ESP8266 WiFi OTA Remote File Management

Introduction: ESP8266 WiFi OTA Remote File Management

I have been working with ESP8266 micro controllers long enough now to have a number of them deployed in my environment. I have a couple setup testing my MQTT broker(s). I have a couple sending temperature, humidity and barometric pressure readings to the cloud. I have some running for debugging and testing on my bench. I even have one running my irrigation system.

In order to make any changes to the software on any of these controllers, I have to physically attach a FTDI and USB cable to a computer and connect via COM port. This can become very problematic for ESPs deployed in hard to get to or just inconvenient locations. Its also time consuming.

In my minds eye, I can see the possibility of deploying dozens or even hundreds of these inexpensive devices to control/monitor a myriad of situations. Making changes to the software of many of these devices by physically connecting to each is impractical and will keep these great little controllers from reaching their full potential in the world of IoT.

However, with the proper tools, it is relatively easy to manage these controllers over their Internet/LAN connection!

With NodeMCU, it is possible to create and deploy a remote file management system. With the ability to remotely upload, remove, compile and run files, many more options become available. Such as, the ability to push new or updated software to these controllers in mass, remotely or OTA (over the air).

This Instructable will show you how to manage your ESP8266s over WiFi.

Step 1: Overview

Briefly, the work flow is as follows:

PHP files served via web browser will collect the users intentions to either: Read the controller status, upload a file, reboot the controller, remove files, compile files or run files on the ESP.

Once the command is collected from the user, PHP formats the data/information and sends it to the ESP via fsocket.

When the data/information is received at the ESP, LUA files parse the data and carry out the commands.

The possibility of managing ESPs over WiFi rests with the fact that the dofile() command loads the script into memory. Once in memory, the file can be removed or replaced in the ESPs file system with an edited file.

As an example. the init.lua file is automatically loaded into memory on boot up. If there is a command in this file to blink a LED or read a pin, the ESP will continue to carry out those tasks even if the init.lua file is deleted from the ESP. If a new init.lua file is uploaded or sent via WiFi that doesn't contain commands to blink a LED, nothing will happen until the file is executed ( dofile() ) or the ESP is rebooted. Then the new init.lua file will be executed and the LED will stop blinking.

We will use a web browser to send the new init.lua file and a command to restart.

There are some caveats to this solution, we will cover those later.

What you will need:

  • A webserver running on the same LAN as your ESP8266 or the ESP8266 exposed to the Internet.
  • Access to the webserver's root. You will need to upload the PHP files to a root directory in under www/html (apache) or \Inetpub\wwwroot (iis). Simple website business.
  • An ESP8266 flashed with NodeMCU and the ability to upload initial files via an IDE.
  • Some PHP and LUA understanding.
  • Source files for this project.

Step 2: Communication With the ESP8266

The ESP running NodeMCU has a number of ways to communicate with the outside world. TCP/UDP Client, TCP/UDP Server, telnet and MQTT. Since we will be sending simple text strings to the ESP, any of these options can be used.

MQTT would be a great solution for this project. However MQTT has a few more requirements, we will be using NodeMCUs TCP Server.

An ESP flashed with NodeMCU and configured as a TCP Server is capable of sending and receiving data to and from a web browser. This allows us to send files and commands to the ESP and also to query the ESP for certain information. For our purposes, when queried, the ESP can tell us how much memory is left and list the files in its file system.

Sending data to the ESP TCP server can be sent via number of methods. Some of the more conventional are 'GET' and 'POST'. Both of these include lots of header information and other information data of no use to us. The GET protocol is limited 256 characters in most configurations. ~256 characters would be a small .lua file. Although it could be done, sending files via multiple GETs would be difficult and error prone. These methods are slow and browsers can be troublesome.

PHP's fsockopen command is the tool of choice for this project. It is a one to one connection and very fast. We write to the socket just like writing to a file. And!! it only sends data written to it. No header info!

There is a limit to the number of characters sent in a fsockopen session. This is controlled by different TCP layer and most connections are limited to ~1460 characters. If the lua file we are sending is larger that 1200 characters (ASCII), our PHP file will close the session, open another, and continue to send the next 1200 characters of the file until EOF is reached. The LUA file on the ESP will either write or append to the file as it receives the data.

Step 3: The PHP Side

The PHP part of the project resides on the web server attached to the common LAN. This collection of scripts format the data and manage the transfer to the ESP. Files in /filbin folder can be sent to the ESP.

The php files for this project consist of:

  • index.php -- The GUI file for the user to send files/commands to the ESP.
  • controllerIP.txt -- Text file containing the IP number of the target ESP.
  • writeIP.php -- Saves the IP number to controllerIP.txt
  • send_Compile.php -- Sends the filename and 'compile' command to ESP
  • send_Delfile.php -- Sends the filename and 'delete file ' command to ESP
  • send_Dofile.php -- Sends the filename and 'dofile' command to ESP
  • send_file.php -- Sends the filename, command to write or append and a string containing the file to ESP
  • send_Restart.php -- Sends a restart command to ESP

These files along with an empty sub-directory called 'filebin' will need to be uploaded to the server root directory of the machine hosting your web server.

The directory /filebin contains files to be uploaded the ESP are stored.

In my previous example discussing the init.lua file blinking a LED, the /filebin directory is where you will put the edited/new version of the init.lua file. When you are finished editing, simply click on the file name and it will be sent to the ESP.

The index.php file (image above) uses simple forms and buttons to send information to the associated php execute file.

Clicking the "ReBoot ESP" button sends the IP number to the send_Restart.php file. The send_Restart.php file opens the socket to the IP, formats the message "**command**restart**" and writes that string to the ESP. META http-equiv= refresh is used to redirect back to index.php after 5 seconds.

    All the commands are similar with the exception of writeIP and Controller Status.

    The 'Controller Status' button is simply linked to the controllers IP number with "/status" appended to the IP.

    Clicking on any of the files in "Files in '/filebin' available for upload:" area sends the filename and IP number to the 'send_file.php' file.

    The 'send_file.php' reads the target file into an array, then begins to reassemble the file into one string prepended with the command "**command**Newfile**\n". If/when the string exceeds 1200 characters the socket is closed. Another socket is opened and the string continues, this time pre-pended with "**command**Apdfile**\n"

    Step 4: The ESP8266 LUA Side

    The .lua files on the ESP8266 do the heavy lifting of parsing the strings coming over via fsockets. They 'catch' the payload (pl) coming from the webserver and parse for instructions.

    In this example four files are required:

    • init.lua
    • servernode.lua
    • wifi_tools.lua
    • getstatus.lua

    Initially, these files have to be uploaded to the ESP via FTDI and COM port.

    init.lua, servernode.lua and wifi_tools.lua must ALWAYS be in the file system of the ESP. They have to be available to "dofile()".

    This is one of the caveats mentioned earlier.

    When the ESP boots, it MUST to boot into a server mode and it MUST be able to access "dofile()" wifi_tools.lua.

    If you 'format' the controller, its back to a FTDI and a COM port. Yes, you can format via WiFi.

    With the exception of "wifi_tools.lua" and "servernode.lua", these files can be removed and uploaded between reboots.

    !! "wifi_tools.lua" and "servernode.lua" can only be overwritten, never removed !!

    The size of "wifi_tools.lua" and "servernode.lua" normally, cannot exceed 1200 bytes (ASCII characters, including whitespace and linefeeds)

    If "wifi_tools.lua" is larger that 1200 bytes, it will be partially written and will cause 'panic' when the ESP tries to dofile from the servernode.lua to append the rest of the file.

    "servernode.lua" is a generic TCP server file, code can be found on the NodeMCU api site with this addition:

    • if string.sub(pl, 0, 11) == "**command**" then
    • dofile("wifi_tools.lua")
    • end
    • if string.find(pl, "status") then
    • dofile("getstatus.lua")
    • tmr.delay(250)
    •"info.lua", "r")
    • conn:send(
    • file.close("info.lua")
    • end

    The variable "pl" is the payload from the fsocket connection, this is the string generated by the PHP files server-side.

    If **command** is found than the data string is parsed by wifi_tools.lua for exact details and commands.

    The 'getstatus.lua' routine is a bit convoluted, however it allows us to see which files are actually on the ESP in a near real time fashion. It also demonstrates the ability to create and serve files dynamically.

    Step 5: Final Thoughts and Source File Information.

    There is a learning curve associated with this project and stable managing of the ESP. One has to bear in mind how files are affected by other files.

    For example, in this setup the init.lua calls the servernode.lua file, the servernode.lua file may call the wifi_tools.lua file. If this chain is broken, the ESP crashes and restarts.

    If init.lua or servernode.lua are causing the panic, the ESP will loop forever. Probably nothing short of re-flashing the chip will cure this.

    Only add the dofile() command when you are sure its working correctly by manual debugging.

    It is good practice to run files manually during debugging to avoid these loops. With this project that isn't always possible and some other form of debugging must be employed. Many print() commands come in handy. Adding print commands and other debugging code to wifi_tools.lua is not a problem. The file can be as large as you like.

    In fact you can add more commands to such as starting and stopping timers and even Format if you like. :)

    Just remember you will not be able to upload/overwrite via WiFi if wifi_tools.lua or servernode.lua exceeds 1200 bytes. FTDI and COM ports still work if you have a wifi_tools.lua or servernode.lua file larger that 1200 bytes.

    There is no size limit on any other files.

    This is a fairly complicated Instructable. A lot of things have to happen just right for this to work. I have tested this on my setup with a number of different ESPs but I can't possibly foresee all combinations of variables.

    My setup is running Apache2 and PHP Version 5.4.on Linux. I believe linux based Apache2 systems are similar enough to not cause issues.

    Running on IIS or IOS will probably work, however, if it doesn't, look hard at line feeds and EOF handling. Possibly forward and backslashes in file addressing.

    Chrome, IE and Safari seem to be ok. Chrome/Safari on my Iphone have no problems. Yes, I update/configure my ESPs from my Iphone.

    I am connecting to my ESPs over a private LAN. I have not tested this over the Internet.

    I have posted all the source files on GitHub. Use them in a test environment!

    Of course, no guarantees!

    All source files here:

    GitHub sourse files

    Good luck managing your ESPs over WiFi!

    1 Person Made This Project!


    • Puzzles Challenge

      Puzzles Challenge
    • Metal Contest

      Metal Contest
    • Lamps Challenge

      Lamps Challenge



    Question 4 years ago

    can't make it.

    It says:

    "lua: init.lua:11: bad argument #1 to 'config' (config table not found!)
    stack traceback:
    [C]: in function 'config'
    init.lua:11: in main chunk
    [C]: ?"

    I have the latest bild from NodeMCU firmware and using Esplorer.

    Any advice on how to carry on?


    5 years ago

    get status should works?

    in my all is ok but this option not work.

    I see the all files lua, and do not see any functions for that

    Only write this file (info.lua)

    The server send request to client, and lua should send this info.lua to server, but nothing are doing..this is not impemented yet?


    5 years ago

    I have been reading some recent articles on securing IoT devices. OTA updates needs to be done with caution. At least be aware of the issues. Especially if you plan on putting it in control of devices that could cause problems if misused. I'm still trying to search for a variety of solutions. It's one area that still needs work.


    5 years ago

    Hey, thanks for the solution, seems like it would save a lot of time in the future.

    Just a quick quesiton, if any1 can help me as this is my first day with the nodemcu and im kind of lost. I am trying to host a second web server that would receive commands to operate relays. How can i have both functionalities? add the code i got from this guys here ( somewhere so the ESP can run it and expect GET commands.



    5 years ago

    I have been using this for quite sometime. I decided to add a sqlite database to the project so I could keep up with many modules spread out around by house.


    Reply 5 years ago

    Great work!!

    Thank you!


    6 years ago


    I am trying to get it to work, but without success so far. Same picture as described by LeonardoF21: it reboots, it uploads files (but not really. It tells that the file has been uploaded, but when I check on ESP, it is not there.

    Here is the output I see (with ESPlorer) after pressing the reset button:

    NodeMCU custom build by
    branch: master
    commit: c8037568571edb5c568c2f8231e4f8ce0683b883
    SSL: false
    modules: node,file,gpio,wifi,net,i2c,spi,tmr,adc,uart,ow,cjson,dht,sntp
    build built on: 2016-01-20 05:51
    powered by Lua 5.1.4 on SDK 1.4.0
    System Info:
    NodeMCU 1.4.0
    Flashsize: 4096
    ChipID: 91198
    FlashID: 1458376
    Flashmode: 2
    Heap: 38520

    File system info:
    Total : 3426401 Bytes
    Used : 15813 Bytes
    Remain: 3410588 Bytes

    Server running...

    This is on a NodeMCU 1.0 board

    I have downloaded your files from github yesterday, so it is an actual version.

    Any ideas ?

    What worries me is IP: nil (output from init.lua)

    When I enter

    = wifi.sta.getip()

    in ESPlorer, I get the IP, netmask and gateway as return.

    Any ideas ?

    Thank you for your assistance



    Reply 5 years ago

    i have same problem please any one guide for that problem


    5 years ago

    Hi, just want to ask. Can I used this for uploading Arduino sketch remotely anywhere in the world? If so, would you mind sharing it with me? I really dont have any idea. :( Really appreciate the help.


    5 years ago

    Hi, im kinda stupid, so here's my question.. does this require you to run .lua file for your program? or can you run .ino file that was created in Arduino IDE for the esp8266?


    5 years ago

    Hi, Tough the steps works partially like ip address change, rest all not functioning. let me know if any other steps could be of use and possible share. thanks.


    6 years ago

    It work great and freely ESP8266 from USB cable.

    I use EasyPHP Devserver as WebServer.

    No found any doubt on your code.

    Anyhow I add enduser_setup at first line of init.lua for easily to define SSID , Password and IP Address.


    Reply 6 years ago

    Glad you have found this useful. Your post sounds interesting. Would you share your enduser_setup code?


    Reply 6 years ago

    Below is init.lua code.

    MyPersonalxxxxxx will be shown on wifi list.

    Connect to it and browse to via webbrowser.

    Select your router ssid, fill password and save.

    Wait and see status. You can see IP address.

    Refresh webbrowser, If status not change.

    * build firmware with enduser_setup module.

    print("Start "..node.chipid())


    wifi.ap.config({ssid="MyPersonal"..node.chipid(), auth=wifi.OPEN})




    print("Connected to wifi as:" .. wifi.sta.getip())




    function(err, str)

    print("enduser_setup: Err #" .. err .. ": " .. str)



    function doServer()


    --wifi.sta.config("front-tech1","frontwheeltech") --- SSID and Password for your LAN DHCP here


    tmr.alarm(0,1000, tmr.ALARM_AUTO, function()

    if wifi.sta.getip() ~= nil then


    print("System Info: ")

    print("IP: ")


    majorVer, minorVer, devVer, chipid, flashid, flashsize, flashmode, flashspeed =;

    print("NodeMCU "..majorVer.."."..minorVer.."."..devVer.."\nFlashsize: "..flashsize.."\nChipID: "..chipid)

    print("FlashID: "..flashid.."\n".."Flashmode: "..flashmode.."\nHeap: "..node.heap())

    -- get file system info

    remaining, used, total=file.fsinfo()

    print("\nFile system info:\nTotal : "" Bytes\nUsed : "..used.." Bytes\nRemain: "..remaining.." Bytes")


    dofile("servernode.lua") -- calls servernode.lua





    Reply 6 years ago

    Good work! Seems NodeMCU can send multiple conn:sends again?


    Reply 6 years ago

    NodeMCU seems very easy panic and restart, I have tried many block size. Finally I found it can receive around 1024 bytes after every GC, so I split the file into many 1 KB request and send it in sequence.


    6 years ago

    This is GREAT!!!


    I was wondering if there were a way to upload/modify files on the ESP without connecting to it.

    And this is amazing.

    I just needed to adapt to my actual init.lua and then it's working perfectly!!!!


    Reply 6 years ago

    I'm glad you found this useful. Thanks for the kind words!


    6 years ago

    The php code pushes files to the ESP. I think you could use the ESP to fetch files from a server, however, large files would pose a challenge. Large files need to be sent in chunks. The php client does the heavy lifting in that arena. Thanks for the input, you have me thinking along some new lines!