Introduction: ESP8266 OTA LUA With WEB UI MANAGEMENT / Nodemcu Firmware

Picture of ESP8266 OTA LUA With WEB UI MANAGEMENT / Nodemcu Firmware

Over The Air (OTA) functionality for ESP8266 using nodemcu firmware. Ability to remotely update your scripts using web management. The solution consist of two parts - LUA scripts for your ESP8266 and PHP+Mysql running on your server. Using this solution you will be able to manage all your ESPs quite easily - no more USB file uploads.

Hi all,

let me introduce my project a little bit. I already play with esp8266 for a long time, a few of devices are already a part of my home automation system like irrigation system or temperature and humidity sensors. A time ago I realised that time to time I need to upgrade the scripts running there due to some new features or just make them compatible with my HA platform APIs and that was the root cause why I decide to code a OTA for nodemcu firmware. It;s not really a OTA but the main feature is to be able to update the scripts via common management using IP world (instead of USB interconnection with my Mac). The result is a set of LUA scripts and WEB management written in PHP.

What you need?

  1. ESP8266 with nodemcu firmware
  2. webserver running mysql and php (raspberry pi with rasbian is well enough)
  3. my scripts which can be downloadable via github.com - here

How it works?

It consist of two parts - client software and server side. The client software consist of few lua scripts, which communicate with the server side application written in PHP & Mysql using Bootstrap and HTML.

Server side

Using Management UI you are able to manage which scripts should be downloaded to esp and which should be run after the successful upgrade. All esp's are recognised by the chipid unique value.

Main features:

  • easy to catalogue the esp devices including some notes
  • heartbeat - see if it's up and running
  • force to update the scripts
  • add, modify and delete files associated with esp
  • edit lua files

Client side

The client side was written to be gentle to memory usage and allocate as much as possible for your scripts. That's why the process how it handle a little strange but really useful.

  1. the ESP will boot in AP mode with simple UI using which you are able easily define main parameters, like wifi credentials and your server ip and how often to check for update
  2. in next step the ESP will download and compile all necessary files
  3. and finally reboot itself and start script with boot flag
  4. in defined time period it will check for update and as soon as new update is available it will restart, download it, compile it, restart and boot and run the your script

Please be aware that your scripts should be tested before apply via OTA. If it cause the crash of ESP then you need to recover it via USB (just delete the s.txt file, which include the configuration data)

Step 1: Install the Scripts Esp8266

Picture of Install the Scripts Esp8266

I don't wanna describe how to upload the scripts to ESP, there exist a plenty of howto's using ESPlorer or other software. Just download the files from my github repository and upload these files to ESP

  1. init.lua
  2. server.lua
  3. client.lua
  4. config.htm

Now you need to turn it on. It will boot in AP mode with ESSID Wifi_to_Config. Please join the wifi network and open the website http://192.168.4.1. There will be simple form which need to fill in.

The form includes:

  • Essid - your wifi essid name
  • pwd - password
  • host - it's ip of your server, which hosts the management php scripts
  • domain - domain name
  • path - folder where you upload the management scripts on the server
  • update - how often will esp contact the management server to check if new update is available

That's all what need to be done on ESP.

Step 2: Install Management Scripts

Picture of Install Management Scripts

A part of my github repository you can find in folder WebUI_MNG the server part of the OTA. Using this server part you are able to manage your deployed ESPs. Server can be placed inside local network or suing public internet.

Create a folder on your webserver (need to run apache2, mysql and php, personally I use Raspberry Pi with Raspbian). For my guide I decided to you folder "esp". Upload there all files from folder WebUI_MNG.

In next step you need to create database which name is esp and grant access for your user. The database is stored in folder sql. Just import the sample one. Then please edit the config file - config.php and enter your database name and credentials.

If everything ok, you should be able to enter Management UI by enter in your browser http://ip_of_your_server/esp/

Step 3: Configure the ESP Via Management UI

Picture of Configure the ESP Via Management UI

Using the Management UI you are able to remotely configure your ESP, especially to update your scripts.

First of all you have to create a new node to operate. Just click red button - Add new Node and a simple form will be shown. The most important is to correctly setup the ChipID value of your ESP. Such a info you can find during the boot of your esp, just check the boot log using ESPlorer - nodeID is: 8775412 And finally you can add the lua scripts which should be placed on esp (you are able to define also txt, html, mono and other file types). The UI support multiple file upload.

As soon as you press the Submit button, then your server will take care about your new esp.
The other functionalities of management will be described later.

Step 4: Deploy the Esp

Picture of Deploy the Esp

If the esp is correctly configured, we are able to power it on. Now let's describe what will happen.

  1. the esp will join the wifi network
  2. connect the server and ask for filelist (it's a list of scripts which should be downloaded to the esp)
  3. download each file, if it;s script then it will automatically compile it
  4. as soon as everything is done, the it will reboot itself
  5. it will boot up
  6. join the wifi and initialise periodical update check
  7. execute the script

In my case script1.lc will be executed - "Hello world". Using management you can configure a software/program which consist of several lua, hmtl, text or other types of files.

Step 5: Some Useful Notes

Picture of Some Useful Notes

Basic functionalities of the management:

  • able to force update of your esp and deploy a new scripts
  • edit script files (it's not really a lua editor, but if you need to modify just a part of your code, the it's quite useful)
  • add, remove files for each of your esp

Functionalities:

  • UPDATE - it will mark the esp to be updated. As soon as the esp will contact server (heartbeat) then management will force the update process
  • EDIT - just to edit info like name, description and most important chipid value
  • DELETE - you will delete the node from management
  • FILES - modify the files associated to esp
  • BOOT FLAG - if you associated several files to the esp, then please mark using this button a script, which should be executed during the esp boot process. If none will be defined, the esp will execute the first one (first one from the file list)
  • EDIT - edit your script files

Step 6: Important Notes

Due to fact that I would like to optimise the lua OTA scripts really a lot to achieve as much as possible of free memory for the main scripts, some possible errors or misconfigurations from server side are not handle correctly and possible cause a reboot of your esp.

Before you deploy the script to your esp using OTA, please be sure that it was tested and run correctly.

Enjoy it.

Comments

VanjO1 (author)2017-02-22

Hi, tried this tutorial unfortunately i'm having this error when updating my script "

panic unprotected error in call to lua api", what seems to be the problem?

slarti.fartfast (author)2016-11-05

Sounds like a good project! I'm a bit confused about what scripts the ESP8266 ends up retaining; I see we can nominate a "boot" script which obviously should control all our other scripts; are there any constraints re the lua code we upload? e.g. can our ESP8266 have a web server, or would that conflict with the OTA code?

Jubudu made it! (author)2016-11-01

I agree: a cool project. Some smaller issues during MySQL configuration for this project and in server.lua. I will report on Github.

Setup should be better structured step by step including steps of integration in Apache and MySQL, if this tool should become useful for beginners...

Thank you.

kovi44 (author)Jubudu2016-11-02

Yes, you're right, there is still some bugs in the code, which I will try to fix soon. There's a plenty of possible implementations of my lua code. I expect that everyone will modify it based on his needs (like turn off the update check due to memory optimisation, or turn off the compile process, add button functionality to do the reset of settings and many others) I will try to incorporate in the code as many options as possible ,which will be possible to enable, disable based on user needs.

DIY Hacks and How Tos (author)2016-10-23

Cool project. Do you have any more pictures of the setup.

Thx. Will try to add more pictures.