Introduction: NodeMCU WebIDE

This instuctables show how to install NodeMCU WebIDE to ESP8266 board.

It also include several examples to brief how to use the WebIDE.

Step 1: Introduction

ESP8266 Board is a WiFi MCU board and ironically you always connect to a USB serial cable while development.

NodeMCU WebIDE aim to break this situation.

Once you install NodeMCU WebIDE to the ESP board, you can detach the USB serial cable and all development works on WiFi only!

P.S.

ESP8266 still require a battery for operation :P

P.P.S.

Don't throw the USB serial cable away, it still required for upgrading WebIDE or NodeMCU firmware :D

Step 2: Design Overview

Develop ESP web IDE challenge and solution

  • Compiler vs Interpreter
    Web browser (Javascript) unable to compile binary for ESP8266 MCU but it can send and save plain text to HTTP server if it have appropriate AJAX API. So I look forward to the scripting language for ESP8266. I can find 4 options for ESP8266: Lua, BASIC, MicroPython and JavaScript. Lua (NodeMCU) seems have better ESP8266 API support, so I choose NodeMCU.
  • MCU resources
    ESP8266 is very limited in memory and processing power, it is not capable handle all IDE features. Conversely, current mobile is much more powerful. ESP8266 act as a "thin web server", only serve static HTML, Javascript files. And also an AJAX file API for saving edited files from the web browser. All other features such as fancy UI, editor functions, IDE logic, etc, handled by the browser and Javascript. Thanks to the work done of nodemcu-httpserver and codemirror projects, the NodeMCU WebIDE become feasible!
  • WiFi transfer rate
    I found some mobile browser, like iOS Safari, unable to retrieve file over around 50 KB. I guess it is caused by client side request timeout. So it is required to trim, shrink and split all web resources files into not more than 50 KB. But at the same time it still required to reduce the number of files to optimize retrieve round trip. There are various tools I have used, such as CSS Minifier, codemirror compressor and gzip. After the file size arrangement, it still need to arrange Javascript load the resources files one by one in sequence to reduce the chance of ESP8266 memory outage.
  • POST request size limit
    Since ESP8266 is very limited in RAM, it cannot handle the HTTP POST data that over around 1 KB. A normal Lua code file can be around 1 KB ~ 10 KB, save a code file from the web IDE to ESP board flash require split data in multiple POST request. It is handled by Javascript and NodeMCU WebIDE file API.

Step 3: Preparation

ESP8266 Board

ESP-12E or related dev board preferred, since it have 4MB flash and can use option 1 (simple one) install method.

USB Serial Cable

If you are using dev board like NodeMCU or WeMOS, it is already included on the board; If you are using ESP8266 board like ESP-03, ESP-07 or ESP-12E, you may refer my previous instructables (ESP-03, ESP-07 and ESP12E) how to connect ESP board with USB serial cable.

Step 4: Install Option 1: Flash Full Image

The pre-built flash image is extracted from ESP-12E, if you are also using the ESP board with 4 MB flash, you can use this install method:

  • Download NodeMCU WebIDE flash image:

https://github.com/moononournation/nodemcu-webide/...

  • Unzip the flash image:
gzip -d nodemcu-webide.bin.gz
  • Flash the image

Windows

    1. Download latest ESP8266 Flash Download Tools if not yet: https://espressif.com/en/support/download/other-t...
    2. flash the image to ESP board

    Linux / OSX

    1. Download esptools if not yet: https://github.com/themadinventor/esptool
    2. write flash
    esptool.py --port /dev/YOURPORT write_flash 0x0 nodemcu-webide.bin

    Step 5: Install Option 2: Flash NodeMCU, Upload Files

    This is more advanced option for install, it can install in any flash size.

    Here is the summary:

    • Custom build NodeMCU: http://nodemcu-build.com/
      Please select dev branch and here are my selected modules: adc, bit, cjson, coap, crypto, dht, encoder, file, gpio, http, i2c, mdns, mqtt, net, node, ow, pwm, rtcfifo, rtcmem, rtctime, sntp, spi, tmr, u8g, uart, ucg, wifi, ws2812
    • Wait to receive the build finished email and download the integer version firmware
    • Flash NodeMCU firmware to ESP board
    • Upload all nodemcu-webide files in bin folder to ESP board: https://github.com/moononournation/nodemcu-webide

    Step 6: Interface Guide

    File list

    Editable files in ESP board flash, click the file item to select.

    Selected file

    Selected file item will be highlighted and loaded to the editor. Selected file can use rename and delete button for further actions.

    New file button

    Create a new file in ESP board flash, default filename is newfile1.txt and then newfile2.txt, etc. The filename can be renamed by select the file and then press the rename button.

    Rename button

    Rename selected file item.

    Delete button

    Delete selected file item.

    Undo button

    Undo last editor changes.

    Save button

    Save the editor changes to ESP board flash.

    Preview button

    Open selected file in a new web browser tab.

    Editor

    The code editor is powered by Codemirror.

    Step 7: Setup NodeMCU WebIDE

    All NodeMCU WebIDE configurable options stored in config.lua file.

    There are 2 ways to modify the config.lua:

    • ESPlorer or other ESP access tool upload edited config.lua through USB serial cable
    • Use NodeMCU WebIDE edit the config.lua

    It is a good starting point for familiar with NodeMCU WebIDE.

    Here are the steps using NodeMCU WebIDE to edit the config.lua:

    1. Power on the ESP board
    2. Connect to ESP board AP WiFi, SSID is "ESP-XXXX" where XXXX is the ESP chip ID, and the default password is "PassXXXX", it is case sensitive, remember replace XXXX with the chip ID show in SSID
    3. Browse to http://192.168.111.1/
    4. Prompt authentication dialog, default user: "login", password: "Passw0rd", case sensitive
    5. Wait a few minutes for initial load all resources files
    6. Select config.lua in file list
    7. Edit parameters in the editor, please refer below parameter list for further details
    8. Press save button
    9. Wait the status bar at the top show save success
    10. Reboot the ESP board
    11. New configuration applied

    Parameter List

      • conf.auth.enabled : enable HTTP basic authentication, default true, set to false if you feel annoying and don't care about security
      • conf.auth.realm : the name show in login dialog
      • conf.auth.user : default "login", change to your user name, case sensitive
      • conf.auth.password : default "Passw0rd", change to your own password, case sensitive
      • conf.wifi.mode : default wifi.SOFTAP, set to wifi.STATIONAP if you would like to connect to your own AP. Please be aware this setting, I found wifi.STATIONAP mode make ESP AP become unstable if your own AP not within the connection area.

      • conf.wifi.stassid : your own AP SSID, case sensitive
      • conf.wifi.stapwd : your own AP password
      • conf.wifi.ap.ssid : ESP AP SSID, default "ESP-XXXX", where XXXX is the ESP chip ID
      • conf.wifi.ap.pwd : ESP AP password, default "PassXXXX", where XXXX is the ESP chip ID
      • conf.wifi.apip.ip : ESP AP IP address, default 192.168.111.1
      • conf.wifi.apip.netmask : ESP AP IP net mask, default 255.255.255.0
      • conf.wifi.apip.gateway : ESP AP IP gateway, default 0.0.0.0, it can avoid mobile phone try to use ESP board access the Internet

      Step 8: Example 1: Hello-world.lua

      Traditional first programming example.

      In order to call from the nodemcu-httserver, all user code must under the return function block.

      return function (connection, req, args)<br>  dofile('httpserver-header.lc')(connection, 200, 'html')
      
        --insert user code here, return output in HTML format by connection:send(html_output)
      
      end
      

      This example simply use a connection:send() function to return the hello world HTML text to the web browser.

      Step 9: Example 2: System-info.lua

      This example shows the ESP board basic system information.

      Step 10: Example 3: Ws-echo.html

      This example show how to use a HTML5 new protocol, WebSocket, make a low latency communication channel between web browser and ESP board.

      In my test with NodeMCU WebIDE platform, Ajax request always have response time over 1 second; using WebSocket can handle up to 6 requests in a second.

      However, please be noted that, my WebSocket code still at a very early stage. It is noticeable memory leakage for every WebSocket connection. The ESP board may reboot after attempt a few WebSocket connection.

      Step 11: Example 4: Led-matrix.html

      This example show how to use a HTML+Javascript fronted (web browser) and a Lua backend (ESP board) to make a simple interaction from the mobile to the ESP board.

      Please refer to my previous instructables for further details:

      https://www.instructables.com/id/IoT-LED-Matrix/

      Step 12: Example 4: Ws-led.html

      This example show further WebSocket application for low latency communication.

      You may notice it have seconds of latency in previous LED Matrix example. This example replace Ajax logic with WebSocket, the response time is much better.

      Step 13: Example 5: Robot.lua

      This example show how to use Lua timer function to make a sequential time interval operation. You may change the steps string value for designing your steps.

      Please refer to my previous instructables for further details:

      https://www.instructables.com/id/Robust-Remote-Toy-...

      Comments

      author
      msameer39 made it! (author)2017-06-28

      awesome brother. Ne how

      author
      陳亮 made it! (author)陳亮2017-06-28

      thx

      author
      mrgw98 made it! (author)2016-08-08

      What keyboard case is that?

      author
      陳亮 made it! (author)陳亮2016-08-08

      it just a white bluetooth keyboard DIY added a plastic support plate :P

      author
      dthomas6184 made it! (author)dthomas61842017-02-07

      how did you construct the keyboard?

      author
      陳亮 made it! (author)陳亮2017-02-08

      you mean how I add a mobile support plate? this is my old diy, may be post another instructables later.

      author
      harrytuttle made it! (author)2016-11-06

      "Since ESP8266 is very limited in RAM, it cannot handle the HTTP POST data that over around 1 KB."

      This sounds solvable. I didn't like the size and feature matrix of nodemcu-httpdserver, so I wrote a very tiny httpd server for nodemcu too: it is less than 4KB in a single file, leaving over 31KB free according to node.heap() (and there's also removable websocket support). I also reused and tweaked the cool ide from the ESPAsyncWebServer project. I can POST over 500KB of data with no problems in a multipart form request: after finding the start boundary the callback function receives all the data continuously in chunks of about 1,5KB stopping at Content-Length bytes. I tested the upload with chrome and firefox. Here's my approach:

      https://github.com/harrytuttle/nodemcu-httpd/

      author
      陳亮 made it! (author)陳亮2016-11-06

      seems great! I will try this out!

      author
      JohnP493 made it! (author)2016-08-21

      I got it loaded and working. I had a bit of trouble getting it to load automatically, but solving that helped me learn a bit about Lua and NodeMCU. The only thing I noticed is the icons are not loading. The "New File, Rename, Delete, Undo, Save and Preview" icons are not showing.

      If I drag the mouse across them they darken to show where they are supposed to be, and clicking works. But I already have clicked the wrong one (more than once) and had to reload everything to replace what I damaged. Any idea why they are not showing? Are they loaded in a specific file that maybe I am missing? I loaded all of the ones in the BIN folder.

      Oh, and it is the same on three browsers, so it is not a browser issue.

      Thanks again!

      author
      陳亮 made it! (author)陳亮2016-08-22

      Hi John, I found some compatibility issue on inline SVG background image with some browsers. Chrome and Safari just works well. I have updated all SVG image from plain text to base64 encoding for catering this. You may use the latest code / image to test it again.

      author
      JohnP493 made it! (author)JohnP4932016-08-22

      That worked! IE and Firefox now show correct. MS Edge still does not load them, but I can work in one of the others. I just like Edge for things like this because then I can use my main browser(s) for other things and switch back and forth quickly.

      author
      JohnP493 made it! (author)JohnP4932016-08-22

      Never mind. Rebooted and now it works in Edge. Great work! Where can I buy you a beer?

      author
      JohnP493 made it! (author)2016-08-15

      I am waiting for my ESP8266 boards to arrive and getting ready. I like the WebIDE concept. What build of NodeMCU/Lua does your 4M download come loaded with?

      Oh, and thanks! This is a great tutorial that should make this easy.

      author
      陳亮 made it! (author)陳亮2016-08-16

      The build come from NodeMCU custom builds website with dev branch options

      author
      gkal made it! (author)2016-08-11

      Thanks for the instructable. I cannot flash your included image. I get error

      Error:Set ESP8266 Address timeout.

      Can you suggest modules for the custom build prior of uploading the files from bin directory?

      author
      陳亮 made it! (author)陳亮2016-08-11

      Hi, gkal. I have yet see this error message before, can you give me more details about this error? What tools are you used to flash the image? Which serial port you selected?

      author
      gkal made it! (author)gkal2016-08-11

      I am using the serial port that i use to communicate, the same that the program recognizes and i am using the esp8266flasher that you suggest and i am always using. I can flash nodemcu and micropython with the same method.

      Thank you for fast reply

      Clipboard-2.jpg
      author
      陳亮 made it! (author)陳亮2016-08-11

      I have yet encounter this error before. Google tell me it may caused by Vcc not connecting to 3.3 V. Hope it can help you.

      author
      gkal made it! (author)gkal2016-08-11

      Finaly managed to flash your complete binary file using espressif flash tool v3.3.6

      https://espressif.com/en/support/download/other-tools

      Thank you

      author
      陳亮 made it! (author)陳亮2016-08-11

      good to see that, I have updated the download link, hope it can help other people.

      author
      gkal made it! (author)gkal2016-08-11

      Thank you again. After connecting to the esp when I authenticate 192.168.111.1 the esp reboots. I tried a couple chargers and battery banks with the same issue. I used Firefox and CM browser.

      author
      陳亮 made it! (author)陳亮2016-08-11

      it most likely caused by failed to load some files. If you still connecting to ESPlorer, try scroll up to find the PANIC message. Most of this type error can be solved by lower the flash speed and reflash the image.

      author
      gkal made it! (author)gkal2016-08-11

      I have flashed it again in 115200 baud rate and kept esplorer open.

      When i authenticate to esp8266 the message is

      PANIC: unprotected error in call to Lua API (cannot open httpserver-static.lc)

      I uploaded manually httpserver-static.lua which in the first run compiled and now it works flawlessly.

      Also is there a way to extract or view the files in a binary image file as yours webide.bin?

      Thank you for the project and great support

      author
      gkal made it! (author)gkal2016-08-12

      I flashed it again with different baud rates. It seems that the httpserver-static.lc file is missing from the binary. Uploading the corresponding lua file fixes the problem in the first run. Try it yourself if you have time and upload a binary that includes the httpserver-static.

      Thank you again

      author
      陳亮 made it! (author)陳亮2016-08-14

      I have updated the image with some latest code change, you may test it in your convenience.

      author
      陳亮 made it! (author)陳亮2016-08-12

      I also have this problem in one of my ESP-12E board. I will try to grab the image before first compile in next update, hope this can help it.

      author
      陳亮 made it! (author)陳亮2016-08-12

      I also have this problem in one of my ESP-12E board. I will try to grab the image before first compile in next update, hope this can help it.

      author
      陳亮 made it! (author)陳亮2016-08-12

      I am using esptools.py read_flash function to extract the full image from an ESP-12E board.

      author
      olivierh11 made it! (author)2016-08-10

      Can you share the pcb schema and the battery you use?
      Have you an idea about the battery lifetime?

      author
      陳亮 made it! (author)陳亮2016-08-10

      You may find the connection in my previous instructables:
      https://www.instructables.com/id/ESP8266-Bread-Board-Testing/
      The battery is a rechargable, LIR2032. I have yet time the uptime, but I estimate it can run around 1 hour if not use any ESP8266 power save feature.

      author
      olivierh11 made it! (author)olivierh112016-08-11

      Thanks :)

      author
      陳亮 made it! (author)2016-08-11

      I have encounter this error in some ESP board before, double check the USB serial cable is well connect and try to lower the flash speed. I can solve this problem when lower the flash speed to 20 MHz DOUT.

      author
      RobotHacker made it! (author)2016-08-10

      Thank you for making a flashable bin file for this. I had tried uploading the individual files from your GitHub but I ended up finding out that there is a known bug in the older SDK SPIFFS that only allowed files up to about 26KB, so I could not upload the codemirror.js.gz to my ESP8266. I just flashed it with the bin and everything looks good so far.

      author
      陳亮 made it! (author)陳亮2016-08-10

      Yes, I found it is a big barrier for upload the source file, so I make this lazy pack.

      author
      mmiscool made it! (author)2016-08-09

      Seems very similar to how esp8266 Basic works.

      author
      陳亮 made it! (author)陳亮2016-08-10

      Yes, very similar, just more fancy UI.

      author
      airsb made it! (author)2016-08-09

      pitty that it is for lue, arduino web ide for esp would be a hit

      author
      陳亮 made it! (author)陳亮2016-08-09

      As mentioned in the instructables, you cannot compile binary file in web browser, so you need to choose from the scripting languages. Lua can be as powerful as Arduino, and it is very easy to learn. Please give it a chance.

      About This Instructable

      8,344views

      58favorites

      License:

      Bio: Do it yourself if you cannot buy it!
      More by 陳亮:Portable WiFi AnalyzerIoT Power Consumption ConcernATtinyPowerMeter
      Add instructable to: