ServDuino - Build Your Own Arduino Web Server

318,002

230

51

About: I like to make things that move, sense, calculate, compute, blink, and make noise. I like making things that create high voltages, electrical arcs, and can light fluorescent bulbs at a distance. I like to d...

The Arduino Ethernet Shield is a sweet toy!!!  Everybody, from the beginner who got their Arduino yesterday and hasn't slept since, to the hard core pro who learned c as a second language, can learn how to use the Ethernet Shield and have lots of fun with it.  Its that awesome! As soon as I earned enough to get one, it was mine, but it took a while for me to figure out how to get my site running and on the web, and I had some trouble with the software, but once I got started, I designed a whole web server, which included a traffic meter, LCD screen, and SD card storage!

This instructable is going to show you how you can make your own "ServDuino" web server, and get it on the web without some of the roadblocks that I, and many others have faced.


Some of this instructable is a work in progress - specifically the code on the programming step.  Not all of the code was released as of the publishing of this instructable because I am always improving or writing new code for my projects.  I am also seeking help from the instructables community in writing code, not because I am lazy, but because I know that I don't have the most efficient code, as I have been programming in c for only a few years, most of which is spent in the world of Arduino.

Don't forget to rate!
I entered this into the Microcontroller contest, so if you think its awesome, or it helps you off the ground with the ethernet shield, vote for it! :)

Step 1: What You Will Need

For this project, you need only 2 things minimum to make it work.  Everything else is extras that you can include if you want to.

The Required Parts are:
An Arduino
An Arduino Ethernet Shield - Note:  If you have an older shield, you will not be able to use an SD card with it.  If you don't have a shield, get the newest version!!!

Optional Parts:
A box to put it in - Recommended
A Servo
An LCD Screen
Some LEDs
An SD Card

Even More Optional Parts - These are for external controls and/or sensors:
A Second Arduino
Whatever sonsers and controllers you want
Either a Long Cable or a wireless Transciever (like an Xbee)

Step 2: Getting Started on the Web

The first thing you will need to do is put the Ethernet shield on the arduino.  If you need help, line up the pin names on the shield to those of the Arduino.  Then plug it in to your computer and connect an ethernet cable from your router to the Arduino.  Open up Arduino IDE and run the WebServer example under arduino-00xx -> libraries -> Ethernet.

Open up your favorite web browser and type http://192.168.1.177 .  You should see something like the image below.  If it does not work, DO NOT PANIC!!!   You probably have an Internet Service Provider the blocks port 80, the standard http port.

To test this, we will try one of the following ports 8080 or 8081.  Go to the WebServer sketch and find the line that reads: Server server(80);  It is at the begining of the sketch.  Change 80 to 8080 or 8081.  Now upload.  To get on your site, however you will need to type: http://192.168.1.177:8080 or http://192.168.1.177:8081 .  If it works, Great now you can move on to the next step!!!  A great place to look at for forwarding help is portforward.com .

Before you get excited and call up your friend to get on, we have a problem.  The IP address you site has can only be accessed by a computer connected to your router.  Nobody else can access it.  How do we solve this??? We use port forwarding.  This can be easy or hard, depending on how you and your router make it.  You will need to access you router's settings page and search until you see a link or tab or whatever that is named port forwarding, or port forwarding/port triggering, or similar.  Go to that page, and select a new http service.  Type the IP of your webserver, and set the starting and ending ports to 8080 or 8081.  Save changes and now people can access you website from outside your router.  But you will need a new IP. Your router's IP.   Go to www.whatismyip.com .  No go to your IP at http://this.is.my.ip:8080 or http://this.is.my.ip:8081. With this and is and my and ip being the respective numbers of your address.

Now call your closest friends up and have them look at it.  But wait!!! There is still a problem, remembering all of those numbers can be a pain! We are not very good at remembering strings of numbers, but we are good at remembering words and names.

To fix this you will probably want to get an actual domain name, like example.com.  The easiest way to do this is to go to dyndns.com and sign up for a free account and choose a free domain name.  These include yourname .dyndns-something .com or yourname .dyndns.some2or3letterending .  Now set this domain to your router's ip.  To go to your site, you will type http://yourname.dyndns.whatever:8080 or 8081, depending on your server port. The http:// is needed in Internet Explorer, but not Firefox for ports other than 80.

The last step you can take (optional) is to make a webhop so that you don't have to type the :808x onto the end.  I have http://teslalingeweb.dyndns-web.com pointed to my router's ip, and set http://teslaling.dyndns-web.com as a webhop to http://teslalingeweb.dyndns-web.com:8081 .  For websites on port 80, the http:// is not necessary.

Now that you have the site on the web, you can build the server, and design your site!!

Step 3: Building the Web Server

While I am going to show you my web server design, it is totally up to you how you would like to layout and design it, but here are some guidelines.

I would suggest putting the arduino in a corner, so that there is access to the power, usb, and ethernet port and the SD card slot.

I would also reccomend using a sturdy box, the one I used wasn't very sturdy, and now the weight of the servo is making it sag in the middle.

If you plan to attach an lcd screen, solder wires to it, and connect them to the ethernet shield before putting it in the box, 8 wires are a lot to connect inside of the box!!

Step 4: Connecting Your Devices

As with everything else with this project, you have a choice in what other stuff you want to add to your ServDuino.  I have a few suggestions and rules that you should follow to make this a successful build.

The Servo.
The servo is simple to connect, attach the red wire to +5, the black to gnd and the yellow (or third wire) to a pwm pin.  On the main board the pwm pins include 3,5,6,9,10,11.  Pins 10 and 11 CANNOT be used, however, because they are used for the SPI communication with the ethernet shield and SD card.  I would reccomend not using pins 5 or 6 either if you are using a delay or millis() because there is a jitter that can be annoying if your ServDuino is next to the computer!

The LCD Screen.
 A character LCD screen can be connected with as few as 6 data pins.  Which is good because there are exactly 6 digital pins left over. Now before you comment and tell me that there is actually 9 pins, let me explain.  We've used pins 13,12,11,10 and our choice of 9,6,5 or 3.  Thisl leaves 9 pins. BUT, we need a pin for the SPI communication to the SD card, which is pin 4.  Next, we must remember that the Arduino uses the UART to upload its sketches, so we don't want to use pins 0 or 1, just in case.  Now how do we connect the LCD screen with only 6 pins, like this:

Vss is tied to ground.
Vcc is tied to +5 volts.
Vo can be tied to ground, or you can attach a potentiometer to it.
RS is tied to a data pin. Remember what pin!!! Or better yet, write it down!
R/W is tied to ground.
E is tied to a data pin.  Write it down!
DB0-DB3 are not connected.
DB4-DB7 are tied to data pins.  Write it down!!
Backlight pins can be tied to their respective voltage source.

Analog Sensors:
Analog Sensors are very straight forward, but on the newer ethernet shield, and maybe the older ones too, there is 10K ohm resistors tied to +5v on analog pins 0 and 1.  Watch out!!

Xbee and Serial communications:
Well, I don't have very much to offer here, I don't have any Xbees...yet...but I would like to offer some suggestions.  Anytime you need to upload a sketch, you will need to either switch out the xbee jumper or disconnect the serial wires (if you are not going wireless), so you may want to try putting a jumper select on the server box.  If you don't you will have to rip it apart to upload a sketch. That's not fun, trust me.

Step 5: Designing Your Site

Here is where you have to start thinking about what you want your site to look like and do, so that you can choose how you want to store your site's information.

If you want to have a very basic site, maybe only one page, or you want to be able to control some led's or light switches or whatever your heart desires, you should go with a site saved in the Arduino's ram.

If you want to have similar capabilities, but you want to have more than 1 page, you will want to consider using the code with PROGMEM.

If you want a regular site with pictures and videos and lots of pages, or whatever, you will want to look into using an SD card.

Limitations and features:
With the ram option, the site can be only like 1500 characters (of html codes, not of actual text) which is very limited, but it is good for a simple led (the led could be anything actually) controller.

With the PROGMEM option, you are much less limited.  With an atmega328, you can store up to 12,000-13,000 html characters!!!  This is suited for home automation or a multipage site w/o an SD card, but you cannot store pictures or files on it, and you are still limited to 12k characters.

The SD card is the best choice for a large site, with lots of scripting, pictures, pages, etc. But as of now, it cannot be used to control leds over the web, but I'm working on it though! I am also working on directories and using the SD library instead of the sdfatlib (but don't expect it for a while because my main focus is on a led controller and directories, unless the 'ibles community would like to help!)

Step 6: Storage Options

This is a more in depth view of the storage options I explained in the previous step.

Ram:
The first option is to store the site in the Arduino's ram.  This is the simplest way because you simply call the client.print() function and write your html code inside it.  It is limited to about 1500 characters, because the Arduino only has 2048 bytes of ram.* 
Ex: client.print("<html>...</html>");

Program Memory:
Wouldn't it be great if you could store the code in the program memory instead of the ram.  Well, actually, the above option does that, but the compiler then makes the micro read all of that out into the ram upon start up.  How do we fix that? We use the PROGMEM Library, pgmspace.h.  This tells the compiler that we want to leave the data in the program memory.  Now we are able to have 12k characters.*  The only problem is that the micro tends to freeze up easily, but I have a workaround for that in the Going further step.

The SD Card:
When It comes to storage amount, the SD card is definitely the best choice.  You can have up to 64GB of memory!!  The biggest problem is that it is not recommended for the Duemilanove with ATMEGA168 or older, because it compiles at about 16000 bytes at minimum.  I rarely have freezing problem with the SD Card, unlike with the PROGMEM code, and I am very close to getting a working led controller code.

Step 7: Programming!!

Here is where the fun - or frustration - begins!!  You are now ready to upload the sketch of your choice and test your web server!!!

I have a bunch of sketches here for download.  I will be listing them in the following categories: RAM, PROGMEM, SD.  I will give a quick rundown of each sketch and what its functions are. I will provide a sketch with each new function.  It is up to you to combined them in whatever combination you want.

RAM:
RAMWebServer_Basic - Basic Ram WebServer Sketch.
RAMWebServer_Control - Code to control and LED included.
*More coming soon!!

PROGMEM:
*More Coming Soon!!

SD:
IMPORTANT:  The SDFat Files will not compile if you don't have SDFatLib and/or you have the SD.h library (included in Arduino-0022)
SDFatWebServer_Normal - Basic SD Card WebServer Sketch
SDFatWebServer_Fast - Contains a buffer between SD card read and Client write functions, making uploading much faster
SDFatWebServer_Normal_404 - Has code for a customized 404-file not found page, called 404error.htm
SDFatWebServer_Fast_404 - Has code for the 404 page and the buffer
SDFatWebServer_Normal_LCD - Has code for an LCD screen hit counter
SDFatWebServer_Fast_LCD - Has code for the LCD and the buffer
SDFatWebServer_Normal_Servo - Support for a web traffic servo meter.
SDFatWebServer_Fast_Servo - Servos and Buffers, what better?
*More coming soon!!

Step 8: Going Farther

If all went well, you now have a working ServDuino web server of your own!!  Now you can try different codes, and even write your own, and if it is good, send it to me and I will post it on the instructable and credit you for it!!!

PROGMEM fix:
The server likes to freeze up a lot when running using the PROGMEM storage.  An easy fix would be to use the watchdog timer, but the problem with that is that when it resets, the Arduino goes into an infinite loop, because there is something messed up in the bootloader.  The easiest thing is too make an external watchdog with a 555 timer, but if you want to keep it software based, you will need the ladyada bootloader .  This enables you to use the watchdog timer to reset the frozen Arduino.  Now you have another problem with going software though.  You still need to reset the ethernet shield.  How? Cut the reset pin off of the shield then run a wire from it to a pin of your choice.  Now you just set the pin high in setup() then bring it low for a few microseconds before you initialize the ethernet chip later on in setup().  A watchdog can also be useful in the RAM and SD options.

SD Card favicon.ico:
When I was testing the SD Card codes, I had the USB cable plugged into a computer and had a terminal monitoring the incoming serial.  When somebody gets on the site, the Arduino sends what page they just requested.  I noticed that whenever somebody went to the homepage, there was also a request for a favicon.ico.  The favicon.ico is the little icon in the address bar, on the tab, and in the favorites folder next to the site.  To get your own favicon.ico, you will need to find an online favicon generator, (google that) make your icon, then download it and put it on your sd card, in the root directory (not in any folders).  If you have the site saved as a favorite, delete the favorite, and go to your site.  You should see it appear.  Note:  If you go into paint and make one and save it as an .ico, internet explorer will reject your favicon.  It will work in firefox and chrome, but not in ie.  If you want you can also make the browser request the favicon by putting this into your homepage html file(index.htm): <link rel=shortcut icon type=image/x-icon href=favicon.ico>

Now go have fun, and rule the web!!

Share

    Recommendations

    • Toys Contest

      Toys Contest
    • Make it Glow Contest 2018

      Make it Glow Contest 2018
    • Big and Small Contest

      Big and Small Contest

    51 Discussions

    0
    None
    marco_righe

    3 years ago on Introduction

    Thank you !!! the code :

    if (c == '\n' && currentLineIsBlank) {

    // send a standard http response header

    client.println("HTTP/1.1 200 OK");

    client.println("Content-Type: text/html");

    client.println();

    client.println("your html code will go here. You cannot have any double quotes in it, however, because that will end the client print function "like this");

    break;

    }

    if (c == '\n') {

    // you're starting a new line

    currentLineIsBlank = true;

    }

    else if (c != '\r') {

    // you've gotten a character on the current line

    currentLineIsBlank = false;

    }

    }

    }

    make my project a lot more fast and reactive!!!!

    I have already completed the controlling of devices through internet using buttons.I used Arduino Uno+ SD CARD+ Ethernet sheild. Now I want to modify it so that used can send text commands through form submission to control the same.

    I could do it when I was using the Arduino Uno Memory for HTML code but now facing problems when I am using SD Card for HTML. Expecting the code module for the same.

    0
    None
    rinksrides

    3 years ago on Introduction

    I had something similar to this working earlier this year (March), but ran out of time to finish it till this winter it used the SEED ethernet 1.1 without SD. Boy that was interesting to figure out how to share the ISP header with two conflicting shields, but I got it to work reliably with some external logic (looked like a rats nest). Home automation at it's best. 70m/150W of LED strip lights, under cabinet lighgting, oh, and controlled HUGHES lights also from a webpage with a colormap. What a PIA that was to figure out, needed two separate maps cuz hughes doesnt use standard RGB colors in their lights. the green is more lime green, the blue is more like teal, even the red is a bit washed out. Nice writeup though. Side note, If you have a MEGA ADK, you can use the USB lib's to access an external HDD/SSD, so you wont be limited to an 64GB SD..

    0
    None
    Topcam

    5 years ago on Introduction

    Hello, my English is not so perfect. (GOOGLE German ->ENGLISCH)
    Catch just me with the Arduino and its potential deal.

    Since I need a web server that reads all data from the SD card
    I have the script on my Arduino SDWEBSERVER_FAST_LCD.

    Running with the DYNDNS perfect.
    Only when I: http://NAME.DYDNS.COM/test/ (here in directory there is an index.htm) write, appears donload windows.
    I give but a http://NAME.DYDNS.COM/test/ index.htm
    goes to the page.

    There's an error in your script?


    My question 2:

    I would now like Led and power switch on the Web, but the code is not in the store Adruino special embed in the HTML file.
    Adjust so I just need to code the HTML and not always in the code Ardoino.

    Is that how and when do you do that?

    Thank you.

    0
    None
    TeslalingKrayzi99

    Reply 6 years ago on Introduction

    Yes. In the _404 sketches, there is the option to have a custom 404 page. You just have to make the page and either name it 404error.htm like what is in the sketch or put the name of your 404 page as the name to replace 404error.htm. The line has a comment after explaining what it is. Look for that then find the filename and replace it with your file's name.

    0
    None
    charlesfries

    6 years ago on Introduction

    Just for people receiving errors when using the SD sketches, "Server" must be renamed to "EthernetServer" and "Client" must be renamed to "EthernetClient"

    1 reply
    0
    None
    Teslalingcharlesfries

    Reply 6 years ago on Introduction

    Yeah, sorry. I need to update some of this instructable when I get the chance, including making the sketches work with the new ethernet libraries.

    0
    None
    botronicsbotronics

    Reply 6 years ago on Introduction

    I got a report from someone that the DX one doesn't work well. They use a different chip than the Arduino standard chip. The drivers don't work with Arduino 1.01. The Arduino-brand ethernet shield works perfectly. Newark has them:
    Newark Part Number: 65T3485

    0
    None
    Vee20

    6 years ago on Introduction

    My server just won't work. I'm using google chrome. I changed the port to 8081. Is there anything else that I should check? I have already disabled MAC filtering on my router. Need help urgently as I can't get any further with my project without this step.
    Thank you so much for the helpful Instructable.

    0
    None
    N.fletch

    6 years ago on Step 2

    Hello, I am a follower of yours and I wanted to say thank you for this Instructable you posted. I used this quite a bit with a wireless bridge to create my latest robot!

    0
    None
    Krayzi99

    6 years ago on Step 8

    Great Ible! This is EXACTLY what i have been looking for! on a side note, if you want the website to appear like http://www.TEXTHERE.com/ instead of something like http://192.168.0.51 then go to no-ip.org, and they'll redirect people to your ip. Thats what I'm gonna do when I build my server XD.

    2 replies
    0
    None
    Krayzi99Krayzi99

    Reply 6 years ago on Step 8

    just noticed you had something from dynds... no-ip is also good, so choose that if ya dont like dynds.

    0
    None
    TeslalingKrayzi99

    Reply 6 years ago on Introduction

    actually, dyndns doesn't provide that service anymore :( I am going to rewrite that set anyway since it can be confusing and misleading.

    Good luck with your server! Glad you found this 'ible helpful!

    0
    None
    Krayzi99

    6 years ago on Introduction

    Is there a way to combine 2 of these for faster connection? I would be willing to make 3 of these if i could. I'm thinking it could be as easy as having all 3 having the same ip, but I don't know. Any suggestions?

    1 reply
    0
    None
    TeslalingKrayzi99

    Reply 6 years ago on Introduction

    I don't know how or if you could combined them for a faster connection. All having the same IP would NOT work though. If all three tried to respond to a request, there would be a network collision. It would be like 3 cars pulling up to a 4 way intersection and all turning onto the same road at the same time :)

    Unfortunately, I think the only way to speed up the connection would be to use parallel communication with the wiznet chip. That would require a rewrite of the ethernet libraries.

    As to the FTP, I don't think it would be possible. The Arduino libraries only allow 1 open port, and to have http plus ftp would require 3 ports (ftp takes 20 and 21). Plus you would need a way to store the files to the card...Things would become very complicated very quickly.