Introduction: WEBSD

This is a development board for the PIC24F series of microchip, with:
  • USB ( plug it to any computer)
  • Ethernet ( plug it to the internet)
  • SD card ( save and read all data you will need)
I developed this thing as a learning platform, it worked ( still is). So if you always want to go a step further in microcontrollers as i want, it's a nice way of doing it . The problem i had is that microchip examples are for their development board, which i don't have access. So i developed my one board and examples, and i saved all the steps so it would be easier to someone to get started.Going from a simple ping to a webpage with ajax.

 A video showing the project:

 (Obs:the page is no longer available, iT consumes some power and a port on my  router.)
Just to give a head start see some of the things i have done so far with this board:
  1. An Touch keyboard, usable as a standard USB keyboard.
  2. A Video Game ( pong and snake games) , with TV output, and connection to a PC Keyboard ( the old PS/2 ) for control.
  3. Picture frame,  reads pictures form the SD card and show them on the TV.
  4. Controling a 5 volt LCD and some leds, with just 3 wires ( the MCU is 3.3V)
  5. Web server, where you can see some status of the board as temperature, the position of a Pot and a Button,  control some leds and the LCD ( all this from the internet)
  And the things i know before i started ( just a checklist): It all started when i saw this project:

web-server-on-a-business-card-part-2 ( thank you HACKADAY)

The idea of controlling something trought the internet was really nice, so i bought the chip. But when i was delivered classes started. One Year later, i started to read a promising book:

Programming 16-Bit PIC Microcontrollers in C: Learning to Fly the PIC 24 

I was the 24F, i started right away. Breadboard, wire-mess, and the first programs were running. But as i advanced in the book i felt the need for a PCB. Hackaday project was my base, but i wanted i little bit more. An USB connector, and some pins fore expansions.The WEBSD board was born.

After I developed some extra boards to plug. If i want to do a new thing with the board, just design a simple expansion board. 

Follow the next pages to see more about the project. Project files are annexed on this page.

Arthur Benemann 2011 Brasil

Step 1: Main Board

Jump this if you have no interest on the design part of the project.

The main board was been based on the hackaday project. It's name come from the two main thing on it:
  • WEB - Internet connection
  • SD - The memory card
I was one of my first projects with a PIC24F, and the power of this chips and C combined scared me.

Layout was easy because of the PPS (Peripheral Pin Select) feature of this family. I allows the I/O of the digital peripheral to be assigned to almost any pins. But i was low on the pin count, but squeezed 4  pins out of it for expansions.

Power is done by a small power supply rated for 5v 500mA ( a standart 7805 PS). I do not chose USB power because of the current draw by the Ethernet chip.

To put all the connectors on one side , and to keep a small size, a double layer board was needed. i got addicted by smd parts, they can be faster to solder, and there is no need to drill so many holes in the board. Unfortunately i didn't bought the Smd version of the ENC28j60 so it is a DIP part.

The MCU choice was simple, i wanted to learn to use the 24F family, USB was a nice feature to explore, SOIC package (QFN packages still frightens me). So i just pick the one with the biggest program memory.

The ICSP programming connector is for the pickit2 footprint, i is used also as a rs232 serial port. You can see the communications using the terminal provided with the pickit2 or with a external serial port ( allows for more speed)


Part-list  ( with mouser reference) :
QtyValueDeviceParts
3 redLEDCHIPLED_0805 LED1, LED2, LED3
8 0.1uFC-EUC0805 C2, C4, C6, C7, C9, C10, C11, C12
2 1uFC-EUC0805 C5, C8
1 2K32R-EU_R0805 R12
2 10kR-EU_R0805 R1, R5
1 10uFCPOL-EU153CLV-0405 C3
1 25MhzCRYSTALHC49UP Q2
2 22pfC-EUC0805 C15, C16
4 49R9 1%R-EU_R0805 R8, R9, R10, R11
1 60ohmsI_0805 L1
1 330RR-EU_R0805 R4
2 330RR-EU_R0805 R2, R3
1 AP1117E33V_REG_LM1117SOT223 IC1
1 ENC28J60-DILENC28J60-DIL IC2
1 J1006LONGEthernet Connector RJ1
1 PIC24FJXXGB002PIC24FJXXGB002 U$1
1 SDSD U$2
1 USB-MB USB-MB U$4
1 power jack JACK-PLUG1 J2

Step 2: TempLedPotButton Board

Jump this if you have no interest on the design part of the project.

The first expansion board i designed, as simple as it can be.

I has exactly what the name says:
  • Temp - Temperature sensor, one  LM35 power from the 5v supply , which gives 10mV/°C.
  • Led -  no mystery here, a led an a resistor
  • Pot -  Potentiometer connected as a voltage divider, just waiting to be read by the ADC
  • Button - a pushbutton to the ground, no pull-ups are used as the PIC already has them.
I have i problem here in Brasil, it's hard to get parts. I have to buy from the USA ( paying a lot of taxes , up to 200%) just to get a couple of "female headers". So i know that the connectors i used are made for cables. Nice headers would be much better (like those in the arduino).

I was experimenting with my board manufacturer capabilities, so fell free to increase the size of those tracks ( 8mil tracks are hard with the toner transfer method).

Step 3: Touch DirectKey Board

Jump this if you have no interest on the design part of the project.

I had already  written a routine for touch sensing in assembler for the PIC16, but the 24Fs has  another nice feature the Charge Time Measurement Unit (CTMU). Using it as a current source it's a lot easy to measure capacitance. 

So a simple explanation of the touch sensors: when you put your finger close to the big square on the board you increase the parasitic capacitance of that track. The micro continually monitor the line for changes on the line detecting a button press.

No big things on this board, just a piece of acrylic on the top  to simulate the front panel of something ( i broke one corner when i was cutting). I made a simple silk screen on the board to identify the buttons. For a better performance the tracks must be as close as possible to the finger, but i put them on the bottom side of the board because if i can make it work like this, it will work well when used normally.

Step 4: LCDSerial

Jump this if you have no interest on the design part of the project.

No development board would be complete without a LCD.But there is a big problem here, in fact two:
  • The power for this PIC is 3.3V, and all alphanumeric LCD i have at hand are only 5v. There is the possibility for connecting 5V inputs to some of the pins on the micro, but i had use all of them. ( if you take out the USB pins there isn't enough pins left)
  • Even if it was a 5V part, i had only left 4 expansion pins, the lcd needs at least 6 ( nimble, only write mode)
So i looked around until i find the solution on a PICLIST post, i little more and i found this. One 74HCT595chip would do the job of the level conversion. I think it would be better explained by "dangerous prototypes", so take a look over there.

But why stop on a LCD if you are using a shift register? Lets put some Leds on this thing. Another '595 in series and we are good to go.

The hidden position of the contrast potentiometer is worth mention. i projected with a lcd, but soldered other so it stayed in the wrong position.


Part-list (with mouser reference):
QtyValueDeviceParts
8LEDCHIP-LED0805 LED1, LED2, LED3, LED4, LED5, LED6, LED7, LED8
1 TAC_SWITCH S1
1 0.1uFC-EUC0805 C1
1 1KR-EU_0204/5 R2
1 10KTRIM_EU-LI10 R1
2 74HC595N 74HC595N IC1, IC2
8 330R 0805R-EU_R0805 R3, R4, R5, R6, R7, R8, R9, R10
1 HD44780-LCD HD44780-LCD LCD1

Step 5: Video Keyboard Board

Jump this if you have no interest on the design part of the project.

This board was designed to be connected to a PC keyboard. It's the old PS/2 keyboard. It works at 5v so we must apply a little trick here, there are some resistors that rely on the clamp diodes from the PIC to lower the voltage to 3.3V. The connector i took from a old ATX motherboard.

A simple DAC made of two resistors is used to generate the video signal.I had no PCB mount RCA connector, than i used a molex connector wired to a RCA male plug.

The molex-RCA adapter is simple, GND of the board must connect to the shield of the RCA, VideoOut from the board to the center pin of the RCA.


Step 6: Code

It took me some months to learn how to code everything that this board does now, so i will divide the code in some parts:
  • Ethernet connection
  • Serial LCD
  • Touch
  • Video
So jump to the page of your interest.

All code was compiled with C30 lite version from microchip ( free).

Below its a table describing what is necessary to run each example.


Board Hardware
TempLed
PotButton
LcdSerial Video
Keyboard
Touch DirectKey SDcard
(micro)
Keyboard
(PS/2)
SDTest - - - - X -
Ping - - - - - -
HTTP - - - - - -
HTTP2 - - - - - -
Dynamic Variable  X - - - - -
GET X - - - - -
Autentication X - - - - -
AJAX X - - - - -
Ethernet LCD - X - - - -
TCPIP demo  X - - - - -
LCDSerial - X - - - -
Touch LIB - - - X - -
USB touch - - - X - -
Video - - X - - -
SDPicture - - X - X -
Games - - X - - X

Step 7: Ethernet - the Beggining

So lets begin. The main board is ready what we do ?

-------------------------------------------- Powering Up ---------------------------------------
Just power it up and check if nothing smokes, the power led must be ON. Now try to connect one Ethernet cable from your router, or PC. One of the leds near the ENC28J60 should light indicating link acknowledged, and the other blink meaning that there is RX activities ( your router is trying to connect to the board). 

----------------------------------------------- SDtest ----------------------------------------------
If you got here Great ! Means that the Ethernet part of your board is working. now you can test the MCU and SD card by running the demo SDTest. Burn, power up the board with a SD card inserted on the socket. A text file will be created on the card if everything runs well. This means that now you can start playing with microchip file system library's : ).

---------------------------------------------- Ping Test --------------------------------------------
Now things will get interesting, let's connect it to a network.You can do this by 2 ways, connecting with a crossover cable direct to a PC. Or ( this one is best) connect it to a domestic router with DHCP enabled ( i use a di-524). Static ip isn't hard, but the demos are programed for DHCP. 

Program the board with the Ping demo.

Just ping the board to see if TCPIP stack is working. To do a ping test open a command prompt and type "ping websd" , it should return the IP addresses of the board and the time it took to get a response.

if it doesn't work you can try to ping directly to the ip address of the board, you can find this add by looking in you router page, or by using a tool provided by microchip "Ethernet Discoverer" ( this is the easiest way).

There are some debugging stuff that i send trough RS232 ( not all demos). Plug it to have a little more fun.

--------------------------------------------------- HTTP -----------------------------------------------
Congratulations, if you get here the TCPIP stack is operational ! Now let's make a HTTP server.

Demo number 3 - "HTTP", is a mini http server. I uses the old server from microchip with MPFS to store webpages in the program memory. The must be stored in the program memory because it doesn't have a external memory chip.To prepare the webpages you must use a small microchip program called MPFS. WebPages.bat its a batch file i create that can be used to process the webpages for the mpfs, it can be included in MPLAB as a pre-build option, so when compiling the webpages will be automaticly updated. 

To see this example working you must:
Compile, burn. Now try to ping the board, then open the page trought a browser.It can be opened in a browser by openein the url : " teste/" or "IPADD" , where IPADD it's the network address of the board.

If sucesseful a webpage with a hello world message will open , putting a smile on your face.

--------------------------------------------------- HTTP2 -----------------------------------------------
It's the same stuff as the "HTTP" demo, bu uses microchip HTTP2 module, and MPFS2. These have a lot more functionality with will be explored in the next examples.

Just compile,burn, pen in a web browser. But now note that TCPIP version and build dat, will change acordling to when it was built. These two are dynamic variables.

Step 8: Ethernet - Middle

Ok you have a web server, it has links, text, images. But it still seems to be missing something, we are not at the 90s anymore. What we need is a way of interacting with the world so:


------------------------------------------- Dynamic Variables ---------------------------------------------
Now just remember to plug the TempPotLedButton board.

Dynamic variables is the way microchip chose to implement this interaction. You put something like "~temp~" in your webpage code, then when the MPFS reads this it is changed to call to a subroutine in the program.The routine called is "HTTPPrint_temp" on the "CustomHTTPApp.c" file. There you can return any stric to be show in the webpage with the command "TCPPutROMString()". All calls to the print subroutines are in the "HTTP_print.h" file that is generated automatically by msfs. 

The variables implemented in this demo are the status of the button, temperature of the board, position of the potentiometer. The values are captured when the webpages is being loaded, so an F5 needs to be pressed to refresh the page with new values.

I added a "ipconfig" page to show the configuration of the board TCPIP protocol.

---------------------------------------------------- GET form --------------------------------------------------------
Until now we have a http server that can display real world variables like temp, status of  a button... Now it's time to start controlling things from the web.To do this we will use a html form.There are two methods of processing data for html forms: GET or POST. GET it's simples but has a limit on the size of the data. POST has no limit, but is a little more difficult. TO just toggle a led get method will be sufficient.

Get is done by a form on the webpage, it append some data to the URL of the webpage like:
"index.htm?led=on",  where Led is the variable, and on is the state.

Microchip HTTP2 server call a routine called:"HTTPExecuteGet" when a get form must be processed.In this routine we must check if it's the right webpage, look for the variable in the url, check it's state, and do something with it like toggling the LED.

It's just necessary to burn and open the browser to see this working.

------------------------------------------------- Authentication ------------------------------------------------
The page ipconfig has some info about the local network, maybe its better to protect this info. y Fortunately Microchip has done almost all the job for us (nice). Just need to add the authentication module to the HTTP server.

A routine in customHTTPApp.c a routine checks if the webpage being open needs authentication, and other routine checks the password. Simple as it could be.

Just burn the demo, and try to open ipconfig page, default config is :
user: "arthur"
pass: "admin"



Step 9: Ethernet - the End ?

Is this the end of the Ethernet demos? Probably not, i think that this has just scratched the world of things that can be done with board like this. Now that we covered the basics lets see some of the latest codes i have done.

---------------------------------------------------------------- AJAX ----------------------------------------------------------
Least demos showed how to blink the led and read some stuff from the board. But every time you want to blink the led or get a new reading from the board a refresh must be done, reloading the entire page again. 

The solution that microchip and many othes adopt is to use ajax . It was developed to take some of the load to the client machine, according to Wikipedia:" web applications can retrieve data from the server asynchronously in the background without interfering with the display and behavior of the existing page".

There is a script on the webpage that query only the dynamic variables to the server in a timed fashion.This way the server load is decreased, also the calibrations of the temp sensor, and range scaling of the potentiometer can be done in the webpage, moving the floating point operations to the PC and not to the PIC.

Things on the PIC side doesn't change too much, just the calls to the temp and pot variables do not need to do the scaling of the values.

JavaScript on the webpage change also the size and color of a table as a way to improve the visualization of the potentiometer value ( changing color too).

Burn, plug the TempPotLedButton board, and have some fun.

The code for the auto reload ajax script was taken form the microchip "TCPIP Demo App".

-------------------------------------------------------- Ethernet LCD ------------------------------------------------------
To understand this better read the page where  i explain the LCD routines first.

Reading variables from the board, and having updates in real time it's a useful feature. But another nice one i think is controlling things from the web. I know that the GET example already did that, but with ajax things are a lot more fun. If some one clicks to blink a led while you are in the page you will also see it status changing. Putting messages on a lcd over the internet its nice too.

So i developed this board and demo for this function demonstration outputs, a message can be send to the LCD trough the webpage, and the status of the leds can also be changed.

As i whanted to keep this demo online for some time a hits counter would be nice. Therefore i created a variable that is increased every time the "print_builddate" routine is called ( on page loads). Its value is self refreshed by the ajax script. Its a 32bit unsigned integer, it will not overflow ( hehe 2^32 = 4.294.967.296 hits).

If all goes well ( my ISP doesn't block me) you should be able to open my WEBSD board on the address ( it might be slow , as i don't have a good internet connection):

http://websd.no-ip.org/

Putting this online was a little hard. I have a domestic ADSL connection, which means no static ip address, and to get things worst my IPS blocks port 80 and 8080 ( common ports for HTTP servers) .
The solution was to use no-ip redirection from port 80, dynamic ip (no-ip and my router) , and redirection again to the  80 port on the ip-add of the board in my local network. port 80 in my internal network so i can access it directly.

-------------------------------------------------- TCPIP demo App -------------------------------------------------------
With a few modifications TCPIPdemo from microchip exemples can be run on this board using internal flash. But because of program size constrains maximun otimizations must be used.
Some features where cut because of program size constrains.Button display works, and the led on bord is controled by the second led ( left to right) on the webpage.

Step 10: Video

Like i said on the beginning, i started this project because i was reading the book:
Programming 16-Bit PIC Microcontrollers in C: Learning to Fly the PIC 24

In one of the chapters it shows how to generate a composite video signal (NTSC an PAL) with the SPI module of the pic. So here are the results of playing with that.

----------------------------------------------------------- Video--------------------------------------------------------------
This project it's a following  the books page by page. The best way to understand deeply what is going on is reading the book. So for this i will left just the code  with no explanation (except for the comments in the code).

Include one of the folowings in the project souce in MPLAB:

 CleanScreen.c      - Blank NTSC frame
 TestPatern1.c        - Test pattern to see if video buffer i
 CornerBox.c           - 4 boxes on the corners, to see if all pixels are showing up
 RandomPatern.c  - test of the plotPixel function
 LineTest1.c            - trying to draw a line
 LineTest2.c            - Bresenham algorithm for lines
 graph1.c                 - 2D graph, calculated by the pic
 graph2.c                 - 3D graph, same as before but nicer
 mandelbrot.c         -  the Mandelbrot Set draw on the fly

( i suggest that you follow the order to see how thing are developed)

Then compile, and burn the pic to see it working.

--------------------------------------------------- SDPicture ----------------------------------------------------------------
Thanks HACKADAY for the inspiration and explanation of some stuff of this project. Wikipedia also helped a lot.

This project simply read the SD card looking for monocromatic , 256x192 (screen resolution), BMP files on the card.

Than it decodes it and transfer the image to the video buffer, showing them at the TV screen.

It was based on this project from hackaday: digital picture frame
Wikipedia was extremely helpful to make me understand the BITMAP files: BITMAP

-------------------------------------------------------- Games --------------------------------------------------------------
A hint in the closing chapter of the book instigated me. Double buffering could be used to make fluid motion on the screen for things like games. 

Double buffer is a method  used by every video card to produce graphics. One buffer is show on the screen, while the other is being draw. When the drawing is done buffers are swapped.

That was it, i needed to build a video game. But how to connect some controls to this board , 4 pins would not be enough for a direct connection. Another PIC connected via a serial port would be OK. But from a previous chapter of the book i have made some routines a connect a PS/2 keyboard . Only two pins needed, just apply the resistor trick to change the level form 5v to 3.3v  ( the keyboard is 5v). 2 pins for the keyboard+ 2pins for the video out , Video Keyboard board was done.

Now just chose a game, first choice SNAKE. The code was written in a top-down fashion. An structure to hold the snake variables, routines to draw the snake, draw the block of the body...
Is nice to check the code of this one.

It was a nice game, but multi player is better. Why not invoke one of the classics : PONG. This was fast to code, but needs still need some polish.  I hope to post a video of these video games.

Step 11: Touch and USB

I'm working on a project that needs a USB touch Keyboard. The WEBSD board was perfect for developing a quick prototype. But i couldn't make the microchip touch sensing library's to work.But i had already written some touch routines for the PIC16F in assembler. Then i build my own routines.

------------------------------------------------------ Touch LIB -------------------------------------------------------------
My own touch sensing routines.
Take some of the comments on the code to send a lot of data via RS232. It's nice to see the variables running in a graph with a terminal like Tinybld.

I won't explain to much here, microchip has a ton of material about touch sensing, so look here for more info.

This program is just a test of the Touch LIB, it keeps sending the status of each button on the board via serial ( pressed or not).


------------------------------------------------------ USB Touch --------------------------------------------------
It's a touch usb keyboard, the pc recognize it as a standart USB keyboard, a lot like one of my previous projects IRUSB . But isntead of keys it has touch sensors.

The code is a fusion of the Touch Lib developed above, and the usb keyboard demo from microchip.

Step 12: Final Words

Just some final words.

I hope to continue this project, with things like Google power meter, and others.Comments are always appreciated.

I want to thank microchip, and all those guys who write application notes and example code.
Hackaday for your well explained projects.

Fell free to use my code, just don't use it commercially.

Arthur Benemann 2011 Brasil.

Microcontroller Contest

First Prize in the
Microcontroller Contest

3rd Epilog Challenge

Participated in the
3rd Epilog Challenge