Streaming Video From SD Card to Nokia LCD With Arduino

14,844

48

32

Introduction: Streaming Video From SD Card to Nokia LCD With Arduino

For a long time I thought playing back video with audio was not possible on the Arduino. Then one day I decided to give it a shot and it turned out to be simpler than I thought.

I decided to write the steps in an instructable for the benefit of others considering trying it out themselves.

This write-up is about how to create such a video for playback. It is not intended to be a full-fledged instruction on how to build a device like this !

However, I will tell you how to get one or instructions for building one, if you are interested.

Step 1: The DIY "Gamebuino" Device and How to Get One

The "Gamebuino" or actually a DIY "Fakebuino" used in this video is just a Nokia 5110 monochrome lcd display wired up to a Arduino Nano (ATMega328). Also included in my build is a SD-card breakout and a PAM 8403 amplifier breakout. All components can be bought very cheap on Ebay. Simply search with the terms in bold above.

Alternatively, you can buy a device like this ready-made at www.gamebuino.com. Aurelien Rodot, the mastermind behind the Gamebuino device has open-sourced his hardware and library. You will find all the information you need by searching his project website.

I have included a Fritzing schematic of how my own device is wired up. Just be aware, that it is not a 1:1 Gamebuino-compatible device, it is just my own version of the thing. Take note: the PAM 8403 chip is not a chip, the pins refer to connections on a PAM 8403 breakout.

Also, I will not explain how to use a Nokia 5110 LCD / SD card breakout with Arduino, even if asked. Both topics have been extensively covered by others and you will have to do your own research.

Step 2: Ripping the Video Into Numbered Bitmap Frames With Ffmpeg

The video used in this demonstration was saved from Youtube. Again, I will not explain that process here. Just Google it. In any case, I saved the video as a low resolution 3gp file, since I knew it was going to end up in a very very low resolution in the end anyhow.

So, after you have yourself a source video you wish to rip, you need to get and install yourself ffmpeg if you do not already have it.

https://www.ffmpeg.org/download.html

The video was ripped into 84x48 grayscale bitmap files at 20 frames per second using the following command line:

ffmpeg -i myvideo.3gp -r 20 -f image2 -vf format=gray -s 84x48 .\output\%d.bmp

Explanation of parameters:

  • ffmpeg is the program. If installed correctly, you should be able to call this from any directory on your Win/Linux box
  • myvideo.3gp is the name of the source video. You need to be in the directory of the source video when you run this command. Otherwise ffmpeg will not find your source video. The video can be any format, here it just happens to be a .3gp. Could be a a .mpg or .mp4 or whatnot.
  • -r 20 is the frame rate at which we will rip the frames, 20 frames per second
  • -f image2 is the output format (output video as image frames, not a single video file)
  • -vf format=gray means make the output format grayscale
  • -s 84x48 is the pixel size of the output frames (84 by 48 pixels for Nokia 5110 LCD)
  • .\output\%d.bmp is the output directory and output naming for the ripped frames. You need to make a "output" folder before you run this command ! The %d.bmp simply means to name files as follows: 1.bmp 2.bmp 3.bmp ...

Step 3: Ripping the Audio to 8-bit RAW With Audacity

Go get and install Audacity, if you haven't got it already.

http://audacity.sourceforge.net/download/

Audacity needs both the LAME and ffmpeg libraries to work !! Go check from Preferences->Libraries that Audacity lists both the LAME and the ffmpeg version. If no version numbers are given, it means that Audacity has not located the libraries and the next steps will not work ! Again, I am not going to give technical advice on using Audacity. Seems harsh but believe me, there are hundreds of tutorial on Audacity already. I am not going to repeat that information here.

Next, open up the video file in Audacity. Thanks to ffmpeg, it can directly rip the audio into Audacity (this is why we need the libs in the previous steps, you see).

NOW, set project rate at 10080 Hz (see image). This is important because it will sync the video and audio. As we will be playing back video at 20 frames per second = 504 x 20 = 10080 bytes of image data per second, therefore we need also 10080 bytes of audio per second. THIS IS THE MOST IMPORTANT TRICK IN THE WHOLE THING !

NEXT, export audio as 8-bit unsigned headerless RAW (see image)

SAVE file as "audio.raw" into the output folder containing the ripped .bmp frames

Step 4: Dithering the Frames With Imagemagick "mogrify"

Next, we need to make 1-bit black & white images from the grayscale frames we ripped.

For that, we need to download and install Imagemagick:

http://www.imagemagick.org/script/binary-releases....

If imagemagick is correctly installed, you should be able to run it from command line in any directory, on both Windows and Linux.

WARNING: Mogrify will operate directly on the .bmp files. If you want to keep the originals that you ripped, back them up before you proceed.

Now, go to the output directory, where you have the .bmp frames and the audio.raw.

To convert to 1-bit monochrome using simple threshold:

mogrify -dither None -monochrome *.bmp

To convert to 1-bit monochrome using Floyd-Steninberg dithering:

mogrify -dither FloydSteinberg -monochrome *.bmp

THIS WILL TAKE TIME, EVEN ON A FAST MACHINE

Step 5: Rejoining Video and Audio Into a Single Stream

Now, we need 2 tools that I wrote myself.

You can either download Windows executables I made (don't worry, they are simple command line utilities, there's no viruses or that sort of bulls*) from the link I provide here, or you can copy paste the code and compile in your favourite environment.

1. Put jvid.exe (or the executable you compiled from jvid.cpp) into the output folder and run it

This will read the .bmp file header, decide if foreground / background need to be flipped, strip the image data from the data area and join all frames image data into one chunk into a file called output.jvd

2. Put jmerge.exe (or the executable you compiled from jmerge.cpp) into the output folder and run it

This will join together output.jvd (the image stream) and audio.raw (the 8-bit raw audio file you created earlier. BOTH have to be in the same directory as the jmerge executable. The output is a file called video.jvd

VIDEO.JVD is the end result video file that will be put on the SD card !

Question: why didn't you make a nicer interface four your tools ? Why did you choose such silly names ?

Answer: this was an experiment. I am not interested in developing this further. Look at the code and make better tools yourself.

EDIT: In case you get "missing libgcc..." with jvid.exe or jmerge.exe I added libgcc runtimes (Libraries.zip) to the downloads. Download and add them to the same folder if you get "missing libgcc..." error.

Step 6: Playback Program (sketch) for the Arduino

Now that we have copied video.jvd onto the SD card, we only need the player program on the Arduino/Gamebuino/Fakebuino/whatever you have created to play back the video.

In order to use this sketch, you need 2 libraries for the Arduino

1. TimerOne library

https://code.google.com/p/arduino-timerone/downloa...

2. PetitFS library (already included in the Gamebuino library, if you downloaded that)

http://elm-chan.org/fsw/ff/00index_p.html

There are some explanations within the sketch on how the code works but basically you are on your own. Can't be bothered to clean it up. Sorry.

Step 7: Conclusion & Caveat Emptor

Caveat Emptor

[Latin, Let the buyer beware.] A warning that notifies a buyer that the goods he or she is buying are "as is," or subject to all defects.

What I mean: use all this information on your own risk. Tech support on this article will be minimal, due to lack of time.

Conclusion

Trying to cram a video onto a 84x48 pixel screen was very fun. I had no previous knowledge of the tools needed for this excercise, I figured it out pretty much by myself - I knew what I needed, though, thanks to prior work by others.

Thanks to Myndale for his Gamebuino audio streaming demo, and to Elm-chan for the wonderful PetitFs library. Thanks to Aurelien Rodot for inspiring with his Gamebuino project. Thanks to Joseph Rautenbach, who did the Rick Astley video on Arduino, which had no sound, but which got me thinking about how it could be done. You can see his video here.

1 Person Made This Project!

Recommendations

  • The 1000th Contest

    The 1000th Contest
  • Battery Powered Contest

    Battery Powered Contest
  • Hand Tools Only Challenge

    Hand Tools Only Challenge

32 Discussions

0
wwasantha
wwasantha

4 months ago

Dear jonne, if i need to convert whole set of dithered images in single shot in to their invert colors (negative appearence) by command prompt of the pc;what should be the command you suggest, please help me it is really really need now

0
wwasantha
wwasantha

4 months ago

Dear Jonne,
I made this project successfully. but after playing a video it seems playing audio and video a little bit faster it causes audio deformity human voices like voices of robots. audacity and dithering seems to be ok i think change occurs wen audio-video rejoining by jmerge.exe and jvid.exe
you told the above exe tools wrote by your self. Did you do any alteration in the speed of rejoining in these tools program before uploading? could you please reset the speed normal/ reduce the speed normal.

0
wwasantha
wwasantha

Question 5 months ago

please be kind to explain in detail step 4 and 5 if anyone done this project successfully ??? i'm stuck at that level now. please help.

1
jonnection
jonnection

Answer 5 months ago

Could you be more specific what is the problem you are having? Morgrify, or the jvid & jmerge tools?

0
wwasantha
wwasantha

Reply 5 months ago

thanks in advance for replying me with busy work schedule!!!
i still stuck at mogrify. wanted to complete this project !!!

please give me one simple help on step 4 please tell me COMPLETE COMMAND SENTENCE for mogrify dither floydstetnberg of grayscale bmp prepared in step3 as "mogrify -dither FloydSteinberg -monochrome *.bmp" not enough to do this.THANKS IN ADVANCE. HAVE A NICE DAY.

0
wwasantha
wwasantha

Reply 5 months ago

ITS DONE,BUT I HAVE MY LAST QUESTION
PetitFS.h is available Gamebuino library of given link is giving errors on uploading says "PFFS is not declared in this scope" attempted all possible library file and failed. now i need your help WITHOUT SAYING NO.
i am at near the line to end the race.

please give me a link to download this PetitFS library


Arduino: 1.8.12 (Windows 7), Board: "Arduino Nano, ATmega328P (Old Bootloader)"
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino: In function 'void setup()':
F9BBP7VI7VQL38Q:121:11: error: 'PFFS' was not declared in this scope
int a = PFFS.begin(10, rx, tx);
^~~~
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino:121:11: note: suggested alternative: '_FFS'
int a = PFFS.begin(10, rx, tx);
^~~~
_FFS
F9BBP7VI7VQL38Q:133:7: error: 'pf_read' was not declared in this scope
pf_read((void *)&buffers[0][0],BUFFER_SIZE,&br);
^~~~~~~
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino:133:7: note: suggested alternative: 'fread'
pf_read((void *)&buffers[0][0],BUFFER_SIZE,&br);
^~~~~~~
fread
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino: In function 'void loop()':
F9BBP7VI7VQL38Q:148:3: error: 'pf_read' was not declared in this scope
pf_read((void *)&buffers[1][0],BUFFER_SIZE,&br);
^~~~~~~
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino:148:3: note: suggested alternative: 'fread'
pf_read((void *)&buffers[1][0],BUFFER_SIZE,&br);
^~~~~~~
fread
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino: In function 'void forward()':
F9BBP7VI7VQL38Q:178:9: error: 'pf_lseek' was not declared in this scope
res = pf_lseek((frame + SEEKFRAMES) * 1008);
^~~~~~~~
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino:178:9: note: suggested alternative: 'fseek'
res = pf_lseek((frame + SEEKFRAMES) * 1008);
^~~~~~~~
fseek
F9BBP7VI7VQL38Q:185:3: error: 'pf_read' was not declared in this scope
pf_read((void *)&buffers[0][0],BUFFER_SIZE,&br);
^~~~~~~
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino:185:3: note: suggested alternative: 'fread'
pf_read((void *)&buffers[0][0],BUFFER_SIZE,&br);
^~~~~~~
fread
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino: In function 'void rewind()':
F9BBP7VI7VQL38Q:201:12: error: 'pf_lseek' was not declared in this scope
res = pf_lseek((frame - SEEKFRAMES) * 1008);
^~~~~~~~
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino:201:12: note: suggested alternative: 'fseek'
res = pf_lseek((frame - SEEKFRAMES) * 1008);
^~~~~~~~
fseek
F9BBP7VI7VQL38Q:205:11: error: 'pf_lseek' was not declared in this scope
res = pf_lseek(0);
^~~~~~~~
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino:205:11: note: suggested alternative: 'fseek'
res = pf_lseek(0);
^~~~~~~~
fseek
F9BBP7VI7VQL38Q:209:3: error: 'pf_read' was not declared in this scope
pf_read((void *)&buffers[0][0],BUFFER_SIZE,&br);
^~~~~~~
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino:209:3: note: suggested alternative: 'fread'
pf_read((void *)&buffers[0][0],BUFFER_SIZE,&br);
^~~~~~~
fread
exit status 1
'PFFS' was not declared in this scope
This report would have more information with
"Show verbose output during compilation"
option enabled in File -> Preferences.

1
jonnection
jonnection

Reply 5 months ago

PetitFS was written by elm-chan and is available here:

http://elm-chan.org/fsw/ff/00index_p.html

Now, "pffs is not declared" means either the petitfs lib has changed or (more likely) you have not put the library in the correct place in your Arduino.

I can not offer Arduino help because I have not been using Arduino for a long long time

0
wwasantha
wwasantha

5 months ago

my Step 4: Dithering the Frames With Imagemagick "mogrify"

mogrify dithering3.pngmogrify dithering2.pngmogrify dithering4.pngmogrify dithering.png
1
Marcel_X
Marcel_X

2 years ago

Took a while to get it running, but works like a charm now! Tools used in this instructable have evolved, so you have to use "magick mogrify -dither FloydSteinberg -monochrome *.bmp" now to get the dithering to work.

If you have a stereo track in your video stream, make sure to mix them to mono in Audacity.

Great job!

0
wwasantha
wwasantha

Reply 5 months ago

please be kind to explain in detail step 4 and 5 if anyone done this project successfully ??? i'm stuck at that level now. please help.

0
IanA118
IanA118

1 year ago

I know this is late, hope you can see my comment but Iam having a hard time about the petit fatfs library. I clicked the link, extract the zip file and put it on my libraries in arduino but still shows an error. I also download and install the gamebuino library and still have the same error. hope you can help me with this. I like this project very much. Thank you!

0
jonnection
jonnection

Reply 1 year ago

Hi Ian

Could you copy-paste the error text?

Jonne

0
IanA118
IanA118

Reply 1 year ago

Ive fixed the problem now.. Thank you for this project

0
wwasantha
wwasantha

Reply 5 months ago

please be kind to explain in detail step 4 and 5 if anyone done this project successfully ??? i'm stuck at that level now. please help.

0
IanA118
IanA118

Reply 1 year ago

i successfully burn the nano with the boot loader but I cant upload any sketches using the gamebuino as the board. can you help me with this? Thank you very much

ads.png
0
IanA118
IanA118

Reply 1 year ago

I have another problem. do I need to burn bootloader of gameduino to arduino nano before uploading the code?

0
IanA118
IanA118

1 year ago

Ive done it!, but I have a little problem, is it possible to adjust the contrast in your code??

0
jonnection
jonnection

Reply 6 months ago

Hi! There is no way to adjust contrast in the code itself, sorry. You need to look at the LCD initialization and the LCD datasheet to fix it.

1
wwasantha
wwasantha

Tip 6 months ago

this is not a instructable THIS IS A PUZZLE, so every one come and comment here to sort out hidden steps and things once you complete this project successfully .its a help for all interested this