Streaming Video From SD Card to Nokia LCD With Arduino
Intro: Streaming Video From SD Card to Nokia LCD With Arduino
For a long time I thought playing back video with audio was not possible on the Arduino. Then one day I decided to give it a shot and it turned out to be simpler than I thought.
I decided to write the steps in an instructable for the benefit of others considering trying it out themselves.
This write-up is about how to create such a video for playback. It is not intended to be a full-fledged instruction on how to build a device like this !
However, I will tell you how to get one or instructions for building one, if you are interested.
STEP 1: The DIY "Gamebuino" Device and How to Get One
The "Gamebuino" or actually a DIY "Fakebuino" used in this video is just a Nokia 5110 monochrome lcd display wired up to a Arduino Nano (ATMega328). Also included in my build is a SD-card breakout and a PAM 8403 amplifier breakout. All components can be bought very cheap on Ebay. Simply search with the terms in bold above.
Alternatively, you can buy a device like this ready-made at www.gamebuino.com. Aurelien Rodot, the mastermind behind the Gamebuino device has open-sourced his hardware and library. You will find all the information you need by searching his project website.
I have included a Fritzing schematic of how my own device is wired up. Just be aware, that it is not a 1:1 Gamebuino-compatible device, it is just my own version of the thing. Take note: the PAM 8403 chip is not a chip, the pins refer to connections on a PAM 8403 breakout.
Also, I will not explain how to use a Nokia 5110 LCD / SD card breakout with Arduino, even if asked. Both topics have been extensively covered by others and you will have to do your own research.
STEP 2: Ripping the Video Into Numbered Bitmap Frames With Ffmpeg
The video used in this demonstration was saved from Youtube. Again, I will not explain that process here. Just Google it. In any case, I saved the video as a low resolution 3gp file, since I knew it was going to end up in a very very low resolution in the end anyhow.
So, after you have yourself a source video you wish to rip, you need to get and install yourself ffmpeg if you do not already have it.
https://www.ffmpeg.org/download.html
The video was ripped into 84x48 grayscale bitmap files at 20 frames per second using the following command line:
ffmpeg -i myvideo.3gp -r 20 -f image2 -vf format=gray -s 84x48 .\output\%d.bmp
Explanation of parameters:
- ffmpeg is the program. If installed correctly, you should be able to call this from any directory on your Win/Linux box
- myvideo.3gp is the name of the source video. You need to be in the directory of the source video when you run this command. Otherwise ffmpeg will not find your source video. The video can be any format, here it just happens to be a .3gp. Could be a a .mpg or .mp4 or whatnot.
- -r 20 is the frame rate at which we will rip the frames, 20 frames per second
- -f image2 is the output format (output video as image frames, not a single video file)
- -vf format=gray means make the output format grayscale
- -s 84x48 is the pixel size of the output frames (84 by 48 pixels for Nokia 5110 LCD)
- .\output\%d.bmp is the output directory and output naming for the ripped frames. You need to make a "output" folder before you run this command ! The %d.bmp simply means to name files as follows: 1.bmp 2.bmp 3.bmp ...
STEP 3: Ripping the Audio to 8-bit RAW With Audacity
Go get and install Audacity, if you haven't got it already.
http://audacity.sourceforge.net/download/
Audacity needs both the LAME and ffmpeg libraries to work !! Go check from Preferences->Libraries that Audacity lists both the LAME and the ffmpeg version. If no version numbers are given, it means that Audacity has not located the libraries and the next steps will not work ! Again, I am not going to give technical advice on using Audacity. Seems harsh but believe me, there are hundreds of tutorial on Audacity already. I am not going to repeat that information here.
Next, open up the video file in Audacity. Thanks to ffmpeg, it can directly rip the audio into Audacity (this is why we need the libs in the previous steps, you see).
NOW, set project rate at 10080 Hz (see image). This is important because it will sync the video and audio. As we will be playing back video at 20 frames per second = 504 x 20 = 10080 bytes of image data per second, therefore we need also 10080 bytes of audio per second. THIS IS THE MOST IMPORTANT TRICK IN THE WHOLE THING !
NEXT, export audio as 8-bit unsigned headerless RAW (see image)
SAVE file as "audio.raw" into the output folder containing the ripped .bmp frames
STEP 4: Dithering the Frames With Imagemagick "mogrify"
Next, we need to make 1-bit black & white images from the grayscale frames we ripped.
For that, we need to download and install Imagemagick:
http://www.imagemagick.org/script/binary-releases....
If imagemagick is correctly installed, you should be able to run it from command line in any directory, on both Windows and Linux.
WARNING: Mogrify will operate directly on the .bmp files. If you want to keep the originals that you ripped, back them up before you proceed.
Now, go to the output directory, where you have the .bmp frames and the audio.raw.
To convert to 1-bit monochrome using simple threshold:
mogrify -dither None -monochrome *.bmp
To convert to 1-bit monochrome using Floyd-Steninberg dithering:
mogrify -dither FloydSteinberg -monochrome *.bmp
THIS WILL TAKE TIME, EVEN ON A FAST MACHINE
STEP 5: Rejoining Video and Audio Into a Single Stream
Now, we need 2 tools that I wrote myself.
You can either download Windows executables I made (don't worry, they are simple command line utilities, there's no viruses or that sort of bulls*) from the link I provide here, or you can copy paste the code and compile in your favourite environment.
1. Put jvid.exe (or the executable you compiled from jvid.cpp) into the output folder and run it
This will read the .bmp file header, decide if foreground / background need to be flipped, strip the image data from the data area and join all frames image data into one chunk into a file called output.jvd
2. Put jmerge.exe (or the executable you compiled from jmerge.cpp) into the output folder and run it
This will join together output.jvd (the image stream) and audio.raw (the 8-bit raw audio file you created earlier. BOTH have to be in the same directory as the jmerge executable. The output is a file called video.jvd
VIDEO.JVD is the end result video file that will be put on the SD card !
Question: why didn't you make a nicer interface four your tools ? Why did you choose such silly names ?
Answer: this was an experiment. I am not interested in developing this further. Look at the code and make better tools yourself.
EDIT: In case you get "missing libgcc..." with jvid.exe or jmerge.exe I added libgcc runtimes (Libraries.zip) to the downloads. Download and add them to the same folder if you get "missing libgcc..." error.
STEP 6: Playback Program (sketch) for the Arduino
Now that we have copied video.jvd onto the SD card, we only need the player program on the Arduino/Gamebuino/Fakebuino/whatever you have created to play back the video.
In order to use this sketch, you need 2 libraries for the Arduino
1. TimerOne library
https://code.google.com/p/arduino-timerone/downloa...
2. PetitFS library (already included in the Gamebuino library, if you downloaded that)
http://elm-chan.org/fsw/ff/00index_p.html
There are some explanations within the sketch on how the code works but basically you are on your own. Can't be bothered to clean it up. Sorry.
STEP 7: Conclusion & Caveat Emptor
Caveat Emptor
[Latin, Let the buyer beware.] A warning that notifies a buyer that the goods he or she is buying are "as is," or subject to all defects.
What I mean: use all this information on your own risk. Tech support on this article will be minimal, due to lack of time.
Conclusion
Trying to cram a video onto a 84x48 pixel screen was very fun. I had no previous knowledge of the tools needed for this excercise, I figured it out pretty much by myself - I knew what I needed, though, thanks to prior work by others.
Thanks to Myndale for his Gamebuino audio streaming demo, and to Elm-chan for the wonderful PetitFs library. Thanks to Aurelien Rodot for inspiring with his Gamebuino project. Thanks to Joseph Rautenbach, who did the Rick Astley video on Arduino, which had no sound, but which got me thinking about how it could be done. You can see his video here.
32 Comments
wwasantha 3 years ago
wwasantha 3 years ago
I made this project successfully. but after playing a video it seems playing audio and video a little bit faster it causes audio deformity human voices like voices of robots. audacity and dithering seems to be ok i think change occurs wen audio-video rejoining by jmerge.exe and jvid.exe
you told the above exe tools wrote by your self. Did you do any alteration in the speed of rejoining in these tools program before uploading? could you please reset the speed normal/ reduce the speed normal.
wwasantha 4 years ago
jonnection 4 years ago
wwasantha 4 years ago
i still stuck at mogrify. wanted to complete this project !!!
please give me one simple help on step 4 please tell me COMPLETE COMMAND SENTENCE for mogrify dither floydstetnberg of grayscale bmp prepared in step3 as "mogrify -dither FloydSteinberg -monochrome *.bmp" not enough to do this.THANKS IN ADVANCE. HAVE A NICE DAY.
wwasantha 4 years ago
PetitFS.h is available Gamebuino library of given link is giving errors on uploading says "PFFS is not declared in this scope" attempted all possible library file and failed. now i need your help WITHOUT SAYING NO.
i am at near the line to end the race.
please give me a link to download this PetitFS library
Arduino: 1.8.12 (Windows 7), Board: "Arduino Nano, ATmega328P (Old Bootloader)"
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino: In function 'void setup()':
F9BBP7VI7VQL38Q:121:11: error: 'PFFS' was not declared in this scope
int a = PFFS.begin(10, rx, tx);
^~~~
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino:121:11: note: suggested alternative: '_FFS'
int a = PFFS.begin(10, rx, tx);
^~~~
_FFS
F9BBP7VI7VQL38Q:133:7: error: 'pf_read' was not declared in this scope
pf_read((void *)&buffers[0][0],BUFFER_SIZE,&br);
^~~~~~~
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino:133:7: note: suggested alternative: 'fread'
pf_read((void *)&buffers[0][0],BUFFER_SIZE,&br);
^~~~~~~
fread
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino: In function 'void loop()':
F9BBP7VI7VQL38Q:148:3: error: 'pf_read' was not declared in this scope
pf_read((void *)&buffers[1][0],BUFFER_SIZE,&br);
^~~~~~~
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino:148:3: note: suggested alternative: 'fread'
pf_read((void *)&buffers[1][0],BUFFER_SIZE,&br);
^~~~~~~
fread
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino: In function 'void forward()':
F9BBP7VI7VQL38Q:178:9: error: 'pf_lseek' was not declared in this scope
res = pf_lseek((frame + SEEKFRAMES) * 1008);
^~~~~~~~
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino:178:9: note: suggested alternative: 'fseek'
res = pf_lseek((frame + SEEKFRAMES) * 1008);
^~~~~~~~
fseek
F9BBP7VI7VQL38Q:185:3: error: 'pf_read' was not declared in this scope
pf_read((void *)&buffers[0][0],BUFFER_SIZE,&br);
^~~~~~~
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino:185:3: note: suggested alternative: 'fread'
pf_read((void *)&buffers[0][0],BUFFER_SIZE,&br);
^~~~~~~
fread
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino: In function 'void rewind()':
F9BBP7VI7VQL38Q:201:12: error: 'pf_lseek' was not declared in this scope
res = pf_lseek((frame - SEEKFRAMES) * 1008);
^~~~~~~~
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino:201:12: note: suggested alternative: 'fseek'
res = pf_lseek((frame - SEEKFRAMES) * 1008);
^~~~~~~~
fseek
F9BBP7VI7VQL38Q:205:11: error: 'pf_lseek' was not declared in this scope
res = pf_lseek(0);
^~~~~~~~
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino:205:11: note: suggested alternative: 'fseek'
res = pf_lseek(0);
^~~~~~~~
fseek
F9BBP7VI7VQL38Q:209:3: error: 'pf_read' was not declared in this scope
pf_read((void *)&buffers[0][0],BUFFER_SIZE,&br);
^~~~~~~
C:\Users\Intel\Downloads\VIDEO PLAY IN NOKIA LCD - ALL FILES\A-ha Take on Me video playback on Arduino)\Streaming Video From SD Card to Nokia LCD With Arduino\F9BBP7VI7VQL38Q\F9BBP7VI7VQL38Q.ino:209:3: note: suggested alternative: 'fread'
pf_read((void *)&buffers[0][0],BUFFER_SIZE,&br);
^~~~~~~
fread
exit status 1
'PFFS' was not declared in this scope
This report would have more information with
"Show verbose output during compilation"
option enabled in File -> Preferences.
jonnection 4 years ago
http://elm-chan.org/fsw/ff/00index_p.html
Now, "pffs is not declared" means either the petitfs lib has changed or (more likely) you have not put the library in the correct place in your Arduino.
I can not offer Arduino help because I have not been using Arduino for a long long time
jonnection 4 years ago
Try this library (greiman's version of PetitFS)
https://github.com/greiman/PetitFS
It is the "Arduino wrapper" version you need
wwasantha 4 years ago
Marcel_X 6 years ago
Took a while to get it running, but works like a charm now! Tools used in this instructable have evolved, so you have to use "magick mogrify -dither FloydSteinberg -monochrome *.bmp" now to get the dithering to work.
If you have a stereo track in your video stream, make sure to mix them to mono in Audacity.
Great job!
wwasantha 4 years ago
IanA118 5 years ago
jonnection 5 years ago
Could you copy-paste the error text?
Jonne
IanA118 5 years ago
wwasantha 4 years ago
IanA118 5 years ago
IanA118 5 years ago
IanA118 5 years ago
jonnection 4 years ago
wwasantha 4 years ago