Wireless SD Card Reader [ESP8266]

18,427

112

21

Introduction: Wireless SD Card Reader [ESP8266]

About: Electrical Engineer and a programming hobbyist! I love to build exciting stuff!

USB was supposed to be universal, and the main goal was to make a hot-swappable, super easy to interface with other devices but over the years the idea went haywire. There are so many different variants of these USB ports which is so frustrating at times and the way these works totally contradict their name [USB - Universal Serial Bus] because Every USB receiver should be compatible with any USB device! You can't plug in your USB stick or a keyboard inside a charger and expect it to work.

But the concept sounds too good! That's why to initiate this "Universal-Port" concept I started with a simple project "Wireless Card reader"

This fulfilled all my wishes, all I have to do is to just plug it inside any USB receiver, it doesn't matter which one!

As soon as you plug it in, it creates an access point where we can connect and then connect to the access point and just open up any FTP client application in any compatible device. With this setup, we can copy as well as save files to the SD card wirelessly!

Supplies

These are the list of products which can help you do this project with ease

(Affiliate Link)

Step 1: SD(Secure Digital) Card

SD stands for Secure Digital, it’s similar to your Pendrive but with a smaller footprint and much cheaper price.

When we have to use this with any of the microcontrollers there are 2 options, one is SDIO and SPI. Almost all the SD cards share many standard features and have the same physical and electrical specifications. The actual differences between the SPI and SDIO are mainly on the software level. You can read more about it in this Link.

For now, let’s just say SDIO is faster but harder to implement and SPI is slower but easier to implement. Since most of the microcontrollers support SPI by default we'll just stick on to it.

SD card pinout for SPI

Pin-1 - CS (Chip select)
Pin-2 - DI (MOSI)
Pin-3 - GND
Pin-4 - VCC
Pin-5 - SCLK
Pin-6 - GND
Pin-7 - DO (MISO)
Pin-8 - NC
Pin-9 - NC

Step 2: SD Card Adapter Modification

You can use any SD card modules which support Arduino and esp8266 but for the purpose of this project, we'll use the microSD card adapter and will modify in such a way that we can use that instead of the module.

First, clean the contacts of the SD card adapter. Then use angled header pins and solder the pins directly to the adapter contacts. Once the soldering is done, check the contacts between the header pins to check for any short circuit. Remove the black separator one by one, so when we place it back, it would flush with the PCB.

Cut the PCB in such a way that it fits perfectly with the SD card adapter and has some extra space, to add the male USB port.

You can also do the same process with the SD card instead of the adapter, but it's quite risky if you damage it.

Step 3: USB Connection

We need to power the SD card, For that, we'll be using the USB receiving port itself. So we'll use a male USB port. This usually has 4 pins, where 2 middle pins are used for data transfer and 2 extreme pins are used for power and ground. Since we just need power, I will cut data pins and just keep the GND and VCC.

Then placed the male USB port in front of the SD card where we made some space earlier, then solder it in place. This did not solve any power issue yet! Because the SD card requires 3.3v but the USB supply is standard 5V if you just plug this in the supply, you will probably fry out the SD card (But no damages will be taken by the microSD adapter).

To solve this we'll use a 3.3V regulator and connect the input of the USB supply to 3.3V regulator i.e connect GND of the USB to pin 1 of the regulator and connect pin 3 of the regulator to +5V of the regulator. Finally, solder the pin 3 (output pin) and ground of the regulator to the SD card.

This will setup the power for the SD card. You can check the circuit diagram for a more detailed connection.

Step 4: Putting Everything Together With ESP-12E

Now to read and write the data from the SD card we'll use the Esp12E wifi module, even though it's slower than esp32. But it really doesn’t matter which one you choose, I will tell reason in the latter steps.

First solder the EN(enable pin) to the VCC of the esp12E, this will turn on the IC. If this is not connected to the HIGH signal, the IC won't turn on. Then place the esp12E on the back of the PCB board and solder the SPI pins of the esp12E to the SPI pins on the SD card. For the detail, connection checks the circuit diagram.

Step 5: HTTP VS FTP

Before programming, I did some research on how downloads and uploads work, that’s when I stumbled across the word FTP. Basically FTP stands for file transfer protocol, this protocol is used to transfer files between servers and client and it’s totally different from the regular HTTP where the client and server send and receive requests/response which is very small in size.

FTP is faster than the HTTP in transferring files because it was specifically made for it. So, I wanted to implement this in this project. Where an FTP server runs on the esp-12E and we can push and fetch data through this FTP to the SD card.

Step 6: Figuring Out the FTP Library

I couldn’t find any FTP library that’s very actively developed or specifically made for esp8266. But with some digging I came across David Paiva who ported an Arduino version of the FTP server to esp8266 but with the SPIFFS support and not the SD card.

But with little more effort, I found someone who did some work on David Paiva library to convert the SPIFFS to the SD card. But when I tried to use this, I faced 2 issues. First, the page where I found this out was in Korean, So I had to literally sit and translate everything to know what was going on before I could do anything with it. Then the second problem was, I had to modify the existing SD library to support the changes he made but that felt very clumsy.

So, I compared both this library, one from David Paiva and the other one from the Korean website, Then made some minor changes and made the whole thing into a single project so there is no need to install any library of any sort. You can check out the code from my Github account.

Step 7: Programming the ESP-12E

ESP-12E doesn't come with an inbuilt programmer, so we need to use an external programmer like the FDTI module. So I made an adapter with a few wires and female header pins, With this, we can temporarily solder the esp12E and program it using the FTDI module.

Connect the GND[esp12E] to GND, Rx[esp12E] to Tx, Tx[esp12E] to Rx, GPIO15[esp12E] to GND, GPIO0 [esp12E] to GND, VCC[esp12E] to VCC of FDTI module.

Then upload the code from Github using the Arduino IDE.

Once the program is uploaded you can desolder the wires which were connected to program the esp12E.

Step 8: Finishing the Project!

Just Put in any microSD[32 GB max] card inside the adapter and plug the whole device into any USB compatible device, That should power things up! But there are few things to consider, make sure the USB port output current is greater than 1amp, just to be on a safer side. Because the Esp12E module consumes more current when it's transferring files.

Step 9: Using the Device

As soon as the device is powered the device creates an access point called SD Reader. Connect to this access point using the password which is on the code. Then Depending on which device you are using to connect to the 12E download the respective FTP client software if you're using PC download WinSCP or Filzella and if you're using an Android device download AndFTP.

Once the installation is done, open the AndFTP and fill in the credentials to set up the FTP client. In my case, I left the user name and the password to default "esp8266" in the code. So, use that for the user information and for the host use 192.168.12.7. Finally, connect to the FTP server.

Once That’s done, you can download any files from the SD card as well as you can upload files from your phone to the SD card.

You can see the video to know how it works!

Step 10: Final Thoughts

But before jumping into the conclusion that it's a very handy device to have, let’s take a step back.

Even though it does what I want, it is utterly slow! For just 4 files(each ~100Kb) it takes roughly 30secs, and if you try with a larger file like 10MB it will take around 3-4mins to complete. There are ways to optimize this, and from the page where I referred he was able to get roughly 450kbs read speed. (With Esp32 and SD_MMC library transfer speed may be around 1MB/sec)

The reason why I stopped the project here and didn't try optimizing it was because of 2 reasons. First reason, I really wish, along with the FTP server I could still use the USB data line to transfer data, but it’s not supported in esp8266 or esp32. And the second reason is that I couldn’t get enough speed to transfer the files over FTP. These are also the same reason as to why I didn’t bother to use the esp32 instead of the esp12E.

But I think some of these problems can be solved if we can use the esp32 S2 boards which supports full speed on the go USB. Maybe I can do that for another instructable XD.

1 Person Made This Project!

Recommendations

  • Summer Fun: Student Design Challenge

    Summer Fun: Student Design Challenge
  • Water Speed Challenge

    Water Speed Challenge
  • DIY Summer Camp Contest

    DIY Summer Camp Contest

21 Comments

0
Andred56
Andred56

7 weeks ago

Hi what do I need to do to get this code to work on wemos. Its say it can't load to wemos

0
Neutrino-1
Neutrino-1

Reply 7 weeks ago

It should work with wemos as well, can you post a screenshot of the error you are facing?

0
ssfida2006
ssfida2006

Question 4 months ago

Dear sir, your project is great! I congratulate you on your hard work.
Would you please show me a way how to use this project without using access-point, because I want to use my router in this project. I want to connect the esp8266 to my router, and also the FTP client to the router. I have a Desktop PC, which doesn't have an inbuilt WIFI module, so I am unable to use this method with my PC unless I use my router in this project. I can use my phone for this, but if I could use it on my PC, it would be more helpful. I am waiting for your reply.

If someone else knows how to switch from AP mode to router mode, please also reply.
Thanks!

0
Neutrino-1
Neutrino-1

Reply 4 months ago

(wireless_SD.ino file)
Replace line 5 and 6 with your route SSID and password and replace from line 23
WiFi.mode(WIFI_STA);
WiFi.disconnect();

delay(100);

WiFi.config(apIP, gateway, subnet);

WiFi.begin(ssid, password);

This would make the esp8266 work with your router.

1
rudyauction509
rudyauction509

4 months ago

Just a follow up. I had enough fun getting this working on ESP32 that I decided to get it working with 4 bit SDIO on ESP32-CAM. Got it working easily, just replaced every SD call with SD_MMC. Speeds are near identical to the SPI interface, apparently limited by WiFi and not SD read/write time. Note that, at least on my board, accessing the microSD card causes the flash LED to strobe.

1
rudyauction509
rudyauction509

Reply 4 months ago

Follow up to my follow up (lol): after switching power of the ESP32-CAM from the USB-TTL chip to its own USB cable, I’m now getting 600KBPS read and 260KBPS write, about double what the original project could handle.

0
Neutrino-1
Neutrino-1

Reply 4 months ago

That's really impressive!

0
Neutrino-1
Neutrino-1

Reply 4 months ago

Flash led is connected to GPIO-4, same as DATA-1 pin on the SD card. This won't be a problem when you use the SD card in SPI mode but in SDIO it uses DATA-1 along with other pins, maybe this is the reason for strobes.

pinout esp32.PNG
0
michal.zima01
michal.zima01

5 months ago

hello. I really appreciate your work. It's is amazing. It's almost perfect, there's only one thing that I want to know...how can you access different directories and upload/download files from there...I tried to find out how to do it for last 4 hours but I couldn't do it. The problem is, I can see them, but only as a undefined files with 0 bytes and if I try to open them, they only open in text editor and print some weird stuff. I will be very happy if you could help me vith this. THANKS!

1
rudyauction509
rudyauction509

Question 7 months ago on Step 3

Does anything need to be changed to use this code on an ESP32? Just got my 1st a few days ago and a tiny FTP server for small files could be very handy.

0
rudyauction509
rudyauction509

Answer 6 months ago

Never mind. Fought with this for 16 hours straight. I can connect to the access point, that’s as far as I can go. Can’t access microSD, can’t start an FTP connection. I give up. Best I can get is “target device is actively refusing connection”.

0
rudyauction509
rudyauction509

Reply 6 months ago

Ok looks like my problem is hardware related. I ordered a 3 pack of microSD breakouts, and all 3 fail to read any of my known good cards (ranging from 4gb to 64gb) on any of the 15 microcontrollers I tested them on (yes I fed the breakouts the 5v needed to feed their 3.3v regulator). I’ll try again with new microSD breakouts from a different brand when they arrive.

0
rudyauction509
rudyauction509

Reply 6 months ago

WORKING on new microSD breakouts, software wise all I had to do is change the WiFi library from ESP8266WiFi.h to just WiFi.h, and change the CS pin on the microSD. Also, as of Arduino 1.8.13, the other files must me opened manually, but only the first time.

0
Neutrino-1
Neutrino-1

Reply 6 months ago

Sorry couldn't help your sooner, But you have done a great job! Hope your solution helps others who is facing similar issue.

0
rudyauction509
rudyauction509

Reply 6 months ago

Thanks, was definitely difficult for me because this is the most advanced project I’ve taken on so far, though I was surprised how easy the code was for me to understand. Was very satisfying to finally see that transfer happening. I had a few more intermittent issues after 1st getting it going, though they were fixed when I moved the project off of the breadboard and onto a permanent PCB prototype board.

image.jpg
1
amansinghaljpr
amansinghaljpr

1 year ago

Trust me man the work which you did is really really amazing. It's actually people like you who changed the world with their little inventions. I really imagine how would the system perform once you are on esp32 chip with the code and Library optimizations. Also so I would be more than happy if we could ever collaborate on a project . amansinghaljpr@gmail.com this is my mail just ping me and let's become friends.

0
Neutrino-1
Neutrino-1

Reply 1 year ago

Thanks man! I will ping you.

1
banman11
banman11

1 year ago


Very nice project.
Really like your comprehensive video on this project.
I know you said that the speeds were a little slow for your liking. However for some users it will not matter. The ability to have wifi accessible data in a small package is a good thing. What would the theoretical upper size limit on the SD card with your current setup?
Well done!

0
Neutrino-1
Neutrino-1

Reply 1 year ago

Thank you! As far as I know it can support up to 32GB.