Arduino Based Universal Data Logger

In an era where microcontrollers costing less than a meal are more powerful than the computers that took us to the Moon, and where a 32 GB SD card can hold millions of times more data than those early space missions, data logging is no longer a significant challenge.

However, professional data loggers are not necessarily cheap, while building one is not particularly difficult.

This Instructable describes how I developed a low-cost yet practical data logger that can record a single channel every 7 ms and up to eight channels at similar intervals.

The aim of this project was not only to develop hardware for data storage, but also to create a clean computer interface that makes configuring the data logger and downloading data as seamless as a professional system.

Supplies

The data logger is based on an Arduino Nano, which uses its eight ADC pins to read analog values and writes the data to an SD card using an SD card module.

The detailed bill of material is attached in the pdf in this step.

Please refer the details pdf file for the qty of each component. In case there are any clarifications required, please drop a comment below or you can reach out to me with the contact me section in my profile.

Step 1: CAD Model

The entire assembly is split in three major parts.

1. The main body
2. It acts as the chassis on which the entire sub assembly rests
3. The banana connector assembly
4. It has 12 banana connectors
5. 8 analog channels (green)
6. 4 ground reference (black)
7. As the T-clamp arrangement to grip the rails on the main body
8. Arduino Assembly
9. Hosts the main PCB with the Arduino microcontroller and the SD card.
10. Has buttons, LEDs and the buzzer to act as the HMI for the data logger
11. As the T-clamp arrangement to grip the rails on the main body

Step 2: Excel Configurator and Data Retriver

While designing the configurator, I wanted to ensure that anyone could easily open and use it. Hence I decided to use MS excel and it's versatile vba functionality to load the configuration into the data logger and to extract the data out of the logger.

The configurator can be downloaded from the link aside: Link

And the most convenient way to load the configuration into the logger is the same way we will be extracting the data out, i.e. the SD card. The excel configurator is designed such that:

1. When an SD card is available with the PC, it will be recognised by the excel VBA, and in case there are more than one, the user can select the removable drive where the configuration is to be stored.
2. In case the SD card is inserted post launch of the Userform, the user needs to click on the refresh button which will fetch any and all removable drives which are available with the PC.
3. Once the SD card is selected, the user may decide on which channels are to logged. The 8 channels are listed in the Userfrom and by-default, the code will mark all channels as active. The user may click on the off button to turn it off. (Note that this selection is only temporary and it has to be pushed on to the SD card for the data logger to behave in the said pattern)
4. The user is also supposed to select the delay between two data logs.
5. Note that is minimum setting, and if the value is put too low, the data logger can take more time than what is mentioned here.
6. When operating all 8 channels, the data logger can take upto 18ms between two readings and
7. When operating only one channel, the data logger can take upto 8ms between two readings
8. Once the channels are set and the delay time is put in, the user can click on save config button, which will push the data to the two config files which are saved onto the SD card. The Arduino uC inside the data logger can read these two files and based will log data based on details in these two files, will know which channels are to be logged, and at what interval

Step 3: Building the Electronic Circuit

The schematic of the data logger is attached in the pdf file in this step. The heart of the data logger is an Arudino Nano uC. It uses the 8 analog channels to read the data, and stores the data on to the SD card.

To control the data logger, two on-board tactile push buttons are provided. And for the user to be able to understand the status of the logging, the system has an on board RGB led, and a buzzer to feedback about the status of the log.

Please note that since the input channels are directly connected to the analog input channels of the Arduino micro controller, no input should cross the threshold of 5V. Beyond 5V, we risk permanently damaging the micro controller.

Further when soldering the physical circuit together, please make a note of the physical location of the buttons and the LED lights on my perf board. The cutouts for the LED, the buttons and the buzzer are based on the location of the component on my board, and hence in case you are replicating the project, kindly make sure that you solder the components on the PCB at the same location as my circuit.

Step 4: Mechanical Assembly

The steps to mechanically assemble the device are as follows:

1. Add the nut in the T clamp
2. Add the nut in the PCB holder
3. Place the PCB inside the circuit holder
4. Add the button cover inside the square cavity for the covers
5. Place the PCB along with the two holders into its spot and tighten the bolts holding the PCB to the Ardino plate. Tighten the PCB on the Arduino plate with M3-16L CSK bolts
6. Add the T- clamp with the bolt to its spot, and holt it in place with the M3-16L CSK bolt. Do not tighten it.
7. Assemble all the banana connectors on to the banana connector plate. (Not shown in the images). Add the T-clamps like mentioned in step 5
8. Make wiring connections between the banana connectors and the main PCB.
9. Slide the banana connector plate first on to the main body followed by the arduino plate. Tighten the M3-16L CSK bolts to hold the banana plate and the Arduino plate on to the main body

Step 5: Arduino Code

This project had quite a few iterations before it reached its final specifications.

Background:

Original idea was also to include an RTC and a display so that the logger can log the abosolute time when storing data. However, unsurprisingly, this was too ambitious for Arduino. And with the advent of AI, my thought this might be the most trivial part of the project.

But ehh! Unfortunately, I simply ran out of memory when just importing these libraries. And, when you dump the entire project on to AI, the code that it designs is wayyyyy to sophisticated, and this project was no exception. The original codes were complicated to such extent that the uC had stability issues and in addition to that it did not even have ram spared to store the data for one of the logs so that the number of writes to the SD card can be kept in control.

However, I acknowledge and thank AI for helping me out! 🍻

Compressed ZIP file: Link

Step 6: Uploading Configuration and Downloading Data

Uploading configuration and Downloading Data: (Link)

1. Insert the SD Card into the computer
2. Launch the DataLogConfigurator.xlsm - Macro enabled worksheet. Your sd card should show up in the removable drive tab. In case it is not seen, click on the refresh button
3. Note: if there are more than one removable drives attached to the computer, kindly ensure you select the SD card which is to be connected with the data logger
4. Select the channels for which you wish to store data. By default, all channels will be turned on. Ensure you turn off the ones that you dont need.
5. Select the time delay between two intervals. Please note, the fastest the data logger can store data is with 7ms between reading when using only one channel and 17ms when using all 8 channels. Hence, if the delay entered in the data logger config is less than the said value, the writing speed will be a limiting factor and the system will be capped at 7 and 17ms, for 1 and 8 channels, respetively.
6. (Refer next steps to understand starting and stopping the data log)
7. To download the data to the computer, insert the SD card into the computer and make sure the configurator recognises the SD card.
8. Click on extract logs button.
9. The data logger will ask for an optional userform where the user can input the details of the test or event that was being logged. Say if the channel 1 was logging voltage and channel 2 was logging current for a DC motor, the user can enter the said details in this user from so that one can understand which channel was used for what, later.
10. Click on Save button. The data logger will store all the data on to the computer and at the same time will delete the logs from the SD card
11. The documents will be stored in a folder named Extract_YYYY-MM-DD-HH-MM-SS.
12. Please note that the date and time mentioned here is the time of extraction and not the time of logging.
13. Each file which is present on the SD card will be stored in the above mentioned folder with serial number as the suffix, eg:
14. Extract_YYYY-MM-DD-HH-MM-SS_1
15. Extract_YYYY-MM-DD-HH-MM-SS_2
16. Extract_YYYY-MM-DD-HH-MM-SS_3, etc
17. The folder Extract_YYYY-MM-DD-HH-MM-SS will be placed inside a folder called Data Logs, which is stored in the same location as the DataLogConfigurator.xlsm file.
18. Post save, the data logger will ask if you wish to open the last stored file in the datalogger. If the user clicks yes, the data logger will open the file Extract_YYYY-MM-DD-HH-MM-SS_XX, with highest XX number
19. The user can click on the LogFolder button on the user form, and it will automatically open the Data Logsfolder

Step 7: Using the Data Logger

Insert the SD card into the data logger and power it on, either with a power bank or with the computer. In case the data logger does not recogonise the SD card, the data logger will give an error with following beep pattern and a red light: Link

To start the data logging normally, proceed with following steps: (Link)

1. Insert the SD Card and power it on with the computer or power bank. If the data logger detects the SDand the config, it will give a single beep and a green light as shown at timestamp 0:07
2. To start the data logging, press and hold both buttons simultaneously. The system will give a long beep, followed by a short beep, post which the system will start data logging. The green led will start blinking, indicating that the system is now logging data.
3. In case you wish to turn off the led, but keep data logging, press the top button once. The setting can be reversed by pressing the top button again. (0:30)
4. In case you wish to turn off the buzzer functionality, press the lower button once. The setting can be reversed by pressing the top button again. (0:39)
5. To stop the data logging, the data logger will give a long beep, followed by a short beep, indicating that the data log has been saved. The green blinking led will return to a solid green indicating that the system is now ready to start a new data log. (0:47)

Power failure:

1. The data logger stores a variable on to the SD card indicating if it is recording data or ready to log data. This variable is read at the startup the the datalogger.
2. If the power to the data logger had failed during the logging, the variable would read as data log active.
3. If the data logger recognises an interrupted data log, it will automatically start the data logging at startup with out any manual intervention. It will also give an indication that the data log was interrupted, in the csv file that it is generating.