loading

Step 7: SCRUBBING & FORMATTING DATA WITH A SPREADSHEET

With enough road data captured on the SD card, we now start the data scrubbing and re-formatting in preparation for analysis and visualization in Excel, GPSvisualizer, or our favorite data analysis and charting tool. 

Remove the SD card from the socket and insert into PC SD card reader. Copy the GPS.log file to your PC and Run Excel (or your favorite spreadsheet application). The steps below are for Excel 2007
  • STEP 1: From Excel, File/Open and select GPS.log. Make sure you select File Type All Files (*.*) else you will not see the file GPS.log listed. Open file. This will launch the Text Import Wizard.
  • STEP 2: Select Delimited radio button. Click Next.
  • STEP 3: Select Comma check box only. Cick Next.
  • SETP 4: Excel will import the GPS.log file into columns and rows. The columns are ordered  in this manner: X, Y, Z, NMEA output type, UTC Time, Status A: A=data valid or V=data not valid, Latitude, N/S: Indicator N N=north or S=south, Longitude, E/W Indicator: E E=east or W=west, Speed over ground, Course over ground, Date, Magnetic variation, Variation sense: E=east or W=west, Mode A: A=autonomous, D=DGPS, E=DR, Checksum.
  • STEP 5,6: In column 'F' you will see one letter either A or V. A means valid fix. V means invalid data. So delete all rows that are invalid.
  • STEP 7: Also, delete jumbled lines. 
  • STEP 8: Keep columns C (z-axis), G (Latitude), and I (Longitude) but hide other imported columns. 
  • STEP 9: Add header labels to the top of the remaining three columns: N, Latitude,  Longitude.
  • STEP 10: Now select and copy to clipboard the range of rows you wish to map in GPSvisualizer and don't forget to also copy the columns header labels. 
With the selected data in the clipboard, we are ready to paste it into GPSvisualizer so we can map and analyze our logged data.
<p>Can anyone help please? I am new <br>to this, and been trying to put together the project. So far I have everything <br>assembled as instructed. However, as soon as I try to verify the code, I get <br>this error:</p><p>Arduino: 1.6.5 (Windows 7), Board: &quot;Arduino/Genuino <br>Uno&quot;</p><p>C:\Program Files <br>(x86)\Arduino\libraries\SD\src\utility\Sd2Card.cpp:26:17: fatal error: SPI.h: <br>No such file or directory</p><p>#include <br>&lt;SPI.h&gt;</p><p>^</p><p>compilation terminated.</p><p>Error compiling.</p><p>What do I need to do to fix this?</p>
<p>Newer versions of Arduino IDE can't find the SPI library, so you need to add them by adding the following line at the start of sketch:<br>#include &lt;SPI.h&gt;<br></p>
<p>I'm no expert but had this same problem on another project. All of the files (including the additional libraries) need to be in the right folder. The Arduino IDE seems to be a bit picky about this (on Linux at least).</p>
<p>can i have the code for storing Accelerometer ADXL335 data into SD card with time staps (say every 5 min interbal)</p><p>and how can name the stored file as a YYYYMMDDHHSS format </p>
<p>could not get code to compile</p>
<p>I made an equivalent tool in python that print real time data from ADXL345 accelerometer.</p>
<p>like</p>
<p>I made an equivalent tool in python that print real time data from ADXL345 accelerometer.</p><p><a href="https://github.com/mba7/SerialPort-RealTime-Data-Plotter" rel="nofollow">https://github.com/mba7/SerialPort-RealTime-Data-P...</a></p><p>may be it will be helpful for someone</p>
<p>I made an equivalent tool in python that print real time data from ADXL345 accelerometer.</p><p><a href="https://github.com/mba7/SerialPort-RealTime-Data-Plotter" rel="nofollow">https://github.com/mba7/SerialPort-RealTime-Data-P...</a></p><p>may be it will be helpful for someone</p>
<p>I need create a file GPS.log in SD card before do this work ? Or the file GPS.log will be created by code?</p>
<p>You can read the accelerometer more often than the GPS, so there should be no need to interpolate the accelerometer reading.</p>
<p>great</p>
<p>good</p>
<p>super</p>
<p>super</p>
<p>In this tutorial I find what I need to start interfacing the gps sensor with an Arduino. And because I want to help many more hobbyists to start building robots, I share this tutorial on my post http://www.intorobotics.com/gps-sensors-tutorials-resources/. Thank you!</p>
<p>Hello techbitar - I'm thinking of building an adaption of your project but am considering changing the SD card adapter to the microSD card shield from Sparkfun: https://www.sparkfun.com/products/9802</p>
True but not all sensors (temperature, humid, etc.) possibly used by others will be as fast as the accelerometer I am using in my project.
Hello techbitar, <br> <br>I just ordered all the hardware for this project, and I should be beginning it some time next week. I just had a question. Obviously all of these components would need to be powered up whilst in car data collecting while driving, so would one of these do the job: http://www.radioshack.com/product/index.jsp?productId=3802146 . Also, I got this for the 'in-home' programming and construction of the actual project: https://www.sparkfun.com/products/9442? ..... Do you think that'll be good as well? I would assume so.
The Arduino Uno specs are as follows: <br>Input Voltage (recommended) 7-12V <br>Input Voltage (limits) 6-20V <br> <br>As for current, I have tested my Bump-O-Meter with a USB connection which provides 5V and up to 0.5A. I did not run into any issues but I did not perform any exhaustive tests. <br> <br>While driving around in my car, I powered the Arduino Uno via the DC plug using a battery brick made of 6 X 1.2V (NiMh) = 7.2V @ 2A. That voltage is close to the minimum recommended voltage. It worked but of course the moment the battery dropped to 6V and below I am sure I would have ran into all sorts of issues. <br> <br>If I go production with this, I would use a 3S LiPo for a cool 11.1 Volts and 1.5A or 2A current for safety margin in case I need to add additional power hungry gizmos to the bump-o-meter. <br> <br>Check you choices of power supplies against the min/max voltage specs and add a comfortable margin to the current with an eye on your future plans. then decide what fits your requirements best.
I was riding my bike recently and remarked to another rider that it would be useful to quantify just how bad our roads are. My thinking is to first breadboard components as you have laid them out, and then substitute an Arduino Micro and micro SD card reader for compactness. While I like LiOn batteries, a 3S 11.1V 2 amp battery is fairly large (and heavy and needs a requisite charger). Do you have an idea for a compact battery with a 3-hour runtime?
Hello techbitar - I'm thinking of building an adaption of your project but am considering changing the SD card adapter to the microSD card shield from Sparkfun: https://www.sparkfun.com/products/9802 <br> <br>Do you see any obvious problems with this change, and would it break the code you've written? Cheers
I have not tried it but if it Sparkfun's SD adapter does not work out of the box you may need to tweak my circuit and/or code.
You can read the accelerometer more often than the GPS, so there should be no need to interpolate the accelerometer reading.
True but not all sensors (temperature, humid, etc.) possibly used by others will be as fast as the accelerometer I am using in my project.
You're welcome.

About This Instructable

150,015views

479favorites

License:

Bio: Did I unplug the solder iron?
More by techbitar:IR Remote Control Station for Android - TURN THE TV DOWNSensoDuino: Turn Your Android Phone into a Wireless Sensors Hub for ArduinoModify The HC-05 Bluetooth Module Defaults Using AT Commands
Add instructable to: