loading

Step 7: SCRUBBING & FORMATTING DATA WITH A SPREADSHEET

With enough road data captured on the SD card, we now start the data scrubbing and re-formatting in preparation for analysis and visualization in Excel, GPSvisualizer, or our favorite data analysis and charting tool. 

Remove the SD card from the socket and insert into PC SD card reader. Copy the GPS.log file to your PC and Run Excel (or your favorite spreadsheet application). The steps below are for Excel 2007
  • STEP 1: From Excel, File/Open and select GPS.log. Make sure you select File Type All Files (*.*) else you will not see the file GPS.log listed. Open file. This will launch the Text Import Wizard.
  • STEP 2: Select Delimited radio button. Click Next.
  • STEP 3: Select Comma check box only. Cick Next.
  • SETP 4: Excel will import the GPS.log file into columns and rows. The columns are ordered  in this manner: X, Y, Z, NMEA output type, UTC Time, Status A: A=data valid or V=data not valid, Latitude, N/S: Indicator N N=north or S=south, Longitude, E/W Indicator: E E=east or W=west, Speed over ground, Course over ground, Date, Magnetic variation, Variation sense: E=east or W=west, Mode A: A=autonomous, D=DGPS, E=DR, Checksum.
  • STEP 5,6: In column 'F' you will see one letter either A or V. A means valid fix. V means invalid data. So delete all rows that are invalid.
  • STEP 7: Also, delete jumbled lines. 
  • STEP 8: Keep columns C (z-axis), G (Latitude), and I (Longitude) but hide other imported columns. 
  • STEP 9: Add header labels to the top of the remaining three columns: N, Latitude,  Longitude.
  • STEP 10: Now select and copy to clipboard the range of rows you wish to map in GPSvisualizer and don't forget to also copy the columns header labels. 
With the selected data in the clipboard, we are ready to paste it into GPSvisualizer so we can map and analyze our logged data.
<p>Hi im just wondering whether it will able to track the location like within a room because i want to implement this in a lego track for detecting faulty track.</p><p>Can you assist me with this?</p>
<p>Can anyone help please? I am new <br>to this, and been trying to put together the project. So far I have everything <br>assembled as instructed. However, as soon as I try to verify the code, I get <br>this error:</p><p>Arduino: 1.6.5 (Windows 7), Board: &quot;Arduino/Genuino <br>Uno&quot;</p><p>C:\Program Files <br>(x86)\Arduino\libraries\SD\src\utility\Sd2Card.cpp:26:17: fatal error: SPI.h: <br>No such file or directory</p><p>#include <br>&lt;SPI.h&gt;</p><p>^</p><p>compilation terminated.</p><p>Error compiling.</p><p>What do I need to do to fix this?</p>
<p>Newer versions of Arduino IDE can't find the SPI library, so you need to add them by adding the following line at the start of sketch:<br>#include &lt;SPI.h&gt;<br></p>
<p>I'm no expert but had this same problem on another project. All of the files (including the additional libraries) need to be in the right folder. The Arduino IDE seems to be a bit picky about this (on Linux at least).</p>
<p>can i have the code for storing Accelerometer ADXL335 data into SD card with time staps (say every 5 min interbal)</p><p>and how can name the stored file as a YYYYMMDDHHSS format </p>
<p>could not get code to compile</p>
<p>I made an equivalent tool in python that print real time data from ADXL345 accelerometer.</p>
<p>like</p>
<p>I made an equivalent tool in python that print real time data from ADXL345 accelerometer.</p><p><a href="https://github.com/mba7/SerialPort-RealTime-Data-Plotter" rel="nofollow">https://github.com/mba7/SerialPort-RealTime-Data-P...</a></p><p>may be it will be helpful for someone</p>
<p>I made an equivalent tool in python that print real time data from ADXL345 accelerometer.</p><p><a href="https://github.com/mba7/SerialPort-RealTime-Data-Plotter" rel="nofollow">https://github.com/mba7/SerialPort-RealTime-Data-P...</a></p><p>may be it will be helpful for someone</p>
<p>I need create a file GPS.log in SD card before do this work ? Or the file GPS.log will be created by code?</p>
<p>You can read the accelerometer more often than the GPS, so there should be no need to interpolate the accelerometer reading.</p>
<p>great</p>
<p>good</p>
<p>super</p>
<p>super</p>
<p>In this tutorial I find what I need to start interfacing the gps sensor with an Arduino. And because I want to help many more hobbyists to start building robots, I share this tutorial on my post http://www.intorobotics.com/gps-sensors-tutorials-resources/. Thank you!</p>
<p>Hello techbitar - I'm thinking of building an adaption of your project but am considering changing the SD card adapter to the microSD card shield from Sparkfun: https://www.sparkfun.com/products/9802</p>
True but not all sensors (temperature, humid, etc.) possibly used by others will be as fast as the accelerometer I am using in my project.
Hello techbitar, <br> <br>I just ordered all the hardware for this project, and I should be beginning it some time next week. I just had a question. Obviously all of these components would need to be powered up whilst in car data collecting while driving, so would one of these do the job: http://www.radioshack.com/product/index.jsp?productId=3802146 . Also, I got this for the 'in-home' programming and construction of the actual project: https://www.sparkfun.com/products/9442? ..... Do you think that'll be good as well? I would assume so.
The Arduino Uno specs are as follows: <br>Input Voltage (recommended) 7-12V <br>Input Voltage (limits) 6-20V <br> <br>As for current, I have tested my Bump-O-Meter with a USB connection which provides 5V and up to 0.5A. I did not run into any issues but I did not perform any exhaustive tests. <br> <br>While driving around in my car, I powered the Arduino Uno via the DC plug using a battery brick made of 6 X 1.2V (NiMh) = 7.2V @ 2A. That voltage is close to the minimum recommended voltage. It worked but of course the moment the battery dropped to 6V and below I am sure I would have ran into all sorts of issues. <br> <br>If I go production with this, I would use a 3S LiPo for a cool 11.1 Volts and 1.5A or 2A current for safety margin in case I need to add additional power hungry gizmos to the bump-o-meter. <br> <br>Check you choices of power supplies against the min/max voltage specs and add a comfortable margin to the current with an eye on your future plans. then decide what fits your requirements best.
I was riding my bike recently and remarked to another rider that it would be useful to quantify just how bad our roads are. My thinking is to first breadboard components as you have laid them out, and then substitute an Arduino Micro and micro SD card reader for compactness. While I like LiOn batteries, a 3S 11.1V 2 amp battery is fairly large (and heavy and needs a requisite charger). Do you have an idea for a compact battery with a 3-hour runtime?
Hello techbitar - I'm thinking of building an adaption of your project but am considering changing the SD card adapter to the microSD card shield from Sparkfun: https://www.sparkfun.com/products/9802 <br> <br>Do you see any obvious problems with this change, and would it break the code you've written? Cheers
I have not tried it but if it Sparkfun's SD adapter does not work out of the box you may need to tweak my circuit and/or code.
You can read the accelerometer more often than the GPS, so there should be no need to interpolate the accelerometer reading.
True but not all sensors (temperature, humid, etc.) possibly used by others will be as fast as the accelerometer I am using in my project.
awesome and super useful..
SUPER Tutorial and explanation!! Thanks.
Does anyone know why I am getting this: http://imgur.com/AeUVEwj , and not what the instructable shows? Any help would be much appreciated, especially seeing how far I've gotten.
&quot;We have to level-shift the Arduino's 5V signals to 3.3V before we can connect them to the SD card.&quot; &nbsp;<strong>Why?</strong><br> <br> Can I <strong>power the SD</strong> card via a <strong>3.3V output of the SN74AHC125?</strong> I just have the SD socket and have got all components except the LM1117 to build a circuit like the LC STUDIO.
Wikipedia says &quot;The power consumption of microSD cards varies by manufacturer, but is in the range of 66&ndash;330 mW (20&ndash;100 mA)&quot;
&quot;Just because this SD card socket can be powered with 5V or 3.3V, we still can't connect 5V Arduino pins to the SD card socket's pins&quot;. <br> <br>Also ask why? If the arduino's 5V socket can hold 500mA shouldn't I be able to connect this socket to the shield's 5V socket directly?
Hw to make this data logger into real time to upload the SD card contents after formatting into any GPS visulaizing website automatically !! <br>Can any one help me !!!
everything is well
does the GPS data logger always have to be connected to the internet or all it needs is a power source? i need an urgent reply coz im about to start making it please help me hehe tnx in advanced
there's no need to connect the gadget to the internet.
oh i get it thank you very much<br>
You can read the accelerometer more often than the GPS, so there should be no need to interpolate the accelerometer reading. Either you would interpolate the position reading (write 10 accelerometer readings for every GPS reading and do the interpolation in post processing,) or characterize the 10 accelerometer readings (likely take RMS or peak) and record once per GPS reading.
actually this is my 2nd time to use the arduino and im a student, most of the materials that you use is available in the Philippines but they differ on the look and sizes but the function remain the same <br> <br>SN74AHCT1G32DCKR 2 input OR Gate tsop-5 &lt;--- is this IC are allowed to use? i find it hard to find the IC SN74AHC125 as level shifter from 5V to 3.3V <br>or do you have any alternatives?
Google it and see if others have used it and what are their impressions. then check the datasheet and see if the specs meet your requirements
ok tnx for the reply
awesome and super useful....keep going :)
this is a LOT better than your previous version, i am planning to build it, i am from the Philippines, is it okay if i present it to my school? anyway i hope you can help me out if i encounter trouble in making this prototype :D <br>GOD bless thank you :)
Hi tolstoyan, by all means you can do whatever you want with the design. But if this is your first Arduino project, may I suggest that you start with something more basic (so many nice Arduino projects on instructbales.com) before you start with the geo data logger. I have provided point to point wiring diagrams both visually and textually so you can implement this with little need for intervention from anyone. if you use the same parts and wire them according to my instructions, the accompanying software will also work as-is. I would love to see some videos of your final project. Best of luck and keep me posted please.
Hello, this looks pretty awesome. I am learning to fly a Powered Paraglider and would live to have some type of instrumentation that gives similar results to what you have here. Just a simple GPS or Variometer seems like minimal information. Thanks for the post ill keep up with you to see how things progress. If you ever need an aviation instrument tester let me know!
Thanks, Scubamatt. I am not sure I will be flying things with my gadget any time soon :) but I think the nice folks at DIYDRONES.com have great guides and tools for what you have in mind. Check this out: http://diydrones.com/profiles/blogs/a-newbies-guide-to-uavs <br>
Air quality depends a lot on time of day. This seems like a good way to measure that variation on a local level over many, many trips through the same area at different times of day.
good point.
I would like to use something like this to log mileage driven in my informal two-household car-sharing program, for the purpose of sharing expenses. We determined that we don't need to know where the car has been, but would like to know just the amount of time it is driven + time it is away from the garage, for each user. So I think it's three sensor inputs needed to log three pieces of info: <br>1. am i away from the garage? <br>2. if so, am i moving? <br>3. if moving, who is driving me? <br>And the rest can be worked out offline. Any ideas?
my first impression is that you will have write a program to analyze the data generated by the geo logger. we already capture location, speed, and date/time. Since know your garage's latitude/longitude you can analyze logged data ranges that start and end with your garage's location.this signifies a completed round trip. then you calculate difference of time/date when you left the garage vs when you returned. that's the duration of your journey. then you scan the previous data for in between locations to look for periods where location does not change much which should mean the car is stationary. subtract the time for the stationary locations from total round trip time to determine for how much time the car was moving. this is very preliminary approach but you get my drift. the solution will be mostly programmatic.

About This Instructable

166,166views

521favorites

License:

Bio: Did I unplug the solder iron?
More by techbitar:IR Remote Control Station for Android - TURN THE TV DOWN SensoDuino: Turn Your Android Phone into a Wireless Sensors Hub for Arduino Modify The HC-05 Bluetooth Module Defaults Using AT Commands 
Add instructable to: