INTERNET CONNECTED DEVICES AND TIME SYNCHRONIZATION

lot of devices that are connected to the internet uses NTP protocol to synchronize internal clocks in a periodic way. Personal computers (PC) are the most common case. Those devices usually have an internal clock (RTC), but probably lost a second or more every day. After some days the offset could be from some minutes to hours.

With the near omnipresence of the internet, a lot of connected devices, to lower manufacturing costs, don't include an RTC, so every time a timestamp is needed a call to an NTP server is made.

What if at some point in time, internet connection is lost and hundreds of devices need a timestamp to do some stuff?

How to time synchronize hundreds of wireless connected devices, but in an scenario with no internet connection, i. e. in a faraway forest where there is no cell phone signal?

How to time synchronize industrial devices in factories, where for security and/or paranoia reasons, devices are interconnected but physically unplugged from the internet?

For more info, please visit: Absolutelyautomation.com

Step 1: REQUIRED COMPONENTS

Here is the list for the required components for this project with the absolute lowest prices! If you find something cheaper somewhere, please msg me!

ESP8266-01 ► ( http://goo.gl/o4fAfJ )

GPS Module ► ( http://goo.gl/M8Y6pR )

USB to TTL interface ► ( http://goo.gl/da82Uc )

Step 2: WIRELESS NTP SERVER WITHOUT INTERNET CONNECTION

WIRELESS NTP SERVER WITHOUT INTERNET CONNECTION

If an internetless reliable time source is needed, there are multiple choices: Radio time signals, atomic clock, gps and others.

Time signal receivers are a low cost option, but their effectiveness depends on how close are to the emitting station. Also, there are different frequencies depending on the country or geographic zone, so is not a very universal alternative for every site.

Atomic clocks are the best option if a very high degree of precision is needed. Some time ago atomic clocks were big, not very portable and very power hungry. However, with the advances in miniaturization now is possible to get a chip sized atomic clock that doesn't need more than a few milliwatts to work. If cost is not a determinant factor (prices around thousand dollars), this is the option to choose!.

Global Positioning System GPS or equivalent systems in other nations ( GLONASS, GALILEO, BEIDOU) are an intermediate option that brings more flexibility than radio time signals (well, some clear sky view is needed) and with a price cost much less than atomic clock.

If a very high degree of robustness is needed, for example to make a homemade nuclear reactor or if big amount of devices doing millions of requests for second will be connected, an already made solution is preferred. But if something simpler is wanted, for experimenting purposes, and offsets of one second could be tolerated, is possible to build one spending around $10 USD using anESP8266-01 and a GPS module, also a handheld GPS with serial NMEA output could be used.

Step 3: DATE AND TIME PROTOCOLS: DAYTIME, TIME AND NTP

DATE AND TIME PROTOCOLS: DAYTIME, TIME AND NTP

Since the very first practical computers started to work, the idea of having them time synchronized was revolving around. There are many protocols for this, some of them are: daytime, time and NTP.

Daytime protocol:

This protocol is described in RFC867. It's one of the oldest and practically not used now. Some time servers still provide it for educational purposes and as an alternative for very very old hardware/software applications that still uses it. This protocol works over port 13 and time/date information is sent in clear text. There is not a specific recommendation for the format used, the only exigence is that the information could be read by a human.

Time protocol:

This protocol is described in RFC868 and works over port 37. Time/date information is coded as seconds elapsed since 00:00:00 (midnight) of January 1 1970 in a 32 bit number.

NTP protocol:

NTP was conceived to bring time/date information, is described in RFC5905 and have a precision in the order of milliseconds. Is based on a modified Marzullo's algorithm to take in count the variable delay of the information packets. This protocol is very widespread used, information packets travel over UDP connections on port 123 to minimize the processing time.

Step 4: ESP8266 + GPS

ESP8266 + GPS

This project started as an improvement of Ray Burnette's Tardis Time that basically has the same hardware elements, but doesn't send time/date information using standard protocols, so a small listener application must be developed in every different device to be synchronized. This project takes advantage of the built-in defacto time synch apps in personal computers, raspberry pi, and could be used by the NTP client libraries for the ESP8266.

The GPS module used was an EM-506 which don't have a PPS signal. Also a handheld GPS receiver with RS232 NMEA output could be adapted for the same purpose. The ESP8266 was programmed using SDK version 1.5.2. Don't forget that some kind of USB to TTL 3V interface is necessary for programming!. The software could be described as 3 big parts:

SERIAL DATA RECEPTION

ESP8266's UART has a hardware FIFO with a max capacity of 256 characters and multiple sources of interruptions. The character received threshold and timeout interrupts were used in the program.

Character threshold interrupt was used to fire automatically a function when some amount of characters reaches the UART's FIFO. In this particular case as NMEA strings are expected from the GPS, and they are around 30-80 chars long, a threshold number between them was used. This function reads characters from the FIFO and writes them to a circular buffer.

Timeout interrupt is used when only a few characters were received (less than threshold amount) and no more characters were detected in a determinate time interval. In this particular case a timeout of 10 byte-time was programmed.

This function need to be executed quickly to return from the interrupt as fast as possible, so no processing is done here, just read-and-copy.

STRING PARSING, ACTUAL TIME/DATE AND MICROSECONDS CORRECTION

Serial ISR writes received characters in a circular buffer which max size should be twice the length of the longest NMEA string to be parsed. In this particular case the string that begins with the $GPRMC header. This particular string holds time/date information. The function that processes strings is called by the ISR once all received characters were written to the circular buffer using the messaging system provided by the SDK system_os_post.

NMEA strings have variable length, so absolute character position should be avoided to parse values. The number of delimiters in this case commas "," is constant for a specific type of string and were used to extract the time and date values. A sample string looks like this:

$GPRMC,201705.000,A,0000.0000,N,00000.0000,W,1.10,265.50,120816,,,A*79

Between comma "," delimiters #1 y #2 lies the string 201705.000 which means 20 hours, 17 minutes and 05,000 seconds. Between comma "," delimiters #9 y #10 lies the string 120816 which means Day 12, month 08 (August) and Year 16 (2016). Additionally the system's microseconds ring counter (not from GPS) is stored, to make adjustments and deliver a more accurate timestamp.

DELIVER DATA, SOCKETS

Three independent sockets were created to listen to any type of service: daytime, time or NTP. For daytime, when a query is received, data are read from the global variables that holds the date and time information and transformed to a human readable text string. For the time service case, date and time information should be encoded in a suitable way, the helper function SecondsSince1900 do the task.

In the case of NTP, the received packet should be temporarily stored, some information needs to be modified/updates and sent back as an answer. Due to the GPS used in the project only brings time/date information each second, and an NTP query could occur in the middle of that interval, to mitigate a bit this problem, the value of the system microsecond ring counter is read using system_get_time() every time a new GPS frame is received. When an NTP query is received, that counter is read again, and with these two values the time elapsed between the last GPS update and NTP query reception could be estimated. This time is added to the time stored in global variables (the one that is updated every second) and send in the answering NTP packet.

Step 5: TESTS AND CONCLUSIONS

TESTS AND CONCLUSIONS

- Standard apps were used (In this particular
case Linux apps) to test time services programmed in the ESP8266, like netca, rdate, ntpdate .

- Due to its small size, low power and wireless connectivity, could be installed in places with better clear view of the sky: near windows, rooftops, etc.

- With a GPS with a PPS, more precision could be achieved.

- Could be solar powered ( with batteries for the night ) to get a totally wireless continuous operating system.

- Some signal level converted have to be used depending of the type of the GPS like 5V TTL to 3V TTL or RS232 to 3V TTL.

For more info, please visit: Absolutelyautomation.com