Introduction: [15min] Weasley Clock / Who Is Home Indicator - Based on TR-064 (beta)

UPDATE: See step 3.

Long story (TL;DR below):
Some time ago I wrote a small bash-script on my computer which would scan the network for registered devices and compare their hostnames against a list with associated names. Every time a device would log-in or out the network I'd get a notification. Like this I'd have an idea who is home (not only is it sometime nice to know who is home to ask them if they want to join a meal or such, but it can also potentially help you avoid awkward situations). Problem is, that this method is not very reliable. Unfortunately some devices don't reply pings (e.g. smartphones) and seem to go into some power-save mode are thus not reliably detectable with commands like nmap . I tried to overcome this by requiring at least 10 negatives before finally announcing that the device left the network. All in all it worked, but slowly, unreliably and not very nicely. Also I wanted to be able to check who is home, without opening my computer.

So I had these ESP8266s lying around - and wanted to play with them. But I didn't wan't to build the 10.120.124.812th ESP-based weather station or such (why would you want to monitor the weather with your crappy ±2°C thermometer when you can just visit some website anyway?). So the plan was made. Unfortunately it turns out that the ESP seems to not be equipped to properly ping (which was bad to begin with, remember?) let alone do a full-blown network scan. I considered writing some library for that - but since my C++ is very limited, this is my first ESP-project and I'm not a genius with network stuff either I scrapped that idea.

After doing some research, I found out, that my router (brand Fritz!Box) actually has an API! YAY!!!! The protocol is called TR-064. Such an inspiring name, right?

Doing some research into the matter, I found a (rather crappy) documentation online. It took me some time to work it out. I might post a more hands-on instructable in the next days (and how to communicate with/test this API using a browser addon).

Once I understood the API, I started using it from my ESP. Long story short: I got a duck-tape-and-wd40 solution running and doing what I wanted (querying the API for connected devices, compare them to a list of MAC-addresses and switching LEDs accordingly).

But I wanted a more robust and flexible solution and this API has so much more to offer (you can change settings (turn on/off the wifi, change password, open/close the guest wifi, change signal power), restart the router, query the connection speed, ...) - so I decided to create a universal TR-064 library!

TL;DR;
Device that indicates via LEDs who is currently at home. Works great, has no great housing yet (hey - noticed the beta in the title?). Uses SOAP-protocol to communicate with router. Wrote Arduino library to make that happen.

Required knowledge
• Basic Arduino/ESP knowledge
• If you assembled a Blink example with your ESP, you'll be fine!
• If not, you can follow the first two steps of this instructable
• Soldering might come in handy, if you want to leave the breadboard stage
• Some 3D designing an/or woodworking might come in handy for a housing


BOM
• ESP8266 + Development-Board or other TTL connector + cable (around 3€)
• Power Source (~2€)
• A few LEDs and resistors (~1€)
• (Casing)


=> Around 4-6€ for a minimal design. That means for me, that I can easily hang a few around the house.
If you are lazy, you can also use a development-board with included LED's (see step 4 for more detail).

Github project link [https://github.com/Aypac/Arduino-TR-064-SOAP-Library] for quick reference.

Step 1: Set Up Your Router (optional, But Advised)

The protocol was designed as a general protocol for routers, but I only know that (most) Fritz!Box routers (very common at least in Europe) use it. Don't know about other brands. So I'll assume, that you are using a Fritz!Box. If you can make this work (or fail to do so) on any other router, I'd be happy to add it to this instructable (maybe even start a short compatibility list?).

This step is not necessary, you could also just use your admin account, but that is not advisable for obvious reasons (might be intercepted or extracted from your ESP, you might accidentally change stuff you don't want to, ...) - so I advise you to do it.

Anyway, here we go:

  1. Log in to your Fritz!Box by typing fritz.box in the url-bar of your favourite browser.
  2. Find the page for creating a new FRITZ!Box user (you need to have advanced it should be under System, if you can't find it, try updating you device).
  3. Create a new account as you see in the picture (using a different user name/password!).
  4. Logout.

Step 2: Code

  1. You can get the library from Github.

  2. In the examples you will find a file called home-indicator.ino, which is the code for this project. Place the examples folder in your arduino IDE project home folder and the other folder into the libraries folder in the same project home folder.

  3. (Re)Start you IDE.

  4. Open home-indicator.ino in the IDE and enter your settings. It should be quite obvious what goes where. If you have any questions, let me know!

  5. Now you can compile it and transfer it to your ESP!

Step 3: Hardware (beta)

Yeah... This is the step, where I still need to work some more. But I guess there is enough material in the web to figure something out :)

The setup as in the pictures is just a diode with a ~100Ohm resistor wired to the GPIO{5, 4, 0, 2}, which are the D1, D2, D3, D4 (in that order) pins of my MCU ESP8266 board and the little orange cable taking the cathodes to GND (marked G on my board).

If you are lazy, you can do what you see in the second image. Just use a development-board with included LED's (like the "ESP-202", see pictures - around 3.50-5.00€ including shipping). You can then just hook it to USB power or use the included battery pack, draw a interface on a piece of paper and stick it to the wall. Done.

For this specific board the ports are:

int userPins[numUser] = {5,4,2,14,16};

from top to bottom.

Step 4: Outlook/References

This is what I still plan:

1. Making a nice housing (obviously)
2. Adding a nice GUI for managing users/LEDs/MACs/known devices
3. Improving the library (see Github for that)
4. Fix: sometimes the LEDs react sluggishly (it takes a little until they turn off, after a device left) for no obvious reason
5. Hook the thing up in the hallway and maybe a few more around the house


Would love to get some input from you guys! And as always, it makes me really happy to see others build shit you thought up, so please take the minute to comment and post pictures! :):)

____________________________________________________________

Since I'm somewhat a late-comer for the IOT-Contest, I'd doubly appreciate if you'd vote for me (up there ↗ somewhere I guess)!
____________________________________________________________

Some references

I'll link some stuff here, that might help you:

TR-064 reference
Some guy messing around with the shell and TR-064 (good to get a grasp of the API)
• Harry Potter Wikipedia: Weasley Clock entry
Some guys building a beautiful, but complicated Weasly Clock*

____________________________________________________________

*It's based on an app for your phone which is taking your location. If you are in certain pre-defined areas, the according light on the clock will turn on. For me that was too complicated and had to many places where it could break/fail:

• Apps need to be kept up to date
• Everyone has to use these apps
• And be okay with the impact on privacy
• The app only works with internet connection
• You have to use an intermediate server for interfacing the communication
(unless you want to set-up you esp as server and open you home network for outside traffic - yay, yet another unsafe IOT device directly connected to the internet!)
• The intermediate service might go offline one day/change its API,
• ...You need more software. More software means more places to do an error ;)

But, they did an amazing job and if you want to do something like that, head over to their site, it is really great!

Comments

author
OleU1 made it! (author)2017-07-01

Hello,

now the DECT 200 woks fine with your library.

The space in the NewAIM is needed.

The documentation from AVM (x_homeauto.pdf) is a mess. "X_AVM_DE_Homeauto1" instead of "X_AVM-DE_Homeauto1", "AIM" instead of "NewAIM"... So anyone who ist interested in their own aplications: Allways check the xml´s, dont belive AVM docs.

I am very happy about the library; now I can build my own remote controlls for the DECT 200. This is very very usefull: If it is dark, I do not wake up. So my sleepingroom has a DECT 200 for switching the light on by timer the same time, the alarmclock rings. Switching off must be done manually, when leaving the bedroom. And of course, switching on before going to bed and off again before sleeping. Now I can put a remote switch at the door off my bedroom and a switch beside my bed. Perfect!

author
Razanur made it! (author)Razanur2017-07-02

Hi Ole,
thank you! Yeah, 95% of the work for this project went into the library -- it seemed too important an application to not have an arduino library for it. I love your applications (I don't use the DECT at my place, so I've not even thought about it yet :) But this doorbell idea is just genius, I love it. You could even replace your 'classical doorbell' altogether I guess. It would be super nice if you could add your examples (code + schematics or short explanation) to the examples folder in the GIT { https://github.com/Aypac/Arduino-TR-064-SOAP-Libr... }. If you run into any problems or have any suggestions with the library, let me know (issues on github).
(And yeah, I was also kinda disappointed by the documentation from AVM. Just barely good enough to actually implement it. I almost exclusively worked with the XML, which sometimes makes it a little hard to figure out what the different functions do. Some trial and error is apparently always required).

author
OleU1 made it! (author)OleU12017-07-02

Hi Razanur,

my examples are at http://www.ip-phone-forum.de/showthread.php?t=295676
(in German, but as I read in your profile, you are from Hamburg, so that shouldnt be a problem)
Fell free to use them as examples in your libraray.

The doorbell project is only the first step to a bigger project: an FTZ123D12 interface to connect a speakerphone at the door. The FTZ123D12 consists of an powerless Audio (the normal a/b without AC ring-power or DC power for the telephone), the doorbell-input (or some more doorbell-inputs), a power relais-out for turning the power-amplifier in the doorspeaker on, and a relais for dooropener.
The physical Interface is very simple: cut out the line-interface of an old analoge modem. There you get the powerless a/b, an ring-indicator (optocoupler) and a hook-input (electronic relais/optocoupler with relais).
So you can connect the FTZ123D12 (for example: Auerswald Dialog 201) to the lineinterface for audio. The ESP-12F controls all relais necassary and also the hook after ringing. The only problem is call-end-detection. Therefore you have to use the callmonitor, which meens all doorcalls have to go externaly (fritzbox doesn´t watch internal calls). So you have to use external numbers to call yourself or a loopback-cable (second fon-Port of the box connected to analoge-in of the DSL/analog/ISDN connector of the box).
Opening the door would also be realised by the callmonitor: R000123 - as mankind is seen as to primitive to get access to interstellar telephone networks, "000" always fails. But trying a call to E.T. it is detected by callmonitor, so you can use it for trigger actions (as I discovered 10 years ago with callmonitor-plugin on ds-mod, predecessor of freetz). So for about 8 bucks and some electronic trash you get an FTZ123D12-Fritzbox-Interface. If you have an Siedle 4+n (like TM511, very common in germany), the adapter for an FTZ is about 5-20€ (its only the "Gabelschaltung" with some level-adjustments), so this will work also. The old Telekom/Telegärtner CE-Zweidraht Doorline-Protocoll ist somehow more komplex. It seems to be a "multiplexed" FTZ123D12, where ring opens door and hook (with Impulse-Call for 1-4) meens Door-Ring. I got an old doorline for 6€, so maybe I try an interface at last step. Normaly I would say CE-2Draht is a very simple interface, which is the reason it is integrated in so many telekom-switchboards. Even the Gigaset SX353 has a build-in interface. Its only an other interpretation of normal a/b-signals, so AVM could easily implement it via Firmware. It is so old, no patents protect it. But building is for yourself meens to split a/b and multiplex it again, which is more complicated.

Schematics will need a while, as I just ordered some parts from china (optocouplers 20 pcs 1€, 5V to 3,3V voltage converters 5 pcs 1€, ESP-12F for 1,89€/ pcs, all including pakage).
As soon as photos and schematics are available, I will contact you. All I produce is under Creative Common Licence, so fell free to use it in any way.

author
Razanur made it! (author)Razanur2017-07-06

Hey. Thank you for your active contribution!
I uploaded the code into the example folder.

author
OleU1 made it! (author)2017-07-01

Hello,

it is a very nice and usefull project. Most usefull is the library.

For example you can trigger the "Wählhilfe" to let a target number ring:

String params[][2] = {{"NewX_AVM-DE_PhoneNumber", "**799"}};
String req[][2] = {{}};
connection.action("urn:dslforum-org:service:X_VoIP:1", "X_AVM-DE_DialNumber", params, 1, req, 0);

In the Phonebook of the Fritz!Box as ShortDial **799 you can define any target. "**9" as target will let all telephones ring, so if you connect the doorbell via optocoupler to the ESP, all your telephones ring. Very nice, if you dont need a voice connection to the door, but only the information, that sombody is at the door.

While the Wählhilfe functions very nice, It do not on switching an DECT200. Due to documentation i tried following:

String params[][2] = {{"NewAIN", "123456789"}, {"NewSwitchState", "TOGGLE"}};
connection.action("urn:dslforum-org:service:X_AVM-DE_Homeauto:1", "SetSwitch", params, 2);

Where 123456789 is the AIN seen on Webinterface of the FritzBox. The xml says "NewAIM", the documentaion from AVM said "AIM" as parameter, so I tried both. Neither one worked.

It would be so nice to have a simple remote switch für the DECT200.

Have you any idea?

author
Shahid Khattak made it! (author)2016-12-07

Thanks for sharing