Control Chromecast With Gestures Using APDS9960 and NodeMCU

Ages back, when Google's Chromecast was without the Android TV, I came up an idea to control the Chromecast with a Raspberry Pi. This was for those who wanted a tactile experience and who did not mind tossing their device to their children. But there was an issue with that. Raspberry Pi had an OS running and had to be booted which naturally took time and can't be shutdown abruptly which would corrupt the OS installation.

So here is Version 2 of the project and its a Covid-19 special. Non-contact based remote control for your Chromecast. That is, you can control your Chromecast with hand gestures. No more shouting "Hey, Goooogle" or getting distracted with your phone forgetting to pause your Chromecast. This overcomes the disadvantages of the Pi based remote control that I just mentioned above. This will serve as a good activity time project for the tiny tots in your home and they will surely have fun interacting with the device.

Supplies

Hardware:

Wemos D1 Mini (any other NodeMCU will do as well).
APDS9960 Sensor.
Jumper wires.
3mm LED (any color).
220 Ohm 1/4 W resistor.

Tools (if you are going to enclose the hardware in a case):

Box cutter.
Electrical tape.
A plastic box.
Electric hand drill.
A piece of MDF/Wood.
General purpose PCB (4cm x 2.5cm).
M2.5 screws.

Step 1: Hurdle No1 - Wemos and APDS9960 Sensor Integration

There were hardly any working references available for using Wemos D1 Mini/NodeMCU with ADPS9960 sensor in the internet. So I knew that I was venturing into the unknown territory.

So to start with, I first connected the APDS9960 sensor to the Wemos as per the attached wiring diagram to check if the Wemos detected the sensor at all. I used the I2Cdetect library and check for the APDS9960 Sensor's address. I could see that the sensor was at 0x39 I2C address. That was a positive start.
Next step was to check the if the Wemos read the gestures. So I downloaded the Sparkfun's APDS9960 sensor library and use the built-in example. It was a bummer. The code would not even compile. Going through various forums, the issue was likely IRAM related. None of the suggested solutions worked. That was when I came across a fork of the Sparkfun's official library. That was tailored to work with Wemos and when I tested it out it did work, but had to do some minor changes. You can download the copy of the modified library from here.
I gave the Wemos a test run with the modified library and it did work. I was able see the gestures getting detected in the serial monitor.

That was one huge hurdle that was crossed.

Step 2: Preparing the Project Enclosure - the Cutting

I did not want to add new and more plastic waste in this world by 3D printing something. Instead I decided to recycle a sweatbox that I had in plenty with me. This was the same box that I used in my previous Crypto Ticker project.

I decided to partially expose the Wemos's USB connector and RESET button outside the enclosure. This way, I need not take out the Wemos out of the enclosure for updating the code or re-purposing the setup for some other display based project. This will also allow me to RESET the Wemos when desired.

First I inspected the plastic enclosure for any defects. While inspecting the enclosure, I saw that the plastic was partially translucent.
The partial translucency in fact helped my cause. I placed the Wemos inside in the desired position and I could see it's outline from outside.
With the Wemos's outline as reference, I used a piece of electrical tape to mark the dimensions of the cutout.
In the fourth image, you can actually see the tape's border matching the Wemos's height.
Using a box cutter, I cutout the plastic to make room for the Wemos.
Similarly, made a cutout on the lid for the gesture sensor.
I hot-glued the Wemos in its place and stuck some paper tape on the exposed Wemos to temporarily protect it from dust and other elements
On the back of the lid, I stuck a piece of small wood and held it in place with some electrical tape. On a hindsight, I could have used a duct tape here, my bad.

Step 3: Preparing the Project Enclosure - the Drilling

This part involves the use of power tools. Always take the necessary precautions and wear the required safety gear while operating the electric power tools. Also, pay full attention and do not get distracted.

Drilled the following holes.

2 holes on the lid for two M2.5 Screws to hold the backplate in place.
1 hole of approximately 5mm diameter for the sensor.
1 hole of approximately 3mm diameter for the 3mm indicator LED.

After drilling the front two holes, secured the backplate with two M2.5 screws and then removed the temporary electrical tape that was holding it in place.

Step 4: Preparing the Project Enclosure - the Paint Job

First, I buffed up the plastic surface with some sand paper to make the surface rough. This was to make sure that the enclosure would hold the paint well.

Then, I masked the parts of Wemos that was protruding with some paper tape.

After roughing up the enclosure surfaces, gave it a good dose of black spay paint and left it to dry for 48 hours. The paint could have dried earlier, but did not want to check just in case.

Step 5: Hurdle No 2 - My Mistake

After putting altogether as per the attached schematic, tried to see if the device could pickup any gestures with sample code previously used. The sensor was working, as you could clearly see the infrared light from the sensor. But there was nothing on the Serial Monitor.

Tried to see if I could figure out anything from the datasheet and it was a dead end. At this point, I had two probable causes:

The indicator light was interfering with the detector, or
The hole was not large enough for the detection cone of the sensor

Just to check on both, took out the LED and the sensor out of the enclosure and while holding the LED just above the sensor, tried some gestures and it was working. So the culprit was the hole.

So i decided to fully expose the sensor just to not run into similar issues again.

Step 6: Putting It Together the Second Time and Properly

I measured the seize of the wooden backplate that was previously used and replaced it with a double sided PCB. Drilled the matching holes on the PCB and held them in place securely using two M2.5 screws. The LED and the connections to the sensor were bonded to the backplate with some hot glue.

The connections were made using just the jumper wires. Not clean, but I did not mind as they going to be unseen inside the enclosure. Instead if you want something neat, I have prepared a gesture sensor shield for the Wemos. You can find the Gerber files here. For PCB fabrication, you can use JLCPCB's service. With their online Gerber viewer, you can verify the PCB design before ordering and they have different PCB colors to choose from. Use this link https://jlcpcb.com/IAT for great discounts and minimal PCB fabrication costs. If you want to create your own design, you can modify the board design with the Fritzing file I have provided here.

I modified the sample code from the ArduCastControl library and combined it with the modified gesture sensor code. You can find the modified libraries here and the sample code here.

There something else you need to mind. The code is dependent on other libraries as well.

ArduinoJson version=6.18.5
Arduino_JSON version=0.1.0
SnappyProto version=0.1.2

For your own use, make sure to declare the IP address of your Chromecast and also your WiFi SSID and WiFi password in the code/script. The locations for these declarations have marked clearly.

After compiling and verification, I decided to power the setup with a power bank. I could place this easily on the sofa armrest and control the Chromecast.

Step 7: Purpose of the Indicator LED

I have assigned specific behavioral patterns to the indicator LED.

If the LED blinks every 1 second, it means that the WiFi connection has got disconnected.
If the LED blinks in long intervals, it means that it is unable to shake hands with the Chromecast.
If the LED pulses constantly, it means that its constantly communicating with the Chromecast.
If the LED momentarily blinks while pulsing, it means that it has picked up a gesture.

Step 8: Demo: and the Controls

The gesture sensor can inherently detect 4 gestures:

Right swipe.
Left swipe.
Top swipe and
Bottom swipe.

Thanks to Sparkfun, we have two additional gestures:

Far swipe and
Near swipe.

For Far swipe, you need to start close to the sensor and take your hand away and for Near swipe, its the opposite, you need to start Far away and bring your hand close to the sensor.

I have integrated these 6 gestures with 6 different Chromecast controls.