Introduction: Use the Force... or Your Brainwaves? (multifuctional Thought Controlled System)

About: Hi, I'm Tamas. I like to build all kinds of electronic gadgets, hope you'll like my ideas. I'm not so active on the site these days, so sorry for the late replies. (Oh, and sorry for my broken English in these…

As a kid I always wanted to have a special ability, like moving objects with thoughts or flying. I tried to move objects with “the power of my mind” of course with no success. Well, yes, I was and I am a huge Star Wars fan, but I also like electronics and programming, so why not combine these two things to build an incredible project?

You can find all parts, codes and libraries with links at the Hardware and Software section.

The main concept is that using different brainwaves the user will be able to control a robot, the cursor on his PC or turn on/off lights in his home, while a microcontroller (Particle Photon) creates an online analyzing about the user’s brainwaves. This is a perfect method to detect diseases or stress and sleepy statement. This also can be used to help people live better their life, learn to control their emotions and how to be always happy. The human brain needs balance, to be healthly, we should sleep, think, move... Using the Particle Photon and the webserver created by the founders of the Particle we can help people to find their natural brain-balance. A childhood dream guided me to this project, and I'm very happy that I made it.

So yes, with this device you can actually use the Force...But if we put Star Wars and the Force away for a bit, this device is not only for lazy people or for fans, it’s made to give back an ability for people who are fighting each day with their disability. The robot that I designed works the same way as a wheelchair, the home automation system would help for the user to control lights or TV-s even if he/she can’t move. The PC controller mode can be useful even for those who is perfectly healthly. I think all of us wants to play computer games or just surf on the internet using only thoughts.

Watch this short introduiction video about the story of the project:

Step 1: The Science Behind the Project

Electroencephalography (EEG) is an electrophysiological monitoring method to record electrical activity of the brain. It is typically noninvasive, with the electrodesplaced along the scalp, although invasive electrodes are sometimes used in specific applications. EEG measures voltage fluctuations resulting from ionic current within the neurons of the brain. Diagnostic applications generally focus on the spectral content of EEG, that is, the type of neural oscillations (popularly called "brain waves") that can be observed in EEG signals. (thank you Wikipedia) But we're going to use two very accurate values.

  • Alpha is the frequency range from 7 Hz to 14 Hz. It emerges with closing of the eyes and with relaxation, and attenuates with eye opening or mental exertion.
  • Beta is the frequency range from 15 Hz to about 30 Hz. Low amplitude beta with multiple and varying frequencies is often associated with active, busy or anxious thinking and active concentration.

Step 2: How It Works?

I used four microcontrollers to bring this project to life: an Arduino Mega, an Arduino Leonardo, an UNO and a Particle Photon. The Mega is the brain of the project that receives signals from the headset, analyzes then forwards commands to the other devices. Transmits every data forward to the Photon that creates the webserver. The Leonardo controls the mouse on a PC and the Uno is used to receive IR (infrared signals) and controls the robot. As I said the device is able to control three different things (or more if you want to program anything else). Now I’ll call these three different things channels. The device switches between these channels if you close your eyes:

  • Close your eyes for 1-2 seconds: switches to home automation featurette
  • Close your eyes for 2-4 seconds: switches to robot controller mode
  • Close your eyes for 1-6 seconds: switches to mouse controller mode
  • I used relays to make the home automation featurette you can connect anything to them: TV, light bulbs, wall outlets anything you want

Step 3: The Wireless Headset

I love hacking toys so I bought a brainwave sensing toy called Necomimi, that is able to move its ears depending on your brainwaves (attention, meditation). This is the most useless toy that I’ve ever saw, but there’s inside a small chip that reads brainwaves, filters noise and gives a very good signal. Works with UART (Serial) interface, so with some hacking we can use Arduinos to read brainwaves. The role of this headset is to transmit brainwaves wirelessly to the central server. Nobody wants cables on his head so I created this comfortable and user friendly headset.

Go down to see how to take apart and make a wireless Bluetooth headset out of it.

Step 4: Controlling the Home Automation System

If you are switched to home automation mode increase your attention levelupper than 70 to turn on first relay, increase meditation level upper than 70 to turn on second relay, and increase both of them upper than 70 to turn on third relay. It's a bit tricky but isn't imposible after some mind-training. If is already turned on, use the same command to turn it off (so if is turned on reach again 70 to turn it off). When the system is done you can connect any high-voltage device (light, TV, PC, cooler fan, anything you want) to the relay module.

Step 5: Controlling the Mouse

The cursor is controlled with emotions. If attention level is under 25 moves left, if is between 25 and 50 moves right, if is between 50 and 75 moves up and if isbetween 75 and 100 moves down. You can click by increasing meditation level. As I said before the attention level can be easily controlled with emotions (for me).

Step 6: Controlling the Robot

The robot has only three different statements: stop, turn left, and move forward. Why only two movement options? Because these two directions are enough to move the robot anywhere you want. Use your meditation level to spin with the robot, and when you reached the direction where you want to move stop the robot then move forward with the attention level. It's a better method to control the robot and even beginners, who can't control their attention/meditation values so good, can play with this robot.

Step 7: Gaming

All of us likes video games. And yes, with the cursor controller you actually can play your favorite PC games with your thoughts. But wait, the headset is also compatible with any of the Neurosky games. Clcik here to view them. I tried out a lot of them and it was incredible. I used just the power of my mind and played car racing games, advanture gamesit was awesome!!

Step 8: My Plan

This is an older picture about my plan, but succeeded so I’m very happy with the results. The headset transmits BT (Bluetooth) signals to the Arduino Mega that analyzes the incoming bytes and depending on the user’s thought controls the different features. It was very hard to find out the best ways to transmit this lot of data, but I choose Bluetooth instead of WiFi. At first time I wanted use the Particle Photon as a data transmitter, but that little guy got a better role in the making of the webserver. That was the biggest modification in the whole project. (On the picture you can see the actual plan). I used homemade Arduino modules, because I love design my own circuits. You can buy these modules online if you want or build yourself with me.

Step 9: A Half Year of Learning and Experimenting

I worked a lot on this project and yes, I started almost half year ago, but with the Photon things accelerated. It was very easy to use their dashboard. By the way there are many tutorials on the internet how to hack EEG toys, that helped me a bit, but they didn't have any extra functions. So I decided that I'll hack this Necomimi toy and using my creativity I created this device that has much more features than blinking a LED. Hope you'll enjoy!

Let’s get started!

Step 10: Gathering Tools and Parts

All of my parts are from the GearBest.com. It's an online store just like the eBay, but the parts arrived much more faster. So if you want to build you own game controller, robot controller or Force detector start with buying the parts! (click on them to view purchase links)

I strongly recommend to buy the parts now, because on the site is a 2 year anniversary celebration with extremly low prices. If you need Arduinos or anything else you can buy now for half price.

Tools Needed:

  1. soldering iron
  2. solder
  3. glue gun
  4. self-adhesive tape/double sided tape
  5. wire stripper
  6. wire cutter
  7. rotary tool

Hardware:

Software:

Step 11: Hacking the Necomimi Toy

We want to modify this EEG toy to transmit data via Bluetooth so first take apart the case. The screws are under the sticker. Remove the sticker and the back of the device and you’ll find inside a small circuit. The one that is under the main circuit is the Neurosky TGAM chip. This is connected with four header pins to the main microcontroller board so take a soldering iron and remove this circuit carefully. Now solder three wires to the GND-pin to the VCC-pin and to the T-pin. The T-pin is the transmitter pin with 57600 baud rate, this sends data packets to our microcontroller, but I connected this directly to a HC-06 slave BT module. The HC-06 is set to 9600 baud rate but don’t worry we’ll fix this problem. If you soldered the three wires to the you can build in your own rechargeable power source. I used a 500mAh Li-Ion battery, a USB charger circuit, a 5v step up circuit and two resistors (100 ohm and 200 ohm) to ensure a perfect 3.4 volt power supply for the chip and for the Bluetooth module. Follow the schematic to build the circuit that is needed in the headset. If the circuit is done configure the Bluetooth module.

Step 12: About the Chip

The Neurosky ThinkGear AM is designed to filter, amplify and send data through serial interface. More info here. You can see the schemantic here too.

Step 13: The Schematic / Electronics

Follow the instructions and the pictures below and create your own wireless EEG headset. Each comment is for one picture.

  1. Solder wires to GND, VCC and T pins.
  2. Wire soldered to pin "T".
  3. Voltage divider to reduce 5 volts to 3.3 volts for the chip.
  4. The Lithium Charger, the 5v step up, the voltage divider and the chip together.
  5. Use header pins to connect them to the Bluetooth module.
  6. The done electronics, ready to play...

Isn't so hard to build...

You'll need a 500mAh Lithium-Ion battery, a 5v voltage step-up module to create a stable voltage then a 100 Ohm and 200 Ohm resistor to reduvóce voltage to stable 3.3 Volts. I soldered female header pins to power up and transmit data to the bluetooth module.
The most important part is to connect the the "T" pin with the "RX" on the Bluetooth module. This brings our project to life.

Step 14: Configuring Bluetooth Modules

HC-06: First load up the sketch named “HC_06_Bluetooth” to an Arduino UNO then connect the Bluetooth module as the schematic shows. I found this schematic on Artronix team's website and seemed useful.

Open your Serial Monitor in the Arduino IDE and wait until the Arduino configures the BT module. Now your Bluetooth module is set to 57600 baud rate. You can try out a lot BCI (Brain Computer Interface) apps, because this hacked bluetoothNecomimi toy will be compatible with every Neurosky apps.

//HC-06-Bluetooth
void setup() { <br>
// Start the hardware serial.
Serial.begin(9600); // default HC-06 baud rate
delay(1000);
Serial.print("AT");
delay(1000);
Serial.print("AT+VERSION");
delay(1000);
Serial.print("AT+PIN"); // to remove password
delay(1000);
Serial.print("AT+BAUD7"); // Set baudrate to 576000 - eg Necomimi dafault
delay(1000);
Serial.begin(57600); //
delay(1000);
}
void loop() {
}

You can use these apps to learn to control your brainwaves and values, like attention or meditation. You can try a lot of cool games, but I recommend these apps (they are compatible with PC, iOS and Android):

http://store.neurosky.com/products/brainwave-visua...

http://store.neurosky.com/products/eeg-analyzer

http://store.neurosky.com/collections/android/prod...

HC-05: Then use the "HC_05_Bluetooth" and load up to your Arduino the same way like before. Connect "EN" pin of the HC-05 to the 3v3 of the Arduino. You should write the adress of your HC-06 module in the code. Check the adress of the BT module with an Android smartphone, like me. Replace ":" (double dottles) with "," commas in the code.

//HC-06-Bluetooth<br>void setup() {
// Start the hardware serial.
Serial.begin(9600); // default HC-05 baud rate
delay(1000);
Serial.print("AT");
delay(1000);
Serial.print("AT+VERSION");
delay(1000);
Serial.println("AT+ROLE=1"); //set the HC-05 to master mode
delay(1000);
Serial.println(" AT+LINK="copy here your adress"); //now the module should connnect automatically
delay(1000);
Serial.print("AT+UART=57600,1,0"); // Set baudrate to 576000
Serial.begin(57600); //
delay(1000);
}
void loop() {
}


And you should change the adress in the code: 20:15:09:15:17:82 ==> 2015,09,151782 This way the HC-05 module can recognise the adress. So just leave some ":", because they aren't neccesary.

Step 15: The Remote Controlled Robot

To make this inexpensive robot I used 38kHz IR technology, that is used in TV remote controllers. I bought from Gearbest a chassis for my robot. The IR receiver is salvaged from an old TV and the Arduino is also from the Gearbest.

The motor driver circuit-You’ll need these parts:

  • 2 Screw Terminals
  • L293D IC3
  • Header Pin (90 degrees)
  • 1k Resistor
  • Red LED
  • Wires
  • A PCB Board

I used some copper wires and following the schematic connected the pins of the IC to the header pins. Doesn't matter that which pin goes to which header pin, just remember that were did you connected them. The LED is connected in series with the resistor and in paralell with the 5v VCC.

Step 16: Assemble the Chassis

This step was very easy, used a screwdriver and some crafting skills I built the frame in five minutes. After this is coming the harder part, we should build a motor driver circuit. To control my motors I choose the L293D IC that can drive two motors. Look at the schematic to build circuit.

Connect the parts to the Arduino

I used jumper wires to connect the sensor and the motor driver to the Arduino.

Arduino Pin ==> Motor Driver

4 ==> 15

5 ==> 10

6 ==> 7

7 ==> 2

VIN ==> 8

5v ==> 1, 9, 16

GND ==> 4, 5, 13, 12

So look at the schematic of the L293D module, then connect its pins to the Arduino UNO as I wrote here. The 5v pin of the Arduino should be connected to the 1, 9, 16 pins to enable the IC's motor driver function. Then finally use the screwterminals to power up the motors.

Step 17: Writing Codes…

Using the IRremote library I created a code that reads 38kHz infrared signals, decodes them, then moves the robot. (Download the library at the Code section).

I added explanations in the code, but the substance is that decodes IR signals coming from the main server, then depending on what the user wants, move the motors than will move the robot forward or turns left. Downloload the code:"Robot_Code". Load up this to your Arduino and and your robot will complete.

Step 18: The Main Server (Arduino Mega, Leonardo, Photon)

The server reads incoming data packets from the wireless headset. We’ll use the configured BT module to ensure the connection between the headset and the server. The Arduino Mega is the brain of the circuit, everything is connected to this microcontroller: Bluetooth, infrared transmitter LED, webserver, and mouse controller. The code is a bit complicated, but I added explanations to understand.

Step 19: The Case

It’s simple but looks great. I cut two 18x15 cm plates the smoothened the edges with sandpaper. Using some screws, I connected them to each other.

Step 20: The Relay Circuit

You’ll need these parts:

  • 2n2222 transistor (3 PCS
  • )germanium diodes (3 PCS)
  • 330 ohm resistors (3 PCS)
  • 1k Ohm resistors (3 PCS)
  • 5v relays (3 PCS)
  • hook-up wiresheader pins
  • PCB board
  • A picture worth more than a thousand words, so instead of trying to writing how should you connect the parts on the PCB look at the schematic If the “Signal” pin gets signal from the Arduino the relay will turn on. The transistor amplifies the signal to ensure enough power fot the relays. We will use 37-38-39 pins to control HIGH-LOW levels of each relay.

Step 21: Install Parts in the Case

To install the parts on the plexy-glass case I used some double sided white tape. This loks noce and hold the circuits on the case pretty strong.

Step 22: The Server-Circuitry

This server is software based so making the circuit isn’t so hard. (See schematic downer). You just need to power up the microcontrollers and make connection between them. The IR LED is connected to pin D3 and the relays to 37-38-39. D16 of the Arduino Mega goes to RX of the Photon and the D18 to the RX of the Leonardo.

Step 23: Connection With the Robot

The IR Leds are connected to the digital pin D3 and with the IRremote library we send codes to the robot. It's pretty simple. The IR codes must be the same in the robot's code. If you think you're done you can test it with your camera. The infrared light looks purple on a photo, cameras can detect the IR light. This trick always works.

Step 24: The Code

Use the "Arduino_Mega_Server" code in the Software part.

I suffered a lot while wrote the codes. I should program three different microcontrollers, so it was a big challenge. I used IRremote, Brain, SofwareSerial and Mouse libraries. (see download links at the Software part). But now the code is done and works so you just should upload to your microcontrollers. Download the .ino file or copy/paste the code in your IDE and use it. The code to the Particle Photon should be uploaded via the browser IDE. To make this register to theParticle Build. And connect your laptop to your microcontroller. I was really surprised that this happened almost automatically, I just added my Device ID number.

Before loading up the codes be sure nothing is connected to the RX/TX pins. So disconnect your Bluetooth module from the Mega, and disconnect the Mega from the Leonardo and the Photon.

Step 25: The Mouse Controller

The Arduino Leonardo controls the mouse.

Step 26: The Webserver

I wanted to add an IoT (Internet of Things) function in my project, so I made anonline data logger using the Particle Photon. Depending on what you make with the device the Photon creates a personal server, and writes the data in the Cloud.

This may seem scary at first time, but imagine that you can detect if you are stressful (attention level increases and decreases quickly) or if you should sleep (meditation level is always higher than 80). This webserver may help you to live healthlier.

Step 27: Coding in Browser

The Particle also has an online dashboard where you can publish any kind of data with the "Particle.publish();" syntax. We should have to say a big "thank you" for the developers of the dashboard. They saved a lot of time for us.

Writing codes in a browser?

It was intresting to write codes on a web page, but worked the same way like the normal IDE. The code was uploaded wirelessly. This microcontroller would be a very useful tool in everyone's toolbox.

Step 28: Webserver Appearance

The dashboard looks somehow like this if you worked fine. Shows Attention and Meditation, and you can check them anytime, at your dashboard URL. For more info about the dashboards click here.

Step 29: A Quick Upgrade for Those Who Want to Control Only a Robot With Mindwaves

You don't want to build the build the whole project, only a mindwave controlled robot? Don't worry if you're a beginner I thought even on you. I made a code with explanations that needs an Arduino Mini Pro and an IR LED and of course the headset to control the robot. I know that many of my readers wants to make a simple and fun weekend project so here's the description and the code for you.

Use the "RobotControlllerHeadset" code inthe Software section. Connect an IR LED to the D3 pin on the Mini Pro (Or Arduino UNO) and connect the T pin of the NeuroSky chip to the RX pin of the Arduino. Power up with 3.3 or 5 volts and you're done. So build the robot, then use this code in your headset:

#include <irremote.h><br>
#include <irremoteint.h></irremoteint.h></irremote.h>
#include <brain.h></brain.h>
IRsend irsend;
Brain brain(Serial);
const int ledPin = 3; 
long interval = 500; 
long previousMillis = 0;
int ledState = LOW; 
int medValue;
void setup() {
 // Set up the LED pin.
 pinMode(ledPin, OUTPUT);
 // Start the hardware serial.
 Serial.begin(57600);
}
void loop() {
 // Expect packets about once per second.
 if (brain.update()) {
 Serial.println(brain.readCSV());
 // Attention runs from 0 to 100.
 medValue = brain.readMeditation();
 }
 // Make sure we have a signal.
 if(brain.readSignalQuality() == 0) {
 // Send a signal to the LED.
 if (medValue > 65) {
irsend.sendNEC(0xFF10EF, 32); 
 delay(40);
 }
 if (brain.readAttention() > 65) {
irsend.sendNEC(0xFF18E7, 32); 
 delay(40);
 }
 } 
}

Step 30: Becoming an IoT Jedi (Testing)

After making everything that I explained here, only one thing is left: try it out. Use the power of your mind, use the Force of the neurons in your brain, move your little robot, control your home, control everything with thoughts...

Now you become a real IoT Jedi!

The device has many features and it's very fun to play with. But as I said before it isn't only a toy, featuring biology and electronics we created a device that may help for people, who fight each day with their disabilities, to get back their natural capabilities.


The robot can be controlled easily, the relays aren't big challange too. You only should learn how to control your attention levels to control the mouse. On the picture I'm trying to exit an app with my brainwaves. More practice guarantees better experience.

Step 31: Thank You for Watching!

I really hope that you liked my project and please write your opinion in comments. I wanted to make a video about the device, but coding was hard so I haven't enough time to make a short movie, but I'll publish it very soon on my Youtube channel and I'll update this project presentation too.

Being the richest man in the cemetry doesn't matter to me... Going to bed at night saying we've done something wonderful... That matters to me - Steve Jobs

May the Force be with you, always!

Full Spectrum Laser Contest 2016

Participated in the
Full Spectrum Laser Contest 2016

Gaming Contest

Participated in the
Gaming Contest