Virtual Reality Telepresence With Intel Edison (Intel IoT)

This is a project we worked on at Intel IoT Roadshow 2016.

The Intel Edison compute chip is a rather powerful board with built in WiFi and bluetooth capabilities. This makes it perfect for some slightly more intensive IoT applications.

I used a google cardboard and mapped the user's head movements to a camera mounted on a servo motor, making it look like a full 360 degree video.

Step 1: One the Phone

I used Trinus VR to display a camera feed from my laptop to my phone. Trinus also translates head movement to mouse pointer movement, which we can then capture.

Trinus requires a mobile app that is compatible with google cardboard, and a desktop app.

Download and install them.
http://trinusvr.com/

Step 2: On Your PC

Since Trinus moved the mouse pointer based on head rotation, I could capture this mouse movement and use it to sent requests to the Edison.

I did this with a little Python script that uses the win32 api to capture the mouse position every few milliseconds and make a get request to the nodejs server running on the Edison.


Here is the code:
import win32api

import time import urllib2

running = True width = win32api.GetSystemMetrics(0)/2 height = win32api.GetSystemMetrics(1)/2

while(running): x, y = win32api.GetCursorPos() win32api.SetCursorPos((width,height)) if(x-width >= 100 or y-height >=100): print "Vert: %s, Hor: %s" % (x-width, y-height); time.sleep(0.1) if(x-width>5): urllib2.urlopen("http://192.168.21.207:8081/right") if(x-width<-5): urllib2.urlopen("http://192.168.21.207:8081/left")

Step 3: On the Edison

The Intel Edison compute chip runs a full linux distro called Yocto. It also comes with Nodejs as well as a bunch of preinstalled libraries for interacting with the hardware ports.


Code:
var express = require("express")

var app = express() var mraa = require("mraa") var pwm = new mraa.Pwm(3) var groveSensor = require('jsupm_grove'); var led = new groveSensor.GroveLed(6);

var led2 = new groveSensor.GroveLed(5);

pwm.enable(true); pwm.period_us(2000); var value = 30;

//pwm.write(value);

app.get('/', function (req, res) { res.send('Hello World!'); });

app.get('/right', function (req, res){ pwm.write(0); setTimeout(function(){pwm.write(3);console.log("IT IS DONE");}, 100); led2.on(); setTimeout(function(){led2.off();},100); res.send('ryt'); })

app.get('/left', function(req, res){ console.log('hit') res.send('left'); led.on(); setTimeout(function(){led.off();}, 100); //insert action here })
app.get('/stop', function (req, res){

pwm.write(5); // setTimeout(function(){pwm.write(3);console.log("IT IS DONE");}, 100); res.send('stoppin'); }) var server = app.listen(8081, function () { var host = server.address().address var port = server.address().port console.log("Example app listening at http://%s:%s", host, port) })



Stick this in a javascript file and run it.


I also used a Grove starter kit to power two LEDs and a servo motor that had the webcam mounted on it. The webcam feed is then streamed and can be accessed from any browser.

https://github.com/drejkim/edi-cam

This repo has a great way to implement the stream.