Introduction: Stream PC Desktop to Phone for Virtual Reality (Now With Head Tracking!)

Picture of Stream PC Desktop to Phone for Virtual Reality (Now With Head Tracking!)

I've loved virtual reality for a long time. Although you won't need one, I finally got my hands on a 3D Printer, and was able to print a VR headset.

Unfortunately, this doesn't come with any electronics. I've been using my iPod Touch as its screen.

This tutorial will teach you how stream your PC to your Android or iOS device with minimal lag, and head tracking. I've been able to watch movies and play PC games such as Portal in stereoscopic 3D, with full head movement.

Let's get started!

Step 1: Currently Available Apps for Streaming

Picture of Currently Available Apps for Streaming

There are five programs that I use to stream my desktop to my phone/iPod. All of them have different pros and cons.

1. Kinoni Remote Desktop is probably the least laggy streamer out there. It can be downloaded at http://www.kinoni.com/

Pros:

I ran many games like Portal and saw no perceptible lag while in game.

High framerates.

Free!

Cons:

The audio quality isn't great.

Android only.

2. Trinus Gyre is definitely the second best. The streaming is just above average, but what really puts it ahead is its built in head tracking. The moment you start streaming, depending on your settings, the mouse should move automatically as you rotate your head. It can be downloaded at http://oddsheepgames.com/?page_id=134

Pros:

Built in head tracking? Yes, yes, yes!

Relatively high framerates.

Cons:

Android only.

Head tracking needs a some work.

Costs money (5.27 USD for the app; the PC streamer is free)

3. Kainy would have been second on this list a few months ago, I didn't know about Trinus Gyre yet (it might not have been out). It can be downloaded at http://kainy.com/download.html

Pros:

Kainy can stream up to 60 FPS (frames per second) with minimal lag.

Cons:

Free version has ads, which makes it unsuitable for VR gaming.

Android only. (Support to iOS supposedly coming soon)

4. Splashtop is number four because it only streams at 30 FPS, with pretty noticeable lag/jitter. It can be downloaded at http://splashtop.com

Pros: iOS AND Android compatibility! Yay!

Free on Android!

Cons:

Slow, especially on iOS devices. Like, really slow.

Costs money for the iOS app (5 USD)

Unusable for VR gaming.

5. Teamviewer is actually not that bad considering that it was originally intended for business. It can be downloaded at http://teamviewer.com

Pros: iOS AND Android compatibility! Yay!

Free!

Cons:

Lag, Lag, Lag. Everywhere. Unusable for VR gaming.

Step 2: Head Tracking

Picture of Head Tracking

I have 2 favorites for head tracking.


1. A Small Gyro Mouse is actually incredibly fast. If there's any lag, it's imperceptible to me! I ordered a Tronsmart one at http://www.amazon.com/Tronsmart-TSM-01-Keyboard-Mo...

Pros:

Wireless

No lag that I can notice

Relatively accurate tracking

Rechargeable batteries included, along with a charging cable.

Pretty small.

Cons:

Looks a goofy when you mount this to your head, whether it's by rubber banding it to your headphones or anything.

Difficult to mount reliably to headset. Easier to just mount to headphones.

2. Trinus Gyre was mentioned in the previous page. If you didn't read it, it's a streaming program that uses your smartphone's gyroscope to emulate mouse movement. Download here: http://oddsheepgames.com/?page_id=134

Pros:

No need for extra sensor.

Looks better than putting a remote on your head. (That's what the Tronsmart Air mouse looks like)

Cons:
Android only.

Only as fast as your WiFi and your smartphone.

Tracking is still in development; it needs some work. (Slight lag, drift, etc)

Step 3: Conclusion

Picture of Conclusion

Now that you have read this tutorial (and hopefully downloaded some software too) you should be able to play some VR games with your mobile phone VR headset.

I hope you enjoyed!

If you have any questions, please PM me!

Comments

LucasK1 (author)2014-11-12

For those with a nVidia GPU, you can use Limelight for streaming

BrendanC1 (author)LucasK12014-11-13

Excellent point! I didn't include Limelight because the requirement doesn't end at nVidia. You need a pretty good GPU... It's gotta be GTX, and in the 600 series. Sure, a GTX 650 ti boost won't break the bank, but I have a GTX 560 ti, and wasn't even able to try it. Therefore, I could not include it in my review.

Squidyman (author)BrendanC12015-04-22

I can use limelight (GTX 960), but you can only stream games using it. Until that changes, I don't see it being ideal

BrendanC1 (author)Squidyman2015-04-22

Well, as of right now, the main use Virtual Reality has IS gaming. I have yet to honestly see an incredibly innovative virtual reality application that doesn't have some sort of gaming aspect. Even so, it is too bad that limelight only works with NVidia GameStream enabled games. I'm sorry that this specific solution isn't working for your needs. Definitely check out the options that are featured in the i'ble though!

SergioE29 (author)BrendanC12017-10-04

Maybe you are not seeing autocad vlogs but there are some serious applications in engineering.

sumsboy. (author)BrendanC12016-01-29

you can add remote desktop in Geforce experience

look for mstsc.exe

locatd in C/windows/system32

then you can play any game and watch movies

Squidyman (author)BrendanC12015-04-23

I am a bit different in the sense I am looking to use the googles as an FPV set for my quadcopter. I need to find a program that takes a AV to digital converter and sends it to my phone for wearing with my goggles. I think some of the programs you listed will do that.

Justins14 (author)LucasK12015-04-02

can you only play steam games with it? thats all im getting the options for

BrendanC1 (author)Justins142015-04-22

As of right now, the games you can play with Limelight are the games that are supported with NVidia's Gamestream.

This list (http://shield.nvidia.co.uk/pc-game-list/) should get you started on which games are supported. There are a bunch of games that aren't mentioned on that page that ARE supported though, so don't get discouraged if it's not on the page. But to answer your question, the majority of the compatible games are also available on Steam. The two things aren't related though... if you buy Portal 2 on a disk (not Steam) it will still show up on Limelight.

ddas3 (author)2016-12-25

how to get kinoni for mac ? only windows pc version is available

Zenoscope (author)2016-09-14

thanks for the 'ible, I've used trinus vr (formerly trinus gyre) for the streaming and freepi for the head tracking. works pretty well, trinus does sbs and you can change the interpupillary distance as well.

rad-io (author)2016-02-25

You can try Remotr. I'm using it for streaming games from my PC, it's the best available today :)

Arief98 (author)rad-io2016-09-06

It doesn't support SBS display right?

NathalieB18 (author)2016-08-05

Hi, I have my own headset and an iPhone.. I would like to use this simply to see my desktop in the goggles.. I installed splashtop but that doesn't do the split screen.. Any ideas on how to acheive this ?

peter---peter (author)2016-05-13

my antivirus went all bananas on kinoni, is this safe to use?

BrendanC1 (author)peter---peter2016-05-13

Hmm, that's odd. The last time I used it, it was definitely virus-free.

It's possible that has changed, so be careful.

If you're using McAfee or Norton you can safely ignore the warning though.

MakerMan234 (author)BrendanC12016-07-26

I herd of trinus and i found out that there is a trial version for the android

and a lot of people had no problems with it

JilkO1 (author)2016-07-04

I used trinus gyre and it works great, a little lag but not much. I just got the vr box 2.0, stuck a gyroscopic / air mouse on the strap and linked my phone with trinius VR. It costed about £40 but it is worth it!

MakerMan234 (author)2016-04-30

I herd of trinus and i found out that there is a trial version for the android

and a lot of people had no problems with it

fpiras (author)2016-04-15

I made my DIY Virtual Reality headset for Ipad mini: https://www.youtube.com/watch?v=O-7CeCbD6Yk

jayants7 (author)2016-04-07

thnx

JamesG123 (author)2015-12-29

I got everything working, only problem is there are no instructions to get my screen SBS.

This is quite cool. I have an old Logitech MX Air Mouse, I just put it in a hat on my head, put GTAV in FP view, and it works. Using Kino to stream my pc screen to my phone, got my phone in the View-Master VR headset. But I need my pc screen to show in SBS. Any tips?

qSheep (author)JamesG1232016-02-02

You could trie these sofwares :

- Vireio Perception (Free and opensource)

- TriDef 3D (not free : $39.99)

symedley (author)2016-01-11

I missed the part where you can get Splashtop to work with cardboard. I think I'm missing something very basic. Any help appreciated.

MarkL96 (author)2015-11-10

sorry to hear you have laggy experience with Splashtop. Splashtop is in fact well known for its high performance remote 3D gaming. if your computer has NVIDIA or AMD/ATI GPU, Splashtop Streamer is able to take advantage of hardware acceleration. In fact, NVIDIA CEO Jensen was on stage at CES showcasing Splashtop remote gaming (60+ fps at <33ms latency, all over a wifi network at CES). Potentially, there has been some set up issue. We'll be glad to help. contact us at support@splashtop.com thanks

ChaseW3 (author)2015-10-20

With spashtop how do I turn on dual veiw

RobertR66 (author)2015-09-06

It makes me sad that it looks like VR isn't for ios... :(

QuinnH2 (author)2015-08-28

Why is there no mac support on any of these except for Splashtop?

DavidD58 (author)2015-07-31

Perhaps I missed it but I didn't see mention of any video player I can use that supports tracking.

Dirkalurkastan (author)2015-07-20

Forget streaming check out this tiny hdmi monitor: http://store.smallhd.com/store/501-monitors-bundle...

Just wire to pc, run tridef, strap it, cardboard and gyro mouse to head. Bang, done.

cheaper ones coming hopefully: http://pre-order.cmoar.com/collections/hmd/product...

Dirkalurkastan (author)2015-07-19

Has anyone tried Intugame?

KevinR21 (author)2015-07-17

MarkH21 (author)2015-04-12

Great setup! Any options for streaming my whole PC desktop (duplicated on the Android screen, but without 3D experience)? Aiming to do programming on the sun ;-)

BrendanC1 (author)MarkH212015-04-12

Thanks! If you're planning on using a streamer for programming, i highly recommend Splashtop. Its latency is incredibly low when you're not gaming. Splashtop doesn't come with SBS 3d support, so it's perfect for you.

MarkH21 (author)BrendanC12015-04-12

Will check it thanks!

Got the missing link too, that is duplicating the PC screen to the left and right side of the phone screen (I guess there is no such option in any streamer). Solution: need to run two vnc-clients (or any remote desktop sessions) pointing to the same actual desktop. Can easily do this by running a single vnc-server with the right geometry on a Linux server (i.e. half width of phone screen x height of phone screen), then opening two client sessions from say a Windows ;-)

XYZAidan (author)2015-03-29

Nice instructable, Brendan!

nishantkinshu (author)2015-03-28

for head tracking, instead of buying a separate gyro mouse, you can also use a spare phone (this is what i am doing :P), and run a 3d mouse application on it(using monect 3d mouse).

works for me.

otykier (author)2015-01-03

Are there no options for using the accelerometer/gyrometer inside the phone/tablet, for iOS

BrendanC1 (author)otykier2015-01-18

Unfortunately, I don't know of any. If you do find one, let me know!

TheEpicHunter (author)BrendanC12015-03-08

There is a way, but it's pretty complicated and requires 2 iOS devices, one for tracking and one for streaming, mainly because the tracking app dosn't stay open in the background, but if you have iOS 7 and can install Background Manager then you only need 1. Here's the info: http://www.mtbs3d.com/phpbb/viewtopic.php?f=139&t=15484 I've gotten it to work quite well, so I hope you can too. Oh, you will also need a program called SakasaMouse, which allows you to reverse the X and Y of your mouse, as the mouse output of FreePie is reversed.

BrendanC1 (author)TheEpicHunter2015-03-20

Interesting. If you were to jailbreak your iOS (which I do not in any way suggest), do you think there's a tweak to keep them both running at once?

dacian.solgen.3 (author)2014-10-19

i wish that there was a app for iOS with VR & Head track feature.

There is a way, but it's pretty complicated and requires 2 iOS devices, one for tracking and one for streaming, mainly because the tracking app dosn't stay open in the background, but if you have iOS 7 and can install Background Manager then you only need 1. Here's the info: http://www.mtbs3d.com/phpbb/viewtopic.php?f=139&t=15484 I've gotten it to work quite well, so I hope you can too. Oh, you will also need a program called SakasaMouse, which allows you to reverse the X and Y of your mouse, as the mouse output of FreePie is reversed.

VinhDicate (author)TheEpicHunter2015-03-19

Could you possibly make a step by step? That link is old and confusing, i also feel there is alot of information not needed. My VR janky set up, iphone 5, IOS 8, TeamViewer, and Sensor Data. i'm trying to view virtual reality videos with head tracking. Not so interested in games yet.

PabloR7 (author)2015-03-08

Hi, thanks for the information!

I'm curious to know how could you extract the output of the Tronsmart mouse into the host program.

RitikS (author)2015-03-06

Use freePIE, it records input from your phone and other input devices. Only for windows.

DragonSkills23. (author)2015-02-05

I've tried all the application in the article and i can say that Kainy is the best app for cardboard/oculus vision, it cost less than trinus Gyre and the headtracking is just perfect with my samsung s4. Instead with Trinus Gyre , freepie (using all the script around the web) and opentrack not seems to work well, mabye is the s4 inself but i dubit. The only cons it that with kainy you need an external side by side 3d software.

Thanks.

Hey there! Thanks for reading!

Yes, I've heard many times that Kainy is great for streaming. I haven't had as good results with it as I have with Trinus, so I put it third on the list. These best of these five options often depends on your system. For example, on 5ghz WiFi, I often get better results with Trinus than with Kinoni. However, on most WiFi networks, Kinoni seems to have less jitter AND latency.

I'm glad that you found a good solution. Once again, thanks for reading (and commenting!).

bobes360 (author)2015-01-18

nice instructable but I have some questions : trynus gyre doesn't work for games and kinoni remote with tridef3d doesn't support control using gyroscope and I haven't found any program to do so ... please help.

BrendanC1 (author)bobes3602015-01-18

Actually, Trinus Gyre is intended to work specifically for 3D games. Can you provide any information as to why Trinus Gyre doesn't work? I may be able to help.

About This Instructable

554,133views

121favorites

License:

More by BrendanC1:Make a dumb device smart with Pilight and HomeKitUse Printrbot's Autoleveling Probe with Cura!Stream PC Desktop to Phone for Virtual Reality (Now with Head tracking!)
Add instructable to: