Introduction: Voice-controlled PC Assistant

Picture of Voice-controlled PC Assistant

Ok so I know that there are a lot of these out there already, but I couldn't find a voice controlled PC assistant that I really liked, so I made my own. The assistant will do everything from tell jokes to opening programs, and I even coded a little script to help play the popular free game, Team Fortress 2. I have used a fantastic program called 'GlovePIE' to make her, which uses its own programming language. In this Instructable I will teach you how to install and use the assistant. You should learn enough just from looking at the code itself to be able to edit and write your own voice controlled assistants, but if you do want a separate instructable on how to code in GlovePIE just ask for it in the comments. 

Note that this is not an intelligent AI, you can't really hold a full conversation with it like you can with a bot like cleverbot. It only responds to what it is coded to do. I have coded simple conversations though, e.g. how are you, fine thanks etc. 

And yes, I did call it Jarvis, because it was inspired by the Iron Man film. Unfortunately, I don't think that there is a way to change to bot's voice from the default female voice, so we're stuck with a female Jarvis for now. 

A list of commands is on step 3


LOOK AT THIS:
How to control JARVIS, from your phone!! - instructables.com/id/Voice-control-your-PC-from-your-phone-android/
How to make JARVIS control videogames!! - instructables.com/id/Voice-control-games-using-GlovePIE/

Step 1: Preparation

Picture of Preparation

1. First you need to install GlovePIE. This is simple enough, just go to this link:

http://glovepie.org/glovepie_download.php

(just click the link that says that you are 100% Green)

2. Once downloaded, extract the zip file into a folder called 'GlovePIE' on your desktop.

3. Double-click the file named 'PIEfree' to open GlovePIE

Next you need to download my code to use in GlovePIE. Just go to this link:

http://www.mediafire.com/view/vtzqb1vyc53gmm2/jarvis.PIE

Once downloaded, just drop it into your desktop

Step 2: Getting It to Work

Picture of Getting It to Work

Go back to GlovePIE and go to file>open, then navigate to your desktop and open the Jarvis.PIE file. 

Now all you have to do is click 'run' and you are good to go!

The next step will tell you how to use the assistant, and what she can do.

Step 3: Commands

Picture of Commands

Jarvis has two modes: listening and not listening. While Jarvis is not listening, you can say whatever you like and it will not respond to you (except for a few key words). When Jarvis is listening, it will be able to respond to you.

To get Jarvis to listen, say:
'Jarvis' or 'Hi Jarvis' or 'Hello Jarvis' or 'Jarvis listen'.

Once you have said this, Jarvis will continue to listen until you tell her to stop.

To get Jarvis to stop listening say:
'Thanks Jarvis' or 'Bye Jarvis' or 'thanks' or 'shut up'.



Commands you can say while Jarvis is not listening:

1. Toggle whether Jarvis listens or not - 
Explained above

2. Lock or unlock Jarvis - 
If you say 'lock' or 'I want you to respond to no one but me', Jarvis will shut down until she hears the password. Upon hearing the password, she will regain functionality. The default password is 'potato', you can change this in the code.

Commands when Jarvis is listening:

1. Show/hide the window - 
If you don't like having a window full of code open just say 'hide' to make it disappear. Say 'show to make it reappear.

2. Change your name - 
I have put 3 default names in, these are 'sir', 'sire', and 'overlord'. Say 'call me overlord' to make Jarvis address you as overlord. If you want a different name, you will have to put it in the code yourself. It's inconvenient, but its how GlovePIE works.

3. How are you - 
Say 'how are you' or 'how's it going' to chat with Jarvis

4. Type
Jarvis is not a dictation bot, so will not automatically type words from what you say, but she can type letters. Say 'start typing' to be able to type letters individually, and say 'stop typing' to stop Jarvis responding to individual letters. Say 'is typing enabled' to find out whether she is in typing mode or not.

5. Tell jokes
Jarvis has a bank of 10 or so cheesy jokes, just say 'tell me a joke' or 'say something funny' to hear a random joke.

6. Close window
Jarvis can 'Alt-f4' a window. Just say 'close window', then when she asks if you are sure you want to close it say 'yes'. (say 'no' to cancel)

7. Open programs
Jarvis can open programs and websites just say 'open chrome' for example.
List of default programs: notepad, google, ebay, facebook, youtube, gmail, spotify, steam, chrome, skype
In the next step I will explain how to change these programs.
Not all of these might work for you, as the programs might be in a different location on your PC. Again, I will explain how to fix this in the next step

8. Music
Just say 'play' or 'pause' to start and stop a music track

9. Alt-tab
Say 'cycle windows' to alt-tab. Say next to choose a different program, and say 'ok' or 'that one' to select it.

10, TF2 controls
If you play tf2, then I have coded a couple of controls here. Say 'enable tf2 control' to enable it. (and disable tf2 control' to disable it)
If you say 'medic', 'spy', or 'help' in game, then your character call out the corresponding phrase. Not a game-changer, but is a bit more convenient.

11. Screensaver
say 'sleep' to go into a screensaver, say 'wake up' to close the screensaver

12. Show desktop
Say 'show desktop' to show the desktop 





Step 4: Adding Your Own Programs

Picture of Adding Your Own Programs

If you want to add or edit your own programs to the programs list, scroll down in GlovePIE until you see the following code:

if said("open [insert program]", 5) then
    say("opening [insert program]")
    Execute("[insert program path")
end if

Insert the name of your program into the 'insert program' square brackets. Finally, follow these steps to insert the program path:

1. Navigate to your program location, navigating to a shortcut is fine, for example, using a shortcut on your desktop is fine.
2. Right click on the icon 
3. Press properties
4. Copy (Ctrl+C) all the text inside the box called 'target' and paste it into the 'insert program path' section

It should look like this:

if said("open notepad", 5) then
    say("opening notepad")
    Execute("C:\Windows\system32\notepad.exe")
end if




Step 5: Final Notes

Picture of Final Notes

If you need an instructable on coding in GlovePIE then just comment below, but just looking at the code should give you enough information to get started.

If you happen to have displayfusion installed, you can say 'move window to the other screen' to switch screens.

Oh and if you have your speakers up too loud, Jarvis will hear herself and start having weird conversations with herself. 

I think that's everything. If you have any ideas about more features that could be added I would love to hear them!

Comments

mArisch (author)2017-06-30

Hey, I made an instructable for how to get Google Assistant on your PC/Mac/Linux
https://www.instructables.com/id/Google-Assistant-for-PCMacLinux/

Check it out-it's my first Instructable!

Jack Akkiarto (author)2015-12-29

Hey, i don't know if you can help me or if this is dead xD

First wow, this is amazing! I made a pretty nice assistant in less then 4 hours (still much to do), because your instructions are very helpful and easy to understand.

But i want to change the voice (to german male) and i dont know where to get new voices (pls free) or how to use them after.

Would be very happy if you could help me

Oh, and sry if this post is wrong somehow, this is myy first comment ^^

RohitA1 (author)2015-03-26

hey this program is amaizing but can we make it a dictation bot by programming or by any other meathod

OptimumAlliance (author)RohitA12015-03-27

Not as far as I know sorry, it doesn't translate what it hears into words it just compares what it hears to whatever you've told it to listen for in the code.

MageE1 (author)OptimumAlliance2015-10-17

Hey Optimum I was wondering if you could add something like "Jarvis Scroll down" and have it scroll down and the same for up and of course "Scroll faster" and "Scroll down" mainly because I can't find out how to input scrolling into glovepie

danielcox4357 (author)2015-02-05

is there a way to type with Jarvis

MageE1 (author)danielcox43572015-10-17

yes say to jarvis start typing and then say letter by letter the word/words you want to type and then say stop typing to disable typing with jarvis

MageE1 (author)2015-10-17

i would like to know how to use "jarvis" to scroll down/up

DamageReport (author)2015-08-06

If you have other voices installed in the Microsoft speech settings, how do you enable them in GlovePie?

KrisztiánK (author)2015-08-03

Hi!

I have some problem. It doesn't work for me. I think the reason is, the operation system of my computer is not english, and at my language windows 7 doesn't have speech recognizer. What do you think? Is there any solution for that, or should I somehow change the language of my op system?

RohitA1 (author)2015-05-28

Is their any method through which we can start computer by using smartphone with using ur program

ArendG (author)2015-05-14

Hi m8

i was looking around for a good jarvis free maby i found it here, NOT TESTED YET
me just learn coding after playing games 30 years straight.
and i notiched u got a female voice, probaly standaard windows speech to text.
If u want the real jarvis u must buy or get a good cracked, IVONA 2 Speech, British Brian Voice, That wil install in your windows speech and recognition . u can change there the voice to Brain/Jarvis ;-)

jemson.pagalan (author)2014-12-27

hi i jarvis version 1.0, i like much than the latest version, but i want to know, is it possible to control programs using jarvis,?

Sometimes. It only works for programs that use or support keyboard shortcuts. Find the keyboard shortcut for the action you want to perform, and program jarvis to execute that shortcut on demand. An alternative, and somewhat finnicky method is to use xy commands. If you know the xy position of a particular button on the program, you can tell jarvis to move the cursor to that position and click, however this relies on the window being in the same place every time.

AnhN3 (author)2014-12-13

i want to set command shutdown computer and want it ask me "are you sure" to i can answer yes or no but already used the "close window" command but it doesn't work please write of me that command

OptimumAlliance (author)AnhN32014-12-14

If you want to shut down your pc you can create a batch file to actually shut down the pc and link a voice command to that file, here's a website for creating the batch file: http://www.robvanderwoude.com/shutdown.php

AnhN3 (author)OptimumAlliance2014-12-14

thanks but if i want to play a song i want in window media player how can i create a command that when i said "play music" it ask me "which song" and i can tell it the song name i want

jason.pavelock (author)2014-10-26

it wont let me change the default voice, how do i do so?

I'm afraid it can't be done in GlovePie as far as I know, sorry

Edit: scroll down and check out Nishant Shubham's comment, he seems to have found a way to do it

cenkogtk (author)2014-10-11

I have a problem. I press the run button, and say jarvis, but nothing happens. What should I do?

OptimumAlliance (author)cenkogtk2014-10-26

Try assigning the 'Jarvis' command to a keyboard key first to test that the code all works

If it does work with the keyboard, mess around with the voice sensitivity

cenkogtk (author)2014-10-11

Nice project btw :)

OptimumAlliance (author)2014-04-19

Yeah sure you can use it

OptimumAlliance (author)2014-04-18

It would be fairly easy to just have it open a website with the weather on it, however I'm not sure if it is possible for it to read out the website. It's possible that you could use a 'screen scraper' program in conjunction with glovepie, I'll look into it and get back to you.

calebhk98 (author)2014-04-16

Can it work by typing?

You can assign keypresses to the functions e.g. if pressed(f8) then var.mute = true end if

Another way to do this is: key.f8 = //whatever output you want

Activeproject (author)2014-03-04

Hi there,

may i say you are brilliant sir! thanks a lot , great project and great work! if I may, what would be the best place to learn how to write this program from? I don't think there are many books or literature floating around..

Hello! I made an instructable: http://tinyurl.com/l8wztmq which gives a basic overview of coding in GlovePIE and applies it to a videogame. Another great way to learn is by looking at the code itself and using it to write your own (which is how I learnt), or if you like, I could make a separate instructable on more general, non-videogame centered coding. Feel free to direct message me if you have any coding questions or would like some more support by going to my profile and clicking 'direct message'.

Nishant Shubham (author)2014-02-21

One of the best projects I have seen so far. Thanks for this awesome project.

thank you:)

Actually there is a way to get the male voice...............just download ivona and crack it............I am using the Ivona 2 Brian UK English Male voice.........It only has a 30 day trial period so be sure to use this - http://kickass.to/ivona-cracked-easy-setup-phr34k-t7816987.html .............the program will suggest to download the HAL reader but i personally use the espeak TTSapp program.

oh cool thanks, now Jarvis can sound more like the actual Jarvis

danilo.bar1 (author)2014-02-21

I saw that the front page of glovepie.org was hacked. Are you sure that the download is safe?

However, excellent topic!

It will be one of the next softwares that I will install. :-)

Yeah I saw that too, but the download link I've provided seems to be fine, I scanned it with my antivirus program. I managed to get to the glove pie downloads page through a link in the PC world website

About This Instructable

41,936views

89favorites

License:

More by OptimumAlliance:Cheaply Record Studio-quality MusicElectric Cigar-Box guitar w/ Built in amplifierCheap Product Photography in Under an Hour (Soft box)
Add instructable to: