Voice-controlled PC Assistant

About: Computer Science student in the UK

Intro: Voice-controlled PC Assistant

Ok so I know that there are a lot of these out there already, but I couldn't find a voice controlled PC assistant that I really liked, so I made my own. The assistant will do everything from tell jokes to opening programs, and I even coded a little script to help play the popular free game, Team Fortress 2. I have used a fantastic program called 'GlovePIE' to make her, which uses its own programming language. In this Instructable I will teach you how to install and use the assistant. You should learn enough just from looking at the code itself to be able to edit and write your own voice controlled assistants, but if you do want a separate instructable on how to code in GlovePIE just ask for it in the comments. 

Note that this is not an intelligent AI, you can't really hold a full conversation with it like you can with a bot like cleverbot. It only responds to what it is coded to do. I have coded simple conversations though, e.g. how are you, fine thanks etc. 

And yes, I did call it Jarvis, because it was inspired by the Iron Man film. Unfortunately, I don't think that there is a way to change to bot's voice from the default female voice, so we're stuck with a female Jarvis for now. 

A list of commands is on step 3


LOOK AT THIS:
How to control JARVIS, from your phone!! - instructables.com/id/Voice-control-your-PC-from-your-phone-android/
How to make JARVIS control videogames!! - instructables.com/id/Voice-control-games-using-GlovePIE/

Step 1: Preparation

1. First you need to install GlovePIE. This is simple enough, just go to this link:

http://glovepie.org/glovepie_download.php

(just click the link that says that you are 100% Green)

2. Once downloaded, extract the zip file into a folder called 'GlovePIE' on your desktop.

3. Double-click the file named 'PIEfree' to open GlovePIE

Next you need to download my code to use in GlovePIE. Just go to this link:

http://www.mediafire.com/view/vtzqb1vyc53gmm2/jarvis.PIE

Once downloaded, just drop it into your desktop

Step 2: Getting It to Work

Go back to GlovePIE and go to file>open, then navigate to your desktop and open the Jarvis.PIE file. 

Now all you have to do is click 'run' and you are good to go!

The next step will tell you how to use the assistant, and what she can do.

Step 3: Commands

Jarvis has two modes: listening and not listening. While Jarvis is not listening, you can say whatever you like and it will not respond to you (except for a few key words). When Jarvis is listening, it will be able to respond to you.

To get Jarvis to listen, say:
'Jarvis' or 'Hi Jarvis' or 'Hello Jarvis' or 'Jarvis listen'.

Once you have said this, Jarvis will continue to listen until you tell her to stop.

To get Jarvis to stop listening say:
'Thanks Jarvis' or 'Bye Jarvis' or 'thanks' or 'shut up'.



Commands you can say while Jarvis is not listening:

1. Toggle whether Jarvis listens or not - 
Explained above

2. Lock or unlock Jarvis - 
If you say 'lock' or 'I want you to respond to no one but me', Jarvis will shut down until she hears the password. Upon hearing the password, she will regain functionality. The default password is 'potato', you can change this in the code.

Commands when Jarvis is listening:

1. Show/hide the window - 
If you don't like having a window full of code open just say 'hide' to make it disappear. Say 'show to make it reappear.

2. Change your name - 
I have put 3 default names in, these are 'sir', 'sire', and 'overlord'. Say 'call me overlord' to make Jarvis address you as overlord. If you want a different name, you will have to put it in the code yourself. It's inconvenient, but its how GlovePIE works.

3. How are you - 
Say 'how are you' or 'how's it going' to chat with Jarvis

4. Type
Jarvis is not a dictation bot, so will not automatically type words from what you say, but she can type letters. Say 'start typing' to be able to type letters individually, and say 'stop typing' to stop Jarvis responding to individual letters. Say 'is typing enabled' to find out whether she is in typing mode or not.

5. Tell jokes
Jarvis has a bank of 10 or so cheesy jokes, just say 'tell me a joke' or 'say something funny' to hear a random joke.

6. Close window
Jarvis can 'Alt-f4' a window. Just say 'close window', then when she asks if you are sure you want to close it say 'yes'. (say 'no' to cancel)

7. Open programs
Jarvis can open programs and websites just say 'open chrome' for example.
List of default programs: notepad, google, ebay, facebook, youtube, gmail, spotify, steam, chrome, skype
In the next step I will explain how to change these programs.
Not all of these might work for you, as the programs might be in a different location on your PC. Again, I will explain how to fix this in the next step

8. Music
Just say 'play' or 'pause' to start and stop a music track

9. Alt-tab
Say 'cycle windows' to alt-tab. Say next to choose a different program, and say 'ok' or 'that one' to select it.

10, TF2 controls
If you play tf2, then I have coded a couple of controls here. Say 'enable tf2 control' to enable it. (and disable tf2 control' to disable it)
If you say 'medic', 'spy', or 'help' in game, then your character call out the corresponding phrase. Not a game-changer, but is a bit more convenient.

11. Screensaver
say 'sleep' to go into a screensaver, say 'wake up' to close the screensaver

12. Show desktop
Say 'show desktop' to show the desktop 





Step 4: Adding Your Own Programs

If you want to add or edit your own programs to the programs list, scroll down in GlovePIE until you see the following code:

if said("open [insert program]", 5) then
    say("opening [insert program]")
    Execute("[insert program path")
end if

Insert the name of your program into the 'insert program' square brackets. Finally, follow these steps to insert the program path:

1. Navigate to your program location, navigating to a shortcut is fine, for example, using a shortcut on your desktop is fine.
2. Right click on the icon 
3. Press properties
4. Copy (Ctrl+C) all the text inside the box called 'target' and paste it into the 'insert program path' section

It should look like this:

if said("open notepad", 5) then
    say("opening notepad")
    Execute("C:\Windows\system32\notepad.exe")
end if




Step 5: Final Notes

If you need an instructable on coding in GlovePIE then just comment below, but just looking at the code should give you enough information to get started.

If you happen to have displayfusion installed, you can say 'move window to the other screen' to switch screens.

Oh and if you have your speakers up too loud, Jarvis will hear herself and start having weird conversations with herself. 

I think that's everything. If you have any ideas about more features that could be added I would love to hear them!

Share

    Recommendations

    • Optics Contest

      Optics Contest
    • Plastics Contest

      Plastics Contest
    • Audio Contest 2018

      Audio Contest 2018

    37 Discussions

    0
    None
    mArisch

    1 year ago

    Hey, I made an instructable for how to get Google Assistant on your PC/Mac/Linux
    https://www.instructables.com/id/Google-Assistant-for-PCMacLinux/

    Check it out-it's my first Instructable!

    0
    None
    Jack Akkiarto

    2 years ago

    Hey, i don't know if you can help me or if this is dead xD

    First wow, this is amazing! I made a pretty nice assistant in less then 4 hours (still much to do), because your instructions are very helpful and easy to understand.

    But i want to change the voice (to german male) and i dont know where to get new voices (pls free) or how to use them after.

    Would be very happy if you could help me

    Oh, and sry if this post is wrong somehow, this is myy first comment ^^

    0
    None
    RohitA1

    3 years ago

    hey this program is amaizing but can we make it a dictation bot by programming or by any other meathod

    2 replies
    0
    None

    Not as far as I know sorry, it doesn't translate what it hears into words it just compares what it hears to whatever you've told it to listen for in the code.

    0
    None
    MageE1OptimumAlliance

    Reply 3 years ago

    Hey Optimum I was wondering if you could add something like "Jarvis Scroll down" and have it scroll down and the same for up and of course "Scroll faster" and "Scroll down" mainly because I can't find out how to input scrolling into glovepie

    0
    None
    MageE1danielcox4357

    Reply 3 years ago

    yes say to jarvis start typing and then say letter by letter the word/words you want to type and then say stop typing to disable typing with jarvis

    0
    None
    MageE1

    3 years ago

    i would like to know how to use "jarvis" to scroll down/up

    0
    None
    DamageReport

    3 years ago on Introduction

    If you have other voices installed in the Microsoft speech settings, how do you enable them in GlovePie?

    0
    None
    KrisztiánK

    3 years ago on Introduction

    Hi!

    I have some problem. It doesn't work for me. I think the reason is, the operation system of my computer is not english, and at my language windows 7 doesn't have speech recognizer. What do you think? Is there any solution for that, or should I somehow change the language of my op system?

    0
    None
    RohitA1

    3 years ago

    Is their any method through which we can start computer by using smartphone with using ur program

    0
    None
    ArendG

    3 years ago on Introduction

    Hi m8

    i was looking around for a good jarvis free maby i found it here, NOT TESTED YET
    me just learn coding after playing games 30 years straight.
    and i notiched u got a female voice, probaly standaard windows speech to text.
    If u want the real jarvis u must buy or get a good cracked, IVONA 2 Speech, British Brian Voice, That wil install in your windows speech and recognition . u can change there the voice to Brain/Jarvis ;-)

    0
    None
    jemson.pagalan

    3 years ago on Step 5

    hi i jarvis version 1.0, i like much than the latest version, but i want to know, is it possible to control programs using jarvis,?

    1 reply

    Sometimes. It only works for programs that use or support keyboard shortcuts. Find the keyboard shortcut for the action you want to perform, and program jarvis to execute that shortcut on demand. An alternative, and somewhat finnicky method is to use xy commands. If you know the xy position of a particular button on the program, you can tell jarvis to move the cursor to that position and click, however this relies on the window being in the same place every time.

    0
    None
    AnhN3

    3 years ago on Introduction

    i want to set command shutdown computer and want it ask me "are you sure" to i can answer yes or no but already used the "close window" command but it doesn't work please write of me that command

    2 replies
    0
    None
    OptimumAllianceAnhN3

    Reply 3 years ago on Introduction

    If you want to shut down your pc you can create a batch file to actually shut down the pc and link a voice command to that file, here's a website for creating the batch file: http://www.robvanderwoude.com/shutdown.php

    0
    None
    AnhN3OptimumAlliance

    Reply 3 years ago on Introduction

    thanks but if i want to play a song i want in window media player how can i create a command that when i said "play music" it ask me "which song" and i can tell it the song name i want