Introduction: Controlling Mouse and Keyboard Actions Using Python

This is an instruction on how to automate your tasks that require the use of mouse and keyboard. The instruction uses Python programming language, and its “pyautogui” library to write simple lines of code to control the mouse and keyboard. The instruction is intended for anyone aspiring to learn about writing a simple computer program to experts in programming that do not know about automating tasks with Python. This instruction is only meant to be an introduction to automating your tasks and Python and is not supposed to be a programming tutorial.

What are the benefits of being able to control mouse and keyboard?

  • Useful to automate tasks that you must repeat every time.
  • Used to automate playing simple games and can be added to machine learning projects to mimic the human player.
  • Precision control with accuracy of mouse pointers as well as typing speed can be utilized in various areas such as security, running simulations, and so on.

Step 1: Watch the Video Instead.

This is the same instruction but in video format for easier explanation.

Step 2: Installing Python

We need to install Python to start, to do this go to "https://www.python.org/downloads/". You will see a button that says "Download Python" whichever operating system you have, you can follow the same method to get python. Once python is installed, it will be automatically added to the "path" of your machine. This will be useful when installing Pyautogui library.

Step 3: Installing PyAutoGui Library

Pyautogui library is a libary in Python language that helps us to automate mouse and keyboard actions. To install pyautogui, go to PyAutoGui documentation and you can follow the steps or you can just open your command prompt (or terminal in Mac and Linux) and type in "pip install pyautogui" and press enter.

Step 4: Simple First Program to Move the Mouse Around.

  1. You can use any text editor of your choice or you may use any IDEs (Integrated Development Environment) if you are familiar with programming.
  2. Open your text editor and type as shown in the figure.
  3. Or you can also (in the command prompt) type "python" and press enter, then type the same code.

Explanation of the code line by line:

  • The first line imports the library to be used in the program and denoted with “pt”, an optional step to make it easier to use the library.
  • pt.moveTo(x, y) moves the mouse cursor to the location x and y of the desktop.
    • In computer graphics, the origin is at the top-left of your monitor.
    • X-axis increases towards the right of the origin.
    • Y-axis increases downward from the origin. (opposite of the coordinate system in mathematics.)
  • pt.click() : clicks the left mouse button once.
  • So, the code simply moves the mouse to the coordinate (10, 10) and performs a mouse click.

Step 5: Running the Program

1. Save the file as [filename].py, example: “tutorial.py” (without the quotes). o If your text editor saves with .txt extension by default click on the “Save as type”, change the option to “All Files” or “.py” in place of “.txt”. 2. In windows, navigate to where you have saved your file. Click on the file and you should see “Open Windows PowerShell” or “Open Command Prompt” (shown in Figure 3). Click on either of the options.

3. In Linux or mac, open the terminal and find the “pyautogui_tutorial.py” file and open the directory. 4. Type “python [filename].py” or “python tutorial.py” depending on the filename of the program you saved.

Step 6: Running the Code in Your Command Prompt (terminal)

python tutorial.py (replace tutorial.py with your file name.)

Step 7: There Is So Much More to Do With This Library.

These are some of the example codes that you can use with Pyautogui library. Once you get used to some of these methods, consider combining multiple lines to mimic something you do regularly. For further information read the PyAutoGui documentation at https://pyautogui.readthedocs.io/en/latest/.