-CPU (Central Processing Unit, or processor)
The single most important part of your machine. This is the brain of your computer, computing everything from word processing the the physics in your hardcore shooter games.
-RAM (Random Access Memory)
This stores everything that is being used in real-time. When the computer is off, RAM is empty.
For a gaming PC, you will need a discrete (seperate) graphics card. Using the onboard chipsets of motherboards and integrated graphics of processors just won't cut it when slaughering dragons in Skyrim.
This is where the operating system (Windows), your games, music, and videos are stored! You need this (kinda.)
-PSU (Power Supply Unit)
Pretty straight forward, this supplies power to every component in the computer.
This is for your DVDs and CDs
Everything connects to the motherboard in some way. It connects components the way roads connect people.
For recommendations regarding the above parts, I recommend visiting the tomshardware.com forums.
NOTE: This list does not include the OS (Operating System)