Step 1: Components
There are many types of component that are important when building circuits such as LEDs, capacitors and transistors. All of these components are available at RadioShack or other stores and they are key components for a beginner.
555 Timer - A type of IC (Integrated Circuit) used to provide time delays, such as an oscillator.
555 Timer @ RadioShack
Batteries - Devices used for storing chemical energy for transformation into the electrical energy that powers the circuit.
Batteries @ RadioShack
Capacitors - A Used to store energy in an electric field. They come as electrolytic and ceramic variants and in many different sizes.
Sunrise Alarm Clock
Capacitors @ RadioShack
LEDs - (Light-Emitting Diode) A semiconductor light source.
LEDs @ RadioShack
Resistors - Implements electrical resistance as a circuit element. These are often used with LEDs.
Resistors @ RadioShack
I found this useful when making large LED arrays: http://led.linear1.org/led.wiz
Voltage Regulator - Electrical regulator designed to automatically maintain a constant voltage throughout a circuit. A specific type of transistor.
USB iPhone Charger
5V Voltage Regulator @ RadioShack
Wire - Metal strands used to carry electrical current.
Wire @ RadioShack
Switch - Able to break electrical circuit.
Switches @ RadioShack
Transistor - Semiconductor used to amplify and switch electronic signals.
Transistors @ RadioShack
Step 2: Tools
A fully equipped tool kit is necessary for creating electronic projects. A small bag or container is useful in making your kit fully portable, which enables you to use multiple workstations.
Breadboard/Protoboard - Used for prototyping electronic circuits.
Breadboard @ RadioShack
Soldering Iron + Accessories - An essential tool in constructing circuits. The hot soldering iron melts the solder on the joint between to components to form an electrical joint.
Soldering Iron @ RadioShack
Zip/Cable Ties - Useful for binding cables into a neat form.
Heat Shrink - Provides support and insulation for wiring.
Heat Shrink @ RadioShack
Sugru - Silicone based putty that sticks to objects for multiple purposes and cures over time.
Multimeter - Electronic measuring instrument.
Multimeter @ RadioShack
Work Light - Provides useful light for working on electronics.
Craft Knife - Small, sharp knife. Especially useful for making precise incisions.
Glue Gun - Strong adhesive
Glue Gun @ RadioShack
Multi-tool - Versatile hand tool with multiple functions.
Step 3: Useful Links
Here are some other sites that will give you more information about basic electronics:
Basic Electronics by randofo
How-to Breadboard by amandaghassaei
If you find it too hard or too expensive to buy the individual parts, I would recommend buying a complete project kit which would include everything you need for that particular project as well as assembly instructions.
After learning the basics you can move on to more advanced electronics such as Arduino or Raspberry Pi. Both of these mediums offer easy to learn programming as well as strong communities that are willing to help beginners.