I2C is a communication bus designed by Philips, for chips to communicate with each other on a PCB. It is commonly used, however, for connecting sensors, such as the two examples later in this instructable and port expanders, because you can have multiple devices on the same two pins.
Step 1: Install R-Pi Image
There is an easy setup guide on the wiki, just follow it through.
When you have got it installed, run the config tool, and get everything going nicely. In my case, I am running it headless via SSH, which is enabled as default, at firstname.lastname@example.org.X (check on your router to find the IP).