DRCURZON did an awesome job describing how to build a complete Raspberry Pi Web Server to run on your home network. This will take you far beyond that to a pi hosted site with world-class performance for a worldwide audience. These instructions use a high performance web server called Nginx, then we front end it with a FREE Content Delivery Network (CDN) provider who mirrors your content to deliver awesome global performance (nearly 30 data centers today & growing) while also providing world-class security & Distributed Denial of Service (DDoS) protection. This ensures that your Pi's content can be seen by thousands of people, all at the same time, without any of them ever actually touching your home server. This will give your site blazing performance & security, all while running on your little $59 Raspberry Pi. Also since we're using Nginx we'll show how to easily host several websites on the same Pi & configure the CDN to support this.
I chose to order my parts from an electronics warehouse, Element14. I've ordered several Pis over the past year & numerous other toys to plug into it so if you want to join this community, this is the way to go. Once on their emailing list you get a real feel for new projects.
Note since I bought my last Pi earlier this year they've introduced the model B+ which has a few minor enhancements as outlined above so let's order that, click on the links below to buy the parts:
I'm going to assume you have a network cable, USB keyboard & a display with an HDMI cable.
There are kits that come with all the above parts, and some are even are pre-assembled, but that takes all the fun out of it.
*Note the SD card referenced above comes with a pre-installed version of Linux for the Pi if you would like to take the easy route this would be it. One reviewer commented that this release is a bit behind. You could save a buck or two perhaps using another SD card, or you might also want a bigger one, this is for you to decide. I would opt for picking up a bigger SD card from some high volume site like Amazon, perhaps this 16GB one for $9.36, although they have a nice 64GB for $30, but that will blow our budget.