You could just go out and buy plants that are already a few months grown, but that can get expensive in a hurry. Like most people I try to do things as cheap as possible. I prefer to think I do things as cheap as reasonable, and the way I will tell you how I do it also recycles some of your waste. It is also cheaper to start with seeds. You could also buy greenhouse kits, but those too can get expensive fast.
I can start my garden for $20 (+tax) and a little time.
-- Seeds, $5
-- Bag of planting dirt, $5
-- Fertilizer, $10
You could save $5 if you used dirt from your yard or garden, but right now everything around me is mud, or frozen mud. its worth the $5 to me to start with some good dirt thats ready to plant. If you planned ahead from last garden season, you may be able to save more money but saving seeds from last years plants. If not, you can always plan to save seeds from this years harvested plants.
Along with those items you need something to put them in. Here is where you can save a lot of money and do some recycling. the big tip is to use empty milk jug. 1 gallon, or 1/2 gallon.
--Knife and or scissors