Step 1: Picking out a bass
The first step of picking out a bass is NOT deciding the budget, however if you truly never have played a stringed instrument before a bass you get a Walmart with the amp should be sufficient.
If you have played guitar before, or don't ever plan on buying a new bass, or just have played before, go to a music store. Ask to try out the basses, and they will try to talk you into a bass. Just ignore it, and try out any bass that looks good to you. What you should look for in a bass is whether you will be playing only rock, only jazz, or a little of each. If you only plan on playing rock, I suggest a precision bass, they have a deeper tone. A jazz bass is the opposite, it has a brighter tone. Personally, even though i play mostly rock I love my Jazz bass, it seems more... well... customizable. You will be able to tell the difference even if it is not in the name like Fender. A jazz bass has a narrow neck towards the tuning parts, but it gets wider towards the body of the bass, therefore the strings spread apart. A precision bass is different in that the strings stay more or less the same length apart down the whole neck.
When you are looking at basses, don't look at the price tag, that comes next. If you fell in love with a bass, tell the workers there why and they might have a cheaper version. Also, ask for someone who can play the bass, people who don't will not know much about a bass.
Fretted over Fretless: For a first bass, get a fretted. They are easier and you can slap them and such. For a secondary bass get a fretless if you want. That does not mean don't try a fretless out, they are really fun to play.
Full sized over 3/4: Get a full size, even if you are young. You will grow into it. It will also get your fingers used to reaching.
4 String over 5 or 6 string: For your first bass, get a 4 string. Until you feel you need another string (most of the time the string is lower) stay with a 4 string bass.
24 Fret over 21 Fret: Actually don't worry about that. Any of the two will do. The 24th fret is just another octave.
Active over Passive: Active electronics means you get better sound quality from your bass, mostly more bass. It also takes a battery. However, you can usually shut it off so you can play it like its passive if you want that sound.
Solid over Semi-Hollow/Hollow: GET A SOLID! Although the hollow ones may look cool, once you crank up the volume the air inside will resonate giving it a muddy tone. Unless you are planning on always playing softly, get a solid bass.
Of course, you can go against all those if you truley fall in love with a bass. Just i suggest you pass the bass with someone who plays the bass. They will hopefully tell you if they like it. Also, before you buy it, go to a different music store if you can and try a few basses there, and compare prices.
Strings: Don't worry, unless you are really good at the bass you probably will not notice the difference.
Some other tips: Unless you think you are good at the bass, I suggest you don't crank up the volume on the amp at the store, it is pretty embarrassing to have no idea what to do and just hit random notes, most people will know that you have no idea what you are doing.
Looks are a lot in a bass, only try the basses out you like the looks of. It is no fun having an ugly bass.
Get a hardcase, they are sturdier than softcases and more protective.
Get an bass amp, not a guitar amp.
Used basses ARE good, they will not lose sound quality. They may even gain it. You can also get a great deal on a used bass, I saved more then $500 on my Fender Jazz Bass, (the first picture in the intro)
My suggested price ranges when you do end up buying a new bass:
100-300-Beginner. Keep this bass for a few years until you get good and decide to stick with the bass
300-500-Intermediate. Keep this bass for a while, until you feel the instrument is holding you back.
Above-Good. The store just made some cash, and you have a bass thats really good. You probably don't need a bass that good until you are good.
My suggested price ranges when you do end up buying a used bass:
This is harder, but I suggest 300-500
I got mine for 400, it's a $900 bass. I also got a really good deal on it though, they were having a sale.
Amps: The bigger the better, don't get a used one. The more watts the more punch you will feel when you hit a note as well. As a beginner, you probably only need 20-30. As you get better, trade it in for a bigger amp. You really shouldn't need anything fancy in an amp, no effects or anything, however they are fun sometimes. You're bass should have enough knobs built in to get the sound you want. You could also look into tube amps, or even build one! Build a Tube Amp
For the case, depending on how good your bass is, any old hardcase will do, they range from 90 to a lot more. I got mine for $90
Strings: Your bass will come with strings! Once you get good enough read the Strings step in this instructable.
Also, get a tuner if your amp does not have one built in, and a cord. Just ask the store workers for one and they will give/sell you one. It really does not matter on the type of cord or tuner, anything will work.