If you already have the console you want to use, you don't have to buy one. If you don't, though, read on...
Old systems can easily be found in attics, closets, garages, and many other places. See if you can buy one for cheap from a friend. Try yard sales. You can always get stuff for cheap at those.
If you can't find an NES from any of the above sources, then you have to resort to eBay. Try to look for one with a case that's in bad shape. We will be taking the board out, so the condition of the case is not relevant. Also, try to find a seller that is in your country, so shipping will be low. Obviously, make sure the description says that the NES works, and check the member's feedback to make sure they are a reliable seller.
But what about those SuperHappyMegaJoy 72,000 in one things?
Glad you asked. The NES clones, or NOAC's (Nintendo On A Chip) could also be used for portables.
Here is one for $25: Link
. It only takes 60-pin cartridges, though (Famicom), so you'll need to wire a 60 to 72 pin adapter (try some google searches, couldn't find a guide on this one). You could also try a Super Joy 3. They're good for NESp's, but also only take Famicom cartridges.