Don't put non-rechargable batteries in the holder though, because you will wind up with 6 volts instead of five. The USB socket is supposed to supply five volts.
One nice thing about this setup is that the batteries stick together as a set of four. They get drained together, and they get charged together. Also there are no complicated circuits.
Step 1: Buy a battery holder, batteries, and a charger
You can usually find a battery charger at a thrift store or in the free box at a college dorm building. Often times the charger is missing its power cord, and you can just match it up and be on your way. Buying a charger would be silly with all the free ones out there, ready to go in the trash because nobody wants them.
Most chargers charge as two sets of two (one LED for two batteries) but some charge all four cells as individuals (one LED for each battery). Those are better. Everyone deserves to be treated as an individual.
You can get rechargable batteries in a set of four. Cheap ones are 1500mAh, expensive new ones are almost 3000mAh. Get whatever is handy, and upgrade if you need to. Don't mix batteries that weren't born together, they will end up beating each other up. I am serious.