This is an easy USB charger because you can get the electronic booster board already assembled. Just add the batteries, a couple of capacitors, the usb connector and nothing else.

Step 1: Materials

The main component is the PTN04050, it takes the 2.4 volts from the batteries and raise it to 5 volts for the USB. You can get it from the Texas Instruments site here.
You will also need:
2 100uf capacitor
1 USB female connector
1 battery holder
Wires

Step 2: Assembly

The assembly is pretty easy, just follow the scheme. Try to not overheat the terminal of the PTN04050 and be careful with the polarity of the USB connector.

Step 3: Finish

Find a nice case for your charger, an Altoids tin would be great, but I can't find one of that in my country so I will keep look for a case.
For testing the device at first I used my USB tester and then I tried it with my mp3 player, but there shouldn't be any problem if I were careful with the assembly.