Replacing (AA or AAA powered) batteries in bicycle lights is a pain (and not good for the environment) and bicycle lights with built-in rechargeable Lithium batteries are expensive. This instructable will show how to convert (AA or AAA powered) bicycle lights to use a USB rechargeable lithium battery pack. These battery packs are intended for use as portable cell phone chargers and they are relatively inexpensive (I found a ~2000 mAh pack for less than $8 on eBay).
Step 1: Add a USB Chord to Your LED Lights
Before modifying your LED lights, you should test that they work OK with a 5V supply. There is no typically no current regulation in a cheap LED bicycle light, so the current in the LEDs will increase with the battery voltage. For example the current for my headlight went from 200mA @4.5V to 250ma @5V which is an acceptable current increase for a headlight (and it's brighter now too). However, the current in my taillight went from 50mA@3V to 200mA @5V which is way too much current for the taillight LEDs. To fix this I added a series 30 ohm resistor to compensate for the higher voltage. After adding the resistor, the taillight current is 60mA @5V (close enough).
As a cheap source of USB chords, I used the wires from a couple of broken wired mice. To modify your lights you just need to open your lights, remove the batteries, solder the USB 5V power wires (these are usually the red+ and black- wires of the 4 USB wires) to the battery terminals and run the wires back to your external battery pack.
PS: Sorry, the pics for this step do not show the added resistor. Add it inline with the red+ wire inside the taillight.
Step 2: Making a USB 1-to-2 Y-Connector
Since I am powering both headlight and taillight from the same battery pack, I needed to make a 1-to-2 Y-connector. I used 2x USB extension cables that I found on eBay for $.70 each. I also added a series pushbutton-toggle switch for disconnecting the battery. This switch is not strictly needed. In fact if you decide to use 2 battery packs, you can skip this step altogether.
Step 3: Completed LED Lighting System
Here are a couple of photos of the completed LED lighting system. I placed the battery in a waterproof bag that I'll strap to my rear rack. An under-seat bag is another possible option to locate the battery.