As reactive as Lithium batteries are all you really need to be safe is to have a circuit that controls the voltage going to your battery.
While you could make your own charge controlling circuit, it's far easier to just buy one that you know works. For this you have several options.
Adafruit is now on it's second generation of Lithium Charge Controllers that have several options for power input. They're very nice, but a bit on the large size. Not at all small enough for our Altoids tin sized enclosure.
I sell some small Lithium Charge Controller Modules on my site. These are the ones I use in my kit and in this guide. I like them because they're small, simple, and have status LEDs for charging and when the battery is full. Like the Adafruit charger you can also use the USB port on it to charge up the Lithium Battery if you lack sun. (Something that is very handy for any solar charger.)
No matter what you use, be sure that you know how to use it and what goes where.