So I opened it up and lo and behold the flash chip wasnt soldered in properly.
Anyway here are some tips on how to open it up, fix and put back together.
Step 1: Opening Up
This holds the pocket clip on.
Now carefully take the front buttons off, via the top. They clip in either side so take a small screwdriver and pry it out. (you can see the little tabs on the picture. They are only on the right side, the bottom case clips into the buttons from the left side)
Next find the little clips on the bottom each end. Use small screwdrivers to push them in then gently slide the inner shell out of the metal casing.
Step 2: Fix
The picture below is the flash memory chip. Notice the glob of solder on the left and the unconnected pins on the right. (The top 6 on the left are all N/C so maybe it doesnt matter? I cleaned them up anyway)
Apply lots of solder to all the pins (totally cover it up) then use wick to clean it off, leaving lovely soldered pins.
Step 3: Put It Back Together
Put the board back in its holder, make sure you put in the power switch knob first.
Slide back in then clip the buttons back.
Screw the clip back then you're done!
Step 4: The Battery
It's a lipo soft pack, they are reasonably commonly available, but I dont have a part number for you sorry.
But like another guy said, you can just buy the one that takes a micro-SD card from dealextreme for 7 bucks. This 2G one only cost me $14.