Its all based on partitions, and filesystems. Windows can only read NTFS, and FAT filesystems, whereas Linux can read NTFS, FAT, Ext2, Ext3 (and soon Ext4), and some others, but I'm not quite so up-to-speed on those.
Basically, we split the thumbdrive into two parts, one FAT (so anyone can read it) and the other Ext3 (so any linux user can read it.) Simple.
Step 1: Ingredients
Thumbdrive (AKA USB flash memory stick, jump-drive, pen-drive, etc) The bigger storage the better.
Puppy linux (a ~100M operating system) head to: http://www.puppylinux.org/ (I used version 4.1.1, but they are pretty similar.) on a live CD.. (I'll get to that later)
An Ubuntu installation, with GParted Installed (this is slightly better)
A working computer with a writeable CD Drive (with power, screen, tasteful desktop background, keyboard mouse, etc..)
Step 2: Download and Install Puppy
Once you have downloaded the Puppy Linux ISO file, and write it to a CD, there is an inbuilt ISO writer in Windows XP, so all you need to do is head to where you saved the ISO, and right click, and you should find 'write to CD' or something like that. Finish going through the ISO Wizards steps, inserting the blank CD on command, and so on..
Step 3: Start up GParted
Click 'Menu>System>GParted' (this may differ from version to version) a dialog window should pop up, asking which drive you want to edit, click the one with the make and version of your thumbdrive, then click OK.
A window should pop up, looking something like this: (without the numbers, and arrows)