Here's a simple way of keeping the drive with its cap.
Step 1: The Problem
This is a very simple Instructable to ensure that the cap always comes away with the drive.
Step 2: The Solution.
Use a pair of pliers to cut yourself a 6 inch piece with a joiner on one end.
(Mine is minimum length - Don't make it any shorter.)
If you can't see how the joiner works, stay away from I.Q. tests - You'd only embarrass yourself.
Link the cap and the pendrive with the chain, rejoin it and job done.
Step 3: Much Better
This is a 1GB drive which boots into Vista PE (there's an Instructable about setting this up somewhere) and I use it to run various diagnostic and recovery utilities.