Step 1: Push Down and Turn type caps
There are two ways to fix these. The first and easiest is the Thumbtack method. I use that whenever possible. Sometimes the tip of the thumbtack pierces the cap. So for liquids or anything you think might react with this tiny bit of metal, you can use the second "Toothpick" method. It will also work on caps with a metal inner cap. The items with metal inner caps are often caustic, so I don't usually mess with them.
You could cut, or pry, the outer cap off of the inner cap, but it's sometimes really difficult. If you have to cut it off, it's pretty easy to injure yourself.
A cap with both plastic inner and outer caps is really simple to fix.
Just push a flat head thumbtack into the cap. I guess a push pin will work also. I use the flat heads. They are easy, cheap and they don't stick up off the cap surface.