I've done this repair for lots of people so I figured I'd show how I do it. Sometimes the headphone jack breaks off and there is a long peg left you can grab with very small needle nose pliers. Most of the time the jack breaks off way down in the jack socket so there's nothing left to grab, even if you could get pliers small enough to fit in there. This technique will work even if the jack is broken off all the way at the base.
This procedure is pretty simple and only takes a couple of minutes- but it does require steady hands!
Step 1: Tools
You need a couple of simple tools-
1) A Dremel tool- the smaller rechargeable variable speed model is best but any model should work
2) A 1.0mm hart burr -available here: http://www.cooltools.us/45-Degree-Hart-Bur-1mm-p/drl-324.htm
3) Lots of light (good magnifiers don't hurt either!)
Step 2: Cut a Notch and Remove the Broken Part
Put the burr into your Dremel tool (make sure the burr sticks out far enough to reach all the way into the jack opening) and set your Dremel on its lowest speed. What you want to do is very carefully stick the burr inside the headphone jack socket and cut a small notch into the side of the broken part of the headphone jack. You have to be very careful when doing this as you don't need to cut very far into the broken part of the jack- you only need to cut deep enough so the burr can grab onto the broken part so you can pull it out. The jack opening is really small and it's hard to see in there- you don't want to cut the jack socket and possibly damage your phone.
Turn off your Dremel tool and remove the burr. Turn the phone upside down to remove any metal particles. Now place the burr back into the jack socket and gently feel around until the burr is located in the notch you cut. Apply a little side pressure and carefully pull the burr out and the broken jack part should pop right out with it.
I have yet to see a headphone jack that is broken off lower than the one pictured here so hopefully this works as well for other people as it has for me.
Please let me know if there are any questions- thanks!