Just don't go jamming it in! Figure out what lead is the cathode (-) and anode (+). The anode is the longer of the two leads. You want this on the top, so it lines up best with the positive power rail.
Use a dab of hot-glue to hold the LED in the hole. If you used a 3/16" drill bit, it may be tight. You might have to redrill it out by wiggling the drill bit side-to-side to open up the hole diameter a bit.
NOTE: If and when you do the LED on the other side, the polarities will reverse because you will be approaching the power rails from the other end. Make sure you keep your anode (+) and cathode (-) / red / blue rails sorted out!