Tell us about yourself!
A small cap, maybe 0.1 microfarads in parallel with the switch should do the trick, mostly. Can you post a picture of the completed clock? I'd love to see :)
Question: Is supplying the current from the +5V pin unwise? Isn't that a lot of current going through that one pin that could possibly destroy the Arduino's voltage regulator? Go through VIN instead?