Do some moustache research.
I went with a handlebar moustache because I love the shape - they're very graphic and well recognized!
You could do a Sam Elliot, a Zappa, a Hiltler or Kitler, a Fu Manchu, a Jamie Hyneman... the list goes on and on!
Think long and hard about your moustache. Attain moustache nirvana before proceeding.