You can also recognize queens who look for a nesting place by their behavior. They will fly from hole to hole (in the ground or in walls) looking for a suitable place for a nest.
Having a nest of bumblebees in your garden is not a bad thing. They are very docile creatures that will ony sting you (yes they can sting!) as a very last form of defence when you attack them yourself. Most of the time, they will try to fly away from you or they will drop on their back on the ground with their stinger pointing at you.
If you are an avid gardener, then you should welcome these animals with open arms. They are one of the best pollinators in the animal kingdom. They actually outperform bees. Firstly because they have longer tongues, so that can get in deeper flowers and secondly because of their ability to carry more pollen. They also can't tell their kin where the best flowers are. So bumblebees will go from flower to flower much more than bees, who home in on the best spots.
Having a nest box near your vegetable patch could increase your crops in a significant way. These boxes can hold between 50 and 600 bumblebees (depending on their species) and they will pollinate your garden for about 8 to 10 weeks.
You can buy commercially bred bumblebees in a box for pollination, but in this Instructable I'll tell you how to build your own box and how to start your first bumblebee population.