If you've ever tried to glue things to a very rough surface you probably realized most glues don't work very well unless you use a LOT. Sugru to the rescue. Good use for when you have a little left over from another project. Take a cloths pin (or other convenient clip) and use sugru to attach it to that troublesome surface. It sticks quite nicely to the surface and the rougher the better as far as grip goes. And because it dries to a rubber instead of a hard plastic you can't fracture it off the same way.
Thanks to Matthew Ebisu. He's the designer, developer, (and bug tester), and Vancouver Hackspace for the wall