You want to create a path for the lighter fluid to get to the adhesive underneath the label so it can dissolve it. Some labels are porous and the fluid will soak through. Drop some onto it to see if it does. If it doesn't soak it up, do the following:
Use your piece of sandpaper and CAREFULLY scuff the label without getting too close to the outer edge of the label (you risk scratching what's underneath the label if you do and it's usually not necessary as the fluid will seep underneath the label perimeter from the outside edge). Also be careful not to scratch too deep and tear right through the label, you're only trying to break through the nonporous layer enough for the fluid to seep through.