Once you get the shape very close to what you want, you'll want to sand down the rocket body. This will remove material, and you can count on coarse sandpaper to help you refine the shape of the rocket. During this process, make sure that you don't press down too hard while sanding, or else the rocket will break off of the lathe.
Take an assortment of sandpaper, ranging from coarse grit to fine grit. I used 60, 120, 220, 320, and finally 400.
Turn your lathe up to a higher speed. Starting with a strip of the coarsest grit sandpaper you have, you'll want to hold either side of the strip under the rocket as you sand the full length, smoothing away any of the lines or marks left by your lathe tools. You may have to stop your lathe and check the rocket for a consistent finish.
Once you've got a consistent coarse grit finish, you'll switch to the next finest strip of sandpaper. Use this sandpaper to smooth out any marks left by the coarser sandpaper until you achieve a consistent finish.
Repeat until you get down to the finest grit of sandpaper. It is incredibly important that you get down to a super fine, consistent finish, or else the acrylic won't clear up in the next step.