Chuck the rocket blank in your wood lathe. I stabilized mine using a live center in the tailstock, and I got the blank as close to centered on the lathe as I could by locating the center of the bottom of the blank.
Move the tool rest as close to your blank as possible. Test for interference by manually turning your blank on the lathe, and adjust the location of the tool rest if the blank hits it.
Using a roughing gouge, turn the blank at a relatively slow speed. You're going to need to stabilize the gouge on the tool rest, as you turn the square cross-section into a rounded one. The different textures of the different woods in the blank may be difficult to manage; some woods are prone to chunking out while you are turning them, and that was the case with my walnut.
Once your blank is rounded over, use whatever lathe tools you want to turn your blank down to approximate your intended shape. In my case, I used a smaller roughing gouge and a round nose scraper to shape the blank. If you are new to wood lathing, you should take a look at this Make article where they discuss the different types of lathe tools. You can also bump the speed up slightly. This also is the time when you'll start to deal with the acrylic rod. It'll have a different texture when you're turning than the other sections of your blank, but it handles just fine.
If you lathe away so much material that you start to see a noticeable gap between the tool rest and the surface of your work, stop the lathe and adjust your tool rest up to the new edge of your blank.