Step 4: Internal clean up
Speaker boxes are held together with goopy epoxy-like glues and their insides are never intended to be viewed by anyone, so you'll probably find a lot of glue globs and what not stuck to the box's inner side walls. Take a chisel or knife or whatever and scrape that stuff off, and get rid of any other stuff you might find in there while you're at it (for structural stability purposes, a lot of speakers have little wooden blocks glued into their corners on the inside).
If any chunks of wood have come out with the glue globs, or if you had a missed cut or two when you were slicing out your door, fill in those errors with either some wood filler or some epoxy putty (I recommend J&B Waterweld, as it's the best stuff on the market). After that, sand the inside of your box until you're pleased with its overall smoothness. For the record, I was chasing after Billy Dee Williams level smoothness, so I had to sand mine for a LONG time.