Hi there everyone

I am in a process of building a PVC bike trailer out of 40 mm PN 20 type of pipe.

At the neck of the trailer where the cross-shaped yellow piece swivels horizontally - permitting the trailer turn left and right - , and just under it, where the lower red elbows swivel vertically - permitting the trailer to move up and down - are the two white pipes that i want to shave 1 or a bit more mm off of.

I bought the materials but in a very interesting way the pipes don't fit inside the fittings. They need to be warmed up and then they fit inside one another getting stuck forever. No glue needed. 

If i buy  the next largest fitting size - 50 mm - the gap is just too big. So it's a no go.

That's why I ask you people: How do i shave off 1-2 mm of the white pipes in the picture so they slide in easily in the fittings? I want to point out that unfortunately I don't have access  to any heavy-duty machinery. So a manual way of doing this would be much appreciated. 

P.S. the red dots on the picture are fixed points or dabs of glue if you like. The green dots mark out the movements of the components. For example the yellow cross is glued to the green pipes witch are glued to the red elbows witch in turn are glued to the other green pipes. Here only the yellow cross swivels on the vertical white pipe. :) I hope you got it :)