Step 1: Materials
1 Ice cube tray (as mentioned above)
Enough straws to fit in each compartment
a Small bit of clay
Step 2: Preparation
Next, pit a tiny bit of clay into each end of the straw. This will keep water out of the straw for easy removal later.
Push one straw into each compartment such that it sits near the middle of the cube.
Fill with water.
Step 3: Freeze and Finish
Once your tray is frozen, carefully remove each ice straw.
Removing plastic straw
I found the easiest way to remove the straw was to allow the straws to thaw for about 10 minutes. Then push a skewer through the clay and straw. The friction from the clay pushed out the straw far enough that I could grab it.
You don't necessarily have to remove the straw as long as you remove all of the clay. A clean pipe cleaner would work well for this. This way, you'll get a longer service life out of your straw.
Now go enjoy your Sunday Mimosa or afternoon Bloody Mary :)