Here's what to do about that! Make a fitted hat with a fur or velvet inside. It's about the fastest hat there is, much faster than making a Hex Hat or even knitting a toque.
Step 1: Cut the sides and crown
Measure your head at its largest circumference. We will call this measurement H. From each of your two fabrics, cut a rectangle that is H plus 1.5 inches long, and 6 or 7 inches wide (depending on how big your head is). These will be the sides of the hat.
For the crown, cut an oval shape that's H + 2 inches around, or thereabouts (don't go much smaller than this). I made a paper pattern by folding a piece of paper in quarters, then cutting a line about (H + 2) / 4 inches long (see pic). Cut this out of your outer fabric.
Step 2: Construct the sides
Turn one right-side out and place it inside the other so the right sides are together. It's tidy but not essential to match up the seams. Pin if you like, and stitch around the long edge.
Turn this right-side out now, so the seam is folded to the inside. I find it helpful to stitch the other long edge together at this point. This time the wrong sides are together.