One of the handy properties of felt is that it tends to stick to other felt. It is much weaker than Velcro, but it is strong enough to hold up the weight of small shapes like letters and numbers.
This is a quick project that could even be done without sewing anything (a few safety pins would accomplish the same job, or maybe even fabric glue).
There aren't many in-progress photos, because I temporarily lost the battery charger for the camera... but I believe it is simple enough to follow just looking at the completed project.
Step 1: Materials
Consumables (measurements are approximate):
1 large piece of felt (about 23" wide by 36" long)
1 wooden dowel (1" diameter, 27" long, but you could easily use a different size, or even a coat hanger)
Several smaller pieces of felt in various colors
A bit of yarn (36" long)
Sewing (I used the machine because it was easy, but it wouldn't be too bad to hand sew)
Drill to make a nice hole in the dowel (without a drill, you could just make a notch in the wood to tie the yarn)
Sandpaper (for the dowel)
Note - From other forums, it sounds like wool felt works a bit better than acrylic felt, but I used acrylic here (from the remnants bin at the local craft store), and it worked just fine.
Step 2: Pockets
The first thing I did was to hem the bottom edge, so the tops of the pockets would be rounded and finished-looking. Fold the bottom inch of your big piece of felt forward, and sew one straight stitch all the way across the width of the felt. Trim off the excess felt past your stitch, if you leave too much it seems to make the pockets bulge out.
Fold your hemmed edge up about 5". The rough edge should be concealed in the pocket. Sew the outside edges of your pocket with a straight stitch. I found it easiest to sew from the top down. Now you have one big pocket, 5" deep.
If you like, fold the felt in half to find the midpoint, and sew a divider in the pocket to give you two equal sized pockets. Repeat for each of those pockets, and now you have four equal sized pockets, each 5" deep.
Step 3: Sleeve
I used a drill to bore a hole on either end of the dowel, then sanded the ends a bit to make them a bit more kid-friendly.
Fold 2 or 3" over the dowel so the rough edge of the seam is in back. Use pins to hold the felt in place as you adjust to make sure it will hang level. It's pretty forgiving on that front, so don't stress out about it. All I really did was line up the edges of the sides I brought over the dowel, and it turned out fine.
To sew, just leave the dowel in, and sew a straight stitch all along the side of it, using the dowel itself as a guide.
Tie the yarn through the hole in the dowel after you finish sewing.
Step 4: Accessories
Using the smaller felt pieces, cut up shapes and put them in the pockets or push them against the big piece of felt to get them to stick.
Hang the felt activity board within reach of someone who will get a kick out of sticking letters on, pulling them off, and repeating all afternoon.
[Edit:] It turns out finger puppets work very nicely as dowel-covers, also. And, the result is generally enjoyed by the local populace!
Step 5: Sturdification
If you hang this on a doorknob, the felt can't sit flat against the wall. So, I added a cardboard piece to the back. It is actually two cardboard pieces, but glued together so the corrugation is vertical for one piece, and horizontal for the second piece. That should give it a bit more structural integrity against the pressures of toddlers.
To fit the cardboard in, sew on a long pocket to the bottom, and two triangles to the top, all on the back side.
After a few years of use, it is also clear that the letters have to be wide, or they will stretch and distort. 1" wide at their thinnest point works well, 0.5" stretches.