Fork the Zombies! an Interactive Halloween Game

Trick or Treating is Back! In this Instructable, I'll provide all the ingredients you need to create your own interactive Halloween-themed game. If you've seen any of my previous instructables, you know that we're going to combine an interesting physical controller with a simple concept video game.

Supplies

Projector: Really anything will do, but nice bright projectors (800+ lumens) will allow you to start playing a bit earlier. I used an Asus P3B, which has an integrated speaker good enough to handle the sounds and music. This projector is 720P, which was plenty in terms of resolution.

Jetson Nano (x2). This is an embedded development board made by Nvidia. Every hear of the Raspberry Pi? You could actually sub a Pi in for either of these if you have a bunch of Pi's laying around. Alternately, you could run the whole thing off of a laptop, but it's just not as portable. I'm using one Nano to detect the laser, and one to run the game.

Visible Laser Module: Make sure to specify "Class 1" red laser. Class 1 is designated as eye-safe. We'll be incorporating this into our Forkbow. Also look for a module that runs on 5V, because that will be the operating voltage of our forkbow.

Wifi Router- Nothing special, I had an old one laying around that I dedicated to the game.

ESP32- I used the AIthinker ESP32cam. It costs about $10, and I didn't even use the camera part. This part has to be portable, and it wants to run on 5V so you'll need a battery for this. Know what works great? One of those USB phone chargers. A cheap option is: https://www.amazon.com/BABAKA-Portable-10000mAh-Charging-Compatible/dp/B09C1VK5VT/ref=sr_1_19?dchild=1&keywords=USB+battery+pack&qid=1635955215&sr=8-19

There's a few more things, like a surface to project your game onto (in a pinch you can hang up a bedsheet) and it's nice to set up a barrier like a table that the kids shouldn't cross. BUT- these are things that I figure most people have already.

Now- this looks like a crazy mishmash of parts and you may be thinking "OK, maybe not". I can tell you, this whole thing went from concept to finished in 3 weeks of spare time. It's NOT BAD and if you're building something like this, you've got a terrific community here to help you out.

Step 1: Concept and Rules

In the words of Duke Ellington: "I don't need time, what I need is a deadline." I don't know how the rest of you work, but this is completely me.

For this project, my youngest son (he's 13 now) and I were discussing a cool project that we could add to our Halloween decorations this year. Since our house tends to be zombie-themed, we decided to remain with it. It probably only took us a few minutes to flesh out the entire concept. The key element was forks- my son's idea, and we came up with some rules of how zombies are allergic to forks, and we just needed a way to fling em out there.

Now we've got a concept. Only a few things to work out:

The game really needs to last like 1 minute or less, as this is a party game and we need to keep the line moving
It's got to look cool, so people are interested when they amble up for some candy
We need some awesome physical hardware to play the game.

Keeping those things in mind, here's the separate pieces that needed development. We'll cover each of these below:

The "Video Game" part, which is a herd of ever-increasing zombies headed your way
The "ForkBow" part, the controls that you use to keep the zombies at bay
Hit detect, how do we know WHERE the ForkBow is pointing, and when the trigger is pulled?
Integration- pulling it all together in time.

Step 2: The Video Game

First, let me tell you this was a ton of fun. It's also definitely NOT my major jam, so I was learning a lot the whole way. All of my development was done on a linux laptop, and I chose a 3D engine called "Urho3D" In addition to this entirely free and open source engine, you can find a terrific and supportive community interested in helping you succeed. Interested? Here they are: https://urho3d.io/

Now, Urho3D comes as a source distribution, BUT you can compile it with free tools for just about any platform. As a stroke of ultimate cool, they provide 54 example projects that show you how to implement most of what the engine can do. I started with example 6 "Skeletal Animation". In fact, you can try out this example in webgl here: https://urho3d.io/samples/06_SkeletalAnimation.html

This demo has most of what I need- animated models, walking forward. All I really need to do is substitute in some zombies and I'm most of the way there!

But there's a problem- though I can model static objects in Blender, I never learned to animate a model. With 3 weeks left, I wasn't going to get there in time. Fortunately, there's an entire marketplace of this kind of thing available, and I found a good set of zombie models available with animations for a reasonable price. Here's the one I used: https://www.turbosquid.com/3d-models/zombie-set-animations-25-3d-model-1250738

The model comes in both 'FBX' and 3DS MAX format, but Urho3D requires its own 'mdl' format. From here' Urho3D includes an "AssetImporter". If you just type 'AssetImporter' it gives you instructions to use- and you can change the format for both model and animations to the Urho mdl and ani. This worked great! And soon I had zombies shambling around instead of the warrior in the original example. One Note: when you import a model, you _may_ have to experiment with the model's scale and rotation, these are not guaranteed to be consistent. These factors are called out clearly in the example code, and easy to change.

So I added a format to the game. Your first shot *starts* the game, and zombies begin to spawn at an ever-increasing rate. To control the pace and length of the game, I have the zombies increase in frequency as the game continues. You can only launch 2 forks per second and even if you're careful, you're going to be overwhelmed in about a minute. Best you can do is take as many zombies with you as possible.

The obvious missing piece that I need is some sort of physics, or "hit detection". Urho3D has support for Bullet Physics built right in. Cue the appropriate demo!

https://urho3d.io/samples/11_Physics.html

This demo had it all- it launches an object wherever your camera is pointing, allows for gravity, and lets the chips fall (literally) where they may. I made a few changes- instead of launching a box, I changed it to a fork, and I made it fire wherever you happened to have the mouse pointer, and gave it a speed so it would drop like it was flung from a, you know, fork crossbow. OK, here's how the game came out:

Quick note: If you're looking for the source, no problem! There's a link in the last step.

Step 3: What Are You Pointed At?

I mentioned that I like to use interesting physical controls in a game. We'll get to that in a second, but before we do, let's talk about how we're going to make it work- where exactly, are you pointing? Since you've already figured out I'm using a laser to aim our ForkBow, why don't we put a webcam out there to track it? To filter out most of the non-red light that we don't need, I found a bit of red filter material and cut a small piece to put in front of the camera while we're running. This made it WAY easier to ignore everything else on the screen, focusing only on that red laser dot (if it exists). I mentioned that I'm using the Jetson Nano (specifically the 2GB version) with a webcam to pick up these images, then determine the location of the red spot. You could also use a laptop with a built in webcam. I'll provide the python source code here, but if you're curious about what's going on, try this tutorial which supplies a great example of color filtering:

https://pythonprogramming.net/color-filter-python-opencv-tutorial/

Now that we've got a picture of our screen (and red spot) let's figure out where it's actually located! We do that two ways:

We calibrate our image by projecting a known size checkerboard, and locate all of the inside corners on video. Guess what- there's an openCV function for that in Python, if you want to take a deeper dive you can do so here:https://docs.opencv.org/3.4.15/dc/dbb/tutorial_py_calibration.html We now have a series of mapped coordinates on the projection screen. We turn that into a 'Homography Map' which we can use to account for misalignment, pincushion, keystoning, and just about any other aberration that shows up in our image. Oh yeah- now the location of our dot is absolute! Let's have a quick video demonstration:

Next, since the only thing left in the image (after color filtering) is the dot, we go looking for contours. There should only be one, and the centroid should be the center of it.

All good so far. If there's a dot, we know where it is. BUT, having a spot may just mean you're aiming! To complete the game, you need to tell it to fling a fork! Let's head to the next step for that.

Step 4: The ForkBow

It was 3 days before Halloween, and while I've tested with a laser pointer, I STILL don't have a cool control mechanism to fling forks at zombies. Enter the ForkBow. I started with a 30" long scrap 2x4, and used a band saw to hand cut the general shape. Normally, I measure EVERYTHING twice and do 1/2 hour of calculations before making a cut, so this was good for me. After I had a general shape, I used a belt sander to knock off the corners and splinters. So far I'm in about 5 minutes. From there, I drilled a hole in the front to match the diameter of my laser module, cut a small trench in the top to run power wires to the laser, and drilled a few extra holes to route the wiring. I then hot-glued the laser and wires in place. From there, I designed a few pockets to hold the ESP-32 (more detail on this in a bit) and the batteries. I cut those pockets with my CNC. Please note- I built a CNC last year, and so now I use it for everything. Cutting pockets? CNC. Making a wooden marble run? CNC. Got to sign a bunch of documents? Scan your sig, strap a pen in, and do it with the CNC. The important thing is you don't need a CNC to do this project, you can get by just fine with a hand drill and a wood chisel. In spite of the fact that the whole Forkbow took like an hour to make, it generated more questions on Halloween than anything else: "Did you make that crossbow yourself? How?"

For the trigger well, I just drilled a couple of 3/4" holes with a hand drill, and chiseled the walls flat. The trigger itself is a microswitch, but it has a 3D printed shell over top. Imagine a piece of U-channel with a hole drilled through it, and you've got the idea. The microswitch itself is connected from ground to GPIO0 on the ESP32. Looking to get started on the ESP32? Great! I'd recommend starting over here: https://randomnerdtutorials.com/program-upload-code-esp32-cam/ where Sara and Rui Santos will walk you through basics, then more advanced use of these wonderful boards. You can also find some great instructables!

I covered the ESP32 pocket with a chunk of polycarbonate that I had laying in the shop. The other pocket contained the batteries. Though I didn't need to change them all night, I did leave that pocket open just in case.

For hardware- that was it. Plugged a battery in at the beginning of the evening, and it ran great. Every time you hit the trigger, the ESP32 (connected to the same wifi router as everything else) would generate a UDP packet towards the hit detector. Why the hit detector? To keep traffic at a minimum. The hit detector is always hit detecting, but doesn't report to the game until it sees this UDP packet. I'll provide the source (Python) and a block diagram in the "Integration" step.

Speaking of source, if you're feeling a bit curious, the source of the ESP32 code can be found here (not- it's arduino based!) https://github.com/pilesofspam/fork-the-zombies/tree/main/lasergun

Step 5: Integration!

Now we've got a game, a way to locate our laser spot, and a unique control mechanism. Let's put it all together!

I put together a diagram above to show how things go together. The Nanos connect by wire to a router, one has a webcam, the other has the projector. Your ForkBow is wireless (note in the source code your wifi SSID and password). How do these things communicate? Well, the ForkBow is old-school, it's just firing a UDP packet (check out the code! I promise it's short) but everything else needs to provide a myriad of information. To relay this information, I chose a library called "ZeroMQ". This library is cross platform, brokerless, and available in just about any language you could imagine. Just getting started? I recommend the Python bindings. To get you going, check it out here.

A couple of tricks to make it all go:

1. It takes about 1/4 second before a laser spot becomes a known point on the screen, so the laser detect is running continuously. That way we've got a spot ready when your player pulls the trigger. The only drawback is that you may fire a bit behind if you're in motion when you fire, but nobody seemed to notice.

2. Location: All laser locations are normalized. Remember the checkerboard we saw earlier? When that gets projected full screen, we now have calibration points. From the bottom left of the checkerboard, your X,Y location is 0,0. Proceeding to the top right of the checkerboard, your location is 1,1, and every point in between is a fraction of that. This means that the resolution of your camera (or projector) can change, and the game will still act the same way.

2. SOUNDS- Urho3D allows for 3D sound, which can handle stereo and distance. I took the simple way out and played all sounds out of both speakers at a fixed volume. I chose sounds from freesound.org and I have a list at the bottom here to properly attribute all sound authors.

Step 6: So, How Did It Go?

We had a blast. The image at the top of this step features a gaggle of fairies playing (gaggle? group? herd? anyway...). Pretty much every group of kids that came by gave it at least ONE shot, many stayed for a few plays, just recycling through the line. Most of the accompanying adults gave it a shot too. To keep things clean we kept the groups separated, and made hand sanitizer available to all visitors.

The batteries lasted all night without needing a change, and nothing ever broke or stopped working. We did have a few isolated incidents of "it's MY turn" but the game moved fast enough to prevent this. While I took a bunch of pictures, one thing I never did do was get a video of the game playing, so I'll get one of my older son now:

Step 7: The Source

The Source for this project can be found here:

https://github.com/pilesofspam/fork-the-zombies

To build the game, first install the Urho3D source code from: https://github.com/urho3d/Urho3D

Then apply the above (fork-the zombies) code to Samples/06-Skeletal animation. Make sure that you have the ZMQ development library installed for your platform (Linux, Mac, WIndows, even Raspberry Pi)

The laser detect code can be found in pretty much the same place: https://github.com/pilesofspam/fork-the-zombies/tr...

And you can find the ESP32 code (for your ForkBow) here: https://github.com/pilesofspam/fork-the-zombies/tr...

For this last one, you'll first need to set up the Arduino environment available from https://www.arduino.cc/

All of the code linked in GitHub above is released MIT licence.