Interactive LED Dome With Fadecandy, Processing and Kinect

What

When in Dome is a 4.2m geodesic dome covered with 4378 LEDs. The LEDs are all individually mapped and addressable. They are controlled by Fadecandy and Processing on a Windows desktop. A Kinect is attached to one of the struts of the dome, so movement inside the dome can be tracked and people can interact with the lights.

Why

I'm exploring group experience through shared interaction. I like to make interfaces that lots of people can use at one time. The LED surface of the dome makes a suitable output for a multi-user interface because it's huge, so lots of people can see it. The dome also creates a cosy, spherical space, which encourages people to turn in towards each other. The Kinect works great as a multi-user input because people can move around and affect the depth field at the same time, the only limit is how many people can fit into the space together.

I'm continuously developing new methods of interaction for When in Dome, to see what effect different input methods have, and what works well for different groups of people. I'm particularly keen to see what interfaces encourage connection amongst the friends and strangers inside the dome, and what makes the shared experience feel meaningful and worthwhile.

Where

I designed and built When in Dome as the final project for my Master's, which was Design for Performance and Interaction at the Interactive Architecture Lab, The Bartlett, UCL.

How

Some of the technologies and tools used:

Fadecandy
Processing
Kinect (I got mine from eBay)

Some of the equipment used:

Buildwithhubs hub kit
Treated sawn timber for dome struts
4mm poplar plywood
White diffusion fabric
Black fabric
Aluminium flat bar
Wago connectors
12awg and 24awg cable
5v 30A power supplies
Foam flooring
Jst connectors
Capacitors

Let's go!

There are lots of components to this project which I will talk through, I hope you find something useful and interesting inside!

Step 1: The Dome

Hub Kit

I decided to built my dome with a kit from buildwithhubs and I would definitely recommend them.

They sell kits of connectors and give advice about what materials to purchase for the struts. Most of their domes are used in people's own gardens, whereas mine will be in public, so I also bought their kit of extra secure caps, which keep the struts from popping out if someone leans on the dome.

Size

My dome is 4.2m diameter. I chose this size because it meant that the longest side of the triangles making up the dome would be 1.2m, and this fit nicely into plywood sheets when I came to make the panels to hold the LEDs.

Step 2: Make the Struts

Lengths

I used buildwithhubs' strut calculator to work out the lengths needed to make a 4.2m dome. 30 "shorts" at 1059mm and 35 "longs" at 1209mm.

Material

2 packs of 24 19mm x 38mm x 2400mm sawn timber from B&Q (as recommended on the buildwithhubs site) are enough for one dome. This works well enough but if I was doing it again I would get something that has more lateral strength.

Process

The struts were cut to length using a mitre saw and then I painted them by laying them all out flat on a dust sheet and roller-ing over them. This made a fun timelapse!

Then I bundled them together into batches of 6 at a time and screwed the connector pieces into the ends.

Step 3: Build the Dome

Once the struts are made, building the dome is super easy. I'm not going to talk through the process in detail as there are instructions on the buildwithhubs site and they also provide a booklet.

Step 4: Raise It Up!

I didn't want the LED panels to be right by the floor, as this would mean that a lot of them would be blocked by people in the dome. I also wanted to make the dome higher so it feels more spacious and welcoming.

Legs

I made 50cm tall legs out of 2x4s, and screwed the same connectors into them as the struts.

Then, to increase the strength and structural integrity of the base, I used aluminium flat bar to create X's between each leg section.

The door

I removed one of the horizontal struts to make a doorway, and replaced it with a piece of plywood on the floor to keep the legs at the correct spacing.

Step 5: Plan How the LEDs Should Look

Software

I used SketchUp for my 3D planning work because it is available to use free in an in-browser app. Luckily (as I'm not a 3D modelling expert) I found a geodesic dome model available in the 3D warehouse, where there are loads of models for free.

How many LEDs?

The layout had to take into consideration aesthetics but also power and data distribution. I decided to use 11 Fadecandys (and 11 power supplies) to cover 33 of the triangles of the dome. This would mean that the Fadecandys (and power supplies) would drive 3 triangles each, and that one side of the dome could be open so people could see in from the outside.

This gave me a maximum of 512 LEDs per 3 triangles, as each Fadecandy can drive 8 strips of up to 64 each.

Deciding on a layout

Not all triangles are created equal! My dome is a 2V style, which means it has two types of triangles, equilaterals and isosceles.

I came up with four different potential layouts for the LEDs and took to Instagram to ask people which they liked the best. Style 1 and style 3 seemed to come out on top. Style 3 was my favourite but the concentric triangles in style 3 actually require lots more LED strip than the striped layout, so I decided on style 1. This means there are 8 equilateral triangles with concentric triangle LED layout, and 25 isosceles triangles with striped LED layout.

Since the equilateral triangles are bigger and contain the concentric layout, they use a lot more LEDs each than the isosceles triangles. Therefore I had to split the equilaterals across the Fadecandys.

8 of the Fadecandys control 1 equilateral and 2 isosceles triangles each.
3 of the Fadecandys control 3 isosceles triangles each.

Step 6: More About the LED Layout

With the general layout decided, I needed to work out exactly how many LEDs I would put on each panel. I did this using a combination of a spreadsheet to work out the best way to maximise the Fadecandy's capabilities, and to-scale drawings in Illustrator, so I could see how the layout would look.

Maximising the Fadecandy's capability: Lines and Strips

I mentioned earlier that each Fadecandy can drive up to 8 strips of 64 pixels each. My triangles contain many lines of various pixel lengths, some lines with only very few pixels.

If I was to treat each one of those lines as a strip, I would be losing a LOT of the Fadecandy's capability.

Conversely if I wanted to completely max out the Fadecandy's capability and have 64 LEDs on each strip, I'd need to have some strips that that started in the middle of a line, and that will be confusing to map later.

I had to figure out how best to join the lines together into strips to max out strip capability as much as possible, without splitting lines.

In the end...

Equilateral panels have four strips, made up of:

30, 30 (60 total - red on attached image)
30, 22 (52 total - orange on attached image)
22, 22, 14 (58 total - yellow on attached image)
14, 14, 6, 6, 6 (46 total - green on attached image)

Isosceles panels have two strips, made up of:

23, 28 (51 total - blue on attached image)
3, 7, 11, 15, 19 (55 total - purple on attached image)

Step 7: Lay Out the Fadecandys and Power Supplies

This image shows a flattened view of the surface of the dome.

The LED panels

Each triangle panel has been labelled with a number 1-11, which refers to the Fadecandy that controls it. Each Fadecandy has three triangles, so the triangles also have a letter, A-C.

Other elements

The green boxes show the location of the Fadecandys. Each Fadecandy is mounted on a small panel which also distributes the power, I'll show this in detail in a few steps time.

The purple boxes show the USB hubs. The Fadecandys are connected to a Windows desktop, via these hubs.

The blue boxes show the location of the power supplies, which sit in 3 dryboxes, on the floor around the dome.

Just to make it slightly more complicated

If you compare the location of FC10 and FC11, you'll notice that FC10 is closest to the longest line of its isosceles panels, while FC11 is closest to the shortest line.

Also, if you look at 10C you'll see that the Fadecandy is to the right of it, whereas 10A it's to the left.

I had to take these variations into account when considering how much cable each led strip needed at the start, and when mapping them.

Step 8: Turning Lines Into Strips

This spreadsheet was to figure out how much cable needed to go at the start of every section of LED strip.

How much cable is needed?

Some lines are labelled "jst" which means they are the start of the strip and just need a JST connector.

Some strips have "jst" and a length, which means the strip starts some distance away from the Fadecandy (as we saw in the layout in the previous step), and needs that length of cable to reach it before adding the JST connector.

Some strips just have a length, which means they need to be joined on to the section of strip before them by that length of cable.

Step 9: Preparing the LED Strip

The LED strip

I'm using ws2812b style LED strip, which has three inputs, 5V power, ground and data. Using 3-pin female JST connectors allows me to connect to each of these pins individually. The male counterparts of the JST connectors will supply the power and data.

Soldering

Using my spreadsheet from the previous step, I cut all the LED strip into the required lengths, soldered on the required lengths of cable and the JST connectors. I also put a capacitor at the start of every strip, this is to avoid any peaks in initial current from ruining the first pixel in the strip. (I have had this happen before in previous projects where I didn't add the capacitor, so it's definitely worth doing.)

Sealing

I added some RTV silicone to the exposed part of the strip, covered it in clear heat shrink and blasted it with a heat gun to reseal the waterproof-ness.

Step 10: Make the Panels

Material

I decided to use 4mm poplar plywood to make the panels. I kept it thin to reduce weight. I worked out the total weight of the plywood and contacted buildwithhubs to check whether I was within the weight allowances for hanging stuff off the dome structure. Since the weight is distributed fairly evenly across the dome, it is fine. I would have loved to use acrylic but unfortunately it was out of budget for me for this project.

LED strip attachment

I didn't want to glue the LED strip directly to the panels as I'd like to be able to replace sections of faulty strip, and potentially reuse all the strip at some point, so I decided to make holes in the panels to use cable ties. The dots on the attached image show the layout of the cable tie holes.

Cutting out the panels

There are 33 triangles in total, and I they fit into 9 sheets of 2440 x 1220mm plywood via the layout you see in the attached image.

In an ideal world I would have popped each of the 9 sheets of ply directly into a laser cutter and cut the triangles and the cable tie holes at the same time. Sadly we live in a world where 2440 x 1220mm laser cutters are rare, so the triangles had to be cut out using a track saw.

Sadder still, we also don't live in a world where even a single one of my triangle panels would fit into the laser cutter at school, so I had to laser cut a template of half of each of the triangle designs, and use it to drill the holes by hand.

I also painted the back of the triangles, most of them black and then six random silver ones.

Step 11: Cable Tie the LED Strip to the Panels

This was a lot of cable tying! Luckily I had some friends to help.

Cable labels

I labeled each JST connector with a colour coded cable label, to make it easier when it comes to plug it into its Fadecandy. They are rainbow ordered, so for each Fadecandy there is:

Strip 1- Red
Strip 2 - Orange
Strip 3 - Yellow
Strip 4 - Green
Strip 5 - Blue
Strip 6 - Purple
Strip 7 - Grey
Strip 8 - White

Not an exact rainbow but, that's the colours the labels came in and it works!

(Some of the Fadecandys, the ones that only drive 3 isosceles panels, rather than 1 equilateral and 2 isosceles, only use 6 strips.)

Step 12: Hang the Panels in the Dome

My triangular panels are slightly smaller than the gap between the struts, I wanted them to hang freely in space rather than attaching them solidly to the struts.

Hanging method

Each node of the dome has an eye bolt - these don't come as standard but Buildwithhubs does sell them in a pack. These eyebolts are perfect for hanging stuff on (although be careful not to hang too much weight off one node).

I decided to use paracord and small carabiner clips. The cord is looped through two holes in each corner of the panel. The carabiner clips the cord on to the eyebolt. To tighten up the cord and make sure the panel is correctly positioned in the space, I also added a plastic toggle to each one. This means they can be easily clipped on while loose, and then tightened up afterwards to position them in the centre of the space.

I'm very happy with how the carabiner method turned out. It's very satisfying to clip the panels on to the dome, click click click. It's quick and easy to remove them too.

Step 13: Make Power and Data Distribution Panels

So, we've soldered loads of JST connectors to loads of LED strip, but what do they plug into?

Every strip needs to connect to power, ground and data from the Fadecandy. There are 11 connection panels which hold the 11 Fadecandys and distribute power from the 11 power supplies. I laser cut these panels out of leftover 4mm poplar ply. At the side, there are slots for pieces of velcro, which attach the panels neatly to the dome struts.

Power

Each LED uses a 0.06A at full brightness. This means that the total power needed for 4378 pixels to run at full power is ~1.3kW.

However, I have essentially 11 completely separate power circuits. (They are only connected via -ve via the Fadecandy. Do not connect the +ve of separate power supplies as this is dangerous.) Each circuit is only powering a maximum of 428 pixels, a total of 128W, so the current is at a much safer level.

My power supplies are capable of providing 150W each (30A at 5V).

On the connection panel, the power and ground comes in from the power supply at the bottom, it is then connected to wago connectors, which distributes it across 8 male JST connectors.

Data

The Fadecandy is attached to the left of the panel, and the USB cable comes in from the bottom the same as the power cables.

The data cable of the JST connector is soldered to a strip of single header female pins which plugs into the pins of the Fadecandy. One of the ground pins on the Fadecandy is connected to the ground circuit. (The ground pins are all connected to each other, so it is not necessary to connect them all)

Step 14: Fabric Covering

Sewing together the fabric covering was unexpectedly one of the most difficult and time consuming parts of this project. Luckily I had a friend to help!

Layout

On the flattened diagram of the dome you can see that the cover consists of 5 pentagons which are made up of 5 isosceles triangles each, plus 8 equilateral triangles. We made the cover in this order - sewed together the 5 pentagons first, then joined them together with the equilateral triangles.

(The black sections on that diagram are open and uncovered.)

Measuring

We tried to figure out the measurements for the triangles using maths like normal people, but for some reason it kept coming up wrong and not quite fitting the dome, so in the end we used a piece of polycord through the eye bolts of the nodes to measure the size, and then used this polycord triangle as a template. I don't know why using the known measurements of the struts+node gaps kept going wrong, 3D triangles are confusing.

Pentagons

As we made the isosceles triangles and sewed them together into pentagons, we frequently hung it up onto the dome to check everything was lining up. It's attached to the dome using little pieces of elastic which are sewn to the points where the triangles meet.

Joining it together

Once we had the five pentagons made, we started cutting out the equilateral triangles using the same method - polycord through the eye bolts. Once we sewed two of the pentagons together this way, we realised it was not lining up well at all. So, instead, we decided to hang up all the pentagons in the dome, and pin the equilateral triangles to them in place. Then, once it was all pinned, we took it down and my friend sewed it together into one solid piece.

Pinning it this way was a lot of work, much of it with my arms directly above my head often while trying to pin fabric from the outside of the dome, while standing on the inside. Fun!

Labelling

Along the way, we labelled the pieces with a water soluble fabric pen... these things are great as you can write directly on the fabric and then spritz it with water and the ink will disappear (sometimes it takes a few goes, but it does work)

Step 15: Hanging Up the Fabric

The fabric is hung in the dome by lengths of elastic which are sewn on each point, these tie on to the eye bolts at the nodes of the dome.

Tying and untying the elastics is not as quick as clipping on the panels, so I'd like to replace this method with carabiners or some other clip at some point.

Step 16: Attaching the Kinect

In a bold display of confidence in the universe, at no point did I measure the gap between the panels to make sure a Kinect would actually fit. (Please don't tell my tutors)

You can imagine my joy when it fitted like this.

This picture shows a Kinect v2 but I ended up using a Kinect v1 for reasons which I'll get onto later.

It's just attached to the strut using double sided velcro tape.

Step 17: Floor

The floor is made of interlocking EVA foam mats which I got from B&Q. I've used these for two projects now and it is great for indoors. It's very comfy to sit on.

Outdoors at windy festivals like Burning Man it needs to be secured down all over because the wind will get right under it and lift the whole thing up.

Step 18: That's the Build Done... Onto the Code!

Thanks for sticking with me so far. That is all of the physical build DONE.

Now lets discuss the software.

Step 19: Fadecandy Server

Download the software

The Fadecandy software is available here.

Download the entire github and unzip it.

Run the server

Navigate to the 'bin' folder inside the fadecandy stuff you just downloaded.

Click on fcserver.exe.

This will load a cmd window which shows all the connected Fadecandy devices. In this case, there are 11.

Test the LEDs

Navigate to http://127.0.0.1:7890/ to see the Fadecandy server UI. This shows all of the connected devices again, and allows a tiny bit of control.

Clicking on the test pattern drop down allows you to set all the pixels for that Fadecandy to full or half brightness. It's also possible to make the tiny green LED on the fadecandy itself blink by clicking "identify".

Step 20: Configure the Fadecandy Server

Right now the Fadecandys are all loaded in a random order. Earlier I labelled my triangles 1-11 but there's no way for the computer to know which one is which at the moment. To do that, we need to create a config file.

Which Fadecandy is which

Before we can tell the computer what order the Fadecandys are in, we need to figure out which is which ourselves. I did this by using the browser UI to make each section light up, then noting down which one it was and what its serial number is.

The config file

In the config file we list all of the serial numbers, the index pixel they start at and how many pixels they theoretically control. I say theoretically because I will map the pixels as if there are 512 per Fadecandy, even though in fact there are fewer. This just makes it easier as we know that the first pixel of any Fadecandy is always [Fadecandy number * 512].

The Fadecandy doesn't care that actually each one has fewer pixels than the maximum, and we'll take care of it in the Processing code too.

Loading the config file

Now, to start the Fadecandy server, instead of just clicking fcserver.exe, we need to pass this config file to it.

We do this by opening a cmd prompt inside the bin folder and typing

fcserver config.json

This will now load all of the Fadecandys in the right addresses.

config.json
Download

Step 21: Mapping the Pixels

Dymaxion mapping

Buckminster Fuller (who popularised geodesic domes), also created the dymaxion map, which is a representation of earth as if it were on the surface of a icosahedron. It can be folded to be 3D or flattened out to be 2D.

Much in the same way, I'm flattening out the surface of my dome from its 3D shape to a 2D representation, as shown in the attached image. This 2D representation will be mapped into a Processing canvas where everything I draw on the canvas is immediately shown across the LEDs.

Processing

Processing is a visual programming language. Just like how you'd draw a rectangle in Photoshop using the mouse, you can draw a rectangle in Processing by writing code like this:

rect(100,80,10,50);

That would give you a rectangle starting 100 px in, 80 px down, 10 px wide and 50 px tall.

If you're not familiar with Processing, I greatly recommend Daniel Shiffman's tutorials on Youtube which are as entertaining as they are informative.

Drawing lines of LEDs

Processing works with Fadecandy out of the box. There is a function to set out LEDs in lines, by telling it:

the starting index / address of the LEDs in that line
the actual number of pixels in the line
the x,y location of the centre of the line
the spacing between them
the angle of the line

Drawing triangles

I wrote a function for each of my types of triangles (equilaterals and isosceles). I tell it:

the starting index / address of the LEDs in this whole triangle
the centre of the triangle
the angle the whole triangle is at

From this information it writes out the lines of LEDs, using trigonometry to place them correctly on the Processing canvas.

(You might remember many steps back, I pointed out that due to the location of the Fadecandys, some isosceles triangles start at the longest strip and some at the shortest, and some come in from the left and some the right. This means that I actually have four functions for isosceles triangles)

About addresses

When I say index / address, I'm referring to how the Fadecandy addresses the LEDs.

e.g.

On the first Fadecandy, the first strip starts at 0
On the first Fadecandy, the second strip starts at 64 (no matter how many pixels are actually on the first strip)

On the first Fadecandy, the third strip starts at 128 (no matter how many pixels are actually on the first two strips)

...

On the second Fadecandy, the first strip starts at 512 (no matter how many pixels are actually in the first Fadecandy
On the second Fadecandy, the second strip starts at 576 (... you get the idea)

Code

A "blank" version of my dome code is available on github here.

This code contains the mapping described above but no graphics aside from a circle drawn where the mouse is.

n.b The pixels will only render in this code if you have the Fadecandy server running.

Step 22: Kinect Integration

Kinect 1 or 2?

There are two versions of the Kinect. Kinect v1 worked with the Xbox 360, while Kinect v2 worked with the Xbox One (confusingly).

I'm using a Kinect v1. Part of the reason for this is that its very difficult to extend the length of the USB cable on the Kinect v2 due to the amount of data that is being sent. It requires an expensive and hard to find extension cable. Since my Kinect is mounted at the top of the dome, I am unable to connect a Kinect v2 directly to the desktop on the floor. A ridiculous problem to have but, there we are.

Some of my photographs and video show a Kinect v2, this is because I initially had a set up where I had the Kinect v2 connected to a laptop cable tied half way up the dome, which sent information over OSC to the desktop which controls the LEDs. This worked fine for some applications but once I wanted to use the whole depth feed, I couldn't send it all over OSC so I swapped to Kinect v1.

Installation

I'm not going to talk through installing the SDK and getting the correct cables for the Kinect as there are lots of guides on how to do that. I have the SDK v1.8 installed and, inside Processing I'm using the OpenKinect library.

Step 23: Kinect Depth Tracking

Code

My code is available on github here. It's pretty well commented so have a browse!

This is an overview of what the code is doing:

The Kinect depth camera feed is mapped to colour (e.g. far away = red, close = green), and displayed directly over the LEDs. But there's more to it than that.

Firstly, the colour of each pixel in the depth feed dithers back and forth around it's actual hue, to add a shimmering effect.

Secondly, when starting the sketch, clicking the mouse will take a background reading, then only pixels that are closer than that background reading will display. This stops the floor/any cushions/the structure of the dome from showing up.

There is also a function to reset the background reading every x frames, so if people inside the dome are lying still, they will not show up. This means that actual movement stands out, instead of showing the whole globby mass of depthy nonsense. (I will soon replace this with a lerped version, so the background doesn't do such a "hard" reset but, rather, evolves over time)

There is also a background animation that shows clusters of blobs of colour, the amount of clusters is inversely mapped to the amount of action happening in the dome, so if no one is present or they are still, there is a lot of animation. Then it gradually disappears as more movement happens inside inside.