Multiple Raspberry PI 3D Scanner

Intro: Multiple Raspberry PI 3D Scanner

UPDATE: Check out step 8 for the latest version of my scanner and a download link for the python scripts.

Hi,

I am a big Arduino and Raspberry PI fan and also love 3D printing. I wanted to be able to make a 3d model of my kids and started investigating how to build a 3d scanner. I found a lot of solutions out there, but the problem with most of them is that the subject would have to sit still for a while... well I think it would be easier for me to invent a spaceship that can fly to mars then inventing a solution for my 2-year old son to sit still :-( So none of those solutions where going to work.

I knew I had to come up with a way to instantly take many images at the same time. So I first started to research what cheap digital cameras exists. If I could find a cheap model, I probably could make an automated trigger system using arduinos. But then I would have all the images on many separate SD cards and I was not able to find a cheap good digital camera.

Then I noticed the Raspberry PI and PI camera combination. A "fairly" affordable module, that already is ethernet connected, so I could do the triggering of the cameras using the network and an easy way to download all the images to a centralized place. So my Project (and investment) started.

I bought for this project:
- 40 Raspberry Pies for this project and 40 PI cameras.
- 40 8Gb SD cards
- 1 single 60A 5v power supply to power all the raspberry Pies
- Led Strips and a powerful 12v power supply to power them on

As I am an impatient person I did not build the whole setup at once, I started of with 12 cameras, and was already seriously impressed with the results. So you DO NOT need 40 cameras, especially not if you just want to catch just the front of a persons face.

Here a result output:

UPDATE:
I have included a photo of Britt (the model in the video) being printed in full color by shapeways.

UPDATE2:

I finally was able to make a scan of my little son Hugo (2years old). It was made using 47 Raspberries and using my new softboxes with permanent lights. This allows me to shoot the images with no shadow. Can't wait to receive the printed model from shapeways :-)

STEP 1: Setting Up the Hardware

So I first needed a rig to hold the Raspberry Pies. I initially did some testing with a big round circle I made out of wood, but this was really impractical to work with and hard to walk in and out of. So after some testing, I went with an "individual pole" design. Most programs that turn images into a 3D model need the images to be shoot from different angles. So I settled for each pole to hold 3 Raspberry Pies cameras.

I made the poles out of fairly cheap multiplex wood using a 2mm cutting bit on my CNC machine. This allowed me to pre-drill 2mm mounting holes for the Raspberry, so I just needed 2.5mm screws that would instantly fix the raspberry to my frame. 

For the PI Camera, I designed a small and easy to print bracket (as I need 40 of them, so it needed to be small) that can hold the camera securely and would easily allow me to change the angle the camera would be pointing at.

To fancy up the poles I also added a 1meter strip of 60 LEDs to each one, to provide some extra light for the photos and just because it looked cool :-)

STEP 2: Connecting Everything Up

Connecting 40 computers with ethernet and power was going to be messy, but I wanted to do it as efficient as possible. Unfortunately the Raspberry PI does not support Power-over-Ethernet, so I had to make this myself. I cut 40 ethernet cables, each 5 meters long. I kept all cables the same length so I know that what ever voltage I would lose over this distance would be equal for all and I would be able to adjust this on the power supply to get a very accurate 5v.

As 100mb ethernet only requires 4 of the 8 cables inside an ethernet cable, I could use 2 for providing the 5v to the raspberry. So I ended up putting 80 (2x 40) connectors on the cables using just 6 of the 8 wires (2 not being used). I would say this was one of the most boring and tedious elements of this project :-(

I bought a bunch of female jumper wires, cut them in half and soldered 2 on the end of each network cable, so I could easily just fit this on the ground and 5v pin of the Raspberry Pi.

On the other side, I build a "power distribution board" from my single 60A 5v power supply to where I could easily connect all the 5v and ground wires to coming from each ethernet cable.

STEP 3: The Software

I am using Raspian OS, just the default download from the raspberry pi website.

To collect all the images, I am using a central file server (in my case I am using a Qnap). I configured the raspbian image to connect to the file server using cifs. This is done in the /etc/fstab file.

I am also using the central file server to store my software, so I can make modifications without having to update every raspberry on its own.

After I completed this image, I used dd (on my mac) to clone the SD card 40x for each raspberry.

I wanted to write a "listening" script that each raspberry would run, listening to a particular network broadcast package that would trigger the camera and then save the photo and copy it to the file server. As I want all the images to be stored in a single directory (one directory per shot), I am using the local IP address of each raspberry (the last 3 digits) for a prefix of the filename.

Here the python listening script I am using:

#!/usr/bin/python
import socket
import struct
import fcntl
import subprocess
import sys

MCAST_GRP = '224.1.1.1'
MCAST_PORT = 5007

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(('', MCAST_PORT))
mreq = struct.pack("4sl", socket.inet_aton(MCAST_GRP), socket.INADDR_ANY)

sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)

def get_ip_address(ifname):
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
return socket.inet_ntoa(fcntl.ioctl(
s.fileno(),
0x8915, # SIOCGIFADDR
struct.pack('256s', ifname[:15])
)[20:24])

id = get_ip_address('eth0')

ip1, ip2, ip3, ip4 = id.split('.')

print 'ID: ' + ip4

#create an options file, this file should containt the parameters for the raspistill image cmd
optionfile = open('/server/options.cfg','r')
options = optionfile.readline()
optionfile.close()
print "optons: " + options

while True:
data = sock.recv(10240)
data = data.strip()
if data == "reboot":
print "rebooting..."
cmd = 'sudo reboot'
pid = subprocess.call(cmd, shell=True)
else:
print "shooting " + data
cmd = 'raspistill -o /tmp/photo.jpg ' + options
pid = subprocess.call(cmd, shell=True)
print "creating directory"
cmd = 'mkdir /server/3dscan/' + data
pid = subprocess.call(cmd, shell=True)
print "copy image"
cmd = 'cp /tmp/photo.jpg /server/3dscan/' + data + "/" + data + "_" + ip4 + '.jpg'
pid = subprocess.call(cmd, shell=True)
print "photo uploaded"

To initiate all the raspberries to take a photo, I created a "send script". That would ask for a name. This name is send to the raspberries to include in the prefix of the filename. So I know who the images are from.

Here the python send script:

import socket
import sys
import time

print 'photo name:'
n = sys.stdin.readline()
n = n.strip('\n')

MCAST_GRP = '224.1.1.1'
MCAST_PORT = 5007

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
sock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, 2)
sock.sendto(n, (MCAST_GRP, MCAST_PORT))

The listening script checks the name received. If the name is reboot, reload or restart it does a special action, instead of shooting a photo.

To configure what options I want to use for raspistill (the default image capture software on the raspberry for the PI camera) I am using an options.cfg file to configure this. Again this is stored on the central file server, so I can easily change the options.

I did some testing to see how in-sync all the Raspberry Pies would take the photo. As they all receive the network broadcast package at the exactly same time, I found this worked great. I did a setup test with 12 units all taking a photo of my iPhone running the stopwatch app. Each photo captured he exact same 1/10th of a second.

STEP 4: Processing the Images to a 3d Model

There are different ways how you can turn the photos into a 3d model. Personally I like autodesk’s recap the best.

Go to recap.autodesk.com. If you do not have an autodesk account yet, you can create one for free. It comes with 5Gb of storage, which is plenty enough to create your 3d model.

After you have logged in to recap, click on the new project button. Here you can select the quality of your 3d model (I always choose Ultra, why settle for lower). For the export format, most 3d printer software can directly work with .obj files, so at least select that one.

After that you can upload all your images by dragging them into your webbrowser, or by clicking you get a popup windows where you can select your photos.

Wait for the photos to be uploaded and click next 2 times. The autodesk cloud system will now start to do the hard work and turn your photos into a 3d model. This can take between 15 and 45 minutes. As this is happening “in the cloud”, you can do other things while waiting  [:-)]

When the heavy computing is done you can click in the middle of the thumbnail (on the eye) to see the model in 3d on your webbrowser. Important: this requires the Chrome webbrowser!

When viewing your 3d model, you should see a thumbnail gallery of your images on the left. If all went well, you should NOT have many (or ideally none) in the list called “Not Stitched”. This would mean that those images where not detected. You can try to fix this by manually clicking marking points on the images. But as I said, hopefully the 3d scanner took good enough photos of you that this is not needed.

You can now download the .OBJ file to your local computer for further processing and clean up, like removing the background. The file you download is again a zip file. So you need to unzip this before moving on to the next step. Cleaning up your 3d model. There are many programs available to clean up your 3d model. The easiest program I have found to do this is also a free product from autodesk called “project memento”. You can download memento (Only for windows!) here: http://labs.autodesk.com/utilities/memento

After you have installed project memento, you can directly open up the downloaded .obj file, it should be called mesh.obj. The photos taken by the 3d scanner are 90degrees turned. This means that your model probably has an incorrect feeling for up and down. In the botton left you will get a message “did the orbiting feel off?”. Click on “feels off” to correct this.

You correct the orbit, click for instance on the top of your head, making sure that the arrow is pointing up. Try rotating the model (by keeping your right mouse button pressed). If it feels good now you can accept the setting.

As you can see, you where not only scanned, but also part of your environment. You probably want to cut those pieces away. This is very easy in memento. Just select with your mouse a region of unwanted stuff and then hit delete. By rotating the model around you should be able to easily select and remove all unwanted stuff. Fix your model for 3d printing. To be able to print your model on your 3d printer, you need to make sure you have a watertight model (no holes in it). Again this can easyly be fixed with memento. Click on the top middle of the memento window and you will see a popup to fix your model.

Just walk thru the wizard and any hole issue it finds, click on fix. You need to select if you want a flat or smooth fix. For me usually the flat fix works best. Select “next” again to find more issues. The holes need to be fixed, everything else is optional.

After you have fixed all the holes, click done and now you can export the model as a new .obj file. You can do this by going to the top left corner of the mement program and click on the bended arrow.

You can now specify how detailed you want to save your model. Again here, why go for lower resolution?? move the slider all the way to the right to get the best quality.

You are somewhat done now. You have a printable 3d model. If you want to clean the model more up there are some programs that can help you with that.

Pixologic Sculptris (free): This is a great program to fix little details in your model, like smoothening out areas. it is like a photoshop, but then for the 3d world.

Meshmixer (free): An other free program from autodesk. Meshmixer is great to for instance make a flat bottom on your model. Alternative software to make a 3d model Besides Autodesk Recap there are more options to turn your photos into a 3d model.
Autodesk 123d Catch (free): It works like recap (being a cloud service), it a bit better developed then recap, but does scale down your images. So the end result is a bit less resolution.

VisualSFM (free): This is local (and much more complicated) software to turn your images into a 3d model. You need a powerful graphics card (supporting cuda) to run this software.
Agisoft Photoscan Standard Edition ($179): Also for this software you need a powerful pc and so far I have not found that this produces better results that Recap or 123d catch.

STEP 5: Results

After building this project, there is of course no greater fun then sharing the "machine" with others. So I decided to participate in the Dutch maker faire and offered everyone a free 3d photo shoot. I can tell you it was a very busy but fun 6 hours and we scanned 225 people! You can read more about it on my blog: www.pi3dscan.com

STEP 6: Design Files

Here are the design files I used. The "statief" is an 2d cad file for the poles that I cut out on my CNC.

The camera_mount files are the camera brackets I designed.

STEP 7: Version 2

A few months doing the Makerfaire show, I was invited for another exhibit to demonstrate my 3d scanner. I wanted to make the set more portable and easier to adjust. So I redesigned the frame. instead of using only wood, I designed tripods from cheap radiator pipes. This allows me to easily move the cameras up and down. I already designed new 3d printed brackets for the raspberry and camera that slide over the radiator pipes (40mm).

You can already download the tripod design on thingiverse: http://www.thingiverse.com/anykey/designs and I will soon be adding the raspberry and camera mount to it as well.

I am currently using 47 camera units. This allows me to do full body scan of kids (up to 1.20m). I will need to add more cameras to do full body scan of an adult.

STEP 8: Version 3 - 98 Cameras / Raspberry Pies and High Power LED Strips

Here is my latest setup. Using 98 Raspberry PI units on 19 poles (2 meters in height).

Between each pole is a 2 meter high power white LED strip (19.2watts per meter). On top is some extra LED lights (10 watts per meter) to provide some extra lighting on top of the head/hair

Python Scripts:

You can download the basic python scripts from my website: www.pi3dscan.com

Comments

[theworldvsme]

hi there.. great project.. will it be possible to use ESP32 cam instead of Rasberry Pi?

[Avina Fredslund]

Turns out you don't need the listening script. It is possible to just run the raspistill command through pssh on your laptop. And then have the images saved directly to a Nas drive mounted on the pi, with the -o option in the raspistill command. No need for python scripts, simple bash scripts can be used solely. With almost identical outcome. Not saying it's better. But I mean it works. And it's way more simple than having to learn python.

[Ghostrider513]

Freaking awesome. That many RPis though is way more than I can do. I'm struggling with my first - RPi 3b+. I need to scan some pieces of furniture and wood molding in a Victorian house. None of the molding or trim can be reproduced without a huge sum on money so I bought and built a CNC machine that can do the carving. Now I need to scan those pieces into the computer so I can reproduce them. Can you come up with a more affordable solution, even if we have to move one camera around to capture all the angles? It would be a blessing to have something a non-tech guy like me could build and use. Maybe three cams at most. This is awesome though. How much to make your current version? Do you charge for the 3D models or just the files to make the models? Thanks and outstanding job.

[VinayakNair]

This looks like a very professional solution, and the results are going to be amazing. I have used a Xbox 360 Kinect sensor to scan and 3d Print a model of myself. It came out with colors, but as I'm using a FDM printer with only one filament I printed it in a single color.

Another method is Photogrammetry which is possible using multiple shots from a single camera. But we need to make sure to keep the lighting consistent.
https://youtu.be/rR_uhAODSI8

[lazerx92]

Could this be used for smaller scale scans also? I am wanting to make miniatures, but they might have a older Kinect at my local pawn shop. I really enjoyed your video, btw. Very well done and I think this will help with my own projects once I get the initial setup and fine tuning done.

[the_anykey]

Hi, Furniture does not move, so yes, you can easily scan that with just a single camera like your mobile phone or DSLR. Just take lots of pictures with good overlap and I would recommend using Capturing Reality to process the images into a 3D model. You can "buy" (month or 3 month subscription) via Steam. it is not expensive. Alternatively you can buy agisoft, also not too expensive. Reality capture produces better 3D models, but agisoft is better at texture. So depends a bit what is more important for you.

[hundevandergames]

pardon if thats a stupid question, but is a Mac computer important for that project?

[harriethijssen65]

Hi Richard, awesome setup and great results.
I do have a few questions:

1. instead of ethernet hardwiring, can you use wireless? From the instructable I did not get anything that would require massive performance as long as all the RPi's are synchronized at virtually the same microsecond to prevent movement of the subject. Is the synchronizing of the RPi's the reason for hardwiring?

2. In your later videos you introduce projectors. From what I understand is that you take 2 images within "a very short" amount of time (ie. 50ms). The first image is without projector, the second is with projection (would this be similar to "structured light scanning"?). Can I assume that you have a dedicated camera per projector (presumably a higher end camera with better resolution?)

I'm thinking of building a similar setup and capture small animals (pets), but try to be as economical as possible.

Again, very impressed and am looking forward to actually start to build this setup.

[3dphoto]

Version 3 is very interesting.. about how much to build?

[FrancisT35]

Version 1 is pretty cool and complete! very nice.

About step 2, while your way of managing the cables/connections looks like it would be working just right, i could see a very easy/cheap way of making something cleaner and more heavy-duty/fool-proof.

Having work on many different racks & rigs in many different contexts, i feel like i could bring something to this. I first started working on racks and cases for a live show company called Solotech. What was great over there is that they dont specialize in lets say, audio, or staging etc.. They offer turnkey services for sound(P.A. and live recordings) + lighting (fixtures, control, and the rigging for it) + video (projection/mapping/processing/led displays/recording + closed circuit broadcast + switchers and tally controls and also live digital video processing/effects/clocking and more thru servers working as clusters [ps: this needs to be supported by a whole other data broadcasting system working over optical fibre to move the data from FOH to the processing racks(at the back), process the signal, then push back the source to the projectors/displays/led screen which most most likely back at FOH or beyond hahah]).

After working with all these i tough i knew/learnt most of the rack building insights and tricks but this amazing modular system crossed my path again, and again, and again!

When I left Solotech, i wanted to work in environments that are more controlled and fixed (if you want? lol). So i got into working in recording studios, again, the racking systems were key, but in a very different way. Then i got into working in shooting studios which use very similar racking and mounting systems as the recording studios but for very different reasons!

I eventually stopped working in studios(not totally, but mostly), I got tired of repetitive audio and video editing tasks, and studio setups that are always the same because most people out there aren't very original, its very trend-based, and people do mostly all very similar f****g song/video for 6 months, then its on to something else for another 6 months...

About 3 years ago, i really got into coding! i ended up enrolling in college just to drop out a year and half in, because PHP (sad, sad... I now pray to Javascript and C++), and Ive been working part-time since then, while taking my coding education into my own hands (thanks to all these amazing videos and courses/tutorials available online for free or very cheap!). And of course, the 19" racking prophet came to me again! This time, for servers/datacenters and computer/networking rigs, controllers and more!

All-in-all, its fucking great, look into it!

So lets get back to that messy bay now. easy way i can imagine on top of my head would be to simply use a a 3U sleeve (Gator have cheap ones) and use the slide-in versions of the ethernet connector from Neutrik(thats the company that engineered/produce the famous XLR connector, if you dont know what it is, its the standard Pro Audio connector, like on microphones and soundboard). Neutrik make the same size connector for plate mounting so it can be fit in any standard xlr 1u or 2u plates. I would use this on top on the lanswitch (both mounted in the rackmount sleeve) It could be easily connected to the back of the connectors, and you can fit it from the back(this way the connections will be on the inside); then you can just jump from the connectors to the switch with patch cables(ethernet); to finish it off and making it more fool-proof, just add 1 or 2 cable management bars( Middle Atlantic have a good catalogue of those + all kinds of other useful pieces) and attach the cables to the support bars with mini tie-wraps(this way if the cables gets pulled for whatever reason it will pull on the bar instead of the connections;

I will probably be doing exactly that when i finally build my own rig and will post the result here with the shopping list!

I attached pictures of all the pieces i talked about to this comment + a kind of mini-sketch;

if you have questions, or you just wanna chat about that geeky stuff, it me up!
[e-mail]


I probably wrote too much for such a simple thing, but i was going through planning my rig and told myself i might as well share my thinking process here!

PS: for those who know nothing about shows/concerts FOH means Front of House or the stage/where the performance is taking place. now you know.

[thefuman100]

would it be possible to make a rotating rig on circumference operated by a stepper motor and use maybe 2 upright camera scaffolds with 12 cameras 6 on each program rig to rotate and take photos as it rotates at intervals and using photogrametry software to create a 3d model fore printing.

[johnjacobson190]

I feel like this is a whole lot of work if you're trying to scan full-body at scale - seems better to go with a pre-built solution if you're attempting to run a 3D scanning business and don't have the time to deal with the difficulty of managing something like this. Given that there are solutions in the $20-$30k range (like the Twinstant Mobile http://web.twindom.com/) compared to the $20k or so cost of parts for this DIY (with enough cameras/projectors to make things work well), I'd think that would make more sense. Thoughts?

[Jeffth222]

what photogrammetry software do you feel is best? With 40 cameras can you get a good enough image to 3d print in full color ? Do you have projectors?

[roscowillie]

Richard,

I finally got the script to trigger a three camera wifi setup. It seems to work pretty well. I'm using the Model A, Cam, Edimax Wifi, 8 gb. I'm considering going for 24 (12 x 2) to capture people (15 ft diameter). Is this overkill? What have you settled on for your shooting configuration? Also, have you done any more experimentation with agisoft photoscan?

Thanks again for your creative thinking and tenacious making.

[the_anykey]

Hi, There is never overkill (well unless you use 200+ cameras) :-)

I can not capture a full body adult with just 24 cameras. I currently have 47 working and upgrading this week to 70 to enable full adult scanning.

[bharathy2]

Kool Richard.. Very impressive setup... But I have few basic doubts...

How many people you can scan with your current setup ? I need a setup which can scan single,couple or family... can we achieve using 64 or 100 camera setup ?

does your setup directly push all images to a laptop/computer connected in network for post processing..?? or we need to copy from each SD card ?

if we can use continuos light why not use LEDs ..?? which can be more cost effective and powerful ?

[MannyTheManmoth]

Could you also wire these up with just a router instead of using the gpio pins, assuming you are not using 50+Pis?

[TiagoC3]

Hello! Could you please email me the correct code to [e-mail]

I tried do run the code, but it is without identation. Thank you!

[AzadehA1]

Hi Richard,

I made the network cables using 4 wires and sparing the other 4 wires for power. I managed to power up the pi this way but the pi cannot connect to network although the network cable works when I don't plug in the power wires. I am using the AC to DC convertor power source of desktops.

Any guess what the problem might be?

Thanks,

Azadeh

[Csalway]

Do you control your raspberry pi's with a laptop or by another pi?