DIY 3D Scanner Based on Structured Light and Stereo Vision in Python Language

66,395

307

37

About: Teacher that enjoys working with students

Intro: DIY 3D Scanner Based on Structured Light and Stereo Vision in Python Language

This 3D scanner was made using low cost conventional items like video projector and webcams. A structured-light 3D scanner is a 3D scanning device for measuring the three-dimensional shape of an object using projected light patterns and a camera system. Software was developed based on structured light and stereo vision with python language.

Projecting a narrow band of light onto a three-dimensional shaped surface produces a line of illumination that appears distorted from other perspectives than that of the projector, and can be used for an exact geometric reconstruction of the surface shape. Horizontal and vertical light bands are projected on object surface and then captured by two webcams.

Step 1: Introduction

Automatic 3D acquisition devices (often called 3D scanners) allow to build highly accurate models of real 3D objects in a cost- and time-effective manner. We have experimented this technology in scanning a toy to prove performance. Specific needs are: medium-high accuracy, easy of use, affordable cost of the scanning device, self-registered acquisition of shape and color data, and finally operational safety for both the operator and the scanned objects. According to these requirements, we designed a low-cost 3D scanner based on structured light which adopts a versatile colored stripe pattern approach. We present the scanner architecture, the software technologies adopted, and the first results of its use in a project regarding the 3D acquisition of a toy.

In the design of our low-cost scanner, we chose to implement the emitter unit by using a video projector. The reason was the flexibility of this device(which allows to experiment any type of light pattern) and its wide availability. The sensor can be either a custom device, a standard digital still camera or a webcam. it must support high quality color capture (i.e. acquisition of high dynamic range) and possibly with high resolution.

Step 2: Software

Python language was used for programming for three reasons, one it is easy to learn and implement, two we can use OPENCV for image related routines and three it is portable among different operating system so you can use this program in windows, MAC and Linux. You can also configure the software to use with any kind of camera (webcams, SLRs or industrial cameras) or projector with native 1024X768 resolution. It is better to use cameras with more than two times resolution. I personally tested the performance in three different configuration, first one was with two parallel Microsoft webcam cinema and a small portable projector, second one was with two lifecam cinema webcameras that rotated 15 degrees toward each other and Infocus projector, last configuration was with logitech webcameras and Infocus projector. To capture point cloud of object surface we should go trough five steps:

1. Projecting gray patterns and capturing images from two cameras "SL3DS1.projcapt.py"

2. Processing the 42 images of each camera and capture points codes " SL3DS2.procimages.py"

2. Adjusting threshold to select masking for areas to be processed "SL3DS3.adjustthresh.py"

4. Find and save similar points in each camera "SL3DS4.calcpxpy.py"

5 Calculate X,Y and Z coordinates of point cloud "SL3DS5.calcxyz.py"

The output is a PLY file with coordinate and color information of points on object surface. You can open PLY files with CAD software like Autodesk products or an open source sofware like Meshlab.

http://www.autodesk.com/products/personal-design-a...

Python 2.7, OPENCV module and NUMPY should be installed to run these Python programs. I have also developed a GUI for this software in TKINTER that you can find in step six with two sample data sets . You can find additional information on this subject on the following websites:

http://docs.opencv.org/modules/calib3d/doc/camera_...

http://docs.opencv.org/modules/highgui/doc/reading...

http://www.3dunderworld.org/software/

http://arxiv.org/pdf/1406.6595v1.pdf

http://mesh.brown.edu/byo3d/index.html

http://www.opticsinfobase.org/aop/fulltext.cfm?uri...

http://hera.inf-cv.uni-jena.de:6680/pdf/Brauer-Bur...

Step 3: Hardware Setup

Hardware consists of :

1. Two webcameras (Logitech C920C)

2. Infocus LP330 projector

3. Camera and projector stand (made from 3 mm Acrylic plates and 6 mm HDF wood cut with a laser cutter)

Two cameras and projector should be connected to a computer with two video output like a notebook computer and the projector screen should be configured as an extension to main windows desktop. Here you can see images of cameras, projector and stand. The drawing file ready for cut are attached in SVG format.

The projector is an Infocus LP330 (Native resolution 1024X768) with following specs.
Brightness:650 Lumens Color Light Output:**Contrast (Full On/Off):400:1 Auto Iris:No Native Resolution:1024x768 Aspect Ratio:4:3 (XGA) Video Modes:**Data Modes:MAX 1024x768 Max Power:200 Watts Voltage:100V - 240V Size(cm) (HxWxD):6 x 22 x 25 Weight:2.2 kg Lamp Life(Full Power):1,000 hours Lamp Type:UHPLamp Wattage:120 Watts Lamp Quantity:1 Display Type:2 cm DLP (1) Standard Zoom Lens:1.25:1 Focus:Manual Throw Dist (m): 1.5 - 30.5 Image Size(cm):76 - 1971

This video projector is used to project structured light patterns on the object to be scanned. The structured pattern consists of vertical and horizontal white light strips that are saved on a data file and webcams capture those distorted strips.

Preferably use those cameras that are software controllable because you need to adjust focus, brightness, resolution and image quality. It is possible to use DSLR cameras with SDKs that are provided by each brand.

Assembly and tests were conducted in Copenhagen Fablab with its support.

Step 4: Experimenting With Scanner

For testing the system a fish toy was used and you can see the captured image. All captured file and also the output point cloud is included in attached file,

you can open the PLY point cloud file with Meshlab:

http://meshlab.sourceforge.net/

Step 5: Some Other Scan Results

Here you can see some human face scans and 3d scan of a wall. There are always some outlier points due to reflections or inaccurate image results.

Step 6: 3D Scanner GUI

For testing the 3d scan software in this step I add two data sets one is scan of a fish and another is just a plane wall to see the accuracy of it. Open ZIP files and run SL3DGUI.py. For installation check step 2. Send message to my inbox here for all source codes.

For using 3d scan part you need to install two cameras and projector but for other parts just click on the button. For testing the sample data first click on process then threshold, stereo match and finally point cloud. Install Meshlab to see the point cloud.

http://meshlab.sourceforge.net/

Share

Recommendations

  • Optics Contest

    Optics Contest
  • Plastics Contest

    Plastics Contest
  • Electronics Tips & Tricks Challenge

    Electronics Tips & Tricks Challenge

37 Discussions

0
None
BraidenP

15 days ago on Step 6

1.) Which Linux platform will work best?
2.) Are the cameras usb?
3.) Projector connected to hdmi?

0
None
bdvd

3 years ago

Hi hesamh,

I used your code to make a home made small scanner. It is a very good code. I did some modifications to run it faster. If you want I can share the code with you or if you have it on GitHub I can send it you there.

Best regards,

bdvd

7 replies
0
None
SongT4bdvd

Reply 23 days ago

Hi bdvd,

I have a strong interest in this project, can I get your code, thank you.

lchj

0
None
Salagionbdvd

Reply 2 years ago

Hi bdvd,

I noticed this in the code

x1=-(pxpy[0]-960)*fcr #px1
y1=(pxpy[1]-540)*fcr #py1

Do you know where 960 and 540 come from? And what's the physical meaning of x1, pxpy here?

Thanks a lot.

Best regards,

Salagion

0
None
hesamhSalagion

Reply 2 years ago

960 is 1980/2 and 540 is 1080/2 (resolutions of the camera), these are index of the array for center pixel of the camera

0
None
Byte Chenbdvd

Reply 2 years ago

Hi,could u please recommand some cameras for me?I do not know what to choose.It should be able to build colorful 3D model.THK

0
None
hesamhByte Chen

Reply 2 years ago

You need a camera with software control, some webcams can be used, like some models of microsoft or logitech webcams. It is better to use a camera with no compression and also with global shutter

0
None
hesamhbdvd

Reply 2 years ago

Hi, can you send the code link to me?

1
None
shreyaskamathkmbdvd

Reply 2 years ago

Hello bdvd,

I was planning to make something similar using hesamh's code. Could you please help me out by providing your code too? It would be very grateful.

Thank you,

Best Regards,

Shreyas

0
None
fbujold

Question 8 months ago on Step 1

have you considered/tried using a turntable for the object to cover it in its entirety?

0
None
Kai-SiangG

1 year ago

Hi, hesamh

Thanks for sharing.

In the step 2, you say that it is better to use cameras with more than two times resolution.

How did you get this result?

What are you comparing?

Best regards.

Peter

0
None
CITD

1 year ago

I am currently
comparing photographic, laser and structured light methods of producing point
clouds. Please can you explain the advantage of using your method over
automatic image correlation as two cameras are being used to give a stereo
view? Is it more difficult to match the imaged texture of the object than the
stripes?

2 replies
0
None
hesamhCITD

Reply 1 year ago

By projecting patterns we produce feature point to make correlation accurate

0
None
CITDhesamh

Reply 1 year ago

Thank you for your reply.

Even if there is good texture on the surface of the object do you think that the projected patterns still provide a more accurate surface?

0
None
PatA44

1 year ago

what about scanning something smaller and little more complex like a ps4 controller shell?

0
None
RohitG78

1 year ago

does the chinese projector unic uc 46 with native resolution 800x480 will work ?

1 reply
0
None
hesamhRohitG78

Reply 1 year ago

It works but you should adjust projector resolution inside python programs and change the binary images that will be shown on projector

0
None
paulie_g

2 years ago

What would you estimate the resolution and accuracy to be with the logitech c920s?

2 replies
0
None
hesamhpaulie_g

Reply 2 years ago

Hi Paul, my projector can project more than 100000 different points, so if you catch them with both cameras you can have same number of points in your point cloud

0
None
paulie_ghesamh

Reply 2 years ago

Before I respond, I just want to say that this is an awesome project and I love the fact that you've used python for it as it'll make it super easy for me (and everyone else) to adapt it to our needs and develop it further.

The reason I ask about resolution and accuracy is those are the standard metrics used to determine 3d scanner performance and I wanted to be able to compare your setup with alternatives like a line laser based scanner to see if it's worth spending the extra money to do sls.

You say that you're projecting 100k+ points. Where does that figure come from? Presumably you use more than one pixel for each point? I would think these points are over a rather large area, so what sort of point per inch are we talking if you happen to know? How small were the smallest features you were able to capture reliably? Please understand that I'm not trying to bash your project or even imply that you ought to have answers to all these questions, I'm just trying to get at some sort of data to see if the sls approach is worth it for my project.