A Sandstone Block Built From Lego, Blending Real Objects With 3d Prints

By combining 3d scanning, 3d digital modelling, and 3d printing, it is possible to create amazing and surprising effects, objects that can seamlessly blend between reality and imagination. In this tutorial, I describe the process of making a full colour 3d printed object that seamlessly fits on to a real sandstone wall, producing the illusion that the wall is built from pieces of lego. But the techniques described  have a more general usefulness, almost limitless potential, and  are easier and less expensive than you might think!

Staring with a real physical object, in this case a sandstone brick wall, I create an inexpensive but high quality digital 3d scan using a stills camera and the program Agisoft Photoscan. I then model the lego brick shapes over this scan using 3d software, I make this model fit perfectly onto the scanned wall by using boolean subtraction, I add colours clone stamped from the photos, correct the scale, and finally I do a colour 3d print, which fits seamlessly onto the sandstone wall, producing a novel illusion that would be difficult to achieve by other methods. 

Step 1: 3d Scanning

3D scanning is traditionally achieved with prohibitively expensive laser scanner hardware. However, recent software developments have opened up the possibility of acquiring a similar quality of 3d scan using just a stills camera. I used the program Agisoft Photoscan, which I highly recommend, and a cheep point-and-shoot canon camera. Agisoft Photoscan is easy to use, and not terribly expensive. There are also some free alternatives, ScannerKiller, and 123d Catch.

I took around 29 photos from different angles, and the software automatically derived a 3d mesh from those photos. I chose a sandstone block with a chipped off corner, so that my 3d print could fit into empty space left by that missing corner. The process of using agisoft photoscan is very user friendly, but poor quality photos will tend to result in a less detailed mesh with more artifacts. Try to avoid taking photos that are poorly exposed, or out of focus. The object should remain completely still while you take the photos, if it changes shape or moves it's position then the reconstruction probably won't work. The photos should not be taken from dramatically different angles, there must be some level of overlap and similarity among the photos, or a mesh cannot be derived.

I generated the mesh with Target quality set to Medium, and Geometry type set to Smooth. I generated a texture at 2048*2048 using Orthphoto Mapping mode, and Mosaic Blending mode. Then exported the mesh as an .obj file, and the texture as a .bmp.

These programs tend to work best with very textured objects. The texture gives the software features to lock on to. To get the Best results with smooth untextured objects you may have to apply a random blotchy paint pattern. Blotchy facepaint works really well when scanning people, though it may look rather silly! Shiny objects can also cause problems, because they look different from every angle. Shine may be reduced in a number of ways, talcum powder, dulling spray, or it may be filtered out using polarizing filters, one on the lens, and one on the light source. The filter should be rotated until the shine disappears. Large polarizing filters can be bought online, and it's possible to get very cheap small polarizing filters by cannibalizing the lens from a pair of 3d glasses. Some light sources such as the sky, or LDC screens, are already polarized, so they won't need a filter.

In many cases it is possible to approximate tiny micro details from the scanned object. By applying a "high pass" filter (in PhotoShop, or an equivalent image editing program) to the colour texture map one can create an image that may be used as a bump or displacement map. The results may not be perfectly accurate, but in many cases this will result in a visually convincing level of very fine detail.

Step 2: Correcting the Scale

Agisoft photoscan will produce a very accurate mesh, however this mesh will have an arbitrary scale, it won't have the correct units in size overall. It's impossible for the program to know the absolute scale of the object, a small object close to camera may optically look identical to a large object far away from camera, so the scale must be adjusted.
I used a pair of calipers to take measurements from the sandstone block, then I created a box objects  in 3ds Max with those measured values in their size. I then scaled the scanned mesh so that it matched up with the size of the box objects.
The same process may be achieved with a number of other programs, including free programs like Blender.

Step 3: 3D Modelling

I created the model using 3ds max, but similar results could be achieved using many other programs, including the free program Blender. I often also use the 3d sculpting program Zbrush, which is brilliant, though slightly intimidatingly complex. There are various other similar programs such as Mudbox, 3D-Coat, etc. An excellent free alternative is Sculptris, which is extremely easy to learn.

I built the lego brick shapes using polygon modeling techniques, based on the dimensions of real lego bricks. I first modelled a false chipped off corner, attempting to get as realistic and seamless as possible. Small bumpy details were achieved by using the "displace" modifier, and a procedural noise texture map. I was careful to match the scale of the bumpy details to the scanned mesh. I placed copies of the lego brick model inside the volume of the chipped off corner, as though the sandstone block is made from lego pieces. I then combined my replacement corner model with the lego brick models. This was the most challenging part of the process, and required some careful technical modeling.

Step 4: Cloning Colours

I coloured the mesh by cloning directly from the texture that Agisoft Photoscan generated. I did this in PhotoShop, but the same effect could be achieved in other programs, like the free program the Gimp for example, or zbrush, etc.
The lighting, exposure, white balance of the photos, and the printing process may result in the colours of the 3d print looking different to the real object, so I've created a small colour sample 3d print to help calibrate the colour balance of the textures. By holding the 3d printed colour sample next to the real object I am able to work out what brightness, hue, and saturation to use.

Step 5: Subtracting the Scan Mesh From the Model

The model intersected with the wall, meaning that if I printed it out as it was then it wouldn't fit onto the real wall. To make it fit the real wall perfectly I subtracted the scanned mesh from my 3d modeled mesh using booleans. I did this using 3dS Max, but many programs could achieve the same result.
In 3ds Max booleans are achieved by clicking on the Create tab, choosing Compound Objects, clicking the ProBoolean button, and then picking the model you wish to subtract.
Booleans typically only work with "watertight" models, so I was careful to avoid open edges as I built the mesh. In some cases a simple "cap holes" modifier is enough to make a boolean operation successful.

Step 6: 3D Printing

I printed the model using Shapeways, using a surprisingly inexpensive full colour material called "Full Color Sandstone". Larger models will be more expensive, but my model cost $16.89, and I've had other small models printed for less than $10. I used two texture maps, one for the lego part, and one for the sandstone wall part. Different material id numbers denote different textures, so I ensured that the sandstone parts of the mesh were set to id #1, and the lego parts set to id #2. If you're just using one texture then set all the faces to id  #1, otherwise you may find that your texture map isn't showing up on the mesh.
I saved the texture maps as a .png files, and exported the object as an .X3D from 3dsmax, which required a plugin. I created a zip file containing the mesh and the texture bitmap, and uploaded it on the shapeways website. It's important to check that the scale is correct, 3d applications handle units in different ways,so some double checking, trial and error, and re-scaling may be necessary to get the correct results. I found that I had to scale my models by 39.37%, (the ratio difference between cm and inches).
Prints in the "full color sandstone material" will have a slightly grainy surface, so going over the print with very fine sandpaper can help. I also gave the 3d prints a thin coat of superglue and matte varnish, to help bring out the colour, as well as seal and protect the surface.

Step 7: Painting

Once the model arrived I found that the colours were slightly incorrect, so I painted the printout lightly with acrylic paint to make it match the real sandstone brick more closely. My method for matching colours with paint is to mix a rough estimate the colour, paint that onto a small piece of card, which I then hold against the subject so that I can compare the colours directly. By adjusting the mixed paint to be darker or lighter, cooler or warmer, it is possible to get a very close match.
I've since printed out a small color sample of the full color sandstone material, which can be used to calibrate the colours of texture maps more accurately.

Step 8: Results, Applications, Limitations, and Possible Variations.

These techniques can produce novel and striking effects, surreal blends between different objects, between reality and imagination. Real objects may be cloned, scaled, deformed, and modified in virtually limitless ways. 3d digital modeling may be enhanced by "sampling" from the real world via 3d scanning. The techniques and aesthetics of the digital world may be brought to the world of tangible reality. I use the techniques for artistic purposes, but many they have endless other potential applications.
Similar techniques are already used in the medical field, 3d models of bones and other organs can be acquired via CT or MRI scanning, and modifications are printed out in bio-compatible materials. Potentially in the future 3d prints may even be made using the patient's own cultured cells.
On a more prosaic level, these techniques could be used to repair or customise objects. Custom toys, hobby models, jewelery, cases for electronic devices, prosthetics, etc. The applications are limitless.
The main limitation of this technique is size. Larger 3d prints are exponentially more expensive than smaller prints, and shapeways currently prints at a maximum size of 25x38x20cm using the "full color sandstone" material.
A range of different effects may be achieved through variations on the technique that I describe in this tutorial. One potential example would be to create embossed text on practically any surface, as shown in the image above. For this tutorial I subtracted a 3d scan from a digital model, but it is also be possible to subtract a 3d model from itself, thereby creating a 3d print that can be tiled or tessellated. An example of this is shown in the second image above, I created a 3d printed hand that perfectly fits onto copies of itself, so once it's duplicated (in this case through a silicone mold) it may be tessellated into a sphere shape.
So, to conclude, these technologies open up a wide range of interesting and exciting creative effects that were not possible in the recent past, I'm very excited to see what other people are able to achieve with these techniques.