I recently came across a simple and easy package that can be used to create 3D reconstruction of objects. I wanted to share it and give an easy and practical explanation on how one can create visually appealing 3D models by running a few simple commands, no coding needed. I must emphasize that for keeping it simple, this post will not focus on theory as did the last few posts on binary descriptors, but instead will give an easy and practical guide to 3D reconstruction. Just to give you a taste of what can be done with the package, here’s an example of a 3D reconstruction I made (yes, that me in there):

Here you can see an original image vs. a screenshot of the 3D model:

The package that we’ll use is named OSM-bundler. The package implemented 3 algorithms:

• Bundler – which performs sparse 3D reconstruction
• PMVS (patch multi view stereo) that takes as input the sparse reconstruction resulted from running Bundler and produces a dense 3D reconstruction.
• CMVS (clustered multi view stereo) which is the same as PMVS, only that is faster and should be used when you process a lot of images.

Those algorithms were in fact used in two very impressive projects that ran 3D reconstruction on a very large scale. The first, is “Building Rome in a Day” aimed at (sparse) 3D reconstruction of touristic sites from images found on the web. The following video demonstrates the 3D reconstruction of the old city of Dubrovnic:

The reconstruction was done from about 4600 images. In the second work – “Towards Internet scale Multi-View Stereo” – , the CMVS algorithm was presented and demonstrated how multi-view stereo can be done on a large scale. The following video demonstrates various touristic sites that were reconstructed using images from the web:

In the next section I will explain how to run those packages to produce 3D models.

Installing and running OSM-Bundler

First, download OSM-Bundler from here . Extract it to whichever location you’d like. In my installation, I extracted it directly to C drive, so I now have the following directory from which I’ll run Bundler and PVMS: “C:\osm-bundler-pmvs2-cmvs-full-32-64\osm-bundler”.

Next, download and install python 2.7.6. You should also download and install Python Imaging Library (download the version for python 2.7).  To view the 3D models, you can use Meshlab, which you can download here.

Congratulations, you’re now done with the installation.

 

The procedure for building a 3D model is the following:

1. Take images of the object with your camera from multiple view points. The object should contain texture and shouldn’t have any transparent parts such as windows. Rotate around the object and take an image every few degrees. I used 51 images to create the reconstruction above.
2. Open command line prompt by pressing the winkey (the one with the windows logo) and typing cmd.
3. Use the cd command to go to the folder C:\<Installation Path>\osm-bundler\osm-bundlerWin32 or C:\<Installation Path>\osm-bundler\osm-bundlerWin64 according to your operating system.

>>cd  C:\<Installation Path>\osm-bundler\osm-bundlerWin32/64.

4. Run Bundler on the folder where your images are located:

>>python RunBundler.py –photos=”<full path where your images are located>”

5. Next, Bundler will operate on your image. Once it’s done, it will open a temporary folder with Bundler’s outputs.
6. Now, run PMVS on the bundler output folder using the following command:

>>python RunPMVS.py –bundlerOutputPath=”<BundlerOutputPath>”

7. Once PMVS terminates, you’ll have a model you can view using Meshlab or any other suitable software. In Meshlab, go to file -> Import Mesh. Browse to Bundler’s output folder and in there select the file PMVS\models\pmvs_options.txt.ply.
8. You’re done!

 

Troubleshooting

If the geometry of the model seems broken, the most common reason is that Bundler camera database doesn’t contain the specific camera you used. To solve this, first, right click on one of the images and click on properties. Search for the “Camera model” field and write down its contents (the camera model). For example, for images taken with the Galaxy S4 camera, the “Camera model” is GT-I9505.

The next steps are a bit tricky. You’ll need to find out the CCD width of your camera. To do that, search for the camera’s sensor size (for example Google “<camera name> sensor size”). For example, if you’d google “Galaxy S4 sensor size” and search a bit, you’ll find out that the sensor size is 1/2.33 which according to this page is 6.16 x 4.62. The CCD width is the first value of the two – 6.16.

Now, we need to insert the camera model and the CCD width we found to Bundler’s camera’s database. To do that, run the following command:

>> python RunBundler.py –photos=<full path where your images are located> –checkCameraDatabase

Now, Bundler will prompt you for the camera’s CCD width. Enter it and press ok.

Still having problems?

If the geometry still seems broken, try first to run Bundler and PMVS on the example sets provided in the package. Try taking more images of the object and remember Bundler and PMVS do not work so well for object textureless objects, such as smooth surfaces. If you continue to encounter difficulties, mail me at Gil.levi100@gmail.com and I’ll try to help.

 

References

[1] Agarwal, Sameer, et al. “Building rome in a day.” Communications of the ACM54.10 (2011): 105-112.‏

[2] Furukawa, Yasutaka, et al. “Towards internet-scale multi-view stereo.”Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on. IEEE, 2010.‏