Dense point-cloud reconstruction

I have been working with Bundler and PMVS2 to reconstruct some of my old multi-camera scans from 2007. There is a lot of potential to get some really good output from them, but at the moment I’m getting incomplete results; only one side comes out.

As you can see, these are not complete scans.

Mysteriously, the bundle2pmvs command only processes some of the images with these sets (for other imagesets it does all). While a bit annoying, this can not be the cause of the single-side reconstruction, as there are still images from a full “orbit”:

C:bundlerexamplesphil>bundle2pmvs list.txt ./bundle/bundle.out
[ReadBundleFile] Bundle version: 0.300
[ReadBundleFile] Reading 18 images and 597 points…
[GetJPEGDimensions] File ./p01.jpg: ( 1704 , 2264 )
[GetJPEGDimensions] File ./p02.jpg: ( 1704 , 2264 )
[GetJPEGDimensions] File ./p03.jpg: ( 1704 , 2264 )
[GetJPEGDimensions] File ./p04.jpg: ( 1704 , 2264 )
[GetJPEGDimensions] File ./p05.jpg: ( 1704 , 2264 )
[GetJPEGDimensions] File ./p08.jpg: ( 1704 , 2264 )
[GetJPEGDimensions] File ./p12.jpg: ( 1704 , 2264 )
[GetJPEGDimensions] File ./p13.jpg: ( 1704 , 2264 )
[GetJPEGDimensions] File ./p14.jpg: ( 1704 , 2264 )
[GetJPEGDimensions] File ./p16.jpg: ( 1704 , 2264 )
[GetJPEGDimensions] File ./p20.jpg: ( 1704 , 2264 )
[GetJPEGDimensions] File ./p21.jpg: ( 1704 , 2264 )
[GetJPEGDimensions] File ./p22.jpg: ( 1704 , 2264 )

The “phil” imageset is available in it’s own Flickr set if you want to test out yourself, or as a zip file. The images are pre-undistorted using the ImageIron software, which could be potentially causing the issues. I’ll have a go with an untouched image set tomorrow. Alternatively, it’s possible that the PMVS2 reconstruction only works on one side, and that to get a full result, I will need to reconstruct from a few angles and combine.

Edit 1: I have heard back from one of the PMVS2 developers, Yasutaka Furukawa, who was kind enough to field my question:

PMVS is a multi-view stereo software, meaning, it uses all the images simultaneously. So, we do not do pairwise. It should give you 360 degree reconstruction. My guess is that Bundler fails in registering all the photos. So, Bundler only registers images on one side (the other half have zero camera parameters and are ignored in bundler2pmvs).

It looks likely that Bundler is matching the 16 or so cameras around the wall as the same 3 cameras. It thinks all the perspectives are similar, and is ignoring conflicts in points on the subject (Phil) over the stronger feature match of the cameras. I guess this makes sense; the view of each camera is identical except for the subjects direction.

Edit 2: I have had some better results from masking the background. On looking at the bundler output it bacame apparent that Bundler was matching the (by far most) consistent appearance of the cameras as all the same bank of three. Results are still not 360 degrees, however.

Here are the input images and results:

Edit 3: I’ve managed to optimize the image preparation, to get good results in PhotoSynth at least. I’ve kept the unique pattern, but removed the cameras that were leading to false positives.

Here is the current best imageset:

Here is the most successful (PhotoSynth-based) reconstruction so far: