SfM output to VRML for ICM file

[iscollar]

I'm using an SfM program suite (N. Snavely's Bundler (0,4), Y. Furukawa's PMVS2 or CMVS) with multiple images to make PLY output. To paraphrase the manual: a pinhole camera model is used; the parameters obtained for each camera are a focal length (f), two radial distortion parameters (k1 and k2), a rotation (R), and translation (t). The formula for projecting a 3D point X into a camera (R, t, f) is:

P = R * X + t (conversion from world to camera coordinates)
p = -P / P.z (perspective division)
p' = f * r(p) * p (conversion to pixel coordinates)

where P.z is the third (z) coordinate of P. In the last equation, r(p) is a function that computes a scaling factor to undo the radial distortion:

r(p) = 1.0 + k1 * ||p||^2 + k2 * ||p||^4.

This gives a projection in pixels, where the origin of the image is the center of the image, the positive x-axis points right, and the positive y-axis points up (in addition, in the camera coordinate system, the positive z-axis points backwards, so the camera is looking down the negative z-axis, as in OpenGL). Radial distortion in each image is removed with an intermediate step before running CVMS.

I want to display the images draped over the point cloud from the PLY file from CMVS or PMVS2 using CloudCompare to make an approximate orthophoto with the Z axis vertical. The PLY appears to display correctly in CC. I can compute the FOV from the focal length and image sizes. How should I convert the R, X and t values as above to get the VRML fields for the ICM file required by CloudCompare correctly?

[daniel]

Interesting.

The associated vrml file contains a set of viewpoints (see http://www.lighthouse3d.com/vrml/tutori ... shtml?view).

Example:

Code: Select all

DEF photo1.jpg Viewpoint {
    fieldOfView 0.621379
    position -10.5418 -15.6091 5.95961
    description blabla
    orientation 0.70724 -0.37292 -0.600618 3.74252
}

All the fields must appear in this order, as the code is very old and pretty dumb ('I'll try to fix this in a future version)
- the FOV angle ('fieldOfView') is in radians
- the 'position' is the optical center of the camera
- the 'description' or 'JUMP' fields are ignored (however the 'description' field must be present, see my remark above)
- the 'orientation' field corresponds in fact to a rotation ("rotation relative to the default orientation which points along the Z axis in the negative direction"). It is composed of 4 values: a vector (X,Y,Z) and an angle (radian).

The image name should be just after the "DEF" keyword, and CC will look for the corresponding file in the same folder as the VRML file. Once the data set has been successfully loaded (the job is done in IcmFilter::loadCalibratedImages), it is possible to overlay the image on top of the associated 3D view (the image is displayed in 2D, in full extent, while maintaining its aspect ratio). The OpenGL camera will be setup so that it matches the VRML viewpoint as described above (this job is done in ccGLWindow::applyImageViewport).

- So I guess that 't' and 'R' should roughly be equivalent to the 'position' and 'orientation' fields (you may have however to convert R - which is a matrix I guess? - to the vector+angle form --> you can look at the ccGLMatrix::getParameters method code for instance). In the worst cast you just have to invert them: R' = inv(R) and t' = -inv(R)*t.
- as you said, the FOV should be easily deduced from 'f'
- the radial distortion correction will have to be applied directly to the image (you'll get a new image that will be the one you give to CC. Warning, handling this correctly may be a bit tricky though. At the time this method was developed, the parameters and undistorted image were automatically generated by Trimble's RealWorks software).
- eventually, I don't see what the 'X' output is (is it really part of your tool set output?).

By the way, it would also be nice to extend the ICM format so that CC can handle these parameters directly. If you need help, you can provide me with a full set of data (images, cloud and parameters).