|
3D Scene Analysis from Single Image with Lens Distortion |
INVESTIGATORS
Dr. Yisong Chen et al.
KEYWORDS:
Camera calibration, Lens distortion correction, Nonlinear optimization, Bundle adjustment, Single view metrology, Error analysis.
BRIEF DESCRIPTION
Efficient 3D scene analysis from a single image with lens distortion is difficult. We bridge the gap between perspective lens distortion correction and single view metrology by integrating metric constraints into lens distortion correction. We establish a good mathematical model for efficient parameter estimation, scene measurement and error analysis. Under this formulation, the lens distortion parameters, especially the position of the distortion center, can be precisely recovered. Therefore, reliable 3D measurements can be made from images with lens distortion. The effectiveness of the estimation and the measurement is validated by the associated error analysis algorithm.
(a) Original image(360x288) (b) corrected image without metric constraints (c) corrected image with metric constraints
Figure (a) shows one image captured by a top view camera mounted on the ceiling of a laboratory. Only minimizing the projective constraint error will cause serious deviation of the distortion center and lead to a wrongly corrected image, as shown in Figure (b). In contrast, by considering simultaneously both the projective constraint and the metric constraint we get accurately estimated parameters and a corrected image as shown in Figure (c).
Our algorithm gives rise to a corrected image that corresponds to one taken by an ideal pinhole camera. Following is a example. We can see intuitively that the corrected image is satisfactory. Euclidean measurements in 3D, such as the height of the desk, the length of the keyboard, the mouse mat, and the umbrella, and the height of LCD monitor, can be made in a hierarchical manner. The biggest deviation is only about 2.0 cm.
FUTURE WORK
We intend to extend the above algorithm to multiple view based epipolar geometry computation. The epipolar geometry is computed from the two corrected images and auto-calibration is performed to recover intrinsic parameters of the cameras. The exciting element here is that the aspect ratios and the principal points of both images can be reliably estimated in the lens correction stage. This can greatly reduce the computational burden of auto-calibration by leaving the focal length as the only unknown parameter to be solved, which makes auto-calibration from only two views possible. In recovering focal length, we plan to employ a carefully designed Monte Carlo approach to improve the soundness of the algorithm and avoid instability or singularity. Therefore, Metric 3D reconstruction can be achieved from the auto-calibrated cameras. The planar scene recovering technique in our early project greatly helps to perform such reconstruction. We have achieved success in synthetic scenes (see the illustration in the following figures) and are planning to generalize the method to real world scenes.
(a) Distorted image 1 (b) Distorted image 2
(c) Several views of the reconstructed scene
PUBLICATIONS
Horace H. S. Ip and Yisong Chen, 3D scene analysis from single image with lens distortion, In submission.
