The topic of my team is to calibrate the Olympus SP500-UZ 2D camera and PMDtec PMD[vision] 19k 3D camera
intrinsic (remove the barrel distortion) and extrinsic (calculate the world orientation).
For the calibration we use the Zhang 2-dimensional pattern based calibration algorithm as described in . The implementation is speciﬁed in section 3. For the algorithm it is required to take minimal 3 pictures of the calibration object (from different view angles) for every intrinsic calibration. As intrinsic parameters change with focus and zoom of a camera it is necessary to calibrate each zoom and focus setting separate. For the Olympus SP500-UZ with 52 zoom and 120 focus setting we have to take 18720 pictures and even more if we want better accuracy.
So it is essential to have a fast and easy way to take several pictures automatically. We wrote a tool that controls the camera via a camera SDK and assists us in this process. The functionality of this tool is described in section 2. The process of calibration itself is the topic of section 3.
The tool that helps us in the process of taking several calibration image is called takePic. It is a simple
win32 command line tool written in c/c++ with the help of bloodshed dev-cpp mingw based IDE
The program connects to the camera via the Olympus ryeNV camera SDK. The command line takes 3-4 simple parameters. The exposure time in milliseconds, the zoom setting(focal length) in mm*10 and a focus setting or range of focus.
syntax: cam.exe EXPOPSURETIME ZOOM FOCUS [FOCUSEND]
With the help of the focus range parameter (FOCUS - FOCUSEND) it is possible to take several calibration pictures over a full focus range of the camera in a fast and easy way.
For different focus settings with ﬁxed zoom the view area change is only based on the distortion so it is easy to setup the camera system for one focus setting of a ﬁxed zoom and take the calibration images of this ﬁxed zoom for the whole focus range of the camera.
This reduces the setup of the camera to minimal 156 for all zoom and focus setting with a minimal calibration picture count of 3 images. The program takes pictures in the highest resolution of the camera to get the most image information as possible. Because the pattern is good visible it is sufficient to preview the calibration object via the preview LCD of the camera. So a graphical user interface for the tool is not needed.
To decrease the photograph time the camera LCD gets disabled after the the pictures taking process stars.
Because of the size of RAW images it is at the moment not feasible to take the calibration images in RAW format. This can change in the future. The calibration images are stored in format to ease the calibration more.
The structure of the format is:
More details on takePic can be found in the takePic doxygen documentation (pdf).
We use the Zhang calibration algorithm (see ) to calculate the intrinsic parameters, radial distortion and
In this algorithm the intrinsic is calculated with the help of corresponding feature points from a ﬁxed grid and pictures of the grid from different view angles.
We use a chessboard for the feature point grid (connection points between connected squares). We take calibration images as described in 2 and ﬁnd the feature points in the pictures (semi-)automatic with the help of a corner detection algorithm.
To ﬁnd the chessboard points automatic and calculate the intrinsics from these points we use a modiﬁed version of
the open source tool CamChecker (see ) by Matt Loper from the Brown University (http://www.brown.edu/).
We modiﬁed the tool to overcome some issues.
1. we also have to calibrate a low resolution(120 x 160 pixel) 3D camera with bad intensity pictures where automatic corner detection is not easy
2. we want to output the calibration data in our own calibration database format
3. we want to save the extrinsic guess found by Zhang algorithm ()
To address these problems we added the following modiﬁcations to CamChecker:
1. we modiﬁed the parameter output
(now all parameters are printed into a ﬁle in our own format)
2. we changed the feature point detection tolerance
(we added a parameter to change the tolerance of corresponding square point detection for low resolution pictures)
3.we added a grid for manual setup of the checkerboard points
(for bad intensity pictures)
4. we added the ability to zoom into the picture for better corner point analysis
5. we read out the focal length of a picture to save this data for extrinsic calibration
A detailed description of the way how CamChecker/CamChecker+ detects corner points and calculates the
intrinsic data can be found in the CamChecker+ doxygen documentation (pdf).
For the ﬁrst extrinsic calibration approach we use the extrinsic guess found by CamChecker/CamChecker+
and output the extrinsic data (rotation matrix and translation vector), the intrinsic data and the
grid points into a ﬁle. This guess is then used to be optimized via minimizing the back projection
In the current state of the project we have some errors in this process. the z translation is wrong and x and y translations are slightly wrong. It also happens that the rotation axis are ﬂipped or interchanged (the reason for this error will be described in section 4).
One problem we found so far is the fact that the rotation axis of the extrinsic parameters are ﬂipped
or interchanged. This is because of the point symmetry of the calibration pattern. In an image of
a 45∘ rotated grid you are unable to ﬁgure out if one edge is up/down or left/right. The solution
to this problem would be a pattern with an obvious orientation. This would be no problem to the
underlying Zhang algorithm because this algorithm only needs corresponding points. But it needs
improvements to the pattern detection and analyzing code. This issue is solved in GML C++ Camera
Calibration Toolbox (see A and ) by using a non-uniform chessboard/checkerboard (odd x even
Another problem is the calibration of the 3D camera. The PMDtec PMD[vision] 19k has only one zoom and focus setting so we only need 3 calibration pictures. But analyzing the intensity pictures of the camera is not easy. The chip is only low resolution and very sensitive on reﬂections. Also the ﬁxed zoom and focus forces us to have a certain distance to the calibration object. Both distance and reﬂection force us to use a very good and big pattern. The absence of the PMD camera refused us to test the big pattern for calibrating the 3D camera.
There are several other projects implementing the Zhang or other algorithms for camera calibration.
One is the openCV (open computer vision library by Intel) which is c/c++ library with several functions for checkerboard corner detection and intrinsic and extrinsic calibration based on the Zhang algorithm (). It can be found under .
The GML C++ Camera Calibration Toolbox (by Alexander Velizhev from
Moscow State University http://www.msu.ru/en/ ) is a program that uses openCV and has a easy to use GUI. You can ﬁnd it here .
Another project that implements camera calibration is DLR Camera Calibration Toolbox by the DLR (Deutsches Zentrum f¨ur Luft- und Raumfahrt http://www.dlr.de/). It is a very complex tool set (corner ﬁnder and calibrator) that can output distortion up to 7th order. The program is written in the mathematical programming language IDL. To use the program you have to install the IDL virtual machine ﬁst. You can ﬁnd more information’s on the used calibration algorithms here  and  and more on the program here .
The Olympus SP500-UZ has a 1/2.5inch diameter CCD (according to the documentation).
These 1/2.5inch CCDs have following real dimensions:
|1/2.5inch||5.76 x 4.29 mm||7.2 mm|
You can calculate the angle/ﬁeld of view of a camera with the following formula:
fov = 2 ⋅ arctan(d∕(2 ⋅ f)) with
|f = focal_length||(for the Olympus SP500-UZ lens it is 6.3mm till 63mm)|
|d = dimension||(horizontal , vertical or diagonal dimension in mm)|
|fov||the horizontal, vertical or diagonal angle (corresponding to d) of f.|
|||Wikipedia article on Normal Lenses|
|||Wikipedia article on Digital photography|
|||Vincent Bockaert. Sensor Sizes. Digital Photography Preview online glossary|
|||Wikipedia article on Angle of view|
|||Z. Zhang. A Flexible New Technique for Camera Calibration.|
|||The GML C++ Camera Calibration Toolbox|
|||DLR Camera Calibration Toolbox|
|||K. H. Strobl and G. Hirzinger. Optimal Hand-Eye Calibration. In Proceedings of|
|the IEEE/RSJ International Conference on Intelligent Robots and|
|Systems (IROS 2006), Beijing, China, pp. 4647-4653, October 2006.|
|||J. Weng, P. Cohen, and M. Herniou. Camera calibration with distortion models and|
|accuracy evaluation. In IEEE Transactions on Pattern Analysis and|
|Machine Intelligence (PAMI), 14(10): 965-980, 1992.|
|||Matt Loper. CamChecker|