Date: February 10, 2007
This documentation accompanies an implementation done for the ``Scanning Real World Objects Without Worries'' project of Prof. Marc Alexa during winter term at Berlin technical university. It is also available in PDF format at Documentation.pdf. The doxygen documentation of the project is available at refman.pdf. The talks given for this project are available here:
The idea is to use a PMD depth image camera and two commercial cameras, which are mounted on a tripod. To produce range data from the stereo pair the depth information of the PMD camera is exploited. Once depth images of different views of the object are taken, alignment, registration and meshing has to be performed.
The idea for the alignment is to generate feature points from the range data. Then match those points to construct a rigid transformation between each pair of overlapping images in turn. This proceeding is described in section 2.3.
The range images are treated as point clouds after removal of the background area.
The implementation is a C++ program. The source tree is divided in
sub-folders. It includes a front-end, the algorithms and other
features like exporting and OpenSceneGraph support for the QT widget
set. The outline of the source folder is listed in table 1.
The program stores the set of loaded images in the Controller. The Controller is defined in the file controller/Controller.h. It manages global application data and some inter module communication.
The alignment algorithm is a modified RANSAC see algorithm 1. It is defined in matching/Matching.h. It calculates the matching transformation for the left feature set. The right set is assumed to have an arbitrary transformation . The important parameters of RANSAC are
The result of the alignment is of type FeatureMatching which is defined in matching/FeatureMatching.h. It is a Wrapper for the result of RANSAC. One FeatureMatching has a name, the two matched feature sets and pointers to the corresponding images. The features are copied and new memory is consumed for each matching. In the program the FeatureMatchings are managed by the Controller.
The global matching error is the mean distance of the nearest features
from the transformed images. For feature centers
from the left image and
right image it is
To speed up the NEARESTQUERY operation we use a KDTree implementation by Scott Kircher, that can be found at KDTree.h.
For feature points and we calculate the matching transformation . We implemented two methods for calcuating the transformation. The direct method which takes only matched points and a least squares approach which uses principal component analysis.
Direct calculation contructs a transformation with the following
The transformation calculation among other usefull utility functions is implemented in the file util/NewmatUtil.cpp.
The preview panel is an OPENSCENEGRAPH widget showing the loaded and transformed range images as lighted point clouds. The normals of the points are calculated by examining the direct neighbourhood of a point in the image. A slider alters the size of the displayed points.
The image library shows all currently loaded depth images. Buttons for loading and removal of images are at the top of this docking widget. One can add color overlays to the range files which will be displayed instead. The reset button sets the matching transformation of the active image to identity.
The main part of the application is the matching frame. Here one can choose the two images to be matched. The images are shown next to each other. Here features can be manually added and removed by clicking into the image panel. A left-click adds a feature, a right-click removes the feature under the mouse. Dragging features to a different location is possible.
The lower section provides the tools for feature generation and matching. Controls to generate and remove features of a selected image are located in the features tab. The number of features and skipped features can be entered. The matching tab offers spin boxes for all important parameters of the RANSAC algorithm. It also features controls for matching manually picked features.
Once a matching is aquired for the active pair of images it is drawn in the image panel. Blue lines connect the matched features point. All stored matchings for a pair of images are available in the drop-down menu below the image panel.
If one is satisfied with the alignment result viewed in the preview dock, the range data and matching transformations as well as the feature points can be exported to an XML file. The export functionality is implemented in the file export/Export.cpp. An example output can bee seen at example.pcml.
The definition of the global error can be improved by exploiting symetric queries to exclude unmatchable features from the calculation. This approach can also be used to improve the matching quality of the features. A feature match is correct if the match connects the features from both direction.
To match a series of range images a batch processor is needed. Features generation and matching in turn has to be automated to be usefull in a scanning environment.