The LMDB (Landmark Database) Builder software identifies persistent image features (“landmarks”) in a scene viewed multiple times and precisely estimates the landmarks’ 3D world positions. The software receives as input multiple 2D images of approximately the same scene, along with an initial guess of the camera poses for each image, and a table of features matched pair-wise in each frame. LMDB Builder aggregates landmarks across an arbitrarily large collection of frames with matched features. Range data from stereo vision processing can also be passed to improve the initial guess of the 3D point estimates. The LMDB Builder aggregates feature lists across all frames, manages the process to promote selected features to landmarks, and iteratively calculates the 3D landmark positions using the current camera pose estimations (via an optimal ray projection method), and then improves the camera pose estimates using the 3D landmark positions. Finally, it extracts image patches for each landmark from auto-selected key frames and constructs the landmark database. The landmark database can then be used to estimate future camera poses (and therefore localize a robotic vehicle that may be carrying the cameras) by matching current imagery to landmark database image patches and using the known 3D landmark positions to estimate the current pose.
This work was done by Michael Wolf, Adnan I. Ansar, Shane Brennan, Daniel S. Clouse, and Curtis W. Padgett of Caltech for NASA’s Jet Propulsion Laboratory.
The software used in this innovation is available for commercial licensing. Please contact Daniel Broderick of the California Institute of Technology at
This Brief includes a Technical Support Package (TSP).
Constructing a Database From Multiple 2D Images for Camera Pose Estimation and Robot Localization
(reference NPO-47845) is currently available for download from the TSP library.
Don't have an account?
Overview
The document titled "Constructing a Database From Multiple 2D Images for Camera Pose Estimation and Robot Localization" is a technical support package from NASA's Jet Propulsion Laboratory (JPL). It outlines methodologies for building a landmark database (LMDB) that aids in localizing camera and vehicle positions using multiple 2D images captured from various perspectives.
The process begins with an initial "orbit" around a region of interest, where images are collected along with inertial navigation system (INS)-based camera models. Each frame in this orbit includes stereo range data and a features file that matches features found in a reference frame. A key component of the system is the "slice finder," which determines which slice of data to use based on the current pose of the camera. Each slice represents a specific sector of the orbital path and is promoted to a slice if it contains a sufficient number of landmarks.
Landmarks are defined as features that are observed in multiple frames throughout the orbit, with a focus on minimizing 3D localization error. The document details the steps involved in processing these images, including stereo processing, feature tracking, and 3D frame alignment. The LMDB Builder aggregates feature lists across all frames, removing those that are seen too infrequently and calculating 3D world positions using an optimal ray projection method.
The process also involves calculating re-projection errors for candidate landmarks and removing outliers to improve the accuracy of camera models. The iterative approach continues until the re-projection error stabilizes, indicating that further improvements are minimal.
The document emphasizes that while the LMDB Builder can function independently, it benefits from the integration of the Digital Elevation Model (DEM) Builder, which enhances the overall data quality. The methodologies described have broader implications for robotics and localization technologies, showcasing advancements that can be applied in various fields beyond aerospace.
Overall, this technical support package serves as a comprehensive guide for researchers and engineers interested in leveraging multiple 2D images for effective camera pose estimation and robot localization, highlighting the innovative techniques developed at JPL.
