This software performs two functions: (1) taking stereo image pairs as input, it computes stereo disparity maps from them by cross-correlation to achieve 3D (three-dimensional) perception; (2) taking a sequence of stereo image pairs as input, it tracks features in the image sequence to estimate the motion of the cameras between successive image pairs. A real-time stereo vision system with IMU (inertial measurement unit)-assisted visual odometry was implemented on a single 750 MHz/520 MHz OMAP3530 SoC (system on chip) from TI (Texas Instruments). Frame rates of 46 fps (frames per second) were achieved at QVGA (Quarter Video Graphics Array i.e. 320×240), or 8 fps at VGA (Video Graphics Array 640×480) resolutions, while simultaneously tracking up to 200 features, taking full advantage of the OMAP3530’s integer DSP (digital signal processor) and floating point ARM processors. This is a substantial advancement over previous work as the stereo implementation produces 146 Mde/s (millions of disparities evaluated per second) in 2.5W, yielding a stereo energy efficiency of 58.8 Mde/J, which is 3.75× better than prior DSP stereo while providing more functionality.
The focus is on stereo vision and IMU-aided visual odometry for small un manned ground vehicle applications. It is expected that elements of this implementation will carry over to small unmanned air vehicles in future work. Because the objective is to advance the state of the art in compact, low-power implementation for small robots, highly efficient algorithms that have already been field tested have been chosen. This system combines the sum of absolute differences (SAD)-based, local optimization stereo with two-frame visual odometry using FAST features (Features from Accelerated Segment Test). By exploiting the dense depth map to provide stereo correspondence for the FAST features, it achieves very respectable position errors of 0.35% of distance traveled on datasets covering 400 m of travel. The algorithms used by this system were heavily tested in previous projects, which gives a solid basis for their implementation on the OMAP3530. In the future, cost/performance trade-offs of algorithm variants may be explored.
The novelty of this system is the parallel computation of stereo vision and visual odometry on both cores of the OMAP SoC. All stereo-related computation is handled on the C64x+ side of the OMAP, while feature detection, matching/tracking, and egomotion estimation is handled on the ARM side. This is a convenient division of processing, as stereo computation is entirely an integer process, well suited to the integer only C64x+, while several parts of visual odometry involve floating point operations. The TI codec engine’s IUniversal wrapper was used to integrate the ARM and DSP processes.
The software used in this innovation is available for commercial licensing. Please contact Daniel Broderick of the California Institute of Technology at
This Brief includes a Technical Support Package (TSP).
Stereo and IMU-Assisted Visual Odometry for Small Robots
(reference NPO-48103) is currently available for download from the TSP library.
Don't have an account?
Overview
The document from the Jet Propulsion Laboratory (JPL) outlines advancements in stereo and IMU-assisted visual odometry, particularly for small unmanned ground and air vehicles. This technology is crucial for applications such as reconnaissance and explosive ordnance disposal (EOD). The focus is on enabling 3-D perception and localization, which are essential for autonomous navigation and operation in complex environments.
Key components of the system include a processor board based on the OMAP3530 architecture, which features a 720 MHz ARM processor and a 520 MHz C64x+ DSP. The board is compact and lightweight, measuring 17 x 58 x 4.2 mm and weighing 5.3 grams. It supports various peripherals, including USB, Ethernet, and HDMI, and has an optional WiFi capability. However, it has some drawbacks, such as rumored unreliable connectors and a slow MicroSD card.
The visual odometry system operates at different resolutions and frame rates, with capabilities of 512x384x51 pixels at 14 frames per second (fps) and 320x240x32 pixels at 30 fps. It employs good feature tracking and provides real-time display capabilities. The system is tightly integrated, allowing for precise timing between the camera and the inertial measurement unit (IMU), which enhances the accuracy of the visual odometry.
The software architecture is designed to support the real-time processing requirements of the visual odometry system, ensuring that data from the stereo cameras and IMU can be effectively fused to provide reliable navigation and localization information.
The document also emphasizes the potential broader applications of this technology beyond its initial design for mall robots, suggesting that the advancements could be beneficial in various fields, including commercial and scientific endeavors. The research and technology discussed are part of NASA's Commercial Technology Program, aimed at making aerospace-related developments accessible for wider technological applications.
For further inquiries or assistance, the document provides contact information for the Innovative Technology Assets Management at JPL, indicating a commitment to collaboration and support in the field of innovative technology. Overall, the document highlights significant strides in embedded computer vision and robotics, showcasing JPL's contributions to advancing autonomous systems.
