Tech Briefs

Stereo and IMU-Assisted Visual Odometry for Small Robots

This software performs two functions: (1) taking stereo image pairs as input, it computes stereo disparity maps from them by cross-correlation to achieve 3D (three-dimensional) perception; (2) taking a sequence of stereo image pairs as input, it tracks features in the image sequence to estimate the motion of the cameras between successive image pairs. A real-time stereo vision system with IMU (inertial measurement unit)-assisted visual odometry was implemented on a single 750 MHz/520 MHz OMAP3530 SoC (system on chip) from TI (Texas Instru ments). Frame rates of 46 fps (frames per second) were achieved at QVGA (Quarter Video Graphics Array i.e. 320×240), or 8 fps at VGA (Video Graphics Array 640×480) resolutions, while simultaneously tracking up to 200 features, taking full advantage of the OMAP3530’s integer DSP (digital signal processor) and floating point ARM processors. This is a substantial advancement over previous work as the stereo implementation produces 146 Mde/s (millions of disparities evaluated per second) in 2.5W, yielding a stereo energy efficiency of 58.8 Mde/J, which is 3.75× better than prior DSP stereo while providing more functionality.

The focus is on stereo vision and IMU-aided visual odometry for small un manned ground vehicle applications. It is expected that elements of this implementation will carry over to small unmanned air vehicles in future work. Because the objective is to advance the state of the art in compact, low-power implementation for small robots, highly efficient algorithms that have already been field tested have been chosen. This system combines the sum of absolute differences (SAD)-based, local optimization stereo with two-frame visual odometry using FAST features (Features from Accelerated Seg ment Test). By exploiting the dense depth map to provide stereo correspondence for the FAST features, it achieves very respectable position errors of 0.35% of distance traveled on datasets covering 400 m of travel. The algorithms used by this system were heavily tested in previous projects, which gives a solid basis for their implementation on the OMAP3530. In the future, cost/performance trade-offs of algorithm variants may be explored.

The novelty of this system is the parallel computation of stereo vision and visual odometry on both cores of the OMAP SoC. All stereo-related computation is handled on the C64x+ side of the OMAP, while feature detection, matching/tracking, and egomotion estimation is handled on the ARM side. This is a convenient division of processing, as stereo computation is entirely an integer process, well suited to the integer only C64x+, while several parts of visual odometry involve floating point operations. The TI codec engine’s IUniversal wrapper was used to integrate the ARM and DSP processes.

The software used in this innovation is available for commercial licensing. Please contact Daniel Broderick of the California Institute of Technology at This email address is being protected from spambots. You need JavaScript enabled to view it. . NPO-48103

This Brief includes a Technical Support Package (TSP).

Stereo and IMU-Assisted Visual Odometry for Small Robots (reference NPO-48103) is currently available for download from the TSP library.

Please Login at the top of the page to download.