A bilateral subtraction filter has been implemented as a hardware module in the form of a field-programmable gate array (FPGA). In general, a bilateral subtraction filter is a key subsystem of a high-quality stereoscopic machine vision system that utilizes images that are large and/or dense. Bilateral subtraction filters have been implemented in software on general-purpose computers, but the processing speeds attainable in this way — even on computers containing the fastest processors — are insufficient for real-time applications. The present FPGA bilateral subtraction filter is intended to accelerate processing to real-time speed and to be a prototype of a link in a stereoscopic-machine-vision processing chain, now under development, that would process large and/or dense images in real time and would be implemented in an FPGA.
The present FPGA implementation of a bilateral subtraction filter utilizes a 9×9 window. This implementation was designed to take advantage of the ability to do many of the component computations in parallel pipelines to enable processing of image data at the rate at which they are generated. The filter can be considered to be divided into the following parts (see figure):
- An image pixel pipeline with a 9×9-pixel window generator,
- An array of processing elements;
- An adder tree;
- A smoothing-and-delaying unit; and
- A subtraction unit.
After each 9×9 window is created, the affected pixel data are fed to the processing elements. Each processing element is fed the pixel value for its position in the window as well as the pixel value for the central pixel of the window. The absolute difference between these two pixel values is calculated and used as an address in a lookup table. Each processing element has a lookup table, unique for its position in the window, containing the weight coefficients for the Gaussian function for that position. The pixel value is multiplied by the weight, and the outputs of the processing element are the weight and pixel-value·weight product. The products and weights are fed to the adder tree. The sum of the products and the sum of the weights are fed to the divider, which computes the sum of products ÷ the sum of weights. The output of the divider is denoted the bilateral smoothed image.
The smoothing function is a simple weighted average computed over a 3×3 subwindow centered in the 9×9 window. After smoothing, the image is delayed by an additional amount of time needed to match the processing time for computing the bilateral smoothed image. The bilateral smoothed image is then subtracted from the 3×3 smoothed image to produce the final output.
The prototype filter as implemented in a commercially available FPGA processes one pixel per clock cycle. Operation at a clock speed of 66 MHz has been demonstrated, and results of a static timing analysis have been interpreted as suggesting that the clock speed could be increased to as much as 100 MHz.
This work was done by Andres Huertas, Robert Watson, and Carlos Villalpando of Caltech and Steven Goldberg of Indelible Systems for NASA’s Jet Propulsion Laboratory. NPO-45906
This Brief includes a Technical Support Package (TSP).
Hardware Implementation of a Bilateral Subtraction Filter
(reference NPO-45906) is currently available for download from the TSP library.
Don't have an account?
Overview
The document titled "Hardware Implementation of a Bilateral Subtraction Filter" from NASA's Jet Propulsion Laboratory outlines the development and implementation of a bilateral subtraction filter designed for real-time stereo processing applications. This filter is particularly useful in enhancing image quality by preserving edges while reducing noise, which is crucial for applications such as autonomous navigation and robotic vision.
The bilateral filter operates by computing a weighted average of pixels within a neighborhood, where the weights depend on both spatial and intensity differences between the central pixel and its neighbors. This method allows for effective noise suppression without blurring edges, addressing a common issue in traditional filtering techniques that can lead to artifacts like ringing around edges.
The implementation utilizes a 9x9 pixel kernel in Verilog, optimized for the Xilinx Virtex4 family FPGA. The design is structured into three main components: an image pixel pipeline that generates a 9x9 pixel window, an array of processing elements that perform the core computations, and an image smoother that applies a simple 3x3 center-weighted average. The processing elements utilize lookup tables to apply Gaussian weight coefficients based on the absolute differences between pixel values, allowing for adaptive filtering based on local image characteristics.
The document also discusses the advantages of using the bilateral filter over traditional band-pass filtering methods, which often introduce blurring across image discontinuities. By avoiding this blurring, the bilateral filter enhances the quality of disparity estimation in stereo vision systems, making it a valuable tool for real-time applications.
Additionally, the document highlights the importance of computational efficiency in the design, ensuring that the filter can process images at the same rate as the imager outputs data. This efficiency is achieved through parallel processing and deep pipelining, which are critical for maintaining real-time performance in dynamic environments.
Overall, the bilateral subtraction filter represents a significant advancement in image processing technology, combining effective noise reduction with edge preservation, and is positioned to improve the performance of stereo vision systems in various aerospace and robotic applications. The work exemplifies NASA's commitment to developing technologies with broader scientific and commercial applications.
