In order to perform precision landings for space missions, a control system must be accurate to within ten meters. Feature detection applied against images taken during descent and correlated against the provided base image is computationally expensive and requires tens of seconds of processing time to do just one image while the goal is to process multiple images per second.
To solve this problem, this algorithm takes that processing load from the central processing unit (CPU) and gives it to a reconfigurable field programmable gate array (FPGA), which is able to compute data in parallel at very high clock speeds. The workload of the processor then becomes simpler; to read an image from a camera, it is transferred into the FPGA, and the results are read back from the FPGA.
Harris Corner Detector uses the determinant and trace to find a “corner score,” with each step of the computation occurring on independent clock cycles. Essentially, the image is converted into an x and y derivative map. Once three lines of pixel information have been queued up, valid pixel derivatives are clocked into the product and averaging phase of the pipeline. Each x and y derivative is squared against itself, as well as the product of the ix and iy derivative, and each value is stored in a W×N size buffer, where W represents the size of the integration window and N is the width of the image. In this particular case, a window size of 5 was chosen, and the image is 640×480.
Over a W×N size window, an equidistance Gaussian is applied (to bring out the stronger corners), and then each value in the entire window is summed and stored. The required components of the equation are in place, and it is just a matter of taking the determinant and trace. It should be noted that the trace is being weighted by a constant κ, a value that is found empirically to be within 0.04 to 0.15 (and in this implementation is 0.05). The constant κ determines the number of corners available to be compared against a threshold σ to mark a “valid corner.”
After a fixed delay from when the first pixel is clocked in (to fill the pipeline), a score is achieved after each successive clock. This score corresponds with an (x,y) location within the image. If the score is higher than the predetermined threshold σ, then a flag is set high and the location is recorded.
This work was done by Arin C. Morfopoulos and Brandon C. Metz of Caltech for NASA’s Jet Propulsion Laboratory. For more information, download the Technical Support Package (free white paper) at www.techbriefs.com/tsp under the Electronics/Computers category. NPO-47202
This Brief includes a Technical Support Package (TSP).
Rapid Corner Detection Using FPGAs
(reference NPO-47202) is currently available for download from the TSP library.
Don't have an account?
Overview
The document titled "Rapid Corner Detection Using FPGAs" outlines a novel approach to feature detection in images, specifically aimed at enhancing precision landing systems for spacecraft during the Entry, Descent, and Landing (EDL) phases on other planets. Developed by researchers at NASA's Jet Propulsion Laboratory, this work addresses the need for a control system capable of achieving accuracy within 10 meters, which is critical for successful landings.
The primary focus of the research is the implementation of a Harris feature detector using Field Programmable Gate Arrays (FPGAs). Traditional methods of feature detection are computationally intensive and can take tens of seconds to process a single image, which is impractical for real-time applications during EDL. The FPGA-based solution allows for massively parallel processing, enabling the system to handle one pixel per clock cycle. This results in significantly reduced processing times, allowing a 640x480 image to be processed in as little as 0.9 milliseconds, compared to the seconds required by conventional processors.
The algorithm employed is a variation of the Harris corner detection method, which typically involves calculating Eigen values from image derivatives. However, the FPGA implementation simplifies this by using a predetermined threshold to generate a score for each window of interest in the image. If the score exceeds the threshold, the location is flagged as a corner. This approach not only speeds up the detection process but also minimizes the load on the main processor, allowing it to focus on simpler tasks such as reading images and transferring data to and from the FPGA.
The document emphasizes the relevance of this technology to NASA's future missions, particularly in the context of Mars exploration, where current CPU capabilities are insufficient for the computational demands of real-time feature detection. The FPGA solution offers a low-power, efficient alternative that can operate within the constraints of existing hardware.
In summary, the research presents a significant advancement in image processing for aerospace applications, leveraging FPGA technology to enable rapid and efficient corner detection, thereby enhancing the capabilities of spacecraft during critical landing operations. This work not only contributes to the field of aeronautics but also has broader implications for various technological and commercial applications.
