A proposed complementary metal oxide/semiconductor (CMOS) integrated-circuit imaging device of the active-pixel sensor (APS) type is capable of simultaneously generating the pixel-by-pixel sum and difference of two successive image frames at high speed for real-time operation, while compensating for leakage and suppressing fixed pattern noise (FPN). Frame differencing (and, to some extent, summing) is used in a variety of image-processing applications, including surveillance, range mapping, detection of motion, discrimination against background, compression of image data, and for solar-Doppler magnetographs.
Heretofore, it has been common practice to compute frame differences in software after acquiring image data, but such computation tends to be too slow for real-time operation and to contribute undesirably to the complexity and power demand of an image-data-processing system. In an effort to overcome these disadvantages of frame differencing in software, APS circuits that incorporate capacitive memory elements and associated frame-differencing circuitry within pixels have been demonstrated. Unfortunately, these circuits have been susceptible to unwanted coupling of present-frame pixel signals into pixel memory elements and consequent corruption of the stored previous-frame pixel values. Often, true frame differences are smaller than amounts of leakage, so that the very purpose of frame differencing is defeated. The problem of correcting frame-difference errors is compounded by a need to avoid contributing to FPN through inaccurate subtraction of pixel offsets. Regarding on-chip summing of successive frames: heretofore, there has been no circuitry capable of performing this function.
The proposed imager would include pixel circuitry like that of prior CMOS APS imagers. It would differ from the prior imagers by incorporating redesigned column and global signal-processing circuits that would contain switched-capacitor gain stages (see figure on page 22a). These redesigned circuits would (1) compute frame signals differentially to eliminate FPN; (2) provide different gains for present and preceding frame signals after FPN has been eliminated, in order to compensate for leakage; and (3) sum successive frames by use of modified gain-stage timing.
During exposure of the present frame, charge from the present frame leaks (by optical coupling or diffusion) into the in-pixel storage capacitor,Cp, corrupting the signal stored during the preceding frame. The effect of the leakage is equivalent to that of multiplying the present- and previous-frame signals by different voltage gains prior to the analog computation of the difference between them. Hence, the effect of leakage could be compensated by use of modified gains; in particular, if d is the fraction of signal charge from the present frame that leaks into Cp, then one could compensate for the leakage by making the gain for the present frame 1 + d times that for the preceding frame.
In the proposed circuit, the outputs from the pixels in a given row would be sampled simultaneously at the bottoms of the corresponding columns in column sample-and-hold capacitors (CS1 and CS2). These capacitors would then be successively switched into the global switched-capacitor gain stage, which would include operational amplifiers, feedback capacitors (CIS1 and CIS2), and transistor switches. The gain in each branch would be governed by the ratio between its capacitances. These ratios (CS1/CIS1 and CS2/CIS2, respectively) would be chosen to obtain the different gains needed to compensate for leakage. The gains could be varied by changing capacitor sizes, by digital switching for parallel connection or disconnection of capacitors in increments of 0.01 — a base capacitance.
In order to suppress FPN, it is necessary to effect pixel-to-pixel offset correction and suppression of flicker noise, and this necessitates equal gains. In the absence of a further correction, the use of different gains to compensate for leakage would contribute to FPN. In the proposed circuit, this correction would be effected by reading out each frame pixel value differentially with respect to an externally generated reference potential (Vrefc), and storing the differential value in the charge domain on capacitors. Thus, offset would be corrected for each frame pixel value independently. The offset-corrected charge would then be appropriately amplified by the global switched-capacitor gain stage.
Generation of the sum of pixel values from two successive frames would involve sampling in different ways during the two frames. The signal from the preceding frame, already stored in Cp, would be captured by sampling that signal first, followed by the reset. Inasmuch as the same reset level would be used for computation of the pixel value of the present frame, sampling of the current frame would begin with sampling of the reset voltage first, followed by sampling of the signal voltage. As a result, signals of opposite polarities would be generated in the two branches. Thus, when the difference between global differential outputs is computed, the result would be the sum of successive frame pixel values instead of the difference between them. Additional capacitors (CRH and CSH) would be needed in every column in order to store all four samples for a given pixel while individual columns were scanned out. Measured results indicate residual error of <1 percent, which is constant across the entire illumination range.
This work was done by Bedabrata Pain, Guang Tang, Monico Ortiz, and Suresh Seshadri of Caltech for NASA's Jet Propulsion Laboratory.
In accordance with Public Law 96-517, the contractor has elected to retain title to this invention. Inquiries concerning rights for its commercial use should be addressed to
Technology Reporting Office
JPL
Mail Stop 249-103
4800 Oak Grove Drive
Pasadena, CA 91109
(818) 354-2240
Refer to NPO-21049
This Brief includes a Technical Support Package (TSP).
Leakage-Compensated, Frame-Differencing APS Imager
(reference NPO21049) is currently available for download from the TSP library.
Don't have an account?
Overview
The document discusses a novel complementary metal oxide/semiconductor (CMOS) integrated-circuit imaging device of the active-pixel sensor (APS) type, developed for high-speed, real-time image processing. This device is capable of simultaneously generating the pixel-by-pixel sum and difference of two successive image frames while effectively compensating for leakage and suppressing fixed pattern noise (FPN).
Traditionally, frame differencing has been performed in software after image acquisition, which is often too slow for real-time applications and adds complexity and power demands to image processing systems. The proposed APS device addresses these challenges by incorporating capacitive memory elements and advanced circuitry within the pixels, allowing for on-chip computation of frame differences and sums. This innovation is particularly beneficial for applications such as surveillance, motion detection, range mapping, and image data compression.
The document highlights the technical problems associated with frame differencing, particularly the issue of charge leakage from the current frame into the in-pixel storage capacitor that holds the previous frame value. This leakage can corrupt the previous frame data, leading to inaccurate frame difference calculations. The proposed solution involves redesigning the column and global signal-processing circuits to compute frame signals differentially, thereby eliminating FPN and allowing for variable gain adjustments to compensate for leakage.
The device operates by sampling the previous frame's signal and the current frame's reset voltage in a specific sequence, generating signals of opposite polarities. This method enables the computation of the sum of successive frame pixel values rather than their difference, resulting in high accuracy and low residual error across various illumination conditions.
The document also emphasizes the novelty of this technology, which allows for high-contrast and low-smear frame difference generation, facilitating effective feature detection that would otherwise be obscured by noise. The work was conducted by researchers at Caltech for NASA’s Jet Propulsion Laboratory, and the findings are intended for dissemination through NASA Tech Briefs to reach both government and commercial sectors interested in advanced imaging solutions.
In summary, this document presents a significant advancement in imaging technology, promising enhanced performance for real-time applications through innovative on-chip processing techniques.
