A computational process converts a sequence of digitized video images of the same scene into stabilized, coregistered images of an area of interest within the scene. The process corrects for motion of the area of interest or of the camera (as manifested by rotation, translation, and/or dilation of the raw images). Thus, there is minimal translation, rotation, or dilation in the sequence of output images of the area of interest. The stabilization and coregistration of images can facilitate scientific, engineering, or forensic analysis of the area of interest. Alternatively or in addition, the output images can be used to synthesize a single video image with reduced noise.
Older processes that were developed to serve the same purpose correct for translation but not for rotation or dilation. They are sensitive to effects of parallax (as manifested in differences between velocities of foreground and background objects). Most of them are not capable of resolving image displacements to resolutions finer than one pixel. The present process not only corrects for both rotation and dilation but is also less sensitive to parallax and can resolve motion and achieve registration to within a fraction of a pixel.
The process begins with the selection of an initial or reference video field - the key field - that includes the area of interest. The other fields in the sequence are known as test fields. The area of interest in the key field is identified and extracted for comparison with a corresponding area of the same size in each test field. Cross-correlations between the area-of-interest subimages of the key and test field are computed, and the translational offset (comprising horizontal and vertical displacements) between these subimages is estimated by selecting that offset that maximizes the correlation coefficient.
The areas of interest in the key and test fields are subdivided into blocks of pixels, for which cross-correlations are computed and offsets are estimated, using the previously estimated offset as initial estimates. The blocks are then further subdivided for computation of cross-correlations and offsets. The procedure of subdivision, cross-correlation, and offset estimation is repeated several times, yielding a hierarchy of block sizes (typically, down to smallest block size of 10 by 10 pixels) with corresponding correlation coefficients and offsets.
A data mask is constructed for the offsets at each level of the hierarchy to exclude those offsets that are deemed to be questionable because their correlation coefficients are below an arbitrary threshold value. The final offsets for the test field are calculated as weighted averages of the unmasked offsets for all block sizes; the weights are proportional to the sizes of the blocks. The data mask and the multiplicity of data from different parts of the area of interest help to reduce errors caused by parallax.
The rotation and dilation of the test field relative to the key field are estimated from the curls and divergences, respectively, of unmasked offset vectors. Statistical outliers (beyond one standard deviation) of curl and divergence values are masked out. The final offsets, rotation, and dilation are used to transform the test field into an output field that matches the key field in position, orientation, and magnification. The entire process is then repeated for each subsequent test field, using the offsets from the preceding field as initial guesses to reduce the ranges of offsets that must be searched for maximum correlation coefficients.
This work was done by David H. Hathaway and Paul J. Meyer of Marshall Space Flight Center.
This invention is owned by NASA, and a patent application has been filed. Inquiries concerning nonexclusive or exclusive license for its commercial development should be addressed to
Sammy Nabors
MSFC Commercialization Assistance Lead
at (256) 544-5226 or This email address is being protected from spambots. You need JavaScript enabled to view it.
Refer to MFS-31243.
