Limits of multi-frame image enhancement: a case of super-resolution

Limits of multi-frame image enhancement: a case of super-resolution

A common and important problem that arises in visual communications is the need to create an enhanced-resolution video image sequence from a lower resolution input video stream. This can be accomplished by exploiting the spatial correlations that exist between successive video frames using Super-Resolution (SR) reconstruction. SR refers to the task of increasing the spatial resolution through multiple frame processing.
Multi-frame resolution enhancement methods are of increasing interest in digital image processing and there has been a substantial amount of research in developing algorithms that combine a set of low-quality images to produce a set of higher quality images. Either explicitly
or implicitly, such algorithms must perform the common task of registering and fusing the lowquality image data. While many such processes have been proposed, very little work has
addressed their limits.
In this context, an algorithm designed to operate in the spatial domain is used in a controlled test to compute a higher-resolution image by mapping a model of the image formation process using local sub-pixel shifts among the lower resolution and compressed images of the same scene.
These shifts are determined by way of a rigorous least-squares area-based image-matching scheme that does not require control points.
Statistical results show that the performance of the algorithm does degrade, as would be expected, depending on (1) the amount of noise present in the low-resolution images, (2) the number of low-resolution input images and (3) the magnification factor required to meet resolution requirements.

This publication is copyright. It may be reproduced in whole or in part for the purposes of study, research, or review, but is subject to the inclusion of an acknowledgment of the source. This article was peer reviewed by two independent and anonymous reviewers.