CC BY
17
МАТЕМАТИЧЕСКАЯ ФИЗИКА, МАТЕМАТИЧЕСКОЕ
МОДЕЛИРОВАНИЕ MATHEMATICAL PHYSICS. MATHEMATICAL MODELING
УДК. 004.932
IMAGE SUPER-RESOLUTION METHOD BASED ON A HOLE ARRAY MASK
USAGE
S.V. Blazhevich, E.S. Kaviyeva, Yu.P. Gladkih
Belgorod National Research University, Pobedy str., 85, Belgorod 308015, Russia
E-mail: [email protected]
Abstract
A super-resolution method based on the image registration by a photo-matrix located behind the mask with a hole array is proposed. The mask can take four different positions in relation to the photo-matrix. Each position allows the registration of one digital image using a quarter of each pixel in the matrix. Based on these images one image whose linear spatial resolution two time exceeds the resolution of each registered images is synthesized. The method is very resistant to noise and interference in the process of registration as well as to calculation errors.
Keywords: digital image, super-resolution, photo-matrix, mask with square holes
1. Introduction
Super-resolution methods [1-6] are aimed to increase digital image resolution up to the level exceeding the possibilities of photo-matrix used for its registration. A common feature of these methods is the usage of mathematical treatment of a group of digital images of an object to synthesize one image of higher resolution. Methods of super-resolution allow to obtain new information about the structure of the object on the basis of a group of its images differing one from another by sub-pixel spatial displacements. An optimization of super-resolution method implies the creation of the image registration conditions supplying the required sub-pixel displacements of the images relating to the photo-matrix. The problem of super-resolution is usually underdetermined. To perform the synthesis of high-resolution image, the additional conditions providing single-valued correspondence between the elements of the matrices of the registered images and the elements of the synthesized image are necessary. A setting manner of the boundary conditions supplementing the image synthesis problem was proposed in [7]. This method consists in creating zero boundary conditions for the synthesized image in the registration process by closing a part of each pixel located on the photo-matrix edges by an opaque screen during each exposure. The drawbacks of this method are high sensitivity to the noises and fast accumulation of the mistakes in the process of sequential calculations of pixel values. To decrease these effects, the splitting of the image by separated blocks with the boundary conditions forming for each block can be applied. The extreme result of such a splitting are the blocks containing one "big" pixel (pixel of photo-matrix) surrounded by pixels supplying the boundary conditions in the registration process. In the present work, such limiting variant of the method [7] is proposed for the case when the image with super-resolution built on the base of four images of an object registered by a photodetector array situated behind an opaque mask with a set of square holes.
2. Method essence
The main idea of the proposed method consists in using a mask placed before the photomatrix. The mask is a plane screen with a set of square holes. The holes have the same transverse dimensions as a photo-matrix pixel and are located on the screen against each second pixel in each row and in each column of the photo-matrix (see in Fig. 1). If we move the mask one half pixel along the rows and the columns of the photo-matrix the mask will open one fourth of each pixel of the photo-matrix for registration of the light which carries the image. Now if we move the mask one pixel along the rows or along the columns of the photo-matrix we will find only four similar positions corresponding to different locations of the mask holes relative to the photo-matrix pixels (see in Fig. 2).
nnnnn □□□nn □□□□□
■ 1
■ 1 ■
a
b
Fig. 1. Elements of the image registration system: 1 - the opaque mask with square holes;
2 - the photodetector array (photo-matrix). The photo-matrix is placed behind the mask, which can take
four fixed position in relation to the photo-matrix.
If we sequentially take off the photo-expositions at these four positions of the mask, we will obtain four matrixes of the signals appropriating to the image elements situated on each fourth of each photo-matrix pixel. Thus, we will obtain all the elements of the image whose linear resolution is two time higher than the resolution of the photo-matrix. Then we must only combine the elements into a new matrix. In the process of the photo-registration of the images the screen must take one of its four positions set by the transverse shift of the screen to distance of one pixel (see in Fig. 2) before each exposition. The method gives the linear resolution of the synthesized digital image which two times exceeds the resolution of each registered image. Moreover, this method is very resistant to noise and interference in the process of registration and to calculations errors, because the value of each pixel of matrixes of initial low resolution images is uniquely related to one of the pixels of the synthesized matrix and does not require any additional mathematical treatment.
To clarify the principle of the method let us take into consideration an imaginary 25-pixels photo-matrix consisting of five rows and five columns. Let us place a mask with nine square holes of the same transverse sizes as a pixel in front of the photo-matrix (see in Fig. 1). The holes in the mask are allocated in three rows. The distance between the rows is one pixel. The mask can take four different positions in relation to the matrix by shifting one pixel left or right, up or down. These positions are shown in Fig. 2, where the quarters of photo-matrix pixels, which remain
unclosed by the mask are marked by red color. We can see that in each position of the mask one quarter of each pixel is open for light and can register the corresponding part of the object image on the matrix. During these four exposures of the object made under corresponding positions of the mask each pixel of photo-matrix will register sequentially all four quarters of the image area corresponding to this pixel.
3 4
Fig. 2. The positions of the mask in relation to the photo-matrix used under registration of digital images of an object. In each position the mask opens a quarter of the surface of each photo-matrix pixel for the light carrying the image. All four positions of the screen allow to completely define the matrix
a of the super-resolution image We can define the connections between elements of matrixes b1, b2, b3 and b4 of the registered images of the photography object and the elements of the matrix a of the synthesized image using the Fig. 2:
These matrices allow us to construct the algorithm for the elements formation of the matrix a by renumbering elements of the matrices b1, b2, b3 and b4 of arbitrary size. The investigation of this task shows that to associate the elements of matrix a with elements of b-matrices two algorithms of index generation have to be used:
A B
Fig. 3. The index definition algorithms used in the main algorithm of the high-resolution image syntheses (matrix a) based on the registered matrices b1, b2, b3 and b4, where the index i belongs to the element of one of the matrices b1 - b4, index k belongs to the element of matrix a corresponding to it The index definition algorithms A and B (see in Fig. 3) realized by the means of the software package MathCad can be presented correspondently as programs k0(n) and k1(n):
The main algorithm of the super-resolution method constructed using these index generation algorithms can be presented in the form shown in Fig. 4, where k0(n) and k1(n) are the index definition algorithms presented in Fig. 3 or by programs (2):
Begin
/Input m; n /
' r
/bl, b2, b3, b4/
T
/kO(m), kl(m) /kQ(m),kl(n)/
akO(m){, kO(n)k " bli,k akO(m)i, kl(n)k " b2i,k akl(m)i,kl(n)k-b3i,k akl(m)i, kO(n)k " b4i,k
I
End
k=k+l
Fig. 4. The main algorithm of the considered super-resolution method
In the programming environment of the package MathCad the algorithm of the main program of the image synthesis with super-resolution takes the form of the program (3), where m and n define the dimensions of matrixes b1, b2, b3, b4 of the images registered by photo-matrix; a is the matrix of the synthesized image.
3. Demonstration of the super-resolution method using a digital image
modelling
To simulate the registration process of photography object images let us take the matrix a of the digital image "Mona Lisa" and build out of its elements four matrices b1, b2, b3 and b4, which will be used as input data for synthesis of the digital image a1 with super-resolution.
Fig. 5. The example of the image for simulation the process of the proposed method of super-resolution
(matrix a)
The matrixes b1, b2, b3 and b4 modelling the input (registered) images (see Fig. 6) will be obtained in the programming environment of the package MathCad by the programs:
i^h
bl b2 b3 bj
Fig. 6. The digital images modelling the images registered by the photo-matrix under different positions
of the mask placed before it
Now we will define the elements of the synthesized matrix a1 by the program (2) a1 = a(m, n) that completely recreates the elements of the matrix a.
Let us compare the synthesized image a1 with the registered one b1 (see in Fig. 7) and with a fragment of the averaged image (b1+b2+b3+b4)/4 (see in Fig. 8). These pictures demonstrate the workability of the proposed super-resolution method.
at bl + b2 + M + W
4
Fig. 8. The comparison between the fragments of the synthesized (a1) and the averaged "registered"
image (b1+b2+b3+b4)/4 4. Conclusion
In the present work, a motion-used variant of image super-resolution method, which allows two-time increase of image resolution is proposed. The method is based on four images of an object registered by a matrix of photodetectors (pixels) situated behind an opaque mask with a set of square holes. The method is very resistant to noise and interference in the process of registration and to calculations errors because the value of each pixel of the matrixes of the registered low-resolution images is in one-to-one correspondence with one of the pixels of synthesized matrix and does not require any additional mathematical treatment. The motions of the screen are performed only to the distance equal to the width of one photo-matrix pixel, fixing its position by calibrated limiters and therefore can be fulfilled performed in a very short time not increasing considerably the total exposure time.
Acknowledgements
The present work is supported by the Ministry of Education and Science of the Russian Federation (project of state task No 3.4877.2017/BY).
References
1. Brown L. 1992. A survey of image registration techniques. ACM Computing Surveys, 24(4): 325-376.
2. Chiang M. C. and Boulte T. E. 2000. Efficient super-resolution via image warping. Image and Vision Computing, 18(10):761-771.
3. Elad M. and Feuer A.1997. Restoration of single super resolution image from several blurred, noisy, and down-sampled measured images. IEEE Transaction on Image Processing, 6(12): 1646-1658.
4. Elad M. and Hel-Or Y. 2001. A fast super-resolution reconstruction algorithm for pure translational motion and common space invariant blur. IEEE Transactions on Image Processing, 10(8): 1187-1193.
5. Jianchao Yang, John Wright, Thomas Huang, and Yi Ma. 2008. Image super-resolution as sparse representation of raw image patches. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition: 1-8.
6. Protter M., Elad M., Takeda H., and Milanfar P. 2009 Generalizing the nonlocal-means to superresolution reconstruction. IEEE Transactions on Image Processing, 18(1): 36-51.
7. Blazhevich S.V., Selyutina E.S. 2012. Synthesis method of two-dimensional image matrix with super-resolution. Belgorod State University Scientific Bulletin. Mathematics & Physics,, 2012, 29, N 23 (142): 38-49. In Russian
