# GRADIENT CONTROLLED IMAGE MAGNIFICATION.

Byline: Tabinda Sarwar and Wajiha HabibABSTRACT

One of the most common methodologies to obtain a high resolution image is enlargement of a low-resolution image. Different image interpolation techniques are used for image enlargement. A typical problem with most interpolation techniques is that although smoothing the pixels and keeping the low frequencies in the new zoomed images, they are not able to enhance the high frequencies or preserve the edges equally well. Visually those problems will result in undesirable artifacts like blurring, aliasing or blocking effects. In this paper, an algorithm has been proposed that works in a locally adaptive way, sensing and preserving the edges. By preserving the edges sharpness and applying gradient controlled smoothness mechanism, a quality high-resolution image can be obtained free of aliasing effect. The quantitative and quantitative results demonstrate the superiority of the algorithm over other representative interpolation methods.

Keywords: Magnification, Up-Scaling, Edge-directed Interpolation, High-Resolution Image, Digital Image Processing

1. INTRODUCTION

Given a low-resolution (LR) image, a magnification technique can provide high-resolution (HR) image without much changes in the visual contents of the original image [1]. Enhancing the resolution of an image is complicated due to the localized high-frequency nature of the pixel intensities across edges. To assign a suitable intensity value to unknown pixels in HR image is the task of interpolation technique which is used in magnification [2]. Many traditional techniques are present for the purpose of magnification. Bilinear Interpolation determines the value of a new pixel based on piecewise linear function and bicubic interpolation determines the new pixel value on the basis of cubic function [3]. Figure 1 represents the comparison of both techniques. Other methods, using the B-spline interpolators [4, 5] or the cubic convolution methods [6] have also been proposed. However, all these methods tend to blur the edges and are prone to ringing and aliasing effects.

The basic idea of Edge-Directed Interpolation techniques is to analyze edge information in the LR image in order to aid in the interpolation step [1, 7-22]. Johan and Nishita [1] proposed a method that consists of two steps: first generation of an initial magnified image and second progressively refining the magnified image to produce a high quality HR image. Allebach and Wong [7] used a Laplacian-of-Gaussian filter to create a low-resolution edge map. This low- resolution map is then interpolated to a high-resolution edge map. In the rendering (interpolation) step, they modified bilinear interpolation in order to force interpolation not to cross edges in the edge map. Battiato, Gallo, and Stanco [8] used an adaptive interpolation, which preserves edges information and magnifies the input image. The magnified image maintains its sharpness and smoothness up to some extent.

Li and Orchard [9] proposed an edge-direct interpolation algorithm that estimates the local covariance coefficients estimates from a LR image and then use these estimates to adapt the interpolation at HR image based on the geometric duality between the LR and HR covariances. K. Jensen, D. Anastassiou [10] proposed a non-linear interpolation in which a small neighborhood about each pixel in LR image is first mapped to a best-fit continuous space step edge and then bi-level approximation serves as a local template on which the HR sampling grid can then be superimposed.

Sajjad, Khattak and Jafri [11] proposed an interpolation method that calculates threshold, using which it classify interpolation region in the form of geometrical shapes and then assign suitable values to undefined pixels inside interpolation region while preserving the sharp luminance variations and smoothness. Giachetti and Asuni[12] proposed an algorithm in which first an adaptive algorithm is applied, interpolating locally pixel values along the direction where second order image derivative is lower and then interpolated values are refined iteratively minimizing differences in second order image derivatives, maximizing second order derivative values and smoothing isolevel curves.

He He and Siu [13] predicted the unknown value of HR image by its neighbors through the Gaussian process regression. According to Chughtai and Khattak[14], HR image is created by considering the discontinuities and luminance variations within the LR image and the pixels near the edges are diffused into the edges to reduce aliasing effect. Zhang and Wu [15] proposed an edge-guided non-linear interpolation technique that uses directional filtering (two orthogonal directions) and data fusion (based on linear minimum mean square-error estimation).

Xu and Kin [16] produced a HR image in a way that the pixel value of the LR image is based on the desired changes for its gradient profile. Demirel and Anbarjafri [17] used the stationary and discrete wavelets processing of LR image to preserve the edge information in HR image. Su and Ward [18] studied the relationship between the wavelet approximation sub-image of each edge in LR image and its wavelet detail sub-images. This relationship is then used to predict the edge information of HR image. Wang and Gong[19] used the connected-component labeling concept to enhance the edges of an HR image. Shao and Wei [20] proposed an image enlargement technique utilizing a filtering-based implementation scheme and regularization through coupling bilateral filtering. Shan, Li, Jia and Tang [21] provide an up-sampling algorithm that consists of a feedback loop to recover the HR image information from input LR image.

Morse and Schwartzwald [22] considered smoothing the isophotes of the HR ima ge to reduce the undesirable artifacts. Zhou and Shen [23] improved the traditional cubic convolution method by incorporating directional based interpolation method.

Many reconstruction and learning based magnification techniques have been proposed e.g. Li and Peng [24] proposed an algorithm that learns the gradient field and combine the horizontal gradient map with vertical gradient map to produce HR image and Glasner, Bagon and Irani [25] used the combination of two approaches: multi-image and exampled based super resolution technique, but these techniques require prior information (in form of multiple images or LR to HR mapping information). As every image cannot have prior information, so these techniques are not always successful.

In general, the mentioned methods can improve the visual quality of the HR images but at the expense of higher computational complexity. The HR images produced using them have the sharper edges but are prone to undesirable artifacts for images with complex edges.

In this paper, we aim to develop a magnification algorithm that will preserve fine details of an image while minimizing the computational complexity. The proposed technique is based on edge base interpolation which considers the sharp luminance variation and smoothness of the original image during interpolation, using which it produces a high quality magnified image. Interpolation technique used in the proposed adopts itself according to regional information of the image and assign proper intensity values to undefined pixels in the concerned region, in order to reduce information loss and thus aliasing effect will be reduced.

The remainder of the paper is organized as follows. Section 2 presents the proposed methodology (sub-sections 2.1 represents the expansion phase and sub-section 2.2 represents the interpolation phase). Qualitative and quantitative results of the proposed methodology are discussed in section 3. Section 4 concludes the paper.

2. PROPOSED ALGORITHMS

The proposed algorithm consists of two main phases: Expansion and Interpolation. Interpolation is further divided into three sub-phases: edge sharpness preservation, gradient control smoothing and gradient controlled filling. Figure 2 represents the flowchart oft proposed algorithm.

2.1 Expansion Phase

First phase of the magnification process is Expansion". In this phase, a LR image I of size (n x m) where n for rows and m for columns, is expanded to HR image Z of size (2n- 1 x2m-1). It introduces undefined pixels in between rows and columns of the HR image. Assigning a proper intensity values to these undefined pixels is job of Interpolation process which will be discussed in section 2.2. Let I (i, j ) denotes a pixel in ith row and jth column of LR image I and Z(l,k) denote a pixel in lth row and kth column of HR image Z. Expansion from image I to image Z is given by equation (1).

Equation

Expansion process is diagrammatically explained in figure 3. Figure 3(a) shows an original image I of size (5x5). In LR image I, all pixels have some intensity value assigned to them. Expansion explained in equation (1) and (2) is used to map image I to expanded image Z of size (9 x 9), represented in figure 3(b). In figure 3, black color represents known pixel values of I and white pixels represent undefined pixels. They will be assigned appropriate intensity values using interpolation explained in section 2.2.

2.2 Interpolation Phase

Interpolation phase is the most important phase of any magnification algorithm as the visual quality of the image depends on the accuracy of the interpolation. Interpolation phase consists of three sub-components

2.2.1 Edge Sharpness Preservation

In this phase the interpolator assign value to the undefined pixel depending upon the defined edges. Horizontal and vertical edge direction is found out by taking a 5x5 matrix assuming that edge is passing from the center pixel and for diagonal edges 9x9 matrices are assumed. In this edge preservation phase of the algorithm, approximately 85% undefined pixels of HR image are assigned proper intensity values. An assumption is always made that edge is passing from the center pixel. A 5x5 matrix is made around that pixel say it C2(x,y) as shown in the figure 4. In figure 4, black circles indicate the known pixel values (from LR image); using those values type of edge direction is detected. Keeping in view figure 4, equation 3 is derived. The equations remain same for all types of edges and only differ in the pixel values and positions.(3)

In these equations Ai, Bi and Ci are the values of pixels in HR image where i" varies from 1 to 3 (representing different pixel position and values) and a and AY are variables whose value will determine the edge classification.

There are four types of edges which are shown in figure 5 and 6.

Considering figure 5(a), equation 4 (for vertical edge) is derived from equation 3.

Equation

If equation 4 remains true (with an error of +5%) for Ai, Bi and Ci (where i varies from 1 to 3), then the edge is passing vertically. Value of a+AY is found out using the known values of Ai, Bi and Ci using equation 5.

Equation

The ratio a+AY remain same for all values of Ai, Bi and Ci (with an error of +5%). Then by using all possible combinations of a" and AY" (as grayscale value range from 0 to 255, these combination of a" and AY" will be verified) in equation 4, that combination is selected which satisfies equation 4.

After an edge has been detected, the unknown pixels that exist between the known pixels their value are calculated. Considering pixel P1 (whose location is (x-1, y-2)) and P2 (whose location is (x-1, y)), their value is calculated using the equation 6 and 7. Similarly other unknown pixels that lie between the known pixels are calculated.

Equations

Using figure 5(b), 6(a) and 6(b) as a reference vertical, left diagonal and right diagonal edge can be detected using equation 3, 4 and 5.

Area of LR image i.e. LR unit cell where there is no response of edge after applying equations 3, 4 and 5 (result equal to zero or infinity). This is called a constant region with no point of high contrast as shown in the Figure 6. In the case of Z(x- 2, y-1), value of P" (as shown in figure 6) is calculated using equation 8.

Equation

2.2.2 Gradient Controlled Smoothing

In this phase, the interpolator is responsible for smoothing the HR image. The algorithm scans the image again and sense the smoothness based on the lattice formed around the unknown pixels, as shown in figure 7.

Two scenarios are considered; first if c1 and c2 are not assigned (in both forms of the figure 7(a-b)), then the intensity differences of the original pixels A and B is calculated. If this difference less than a specified threshold then it assigns the average value of both original pixels to P otherwise leaves it undefined. Second if when both c1 and c2 are defined then it also calculates the difference of c1 and c2 and average value of those pixels is assigned to P whose difference is least.

A human eye perceive a change in color (for grayscale images) if a difference of 16 gray-level values exist. So to find the appropriate threshold, values from 1 to 16 were tested on 50 images obtained from [29], using Peak Signal-to- Noise Ratio (PSNR) measured in decibel (db) as the evaluation criteria. Values below 8 and greater than 14 produced very low values of PSNR. Average value of PSNR for 50 test images against threshold value from 8 to 16 are represented in figure 8.

2.2.3 Gradient Controlled Filling

In last phase of the proposed algorithm the holes are filled (undefined pixels left up to this phase). In this phase of algorithm, the median is calculated of the neighbouring defined pixels of the undefined pixel and then assign this calculated value to the undefined pixel. It may happen that all the b-neighbour pixel values are not known, in such case only those pixel values are considered for median whose value is defined. So in this way the more frequent value can be selected intead of generalized value of surrounding pixels.

3. QUANTITAIVE AND QUALITATIVE RESULTS The proposed method was compared with the other four representative interpolation methods, which are directional filtering and data fusion (DFDF) [15],new edge-directed interpolation (NEDI) [9], iterative curvature based interpolation (ICBI) [12] and directional cubic convolution (DCC) [25], The Matlab codes of DFDF, NEDI, ICBI and DCC methods were available from the original authors. These methods were used with their default settings. For thoroughness and fairness of our comparison study, we selected the 50 test images from [29] but due to the limitation of space only eight representative test images are shown in figure 9(a-h). We down-sampled the original HR grey images by a factor of two (in both row and column dimensions) to get the LR images. Using these LR images, desired HR images were reconstructed by mentioned methods.

Since the original HR images are known, we can measure the Structural SIMilarity (SSIM) [26-27] and PSNRs [28] of the zoomed HR images. After the evaluation of quantitative results, the evaluation of the visual appearance (qualitative evaluation) of the interpolated images is required for different methods. Figure 10 and 11 represents two times magnification of Fruit and Bicycle image. You can see that the edges of both the images for proposed methodology are stronger than the remaining methodology.

Tab. 1. SSIM results of the reconstructed HR images by different methods for 2 times magnification

###Method###Test Images

###Fruit###Bike###Rings###Monarch###Lena###Parrot###Airplane###Flower

###DCC###0.9452###0.9716###0.9533###0.9612###0.9669###0.9732###0.9432###0.9612

###DFDF###0.9581###0.9733###0.9612###0.9698###0.9687###0.9798###0.9530###0.9675

###NEDI###0.9770###0.9810###0.9731###0.9712###0.9763###0.9822###0.9912###0.9734

###ICBI###0.9882###0.9881###0.9889###0.9861###0.9789###0.9878###0.9942###0.9781

###Proposed###0.9915###0.9903###0.9932###0.9878###0.9893###0.9902###0.9957###0.9899

Tab. 2. PSNR (dB) results of the reconstructed HR images by different methods for 2 times magnification

###Method###Test Images

###Fruit###Bike###Rings###Monarch###Lena###Parrot###Airplane###Flower

###DCC###25.41###26.92###26.41###26.96###28.53###27.98###22.73###26.98

###DFDF###26.87###27.65###26.92###27.12###28.97###28.32###23.53###27.59

###NEDI###27.95###28.87###27.82###27.73###29.34###28.78###26.86###28.45

###ICBI###29.77###29.12###28.99###28.58###30.12###29.89###28.51###29.87

###Proposed###31.59###30.27###30.59###29.28###31.59###31.73###29.94###31.78

Tab. 3. SSIM results of the reconstructed HR images by different methods for 4 times magnification

###Method###Test Images

###Fruit###Bike###Rings###Monarch###Lena###Parrot###Airplane###Flower

###DCC###0.9481###0.9689###0.9599###0.9712###0.9632###0.9641###0.9455###0.9524

###DFDF###0.9570###0.9712###0.9668###0.9763###0.9689###0.9684###0.9532###0.9589

###NEDI###0.9631###0.9783###0.9701###0.9788###0.9732###0.9771###0.9809###0.9621

###ICBI###0.9824###0.9810###0.9798###0.9813###0.9789###0.9789###0.9865###0.9683

###Proposed###0.9875###0.9822###0.9881###0.9838###0.9811###0.9810###0.9882###0.9787

Tab. 4. PSNR (dB) results of the reconstructed HR images by different methods for 4 times magnification

###Method###Test Images

###Fruit###Bike###Rings###Monarch###Lena###Parrot###Airplane###Flower

###DCC###23.12###25.23###24.02###26.12###26.12###26.23###21.92###26.98

###DFDF###23.96###25.87###24.89###26.89###26.76###26.88###22.87###27.43

###NEDI###24.87###26.32###25.95###27.32###27.12###27.32###23.48###27.98

###ICBI###27.97###26.79###26.21###27.77###27.68###27.89###24.97###28.32

###Proposed###28.63###27.93###27.89###28.02###28.13###28.35###26.23###29.01

Tab. 5. Processing Time (seconds) of the reconstructed HR images by different methods for 2 and 4 times magnification

###Test###Method

###Images###DCC###DFDF###NEDI###ICBI###Proposed

###2 times###4 times###2 times###4 times###2 times###4 times###2 times###4 times###2 times###4 times

###zoom###zoom###zoom###zoom###zoom###zoom###zoom###zoom###zoom###zoom

###Fruit###2.87###6.96###5.74###20.99###12.26###36.64###89.75###141.94###1.33###3.06

###Bike###3.35###6.48###7.59###20.21###11.76###51.07###75.28###288.74###1.22###3.81

###Rings###0.25###0.84###0.54###0.55###0.61###2.79###4.54###15.44###0.13###0.41

###Monarch###2.82###7.27###8.93###21.21###11.88###31.32###78.31###181.45###1.42###3.53

###Lena###2.72###6.88###7.72###19.47###12.56###43.54###49.86###86.73###1.43###3.97

###Parrot###2.88###7.68###7.63###19.85###13.09###26.10###54.83###109.73###1.59###3.22

###Airplane###4.07###7.01###10.13###19.35###11.46###43.77###70.47###186.79###1.38###3.49

###Flower###2.82###7.03###8.25###19.06###12.43###41.78###72.97###168.87###1.81###2.76

Figure 11 and 12 represents four times magnification of Rings and cropped Monarch image. The border effects for DCC, DFDF and NEDI are much strong and aliasing effect has been introduced for ICBI. In figure 12, DCC and DFDF though produce smooth results (excluding the borders) but the HR images are blurred as compared to the proposed methodology.

Figure 13 represents the average value of SSIM and PSNR for 50 test images (both 2 and 4 times magnification were considered) for five different methods. The proposed methodology has the highest value of SSIM and PSNR.

4. CONCLUSION

This research paper proposes an image magnification technique which produces a HR image of high visual quality from LR image. The primary objectives of the digital image magnification technique are to produce an image of size double while maintaining the sharp luminance variation and smoothness of the original image. The magnified image should be free from common artifacts include blurring, blocking and ringing. The proposed algorithm has two main phases, expansion and interpolation. Interpolation itself consists of three sub-phases, preservation of edges sharpness, gradient control smoothing and gradient control filling. . The strength of the interpolation is that all the sub-phases of interpolation consider the edge direction thus preserving the edges information to an extent. One of the limitations of the algorithm is that it doesnt consider or detects noise in an image. So if a LR image has noise than its HR images quality is affected.

The proposed technique is quite competitive as can be seen from the qualitative and quantitative results. Besides producing a good quality HR image, its processing speed is also fast. Only grayscale images were considered, but for colored images further research is required.

REFERENCES

[1] Johan, H., Nishita, T.: A Progressive Refinement Approach for Image Magnification, Proc. of the 12th Pacific Conf. on Computer Graphics and Applications, Seoul, South Korea, pp. 351-360, October 2004.

[2] Gonzales, R.C, Woods, R.E.: Digital Image Processing, Pearson Prentice Hall, Eleventh Indian Reprint, 2005

[3] Parker, J.A, Troxel, D.E: Comparison of Interpolating Methods for Image Sampling, IEEE Transactions on Medical Imaging, Vol. MI-2, No.1, pp. 31-39, 1983.

[4] Hou, H.S, Andrews, H.C: Cubic Splines for Image Interpolation and Digital Filtering, IEEE Transactions on Acoustics, Speech, Signal Processing, ASSP-26(6), pp. 508517, 1978.

[5] Lee, S.W, Paik, J.K: Image Interpolation using Adaptive Fast B-Spline Filtering, Proc. of IEEE Intl. Conf. on Acoustics, Speech, and Signal Processing, vol. 5, pp. 177179, 1993.

[6] Keys, R.G: Cubic convolution interpolation for digital image processing, IEEE Transactions on Acoustics, Speech, Signal Processing, ASSP-29(6), pp. 11531160, 1981.

[7] Allebach, J., Wong, P.W: Edge-Directed Interpolation, Proc. of IEEE Intl. Conf. on Image Processing, Vancouver, Canada, pp. 707710, September 2000.

[8] Battiato, S., Gallo, G., Stanco, F.: A New Edge- Adaptive Zooming Algorithm for Digital Images, In Proc. Signal Processing and Communication, Spain, pp. 144-149, 2000.

[9] Li, X., Orchard, M.T.: New Edge-Directed Interpolation, IEEE Transactions on Image Processing, vol. 10, issue 10, pp. 15211527, 2001.

[10] Jensen, K., Anastassiou, D.: Subpixel Edge Localization and the Interpolation of Still Images, IEEE Transactions on Image Processing, vol. 4, issue 3, pp. 285295, 1995.

[11] Sajjad, M., Khattak, N., Jafri, N.: Image Magnification Using Adaptive Interpolation by Pixel Level Data- Dependent Geometrical Shapes, Intl. Journal of Computer Science and Engineering,, vol. 1, no. 2, pp. 1915-1924, 2007.

[12] Giachetti, A. and Asuni, N.: Fast artifacts-free image interpolation, Proc. of the British Machine Vision Conference, Leeds, UK, pp. 123132, 2008.

[13] He, H., Siu, W.: Single Image Super-Resolution using Gaussian Process Regression, Proc. of IEEE Computer Vision and Pattern Recognition, Colorado Springs, U.S., pp. 449-456, June 2011.

[14] Chughtai, M.A., Khattak, N.: An Edge Preserving Adaptive Anti-aliasing Zooming Algorithm with Diffused Interpolation, IEEE Proc.of the 3rd Canadian Conf. on Computer and Robot Vision, Quebec, Canada, pp. 49, June 2006.

[15] Zhang, L., Wu, X.: An Edge Guided Image Interpolation Algorithm via Directional Filtering and Data Fusion, IEEE Transaction on Image Processing, vol. 15, issue 8, pp. 2226 2238, August 2006.

[16] Xu, N., Kim, Y.: An Image Sharpening Algorithm for High Magnification Zooming, Proc. of IEEE Intl. Conf. on Consumer Electronics, Las Vegas, U.S., pp. 27-28, January 2010.

[17] Demirel, H., Anbarjafari, G.: Image Resolution Enhancement by Using Discrete and Stationary Wavelet Decomposition, IEEE Transaction on Image Processing, vol. 20, no. 5, pp. 1458-1460, May 2011.

[18] Su, W., Ward, R.K.: An Edge-based Image Interpolation Approach Using Symmetric Biorthogonal Wavelets Transform, Proc. of IEEE 8th Workshop on Multimedia Signal Processing, British Columbia, Canada, , pp. 355-359, October 2006.

[19] Wang, J., Gong, Y.: Fast Image Super-Resolution Using Connected-Component Enhancement, Proc. IEEE Intl. Conf. on Multimedia and Expo, Hannover, Germany, pp. 157-160, 2008.

[20] Shao, W., Wei, Z.: Efficient Image Magnification and Applications to Super-Resolution Reconstruction, Proc. of IEEE Intl. Conf. on Mechatronics and Automation, Henan, China, pp. 2372-2377, June 2006.

[21] Shan, Q., Li, Z., Jia, J, Tang, C.: Fast Image/Video Upsampling, ACM Transactions on Graphics, vol. 27, no. 5, article 153, December 2008.

[22] Morse, B.S., Schwartzwald, D.: Image Magnification Using Level-Set Reconstruction, Proc. of IEEE Computer Society Conf. on, Computer Vision and Pattern Recognition, Kauai, U.S., pp. 333-340, December 2001.

[23] Zhou, D., Shen, S.: Image Zooming Using Directional Cubic Convolution Interpolation, IEEE Transactions on Image Processing, vol. 6, issue 6 pp. 627 634, 2012.

[24] Li, J., Peng, X.: Single-Frame Image Super-Resolution through Gradient Learning, Proc. of IEEE Intl. Conf. on Information Science and Technology, Hubei, China, pp. 810-815, March 2012.

[25] Glasner, D., Bagon, S., Irani, M.: Super-Resolution from a Single Image, Proc. of IEEE 12th Intl. Conf. on Computer Vision, Kyoto, Japan, pp. 349-356, September-October 2009.

[26] Wang, Z., Bovik, A.C., Sheikh, H.R., and Simoncelli, E.P.: Image Quality Assessment: From Error Visibility to Structural Similarity, IEEE Transactions on Image Processing, vol. 13, issue 4, pp. 600612, 2004.

[27] Wang, Z.: SSIM Index for Image Quality Assessment, available at http://www.cns.nyu.edu/~lcv/ssim/"

[28] Varan, S., Jagana, A., Kaur, J., Jyoti, D., Rao, D.S.: Image Quality Assessment Techniques pn Spatial Domain, International Journal of Computer Science and Technology, vol. 2, no. 3, pp. 177-184, 2011.

[29] Available at http://www.vision.caltech.edu/Image_Datasets/Caltech1 01/Caltech101.html

Printer friendly Cite/link Email Feedback | |

Publication: | Science International |
---|---|

Article Type: | Report |

Date: | Apr 30, 2015 |

Words: | 4203 |

Previous Article: | THE LONG-TERM VARIABILITY IN MINIMUM AND MAXIMUM TEMPERATURE TRENDS AND HEAT ISLAND OF LAHORE CITY, PAKISTAN. |

Next Article: | DYNAMIC MODELING AND CONTROL OF HYBRID ELECTRIC VEHICLE WITH BATTERIES, ULTRACAPACITORS AND FUEL CELL. |

Topics: |