Image magnification method comparison/Vaizdo didinimo metodu palyginimas.
Image magnification is a process of obtaining an image at resolution higher than taken from image sensor. Image magnification synonyms are interpolation, enlargement, zooming, etc. To create higher resolution image, previous image must be complemented with new pixels and their intensity must be calculated.
Commonly, image magnification is accomplished through convolution of the image samples with a single kernel--typically the nearest neighbour [1--4] bilinear, bicubic [3--5] or cubic B-spline kernel [3, 4, 6] interpolation, triangle based [3, 4, 7] and training-based algorithms [3, 4, 8, 9] that use artificial neural networks.
The main observable problem in image magnification is phenomenon called "magnification blur". Most visible "magnification blur" is on edges (contours). So it is likely that magnified image with great amount of edges will look ugly and image with fewer edges look better.
Unfortunately real image contains a lot of noise. Sources of noise are very different. Thermal noise of image sensor is more visible on dark areas of picture; quantization noise comes on edges and areas with great intensity gradient. The other noise, we call it compression noise, arise during image compression when row image from sensor area is converted to most useful TIF, JPG, PNG or other format. And yet another source of noise is image shrinking. We get a calculation error because shrinking coefficient mostly is fractional number on integer discrete domain.
Actually we don't know what was done with the image previously. Except occasion when we get row image as BMP from well known source. All other image formats have a lot of processing noise. Well known salt and pepper noise with single white and black pixels is almost invisible in image, but brings a great distortion during magnification. It is great that this kind of noise can be filtered out. Unfortunately the noise always grow-up and nothing decrease it. So commonly image is noisy.
Image quality analysis
To estimate quality of image magnification algorithm several methods of error or noise calculation are used. The same technique is used to test compression algorithms.
To test magnification quality a very simple algorithm with three simple steps is used: image for testing is scaled-down in some number, after that it is magnified in the same number, so we have the original and magnified image and it is possible to compare them by some comparison method. The simplest way to compare images is to measure the difference between a pair of similar images pixel by pixel.
The most common difference measure is the mean-square error (MSE). The mean-square error measure is popular because it correlates reasonable with subjective visual quality tests and it is mathematically tractable.  To measure difference for only one pixel square error (SE) is used
SE = [(O(j,k)-P(j,k)).sup.2], (1)
where O(j,k)--pixel from original image; P(j,k)--the same pixel from processed image.
To estimate entire image mean-square error (MSE) is used
MSE = 1 / N.M [[SIGMA].sup.N.sub.j=0][[SIGMA].sup.M.sub.k=0][(O(j,k)- P(j,k)).sup.2], (2)
where O(j,k)--pixel from original image, P(j,k)--the same pixel from processed image, N and M--number of rows and columns in the image.
Root mean-square error (RMSE) is also useful and brings less value
RMSE = 2 [square root of 1 / N * M [[SIGMA].sup.N.sub.j=0 [[SIGMA].sup.M.sub.k=0][(O(j,k)-P(j,k)).sup.2]]. (3)
All these equations to compare processing results will be used here.
Image Magnification Analysis
To get common results a lot of digital images from different sources and with different sizes were chosen. All images were converted to gray-scale to simplify processing. Because whole numbers were used for magnification, images were cropped to avoid fractions during scaling-down. Is the magnification error related with number of pixels corresponding to edges?
Edge ratio for each image was calculated. Edge ratio was calculated as sum of edge pixels on Canny filtered image divided by number of image pixels. Also RMSE for the same set of images with five magnification methods presented in Matlab were calculated. For all methods results and pixel numbers was sorted and indexed. Calculation results are shown in Fig. 1.
[FIGURE 1 OMITTED]
There is evidence, as shown in Fig. 1, that there is no clear relation between Edge Rate Index and image RMSE index. From the same data correlation shown in Table 1 was calculated.
Table 1 shows that there is weak relation between Edge Rate and RMSE of different magnification methods, but there is strong relation between different magnification methods.
Actually when there are two images with different RMSE, for one method, the image with lower RMSE yields the lower RMSE result with any other magnification method.
Also the same result is with other magnification ratio; hence magnification error depends on image content and magnification ratio and a little bit less on magnification method as shown in Fig. 2. Data was sorted with the simplest magnification method Box.
Chart Fig. 2 is very dense therefore a small section of data was picked up and shown in Fig. 3. This figure shows how near are some RMSE with different magnifications methods. Less RMSE is better, so other methods are better except triangle that for some images is worse.
[FIGURE 2 OMITTED]
[FIGURE 3 OMITTED]
[FIGURE 4 OMITTED]
[FIGURE 5 OMITTED]
Where do magnification errors arise? Fig. 4 is original image and Fig. 5 is 2 times scaled-down and then 2 times magnified image.
Both images are almost the same--differences are invisible, but exist. Errors are in some places and have some values. Low value errors are invisible. Therefore to show all pixels with errors, their values are set to maximal intensity, and image is inverted to save ink. Hence dark pixels show pixels with any error Fig. 6.
As shown in Fig. 6 and Fig. 7 there is a weak relation between error pixel map and edge of image. Area with errors is about on the same place as edge but areas with errors are bigger.
Error pixel map Fig. 6 is gray level image where darker pixel shows higher error. Where density of edge is greater the error density is greater too and pixel on error map is darker.
[FIGURE 6 OMITTED]
[FIGURE 7 OMITTED]
The other question is error distribution by error value. Accordingly from previous image set three images that draw lowest, intermediate and highest RMSE values were picked-up. "Ice-cream desert"--left image in Fig. 8 has large flat areas with the same intensity accordingly the RMSE value is low; the middle image--"Great Canyon" has more areas with different intensity and more lines so it draws higher RMSE than the first; the third or right image- "Birch Tree Forest" has a large number of small areas with different intensity--grass on the ground and trunk and branches on sky background that draws a very high RMSE value.
[FIGURE 8 OMITTED]
Original image was scaled-down by factor two, then magnified with five last-mentioned magnification methods by factor two, later difference between original and magnified image pixel by pixel was calculated and modulus of data was calculated. Now it is possible to show pixels number distribution by error value.
The other question is RMS distribution by error value. Previously error distribution by pixel number was shown, but that do not show how error value influence MSE. Hence next three figures show MSE values for each error value and their contribution to all MSE value of image.
In images with higher MSE present higher values errors and that brings greater error rate to MSE, as shown in Fig. 9-Fig. 11.
[FIGURE 9 OMITTED]
[FIGURE 10 OMITTED]
[FIGURE 11 OMITTED]
After analysis and image result observation, greater error values originate in areas with high intensity gradient. For example, third image "Birch Tree Forest" contains a lot of dark and light areas, while "Ice-cream Desert" has big areas with small intensity gradient. As small tree branches noise can bring dramatic grow of RMSE.
Image magnification error distribution depends on various factors. Idea that magnification error can depend on Edge Rate of image is not confirmed, but a weak relation exists. Different magnification methods are related very close. All tested methods show about the same RMSE results for the same picture. Lanczos3 method shows the best results and Box method shows the worst results.
Pixels with errors surround the edges, but can cover wide areas, so magnification errors are located surround the edges. Image noise can increase RMSE value dramatically, concurrently noise can be invisible on image and produce a little intensity ripple but a lot of pixels produce high RMSE.
Human eye do not recognize a small intensity change and little error values below 8 or 16 are invisible; it depends on local image intensity, but calculations draw the RMSE increase. As shown in Fig. 9-Fig. 11 RMSE peak for Cubic and Lanczos method is below 20 of RMSE value, so that errors are about invisible. Visible error number, that RMSE are above 16, decreases very quickly.
Sometimes images with the same visual quality can bring very different RMSE results. Mathematical methods to estimate image quality are stricter than visual quality estimation, but are useful for automatic calculation.
Images with high number of high gradient areas give the higher RMSE value, so that image looks worse.
Also image compression increase error number in magnified image, because most of compression methods are lossy and produce artefacts as ripples near the edge.
[1.] Gonzalez R. C. Woods R.E. Digital Image Processing, 2nd. ed.--Prentice-hall, 2002.--793 p.
[2.] Acharya T., Ajoy K. R., Sammon M. J. Image Processing Principles and Applications.--A John Wiley & Sons, 2005. 428 p.
[3.] Pratt W. K. Digital Image Processing.--A John Wiley & Sons, 2007.--785 p.
[4.] Burger W., Burge M. J. Principles of Digital Image Processing.--Springer, 2009.--428 p.
[5.] Netravali A. N., Haskell B. G. Digital Pictures: Representation, Compression and Standards, 2nd ed.--New York: Plenum Press, 1995.--706 p.
[6.] Unser M., Aldroubi A., Eden M. Fast B-spline Transforms for Continuous Image Representation and Interpolation // IEEE Trans. Pattern Anal. 1991.--Vol. 13.--No. 3.--P. 277285.
[7.] Vysniauskas V. Triangle Based Image Magnification. Electronics and Electrical Engineering.--Kaunas: Technologija, 2006.--No. 6(70).--P. 45-48.
[8.] Freeman W. T., Jones T. R., Paszor E. C. Example-Based Supper-Resolution // MERL, Mitsubishi Electric Research Labs.--201 Broadway Cambridge, 2001.--MA 02139.
[9.] Candocia F. M., Principe J. C. Superresolution of Images Base on Local Correlations // IEEE Transactions on Neural Networks, 1999.--Vol. 10.--No. 2.--P. 372-380.
[10.] Winkler S. Digital Video Quality Vision Models and Metrics.--John Wiley & Sons Ltd, 2005.--175 p.
Received 2011 02 06
V. Vysniauskas, G. Daunys, K. Vysniauskaite
Department of Electronics, Faculty of Technology, Siauliai University, Vilniaus str. 141, Siauliai, 763535, Lithuania, +370 677 63066, e-mail: firstname.lastname@example.org
Table 1. Edge Rate and Magnification RMSE Correlation Method 1 2 3 4 5 5 Edge Rate 1 Box 0,531 1 Triangle 0,553 0,993 1 Cubic 0,548 0,983 0,996 1 Lanczos2 0,546 0,982 0,996 1,000 1 Lanczos3 0,540 0,974 0,991 0,999 0,999 1
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||SIGNAL TECHNOLOGY / SIGNALU TECHNOLOGIJA|
|Author:||Vysniauskas, V.; Daunys, G.; Vysniauskaite, K.|
|Publication:||Elektronika ir Elektrotechnika|
|Date:||Apr 1, 2011|
|Previous Article:||Hierarchical control approach for autonomous mobile robots/Hierarchiniai autonominiu mobiluju robotu valdymo metodai.|
|Next Article:||Voice-based human-machine interaction modeling for automated information services/Zmogaus ir masinos balso dialogu modeliavimas automatinems...|