Printer Friendly

VOTING-BASED HDR COMPRESSION.

1. INTRODUCTION

A highly important characteristic of an image or a video captured with a conventional device is the degree at which the real scene is reproduced. In the field of image processing, the main element controlling the fidelity of the captured image is the luminance of the objects composing the scene. This property describes the quantity of light radiated, reflected or passing through those objects.

In conventional photography, the luminance of the captured image differs from the one of the real scene due to the difference in their dynamic intervals (the ratio between the lowest and greatest possible brightness value). In other words, due to the limited dynamic interval of conventional photographs, there is significant loss of information compared to the original scene.

With technology being is on the rise (mainly because there is a high demand of high quality images in certain fields amongst which autonomous cars or assisted parking sensors), the advent of HDR ("high dynamic range") devices did not come as a surprise. These devices greatly overcome the drawbacks of their conventional counterparts such as LDR ("low dynamic range") and SDR ("standard dynamic range").

There remains, of course, the problem of displaying these HDR images. Up until now, the transition between conventional and HDR TV's, screens and phones has not been completed. Therefore, in this paper the aim is to reproduce an HDR image in LDR format in such a manner that it can both improve the level of details present in the image and display it on a device with limited dynamic range. Far from being a new research field, as it has been studied for approximately two decades, it can still benefit from improvements. The technique is known as "tone mapping" and what it does when applied on an HDR image is to perceptually approximate the real scene by preserving a certain level of details and contrast.

What propelled this technology again after a promising start at the beginning of 2000 was the advent of mobile phone integrated cameras that offered the advantages of expensive digital photography at an affordable price.

Most of the tone mapping algorithms [1][2][3][4] try to compress the dynamic range of the real scene and to reproduce it into a limited dynamic range. Unfortunately, most methods account for a specific class of images or have implicit control parameters based on individual statistics and finding the appropriate value for the parameters generally poses difficulties.

Taking all the above into consideration, this paper aims to describe a new method of obtaining an LDR image from an HDR one by varying different algorithms and their corresponding configuration parameters. As tone mapping algorithms do yield desired results, each for a certain type of input image, the proposed method relies on a voting system. By combining the aforementioned algorithms, the hope is to obtain an improved limited dynamic range image.

In addition to the voting system, a tone mapping algorithm was also devised starting from the existing ones that has a good time complexity and yields satisfying results.

2. IMAGE PROPERTIES

Luminance is a photometric measure which represents the intensity of the light emitted, reflected or passing through objects in a scene and it is basically an indicator of how bright an object in that scene is.

One of the most well-known image formats, the RGB format, lacks such indicator of how bright an object is, but can be either converted into a format which does have it, or the luminance can be computed according to formulas given by the IEC 61966-2-1 RGB standard (fig. 2.1).

3. GAMMA CORRECTION

Gamma correction is an operator that controls the brightness of the entire scene and it is defined by the following expression:

[L.sub.1](x,y) = [L.sub.0][(x,y).sup.[gamma]] (1)

where (x, y) is the pixel corresponding to row x, column y of the scene's matrix representation; [L.sub.0] is the current luminance of the scene; [L.sub.1] is the new value of the luminance and [gamma] is the translation exponent.

According to (5), if [gamma] = 1, the output image is identical to the input image. If [gamma] < 1, the operation is called gamma compression and it is used for brightening the scene. On the contrary, if [gamma] > 1, the operation is called gamma expansion and it is used for darkening the scene.

4. DYNAMIC GAMMA

Any device able to capture or display images has a characteristic called dynamic gamma, defined as the ratio between the highest and lowest pixel value. These values also determine the upper and lower boundaries of the dynamic range.

Unfortunately, conventional devices have a significantly small dynamic gamma (300:1 on average) as opposed to a real scene. This drawback is precisely what lead to the advent of the HDR technology both in capturing and displaying devices.

5. EXPOSURE

The exposure time is the amount of time in which the sensor inside the camera is exposed to light before the image is formed. It is usually measured in seconds or fractions of seconds. By varying the exposure time, a photographer can obtain underexposed and thus darker images or overexposed and thus overall brighter images. Both under and overexposure lead to a decrease in the visible level of details in an image.

6. OBTAINING HDR IMAGES

The most widely used approach for obtaining HDR images belongs to Paul Debevec [8] and consists in capturing consecutive frames of the same scene at different exposure times and combining them in a single image. The resulting image is composed of pixels whose values are proportional to the values of the real scene's luminance.

7. PECULIARITIES

Both capturing and converting an HDR image into an LDR image are subject to anomalies. A possible abnormality is caused by the involuntary movement of the camera between frames and can be easily solved for example by fixing the camera on a steady tripod. Another anomaly is caused by moving objects in the scene. Since the latter is harder to control, peculiarities such as the ghost effect are likely to appear.

Another undesired artifact is the halo effect caused by inverting the contrast between an object and the surrounding details. It can be observed around small objects and it is caused by local algorithms that average the values of bright pixels and their neighbors.

8. LOGARITHMS AND EXPONENTIALS IN HDR

Since using linear functions did not yield the expected results when it came to HDR compression, the focus shifted towards logarithmic and exponential functions.

The logarithm's base plays a vital role in determining its value and the higher the base, the higher values will be mapped to lower ones. The opposite happens for the exponential function.

Applied on HDR images, certain combinations of logarithmic and exponential functions determine a compression of the dynamic range. Darker areas will be dealt with by exponentials in order to brighten them while brighter areas will be subject to logarithmic operations meant to darken and reveal details.

Note the amount of detail revealed by applying logarithmic or exponential functions to the images in figure 2.6 and 2.7.

9. DETERMINING THE QUALITY OF AN IMAGE

Tone-mapping algorithms can yield unexpected and sometimes undesired results such as ghost and halo effects. For this reason, it is of great importance to assess the quality of the output image and this can be done by comparing it to a reference or ideal image.

PSNR

The "peak signal-to-noise ratio" represents the ratio between the maximum power of a signal and the power of the noise corrupting that signal.

PSNR = 20 * log10 ([max([I.sub.r])/MSE]) (2)

MSE = [1/n*m] [[summation].sup.n-1.sub.x=0] [[summation].sup.m-1.sub.y=0] [square root of [[I.sub.R](x,y) -[I.sub.R] (x,y).sup.2]] (3)

where [I.sub.r], [I.sub.t] are the reference and test images; (x, y) is the pixel corresponding to row x, column y of the scene's matrix representation; n, m are the total number of rows and columns and max([I.sub.r]) is the maximum value of the pixels in the reference image.

The bigger the PSNR measure, the better the quality of the image is. The main drawback of this measure is that it does not take human sight into account and therefore cannot provide correct estimates.

SSMI

A measure that does account for the functioning of the human eye (defined by the HVS model) is the "Structural similarity index measurement" [12].

TMQI

"Tone Mapped Image Quality Index" is a measure proposed by Z. Wang and it is based on a modified structural similarity index and a naturalness function based on statistics acquired from natural images.

10. SIMILAR METHODS

Tone mapping algorithms

Tone mapping algorithms reduce large dynamic ranges so the resulting image can be easily displayed on a standard screen. These algorithms can be split into two categories: local and global. A local operator will change the brightness using the current brightness on the selected pixel and a set of properties of the surrounding pixels.

In the case of global algorithms, the brightness compression function becomes the same for all the pixels, in contrast to the local algorithms where it varies depending on the picture fragment.

Similar voting methods

Using the idea of voting, the final results are enhanced, thus subduing the disadvantages of some algorithms.

Voting algorithms are being successfully used in case of image processing methods whose results strongly depend on the input images. Among them are: the binarization method, where the images are being converted from grayscale images to black and white images using an algorithm threshold, image segmentation and OCR ("Optical Character Recognition").

The following method can be used in OCR: at first the image areas that contain text are identified, then on each area a variable number of preprocessing filters are applied. For each filter, the OCR engine can successfully recognize a number of characters with an accuracy score. After that the voting algorithm picks the best fitting filter for each area and the final result is obtained by combining all the picture fragments.

11. THE TONE MAPPING ALGORITHM

The proposed method is a tone mapping algorithm based on a compression function for large dynamic intervals that is applied on the entire image in case of the global approach or on input image blocks if a local approach is favored. The function is a combination of logarithms and exponentials. The logarithmic function restricts a value interval when applied while the exponential one is expanding that interval. By using them in the same time, the brightness can be increased for dark image areas or decreased in case the areas are too bright.

The first step of the algorithm is to establish the HDR image luminance. The RGB color format has no variable that contain the brightness value so it needs to be calculated for each pixel.

The next step is to ascertain the minimum brightness value, the maximum brightness value, the average of the logarithm of the brightness values and the maximum subtraction in the logarithmic space of the brightness matrix. Using these values, the current luminance is being translated into a LDR using the following formula:

[mathematical expression not reproducible] (4)

g (x, y) = [-L(x,y)/[2.sup.medie]log] (5)

where [L.sub.f](x, y) is the value of the pixel at (x, y) from the new luminance matrix, L(x, y) is the value of the pixel at (x, y) from the current luminance matrix and [infinity] is value between [0,2] that controls the luminance of the entire scene (the default is 0.9).

The last step of the algorithm is changing the current luminance with the calculated one.

A local approach of this algorithm can be made using image segmentation then applying the global method of brightness translation into the LDR domain on each picture block obtained by segmentation.

For simplicity, the segmentation will be made on 80x80 blocks.

The blocks must not overlap and their reunion should produce the input picture. If the situation requires it the blocks can be bigger or smaller, depending on the remaining pixels.

After the image segmentation, the global method is applied on each block. At the block level, the results were satisfying: the details were clearer, but when scaling the image, the quality decreased; there were visible brightness differences between neighboring blocks.

A weighted average between the maximum values of each block applied on the entire picture was able to solve this issue.

12. ALGORITHM IMPLEMENTATION EXAMPLE

For ease of implementation, the algorithm can be split into four modules: an input module, a preprocessing module, a voting module and an output module.

The modules communicate with each other using the voting module. The algorithm's architecture resembles the LDR image creation technique, the major differences being found in the voting and preprocessing modules.

The input module reads the input data and begins the voting algorithms. There are two ways of reading the input image: by using a HDR image or by using a set of standard images, each having its own exposure level, which will be merged into a HDR image using a fusion algorithm. Detection and correction methods can also be used to reduce the number of artefacts that may occur during the creation of an HDR image. The set should be made using a static scene to avoid movement blur between frames. To obtain an accurate result from the fusion algorithm, the exposure level used on each frame should be known. The input data should contain three channels: red, green and blue whether it is an HDR picture or a picture set.

The preprocessing module consists in three submodules: a submodule that contains the implementations of tone mapping algorithms, another which contains the control parameters for the algorithms and a submodule that computes the current luminance matrix.

As a first step, the module computes the brightness matrix. Then the matrix is sent to the next step where the main tone mapping algorithms are being applied. The parameters needed may vary for each algorithm, but the luminance matrix is used for all of them.

Voting Module

The output module merges the image blocks resulted from the segmentation, then changes the current luminance with the one resulting from the algorithm and prints the resulting LDR image which has a restricted dynamic interval.

13. THE RESULTS

According to the tests, the results show that the global version of the algorithm runs faster, but does not produce an acceptable contrast between colors, while the local algorithm outputs images with a strong contrast but has a greater time complexity (which could be omitted).

During the program's execution, it can be noticed that the Durand operator makes the scene seem artistic and farther away but emphasizes the details in the center of the image, while Drago's operator makes it seem closer, but the colors are lighter.

The tests have proven that the running time of the global method is logarithmic.

The tests were made on input HDR images, not on LDR image sets to avoid any resulting artefacts that may result from movement. The program was run both globally and locally.

For the local case, on the blocks where the ReinhardTMO algorithm won, there is a minor intensity difference between the adjacent blocks.

The global voting time consists of executing the five algorithms and obtaining the results for each of them. In the case of a global vote, this is the running time of the voting system. The local voting time is the voting time of executing the entire set of algorithms on each block resulted from the image segmentation.

The total running time is the sum of the global voting time and the local voting time.

14. CONCLUSIONS AND FURTHER WORK

The paper has been divided into two parts. The purpose of the first part was to demonstrate the viability of a voting-based method able to choose between different tone mapping algorithms to improve the result. The results were indeed satisfying, but the running time was higher.

In the second part of the work a tone mapping algorithm was proposed with a local and a global version which yields satisfying results in a small amount of time for images with medium-sized dynamic intervals.

A possible future step would be to reduce time complexity with the help of multi-threading programming. Another step is to develop a better method for reducing the difference in intensity between adjacent blocks.

This paper concludes the research carried during the master studies at the faculty of Automatics and Computers from the "Politehnica" University of Bucharest by the first author, thus continuing the work presented in [26].

ACKNOWLEDGEMENT

This work was supported by a grant of the Romanian Ministry of Research and Innovation, CCCDI - UEFISCDI, project number PN-III-P1-1.2-PCCDI-2017-0689 / "Lib2Life-Revitalizarea bibliotecilor si a patrimoniului cultural prin tehnologii avansate" / "Revitalizing Libraries and Cultural Heritage through Advanced Technologies", within PNCDI III.

REFERENCES

[1] E. Reinhard, M. M. Stark, P. Shirley si J. A. Ferwerda, "Photographic tone reproduction for digital images," ACM transactions on graphics (TOG), vol. 21, nr. 3, pp. 267-276, 2002.

[2] F. Drago, K. Myszkowski, T. Annen si N. Chiba, "Adaptive Logarithmic Mapping for Displaying High Contrast Scenes," Computer Graphics Forum, vol. 22, nr. 3, 2003.

[3] J. Kuang, J. Garrett si F. Mark, "iCAM06: A refined image appearance model for HDR image rendering." Journal of Visual Communication and Image Representation, vol. 18, nr. 5, pp. 406-414, 2007.

[4] F. Durand si J. Dorsey, "Fast Bilateral Filtering for the Display of High-Dynamic-Range Images," ACM transactions on graphics (TOG), vol. 21, nr. 3, 2002.

[5] International Electrotechnical Commission, "IEC 61966-2-1: Multimedia systems and equipment-Colour measurement and management-Part 2-1: Colour management-Default RGB colour space-sRGB.," 1999.

[6] "Cambridge in colour," [Interactiv]. Available: http://www.cambridgeincolour.com/tutorials/cameras-vshuman-eye.htm. [Accessed: August 2017].

[7] "Camera Sensor Ratings," DxOMark, [Interactiv]. Available: https://www.dxomark.com/Cameras/Ratings. [Accessed: August 2017].

[8] P. Debevec, "Recovering High Dynamic Range Radiance Maps from Photographs," [Interactiv]. Available: http://www.pauldebevec.com/Research/HDR/. [Accessed: August 2017].

[9] S. Desire si S. Abhilash, "Ghost Detection and Removal in High Dynamic Range Images," Signal Processing: Image Communication, 2012.

[10] A. &. Z. D. Hore, "Image quality metrics: PSNR vs. SSIM," Pattern recognition (icpr), 2010.

[11] G. Wetzstein, "The Human Visual System," [Interactiv]. Available: https://stanford.edu/class/ee267/lectures/lecture5.pdf. [Accessed: August 2017].

[12] Z. Wang si A. C. Bovik, "A universal image quality index," IEEE Signal Processing Letters, 2002.

[13] Z. Wang, E. P. Simoncelli si A. C. Bovik, "Multiscale structural similarity for image quality assessment," Signals, Systems and Computers, vol. 2, 2003.

[14] Z. Wang si H. Yeganeh, "Objective Quality Assessment of Tone-Mapped Images," IEEE Transactions on Image Processing, vol. 22, nr. 2, pp. 657-667, 2013.

[15] C.-A. Boiangiu, M. Simion, V. Lionte si Z. Mihai, "Voting Based Image Binarization," Journal of Information Systems & Operations Management, 2014.

[16] C.-A. Boiangiu si R. Ioanitescu, "Voting-Based Image Segmentation," Journal of Information Systems & Operations Management, 2013.

[17] C-A. Boiangiu, R. Ioanitescu si R.-C. Dragomir, "Voting-Based OCR System," Journal of Information Systems & Operations Management, p. 470, 2016.

[18] F. Banterle, "HDR Toolbox for Matlab," [Interactiv]. Available: https://github.com/banterle/HDR_Toolbox. [Accessed: August 2017].

[19] Banterle, Francesco and Artusi, Alessandro and Debattista, Kurt and Chalmers si Alan, Advanced High Dynamic Range Imaging: Theory and Practice, AK Peters (CRC Press), 2011.

[20] Z. Wang si H. Yeganeh, "TMQI: Tone Mapped Image Quality Index," [Interactiv]. Available: https://ece.uwaterloo.ca/~z70wang/research/tmqi/. [Accessed: August 2017].

[21] Z. Farbman, R. Fattal, D. Lischinski si R. Szeliski, "Edge-Preserving Decompositions for Multi-Scale Tone and Detail Manipulation," [Interactiv]. Available: http://www.cs.huji.ac.il/~danix/epd/. [Accessed: 2017].

[22] "pfstools - High dynamic range images and video," [Interactiv]. Available: http://pfstools.sourceforge.net/index.html. [Accessed: August 2017].

[23] Y. Al-Najjar si Der Chen Soong, "Comparison of Image Quality Assessment: PSNR, HVS, SSIM, UIQI," International Journal of Scientific & Engineering Research, vol. 3, nr. 8, 2012.

[24] Peter, "The HDR Image," [Interactiv]. Available: http://thehdrimage.com/tag/halos-around-trees-in-hdr/. [Accessed: August 2017].

[25] Z. Wang si A. C. Bovik, "Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures," IEEE signal processing magazine, vol. 26, nr. 1, pp. 98-117, 2009

[26] Razvan-Costin Dragomir, "HDR Compression Using Voting" Dissertation Thesis, Unpublished Work, Bucharest, 2017.

Razvan-Costin DRAGOMIR (1*)

Costin-Anton BOIANGIU (2)

(1) corresponding author, Engineer, "Politehnica" University of Bucharest, Bucharest, Romania, razvan.drc@gmail.com

(2) Professor PhD Eng., "Politehnica" University of Bucharest, Bucharest, Romania, costin.boiangiu@cs.pub.ro
Table 1

Device                          Dynamic gamma

LCD                               250:1 - 1750:1
Human eye                        1000:1 - 15000:1
DSLR camera (Nikon D810)        28500:1
Digital camera (Red Weapon 8k)  92000:1
COPYRIGHT 2018 Romanian-American University
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2018 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:High dynamic range image processing
Author:Dragomir, Razvan-Costin; Boiangiu, Costin-Anton
Publication:Journal of Information Systems & Operations Management
Date:Dec 1, 2018
Words:3458
Previous Article:CUSTOMER LIFETIME VALUE AND CUSTOMER LOYALTY.
Next Article:APPLYING PHOTOGRAPHS FILTERS TO IMPROVE PEOPLE AND OBJECTS RECOGNITION USING AN API.
Topics:

Terms of use | Privacy policy | Copyright © 2022 Farlex, Inc. | Feedback | For webmasters |