Printer Friendly


Byline: H. Sarkan, B. Sumali, N. Hamada and Y. Mitsukura

ABSTRACT: Blurred face image restoration algorithm by using image gradient magnitude prior model is presented. In practice, blurred face image restoration is significant because of face recognizing acquired from distant camera. For this ai m, our previous restoration method is modified for targeting face images and utilizing the gradient profile prior. At first, a generic gradient profile feature inherent in face images is revealed. Through an experimental investigation, a novel parametric statistical model is introduced as the prior information on face. Then, an image quality recovering algorithm is proposed by minimizing the distance measured by the difference between a recovered image gradient distribution and the prior one. The actual restoration procedure succeeded by the application of the principal component analysis, which is used to represent clear face image estimation by the linear combination of the obtained principal components.

The optimization is finally carri ed out by maximal matching the gradient distribution of estimate to that of a prior face image. Finally, experiments are conducted to demonstrate the effectiveness of the proposed algorithm.

Keywords: Face Recognition, Image restoration, Gradient Distribution prior, Principal component Analysis, Face Image Analysis


Face image analysis such as face detection, face recognition, and facial expression recognition, has received a great deal of attention in recent years [1,2]. For instance, face recognition is successfully applied to many applications, not only biometric recognition, but also passport control, law enforcement, surveillance, and human-computer interaction. Face recognition is known to offer several advantages over other biometric-based techniques such as fingerprints, hand geometry, hand veins, iris, and retina. One beneficial point is that face images can be acquired from a distance by an inexpensive camera. In addition, unlike other biometric methods, facial image approach is totally non-intrusive.

Though the performance of current face recognition systems is quite well under controlled environment like recognition of passport photograph, major research area still remains in developing effective algorithm even in the unconstrained environments. Such significant challenges are occlusion, face orientation, complex outdoor lighting, and blue and low resolution images from distant camera. One of the main factors that cause severs image degradation is blur. It affects the appearance of face images so that it makes different individuals tend to be appearing more similar. Blur is often happening in real life image acquisition and it is originated from misfocused camera or object motion. This paper addresses the restoration of face image from a single blurred one without estimating blur kernel. The algorithm utilizes a prior model of gradient distribution of face images and our previous PCA-based restoration [3].

The first step of the PCA-based restoration is to create an image subspace which is generated by repeatedly blurring a query face. In the second step, an optimal linear combination of the first some bases of the subspace is found by maximizing its no-reference image quality assessment index (NR-IQA) [4]. As an NR-IQA in this study, the gradient distribution model of face images is exploited.

The organization of this paper is as follows: In Chapter 2, the gradient distribution prior statistic model for clear sharp face images is introduced. Chapter 3 presents PCA-based restoration algorithm with implementing gradient distribution matching to obtain an optimal linear combination of PCA bases. Experimental results are shown in Chapter 4, and this study concludes in Chapter 5.


Existing major approaches for blurred face recognition are categorized into four: (1) blur invariant feature extraction from blurred image and use it for recognition, such as local phase quantization (LPQ) [5], (2) deblur a given degraded face image first and then apply conventional clear face image recognition method [6], (3) direct recognition approach [7], (4) simultaneous deblur and recognition approach [8]. The approaches (2)~ (4) contain some sort of deblurring process or blur kernel estimation, and each of these methods can be combined with the first approach. A typical second approach by Nishiyama et al. tries to estimate blur kernel or blur point spread function from a query blurred face image. In their training stage, blurred training face images are generated by a set of representative blur kernels, then they construct a feature space where the difference of individual kernels are efficiently characterized.

In the test stage, a query face image of unknown blur kernel is compared to each subspace and the closest subspace provides its corresponding blur kernel by which restoration is completed. However, their approach would not incorporate complex blur kernels. The third blurring method copes with the blur face recognition problem by creating a set of artificial face images from the gallery images. For instance, in a recent method [7] underlying blur kernel is estimated as an optimal kernel whose blurring model provides the closest face image to a given query face. This approach does not directly obtain deblurred image itself, but it has to solve convex quadratic minimization in kernel parameter space. For high dimensional kernel model this approach is not efficient from a computational cost viewpoint. The fourth group of approach [8] combines both restoration and recognition of face images together in a loop. They formulate and alternatively solve two minimization problems.

The objective functions are the sum of a model error and a regularization terms.

In general, blind image restoration is an ill-posed inverse problem which recovers the latent image from the blurred observation without any knowledge about the blurring process [9]. One way to solve the problem is to assume an appropriate prior model about the latent image and blur kernel. Lots of restoration algorithms have been developed for natural image restoration where their prior statistic model plays an important role. [10, 11,12]. Most relevant research is the method proposed by Fergus et al., in which a method is proposed for removing the effects of unknown camera shake. Their algorithm exploits natural image gradient prior statistics, which means that photographs of natural scenes typically obey specific distribution of image gradient magnitudes.


Gradient Distribution of Facial Image

The blind restoration algorithm proposed in this study utilizes some sort of image quality measure which needs no reference image information. Several non-reference image quality assessment (NR-IQA) measures are known for evaluating natural scene quality. For instance, studies by Fergus et al. [11] and Shan et al.[12] proposed for solving motion blur restoration. Both methods employ a natural image statistics, which indicates a heavy-tailed distribution of image intensity gradients. This study introduces a novel parametric statistical gradient distribution model as an empirical result from face image data base. Actually, an average gradient distribution curve over given gallery sharp face images is utilized for creating NR-IQA measure of a restored face image.

For a given gray scale image G, image gradient as denoted by (EQUATION), where x, y are horizontal and vertical variables respectively, can be equivalently represented by its magnitude:


Where (EQUATION) are computed by gradient operator such as Sobel, Prewitt gradient operators.

Figure 1 shows an example of (a) original face image, (b) gradient magnitude image, and (c) the histogram of gradient magnitude.

Figure 2 shows (a) the average of gradient distributions of gallery face images (AT and T face data base) and (b) their blur face images, respectively. As we can observe and intuitively recognize from Figure 2, by comparing the distributions of clear and blur face images, the gradient distributions of blurred face images tend to be tightly spread and have a higher peak at a lower gradient magnitude point.

As observed in Figure 2, we introduce a parametric mathematical model


for representing the gradient distributions of clear as well as blurred face images, where x means the gradient variable, a, b, c are real positive numbers. The parameters a, b, c are obtained by curve fitting process such as trust region algorithm [13].

Figure 3 shows a curve fitting result for the gradient distribution of Figure 2 (a).

Fig. 3. Curve fitting of gradient distribution of clear face image.

Restoration Algorithm

The core role of the proposed restoration method of face image is played by our previous blind deblurring algorithm based on PCA subspace estimation [3]. The deviation from the average gradient distribution of gallery face images demonstrated in Figure 3 can measure the image quality of a recovered face image in the stage of an optimization. For a given blur face image g, the proposed restoration method is divided into the following steps:

1. M blurred ensemble {gi} creation:

g1 = g


Where b (s) is Gaussian kernel with the standard deviation of and * means t convolution operator.

2. Principal component analysis

a. Compute mean of ensemble:


b. Centering the ensemble: (EQUATION)

c. Apply PCA to the ensemble of {phi}, and extract the first J principal components: (EQUATION)

3. Estimate a restored image in the J-dimensional subspace spanned by (EQUATION) as the form of


Where the weights [alpha]js (j = 1~J) are determined by solving the following optimization problem:


Where Dg (fJ) means the gradient distribution of the estimate image of fJ, and the average Dg means the average gradient distribution of database face images which is shown in Fig. 2 (a). The minimization problem represented by (7) can be approximately solved step-by-step according to the order of the principal components. Namely, the first step of the optimization is to determine the optimal first weight by solving the following optimization problem in one-dimensional space.


Then, the successive optimization with respect to each weight continues until it reaches J-th weight. The detailed restoration process is described in our previous paper. [3]


The proposed method is applied to some face images from the data base, where Figure 4 to 6 shows the experimental results of restoration. In each figure, the first row from the left shows; blurred face image by Gaussian blue, original clear face image, and restored face image. The image quality with respect to the original image measured by SSIM (Structural Similarity) and PSNR indexes are shown at the next row of blurred and restored images respectively. At the bottom in Figure 4~6 show their gradient distribution profiles (red curves) vs. the average gradient curve (blue colored) of gallery face images. From the results shown in Figure 4 and 5, it can conclude that the performance of the proposed method is sufficiently high. For instance, the improvement gain in PSNAR are around 2 ~ 5dB. On the other hand, the PSNR as well as SSIM of the restored image do not give any improvements in the third case as shown in Figure 6.

Regardless of lower PSNR and SSIM values, our subjective view may apparently recognize some image quality improvement of face image comparing blurred and restored images in Figure 6.


We proposed the PCA subspace blind restoration of face images, where a novel gradient distribution model for face images is adopted to evaluate the restoration degree. As the gradient distribution statistic for face image, simple parametric model was introduced.

The proposed restoration algorithm is developed by incorporating the introduced gradient measure with our previous PCA subspace restoration algorithm. Conducted experiments show that the proposed algorithm effectively restored face images. Unifying the blind restoration method developed here and existing face recognition algorithm would be a significant future work for practical face recognition by distant camera acquisition. Verification of the noise robustness is a future issue of the proposed method.


B. Sumali would like to thank Japan-ASEAN Integration Fund (JAIF) for the MJIIT-ASEAN student incentive to pursue his study.


[1] Stan Z. Li and Anil K. Jain (Editors). Handbook of Face Recognition. Springer-Verlag (2011)

[2] Jafri R. and Arabnia H. R. A Survey of Face Recognition Techniques. Journal of Information Processing Systems, Vol.5, No.2, June (2009)

[3] Sumali B., H. Sarkan, N. Hamada, and Y. Mitsukura, Blind Image Restoration by using PCA-subspace Generation and Image Quality Optimization. 1st International Conference on Advanced Information and Communication Technology (ICAICT2016). May 6 - 7. Chittagong, Bangladesh. (2016)

[4] Mittal A., R. Soundararajan, and A. C. Bovik. Making a "Completely Blind" Image Quality Analyzer. IEEE Signal Processing Letters, vol.20, no.3, pp.209-212. (2013)

[5] Ahonen A., E. Rahtu, V. Ojansivu, J. Heikkila. Recognition of blurred faces using local phase quantization. Proc. ACM SIGGRAPH Conf. pp. 1-4. (2008)

[6] Nishiyama M., A. Hadid, H. Takeshima, J. Shotton, T. Kozakaya, and O. Yamaguchi. cial Deblur Inference using Subspace Analysis for Recognition of Blurred Faces. IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 4, pp. 838- 845 (2010)

[7] Vageeswaran P., K. Mitra, R. Chellapa. Blur and Illumination Robust Face Recognition via Set-Theoretic Characterization. IEEE PAMIVo. 22, No. 4, pp.1362-1372 (2012)

[8] Li S. S. Li, J. Hu, and W. Deng. Simultaneous Blurred Face Restoration and Recognition. 3rd IAPR Asian Conference on Pattern Recognition, (2015)

[9] Gonzalez C., R. E. Woods, Digital image processing. 2nd ed. Prentice Hall, (2007)

[10] Sun J., J. Sun, Z. Xu, and H.-Y. Shum. Gradient Profile and Its Applications in Image Super-Resolution and Enhancement. Trans. Image Process. Vol. 20, No. 6 (2011)

[11] Fergus R., B. Singh, A. Hertzmann, S. T. Roweis, W.T. Freeman. Removing Camera Shake from a Single Photograph. ACM Transactions on Graphics, 25, pp.787-794. (2013)

[12] Shan Q., J. Jia, and A. Agarwala. High-quality Motion Deblurring from a Single Image. ACM Transactions of Graphics, Vol. 27, No.3 (2008)

[13] Yuan, Y. X. Trust region algorithms for nonlinear equations. Information 1, pp.7-20 (1998)
COPYRIGHT 2017 Asianet-Pakistan
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2017 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Sarkan, H.; Sumali, B.; Hamada, N.; Mitsukura, Y.
Publication:Science International
Article Type:Report
Date:Jun 30, 2017

Terms of use | Privacy policy | Copyright © 2021 Farlex, Inc. | Feedback | For webmasters |