Printer Friendly

Grey-level cooccurrence matrix performance evaluation for heading angle estimation of moveable vision system in static environment.

1. Introduction

Sensors are most the important components in measurement. With the data gathered from sensor, data analysis as well as control strategy implementation can be conducted. Due to the importance of the sensor, there are various works conducted in this field to cater issues that arise in this field, for instance, new design of force sensor [1, 2] and tactile display [3].

Particularly in the field of robotics, sensors provide information about a variable been measured in order to control the robot, which is most critical part. This information is processed in order to decide the way the robot should take an action with good performance. There are various kinds of sensors available for this purpose, for example, sonar sensor, position sensor, infrared sensor, and camera. Employment of a camera as a sensor, also known as robot vision system, is an interesting idea due to its capabilities to obtain useful information from surroundings through image acquisition. Depending on the application, extraction of information gathered from robot vision system can be executed in various ways. The extraction process commonly utilises image processing and analysis.

A research in the field of vision system is ranging from hardware development and testing, algorithm development specifically on image processing and analysis, and strategy proposal regarding the way to use vision system for a specific task and its implementation. Conducted previous works include manipulation of image acquisition capabilities of the optical mouse to be used as a sensor [4-6], utilization of panoramic image in visual navigation [7], log-polar imaging application in robotic vision [8], application of sparse visual data without odometry reading [9], obstacle avoidance strategy [10], visual mapping [11], and strategy for robust real-time vision [12].

In this paper, a new approach has been proposed in extracting information regarding the heading angle of vision system using optical flow, with the belief that this work will be beneficial especially in robot navigation. Since it is well understood that some flow vectors generated using this method also contained erroneous vectors which are not representing actual motion, it is necessary to improve the interpretation by integrating a new approach. Here, we proposed the use of grey-level cooccurrence matrix (GLCM) [13] together with optical flow computation as an improvement technique in heading angle estimation. By dividing the image into several areas of interest (AOI), the selection of the most suitable area will be handled by GLCM before an optical flow calculation is made. Additionally, from generated an optical flow vector, heading angle estimation is implemented, and this estimation is further smothered using moving median filter (MMF) in order to eliminate the outlier existed in estimation. The estimation performance is carried out using mean absolute deviation (MAE), and the results from the proposed technique are compared with the estimation result using unmodified optical flow vector field.

2. Grey-Level Cooccurrence Matrix

Grey-level cooccurrence matrix is one technique in texture feature extraction [14]. In general, texture in image shows a repeating pattern of local variation in intensity. It provides information in the spatial arrangement of colours or intensities in an image, characterized by the spatial distribution of intensity levels in a neighbourhood. GLCM is implemented based on statistical approach, where the texture is manipulated as a quantitative measure of the arrangement of intensities in a region. Various works have been done which take a benefit of GLCM, such as in [15-18].

Specifically, GLCM is defined as tabulation of occurrence frequency with regarding to a different combination of pixel intensity values in an image. In this method, consideration has been given to the relation between two pixels at a time which is called a reference and neighbour pixel. This relationship is displayed in the form of cooccurrence matrix. Turning it into an equation, it can be described as

[P.sub.d] [i, j] = [n.sub.ij]. (1)

Here, is denoted as the number of occurrence of the pixel values (i, j) posses at length d in the image. The occurrence matrix Pd has dimension n x n, where n is denoted as the number of grey levels in the image. The grey levels in the image are specified under quantization. The elements of Pd are then normalized using the following equation:

N[i, j] = P[i, j]/[[summation].sup.n.sub.i=1] [[summation].sup.n.sub.j=1] P[i,j] (2)

Through normalization, elements in matrix Pd have a value which is in range between zero and one, which allows to be manipulated as a function of probability.

Through GLCM, texture properties can be captured. Using cooccurrence matrix, feature vector can be extracted in several ways. In this paper, contrast has been selected in extracting feature vector. Contrast is a measure of the local variations present in an image, which are represented as

C(k, n) = [n.summation over (i=1)] [n.summation over (j=1)] [(i - j).sup.2] N[i,j]. (3)

3. Optical Flow

Optical flow technique is commonly used in motion detection and estimation. Optical flow equation is derived based on the assumption that the brightness is preserved unchanged over an entire image sequence and is represented as

[I.sub.x]u + [I.sub.y]v + [I.sub.t] = 0. (4)

Here [I.sub.x], [I.sub.y], [I.sub.t] are denoted as spatial and temporal gradient which can be estimated from image sequence. Meanwhile u, v are denoted as optical flow vector and they are unknowns. Since optical flow equation can be acquired from only a single pixel, hence, these unknowns cannot be solved directly. Regarding this matter, several methods have been proposed in helping to find a solution of these unknowns. In this paper, the method that has been selected is Horn-Schunck method [19].

In Horn-Schunck method, in order to solve these unknowns, optimization constraint is introduced with the assumption that

(A1) motion stays constant within pixel neighbourhood;

(A2) optical flow differs smoothly.

The optimization constraint is defined as

[E.sub.t] = [([I.sub.x]u + [I.sub.y]v + [I.sub.t]).sup.2] + [zeta]([u.sup.2.sub.x] + [u.sup.2.sub.y] + [v.sup.2.sub.x] + [v.sup.2.sub.y]), (5)

Here, [zeta] is a weighting factor, and [u.sub.x], [u.sub.y], [v.sub.x], [v.sub.y] are partial derivatives of the optical components u and v. Moreover, computation of [I.sub.x], [I.sub.y], [I.sub.t] is implemented using the following equations:


In terms of partial derivatives of the optical components u and v, it can be computed from the 4-adjacent position to the position (i, j). Therefore,


In addition, through differentiating and minimizing (5) over u and v at position (i, j), the equation can be rewritten as




From (8), the unknowns are [u.sub.(i,j)] and [v.sup.(i,j)]. The relationship between [[bar.u].sub.(i,j)], [[bar.v].sub.(i,j)], and [u.sub.(i,j)], [v.sub.(i,j)] can be observed. Therefore, iterative scheme can be constructed based on this relationship, as shown in the following equation:


In (10), n is described as iteration step, and [bar.u], [bar.v] are described as the neighbourhood mean of the flow vector u, v, respectively.

4. Heading Angle Estimation Strategy

In this study, a static environment is selected as an experimental scenario. A camera is shifted over horizontal flat surface from left to right, as shown in Figure 1.

Using this camera, a video is captured, and the image sequence is extracted. Extracted images have a resolution of 320 x 176 pixels. At initial stage, all images are converted to greyscale format and then resized to 99 x 99 pixels. Each converted image is divided into nine AOI. Since GLCM is employed, the intensity of the image needs to be scaled to several grey levels. In this study, the image has been scaled to eight grey levels. In each AOI, the GLCM is generated, and a feature vector is extracted using contrast criterion. By using computed feature vector, a suitable AOI is selected.

In the next stage, optical flow field vector is generated using Horn-Schunck method. Vector operation is employed where each flow vector is decoupled to horizontal and vertical components. Summation of each decoupled vector regarding horizontal and vertical axes is implemented, and angle estimation is computed. MMF is applied to acquire smoothness of the estimated angle.

In order to assess the estimation performance, the evaluation criterion used is mean absolute error (MAE). Selection of this criterion is due to the fact that MAE is more robust to outliers, compared to other popular methods such as root mean square error (RMSE) [20]. For that purpose, estimation gathered from the proposed method is compared with the actual value, which is 180[degrees]. MAE is defined as follows:

MAE = [1/M] [M.summation over (i=1)] [absolute value of [e.sub.i]], (11)


[e.sub.i] = [f.sub.i] - [y.sub.i]. (12)

Here, [f.sub.i] is the estimation value and [y.sub.i] is the true value.

5. Results and Discussion

In this experiment, ten images were captured by moving the camera in each second. One hundred images have been captured. The average processing time, starting from image preprocessing until angle estimation, is 0.02 sec. This is the average value based on the computation time to process one hundred images. This processing time is much less than the image sampling time of 0.1 sec. By using GLCM and feature vector extraction based on contrast, the AOI was selected for each image. The example of selected AOI is shown in Figures 2 and 3.

In the AOI selection, the values of feature vectors were calculated, and median value was selected. The reason why median value was decided to be used is because, after operation of GLCM and contrast, diagonal value of cooccurrence matrix is equal to zero, which means that similar grey-level values, occurred next to one another, were excluded. It is also understood that higher contrast value is caused by the effect of large contrast weights. This shows that large deviation on grey-level values occurred next to one another. In this test, the image contrast changed along with the camera movement. This is due to the changes in the image texture, which was affected by the changes of environment structure. Therefore, the AOI which shows the median value among the values generated by GLCM and contrast will be selected. However, the decision is made under the assumption that the contrast in the selected AOI does not have significant contrast difference over the time.

As a result of using the selected AOI, computed optical flow field is shown in Figure 4.

In addition, for comparison purpose, optical flow field created using an image without AOI is shown in Figure 5.

In Figure 5, there exist vectors whose directions are scattered. It is also observed that there are vectors whose its magnitude is too small. These types of vectors are called mislead vectors and irrelevant vectors, respectively. Mislead vectors may cause large errors in estimation because it is not related to an actual motion. The reason why these vectors exist is because of an assumption of brightness constancy, where, in a real situation, this is not easily maintained, especially in outdoor environment. On the other hand, irrelevant vectors show that there is almost small or no motion detected at vectors location. This is due to image texture at that location, which is remaining constant. For that reason, through the proposed method, improvement in optical flow generation is discovered.

Using these results, heading angle estimation is figured out. The estimation results are given in Figures 6 and 7, accordingly.

From Figures 6 and 7, it can be noticed that heading angle estimation shows an improvement when proposed GLCM is applied. Without GLCM, as shown in Figure 7, the estimation fluctuated over time, where extremely large spikes occurred that caused to poor estimation. Hence, through GLCM method, significant oscillation reduction is observed which makes the estimation result improved.

Related to estimation performance, Table 1 shows the results of evaluation conducted using criteria that have been mentioned in Section 4.

From Table 1, the evaluation results of the heading angle estimation which incorporates GLCM show better performance. MAE result without GLCM shows large deviation existence between estimation and actual value. It is because calculated absolute error is contributed from numbers of large oscillations as well as mediocre but rapid oscillations, as shown in Figure 7. On the other hand, the MAE value for proposed GLCM is smaller. This small MAE value is due to the estimation result in Figure 6 which shows large deviations between estimation and actual value that occurred at the beginning, and the deviations become smaller as the estimation becomes stable and reaches actual value. Therefore, most errors that contributed to MAE are from large variations between estimations and actual values at the beginning of the experiment.

In addition, estimation results without GLCM in Figure 7 show that the oscillation which occurred frequently caused difficulties to identify the heading angle of the vision system. This problem can be solved when our proposed GLCM method is applied.

6. Conclusion

In this paper, the effectiveness of the proposed method in estimating heading angle of vision system is presented. Each captured image has been divided into nine AOI, and GLCM contrast has been used to extract feature vector in order to identify the most suitable region that can be used for estimation purpose. Optical flow calculated using Horn-Schunck method has been used as a basis in extracting required information. Using generated optical flow, estimation on angle has been implemented and smothered using MMF. The result shows that an improvement in estimation is observed when compared to an angle estimation based on untouched optical flow. Through the proposed method, invaluable information regarding the system's current status can be obtained.


The authors acknowledge the Ministry of Higher Education Malaysia, Universiti Kebangsaan Malaysia, and Universiti Teknikal Malaysia Melaka for their sponsorship in this study.


[1] A. Song, J. Wu, G. Qin, and W. Huang, "A novel self-decoupled four degree-of-freedom wrist force/torque sensor," Measurement, vol. 40, no. 9-10, pp. 883-891, 2007

[2] G. Song, H. Yuan, Y. Tang, Q. Song, and Y. Ge, "A novel three-axis force sensor for advanced training of shot-put athletes," Sensors and Actuators A, vol. 128, no. 1, pp. 60-65, 2006.

[3] J. Wu, Z. Song, W. Wu, A. Song, and D. Constantinescu, "A vibro-tactile system for image contour display," in Proceedings of the IEEE International Symposium on Virtual Reality Innovations (ISVRI '11), pp. 145-150, March 2011.

[4] J. Palacin, I. Valganon, and R. Pernia, "The optical mouse for indoor mobile robot odometry measurement," Sensors and Actuators A, vol. 126, no. 1, pp. 141-147, 2006.

[5] M. Tresanchez, T. Palleja, M. Teixido, and J. Palacin, "The optical mouse sensor as an incremental rotary encoder," Sensors and Actuators A, vol. 155, no. 1, pp. 73-81, 2009.

[6] M. Tresanchez, T. Palleja, M. Teixido, and J. Palacin, "Using the image acquisition capabilities of the optical mouse sensor to build an absolute rotary encoder," Sensors and Actuators A, vol. 157, no. 1, pp. 161-167, 2010.

[7] F. Labrosse, "Short and long-range visual navigation using warped panoramic images," Robotics and Autonomous Systems, vol. 55, no. 9, pp. 675-684, 2007

[8] V. Javier Traver and A. Bernardino, "A review of log-polar imaging for visual perception in robotics," Robotics and Autonomous Systems, vol. 58, no. 4, pp. 378-398, 2010.

[9] M. Kronfeld, C. Weiss, and A. Zell, "Swarm-supported outdoor localization with sparse visual data," Robotics and Autonomous Systems, vol. 58, no. 2, pp. 166-173, 2010.

[10] K. Souhila and A. Karim, "Optical flow based robot obstacle avoidance," International Journal of Advanced Robotic Systems, vol. 4, no. 1, pp. 13-16, 2007.

[11] L. Wang, H. Li, and R. Hartley, "Video local pattern based image matching for visual mapping," in Proceedings of the 18th International Conference on Pattern Recognition (ICPR 06), vol. 2, pp. 67-70, 2006.

[12] G. Medioni, A. R. J. Francois, M. Siddiqui, K. Kim, and H. Yoon, "Robust real-time vision for a personal service robot," Computer Vision and Image Understanding, vol. 108, no. 1-2, pp. 196-203, 2007.

[13] R. M. Haralick, K. Shanmugam, and I. Dinstein, "Textural features for image classification," IEEE Transactions on Systems, Man and Cybernetics, vol. 3, no. 6, pp. 610-621, 1973.

[14] V. Bino Sebastian, A. Unnikrishnan, and K. Balakrishnan, "Grey level co-occurrence matrices: generalisation and some new features," International Journal of Computer Science, Engineering and Information Technology, vol. 2, no. 2, pp. 151-157, 2012.

[15] K. Padmavasavi, N. U. Kumar, E. V. K. Rao, and M. Madhavilatha, "Performance evaluation of adaptive statistical thresholding based edge detection using GLCM in wavelet domain under noisy conditions," ICGST International Journal on Graphics, Vision and Image Processing (GVIP), vol. 10, no. 3, pp. 35-44, 2010.

[16] M. M. Mokji and S. A. R. S. A. Bakar, "Adaptive thresholding based on co-occurrence matrix edge information," Journal of Computers, vol. 2, no. 8, pp. 44-52, 2007

[17] K. H. Ghazali, M. M. Mustafa, A. Hussain, and F. Engineering, "Machine vision system for automatic weeding strategy using image processing technique," American-Eurasian Journal of Agricultural & Environmental Sciences, vol. 3, no. 3, pp. 451-458, 2008.

[18] C. Lane, R. L. Burguete, and A. Shterenlikht, "An objective criterion for the selection of an optimum DIC pattern and subset size," in Proceedings of the 11 th International Congress and Exhibition on Experimental and Applied Mechanics, pp. 900908, June 2008.

[19] B. K. P. Horn and B. G. Schunck, "Determining optical flow," Artificial Intelligence, vol. 17, no. 1-3, pp. 185-203, 1981.

[20] R. J. Hyndman and A. B. Koehler, "Another look at measures of forecast accuracy," International Journal of Forecasting, vol. 22, no. 4, pp. 679-688, 2006.

Zairulazha Zainal, (1,2) Rizauddin Ramli, (1) and Mohd Marzuki Mustafa (1)

(1) Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia, 43600 Bangi, Selangor, Malaysia

(2) Faculty of Mechanical Engineering, Universiti Teknikal Malaysia Melaka, 76100 Durian Tunggal, Melaka, Malaysia

Correspondence should be addressed to Zairulazha Zainal;

Received 14 March 2013; Accepted 7 May 2013

Academic Editor: Aiguo Song

TABLE 1: Performance evaluation
using mean absolute error (MAE).

Method        Mean absolute
              error (degree)

Without AOI        40.9
With AOI           32.5
COPYRIGHT 2013 Hindawi Limited
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2013 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:Research Article
Author:Zainal, Zairulazha; Ramli, Rizauddin; Mustafa, Mohd Marzuki
Publication:Journal of Sensors
Date:Jan 1, 2013
Previous Article:Augmented reality to support on-field post-impact maintenance operations on thin structures.
Next Article:A tent map based A/D conversion circuit for robot tactile sensor.

Terms of use | Privacy policy | Copyright © 2022 Farlex, Inc. | Feedback | For webmasters |