Image retrieval based on color and shape.
Visual information plays an important role in many fields like education, medicine, commerce, entertainment, fashion and graphic design, architectural and engineering design, crime prevention and others.
The earliest use of the term content-based image retrieval in the literature appears in (Kato, 1992) to describe his experiments into automatic retrieval of images from a database, using low level features: color and shape. Then the term was widely used for describing the process of retrieving desired images from a large collection of images based on their low level features (such as color, shape, texture or combining of these features) that can be automatically extracted from the images.
Content based image retrieval differs from classical information retrieval in that the images have unstructured form-they consist of arrays of pixel intensities with no meaning. Thus there is a need to extract useful information from the raw data.
This paper is organized as follows: section 2 presents related existing work for content based image retrieval; section 3 describes two existing methods for objects representation based on color and shape which will be combined in section 4 to perform image retrieval in an improved way; the experimental results are presented in section 5 and section 6 describes the conclusions and future work.
2. RELATED WORK
Visual features: color, shape, texture and spatial relationships are the basis of any content-based image retrieval technique. Many approaches have been developed for each of these visual features. Color is one of the most widely used visual features in content-based image retrieval. The color histogram method as in (Shapiro & Stockman, 2003) has been the most commonly used representation technique. Based on the color histogram, correlograms were developed to capture the distribution of the pixels in particular areas around pixels of particular colors. For image retrieval based on shape, the shape representation should have the following two important properties (Lu, 1999): (i) each shape should have a unique representation, invariant to translation, rotation and scale; (ii) similar shapes should have similar representations so that the retrieval can be based on the distances among the shape representations. There are two types of shape descriptors: contour-based shape descriptors (which exploit only boundary information--they cannot capture the interior content of a shape) and region-based shape descriptors (which consider all pixels within a shape region). Some existing contour-based shape methods are turning angle (Arkin et al., 1991), centroid radii and histogram distances method (Fan, 2001).
This paper combines two methods for shape representation based on color and shape together with a classification method for retrieving images in an efficient way. The classification will reduce the execution time. First it is performed an initial object classification based on shape using the k-means clustering algorithm. And second, the images are retrieved based on color.
3. OBJECT REPRESENTATION
In the following, a shape will be represented by color and boundary shape.
3.1 Object Representation by Color
A method for image representation by color is the color histogram method (Shapiro & Stockman, 2003). This is a representation of the distribution of colors in an image, derived by counting the number of pixels of each given set of color ranges in a typically two-dimensional or three-dimensional color space.
In (Jeong, 2001) the HSV (Hue Saturation and Value) and RGB color spaces are compared. The HSV color space (based on the artists' tint, shade and tone) is closely related to human visual perception.
There are several distance formulas for measuring the similarities of color histograms, like: the Euclidean distance, the histogram intersection or the histogram quadric distance. These distances are compared in (Jeong, 2001), too. Based on that retrieval experiments the histogram intersection distance gives the best retrieval performance.
3.2 Object Representation by Shape
In (Mocanu, 2004) is presented a method for object representation by shape using its boundary information. Based on its boundary, a shape is described by the edge directions and each edge direction has associated a list of its corresponding radii. The edge directions are represented by the turning angles of each edge. Turning angle as in (Arkin et al., 1991) is defined as the angle formed with a reference axis by the counterclockwise tangent to the boundary of a shape which goes from a boundary point of a shape to the next one. The radii corresponding to an edge is the distance between the centroid of the shape and a sample point from that edge. This representation is invariant to translation, but it is not invariant to scale. To make it scale invariant, all the radii will be normalized by dividing their values by the value of the maximum radii. The representation is not rotation invariant.
The distance between two shapes is computed using the distance between the direction lists plus the distance between the radii associated with each direction.
Using this representation a shape may have more edge directions than other. But it does not mean that the shapes are different, because the shapes' boundaries may be affected by noise. These noisy edges may increase the distance between similar shapes. Thus before feature extraction it is necessary to eliminate noisy edges and reduce the number of data points.
4. COLOR-SHAPE RETRIEVAL
Sometimes it is useful to retrieve images based on combining low-level features. For example if we want to find images that contain a yellow fish with shape as in Fig. 1 it will be necessary to describe and retrieve images based on color and shape.
[FIGURE 1 OMITTED]
A possibility to retrieve images based on these two level features is to combine a shape retrieval method with a color retrieval method in a sequential order--for example (i) first retrieve images based on shape features obtaining a subset S from the initial collection of image which contains similar shapes with the query shape and (ii) second, retrieve images from S, based on color features. For example if we have a collection of images with objects which have shapes as in Fig. 2 --in the first step, all the six images S1 ... S6 will be compared with the query shape and we will obtain the subset S formed by S2, S4 and S6.
To reduce the number of compared images based on shape, the images may be classified.
[FIGURE 2 OMITTED]
The proposed method combines shape retrieval with color retrieval--using a shape classification first. The initial component objects from the image collection are classified using the k-means clustering algorithm based on their shapes (MacQueen, 1967). Using the initial collection of shapes objects from Fig. 2 after classification 2 clusters will result each cluster containing similar objects based on shape as in Fig 3. After this classification the initial step of shape search can be made on a subset of the initial objects--it will be necessary to compare the query shape only with the shapes which belong to the cluster to which belongs the query shape, too. For this example the total number of compared objects is not significantly reduced (6--in case of the sequential form and 5--in case of the improved method) because of the small number of the initial images.
The proposed color-shape method can be described as:
* image segmentation, resulting S = a set of component object
* objects description based on shape
* objects description based on color
* objects classification based on shape
* finding the cluster C to which the query shape belongs
* search the cluster C based on shape, resulting a subset S of shapes similar with the query shape
* search the subset S based on color and determine the subset of similar objects with the query
[FIGURE 3 OMITTED]
5. RETRIEVAL EXPERIMENTS
For testing the performance of the proposed method for image retrieval based on shape and color, a set of 200 fish images was used. Based on precision and recall--as in (Lu, 1999)-the performance of the sequential method for image retrieval based on shape and color and the proposed method are the same. But the execution time is reduced in the case of combining the image retrieval based on shape and color together with the k-means clustering with 15% compared to the case of using these two methods in sequential form.
6. CONCLUSION AND FUTURE WORK
This paper presents a method for content based image retrieval using shape and color. The method is based on two existing methods for image retrieval based on shape--the centroid radii and turning angle method, respectively image retrieval based on color--the histogram color distance combined with a classification using the k-means algorithm. The experimental results demonstrate that using shape classification improves the execution time.
When performing image retrieval based on the low-level features (shape and color ) it may happen to obtain a "red rose" as the result to the query for a "red ball" ( a rose image with a shape similar to the shape of a ball may exist; also the two objects have the same color). This issue may be addressed by annotating images. As future work, the classification step may be used together with a method for image annotation in order to associate keywords to each object from an image.
Arkin, E. M.; Chew, L. P.; Huttenlocher D. P.; Kedem K. & Mitchell, j. S. B. (1991). An efficient computable metric for comparing polygonal shapes. IEEE Transactions on PAMI, Vol. 12, No. 3, pp 232-248
Fan, S. (2001). Shape Representation and Retrieval Using Distance Histograms. Technical Report TR 01-14, Department of Computer Science, University of Alberta, Edmonton, Canada 2001
Jeong, S. (2001). Histogram-Based Color Image Retrieval. Available from: http://scien.stanford.edu/class/psych221/projects/02/sojeon g/ Accessed: 2009-05-20
Kato, T. (1992). Database architecture for content-based image retrieval. Image Storage and Retieval Systems, Vol. 1662, (May 1992), pp. 112-123
Lu, G. (1999). Multimedia Database Management Systems, Artech House Publishers, ISBN 0890063427, Boston
MacQueen, J. B. (1967). Some Methods for classification and Analysis of Multivariate Observations. Proceedings of 5-th Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1, pp. 281-297
Mocanu, I. (2004). Shape Representation and Retrieval Using Centroid Radii and Turning Angle. Scientific Bulletin of the Politehnica University of Timisoara. Transactions on Electronics and Communications, Vol. 2, pp. 146-149
Shapiro, L. G. & Stockman, G. C. (2003). Computer Vision, Prentice Hall, ISBN 0130307963
|Printer friendly Cite/link Email Feedback|
|Publication:||Annals of DAAAM & Proceedings|
|Date:||Jan 1, 2009|
|Previous Article:||Control algorithm of a BLDC motor.|
|Next Article:||Fuzzy system for implementing the cracks control during the continuous casting.|