Automated visual inspection for contamination detection in electronic industry.
Key words: Contamination detection, automated visual inspection concept, practical application of BLOBs analysis.
Machine vision has been widely employed in many applications. Automated visual inspection is the well-known application, which uses machine vision for industrial tasks. Compared with the inspection done by human, automated visual inspection provides more reliable results. Thus, it is more suitable for repeated operations such as industrial inspections. In this paper we describe a novel approach using visual inspection to detect the most frequently occurring defects such as contamination of the parts by using automated visual inspection system. We provide implementation details of the system i.e. algorithms, environmental constraints and experimental results.
Compared with other applications of machine vision, automated visual inspection is widely used because of number of reasons. Automated visual inspection system always operates with environmental constraints e.g. controlled lighting, controlled orientation and limited types of inspected products, which simplify system complexity. Additionally, the quality assurance system can be built using detailed inspection tasks such as inspected regions, locations and pass/fail criteria etc. However, there are two important requirements of this type of industrial applications: one is to achieve high inspection throughput, the system operational time including computational time and waiting time for mechanisms movements must be minimized, and second is visual system must be reconfigurable for inspecting different products. The usual approach used to satisfy the first requirement is simplifying system computation by using simple algorithms incorporated with the products knowledge acquired from the system user. To satisfy the second requirement the system is designed in such a way that its user can redefine the inspection tasks to be carried out by the system.
2. Problem definition
Automated visual inspection is not new technology. It has been developed for many years. Its main advantages are: reducing inspection time, reducing human error such as subjective inspection, providing fast operation and collecting more numerical data. Although there are many automated visual inspection hardware and software commercially available for the industrial solution, but most of them are based on proprietary software which makes other programmers difficult to modify, scale and maintain. Moreover all of them are also imported software, which is very expensive.
There is greater pressure on hard disk drive (HDD) companies to provide lower cost, higher quality and faster time to market products. One of the key solutions for the HDD assemblers is to rapidly adopt and implement automation technology. The need for automation as a key tool for competitiveness cannot be ignored. In an industry where technology changes rapidly and levels of precision are getting higher and higher, the old labor intensive production methods are no longer economically or technically feasible, especially as wages rise.
As a result there is demand from many manufacturers for the user-friendly and open source software which allows their own programmers to modify and adjust the operations of the software according to their specific requirements e.g. an ability to adapt the developed software suited to available hardware of each company, inspecting different types of product etc. However, satisfying all requirements from the industries is not an easy task. Therefore the software development plan is divided into two phases. This paper presents the outcome of the first phase of this plan, which has the objective to develop all necessary software building blocks. The outcome of this phase is stand-alone software which can detect the limited types of defects selected by HDD companies. For the second phase, the outcome will be fully automated visual inspection system, which can satisfy all specific requirements from industries. All developed algorithms, software components are available as open source to the project members.
3. Automated Visual Inspection System Design
Since the software developed by this project is used to inspect the different kinds of products, the software must be flexible enough to be adjusted for inspecting many product. Each of these products requires use of different measurement types with different criteria. Additionally, the available resources in the factory must be utilized to help or guide the system. That is, the knowledge of quality assurance engineer or in-charge person, who knows well about the inspected product and their criterion, must be able to be embedded into the designed system.
We have designed an automated visual inspection system in which the user can define the inspection tasks by selecting the measurement tools and graphically specifying the inspected paths or areas of each tool. After defining the number of measurement tools and their criterion used to judge the product whether it fails or pass. The user can save this information into a file and use information in this file to inspect the products. This is only one possible approach that allows us to make the generic software that can inspect different kinds of products of the companies. Moreover this approach allows us to embed the knowledge of the user, who knows about the inspected products, into the system.
For setting up the inspection task for a particular product, a file containing the information inspected areas, inspected paths, different measurement tools and their criterion, must be created by quality assurance engineers or the in-charge persons who knows well about inspected products and their criterion. This file is called template file in this program. After created, this file can be used by the operator using a user friendly interface to inspect the product. The operation of the system can be classified into two phases as follows.
1. Template preparation phase: This phase is performed by the persons who can provide the accurate information and pass/fail criterion for inspected product.
2. Inspection phase: This phase is performed by an operator. Once the operator selects the template file related to the inspected products, the system use the information in a template file to automatically inspect the products.
[FIGURE 1 OMITTED]
In the designed system, there are line-based tools and area-based measurement tools. Some of the tools can be configured to be either line or area based measurement tools depending on user requirements. The user can specify the inspected paths of the line-based measurement tools by using standard shapes such as circle, rectangle, ellipse or polygon. The user can also specify inspected areas of the area-based measurement tools by using standard shapes such as circle, ellipse, rectangle and solid polygon. This means we break the image into a number of sub-images with require different types of preprocessing and different types of analysis. This makes computational speed faster than processing the whole image for the same preprocessing operations by considering only part of image. This approach makes the designed software more suitable for practical application which needs the different measurements for different product area.
4. Measurement tools
There are 6 measurement tools developed in this research. These tools can be configured to be either line-based measurement tools or area-based measurement tools in the mentioned shapes. They allow user to freely define the required inspection tasks for each product. These measurement tools are detailed as following.
1. Tool for analyzing a histogram of line or area A user can define its pass and fail criteria based on intensity contrast of either pixels along a line or pixels in the specified area. This tool should be the first item in list of inspection tasks because light intensity must be accurate before other inspection tasks are performed.
2. Tool for analyzing intensity profile along the line A user can define its pass and fail criteria based on either a number of edges or a number of features found along the tool path. Edge defined in this research is a transition between dark and bright pixels. Dark pixels have intensity values lower than threshold value. Bright pixels have intensity values equal to or higher than threshold value. Feature is defined as a dark or/and bright series of pixels. It is evaluated based on size defined by user. Thus the features can be either dark features, bright features or both.
3. Tool for analyzing a line or a circle A user can use this tool to detect a line and define pass and fail criteria based on slope, offset and line straightness. A user can also use this tool to detect a circle and define pass and fail criteria based on radius, center and circle roundness.
4. Tool for analyzing colors of the area This is a tool used for examining colors distributions of pixels in the specified area. The color space can be configured to be either red, green, blue (RGB) or hue, saturation, intensity (HSI) color spaces. Pass and fail criteria are based on standard deviations of colors and differences between means of measured colors and reference values.
5. Tool for detecting rotating and translating amounts of product This tool can be used as a reference for other measurement tools. That is, rotating and translating amounts of product, which are detected by this tool, can be used for transforming coordinates of referring measurement tools as shown in figure 2.
6. Tool for analyzing Binary Large Objects (BLOBs) Contamination detection can be one by using this tool. It was designed for counting a number of connected components, which are considered as objects of interest. Before a system can automatically count a number of objects of interest, a user needs to provide the definition of objects of interest to the system by defining boundaries of feature values of objects of interest. There are a number of features used for capturing the definition of objects of interest i.e. area, perimeter, angle of major axis, bounding box size, centoid position and average intensity. These features allow a user to define different definition of objects of interest suited for each application. An operating procedure of this tool is illustrated in figure 3.
[FIGURE 2 OMITTED]
[FIGURE 3 OMITTED]
Acquiring image data from either live high-resolution camera or image file with .bmp extension is the first step of a procedure. Then defining the inspected regions is graphically done by user. The regions can be defined as rectangle, ellipse or polygonal shapes. Compared with all shapes, polygon is the most powerful and the most frequently used shape because it can form an arbitrary shape, which is suitable for product with complicated shapes as shown in figure 4.
[FIGURE 4 OMITTED]
If a tool for detecting rotating and translating amounts of product is set as a reference, a system transforms tool coordinates according to detected rotating and translating amounts. Then, the modified scanline algorithm (Donald Hearm & M. Pauline Baker, 2004) is used for extracting transformed polygonal sub image. Because the polygonal sub image is, in practical, kept in rectangular form in computer memory, the outer region of polygonal sub image is filled by negative value of an intensity of objects of interest. Additionally pixels with negative values of intensity of objects of interest will be ignored by connected components extracting algorithm. For example: outer region of extracted sub image in figure 5a is filled by 255 because in this case, a user needs to detect the dark contaminations, which have intensities equal to 0. Subsequently, thresholding is applied to sub image and the result is shown in figure 5b. The system can provide a user the recommended threshold value, which is selected by using Otsu's method (J.R. Parker, 1997).
[FIGURE 5 OMITTED]
Then, binary morphological operators and connected components extraction algorithm are sequentially applied to binary sub images. There are 4 traditional binary morphological operators implemented in this inspection system, i.e. dilation, erosion, closing and opening. These operators can be used for filling holes on product surface or saparating connected parts caused from applying unsuitable global threshold to local regions. Additionally, a user can freely select a size of structuring element used with these operators.
The connected components extraction algorithm used in this research is modified from two passes algorithm (Ramesh et al., 1995), which is a sequential algorithm used for connected components labling only. The modified algorihtm calculates features of connected components on the fly. That is, while identifying which pixels belong to which components, the modified algorithm also calculates features of each components, i.e. size of a bounding box, area, perimeter, angle of major axis, centoid position and average intensity. The algorithm reduces computational time compared with extracting features of each compenent from label matrices. Based on measured features, we can provide the definition of objects of interest to the system and let the system automatically counts the number of objects of interest found in sub images. Providing the definition of objects of interest can be done by specifying boundaries of feature values of objects of interest. This process can be done iteratively. That is, we observe the feedback from the binary image, which contains the bound connected components indicating detected objects of interest and numerical data in a display list, and then, modify the values of boundaries until we get the required result. After specifying the definition of object of interest, we can also specify pass and fail criteria of the measurement tool by giving the minimum and maximum numbers of objects of interest found in the tool area. For contamination detection, both numbers must be set to zero.
[FIGURE 6 OMITTED]
5. Experimental result
Figure 5b shows all connected components, which are detected by the modified connected components algorithm. Without filtering out by using the definition of object of interest, a number of objects of interest equal to a number of extracted connected components. After the user defines the object of interest by specifying boundaries of its feature values, only connected components, which their features falling in the specified boundaries, are considered as objects of interest. For example : if user specifies the minimum area of objects of interest to 5 pixels, the result will be as seen in figure 7. Compared with figure 5b, figure 7 consists of only connected components which are considered as objects of interest.
[FIGURE 7 OMITTED]
This paper illustrates the practical approach for detecting contamination, which is the most frequently occurring defect in electronic industry. Contamination detection can be done by using tool for analyzing Binary Large Objects (BLOBs).
The inspection tool is embedded in the proposed automated visual inspection system. The proposed system can also be used in many industries because it consists of several inspection tools as described before.
A tool for analyzing Binary Large Objects was designed to count only connected components, which are considered as objects of interest. We used algorithms such as modified scanline algorithm and the modified connected components extraction algorithm, which extracts features of connected components on the fly. Before a tool can automatically count the number of objects of interest, the user needs to provide the definition of objects of interest to a system. This can be done iteratively. The system provides user with numerical feedback via list of connected components features and image feedback via bound connected components in binary image. User modifies boundaries of feature values of objects of interest. After acquiring this information from the expert, this knowledge is embedded into inspection system and a system will use it for automatically inspecting target products. Moreover, a tool for analyzing Binary Large Objects can be used for other applications e.g. part counting, flaw detection and position referencing.
Although the tool for analyzing Binary Large Objects can successfully detect contaminated products, there are 2 points which need improvement as follows.
1. The current version extracts only geometrical features of connected components. In the future, to improve capturing the definition of objects of interest extracting texture features of connected components, e.g. average hue, average saturation, statistical data of co occurrence matrix etc., should be implemented.
2. The current version identifies objects of interest by comparing features of each extracted connected component with boundaries, which are iteratively defined by the user. Although this is the correct approach to acquire knowledge about the product from user, it takes long time to complete. In future, advanced classification e.g. artificial neural network, fuzzy logic system etc., should be employed. This will reduce time used for defining objects of interest. That is, the user just tells the system whether connected components found in example image are objects of interest or not.
The authors would like to thank the following organizations for technical and funding support.
1. National Electronics and Computer Technology Center (NECTEC) National Science and Technology Development Agency (NSTDA)
2. Hitachi Global Storage Technologies (Thailand) Co., Ltd.
3. Seagate Technology (Thailand) Co., Ltd.
4. Western Digital (Bang Pa-In) Co., Ltd.
5. Institute of FIeld roBOtics (FIBO) King Mongkut's University of Technology Thonburi
Donald Hearm & M. Pauline Baker (2004). Computer Graphics with OpenGL, Pearson Education, Inc., ISBN 0-13-015390-7, New Jersey.
J.R. Parker (1997). Algorithm for image processing and computer vision, John Wiley & Sons, Inc., ISBN 0-471-14056-2, New York.
Ramesh Jain; RangacharKasturi & Brian G. Schunck (1995). MACHINE VISION, McGraw-Hill, Inc., ISBN 0-7-032018-7, New York.
This Publication has to be referred as: Kiatpanichagij, K. & Afzulpurkar, N. V. (2006). Automated visual inspection for contamination detection in electronic industry, Chapter 27 in DAAAM International Scientific Book 2006, B. Katalinic (Ed.), Published by DAAAM International, ISBN 3-901509-47-X, ISSN 1726-9687, Vienna, Austria
Authors' data: Mr. Kiatpanichagij K.[irkpong], Associate Professor Afzulpurkar N.[itin] V., Asian Institute of Technology, Thailand, firstname.lastname@example.org, email@example.com
|Printer friendly Cite/link Email Feedback|
|Author:||Kiatpanichagij, Kirkpong; Afzulpurkar, Nitin V.|
|Publication:||DAAAM International Scientific Book|
|Date:||Jan 1, 2006|
|Previous Article:||Improvement of business process for SME on the basis of quality system.|
|Next Article:||IT balanced scorecard as IT governance framework.|