Printer Friendly

An Efficient Multi-Scale Local Binary Fitting-Based Level Set Method for Inhomogeneous Image Segmentation.

1. Introduction

In the process of researching and applying images, people tend to be interested only in certain parts of the image, often referred to as target or foreground; they generally correspond to specific regions of the image that have unique properties. In order to identify and analyze the target, these areas need to be separated and extracted, and then it is possible to make further use of the target, such as feature extraction and measurement. Image segmentation is the technique and process of segmenting an image into distinct regions and extracting interesting objects. Image segmentation is the key step from image processing to image analysis, and it is also a basic computer vision technology. Image segmentation has been paid great attention for many years; so far, a lot of image segmentation algorithms [1-5] have been proposed. In particular, the active contour models [6-7] have been widely used because they are able to provide smooth and closed boundary contours as segmentation results. The level set method [8] is an implicit representation of active contours. Compared to explicit active contour models [6, 9] which utilize parametric equations to represent evolving contours, level set methods represent the evolving contours as the zero level set of a higher-dimensional function, thus making them numerically stable and easily able to handle topological changes. Without loss of generality, we can classify the level set-based active contour models into two categories: the edge-based models [7, 9-11] and the region-based models [12-16]. The edge-based model uses the gradient information of the image to construct the driving force required for the evolution process. Such models are not only sensitive to noise interference but also difficult to detect weak target boundaries. In addition, the final output is heavily dependent on the initial position of the contour. The region-based model constructs the driving force needed for the evolution process based on the regional statistical information of the image. Compared with the edge-based methods, such methods have the following advantages: (1) They do not rely on the gradient information of the image, so they can segment the weak edges, and (2) because the region information adopted is global, it is usually robust to noise. One of the most successful region-based models is the Chan-Vese (CV) model [12], which has been widely used in binary phase segmentation with the assumption that each image region is statistically homogeneous. However, the homogeneity assumption cannot precisely describe the intensity distribution of region with intensity inhomogeneity. Thus, it often fails to segment the images with intensity inhomogeneity.

In order to overcome the segmentation difficulty caused by the intensity inhomogeneity, the researchers have proposed some local region-based segmentation models: they are the local binary fitting (LBF) model [17], the local Gaussian distribution fitting (LGDF) model [18], the local image fitting (LIF) model [19], and so on. These methods generally believe that the images with intensity inhomogeneity satisfy the assumption of homogeneity within a very small local region; that is, within a sufficiently small local image region, we can assume that the intensity of the image is approximately statistically uniform. Thus, by fitting the given image in the sense of local region rather than global region, they can segment the images with slight inhomogeneity.

In practical implementation, they generally use a statistical function with a fixed scale to measure the characteristic parameters of the local region centered at the current sampling point. However, the degree of inhomogeneity between different local regions is usually inconsistent; that is to say, the nonlinear phenomenon of inhomogeneity is very common. Therefore, the practice of fixing the scale for all local regions does not apply to the images with severe inhomogeneity. In view of the universality of the aforementioned issues, to improve the segmentation performance of severe inhomogeneous images, we need to introduce multiscale idea into our model framework.

In this paper, we propose a level set model based on MLBF by introducing the idea of multiscale modeling into the original LBF model and apply it to the practice of inhomogeneous image segmentation. Firstly, an implicit scheme called AOS [20] is utilized to break the time-step limitation of traditional explicit schemes. Under this numerical implementation strategy, the iterative process can take a larger time step, so the evolution curve can quickly converge to the real target contour. Secondly, an automatic initialization strategy driven by a salient target detection mechanism is adopted. By performing a CV [12] model-based segmentation operation on the output of the salient object detection algorithm, we can get the initial curve required for the evolution process. This process is completely automated and does not require any form of human involvement. Thirdly, a level set function update strategy called SFM [21] is used to minimize the number of pixels of the level set function that need to be updated in each iteration. Under the SFM framework, the object to be updated for each iteration is only one-pixel width; thus, the SFM is an extreme narrowband [22, 23] strategy. Obviously, this strategy can further accelerate the evolution of the level set function. Fourthly, to further control the smoothness of the evolving curve and avoid the oversegmentation phenomenon, the regularization term is included in the energy functional. Finally, the multiscale segmentation is performed by minimizing the new formed level set energy functional.

The remainder of this paper is organized as follows. Section 2 is a brief description of the background. Section 3 presents the proposed model. Section 4 gives the three implementation strategies adopted in this paper. Section 5 validates the proposed model by extensive experiments and discussions on a lot of images. Last, conclusions are drawn in Section 6.

2. Background

2.1. Level Set Method. The level set methods implicitly represent the planar closed curve C by the zero level set of a Lipschitz function [phi](x,y,t): [OMEGA] [right arrow] R, such that [phi](x,y,t) >0 if the point (x, y) is inside C, [phi](x, y, t) < 0 if (x, y) is outside C, and [phi](x, y, t) = 0 if (x, y) is on C.

The variable t in the expression [phi](x, y, t) indicates that the level set function changes with time, and its evolutionary process can be expressed by the following equation:

[partial derivative][phi]/[partial derivative]t = V | [nabla][phi], (1)

where "[nabla]" represents the gradient operator, V is the speed function of the evolutionary process, and [phi] is the LSF.

The variational LSF considers LSE as a problem of minimization of certain energy functional E([phi]), that is,

[partial derivative][phi]/[partial derivative]t = -[partial derivative]E([phi])/[partial derivative][phi]. (2)

By using different energy terms to express the information components related to the evolutionary process, the active contours will be freely changed for different application purposes. Thus, the variational level set methods are convenient for developing new segmentation models and have received great attention in recent years.

The core idea of the level set method is as follows: when dealing with the evolution of the plane contour, the level set method does not directly track the position of the active contour, but updates the LSF through the evolution equation shown in (1) and then achieves the purpose of updating the active contour hidden in the LSF [8]. The biggest advantage of this curve evolution style is that the LSF remains a valid function even if the topological change (splitting or merging) occurs in the closed curve which is hidden in the LSF. Figure 1 shows an example of the LSE; the first row represents the three states in the LSE process, and the second is the zero level set curve corresponding to the first row. From this diagram, we can find that the topological structure changes of the evolving curve can be well handled (the topological change here is splitting type) by using the level set expression.

2.2. LBF Model. Li et al. [17] proposed a novel region-based active contour model which takes full account of the local information of the image, and some good segmentation results are obtained on the nonhomogeneous images. The energy functional is defined as follows:

[mathematical expression not reproducible] (3)

where I is the input image, [OMEGA] is the integration region corresponding to the image plane, [[mu].sub.1] and [[mu].sub.2] are control parameters, [K.sub.[sigma]] is a Gaussian kernel function with standard deviation equals to [sigma], and [f.sub.1] and [f.sub.2] are two fitting functions which approximate the local image pixel values inside and outside contour C, respectively.

Similar to other segmentation models based on level set, we still describe the LBF model in the level set framework and use [phi] to represent the LSF. Minimizing the energy functional [E.sup.LBF] with respect to [phi] by using the calculus of variation and the steepest descent method [24], we can easily deduce the corresponding gradient descent flow as

[partial derivative][phi]/[partial derivative]t = -[[delta].sub.[epsilon]] ([phi]) ([[mu].sub.1][e.sub.1] - [[mu].sub.2][e.sub.2]). (4)

[e.sub.1] and [e.sub.2] in (4) are defined as

[e.sub.1] (x) = [[integral].sub.[OMEGA]] [K.sub.[sigma]] (y - x) [[absolute value of I(x - [f.sub.1](y))].sup.2] dy,

[e.sub.2] (x) = [[integral].sub.[OMEGA]] [K.sub.[sigma]] (y - x) [[absolute value of I(x - [f.sub.2](y))].sup.2] dy, (5)

where [OMEGA] is the integration region corresponding to the entire image, and

[f.sub.1] (x) = [K.sub.[sigma]] * [[H.sub.[epsilon]]([phi])I (x)]/[K.sub.[sigma]] * [H.sub.[epsilon]] ([phi]),

[f.sub.2] (x) = [K.sub.[sigma]] * [[1 - H.sub.[epsilon]]([phi])I (x)]/[K.sub.[sigma]] * [1 - H.sub.[epsilon]] ([phi])


In the above equations, we actually use the regularized versions of Heaviside function H and Dirac function [delta] which are expressed as follows:

[H.sub.[epsilon]] (z) = 1/2 [1 + 2/[pi] arctan (z/[epsilon])],

[[delta].sub.[epsilon]] (z) = 1/[pi] * [epsilon]/[[epsilon].sup.2] + [z.sup.2], z[member of]R. (7)

The parameter [epsilon] affects the profile of [[delta].sub.[epsilon]] ([phi]). A bigger [epsilon] will cause a broader profile, which will expand the capture scope but decrease the accuracy of the final contour.

In actual calculation, the construction process of the local binary image ([[mu].sub.1][e.sub.1] - [[mu].sub.2][e.sub.2]) is based on all the pixels in a local Gaussian window; this localization property is the real reason for the LBF model to be able to segment nonhomogenous images. However, when the contour is located at a certain location where [[mu].sub.1][e.sub.1] - [[mu].sub.2][e.sub.2], the local image fitting force will be zero, which leads to the evolution process to be trapped into certain local minima; thus, the segmentation result has a strong correlation with the initial position of the curve.

3. The Proposed Segmentation Model

The energy functional corresponding to the proposed level set segmentation model is composed of two parts: local term [E.sup.L] and regularization term [E.sup.R].

3.1. Local Term. The LBF model has achieved good results on the problem of inhomogeneous image segmentation; however, when the inhomogeneity of the image is severe, the segmentation performance of this model is drastically reduced. The reason is that for a given input image, the model's scale parameter has only one single value. In the real world, more cases are such that the degree of inhomogeneity within different image areas may be different; thus, the scale parameter of the model should not take only one value. In view of this, we introduce multiscale idea into the LBF model to improve the adaptability of the model to severe inhomogeneous images; the following is the local energy term after the introduction of multiscale idea:

[mathematical expression not reproducible] (8)

where [OMEGA] is the image region, N is the total number of Gaussian kernel functions, [f.sub.1,n] (x) and [f.sub.2,n] (x) are the values of [f.sub.1] (x) and [f.sub.2](x) of the LBF model on the nth scale, respectively, [w.sub.n] is the weight factor of the nth fitting energy term, x is the central sampling point, and [mathematical expression not reproducible] has the following expression:

[mathematical expression not reproducible], (9)

where [[OMEGA]] and [[OMEGA].sub.out] correspond to the image regions inside and outside the active contour, respectively, and [[lambda].sub.1] and [[lambda].sub.2] are two positive constant coefficients.

The MLBF model uses a series of Gaussian kernel functions with different scale parameters to deal with the image data with scale nonuniformity (corresponding to intensity inhomogeneity). By using the LBF model as our theoretical basis and introducing multiscale information, the MLBF model can express the global properties of the input image. Because the LBF model is the ideological foundation of the MLBF model, the model of this paper inherits the segmentation ability of the LBF model completely in terms of handling inhomogeneous images. At the same time, the MLBF model is more robust to the position of the initial contour and the noise of Gaussian type.

As mentioned above, it is inappropriate to use the same scale over all local regions when segmenting images that have serious intensity inhomogeneity. Here, we give an example to verify this conclusion.

In Figure 2, we apply the LBF model to segment the images with slight and severe inhomogeneity. Figure 2(a) shows a sample image with slight inhomogeneity (the grayscale range of the target area is 44,193]). We make two markers ([P.sub.1] and [P.sub.2]) on this image, where the x-shaped mark is the position where the current pixel is; the circular mark is the local region (its scale is equals 15 pixels) corresponding to the current pixel. From the two grayscale histograms shown in Figures 2(b) and 2(c), we can see that the histograms of the local regions corresponding to center points [P.sub.1] and [P.sub.2] have similar grayscale distribution ranges. In these two distributions, one of them ranges from 41 to 64, and the other ranges from 157 to 181; the grayscale spans of these two distributions are approximately equal to 13; that is to say, the inhomogeneity of the image does not change greatly when the image data goes from top to bottom. Therefore, when we apply the LBF model to this image, we can get the correct segmentation result shown in Figure 2(d). Figure 2(e) shows another example image, which differs from Figure 2(a) in that it has severe inhomogeneity (the grayscale range of the target area is [25,229]). Similar to the aforementioned approach, we also mark two sampling points on this image; they are [P.sub.3] and [P.sub.4]. From the distribution histograms shown in Figures 2(f) and 2(g), we find that the inhomogeneity of this image is much stronger than that shown in Figures 2(b) and 2(c), and the local region corresponding to center point [P.sub.4] has a higher inhomogeneity than [P.sub.3]. As we expected, the LBF model outputs the wrong segmentation result (as shown in Figure 2(h)) on this image. Thus, in order to accurately segment this type of image, the scales at the local areas indicated by the two circles should take different values.

3.2. Regularization Term. When implementing the traditional level set segmentation models, the upwind schemes are often used to keep numerical stability [24], and the LSF is initialized to be a signed distance function (SDF). Since the LSF is usually very flat or steep near the zero level set in the LSE evolution, this will affect the numerical stability; a remedy procedure called reinitialization is applied periodically to enforce the degraded LSF being an SDF. The researchers have developed a number of computational strategies [22, 25-28] for the problem of reinitializing LSF. Although these methods have achieved some success, the following problems exist: (1) preventing new zero contours from emerging, which may cause undesirable results for image segmentation, such as failures to detecting the interior boundary, and (2) the reinitialization process which requires a lot of CPU time, resulting in an extension of the LSE process.

In order to solve the aforementioned problems, in recent years, some variational level set formulations [28-32] have been proposed to regularize the LSF during evolution, and hence the reinitialization procedure can be eliminated. Among these methods, we choose Zhang et al.'s [32] method as our regularization strategy; the reason is that their method can completely eliminate the reinitialization process and has a strict theoretical derivation.

Under the variational level set framework, the LSE equation can be formulated as

[[phi].sub.t] = -[E.sub.[phi]] ([phi]) = F[delta]([phi]),

[phi] (x,t=0) = [[phi].sub.0] (x), (10)

where [E.sub.[phi]] ([phi]) denotes the Gateaux derivative of the energy functional E([phi]), [delta]([phi]) is the Dirac functional, F is the force function, and [[phi].sub.0] (x) is the initial LSF.

By adding a diffusion term "[epsilon]V[phi]" into (10), we have the following reaction diffusion (RD) equation for LSM:

[[phi].sub.t] = [epsilon][DELTA][phi] - 1/[epsilon] (-F[delta]([phi])),

s.t. [phi] (x,t = 0, [epsilon]) = [[phi].sub.0] (x), (11)

where [epsilon] is a small positive constant and [DELTA] is the Laplacian operator. Equation (11) has two dynamic processes: the diffusion term "[epsilon][DELTA][phi]" gradually regularizes the LSF to be piecewise constant in each segment domain [[OMEGA].sub.i], and the reaction term "1 - 1/[epsilon](-F[DELTA]([phi]))" forces the final stable solution of (11) to "-F[delta]([phi]) = 0," which determines [[OMEGA].sub.i]. Due to the absence of the diffusion term, the traditional LSMs have to regularize the LSF by an extra procedure, that is, reinitialization. That is to say, the gradient descent flow corresponding to the regularization term [E.sup.R]([phi]) can be expressed as follows:

[partial derivative][E.sup.R] ([phi])/[partial derivative][phi] = - [kappa] * [DELTA][phi], (12)

where k is a control parameter.

From (12), we can deduce the expression of [E.sup.R] ([phi]):

[E.sup.R] ([phi]) = [kappa]/2 [[integral].sub.[OMEGA]] [[absolute value of [nabla][phi](x)].sup.2]dx. (13)

In summary, by adding a RD term into the LSE equation, the regularization of the LSF can be achieved, thus completely eliminating the time-consuming reinitialization process.

3.3. Level Set Formulation. With the level set representation, the total energy functional []([phi]) can be reformulated as follows:

[mathematical expression not reproducible] (14)

where [OMEGA] is the entire image region to be segmented, [[lambda].sub.1] and [[lambda].sub.2] are positive constants, and

[M.sub.1] ([phi](y)) = [H.sub.[epsilon]] ([phi](y)),

[M.sub.2] ([phi](y)) = 1 - [H.sub.[epsilon]] ([phi](y)). (15)

[mathematical expression not reproducible]. (16)

Keeping the multiscale fitting energy function [f.sub.1,n] and [f.sub.2,n] fixed, and minimizing the energy functional []([phi]) with respect to [phi], we obtain the following gradient descent flow equation:

[mathematical expression not reproducible], (17)

where [kappa] is the same control parameter as (12), and

[mathematical expression not reproducible], (18)

[mathematical expression not reproducible]. (19)

In order to solve the partial differential equation shown in (17), in this paper, three special strategies are adopted to improve its computational efficiency; the details related to these three strategies are shown in Section 3.

4. Implementation Strategies

4.1. AOS Solver. An explicit scheme is the most popular method for solving (17) [33], but due to the limitations of the Courant-Friedreichs-Lewy (CFL) [24] condition, which asserts that the advancing speed of numerical waves cannot exceed the evolution speed of physical waves. Thus, the active contour can move only a small distance in each iteration; this means that we can only take a small time step. If the active contour is not near the target edge, the evolution process will take a long time to reach its convergence state. In order to remove the restriction on the time step and obtain fast convergence, we introduce the fast AOS [34] scheme to solve the terms which are marked by operator div in (17); however, the existence of [delta]([phi]) leads some differences between our terms and the processing objects of AOS. Fortunately, Chan et al. [12] indicate that [[delta].sub.[epsilon]] ([phi]) can be replaced by [absolute value of [nabla][phi]]; moreover, in the restriction on signed distance function, we have [absolute value of [nabla][phi]] = 1, so our equation will be an appropriate object that can be handled by AOS. The first term on the right side of (17) has no relation with the gradient of the LSF, so it can be treated as a constant. In our previous paper about the AOS scheme, we have given the AOS scheme for the terms which are marked by operator div (for more details, please refer to [20]).

4.2. Salient Target Detection Mechanism-Based Automatic Initialization of Level Set. Since the evolution equation of the level set is carried out in the form of iterations in the concrete execution, we need to set an initial value for the LSF so that our time iteration process has a reference point; the setting method of the initial value is usually divided into two forms: (1) Man-made type. For example, a curve is specified by hand drawing (by sliding the mouse over the image area to complete the interactive design of the curve) or by parameterizing (giving all the associated parameters of the curve), and then the signed distance function corresponding to this curve is taken as the initial value of the level set. (2) Algorithm automatically generated type. For example, the third-party presegmentation algorithm is used to output the rough foreground region of the image, and then the contour of the foreground region is extracted and taken as our initial curve. Finally, the signed distance function corresponding to this curve is taken as the initial value of the LSF. Under different application scenarios, we are free to use these two types of initial curve setting strategies; their convenience is of no doubt, but their respective shortcomings are obvious: The shortcomings of the first approach are people's strong subjectivity, so it usually unconsciously places the initial curve within the neighborhood of the target of interest. As a result, for the execution process, it is impossible to make an objective evaluation of the overall performance of the level set segmentation algorithm. The shortcomings of the second approach are that the output of the presegmentation algorithm cannot be predicted; when the presegmentation effect is poor, the corresponding initial curve must be very disturbing, and there is no doubt that additional interference is added to our level set segmentation task.

It is true that the targets we intend to segment usually have some form of saliency; therefore, we may achieve a predetection of the target area through a salient target detection mechanism, and the outer contour of the output is taken as our initial curve.

Figure 3 shows the performance of some classic salient target detection algorithms on several sample images (the experimental results section also uses this set of images), where the first and last columns are the input images and ground truth, respectively, and the second to the sixth columns are the saliency maps by using the graph-based (GB) algorithm [35], frequency-tuned (FT) algorithm [36], spectral residual (SR) algorithm [37], context-aware (CA) algorithm [38], and region covariance-based (COV) algorithm [39], respectively. In the saliency map, the greater the gray value of these results, the greater the significance of the corresponding position in the input image, and vice versa.

From the saliency target output results shown in Figure 3, we found that the GB model outputs the optimal saliency map; the reason is that its results have the following two characteristics: (1) The pixels in the background area are less salient, and (2) the pixels in the target area typically have a higher saliency value. In view of this, in the experimental part of this paper, we choose the GB model to achieve the detection of salient targets. In addition, we also found a common phenomenon that the output of a significant target detection algorithm is usually a nonbinarized image with a suspected target area; it cannot directly give the external contour of the suspected target area, thus we need to further segment the output of the salient detection algorithm in order to get the initial curve of the level set.

Because the output of the salient target detection algorithm has a high degree of fuzzy characteristics, the external contour of the suspected target area is not obvious; therefore, the traditional threshold segmentation algorithms cannot get the ideal segmentation results on this kind of images. However, the CV model based on the theory of level set has excellent segmentation performance for such edgeless images. Therefore, this paper uses the CV model to segment the suspected target areas of the salient target detection algorithms.

Figure 4 shows the segmentation results of the CV model on the saliency map shown in the second column of Figure 3; in order to demonstrate the initialization performance of this processing mode, the final contour curve is superimposed on the original input image shown in Figure 3. In initializing the LSE of the CV model, we take the simplest way to draw a rectangle directly above the image, which is 5 pixels from each edge of the input image.

4.3. Sparse Field Method for Fast Local Computation. In traditional level set methods, it is needed to update the LSF value in the whole image plane, which greatly increases the CPU time consumption of the operation process. In order to overcome this drawback, a narrow band approach which was initially proposed in [22] and extensively analyzed and optimized in [23] is proposed; its key idea is to deal only with pixels which are close to the latest position of the zero level set in both directions (inwards and outwards), and the SFM [21] takes this strategy to the extreme since it computes the updates on a band of the grid points that is only one point wide. Therefore, the calculated amount increases with the size of the curve length, rather than the resolution of the grid.

The SFM uses lists of points that represent the zero level set as well as points adjacent to the zero level set. By using these lists and carefully moving points to and from the appropriate list, a very efficient representation of [phi] can be maintained.

These lists are implemented as doubly linked lists. These lists have the properties that elements can be added dynamically and that elements can be removed from the middle of the list. This type of data structure is available in most programming languages (e.g., the vector class in C++).

Five lists are used in the SFM to represent five different levels:

[mathematical expression not reproducible]. (20)

Each list holds the x, y, and z location of pixels in the image. In addition to the lists, two arrays are used. The first is the [phi] array. It is the same size as the image domain and should be maintained at full floating point precision. The second array is a label map the same size as [phi]. This label map is used to record the status of each point and provides a way to look up which list a point belongs to. The label map will only contain the values {-3, -2, -1,0,1,2,3}.

The procedure of SFM can be divided into two stages: initialization and contour evolution.

Figure 5 shows an instance of the initialization of SFM. In the LSF evolution stage, the updated status of [L.sub.0] is determined by the level set iteration. Then, by fusing the neighborhood information, the updated status of points around L0 can be deduced and stored in the following five new lists:

[mathematical expression not reproducible]. (21)

Finally, points in the above new lists update their status and one movement of the contour is finished. This process is repeated until convergence is reached.

It should be noted here that the input of the SFM is the updated LSF corresponding to the pixels in [L.sub.0] which is only one pixel wide; by using the AOS strategy to solve [[phi].sup.n+1] shown in (17), we can get the updated LSF.

In addition, from the implementation of SFM, we see that the input LSF can be processed in parallel fashion; thus, we can use GPU or other parallel architectures to obtain further acceleration.

5. Experimental Results and Discussions

In this section, we will carry out a series of experiments on some synthetic and real images with slight and severe intensity inhomogeneity to verify the validity of the model presented in this paper. The experiments were implemented by Matlab R2012a on a computer with Intel Core i7 2.3 GHz CPU, 8 G RAM, and Windows 7 operating system. The CPU time in this paper starts from the time when the salient target detection mechanism-based automatic initialization is completed. Here, we configure the common parameters in the experiments according to the following values: The time step [DELTA]t for the evolution process of LSF is set to 15; the parameters [[lambda].sub.1], [[lambda].sub.2], and k are all set to 1; and the parameter N (the total number of Gaussian kernel functions) is set to 7.

In order to test the generalization ability of the proposed model, we adopted a method of randomly selecting the test images. Some of these images come from the literatures on image segmentation, and some come from the Internet. These images are weakly related to each other.

In the following, we will evaluate the performance of the proposed algorithm from three aspects: accuracy, fastness, and stability; they are represented by contour location accuracy, speed of evolution convergence, robustness against initial contour position, and noise interference, respectively.

The contour location accuracy of the MLBF model is evaluated by applying it to the real images shown in the first column of Figure 3. In this group of experiments, we also give the segmentation results of several other classical models, in which the level set-based segmentation methods have a CV model [12] and geodesic active contours (GAC) model [1], and the traditional segmentation methods include expectation maximization (EM) algorithm [40], Nobuyuki Otsu's (OTSU) algorithm [41], and pulse-coupled neural network (PCNN) algorithm [42]. In order to facilitate the subsequent description, we give each input image a number; the digital numbers corresponding to the input images of Figure 3 are 1 to 6. Figure 6 shows the results of this set of comparison experiments, where the first row is the input images and the initial contours required for the level set-based segmentation methods, and the second to the seventh rows are the segmentation results by using the CV, GAC, EM, OTSU, PCNN, and our MLBF models, respectively. Figures 6(a) and 6(b) follow the same image layout. In order to be consistent with the expression form of ground truth, we use two valued images to represent the segmentation results corresponding to the final evolution curve of the level set.

From this set of real image segmentation experiments (as shown in Figure 6), we found that the traditional segmentation methods are difficult to obtain ideal segmentation results, and they all have different degrees of defects; however, our method outputs accurate segmentation results. The following analysis gives the general cause of the aforementioned phenomenon: (1) The logical basis of the CV model is that the gray value of the target area is uniform. In most cases, this condition cannot be satisfied; for example, on the inhomogeneous images shown in this experiment, the model outputs wrong segmentation results. (2) The GAC model is a level set model based solely on image gradient information, so the model is powerless for weak target edges. (3) Because only the intensity information can be used, the statistical modeling ability of the EM algorithm will be affected, and then its final segmentation performance will be affected. (4) The segmentation process of the OTSU algorithm is based on the histogram; therefore, the spatial information of the image is sacrificed. When the target area of the input image is not uniform, the segmentation result is inevitably deviated. (5) When the PCNN algorithm performs the segmentation task, it searches pixels with similar gray levels only within the neighborhood of the internal connection matrix; in the absence of constraints, some small spurious information will enter into the segmentation results. (6) On the contrary, the proposed method is established within the framework of level set; as a result, we can impose various constraints to filter out the information that does not belong to the target. This is an important safeguard mechanism for accurate segmentation. This is the reason why the output of this paper is the best.

In the subjective visual level of the human eye, our method does output the best results, in order to increase the objectivity of the comparison behavior. Next, we will use four region overlap metrics to compare the performances of the models quantitatively; their definitions are as follows:

(jaccard similarity [43])

JS = N([S.sub.reference] [intersection] [S.sub.test])/ N([S.sub.reference] [union] [S.sub.test]), (22)

(dice similarity coefficient [44])

DSC = 2N [S.sub.reference] [intersection] [S.sub.test] / N [S.sub.reference] + N [S.sub.test], (23)

FPR = N ([S.sub.reference] \ [S.sub.common]) / N [S.sub.reference] (false positive ratio), (24)

FNR = N ([S.sub.test] \ [S.sub.common]) / N ([S.sub.test]) (false negative ratio), (25)

where "[intersection]" and "[union]" represent the intersection and union of two regions, respectively; [S.sub.test], and [S.sub.common] are the output region of the segmentation algorithm, the ground truth, and the common part of two regions, respectively; and N(*) represents the number of pixels in the enclosed set. Obviously, the closer the JS and DSC values to 1, and the FPR and FNR values to 0, the better the segmentation results. Table 1 presents the evaluation results using the four region overlap metrics; it can be seen that the proposed MLBF model has the optimal performance indicators.

The speed of evolution convergence of the MLBF model can be reflected by the number of iterations required to obtain the final target contour and the CPU time required to complete the entire segmentation process. In this group of experiments, we compare it with three other level set models which are also aimed at inhomogeneous images: LBF [17], LGDF [18], and LIF [19]. The detailed performance indicators for this set of experiments are shown in Table 2; the initial curves and the input images here are the same as those shown in Figure 6.

It can be seen from the statistical data in Table 2 that the proposed model which combines the two strategies of AOS and SFM has the least number of iterations and the CPU time consumption under the premise of outputting the accurate segmentation results, and it can decrease the iterations from hundreds of times to dozens of times compared with the level set methods without these two strategies and the total CPU time from dozens of seconds to hundreds of milliseconds. Although the methods involved in the comparison have reached the final state of convergence, their segmentation results all have different degrees of error; Figure 7 shows the final evolution curves and corresponding two valued images.

In order to validate the KL-MLBF's robustness against initial contour position, we compare it with three classic inhomogeneous image segmentation models which have already appeared in the previous comparative experiments. Figure 8 shows the results of this set of comparison experiments, where the first row is the input images and the initial contours of the LSE process, and the second to the fifth rows are the segmentation results by using the LBF, LGDF, LIF, and our MLBF models, respectively; here, we use the usual form of the level set method to express the final segmentation results. By simply assigning the area inside the curve to white and the area outside the curve to black, we can get the binary image corresponding to the segmentation curve. From this set of comparison results, we found that the results of the three methods involved in the comparison are related to the position of the initial contour, while the model in this paper outputs the same correct result under different initialization forms.

In order to verify the proposed model's robustness against noise interference, we carried out the experiments on a set of images with different noise levels. By superimposing different levels of Gaussian noise into a clean original image, we can easily build the experimental data needed here. In the specific implementation, we adopt the imnoise function from Matlab, whose call syntax is "output = imnoise (input, 'gaussian', m, v)," and the precise control of the noise level can be achieved by changing the values of m and v. In this experiment, we use the data shown in Figure 9 as the original pure image; the segmentation results corresponding to different noise levels are shown in Figure 10, where the blue curve is the initial contour and the red curve is the final evolution result. From the final outputs, we can draw the following conclusion: the LBF model is greatly affected by noise and all outputs completely wrong segmentation results, while the proposed model has excellent noise adaptability.

6. Conclusion

In this paper, we propose a level set segmentation model called MLBF. By introducing multiscale ideas into the classical LBF model, the MLBF model in this paper achieves excellent segmentation performance in the segmentation experiments of inhomogeneous images. The introduction of the reactive diffusion energy term makes it possible to ensure the validity of the LSF without the need for reinitialization. In the numerical implementation, we have introduced three strategies to improve the overall performance of the proposed model: (1) the AOS strategy to break the time step limit, (2) the SFM strategy to achieve minimal pixel update range, and (3) the salient region localization strategy to achieve fully automatic initialization. Under the combined effect of the above strategies, the model of this paper has achieved excellent performance in the following aspects: contour location accuracy, speed of evolution convergence, robustness against initial contour position, and robustness against noise interference. In the future study, we intend to further strengthen our model, for example, the introduction of a probabilistic graphical model and other ideas, in order to further enhance its adaptability in natural image segmentation.

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare that there are no conflicts of interest to this work. The authors declare that they do not have any commercial or associative interest that represents a conflict of interest in connection with this work.


This work is supported by the Fundamental Research Funds for the Central Universities of China under Grant No. ZYGX2018J079. The authors would like to thank the anonymous reviewers for their valuable comments and advices.


[1] M. J. Islam, S. Basalamah, M. Ahmadi, and M. A. Sid-Ahmed, "Capsule image segmentation in pharmaceutical applications using edge-based techniques," in 2011 IEEE International Conference on Electro/Information Technology, pp. 1-5, Mankato, MN, USA, May 2011.

[2] D. Cheng, J. Huang, Z. Yu, X. Tang, and J. Yang, "Medical image enhancement based on fuzzy techniques," Journal of Harbin Institute of Technology, vol. 39, no. 3, pp. 435-437, 2007.

[3] D. Mumford and J. Shah, "Optimal approximations by piecewise smooth functions and associated variational problems," Communications on Pure and Applied Mathematics, vol. 42, no. 5, pp. 577-685, 1989.

[4] M. Yasmin, M. Sharif, and S. Mohsin, "Neural networks in medical imaging applications: a survey," World Applied Sciences Journal, vol. 22, no. 1, pp. 85-96, 2013.

[5] T. Leung and J. Malik, Contour Continuity in Region Based Image Segmentation, European Conference on Computer Vision, 1998.

[6] M. Kass, A. Witkin, and D. Terzopoulos, "Snakes: active contour models," International Journal of Computer Vision, vol. 1, no. 4, pp. 321-331, 1988.

[7] V. Caselles, R. Kimmel, and G. Sapiro, "Geodesic active contours," International Journal of Computer Vision, vol. 22, no. 1, pp. 61 -79, 1997.

[8] S. Osher and J. A. Sethian, "Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations," Journal of Computational Physics, vol. 79, no. 1, pp. 12-49, 1988.

[9] C. Xu and J. L. Prince, "Snakes, shapes, and gradient vector flow," IEEE Transactions on Image Processing, vol. 7, no. 3, pp. 359-369, 1998.

[10] Q. Li, T. Deng, and W. Xie, "Active contours driven by divergence of gradient vector flow," Signal Processing, vol. 120, pp. 185-199, 2016.

[11] S. Zhu and R. Gao, "A novel generalized gradient vector flow snake model using minimal surface and component-normalized method for medical image segmentation," Biomedical Signal Processing and Control, vol. 26, pp. 1-10, 2016.

[12] T. F. Chan and L. A. Vese, "Active contours without edges," IEEE Transactions on Image Processing, vol. 10, no. 2, pp. 266-277, 2001.

[13] R. Ronfard, "Region-based strategies for active contour models," International Journal of Computer Vision, vol. 13, no. 2, pp. 229-251, 1994.

[14] S. Niu, Q. Chen, L. de Sisternes, Z. Ji, Z. Zhou, and D. L. Rubin, "Robust noise region-based active contour model via local similarity factor for image segmentation," Pattern Recognition, vol. 61, pp. 104-119, 2017.

[15] L. Liu, D. Cheng, F. Tian, D. Shi, and R. Wu, "Active contour driven by multi-scale local binary fitting and Kullback-Leibler divergence for image segmentation," Multimedia Tools and Applications, vol. 76, no. 7, pp. 10149-10168, 2017.

[16] D. Wang, "Hybrid fitting energy-based fast level set model for image segmentation solving by algebraic multigrid and sparse field method," IET Image Processing, vol. 12, no. 4, pp. 539-545, 2018.

[17] C. Li, C.-Y. Kao, J. C. Gore, and Z. Ding, "Implicit active contours driven by local binary fitting energy," in 2007 IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA, June 2007.

[18] L. Wang, L. He, A. Mishra, and C. Li, "Active contours driven by local Gaussian distribution fitting energy," Signal Processing, vol. 89, no. 12, pp. 2435-2447, 2009.

[19] K. Zhang, H. Song, and L. Zhang, "Active contours driven by local image fitting energy," Pattern Recognition, vol. 43, no. 4, pp. 1199-1206, 2010.

[20] D. Wang, T. Zhang, W. Shi, Z. Wang, X. Yang, and L. Wei, "Moving objects segmentation based on piecewise constant Mumford-Shah model solving by additive operator splitting," Optical Engineering, vol. 49, no. 3, article 037004, 2010.

[21] R. T. Whitaker, "A level-set approach to 3D reconstruction from range data," International Journal of Computer Vision, vol. 29, no. 3, pp. 203-231, 1998.

[22] D. L. Chopp, "Computing minimal surfaces via level set curvature flow," Journal of Computational Physics, vol. 106, no. 1, pp. 77-91, 1993.

[23] D. Adalsteinsson and J. A. Sethian, "A fast level set method for propagating interfaces," Journal of Computational Physics, vol. 118, no. 2, pp. 269-277, 1995.

[24] J. A. Sethian, Level Set Methods and Fast Marching Methods, Cambridge University Press, Cambridge, 1999.

[25] D. Peng, B. Merriman, S. Osher, H. Zhao, and M. Kang, "A PDE-based fast local level set method," Journal of Computational Physics, vol. 155, no. 2, pp. 410-438, 1999.

[26] M. Sussman, P. Smereka, and S. Osher, "A level set approach for computing solutions to incompressible two-phase flow," Journal of Computational Physics, vol. 114, no. 1, pp. 146-159, 1994.

[27] M. Sussman and E. Fatemi, "An efficient, interface-preserving level set redistancing algorithm and its application to interfacial incompressible fluid flow," SIAM Journal on Scientific Computing, vol. 20, no. 4, pp. 1165-1191, 1999.

[28] G. Russo and P. Smereka, "A remark on computing distance functions," Journal of Computational Physics, vol. 163, no. 1, pp. 51-67, 2000.

[29] C. Li, C. Xu, C. Gui, and M. D. Fox, "Level set evolution without re-initialization: a new variational formulation," in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), vol. 1, pp. 430-436, San Diego, CA, USA, June 2005.

[30] X. Xie, "Active contouring based on gradient vector interaction and constrained level set diffusion," IEEE Transactions on Image Processing, vol. 19, no. 1, pp. 154-164, 2010.

[31] C. Li, C. Xu, C. Gui, and M. D. Fox, "Distance regularized level set evolution and its application to image segmentation," IEEE Transactions on Image Processing, vol. 19, no. 12, pp. 3243-3254, 2010.

[32] K. Zhang, L. Zhang, H. Song, and D. Zhang, "Reinitialization-free level set evolution via reaction diffusion," IEEE Transactions on Image Processing, vol. 22, no. 1, pp. 258-271, 2013.

[33] S. Osher and R. Fedkiw, Level Set Methods and Dynamic Implicit Surfaces, Springer, 2002.

[34] J. Weickert, B. M. t. H. Romeny, and M. A. Viergever, "Efficient and reliable schemes for nonlinear diffusion filtering," IEEE Transactions on Image Processing, vol. 7, no. 3, pp. 398-410, 1998.

[35] J. Harel, C. Koch, and P. Perona, "Graph-based visual saliency," in Advances in Neural Information Processing Systems 19:Proceedings of the 2006 Conference, vol. 19, pp. 545-552, Vancouver, Canada, 2007.

[36] R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, "Frequency-tuned salient region detection," in 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1597-1604, Miami, FL, USA, June 2009.

[37] X. Hou and L. Zhang, "Saliency detection: a spectral residual approach," in 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8, Minneapolis, MN, USA, June 2007.

[38] S. Goferman, L. Zelnik-Manor, and A. Tal, "Contextaware saliency detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 10, pp. 1915-1926, 2012.

[39] E. Erdem and A. Erdem, "Visual saliency estimation by nonlinearly integrating features using region covariances," Journal of Vision, vol. 13, no. 4, 2013.

[40] A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," Journal of the Royal Statistical Society, Series B, vol. 39, no. 1, pp. 1-38, 1977.

[41] N. Otsu, "A threshold selection method from gray-level histograms," IEEE Transactions on Systems, Man, and Cybernetics, vol. 9, no. 1, pp. 62-66, 1979.

[42] G. Kuntimad and H. S. Ranganath, "Perfect image segmentation using pulse coupled neural networks," IEEE Transactions on Neural Networks, vol. 10, no. 3, pp. 591-598, 1999.

[43] Q. Zheng, Z. Lu, W. Yang, M. Zhang, Q. Feng, and W. Chen, "A robust medical image segmentation method using KL distance and local neighborhood information," Computers in Biology and Medicine, vol. 43, no. 5, pp. 459-470, 2013.

[44] U. Vovk, F. Pernus, and B. Likar, "A review of methods for correction of intensity inhomogeneity in MRI," IEEE Transactions on Medical Imaging, vol. 26, no. 3, pp. 405-421, 2007.

Dengwei Wang

School of Aeronautics and Astronautics, University of Electronic Science and Technology of China, Chengdu 611731, China

Correspondence should be addressed to Dengwei Wang;

Received 28 February 2018; Revised 23 June 2018; Accepted 28 June 2018; Published 5 August 2018

Academic Editor: Giovanni Diraco

Caption: FIGURE 1: An example of LSE.

Caption: FIGURE 2: An example of inhomogeneous image segmentation based on the LBF model. (a) Sample image with slight inhomogeneity. (b) The histogram of gray level distribution of the local region corresponding to center point [P.sub.1]. (c) The histogram of gray level distribution of the local region corresponding to center point [P.sub.2]. (d) Correct segmentation results. (e) Another sample image with severe inhomogeneity. (f) The histogram of gray level distribution of the local region corresponding to center point [P.sub.3]. (g) The histogram of gray level distribution of the local region corresponding to center point [P.sub.4]. (h) Wrong segmentation result.

Caption: FIGURE 3: The results of different salient target detection algorithms.

Caption: FIGURE 4: Examples of the initial curve determination based on the salient target detection mechanism. The first column: saliency map. The middle column: the convergence result of the CV model. The third column: the visual effect after the contour curve is superimposed on the original image.

Caption: FIGURE 5: One example of the initialization in SFM.

Caption: FIGURE 6: A set of comparative experiments used to verify the contour location accuracy.

Caption: FIGURE 7: The error segmentation results corresponding to the convergence state.

Caption: FIGURE 8: A set of comparative experiments used for verifying the robustness of the model against the initial curve.

Caption: FIGURE 9: The pure input image of the noise suitability test.

Caption: FIGURE 10: A set of comparative experiments used for verifying the robustness against noise interference. The first and second rows are the segmentation results of the LBF and MLBF models under three noise levels, respectively.
TABLE 1: Region overlap metrics of different algorithms, where
"image number 1" to "image number 6" are the serial numbers of
the images shown in the first column of Figure 3 with the same

Input image      Algorithm
                               JS       DSC      FPR      FNR

                    CV       0.4705   0.6399    0.0450   0.5189
                    GAC      0.7111   0.8311    0.1192   0.2132
Image number 1      EM       0.5101   0.6756    0.1170   0.4529
                   OTSU      0.7141   0.8332    0.1952   0.1363
                   PCNN      0.7132   0.8326    0.2099   0.1200
                   MLBF      0.9302   0.9638    0.0381   0.0343

                    CV       0.4818   0.6503    0.0147   0.5147
                    GAC      0.3590   0.5283    0.0989   0.6263
Image number 2      EM       0.8096   0.8948    0.1417   0.0655
                   OTSU      0.0951   0.1736    0.0118   0.9048
                   PCNN      0.1063   0.1922    0.0217   0.8935
                   MLBF      0.8319   0.9082    0.0138   0.0539

                    CV       0.9035   0.9493    0.0453   0.0560
                    GAC      0.6388   0.7796    0.1233   0.2982
Image number 3      EM       0.9024   0.9487    0.0291   0.0724
                   OTSU      0.8426   0.9146    0.1300   0.0360
                   PCNN      0.8800   0.9362    0.0850   0.0416
                   MLBF      0.9301   0.9638    0.0127   0.0286

                    CV       0.7423   0.8521    0.0403   0.2338
                    GAC      0.5879   0.7405    0.0939   0.3740
Image number 4      EM       0.7288   0.8431    0.0697   0.2291
                   OTSU      0.7782   0.8753    0.1119   0.1371
                   PCNN      0.7788   0.8757    0.1370   0.1113
                   MLBF      0.8097   0.8948    0.0332   0.0385

                    CV       0.4534   0.6239    0.0444   0.5368
                    GAC      0.6863   0.8140    0.1038   0.2545
Image number 5      EM       0.3488   0.5172    0.6509   0.0021
                   OTSU      0.0750   0.1395    0.0107   0.9249
                   PCNN      0.0740   0.1378    0.0158   0.9259
                   MLBF      0.8330   0.9089    0.0103   0.0020

                    CV       0.8697   0.9303    0.0199   0.1147
                    GAC      0.7348   0.8471    0.0002   0.2651
Image number 6      EM       0.8707   0.9309    0.0249   0.1095
                   OTSU      0.8923   0.9431    0.0375   0.0756
                   PCNN      0.8729   0.9321    0.0411   0.0931
                   MLBF      0.9005   0.9477    0.0001   0.0133

TABLE 2: Comparison of the speed of evolution convergence.

Input image      Algorithm   Iterations   CPU time (s)

                    LBF         598          80.794
Image number 1      LIF         808          98.521
                   LGDF         536          91.095
                   MLBF          46          0.577

                    LBF         758          96.905
Image number 2      LIF         800          87.217
                   LGDF         653          94.755
                   MLBF          62          0.623

                    LBF         300          34.843
Image number 3      LIF         450          40.935
                   LGDF         500          72.443
                   MLBF          23          0.249

                    LBF         550          53.426
Image number 4      LIF         650          65.756
                   LGDF         250          29.558
                   MLBF          25          0.211

                    LBF         450          58.080
Image number 5      LIF         598          64.185
                   LGDF         200          32.690
                   MLBF          21          0.234

                    LBF         520          22.102
Image number 6      LIF         550          27.404
                   LGDF         260          18.687
                   MLBF          27          0.133
COPYRIGHT 2018 Hindawi Limited
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2018 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:Research Article
Author:Wang, Dengwei
Publication:Journal of Sensors
Date:Jan 1, 2018
Previous Article:Study of Systems Error Compensation Methods Based on Molecular-Electronic Transducers of Motion Parameters.
Next Article:OperaBLE: An IoT-Based Wearable to Improve Efficiency and Smart Worker Care Services in Industry 4.0.

Terms of use | Privacy policy | Copyright © 2020 Farlex, Inc. | Feedback | For webmasters