Computer-Aided Cobb Measurement Based on Automatic Detection of Vertebral Slopes Using Deep Neural Network.
Scoliosis is a spinal deformity in which the spine shows a lateral curvature in the frontal plane . For scoliosis assessment, X-ray is the most economical imaging modality till date. The Cobb angle method  is the gold standard to assess curve severity in posteroanterior (PA) radiographs. To measure the Cobb angle, the most tilted vertebrae (end-vertebrae) at the top and bottom of the spinal curve are identified. As shown in Figure 1, the Cobb angle can be measured manually by determining the angle between the lines, respectively, drawn along the upper endplate of the superior end-vertebra and the lower endplate of the inferior end-vertebra. A patient's physiologic maturity, curve severity, and chances of progression are taken into consideration before making any treatment decisions, such as observation, bracing, and surgery. In general, an increase of 5[degrees] or more between two successive radiographs is considered indicative of curve progression. Therefore, the measuring procedure must be accurate and reproducible for correct diagnosis. Manual Cobb angle measurement requires experience and judgment. There could be errors in Cobb angle measurement if the surgeon selects different end-vertebrae and estimates different vertebrae slopes. Even when the same end-vertebrae are selected, Cobb angle measurements may vary by up to 5[degrees] for the same observer and by 7[degrees] for different observers [3, 4].
In recent studies, scientists have developed computer-aided methods to improve the repeatability of Cobb angle measurements. Zhang et al.  developed a computerized method based on the technique of Hough transform. This method detects vertebral endplates, which are then used to automatically calculate the Cobb angle. In this method, false detection might occur if a vertebra tilts by more than 45[degrees]. To facilitate the identification of end-vertebrae and to eliminate observer variations, Anitha and Prabhu  proposed listing the horizontal inclinations of all vertebrae in terms of slopes by using active contour models and morphological operators. Okashi et al.  developed an automatic method for spine segmentation and curvature quantification from poor-quality X-ray images of mice. They also investigated the applicability of their method to the X-ray image of a human spine, and they believed that their method could be a good solution for low-quality human X-ray images. Anitha et al.  proposed an automated system that extracted vertebral endplates using a customized filter that combined anisotropic, sigmoid, and differential filters. Samuvel et al.  proposed a mask-based segmentation algorithm for automatic Cobb angle measurement. The aforementioned algorithm and the manual method were used for measuring the Cobb angle in six cases. There was a difference of about 4[degrees] between the results obtained by the algorithmic method and those obtained by the manual method. Kusuma et al.  estimated vertebral locations by performing the template matching technique on the subdivisions of radiographs; the technique was based on the sum of squared differences. Then, they estimated the spinal curvature by polynomial curve fitting. In their method, the final measurement was affected by the polynomial order and the number of subdivisions. They achieved an average accuracy of 86.01%. Wibowo and Sardjono  used a modified top-hat filtering method for preprocessing and the gradient vector flow snake for the automatic determination of spinal curvature. Toan et al.  performed Cobb angle measurement by a semiautomatic method based on a deformable model of fuzzy spatial relations. Langensiepen et al.  presented a systematic review of the various techniques used to carry out Cobb angle measurement.
In this study, we proposed a computer-aided Cobb angle measurement method that automatically determined the slopes of vertebrae by using a deep neural network (DNN). DNNs with good generalization capability show promise in the field of machine learning. After being trained using vertebral patches, the DNN could be used to predict vertebral slopes in real time.
2. Materials and Methods
2.1. Radiographic Data. In this study, we used a spine model to generate the training set. This model included 12 thoracic vertebrae and five lumbar vertebrae. This model was made in a high-resolution imaging medium (Sawbones, Vashon, WA, USA) that could be used to produce a realistic image in radiographic environments as real bones. A total of 275 PA radiographs of the spine model in different poses were collected with vertebral slopes in the range of 5[degrees]-50[degrees] (mean: 15[degrees]). Lead markers that could be detected in radiographs were attached to the pedicles of each vertebra in the model. The detected markers were used to calculate the slope of a vertebra; this value was considered as the ground truth slope of that vertebra. In addition, we used the radiographs of 65 patients (51 girls, 14 boys; age: 12.5 [+ or -] 3.6 years) with idiopathic scoliosis. The selection criteria were as follows: (1) diagnosis of idiopathic scoliosis; (2) age between 9 and 18 years; (3) no prior spine surgery; and (4) visibility of pelvis, 12 thoracic vertebrae (numbered T1-T12), and 5 lumbar vertebrae (numbered L1-L5) in the radiographs. In this study, we excluded patients who had other musculoskeletal or neurological disorders. Informed consent was obtained from all patients/parents. This study received ethical approval from the local ethics board.
An image preprocessing protocol was applied to all the radiographs. In this protocol, the spinal radiograph was cropped from T1 to L5 and the images were resized to a standard height of 1000 pixels. Out of the 275 radiographs of the spine model, we randomly selected 235 radiographs to generate training patches for DNN training. Each selected radiograph showed 12 thoracic vertebrae and five lumbar vertebrae. For each vertebra in the radiograph, we randomly extracted 100 patches of 150 x 150 pixels. These patches were downsampled thrice to obtain patches of 50 x 50 pixels; these were represented by vectors of 2500 gray-level values. We conducted DNN training using 399500 patches (235 radiographs, each containing 17 vertebrae; 100 patches were extracted for each vertebra). The performance of the DNN system was tested on 105 radiographs, which included the 65 radiographs of patients and the remaining 40 radiographs of the spine model.
2.2. DNN for Estimation of Vertebral Slope. A DNN was trained to predict the slope of a vertebra as a function of a patch (P). As shown in Figure 2, the network architecture comprised three hidden layers of neurons: h1, h2, and h3. The first hidden layer h1 had 100 neurons whose inputs were a patch of 50 x 50 pixels (i.e., 2500 input units). The second hidden layer h2 had 500 neurons, while the third hidden layer h3 had 50 neurons. In the output layer and the first hidden layer (h1), the neurons had linear activation functions. In the other hidden layers, neurons had hyperbolic tangent functions. The corresponding function of the DNN was a vector mapping A(P): [R.sup.2500] [right arrow] [R.sup.1], where
A(P) = [b.sub.4] + [W.sub.4] tanh ([b.sub.3] + [W.sub.3] tanh ([b.sub.2] + [W.sub.2] (P - [b.sub.1])))). (1)
Here [W.sub.1], ..., [W.sub.4] were the weight matrices of each layer, and [b.sub.1], ..., [b.sub.4] were the bias vectors. The first hidden layer acted as a low-pass filter for multidimensionality reduction based on principal components analysis (PCA). Therefore, [W.sub.1] comprised the first 100 eigenvectors of the patches covariance matrix, representing the PCA basis, and [b.sub.1] was the mean of data patches. The second hidden layer h2 received the latent variable of the PCA as an input. To properly initialize the weights of the network, we pretrained the DNN using autoencoders with a learning rate of 0.1 for each layer. The DNN was then fine-tuned using error backpropagation algorithm. To train the DNN, we used the Python deep learning library Keras with the Theano backend. After the training procedure, the DNN rapidly predicted the slope of any vertebral patch because the DNN performed only simple matrix operations for computing the slope.
2.3. Cobb Angle Measurement Based on Estimated Vertebral Slopes. In the proposed method, the DNN was used to estimate the slopes of vertebrae, and the Cobb angle was calculated from these slope values. The proposed method was not full automatic as the user had to assign vertebral patches. In the PA radiograph, for a spinal curve to be measured, the user selected several vertebrae that contained the upper and lower end-vertebrae by clicking them with the mouse. Then, a rectangle of 150 x 150 pixels was created for each selected vertebra, as shown in Figure 3, and sent to the DNN to obtain the slope of that vertebra. The two vertebrae with the maximum absolute slopes tilting in opposite directions to the horizontal were detected automatically. The Cobb angle was then calculated automatically as the sum of the absolute values of the two slope angles.
2.4. Performance Analysis. The proposed method was tested and compared with the manual measurement method to evaluate its performance. An orthopedic surgeon (S) specialized in scoliosis with 25 years of experience manually measured the Cobb angle in 105 testing radiographs. The results obtained from the surgeon were considered as true values. Then, two examiners measured the Cobb angle in each radiograph using the software developed in this study. Both of them measured the Cobb angle twice over a period of three weeks. Examiner 1 (E1) had worked in a scoliosis clinic for 21 years. Examiner 2 (E2) was the software developer with no clinical experience of measuring the Cobb angle manually. The two examiners were asked to manually measure the Cobb angle in each radiograph twice over a period of one week. The examiners' manual measurements were performed 6 months after their computer-aided measurements. All measurements on the same radiograph were performed on the same curve, although some spines had multiple curves. Statistical analyses were performed using SPSS 16.0 (SPSS Inc., Chicago, IL, USA) software. The intraclass correlation coefficient (ICC, value between 0 and 1) with 95% confidence interval (CI) was used to evaluate the reliability . For error analysis, the mean absolute difference (MAD) of two measurements was also calculated.
3. Results and Discussion
The intraobserver reliability of the proposed method was assessed for each examiner and was compared with that of the manual measurement. Table 1 presents the results of intraobserver analyses. For the measurements obtained by the proposed method, all the intraobserver analyses produced ICC > 0.9 with 95% CI between 0.811 and 0.991 and MAD < 5[degrees]. The results of model radiographs showed that the proposed method had high intraobserver reliability (ICC > 0.98, MAD < 3[degrees]). Compared to the model radiographs, the proposed method showed lower intraobserver reliability for the results of in vivo radiographs. This was mainly because all the training patches were sampled from model radiographs.
Table 2 presents the results of interobserver reliability analyses. By using the proposed method, the interobserver reliability for the model radiographs was high; ICC [greater than or equal to] 0.98 with 95% CI between 0.962 and 0.990, and MAD [less than or equal to] 3[degrees]. These results indicate that the automatic method was consistent regardless of the examiners' experience. However, for the in vivo radiographs, the interobserver analyses produced ICC < 0.9 with 95% CI between 0.689 and 0.973. In the first trial, the MAD was more than 5[degrees]. Therefore, in vivo radiographs should be added in the training process to improve the system performance.
For the model radiographs, the intraobserver and interobserver reliabilities of the proposed method were higher than those of the manual method. For the in vivo radiographs, by using the proposed method the intraobserver reliability of E1 was close to that of E2. For the in vivo radiographs examined by E2, the intraobserver reliability of the proposed method was close to that of manual measurement, while for the in vivo radiographs examined by E1 the intraobserver reliability of the proposed method was lower than that of manual measurement. These results indicate that the reliability of manual measurements depends on the relevant experience of the examiner. The proposed method can outperform manual measurements if the DNN is trained adequately.
In previous studies, the reliabilities of computer-aided methods [5, 6, 8] were less than 3[degrees] for the Cobb angle measurements. For the model radiographs, the results of the proposed method are comparable to those reported in previous studies (i.e., [less than or equal to] 3[degrees]). However, for the in vivo radiographs, the reliability of the DNN system was less than the reliability of methods described in previous studies. Therefore, the training patches should be sampled out from both model and in vivo radiographs to train the DNN adequately.
To evaluate the validity of the proposed method, automatic measurements performed by two examiners were compared with the manual measurement performed by the surgeon. Table 3 shows the results. For the model radiographs, both examiners produced ICC > 0.9 with 95% CI between 0.815 and 0.962. The MAD between the automatic and manual measurements was less than 5[degrees]. These results indicate that the automatic method showed good agreement with the manual measurement method. Even an examiner with little experience could obtain similar results with the automatic method. However, for the in vivo radiographs, all the MAD values were greater than 5[degrees]. Therefore, the DNN system should be trained with adequate number of patches obtained from in vivo radiographs. In this study, the DNN system was only trained with patches obtained from model radiographs.
To explore the effect of network architecture, we trained three networks containing three different numbers of neurons in the second and third hidden layers (h2 and h3): h2 had 500 neurons and h3 had 50 neurons; h2 had 800 neurons and h3 had 100 neurons; and h2 had 500 neurons and h3 had 250 neurons. We collected 680 vertebral patches from 40 model radiographs (i.e., 40 x 17 = 680) with known ground truth slopes to test the three networks. The average errors of the vertebral slope tested with the three networks were 2.0[degrees], 1.9[degrees], and 2.0[degrees], respectively. Therefore, in the proposed system we chose the simplest network with h2 and h3 having 500 and 50 neurons, respectively.
In clinical practice, one of the major sources of Cobb measurement errors is manually drawing lines across the endplates of the end-vertebrae. In this study, we developed a computer-aided system to reduce the human judgment errors in Cobb angle measurements. In the developed system, the user was required to only select the patches. This was the only source of variability in the developed system. The computer-aided system automatically calculated the Cobb angle from the patches assigned by the user. For the model radiographs, the intraobserver and interobserver analyses showed excellent reliability of the proposed method (ICC [greater than or equal to] 0.98) with MAD [less than or equal to] 3[degrees]; this was lower than the 5[degrees] threshold of changes that could influence treatment decisions. Because the DNN was trained with only model radiographs, the system performance was not reliable for in vivo radiographs. The system performance could be improved further by training the DNN with sufficient number of in vivo radiographs.
Cobb angle measurement errors usually occur when lines are drawn manually across the endplates of end-vertebrae. In the proposed method, the slopes of vertebrae were automatically predicted using a DNN that was trained using model radiographs. The intraobserver and interobserver analyses indicated that the measurement variation was lower than the threshold of changes that influence treatment decisions. Although the user is required to assign vertebral patches in the proposed method, the user needs fewer skills to perform this task. By using the trained DNN, the measurement could be completed in real time after the user assignment of vertebral patches. Although we need to collect more in vivo data to improve the performance of the proposed system, the system has the potential to reduce the variation in scoliosis assessment. Further studies must be conducted to determine whether the proposed method can be safely used in clinical assessment of scoliosis.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
The research work was supported by the grants from the Natural Science Foundation of China under Grants 61002049 and 61361010 and from the Key Program of the Yunnan Natural Science Foundation under Grant 2013FA008.
 I. A. F. Stokes, "Three-dimensional terminology of spinal deformity. A report presented to the Scoliosis Research Society by the Scoliosis Research Society Working Group on 3-D terminology of spinal deformity," Spine, vol. 19, no. 2, pp. 236-248, 1994.
 J. R. Cobb, "Outline for the study of scoliosis," The American Academy of Orthopaedics Surgeons, Instructional Course Lectures, vol. 5, pp. 261-275, 1948.
 R. T. Morrissy, G. S. Goldsmith, E. C. Hall, D. Kehl, and G. H. Cowie, "Measurement of the Cobb angle on radiographs of patients who have scoliosis. Evaluation of intrinsic error," Journal of Bone and Joint Surgery American Volume, vol. 72, no. 3, pp. 320-327, 1990.
 D. L. Carman, R. H. Browne, and J. G. Birch, "Measurement of scoliosis and kyphosis radiographs. Intraobserver and interobserver variation," Journal of Bone and Joint Surgery American Volume, vol. 72, no. 3, pp. 328-333, 1990.
 J. Zhang, E. Lou, X. Shi et al., "A Computer-aided Cobb angle measurement method and its reliability," Journal of Spinal Disorders and Techniques, vol. 23, no. 6, pp. 383-387, 2010.
 H. Anitha and G. K. Prabhu, "Automatic quantification of spinal curvature in scoliotic radiograph using image processing," Journal of Medical Systems, vol. 36, no. 3, pp. 1943-1951, 2012.
 O. A. Okashi, H. Du, and H. Al-Assam, "Automatic spine curvature estimation from X-ray images of a mouse model," Computer Methods and Programs in Biomedicine, vol. 140, pp. 175-184, 2017.
 H. Anitha, A. K. Karunakar, and K. V. N. Dinesh, "Automatic extraction of vertebral endplates from scoliotic radiographs using customized filter," Biomedical Engineering Letters, vol. 4, no. 2, pp. 158-165, 2014.
 B. Samuvel, V Thomas, M. G. Mini, and J. Renjith Kumar, "A mask based segmentation algorithm for automatic measurement of Cobb angle from scoliosis x-ray image," in Proceedings of the 2012 International Conference on Advances in Computing and Communications (ICACC '12), pp. 110-113, IEEE, Cochin, Kerala, India, August 2012.
 B. A. Kusuma, H. A. Nugroho, and S. Wibirama, "Spinal curvature determination from scoliosis X-Ray image using sum of squared difference template matching," in Proceedings of the 2nd International Conference on Science and Technology-Computer (ICST '16), IEEE, Yogyakarta, Indonesia, October 2016.
 M. C. Wibowo and T. A. Sardjono, "Spinal curvature determination from x-ray image using GVF snake," in Proceedings of the International Conference on Information and Communication Technology and Systems (ICTS '15), pp. 35-40, IEEE, Surabaya, Indonesia, Sept. 2015.
 N. B. Toan, T. Q. D. Khoa, and V. Van Toi, "Computer-aid cobb measurement of scoliosis using deformable model with fuzzy spatial relations," in Proceedings of the 4th International Conference on the Development of Biomedical Engineering in Vietnam, IFMBE Proceedings, pp. 350-353, Springer, Berlin, Heidelberg, Germany, 2013.
 S. Langensiepen, O. Semler, R. Sobottke et al., "Measuring procedures to determine the Cobb angle in idiopathic scoliosis: A systematic review," European Spine Journal, vol. 22, no. 11, pp. 2360-2371, 2013.
 P. E. Shrout and J. L. Fleiss, "Intraclass correlations: uses in assessing rater reliability," Psychological Bulletin, vol. 86, no. 2, pp. 420-428, 1979.
Junhua Zhang, (1) Hongjian Li, (2) Liang Lv, (2) and Yufeng Zhang (1)
(1) Department of Electronic Engineering, Yunnan University, Kunming 650091, China
(2) Department of Orthopedics, The First People's Hospital of Yunnan Province, Kunming 650032, China
Correspondence should be addressed to Junhua Zhang; firstname.lastname@example.org
Received 9 February 2017; Revised 23 July 2017; Accepted 30 August 2017; Published 3 October 2017
Academic Editor: Jie Tian
Caption: Figure 1: Cobb angle measurement.
Caption: Figure 2: Architecture of the proposed network.
Caption: Figure 3: Vertebral patches assigned by a user.
Table 1: Intraobserver analyses. ICC (95% CI) From model From patients Proposed measurement E1 0.986 (0.976, 0.991) 0.901 (0.812, 0.962) E2 0.985 (0.974, 0.990) 0.908 (0.811, 0.970) Manual measurement E1 0.964 (0.902, 0.985) 0.953 (0.893, 0.981) E2 0.937 (0.851, 0.979) 0.908 (0.809, 0.970) Mean absolute difference From model From patients Proposed measurement E1 2.6[degrees] 4.6[degrees] E2 2.6[degrees] 4.4[degrees] Manual measurement E1 3.6[degrees] 3.7[degrees] E2 4.1[degrees] 4.5[degrees] Table 2: Interobserver analyses. ICC (95% CI) From model From patients Proposed measurement 1st 0.980 (0.962, 0.988) 0.874 (0.689, 0.960) 2nd 0.981 (0.964, 0.990) 0.889 (0.736, 0.973) Manual measurement 1st 0.870 (0.694, 0.955) 0.862 (0.659, 0.945) 2nd 0.892 (0.751, 0.983) 0.878 (0.697, 0.967) Mean absolute difference From model From patients Proposed measurement 1st 3.0[degrees] 5.1[degrees] 2nd 2.9[degrees] 4.9[degrees] Manual measurement 1st 5.1[degrees] 5.3[degrees] 2nd 4.8[degrees] 5.0[degrees] Table 3: Comparison between automatic and manual measurements. ICC (95% CI) From model From patients E1 versus S 0.912 (0.815, 0.962) 0.835 (0.701, 0.914) E2 versus S 0.915 (0.901, 0.927) 0.771 (0.602, 0.874) Mean absolute difference From model From patients E1 versus S 4.4[degrees] 5.4[degrees] E2 versus S 4.9[degrees] 6.6[degrees]
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||Research Article|
|Author:||Zhang, Junhua; Li, Hongjian; Lv, Liang; Zhang, Yufeng|
|Publication:||International Journal of Biomedical Imaging|
|Date:||Jan 1, 2017|
|Previous Article:||Space and Time Resolved Detection of Platelet Activation and von Willebrand Factor Conformational Changes in Deep Suspensions.|
|Next Article:||An Automatic Gastrointestinal Polyp Detection System in Video Endoscopy Using Fusion of Color Wavelet and Convolutional Neural Network Features.|