Pattern Recognition Algorithms for Scoliosis Detection

Bartolome, Jose Ronello T.

De La Salle University Taft Avenue Malate, Manila Telephone number, incl. country code

Bihis, Earl Cristopher G.

De La Salle University 411 Harmony Homes 2645 Sandejas St., Malate, Manila (+632)404-3814

Kampana, Kathereen Celeste J.

De La Salle University 2353 Taft Avenue Malate, Manila (+632)524-9749

onel@ccs.dlsu.edu.ph See, Wayne Steven D.

De La Salle university 711 Camba St., Binondo, Manila (+632)241-8117

earlcristopher@yahoo.com

ckstudmuffin@yahoo.com Serviano, Bettina L.

De La Salle University 2102 Burgundy Westbay Tower P. Ocampo St. Corner Taft Ave., Malate, Manila (+632)302-0660

weynsee@yahoo.com

bserviano@yahoo.com

ABSTRACT
Scoliosis is a painful spinal condition often found in young children that can have serious implications if it is not detected in its early stages. In order to offset the rising number of scoliosis cases, schools have begun screening schoolchildren for signs of this spinal deformity. Scoliosis is detected through manual inspections of x-ray images by trained physicians, but this method is too timeconsuming and impractical to be applied in schools with a large student population. A viable alternative would be an automated scoliosis detection system that would not require the aid of a physician. The proponents devised two methods for the automated detection of scoliosis from x-ray images. The first approach utilizes a combination of feature extraction algorithms like edge detection and c ontrast stretching to extract relevant information from the image needed in diagnosing scoliosis. The second approach utilizes statistical methods to create junction points that serve as basis for approximating the required lines. To test the efficiency of the algorithms, the proponents designed a system that generates scoliosis diagnosis reports from input x-ray images using the algorithms. Tests include determining the presence of scoliosis, and, if detected, the orientation and severity of the curvature. Results show that the feature extraction module has a 60.87% classification rate, while the line approximation has a 78.95% classification rate. For scoliosis detection, the system has an 86.96% accuracy rate using feature extraction, and 89.47% using line approximation.

1. INTRODUCTION
Scoliosis is spinal deformity wherein the spine exhibits a lateral or sideways curvature. If left untreated, the curve may become more pronounced and may require surgery to correct. As scoliosis progresses, the vertebrae (individual bones that comprise the backbone) rotate, causing the ribs to crowd together on one side of the chest and to spread apart on the opposite side. This complication can result in heart and lung problems over a period of years. Unlike poor posture, these curves cannot be corrected simply by learning to stand up straight. Scoliosis can be classified according to its severity, which is measured by determining the angle determined by the curvature. Cobb angle is the current gold standard for assessing scoliosis. It is measured by extending lines along the borders of vertebrae where the curve starts and ends, then measuring the angle formed by these two lines. Scoliosis can be categorized into three types according to its severity: mild (Cobb angle is above 10 degrees); moderate (Cobb angle is above 20 degrees); and severe scoliosis (Cobb angle is above 70 degrees). Scoliosis affects a small percentage of the population, approximately 2 percent 1. The most widespread form of this condition is idiopathic (80 to 85 percent), which means the cause is unknown. In other cases, scoliosis may develop as a result of degeneration of the spinal disks, as seen with osteoporosis, or as a hereditary condition that tends to run in families 2. It usually develops in middle or late childhood, before puberty, and occurs more often in girls than boys. The most common method for

Keywords
Scoliosis, medical diagnosis, x-ray images, feature extraction, edge detection, image analysis, pattern recognition, contrast stretching.
1 2

American Academy of Orthopaedic Surgeons, 2000 Methodist Health Care System, 2004

detecting scoliosis is through manually inspecting thoracic (frontal chest) x-rays. The most important part of the management of scoliosis is the early detection of its presence, as early management may prevent the need for surgery. Most curves can be treated non-operatively if they are detected before they become too severe. But because it is rarely painful, scoliosis often goes unnoticed in a child in its early stages. Schools in America have begun screening schoolchildren for signs of this condition (the S pine Society of Australia recommends that screening for scoliosis should be done on Year 7 and Year 9 schoolgirls). If a physician is to manually inspect hundreds of x -ray images in a short period of time, as some school-employed clinicians are required to do, fatigue can significantly impair his/her judgment. An automated scoliosis detection system can reduce the possibilities of human errors, and may not even require the presence of a physician for diagnosis. Surprisingly little research has been done in the field of computeraided medical diagnosis in the context of spinal deformities. In the current literature, only two papers have specifically dealt with the detection of scoliosis, [2] and [4]. [2] diagnosed spinal deformities using moir topographical images. [4] used computer aidedrasterstereography to detect surface shape deviation for its prognosis. It utilized a PC, a slide projector and a digital camera to acquire the needed images. This paper focuses on detecting scoliosis from a digital x -ray radiograph, and, if the presence of a curvature is detected, classifying it according to severity. We present a novel approach in isolating the spinal chord from the input image, then, using a variety of methods for image analysis, formulate lines that can be used for calculating the Cobb angle. The approaches used for analysis are categorized into two: the first approach uses a combination of feature extraction algorithms like edge detection and Hough Transform to detect vertebral edges; the second approach uses statistical techniques in estimating the location of the vertebral edges. The input images are x-ray radiographs borrowed from three different hospitals in Manila3, Philippines whose images were captured using a digital camera. The next section discusses the steps involved in the spine isolation algorithm. The following sections will then present the two approaches used for establishing the lines needed for calculating the Cobb angle. The calculation of the angle is presented in section 4. We also present some results from experiments conducted to test the accuracy of the algorithms. The final section presents the conclusions of the study, and makes some recommendations for future research.

2. SPINE ISOLATION
To isolate the spinal chord from the input image, we used a statistical-based approach to quantify the area where the spine resides. The information extracted from the input image is used to build its graphical representation. Because the spine can be easily identified as it presents a sharp contrast to the dark background (the rib cage area), pixel intensity is the value used for creating a binary histogram representation of the image. The mean of the image pixels is used as a threshold for filling the histogram. This histogram is then translated into a graph, where the area with the highest concentration of high pixel intensity values is established as the region of interest. To determine the bounds of the area, a simple application of maxima/minima searching is applied. Isolation performance can be improved by first subdividing the image into a set number of regions to localize the values of the histogram, and then contrasting it to heighten the disparity between the pixel values of the spine and its enclosing area. Each region is treated as an image, and the spine isolation algorithm described in the previous paragraph is applied. This method can tighten the extracted area, as well as tolerate higher levels of noise. For example, if the x-ray radiograph contains bright spots in areas outside the spinal area, dividing the images into partitions can limit the noise to a specific region. If the algorithm is applied to the whole image (without partitions), the noise can possibly lead to an incorrect extraction. Figure 1 shows the result using the spine isolation process.

Figure 1. Spine i solation

3. FEATURE EXTRACTION
The lines used for Cobb angle are extensions of the horizontal edges of the vertebrae. In order to detect them, the Hough line detection algorithm is used. However, the quality of each input image varies, and significant amounts of noise can lead to an incorrect detection. In order to retrieve the relevant lines, further image enhancements must be applied. The proposed technique is a three-stage process. The regions retrieved from the spine isolation phase are contrasted as this technique emphasizes the edges of the vertebrae. For the next

St. Lukes Medical Center, Ospital ng Maynila, Metropolitan Hospital

step, edge detection is applied on the images to reveal the prominent contours. The resulting edge map is then line detected using the Hough transform algorithm.

4. LINE APPROXIMATION
As x -ray images g noisier, the harder it is to extract relevant et information from the input. Oftentimes, the edges of the vertebrae are too blurred to be identified by line detection algorithms (or the human eye), making feature extraction impossible. We designed this technique to make estimations on the locations of the lines, based on the low-level data gathered from the image. This method is capable of handling larger amounts of noise, but since the lines are estimated, the measurements are not as opposed to using the actual Cobb lines. This technique uses the actual positioning of the partitions returned by the spine isolation stage in the context of the original image to create a set of connected lines. We used the term junction points to describe the origins of these lines. Junction points are essentially the midpoints of each of the regions. But regions must pass a filtering process before they are elected as junction points. To have its midpoint be elected as a junction point, the width of the region must lie inside the boundary [ |width(x)standard_deviation(all_regions)|, width(x)+standard_deviation(all_regions)], where x is the current region. The regions that passed the filter test will be interconnected through their midpoints (see Figure 7). The junction points will then extended perpendicularly, with respect to the line formed by the current junction point and the junction point of the previous region. As seen in Figure 8, the result should produce n-1 lines, where n is the number of regions that passed the filtering process.

Figure 2. Original image

Figure 3. Image result after applying contrast stretching

Figure 4. Image result after applying edge detection

Figure 6. Regions with Junction Points

Figure 5. Image result after applying Hough line detection

process should ensure that none of the erroneous lines is chosen. The difference between the two angles is the Cobb angle.

6. EXPERIMENTS
To test the performance of the algorithms, we created a prototype system that will accept as input a digital x -ray image, and then output a diagnosis report using the selected algorithms. The algorithms were tested on 42 x-ray images. The images were further subdivided into 2 groups, low-quality and high-quality. This was done to measure the correlation between system performance and image quality. See Figures 9 and 10 for sample images from the two data sets. Low-quality images are defined in this experiment as images where the vertebral edges of the spine cannot be easily distinguished and/or a significant amount of noise is present. Noise can range from any of the following: incorrect camera angle, poor lighting, presence of mold and partial image capture. The proponents have collected a total of 23 low-quality images. The proponents defined high-quality in this context as images where the vertebral edges of the spine can be easily distinguished. The proponents have collected a total of 19 high-quality images.

Figure 7. Connected Junction Points

Figure 8. Line Approximation

5. ANGLE FILTERING AND CALCULATION

The resulting lines from the previous phase will be analyzed and filtered in this stage. We used the angles formed by the input lines (with respect to the y-axis) as the basis for comparison. To filter out erroneous lines, the differences of adjacent angles will be compared against a threshold. If the difference is too large, the line with the angle closest to the average of all the line angles is retained. The threshold plays an important role here, as it must allow gradual transitions, yet also guard against large deviations between two angles. For this reason, the doubled value of the standard deviation of the angles is used as the threshold. In the Cobb method, the angle formed by the lines extending the borders of the upper and lower end vertebra determines the severity of the curve. To identify the needed vertebrae, doctors usually choose the vertebra where the curvature starts as the upper end vertebra and the vertebra where the curvature ends as the lower end vertebra. We achieved similar results by choosing the line with the largest angle as the upper end vertebra, and the line with the smallest angle as the lower end vertebra. The filtering Figure 9. Sample of Low-Quality Image

Figure 10. Sample of High-Quality Image

6.1 Spine Isolation

The algorithm for isolating the spine in a digital thoracic x -ray image differs depending on the color model used. For this reason, the spine isolation algorithm was tested on two color spaces: the RGB model and CIE LUV space. The accuracy of the region of interest detection will also depend on the number of partitions the image will be subjected to. In our set-up, the images can be subdivided to either four or eight regions. The accuracy of the spine isolation algorithm w measured by manually inspecting as whether the returned regions contain vertebral portions of the spine. According to the results (see Table 1 and 2), using RGB and LUV color models produced the same results on low-quality images. Using 4 as the number of divisions gave a better extraction rates (a 10% jump in accuracy for scoliotic images). For high-quality images, there is a slight increase with the use of LUV color space (see Table 3 and 4). As expected, using 4 regions again produced better results. This is because there is a larger room for error in using less partitions, though later experiments prove that using smaller partitions is more beneficial for the algorithms. Table 1. Accuracy of Spine Isolation Algorithm using RGB (Low-Quality Images)

6.2 Feature Extraction

The feature extraction algorithms, namely contrast stretching and edge detection, was tested as a unit to be able to accurately measure their efficiency in detecting relevant features from the images. The feature extraction algorithms underwent several tests using the images produced by the spine isolation algorithm to compute the overall accuracy of this approach. The image was subjected to one of three kinds of contrast stretching algorithm (this test will also consider the bypassing of the contrast stretching altogether) before undergoing one of several edge detection methods. Three contrast methods were used in the setup: percentile contrast, normal contrast and cutoff contrast. The difference between the three is how each limits the range of pixel values to span a desired range. In contrast stretching, a lower and higher pixel value limit is specified. These limits will be the minimum and maximum pixel values that the image type concerned allows. In normal contrast, the l west and highest pixel values o currently present in the image is scanned to produce a scaling function for each point. In percentile stretching, the percentile rank in the histogram is used for the high and low value. For cutoff contrast, a minimum fraction of the histogram peak magnitude is used to limit the values. The accuracy of the contrast and edge detection algorithms was measured by using the Hough line detection algorithm to verify whether the contours of the vertebra can be correctly identified. Verification was done through visual confirmation. The test images were a random collection of 30 regions culled from the test samples. Based on the tests for low-quality images, the Canny edge detector produces the best results (see Table 5). When used in conjunction with percentile contrast stretching, the successful line detection rate is 56.67%; when used with normal stretching, the detection rate is 70%; when used with cutoff stretching the detection rate is 53.33%; and if no contrast is applied, the successful detection rate is 56.67%. The top three edge detectors are Canny, Kirsch and Prewitt. The edge detector that produced the worst results is Robert. According to the tests, the best successful detection rate for highquality images belongs to the normal contrast-Canny edge detection combination, closely followed by the percentile contrast-Canny edge detection and the no contrast-Canny edge detection pairs.

Normal Scoliotic Overall

4 Regions 95.45% 89.58% 92.39%

8 Regions 95.45% 79.17% 86.96%

Table 2. Accuracy of Spine Isolation Algorithm using LUV (Low-Quality Images)

Normal Scoliotic Overall

4 Regions 95.45% 89.58% 92.39%

8 Regions 95.45% 79.17% 86.96%

Table 3. Accuracy of Spine Isolation Algorithm using RGB (High-Quality Images)

Normal Scoliotic Overall

4 Regions 97.92% 100.00% 98.68%

8 Regions 95.83% 100.00% 97.37%

Table 4. Accuracy of Spine Isolation Algorithm using LUV (High-Quality Images)

Normal Scoliotic Overall

4 Regions 97.92% 100.00% 98.68%

8 Regions 97.92% 100.00% 98.68%

Table 5. Accuracy of Contrast and Edge Detection Algorithms (Low-Quality Images)

Canny Kirsch Prewitt Robinson Sobel Shen Robert Percentile 56.67% 30.00% 40.00% 36.67% 46.67% 53.33% 30.00% Normal 70.00% 43.33% 40.00% 33.33% 33.33% 26.67% 26.67% Cutoff 53.33% 30.00% 26.67% 33.33% 26.67% 50.00% 26.67% None 56.67% 30.00% 33.33% 30.00% 30.00% 30.00% 16.67%

the results were compared against the actual diagnosis of a certified physician. The percentage accuracy is taken from the ratio of the number of the correct results against the total number of images in the data set. The combination used for the feature extraction module was Canny edge detection on uncontrasted images, based on its performance after extensive testing. While this might appear to be in conflict with the previous unit testing results (see Table 5 and 6), it was discovered that its performance lowered when faced with regions of differing quality. As the input images (both high and low quality) are composed of regions that are of high or low quality, the algorithms performance suffered because of the lack of a robust parameter thresholding methods. This is especially important because contrast and edge detection methods perform better when their parameters are adjusted to the input image type. In Table 7, the results show that the line approximation approach achieved the best results in scoliosis detection, garnering a 89.47% identification rate. The performance is especially high in identifying the curve orientation of the scoliotic spine for both approaches, but line approximation was still the best, as it achieved a 100% identification rate (see Table 8). In estimating the severity of the curvature, line approximation had a 78.95% accuracy rate, compared to 63.16% for the feature extraction module (see Table 9). For overall results, line approximation (LA) achieved a 78.95% success rate, and feature extraction (FE) accumulated a 63.16% classification rate (see Table 10). Table 7. Comparison of test results in scoliosis identification

Based on the tests (see Table 6), the Canny edge detector once again produced the best results. When used in conjunction with percentile contrast stretching, the successful line detection rate is 93.33%; when used with normal stretching, the detection rate is 86.67%; when used with cutoff stretching the detection rate is 73.33%; and if no contrast is applied, the successful detection rate is 80%. The top three edge detectors are Canny, Kirsch and Prewitt. The edge detector that produces the worst results is Shen. For high-quality images, the contrast methods provide a significant boost in line extraction accuracy, as it can effectively remove certain kinds of noise and simultaneously highlight the relevant edges. According to the tests, the best successful detection rate for highquality images belongs to the percentile contrast-Canny edge detection combination, closely followed by the normal contrastCanny edge detection combination. Table 6. Accuracy of Contrast and Edge Detection Algorithms (High-Quality Images)
Canny Kirsch Prewitt Robinson Sobel Shen Robert Percentile 93.33% 80.00% 80.00% 73.33% 76.67% 70.00% 76.67% Normal 86.67% 86.67% 80.00% 80.00% 80.00% 60.00% 83.33% Cutoff 73.33% 73.33% 76.67% 76.67% 73.33% 63.33% 66.67% None 80.00% 83.33% 76.67% 73.33% 76.67% 60.00% 76.67%

FE LA

Low-Quality Image High-Quality Image 78.26% 84.21% 86.96% 89.47%

Table 8. Comparison of test results in curve orientation identification

FE LA

Low-Quality Image 91.66% 100.00%

High-Quality Image 100.00% 100.00%

6.3 Overall Tests

The two approaches were tested on four categories. In the first category, objective of the test was to determine the accuracy with which a particular method classifies a given image as scoliotic or non-scoliotic. For the second category, objective of the test was to determine the accuracy with which a particular method classifies the curve orientation of a scoliotic spine as either left or right. The next category tests the accuracy with which the algorithms estimate the Cobb angle. For leeway, the estimations that fell within the +7/-7 range of the correct measurement were considered correct. The final category tests the accuracy with which the algorithms matched the actual diagnosis reports of physicians. All

Table 9. Comparison of test results in estimating scoliosis severity

FE LA

Low-Quality Image 47.83% 63.22%

High-Quality Image 63.16% 78.95%

Table 10. Comparison of test results in overall classification rate

FE LA

Low-Quality Image 60.87% 60.87%

High-Quality Image 63.16% 78.95%

7. CONCLUSION
Automated medical diagnosis systems are a fairly popular research area that can potentially yield far-reaching benefits in the medical world. However, relatively few works has been done in the field of x-ray analysis, owing to the fact that the quality of x-ray images varies significantly depending on the x-ray imaging device. In the current literature, no attempts have been made in automating scoliosis detection in x-ray radiographs, despite the fact that trained physicians can easily diagnose spinal deformities through manual inspections. In this paper, two techniques were proposed for analyzing digital x-ray images of the frontal chest area. The first approach uses feature extraction methods like edge detection and contrast stretching to detect the edges of the vertebra to form relevant lines. The second approach uses the midpoints of the different regions of the spinal area to approximate the position of the needed lines. The extracted lines from these techniques are then filtered and analyzed to calculate the Cobb angle of the curvature. According to experiments, the feature extraction strategy achieves 78.26% scoliosis detection rate for low-quality images, and 84.21% for high-quality images. For the line approximation technique, detection rates of 86.96% and 89.47% are achieved for low-quality and high-quality images, respectively. In order to achieve higher classification rates, better parameter thresholding techniques need to be applied for the image operators. We suggest neural networks as a viable method for generating the parameters needed based on the available image information. As evidenced in the tests, the performance of both modules increases if the input images are of good quality. This is especially true for the feature extraction module, which relies on image information to extract the needed edges necessary for measuring

and predicting scoliosis. Low-quality images are usually the result of human-level noise, which is why the proponents suggest a new method for acquiring x-ray images that will be uniform in quality. The proponents believe that an automated x -ray scanning and archiving device will greatly improve the efficiency of the algorithms. Based on the experiments, the results were promising. Although this field still requires further research, the experiments prove that using the listed approaches should merit a closer look. This study may help in detecting other disorders that affect the shape, alignment or function of the spine.

8. ACKNOWLEDGMENTS
Our thanks to the following hospitals: Metropolitan, Ospital ng Maynila, and St. Lukes for allowing us to photograph their x-ray plates.

9. REFERENCES
[1] American Academy of Orthopaedic Surgeons. (2000). Scoliosis. http://orthoinfo.aaos.org/brochure/thr_report.cfm. [2] Kim, H. S., Ishikawa, S., Ohtsuka, Y., Shimizu, H., Shinomiya, T., and Viergever, M.A. (2001). Automatic scoliosis detection based on local centroids evaluation on moire topographic Images of human backs. http://ieeexplore.ieee.org/iel5/42/21028/00974926.pdf. [3] Methodist Health Care System. (2004). Scoliosis. http://www.methodisthealth.com/Ortho/scolio.htm. [4] Zubairi, J. (2001). Applications of Computer Aided Rasterstereography in Spinal Deformity Detection. http://www.cs.fredonia.edu/~zubairi/imavis_402.pdf.