IEEE TRANSACTIONS ON COMPUTERS, VOL. c-21, NO.

7, JULY 1972

667

Texture

Measures ror
0f

Automatic Classification Pulmonary Disease

AND

RICHARD N. SUTTON

ERNEST L. HALL,

MEMBER, IEEE

Abstract-The complete pattern recognition problem is considered for the practical solution to a current significant medical question. Automated screening of chest radiographs for the detection of textural type abnormalities is approached from the view of: 1) preprocessing for standardization and data reduction; 2) feature extraction of characteristic measures (feature selection by optimization of classification accuracy); and 3) overall classification using training and test sets of selected chest radiographs. The disease processes known collectively as interstitial pulmonary fibrosis have been used as a prototype textural discrimination problem as part of a long-term project of automatic screening of chest radiographs. Measures based upon the statistical properties of the spatial distribution of radiograph gray-level sample points serve to differentiate normal from abnornal in a selected group of test cases. Examples of measures that differentiate synthetic patterns are presented. In the absence of a guiding theory, these measures are constructed by intuition and an a priori appreciation of the underlying geometry of the texture. Three examples of texture measurement for automatic computer diagnosis of pulmonary disease are considered. A single texture measurement that produces 92 percent correct classification on a training sample and 84 percent correct classification for the testing of 24 X-rays is described. These examples illustrate the feasibility of automatic computer recognition in radiographs of subtle texture changes produced by pulmonary disease.
Index Terms-Feature extraction, image processing, pattern recognition, radiographic image processing.

Fig. 1. Image classification system block diagram. Preprocessing: lung location, background correction, normalization, etc. Feature extraction: texture measurements, size measurements, etc. Classification: linear discriminant analysis using standard program such as BMD and new techniques. Adaptive supervisor: train on large number of proven cases and test before implementation.

INTRODUCTION

pattern recognition problem for image information may be thought of as consisting of three distinct and separate steps: preprocessing; feature extraction; and classification, as shown in Fig. 1. Preprocessing may consist of such steps as: object location; normalization; or enhancement. Feature extraction, consisting of deriving certain characteristic measurements from the input picture function, is considered by far the most difficult step [1]. Many pattern classification methods are available. One approach to the feature extraction problem, applicable to a restricted, although large class of picture functions, is to derive measures of a statistical nature. Such measures are most appropriate when dealing with scenes containing a repetitive or quasi-repetitive patTvlHE

Manuscript received November 22, 1971; revised March 1, 1972. This work was supported in part by USPHS Grants SM17729, CA06263, and GM01738. A preliminary version of this paper was presented at the IEEE, UMC, Two-Dimensional Digital Signal Processing Conference, Columbia, Mo., October 6-8, 1971. R. N. Sutton is with the Department of Radiology, University of Missouri, Columbia, Mo. 65201. E. L. Hall is with the Department of Electrical Engineering, University of Missouri, Columbia, Mo. 65201.

tern. Another term to describe such a pattern is "texture" [2], [3]. The interpretation of radiographs, specifically standard chest radiographs, is a problem of sufficient practical importance and sophistication to merit consideration as a pattern recognition problem. That in-depth interpretation of these films requires a subtlety that is uniquely human may not be immediately obvious to the uninitiated [4], [5]. Nevertheless, it is quite possible that machine processing might be highly advantageous in certain circumstances of high patient load in a screening capacity. A large number of disease processes affect the lung in such a manner as to produce abnormalities detectable on the X-ray as texture changes. These changes are distinctly contrasted to the many processes that produce well localized discreet lesions. As one step toward the overall goal of automatic machine interpretation, an algorithm that recognizes and classifies abnormalities in texture would be highly useful. The purpose of this paper is to describe examples of a general class of statistical measures that have been found useful in the normal-abnormal classification of certain lung disease processes. These examples illustrate the feasibility of automatic computer recognition of subtle texture differences such as those produced by pulmonary disease. Further details concerning the general topics of clinical chest radiography and the medical aspects of the manifestations of chest disease are contained in

Appendix I.

There exists no optimum method for feature selec-

668

IEEE TRANSACTIONS ON COMPUTERS, JULY

1972

tion. Features extracted must be evaluated for usefulness at the classification stage of the image recognition system. A heuristic approach is indicated for feature selection. Features are selected followed by computerimplemented measurement on a training sample of chest radiographs. Finally, classification is performed to evaluate usefulness of the features. In the following sections, an important preprocessing step, lung location, is considered and an algorithm for automatic lung location illustrated. A general class of texture measurements is then described and illustrated on artificially generated texture patterns. The use of texture measurements for automatic computer diagnosis of lung disease is then illustrated for three types of texture measurements: a general texture measure; Fourier transform domain energy measurements; and directional edge measurements. Finally, the computational problems and results obtained with each are compared.

Fig. 2. Binary image produced by local threshold method for lung location.

angiocardiograms. The method consists of the following major steps. Step 1: The picture is divided into a set of small overPREPROCESSING lapping regions. Step 2: A gray-level histogram is computed for each Preprocessing techniques to compensate for variaregion. tions in radiograph exposure and patient anatomy are Step 3: Histograms with large variances are selected necessary for reliable feature extraction and subsequent as being from regions which most likely contain lungaccurate classification. Several preprocessing techniques have been previously described that may be chest wall-mediastinum boundaries. Step 4: For each selected histogram considered to be used to normalize gray-level distribution, gray-level a mixture of two Gaussian component distributions, the average value, contrast level, variations in subject size, and subject location and orientation within the frame. coefficient of mixture is determined using a least-squares Specifically, the lung location problem will be con- curve-fitting algorithm. Step 5: The resultant mixture of estimated distribusidered here in detail. tions is tested for bimodality in proper gray-level range. We believe that it is important, if automatic pattern Step 6: For each histogram which passes the birecognition is ever to be of practical value, that admodality test, the maximum likelihood, threshold is vantage must be taken of all a priori knowledge of the scene. In this instance we shall guarantee the pre- computed. Step 7: The thresholds for each region are interpoprocessing algorithms that the input shall be a properly lated into a threshold array for each image point. exposed, properly centered, postero-anterior (PA) view, Step 8: A binary decision using this calculated spaadult chest radiograph within the limits normally entially dependent threshold is made for each image point countered in clinical radiography. Object location may be considered as a final goal resulting in a binary image. The lung regions are repre[6], as a preliminary step in scene analysis [7], or sented as black points and other points are white. Details of the implementation of these important simply as a preprocessing step for automatic recognition. The precision required in object location de- preprocessing steps are contained in [8]. Estimates of the midline of the chest, right and left termines the preliminary scanning resolution and thus greatly affects computation time. Since computer-con- sides, and top and bottom of the lungs are made from trolled image scanning systems are available, it is not the x and y gray-level profiles of the picture. A typical unreasonable to consider locating objects at one resolu- binary image is shown in Fig. 2. tion and extracting features at another resolution. FEATURE EXTRACTION Such a procedure is described in this paper. To conserve By appropriate manipulation of the input picture computation time, resolution is kept to the minimum required at each step. The lungs are located at a digital function, one or more features may be extracted to be resolution of 0.75 points/mm while feature extraction used in a classification scheme. One may view the genat resolutions of 0.75, 1.5, and 3.3 points/mm are con- eralized texture measurement problem as a series of mappings as shown in Fig. 3. A texture measure is considered. The method used for locating the lung fields consists structed that will map a texture image onto a correof a local threshold method similar to that used by sponding gray-level image. An average property of the Chow and Kaneko [6] for boundary detection in cine- gray-level values of the output picture are therefore

SUTTON AND HALL: CLASSIFICATION OF PULMONARY DISEASE

669

o 0 0. 0 0 1> 10 0Q4 431 0 ( 5 4 2 0 01 5 4 3 0 0 S 4 2 0 01 4 3 1 00 C 2 100 04 O O O O O c 0 0.0 0 0 000004t

42

0 0 0 0 0 t

0 1 ec 0 o '2 0 0C o 0 23 1 0

0 c

(r)

;II A1Jr Air.r,j

Aij

Ai j.r

A - At.j+rl

Fig. 3. Mapping for texture measurement.

* 4 2 0 0 c S 4 3 0 0 C 5 4 2 0 0 t S3i It Q. 0.< 2 3 C004 Qi._.Q_Q Q.. 000004 000004 0 0 04 0 00 0 04

0
0 u 0 0

a
0 0

Q
4'

related to the average value of the local property of the input picture. Although there is no general theoretical basis for choosing an optimal set of measures in a given circumstance, one may develop insight into the problem by developing measures that select for certain geometrical configurations. Consider the large class of functions M, expressing a relationship between the gray-level values of the picture points A in a scene of the general form
M
=

Fig. 4. Computer-generated pattern of circular objects.

9a000

00000 00001 00013

a0000

000 34 00024 00013 00001 00000 00000 00000

00000 00000

00024

55 5420000 56. 5 4 2 0 0 0 0 4 4 4 3 1 0 0 0 0 2 3 2 1 0 0 0 0 0
0

4 4 4 3 1 0 0 10 0

23 2

100000

00 000000 00a C000Q0

o a a a o o o o
a o o o o o o o o 0 o o o o o o o o

EEEE Aij-AklI I
i

(1)
(2)

o6ooo 00000
a0000 0000o

00000

where

0000o o 0 3 0 c 00004

o o o a o c o o o

=
i

I E
j

Aij-Ai,j+l

0)013 0002 0003' 00024

0001: 0000 0000 0 00
0 0
0

00001

o aoooooooo o o o a
5 6

o o o o 2 3 2 1 0 0 0 0 0 4 4 3 1 000Q0 5 5 5 4 2 0 0 0 0

a linear difference relationship, is familiar and is one of these related functions. We have investigated some of the properties of -a number of these functions for possible use as texture measures. It is apparent that for picture functions with large i and j that the limitations of computation time place certain restrictions on the size of k and 1. Picture functions of the general form

5 5 5 4 2 0 0 0 0 4 4 4 3 1 0 0 040 20 2 1000000 a O a 0t a 0 0 0 0 0000 0 0 0 ^ O GW^ ^ ^ ^

a -o o o

Fig. 5. Computer-generated pattern of wider spaced circular objects.

They consist of roughly circular objects with uniform diameter and spacing. The spacing of the objects in 15 (3) Fig. 4 is approximately for units, that in Fig. 5, 20 units. {Aij; i,j = 1. . . , 256} its ease of computation and A measure selected symmetry is with sample points assuming gray levels 0-63 have been used. Limiting discussion now to that family of M(r) = , + E { Aijpossible picture functions that have identical gray-level distributions (to which many pictures can be made to + A ij- A i,j+r + A 7- A i,j-r } (4) conform by redistribution of pixel gray-level values) a completely random picture is one that satisfies the This measure was applied to the picture functions derequirement that the gray-level values of sample points scribed above, with results noted in Fig. 6. The measure is calculated as a function of increasing be statistically independent, i.e., p(x/y) =p(x). Characteristic deviations from randomness might be indica- "r." Note that the first minimum in each curve occurs at r equal to the center to center spacing of objects. If tive of a specific texture. As an aid in developing insight into the properties of the object of the pattern differentiating scheme convarious measures, we have applied them to a group of sisted of classifying a large number of similar pictures, artificial picture functions of known geometrical proper- a single measure, such as M(15), would accomplish the ties. Two such pictures are those noted in Figs. 4 and 5. separation as shown in Fig. 6.

670
TEXTURE

IEEE TRANSACTIONS ON COMPUTERS, JULY 1972

MEASURES
OF
TEST

PATTERNS

I..

PATTERN -0-20 UNITS CENTER TO CENTER

1S

.20

PATTERN 2 D--0S0 IE UNITS CENTER TO CENTER

II 0

Fig. 7. Normal chest X-ray.

IS

10

20

25

Fig. 6. Texture measurement for computer-generated images.

If the differentiation problem consisted of the separation of pictures with vertical lines from those with horizontal lines, an appropriate measure might be

(5) which would have a small value for lines in the i direction and larger values in- the j direction. In general, we feel that the differentiation of more complicated patterns can in many circumstances be accomplished by appropriate choice of a texture measure or measures of this form. Naturally occurring textures are not as simply described as the examples given,- but certain progress has been accomplished in a selected class of problems.
APPLICATION TO LUNG DISEASE CLASSIFICATION The process described as interstitial fibrosis, attributable to a large number of disease entities, is typically manifested on the standard PA erect chest film as an abnormality in texture. Disease processes in the lung cause certain anatomical structures to be increased in prominence and hence absorb more of the incident radiation causing decreased exposure of the film. This is demonstrated by comparison of Figs. 7 and 8. The abnormal chest film (Fig. 8) has an overall fine pattern, the nature of which has been described as "lacey" or "reticular." Some of the disease entities which are known to cause changes similar to these, and which are a significant public health hazard, are the pneumoconioses, which include anthracosilicosis asbestosis, beryllosis, and a large number of others. Restricting consideration to those measures that consist of some arithmetic combination or comparison between adjacent or near adjacent picture sample points, i.e., local measures, one might expect to find deviations from randomness or a short-range order characteristic of abnormal and normal conditions upon which a classification might be made. We shall describe the implementation of two such

M(r) =

Aij-Ai+r,jI

Fig. 8. Abnormal chest X-ray.

Fig. 9. Restricted area of consideration of chest X-ray.

measures to classify a restricted set of chest radiographs into normal and abnormal groups. In this study the area of consideration in the PA chest radiograph was restricted to a 76.8-mm square area in the region of the right apex, bounded by the projection of the thorax laterally, the clavicle superiorly, and the superior mediastinum medially. The choice of size of the scan area, and the anatomical area is not critical. These areas are outlined schematically in Fig. 9. The readily identifiable structures in the normal film are: two or three ribs, a clavicle projecting more or less laterally, and the structures of the hilum at the lower right with normal vessels and bronchi projecting in a

SUTTON AND HALL: CLASSIFICATION OF PULMONARY DISEASE

671

1.0

0.5

.
Fig. 10. Area of consideration in normal chest X-ray.
---

_ PICTURE I NORMAL
V

-- o PICTURE 2 NORMAL
-O

PICTURE 4 INTERSTITAL FIBROSIS PICTURE 3 INTERSTITAL FIBROSIS

0'

10

20

30

40

50

Fig. 12. Texture measurement M(r) for sample films.

TABLE I SUMMARY OF CLASSIFICATION RESULTS FOR SYMMETRIC TEXTURE MEASURES DESCRIBED IN TEST FOR SPECIFIC VALUE OF M(r)

Classification Results
Texture Measure M(r)
M (30)
-

Percent Correct

Training

Testing
42 46 63 42

71
71 71

M (35)
M (40)

M (45) Fig. 11. Area of consideration in abnormal chest X-ray.

67

fan-like fashion outwardly. The selected area was scanned with the University of Missouri Image Analysis Laboratory's image dissector camera into a 256X256 point array (one scan line per 0.3 mm) with 64 gray levels. Typically normal and abnormal films are shown in Figs. 7 and 8 and again in Figs. 10 and 11.
EXAMPLE 1-A SYMMETRIC TEXTURE MIEASURE

Consider now the measure defined as a function of "r" described previously in (4). This measure was applied to an area of lung in the region of the right upper lobe in the chest films shown previously. The measure was calculated as a function of "r" on four saimple films, two normal and two abnormal, selected only on .the criteria that they be: 1) typical of the pathologic process of typically normal; and 2) that the pictures have similar gray-level distribution functions. Pictures of the sample areas scanned are illustrated in Figs. 10 and 11. The "lacey" or "reticular" pattern

or texture is clearly apparent in the abnormal case. Four films were selected, two typically abnormal, two typically normal, for use as test material for the exhaustive investigation of the texture measure proposed above. Promising measures at a specific "r" value could then be calculated individually on the entire test series of 24 films. The results of this preliminary investigation are shown in Fig. 12. Subsequently, based on the data obtained in this experiment, four specific "r" values seemed to show promise. These were calculated specifically on the entire test series. The slower rise in the curves of the abnormal films in the region of r=30 to 50 presumably attributable to short-range order poorly appreciated visually in the original picture. A feature consisting of a single measure at r=40 in this case might serve to differentiate between normal and abnormal. The classification results using values of the measure M(r) obtained at four values of r are shown in Table I.The metlhod used for classification is described in Appendix II.

672

JEEE TRANSACTIONS ON COMPUTERS, JULY 1972

EXAMPLE 2-A TEXTURE MEASURE COMPARING AREAS OF THE SAME PICTURE Given a scalar point function f(x, y), the gradient is defined as a vector point function denoted as
Vf
Of
= -1 T
Ox

df +-.

ay

(6)

Suppose the discriminating texture is directional and related to edge structure which may be measured by the partial derivative. For a digital computation the partial derivatives may be approximated by directional differences of the form

7f
x

Aij Ai+,,j

Vf

Ai,j

(7)

Fig. 13. Schematic diagram of quadrants considered.

Calculation of such directional differences can be thought of as measuring the "edge content" in the picture function in a given direction. A similar measure that notes change without regard to an increase or decrease might be noted as the absolute value of the directional difference:

Yf I=

Aij- Ai+,j| v

(8)

We have found that the distribution of the edge structure within the picture is of more important diagnostic significance than is the total amount of edge structure. A measure was constructed by subdividing the area of interest into quadrants (Fig. 13), calculating an edge measure in each quadrant, then comparing the values of the measure in adjacent quadrants. This mapping is shown in Fig. 14. The edge measure calculated in each quadrant was the total "edge" in the x direction, or
Ek
=

M(r)

=EaIWL 3 2OuADRAsT4

Fig. 14. Mapping for directional edge measurements.

TABLE II TABULATED DATA FOR CLASSIFICATION USING DIRECTION EDGE QUADRANT MEASURE DESCRIBED IN TEXT

E E Aij- Ai+,,j|,I
reglonk
i.

1, 2, 3, 4

(9)

Ratio of Directional Edge in Quadrant 3/Quadrant 4 at Different Regions

where k is the quadrant number. A consistent finding is that in normal cases more x-direction edge points are in the area corresponding to the anatomical structure, the hilum (quadrant 3), than in the area corresponding to peripheral lung (quadrant 4). The ratio M= E3/E4 (10)
expresses a relationship between the edge content of the regions. This ratio has been found in the test series to be almost always greater than 1 for normal cases, and less than 1 for abnormal cases. Other consistent findings were also noted; however, the one described produced the best discrimination and classification results. The computed edge ratio M is shown in Table II for a training set of 9 normal and 15 abnormal cases. The ratio was computed at three different resolutions. Thus,

M256X256
Normal Cases
1.268 1.219 1.165 1.140 1.030 1.035 1.192

0.998

1.360

M12SX128 1.301 1.282 1.260 1.040 1.269

1.342 1.182 1.141

M64X64
1.289 1.336 1.149 1.251 1.011 1.462
1.032 1.442

M'256X256
0.997
1.164

1.270 1.216

1.275

1.477

1.356 1.129 1.026 1.039 1.199

Abnormal Cases

0.791 0.875 0.839 0.952 0.872 0.879 0.959 0.926 0.930 1.030

0.800

1.01 0.959

0.880 1.400

0.967 0.963 1.063

1.012 0.940

1.250 0.124 1.200 0.898

1.307

0.782 0.874

0.887

0.831 0.989

M2s56X256 corresponds to the computation for the full 256X256 array, M128X128 corresponds to the computation on a 128X128 array produced by averaging each

1.062 0.939 0.968

1.006 1.155 1.113

0.796 0.993

0.908 1.022

1.01 1.078

1.195 1.400 1.201 1.243 1.210 0.983

1.410

0.953 0.928

0.926

1.028 1.058 0.948

0.970
0.766 0.834 0.913

1.372

SUTTON AND HALL: CLASSIFICATION OF PULMONARY DISEASE

673

TABLE III SUMMARY OF CLASSIFICATION RESULTS USING RATIO OF DIRECTIONAL EDGE MEASUREMENT DESCRIBED IN TEST AT VARIOUS RESOLUTIONS
Percent Correct

Classification

Measurement

Training
92
92

Testing
75
84

M256X256
M128X128

M64X64
M'256X256

71
92

50
75

TABLE IV SUMMARY OF CLASSIFICATION RESULTS-MEASUREMENTS MADE UTILIZING AN ANNULAR RING SAMPLING GEOMETRY ON MAGNITUDE FOURIER TRANSFORM

Fig. 15. Energy domain sampling geometry used for sampling the origin centered magnitude Fourier transform of picture data.

Energy Classification Window Classification Threshold Percent Correct Between Percent Correct Annular Annular Ring Ring Training Testing Number Number Training Testing 63 71 0-5 38 63 5 50 71 5-10 63 71 10 50 71 10-15 71 63 15 50 71 15-20 63 75 20 50 71 20-25 63 79 25 67 71 63 25-30 79 30 67 71 63 30-35 35 791

four points in the original array, and M64X64 corresponds to a similar computation on the image, again reduced. Note that a testing classification of 92 percent and a training classification of 84 percent are attained for M128X128 as shown in Table III. Another experiment was performed to test the dependence of the directional edge measurement on the gray-level distribution of the images. The gray-level distributions of the test images were transformed into uniform distributions [1], and the edge measurements repeated. These measurements denoted M26X256 are also given in Table I. The classification results shown in Table IV suggest that edge measurements are insensitive to the gray-level distributions.
FREQUENCY DOMAIN MEASUREMENTS Another method often proposed for measuring texture differences in the films consists of a Fourier domain energy measure [9]. The abnormality in the chest X-rays has been described previously as consisting of a lacey or reticular pattern. A difference between the normal and the abnormal might be detectable in the transform domain at a spatial frequency characteristic of the average size or spacing of the unit structure of the abnormal texture. The measure may be expressed as the integral over a midfrequency range of the Fourier transform magnitude spectrum. The lower bound of the transform domain window may be'chosen to exclude the
EXAMPLE 3

contribution of normally occurring structures such as ribs, while the upper bound of the window might be chosen to exclude the contribution attributable to highspatial frequency noise or minor differences in sharpness between originals. The energy measurements were generated by first computing the two-dimensipnal Fourier tran$form magnitude spectrum, followed by normalization to unit energy. A circular symmetric annular ring sampling geometry was then emplpyed summing the magnitude values between annplar rings, Fig. 15 [1], [9]. An annular ring signature is thus obtained from which a distribution function can then be obtained. Differences between values of the distribution then correspond to total energy in a window. An example of an anular ring frequency distribution function is shown in Fig. 16. Energy distribution measurements corresponding to a frequency spacing of 0.6 lines/cm (5 points) were used as features for pattern classification. Energy values for the frequency windows between these points were also used. The classification results are shown in Table I I I.

DISCUSSION
Three examples of texture measurement for automatic computer diagnosis of pulmonary disease have been investigated. Experiments were computed on a test series of 24 films using symmetric edge, directional edge, and frequency domain energy measurements. From the results of these experiments we found that the directional edge feature provided the best overall classification results. The area of interest utilized in these experiments constitutes approximately 1/4 the entire chest film, and good classification results were obtained using measurements at resolution 128X128. This suggests that sampling the entire film at resolution 512 X512 would be sufficient for classification using this

approach.

674
1.00

IEEE TRANSACTIONS ON

COMPUTERS, JULY 1972

far from exhaustive, we feel that the results justify the conclusion that automatic computer classification of certain pulmonary abnormalities is feasible. Cost effectiveness, however, is quite another matter. We are in the process of expanding our repertoire of approaches to the automatic diagnosis and investigating applications to other types of lung disease.
ANNULAR RING ENERCY DISTRIBUTION
I.

E NERGY DISTRIBUTION

.990

.985
-

10

20

30

40

FREQUENCY

SAMPLE

RING NUMBER

Fig. 16. Frequency domain energy measurements.

The directional edge measure compares edge content in two quadrants of the area of interest. These quadrants also correspond anatomically to areas of peripheral lung in quadrant 4 and less peripheral lung in quadrant 3. As was pointed out previously the relative absence of "edge" structures peripherally is to be expected in the normal case. This particular measure therefore has a basis in normal anatomy. This measure, however, also has merit for another reason. By virtue of the ratio nature of the measure, the influence of differences in technical factors between pictures, such as average grey level and contrast are minimized. The normalization problem deserves emphasis. Throughout the experiments difficulties were encountered due to the technical factors mentioned. Numerous normalization schemes and approaches were tested. When comparisons were to be made between pictures quite different in contrast, linear redistribution of gray-level values of picture elements was found to be of particular merit. The classification results using measurements obtained from the Fourier transform domain were disappointing. This area has appeared promising to others [9] and will receive further attention. While the experiments performed were by no means exhaustive with regard to the selection of energy threshold and windows, from the data it seems unlikely that an outstanding feature might have been missed. We wish to emphasize the preliminary nature of the data presented. While the experiments performed are

APPENDIX I Radiographic examination of the chest usually begins with the exposure of a standardized view, the PA 6-ft erect 14 X 17 film. Commonly an additional view, a lateral projection is obtained at the same visit. After examination of this preliminary film by a medical doctor specially trained in radiologic diagnosis, more views may be obtained. The initial view is stereotyped in projection to simplify the identification and anatomical location of any pathological (disease) process. Due to the public health value of this examination, the deficiency of suitably trained professional personnel, and the stereotyped initial view obtained, it is apparent that an opportunity exists to exploit mechanized interpretation, at least in a screening capacity. Refer to Fig. 7, a reproduction of a normal standard chest film [1]. The film is obtained in the following standardized manner. The X-ray tube to film distance is 72 in. The peak voltage across the tube is usually approximately 140 kV. The patient is positioned with the front of the chest to the film, iback to the X-ray tube. X-rays then pass from posterior to anterior through the patient. Exposure of the film by X-rays that pass through the patient, followed by development, results in blackening of the film in proportion to exposure. The process is complicated by all the nonlinearities experienced in other photographic processes. At the energy commonly used, X-ray absorption is primarily by Compton scattering, the important factor being density of electrons in the absorbing medium. Materials of high atomic number are more efficient absorbers than those of low atomic number; liquids and solids more efficient than gases, etc. This is manifested on the chest film by air-containing structures appearing black, more solid tissues appearing light. Bones, containing calcium, are rather more radiodense radioopaque than soft tissues. Anatomical structures readily noted on the normal chest film are the ribs, the thoracic spine, the heart, and the diaphragms separating the chest cavity from the abdominal cavity. While all structures in the film are examined for abnormality by the radiologist, primary emphasis is on the lungs. Originating in the center of the film radiating laterally in a more or less fan-shaped manner are the shadows corresponding to the bronchi or large air passages and the pulmonary arteries, main blood vessels which carry blood from the heart to the lungs to become oxygenated. The ramifications of the major

SUTTON AND HALL: CLASSIFICATION OF PULMONARY DISEASE

675

branches of these structures from central to peripheral is manifested in the film by light streak-like shadows better seen in the close-up view of the area corresponding to the right upper lobe, as seen in Fig. 10. Disease processes that alter the anatomy of normal structures are manifested commonly by characteristic alterations in the pattern of X-ray absorption. In this paper, we restrict consideration to those processes that are manifested by overall or generalized alterations in the normally expected X-ray absorption pattern. The numerous disease processes categorized by the term "interstitial fibrosis" cause an increase in the amount of tissue adjacent to the small bronchi and blood vessels manifested in the film as a more or less reticular network pattern. A typica' example is presented in Fig. 11. If instead of increased prominence of the interstitium, the alveoli, or air-containing structures where gas exchange takes place, become filled with fluid, a different characteristic pattern is seen. A great number of unrelated disease processes produce interstitial patterns, alveolar patterns, and various mixtures of both. These more or less generalized abnormalities are to be contrasted with localized abnormalities such as small tumors, rib fractures, and localized pneumonia. We emphasize the word "screening." At this time, it seems difficult to imagine that machinery might be developed that could rival the human visual system in complete interpretation of all the fine details. A large class of chest diseases seems to be particularly suited for automatic detection. Within this class are the pneumoconioses, diseases produced by inhalation of toxic materials, often by industrial exposure. From a radiologic point of view, these diseases are commonly manifested as interstitial processes. Rather than attempting to first develop a full diagnostic system capable of replacing highly trained professionals, the aim is for an interim screening capability, a system capable of differentiating between normal and abnormal in a general sense. Such a limited system would be a more realistic scheme at our current level of technology. Workers in high risk areas might have frequent chest X-ray evaluations to determine quickly, simply, and inexpensively whether or not they are developing the pathologic changes characteristic of the prolonged inhalation of toxic material. A mass screening capability is clearly desirable.
APPENDIX I I The pattern classification results presented in this paper were computed by a simple linear discriminant function analysis for a single measurement two-class problem. For a pilot study it is unreasonable to select too many training films because of the large amount of experimentation required for measurement, computation, and selection; however, enough films must be used to provide meaningful classification results. Retricting consideration to a single measurement provides the smallest

number of training films for reliable classification. However, the possibility of failing to achieve sufficiently correct classification is greater. A commonly accepted rule of thumb is that a minimum of 10 samples per measurement per class is required. Therefore, 24 samples for a single measurement two-class problem satisfies the minimum requirements. The training sample classification results were obtained by assuming that the normal films were from class 0, while the abnormal films were from class 02, and that a single measurement x had been made for each film. The linear discriminant function d(x) is of the form d(x) = ao + a1x.

That is, a sample x is assigned to class 0, if d(x) >0 and to class 02 if d(x) <0. The coefficient a, was determined for both the perpendicular bisector and equal variance rule, for Gaussian distributions. The coefficient ao was then determined by an optimization technique. The set of coefficients that provided the greatest classification accuracy was selected. The testing classification results were obtained by using a one-out-of-N jackknifing procedure. Each sample was removed and a decision function computed from the remaining 23 samples. The removed sample was classified. This procedure was continued until all samples were tested.

APPENDIX I I I Consider an input picture function A ij consisting of a repetitive pattern, periodic in the i and j direction with period N and M. An output picture function A ij' is to be constructed (Fig. 3) by the mapping Aij'= A| kl| By an appropriate choice of k and 1, namely: k = N + i; I = M +j such that:

IJ-A

Aij'=

AN-v+iM+j|
-

(as a consequence of the periodicity of Aij), the value of Aij' would then be zero. If. the period of the input picture function were unknown, an experiment consisting of the measurement of the average value of A j' for various N and M would serve to establish the period. If the object of the experiment were to determine the periodicity in the i or the j direction only, a calculation of the average value of the function A = A ij- A,M+j| and Aj' = Ai - AN+i,jI
would produce the desired result. Such an experiment is described in this paper and illustrated in Fig. 6. By knowing the relationship between A and A', and given the distribution of the graylevel values of the input picture function, one can usually calculate the expected value for the output picture function points A ij' and hence the expected value for the average value of A.j'. In practice, how-

676

IEEE TRANSACTIONS ON COMPUTERS, JULY

1972

ever, except for rather simple relationships between A illustrated by the intuitive feeling when viewing Figs. and A', finding the expected value of Aij', given the 10 and 11 that a disproportion of "edge activity" is gray-level distribution of A ,j, is a computationally present laterally in the abnormal chest film. A measure tedious problem. If the object of the transformation was constructed as shown in Fig. 14, and confirmed by A--+A' is to translate some average local property to an testing with classification results. average overall property, with subsequent comparison between pictures and their average properties, normalACKNOWLEDGMENT ization must be made for differences between inputThe authors gratefully acknowledge the assistance of picture function gray-level distributions. many members of the Image Analysis Laboratory and A circumvention of this problem can be accomplished the Department University of by making all the input-picture functions the same by souri, Columbia. of Radiology at the wish to thankMisIn particular they Dr. using a linear redistribution technique. In spite of the G. S. Lodwick, Professor and Chairman of the Departfact that the distribution of the gray-level values of ment of Radiology, University of Missouri Medical the input picture is discrete, the statistics of real pictures at a resolution of 256 X 256 allows for approx- Center, Columbia, and Dr. S. J. Dwyer, III, Director imately linear redistribution. Linear redistribution at of Bioengineering Program, University of Missouri, 1024 gray levels per picture point, followed by conver- Columbia. sion to 64 gray levels for computation of the texture REFERENCES measures, was performed on all input pictures in this [1] E. L. Hall, R. P. Kruger, S. J. Dwyer, III, D. L. Hall, R. W. study. This insures that the distribution of the grayMcLaren, and G. S. Lodwick, "A survey of preprocessing and feature extraction techniques for radiographic images," IEEE level values of each picture, and consequently the Trans. Comput., vol. C-20, pp. 1032-1044, Sept. 1971. average gray level, is as close to identical as the dis- [2] A. Rosenfeld and E. B. Troy, "Visual texture analysis," in Proc. UMR, M. J. Kelley Commun. Conf., Oct. 5-7, 1970, pp. 10-1-1crete nature of the data will allow. 10-1-6. A relationship of the A j' = A ij-A kl for which k and [3] B. S. Lipkin and A. Rosenfeld, Eds., Picture Processing and Psychopictorics. New York: Academic Press, Inc., 1970, pp. 2891 have been appropriately chosen to minimize the aver381. age value of Aij' if a repetitive pattern exists, can be [4] D. L. Hall, G. S. Lodwick, R. P. Kruger, S. J. Dwyer, III, and J. R. Townes, "Direct computer diagnosis of rheumatic heart used as a test function, the sensitive measure being the disease," Dep. Elec. Eng., Univ. Missouri, Columbia, Tech. Rep., average value of Ai>'. By appropriate experimentation June 1971. R. N. Lodwick, "Texture measureand an a priori knowledge of possible textures or re- [5] ments Sutton, E. L. Hall, and G. S.digital image processing techof pulmonary pathology by petitive patterns in the originals, a relationship A--A' niques," in Proc. Conf. Eng. Med. and Biol., Las Vegas, Nev., can be designed. The overall average property of the [6] Nov. 1971, p. 33.3. Kaneko, "Boundary detection of radiographic C. K. Chow and T. output A' can then be used as a feature in a classificaimages by a threshold method," IBM Rep. RC 3203, Dec. 1970. [7] C. A. Harlow and S. A. Eisenbeis, "The analysis of radiographic tion scheme. images," in Proc. Two-Dimensional Digital Signal Processing An example of a combination of experimental and Conf., Univ. Missouri, Columbia, Oct. 1971, pp. 11-1-1-11-1-10. Hall, "Lung location by a local threshold method," Dep. intuitive tailoring of a texture measure is shown in [8] E. L. Eng., Univ. Missouri, Columbia, Image Analysis Lab. Rep., Elec. Fig. 12. As a result of information gained by this preNov. 1971. G. L. Stanley, liminary experiment, specific measures at selected r [9] for G. Lendaris and G.recognition," "Diffraction-pattern sampling automatic pattern Proc. IEEE, vol. 58, pp. 198values are tabulated in Table I. A further example is 216, Feb. 1970.

I,