Академический Документы
Профессиональный Документы
Культура Документы
I. I NTRODUCTION
RTHODONTICS usually examine lateral cephalograms, identify the landmarks based on the prior-knowledge. Their
O x-ray images taken from the side of the face, and analyze
cephalometric landmarks, morphological feature points of hard
performance is highly dependent on the image quality, and
they cannot find some kinds of landmarks that are not on
and soft tissues. The landmarks are employed to measure any characteristic contours. Model based approaches [7]–[9]
the cephalometric components as distance in millimetres and produce models that detect most likely regions including
angles in degree [1]. Identifying landmarks is essential in landmarks. Combined approaches [10]–[12] consider active
contemporary orthodontic diagnosis and treatment planning. shape/appearance models to refine cephalometric landmarks
However, anatomic landmark identification based on the by combining with template matching. They have difficulty
visual assessment of the cephalogram remains a difficult in selecting the representative template or shape and are not
task that requires manual measurements by specially trained robust to noise and variability in individual. Machine learning
clinicians. Unfortunately, the manual process often leads to based approaches [13]–[15] such as neural network and SVM
measurement errors [2]. Such manual identification is con- improve the shape variability and noise tolerance. However,
sidered time-consuming work that takes on average 15 to these approaches require a large number of cephalograms to
20 min from the expert to handle each individual case [3]. improve the accuracy. Identification performance were not
In our case at Osaka University Dental Hospital, a trained very satisfactory in a practical use.
clinician takes about 10 to 15 min for identifying about 40 The recent breakthrough in image recognition technology
landmarks per individual. A fully-automated clinical exam- using deep convolutional neural network (CNN) model [16],
ination of cephalograms would reduce the workload during [17] brings dramatic improvement in diagnostic imaging.
routine clinical service and would provide orthodontists with This methodology can be used to automatically diagnose the
more time for optimum treatment planning. presence of tuberculosis in chest radiographs [18], to detect
Various methods to automate landmark identification have macular degeneration from fundus images [19], and to locate
been proposed. Image processing approaches [4]–[6] usu- malignant melanoma in skin images [20]. It is eagerly desired
ally extract contours regarding to landmark locations and in the field of orthodontics as well, along with the increasing
demands for dental healthcare.
Chonho Lee is with Cybermedia Center, Osaka University, Japan. e-mail: In this paper, we develop a deep neural network based model
leech@cmc.osaka-u.ac.jp.
Chihiro Tanikawa is with Graduate School of Dentistry, Osaka University, for a fully-automated cephalometric landmark identification
Japan. e-mail: ctanika@dent.osaka-u.ac.jp. and demonstrate its feasibility in a practical environment.
Jae-Yeon Lim is with Graduate School of Dentistry, Osaka University, Unlike the recently proposed work [21], [22], two deep neural
Japan. e-mail: j.lim@dent.osaka-u.ac.jp.
Takashi Yamashiro is with Graduate School of Dentistry, Osaka University, networks are trained by multi-scale patches, i.e., cropped
Japan. e-mail: yamashiro@dent.osaka-u.ac.jp. images that contain landmarks, whose rectangle size varies
2
Softmax, 34 nodes
CNN-PC
33 classes
... + none
2x2 MaxPool
2x2 MaxPool
2x2 MaxPool
3x3 Conv, 32
3x3 Conv, 32
3x3 Conv, 64
“S”, 112, 108 “S”, 181, 40 “S”, 35, 103
ReLU
ReLU
ReLU
FC, 1024
FC, 64
...
2 nodes
“N”, 57, 131 “N”, 51, 24 “N”, 138, 31
CNN-PE
2 nodes for
...
for all landmarks
X and Y
Fig. 2. An example training dataset that contains multi-scale image patches Fig. 3. A structure of CNN-PC and CNN-PE (convolutional neural networks
of landmarks with their name and location in the patch. for patch classification and point estimation.)
...
e.g., Resized patches
II. DATASET AND M ETHOD Grid points Sliding multi-scale windows classified as “S”
CNN-PE
Y
for “S”
...
2 0.8
X
...
...
1.5 0.6
Patches
Classified as “Go”
for “Go”
1 0.4
CNN-PE
Y
...
X
0.5 0.2
val_loss val_acc
Candidate patches Scatter plot & Distribution Estimated landmarks 0 0
0 1 2 3 4 5 6 7 8 9 10
Epoch (x100)
“U1”
Multi Root of the upper central incisor TABLE I
modality C OMPARISON RESULTS OF MEAN ESTIMATION ERRORS ( MM ) AND
cluster centroid
STANDARD DEVIATIONS (±) OF HARD TISSUES
Gla
“Go”
Multi
External angle of the mandible, formed
modality Estimation Error
Prn by the posterior border of ramus and
the inferior border of mandible Landmark Saad [8] Tanikawa [23] Proposed
L1 1: S 3.24±2.85 2.10±2.52 2.06±1.70
Go Similar to “Gla”
other soft Most prominent soft tissue point
2: N 2.97±1.85 1.70±1.18 1.67±1.07
tissue in the median sagittal plane between
the supraorbital ridges 3: Po 3.48±2.46 3.63±2.53 2.62±3.40
“Prn” 4: Or 3.42±2.43 2.24±1.35 2.16±6.17
Blurred
Soft tissue point on tip of nose 7: Ar – – 1.95±1.39
13: Go 3.64±1.76 5.06±3.41 3.50±2.20
15: Co – – 3.50±4.29
17: Me 4.41±2.03 1.59±1.07 1.35±1.01
Fig. 6. Examples of ambiguous landmarks (e.g., hard tissues U1 and Go, and 18: Pog 3.66±1.74 1.91±1.42 1.43±1.32
soft tissues Gla and Prn) due to multimodality, similarity and blurred image. 19: Gn 4.22±1.78 1.45±0.84 1.32±4.16
20: Ans 2.70±1.05 2.32±1.34 2.27±7.22
21: Pns – – 2.33±3.63
plot, we compute one mean/median point as a landmark point.
24: Point a 2.55±0.97 2.13±1.36 2.65±8.34
Before the computation, we omit points as outliers, whose
25: Point b 2.23±1.24 3.12±9.46 2.53±3.43
Euclidean distance from the mean/median is greater than two
26: U1 3.65±1.59 1.78±2.29 2.97±2.54
times standard deviation (i.e., ±2σ).
27: U1 c – – 3.50±2.85
There are two notes. One is that the candidate patches
28: L1 3.15±2.30 1.81±1.66 1.67±2.90
may not contain correct landmarks, or are miss-classified
29: L1 c – – 2.30±2.88
due to the performance of CNN-PC. We later show how the
proposed multi-scale patches mitigate the error to improve the
landmark estimation accuracy. Another is that we observed
the two cluster distribution for some landmarks as shown in University Dental Hospital, it dramatically reduces the manual
the bottom scatter plot in Fig. 5. Orthodontists realize this landmarking time that an orthodontist spent about 5 to 7 min
multimodality problem that there are landmarks difficult for to identify 30 major landmarks.
them to determine [23]. For example, as you can see in Fig. 6,
there are ambiguous landmarks such as U1 and Go because A. Estimation error
both right and left side of tooth and jaw are respectively shown
in the cephalogram although it is the lateral view. We later Table I shows the Euclidean distance error of estimated hard
show how the estimation accuracy changes when using the tissue landmarks by the CNN-PE, compared to the ground
mean or median with/without outliers. truth values given by orthodontists. The results are the average
values of 9-cross validation, each of which contains 100 test
and 835 training cephalograms, respectively.
III. E VALUATION The estimation performance for most landmarks improves
All experiments were performed in Ubuntu 16.04 LTS with compared to the previously reported results in Saad [8] and
a quad-core 3.6GHz CPU, 128GB RAM and Nvidia P100 Tanikawa [23], except Point a and U1 that are ambiguous
GPU. Training the proposed CNN models took 10 sec per landmarks related to multimodality problem as described in
epoch for 128 batches. As shown in Fig. 7, the CNN-PC has Fig. 6. Besides, the proposed model also estimates additional
achieved around 93% validation accuracy and 0.4 validation landmarks such as Ar, Co, Pns, U1 c and L1 c, not reported in
loss in 1k epoch. For testing, it takes about 0.7 sec to identify the previous work. The proposed model successfully identified
one of major landmarks shown in Fig. 1. In the case of Osaka landmarks with the range of error between 1.32 mm and 3.5
4
TABLE II
PARAMETERS ( THE LENGTHS OF SEMIMAJOR AND SEMIMINOR AXES , AND
ANGULAR MEASUREMENTS BETWEEN X - AXIS AND THE SEMIMAJOR AXIS )
THAT REPRESENTED THE CONFIDENCE ELLIPSES , AND RELIABILITY
varies based on landmark-dependent criteria examined by [16] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification
orthodontists, and 22 hard and 11 soft tissue landmarks are with deep convolutional neural networks,” Advances in Neural Informa-
tion Processing Systems, vol. 1, 2012.
identified. [17] J. Gu et al., “Recent advances in convolutional neural networks,” arXiv
For evaluation, (i) landmark estimation accuracy by Eu- preprint arXiv:1512.07108, 2017.
clidean distance error between true and estimated values, and [18] D. S. Ting, P. H. Yi, and F. Hui, “Clinical applicability of deep learning
system in detecting tuberculosis with chest radiography.” Radiology, vol.
(ii) success rate that the estimated landmark was located 286, no. 2, p. 729, Feb. 2018.
within the corresponding norm using confidence ellipse, are [19] P. M. Burlina, N. Joshi, M. Pekala, K. D. Pacheco, D. E. Freund, and
computed. The proposed model successfully identified hard N. M. Bressler, “Automated grading of age-related macular degeneration
from color fundus images using deep convolutional neural networks,”
tissue landmarks within the error range of 1.32∼3.5 mm and JAMA ophthalmology, vol. 135, no. 11, pp. 1170–1176, Nov. 2017.
with a mean success rate of 96.4%, and soft tissue landmarks [20] A. Esteva, B. Kuprel, R. A. Novoa, J. Ko, S. M. Swetter, H. M. Blau,
with the error range of 1.16∼4.37 mm and with a mean and S. Thrun, “Dermatologist-level classification of skin cancer with
deep neural networks,” Nature, vol. 542, no. 7639, p. 115, Jan. 2017.
success rate of 75.2%. For testing, it dramatically reduces the [21] S. . Arik, B. Ibragimov, and L. Xing, “Fully automated quantitative
landmarking time from about 5 to 7 min by an orthodontist cephalometry using convolutional neural networks,” Journal of Medical
to 21 sec for 33 landmarks. Imaging, vol. 4, January 2017.
[22] S. Wang, H. Li, J. Li, Y. Zhang, and B. Zou, “Automatic analysis of
Together with anatomical landmark identification, the diag- lateral cephalograms based on multiresolution decision tree regression
nosis of anatomical abnormalities using the landmarks is also voting,” Journal of Healthcare Engineering, November 2018.
significant task for orthodontists. In clinical practice, it is more [23] C. Tanikawa, M. Yagi, and K. Takada, “Automated cephalometry: system
performance reliability using landmark-dependent criteria,” The Angle
useful to analyze angles and linear measurements between Orthodontist, vol. 79, no. 6, 2009.
particular landmarks rather than just point identification. We [24] S. Baumrind and R. C. Frantz, “The reliability of head film measure-
will work on evaluating those measurements based on the ments,” American Journal of Orthodontics, vol. 61, 1971.
[25] R. E. Walpole and R. H. Myres, Probability and Statistics for Engineers
proposed model in the near future. and Scientists. Macmillan, 1978.
R EFERENCES
[1] J. A. Delaire, “Maxillary development revisited: relevance to the or-
thopaedic treatment of class iii malocclusions,” European journal of
orthodontics, vol. 19, 1997.
[2] E. Celik, O. Polat-Ozsoy, and T. U. T. Memikolu, “Comparison of
cephalometric measurements with digital versus conventional cephalo-
metric analysis,” European journal of orthodontics, 2009.
[3] S. Chen, Y. Chen, C. Yao, and H. Chang, “Enhanced speed and precision
of measurement in a computer-assisted digital cephalometric analysis
system,” The Angle Orthodontist, vol. 74, 2004.
[4] J. Cardillo and M. Sid-Ahmed, “An image processing system for the
automatic extraction of craniofacial landmarks,” IEEE Transactions on
Medical Imaging, vol. 13, 1994.
[5] D. J. Rudolph, P. M. Sinclair, and J. M. Coggins, “Automatic computer-
ized radiographic identification of cephalometric landmarks,” American
Journal of Orthodontics and Dentofacial Orthopedics, vol. 113, 1998.
[6] J. M. C. Grau V., Alcaniz M., “Automatic localization of cephalometric
landmarks,” Journal of Biomedical Informatics, vol. 34, no. 3, 2001.
[7] D. B. Forsyth and D. D. N., “Assessment of an automated cephalometric
analysis system,” European Journal of Orthodontics, vol. 18, no. 5,
1996.
[8] A. A. Saad, A. El-Bialy, A. H. Kandil, and A. A. Sayed, “Automatic
cephalometric analysis using active appearance model and simulated
annealing,” Journal on Graphics, Vision and Image Processing, 2006.
[9] W. Yue, D. Yin, C. Li, G. Wang, and T. Xu, “Automated 2-d cephalo-
metric analysis on x-ray images by a model-based approach,” IEEE
transactions on biomedical engineering, vol. 53, 2006.
[10] R. Kafieh, A. Mehri, and S. Sadri, “Automatic landmark detection in
cephalometry using a modified active shape model with sub image
matching,” in Proc. of IEEE International Conference on Machine
Vision, 2008, pp. 73–78.
[11] J. Keustermans, W. Mollemans, D. Vandermeulen, and P. Suetens,
“Automated cephalometric landmark identification using shape and local
appearance models,” in Proc. of the 20th IEEE Conference on Computer
Vision and Pattern Recognition, August 2010.
[12] P. Vucinic, Z. Trpovski, and I. Scepan, “Automatic landmarking of
cephalograms using active appearance models,” European Journal of
Orthodontics, vol. 32, no. 3, 2010.
[13] S. Chakrabartty, M. Yagi, T. Shibata, and G. Cauwenberghs, “Robust
cephalometric identification uusing support vector machines,” in Proc.
of International Conference on Multimedia and Expo, July 2003.
[14] D. Giordano, R. Leonardi, F. Maiorana, G. Cristaldi, and M. L. Dis-
tefano, “Automatic landmarking of cephalograms by cellular neural
networks,” June 2005.
[15] G. D. Leonardi R. and M. F., “An evaluation of cellular neural networks
for the automatic identification of cephalometric landmarks on digital
images,” Journal of Biomedical and Biotechnology, 2009.