Musculoskeletal Imaging Original Research

Kung et al.
On-Call Musculoskeletal Radiographs

Musculoskeletal Imaging
Original Research

On-Call Musculoskeletal
Downloaded from www.ajronline.org by 180.245.204.167 on 11/28/17 from IP address 180.245.204.167. Copyright ARRS. For personal use only; all rights reserved

Radiographs: Discrepancy Rates

Between Radiology Residents and
Musculoskeletal Radiologists
Justin W. Kung1 OBJECTIVE. The purpose of this study was to determine the rate of discrepancy be-
Yulia Melenevsky 2 tween radiology residents and attending musculoskeletal radiologists in interpretation of on-
Mary G. Hochman1 call musculoskeletal radiographs.
Manjiri M. Didolkar 1 MATERIALS AND METHODS. We performed a retrospective review of 2219 con-
Corrie M. Yablon 3 secutive musculoskeletal radiology reports on patients who visited the emergency department
between January 2009 and December 2010. The images were initially interpreted overnight
Ron L. Eisenberg1
by on-call residents (postgraduate years 35), and a final interpretation was rendered the next
Jim S. Wu1 morning by a musculoskeletal radiologist. The reports were evaluated for major discrepan-
Kung JW, Melenevsky Y, Hochman MG, et al. cies, such as missed fractures, osteomyelitis, foreign bodies, tumors, and acute arthritic con-
ditions, which were defined as cases in which a change in clinical management was needed
and required notification of the emergency care provider.
RESULTS. The overall discrepancy rate was 1.8% (40/2219). Fractures accounted for
62.5% (25/40) of missed findings. Fractures involving the upper extremity, particularly the
hand and wrist (2.2% [9/405]), were the most frequently missed. Radial fractures accounted
for 50% (7/14) of the missed upper extremity fractures. Foreign bodies (10% [4/40]) and tu-
morlike lesions (7.5% [3/40]) accounted for the next most common misses. Finally, indepen-
dent resident readings in the on-call setting had little adverse effect on patient care.
CONCLUSION. In the on-call setting, the low discrepancy rate between interpretations of
musculoskeletal radiographs by residents and by musculoskeletal attending radiologists is com-
parable to that reported for other body parts and modalities. Residents should be aware of the
relatively high rate of missed pathologic findings in the upper extremity, especially the radius.

A
t many academic institutions,
Keywords: discrepancy rates, musculoskeletal overnight interpretations of imag-
radiography, residents es are provided by residents, and
a final interpretation is provided
DOI:10.2214/AJR.12.9100
by an attending radiologist the next morning
Received April 18, 2012; accepted after revision [1]. Although independent interpretation is an
June10,2012. important aspect of resident education, provi-
sion of accurate preliminary reports is neces-
1
Department of Radiology, Beth Israel Deaconess sary to ensure high-quality patient care.
Medical Center, 330 Brookline Ave, Boston, MA 02215.
Address correspondence to J. W. Kung
Several studies of the discrepancy rates
(jkung@bidmc.harvard.edu). between residents and attending radiologists
have shown discrepancy rates for major find-
2
Department of Radiology, Georgia Health Sciences ings to be in the range of 0.910% [212].
University, Augusta, GA.
Most of these studies involved CT. Studies
3
Department of Radiology, University of Michigan, evaluating resident interpretation of head
AnnArbor, MI. CT examinations yielded major discrepancy
rates of 1.72.0% [3, 10]. The major discrep-
AJR 2013; 200:856859
ancy rate in the interpretation of body CT
0361803X/13/2004856 examinations was 1.02.3% [2, 9, 12]. One ports obtained in the care of patients with acute
of the largest studies evaluating radiograph-
American Roentgen Ray Society ic, body CT, and neuroradiologic CT exami-
nations showed an overall major discrepancy
rate of 1.0% [11].
Musculoskeletal radiographs can be diffi-
cult to interpret because many fractures are
subtle. Thus the discrepancy rate between res-
idents and attending radiologists may be high.
The purposes of this study were to determine
the rate of discrepancy between radiology res-
idents and attending musculoskeletal radiol-
ogists in interpretation of on-call musculo-
skeletal studies and to identify findings more
likely to be missed. We also evaluated the ef-
fect of the missed findings on patient care.
Materials and Methods
This study was approved by our institutional
review board. We performed a retrospective re-
view of 2219 musculoskeletal radiographic re-
symptoms who presented after hours (weekdays
11 pm7 am, holidays, and weekends) to the emer-

856 AJR:200, April 2013


gency department of a level 1 trauma center. The
images in these cases were initially interpreted by
residents on call, and the final interpretation was
rendered the next morning by a musculoskeletal
radiologist. Imaging was performed between Jan-
uary 2009 and December 2010. At our institution,
Downloaded from www.ajronline.org by 180.245.204.167 on 11/28/17 from IP address 180.245.204.167. Copyright ARRS. For personal use only; all rights reserved
residents take call during postgraduate years 35.
All residents in this study had spent 412 weeks
on the musculoskeletal service before taking call.
In addition, all residents were required to pass an
emergency department examination that includ-
ed commonly encountered musculoskeletal cases.
Musculoskeletal radiographs were defined as ra-
diographs of the appendicular and axial skeleton
interpreted by the musculoskeletal radiology sec-
tion and included cervical spinal and facial radio-
graphs. The images were interpreted by six board-
certified musculoskeletal radiologists with 117
years of experience.
Two radiologists reviewed the final reports
of all studies to determine whether a major dis-
crepancy had occurred. A major discrepancy was
defined as a missed finding that necessitated a
change in clinical management and required no-
tification of the emergency care provider. Dis-
crepancies included missed fractures, osteomyeli-
tis, foreign bodies, tumors, pulmonary nodules or
masses, and acute arthritic conditions. All medi-
cal records in which a major discrepancy was not-
ed were reviewed to determine clinical impor-
tance. The Fisher exact test and chi-square test
were used to compare proportions.
Results
Among the 2219 cases reviewed, a total of
40 (1.8%) major discrepancies were found.
Fractures accounted for 62.5% (25/40) of the
discrepancies. Foreign bodies (10% [4/40])
and tumorlike lesions (7.5% [3/40]) account-
ed for most of the next most common missed
findings (Table 1).
In the 40 cases of major discrepancies,
studies were subcategorized by anatomic lo-
cation (Table 2) to determine whether cer-
tain locations are predisposing factors for a
higher error rate. Among 854 radiographs of
the upper extremity, 19 major discrepancies
(2.2%) were found. Seventeen major discrep-
ancies were found among 1103 radiographs
of the lower extremity (1.5%) and three ma-
jor discrepancies among 243 radiographs
of the spine (1.2%). Miscellaneous radio-
graphs, including rib and facial radiographs,
accounted for one major discrepancy (5.3%).
Fractures accounted for 62.5% of the ma-
jor discrepancies (Table 3). Of these, the high-
est percentage of missed fractures was in the
upper extremity (1.64% [14/854]) as opposed
AJR:200, April 2013 857
On-Call Musculoskeletal Radiographs
TABLE 1: Most Common Discrepancies Between Radiology Resident and
Musculoskeletal Attending Radiologist (n = 40)
Diagnosis No. of Discrepant Cases
Fracture 25 (62.5)
Foreign body 4 (10.0)
Tumorlike lesions 3 (7.5)
Othera 8 (20.0)
NoteValues in parentheses are percentages.
aHip effusion, massive cardiomegaly, gout, particle disease, pulmonary nodule or mass, mortise widening,
atlantoaxial malalignment.
TABLE 2: Major Discrepancies by Location
Major Discrepancy
Location Total No. of Studies No. %
Upper extremity 854 19 2.22
Shoulder and humerus 276 4 1.45
Elbow and forearm 173 3 1.73
Wrist and hand 405 12 2.96
Lower extremity 1103 17 1.54
Pelvis, hip, femur 327 6 1.83
Knee and tibia 388 3 0.77
Ankle and foot 388 8 2.06
Spine 243 3 1.23
Other 19 1 5.26
Total 2219 40
to the lower extremity (0.82% [9/1103]), spine the clinical importance of the missed find-
(0.41% [1/243]), and elsewhere. No signifi- ings. In 20% (8/40) of the cases, the find-
cant difference in discrepancy rate in these ings were recognized independently by the
general fracture locations (p = 0.07) was emergency department or consulting ortho-
found. In the upper extremity, 50% (7/14) of pedic physician. These included six cases of
missed fractures involved the radius. All four fracture and two cases of foreign body. In an-
discrepant interpretations of wrist radiographs other 20% (8/40) of cases, the patients were
were missed fractures of the distal radius (Fig. notified of the injury through the quality as-
1). All three discrepant interpretations of el- surance nurse and were asked to return for
bow radiographs were missed fractures of the treatment but were lost to follow-up. Twenty
radial head. Most of the remaining missed up- of the other 24 patients were admitted to the
per extremity fractures (36%) were fractures hospital from the emergency department or
of the hand. Triquetral fractures accounted for given follow-up appointments with orthope-
40% (2/5) of the discrepant interpretations on dic surgery before discharge. The other four
hand radiographs. In neither case was a wrist patients were discharged from the emergen-
radiograph obtained. In chi-square analysis cy department without treatment despite im-
of location by subgroup, only fractures of the aging evidence of traumatic injury that was
hand and wrist were missed more frequently missed overnight by the radiology resident.
than the rest (p = 0.021). These patients had an orbital floor fracture,
After fractures, foreign bodies accounted a triquetral fracture, a femoral head fracture,
for the next highest percentage of discrepant and C1C2 offset. In the case of the orbital
interpretations (10%). In most cases the for- floor fracture, a CT scan confirmed the ra-
eign body identified by the attending radiolo- diographic finding, and the patient under-
gist was faintly radiopaque and was most of- went surgical fixation 14 days after the initial
ten glass (Fig. 2). presentation. The patient with the triquetral
The medical records of patients with a ma- fracture was recalled 2 days after the initial
jor discrepancy were reviewed to determine presentation, and a splint was applied. In the
 Kung et al.

TABLE 3: Missed Fractures by Anatomic Location counted for the majority of missed elbow
fractures. Williams et al. [15] retrospective-
Major Discrepancy
ly reviewed accident and emergency depart-
Location Total No. of Studies No. % ment images and determined the most fre-
Upper extremity 854 14 1.64 quently missed fractures occurred in the
ankle, finger, and elbow. Our results suggest
Downloaded from www.ajronline.org by 180.245.204.167 on 11/28/17 from IP address 180.245.204.167. Copyright ARRS. For personal use only; all rights reserved

Hand and wrist 405 9 2.22

that radiology residents should pay particular
Distal radius (4) attention to radiographs of the upper extrem-
Triquetrum (2) ity, particularly those of the wrist and hand,
Phalanx (2) and should carefully evaluate the distal radi-
us and radial head.
Metacarpal (1)
Awareness of the findings on musculoskel-
Elbow and forearm 173 3 1.73 etal radiographs most frequently missed by ra-
Radial head (3) diology residents is clinically important. Itri et
Shoulder and humerus 276 2 0.72 al. [16] evaluated the effect of focused missed-
case conferences on the rate of resident miss-
Clavicle (1)
Humerus (1)
Lower extremity 1103 9 0.82
Pelvis, hip, femur 327 3 0.92
Femur (3)
Knee and tibia 388 1 0.26
Patella (1)
Ankle and foot 388 5 1.29
Fibula (1)
Tibia (1)
Talus (1)
Metatarsal (2)
Spine 243 1 0.41
Other 19 1 5.26
Total 2219 25
NoteValues in parentheses are numbers of discrepancies.

case of the femoral head fracture, the nurs-
ing home was notified, and orthopedic fol-
low-up at another institution was arranged.
In the case of C1C2 offset, the patient was
recalled the day after the initial interpreta-
tion, and follow-up CT confirmed the find-
ing. The patient was subsequently cleared by
the neurosurgery department.
Although the interpretation by the muscu-
loskeletal attending radiologist was used as
the reference standard for all 40 cases of ma-
jor discrepancy, in one case of radial head
fracture, the final interpretation was not con-
firmed with follow-up studies.
Discussion
Our major discrepancy rate of 1.8% is sim-
ilar to that in numerous other studies. Cooper
et al. [11] and Ruchman et al. [5] evaluated
after-hours multimodality examinations in-
terpreted by radiology residents and found
overall major discrepancy rates of 1.0% and
2.6%. Cooper et al. further reported a 0.7% Fig. 127-year-old man with distal radial fracture
major discrepancy rate in interpretation of (arrow). Radiograph shows finding missed by
resident.
radiographs. Mann and Danz [13] evaluated
26,421 after-hours radiographs and found a
discrepancy rate of 1.1%. Weschler et al. [9],
Carney et al. [2], and Yoon et al. [12] evalu-
ated major discrepancy rates in the interpre-
tation of body CT scans and found them to be
1.2%, 1.0%, and 2.3%.
We found that the most frequently missed
fractures were present on radiographs of the
upper extremity, particularly radiographs
of the hand and wrist. Radial fractures ac-
counted for one half of the overall missed
findings in the upper extremity. Our find-
ings are in agreement with those of previous
studies. Freed and Shields [14] investigated
the fractures most frequently overlooked by
emergency department physicians and deter-
mined that fractures of the ribs, elbow, and Fig. 226-year-old woman with triangular glass
fragment (arrow) in plantar soft tissues. Radiograph
finger were missed disproportionately often. shows finding missed by resident and confirmed at
In that study, fractures of the radial head ac- removal of the fragment.

858 AJR:200, April 2013


es on musculoskeletal imaging studies. Before
the missed-case conferences, which focused
on musculoskeletal injuries, Itri et al. reported
a miss rate of 1.19%. After the conferences,
there was a significant decrease in the percent-
age of misses by residents (0.87%).
Downloaded from www.ajronline.org by 180.245.204.167 on 11/28/17 from IP address 180.245.204.167. Copyright ARRS. For personal use only; all rights reserved
Misinterpretation of a radiograph by the
on-call resident led to a substantial change in
therapy only infrequently. In 8 of the 40 cases
(20%) interpreted incorrectly by radiology res-
idents, the findings were made independently
by orthopedic surgeons or emergency medi-
cine physicians. In 20 of the 40 cases (50%),
the patients were referred to orthopedic clinic
for follow-up or admitted as inpatients; thus
treatment was not affected. In only two cases
(5.0%) of which we are aware did a discrepant
interpretation lead to a substantial change in
clinical management: the missed orbital floor
fracture and the missed femoral head fracture.
Some limitations of our study deserve
mention. Although our section works hard
to standardize practice patterns to reduce in-
terobserver variability, the radiographs were
interpreted by six musculoskeletal radiolo-
gists, which might have affected our results.
Rhea et al. [17] found that in residents inter-
pretations of emergency radiographs, the res-
idents were correct in nearly 20% of the cases
in which the finding was changed by the staff.
Another limitation was that because our hos-
pital is close to a large pediatric hospital, the
radiographs evaluated were predominantly
those of adults. Finally, we did not encounter
a case in which the attending musculoskele-
tal radiologist noted a false-positive finding
and contacted the emergency care provider.
We hypothesize that in the case of equivocal
fractures, rather than noting a discrepancy,
our musculoskeletal radiologists prefer that
a patient undergo orthopedic surgical follow-
up with subsequent radiography or cross-sec-
tional imaging.
AJR:200, April 2013 859
On-Call Musculoskeletal Radiographs
Conclusion
We found a high rate of agreement between
resident radiologists and attending musculo-
skeletal radiologists in interpretation of emer-
gency musculoskeletal radiographs, the dis-
crepancy rate being only 1.8%. Little adverse
effect on patient care was associated with hav-
ing residents interpret images after hours. Res-
idents should be particularly aware of the rela-
tively high rate of missed pathologic findings
in the upper extremity, especially in the radius.
Acknowledgment
We thank Alexander Brook, PhD for as-
sistance with the statistical analysis.
References
1. Hunter TB, Krupinski EA, Hunt KR, Erly WK.
Emergency department coverage by academic depart-
ments of radiology. Acad Radiol 2000; 7:165170
2. Carney E, Kempf J, DeCarvalho V, Yudd A, Nosher
J. Preliminary interpretations of after-hours CT and
sonography by radiology residents versus final in-
terpretations by body imaging radiologists at a level
1 trauma center. AJR 2003; 181:367373
3. Erly WK, Berger WG, Krupinski E, Seeger JF,
Guisto JA. Radiology resident evaluation of head
CT scan orders in the emergency department.
AJNR 2002; 23:103107
4. Roszler MH, McCarroll KA, Rashid T, Donovan
KR, Kling GA. Resident interpretation of emer-
gency computed tomographic scans. Invest Radi-
ol 1991; 26:374376
5. Ruchman RB, Jaeger J, Wiggins EF 3rd, et al. Pre-
liminary radiology resident interpretations versus
final attending radiologist interpretations and the
impact on patient care in a community hospital.
AJR 2007; 189:523526
6. Stewart NR, Mann FA, Terrell CB, Murphy WA
Jr. Quality assessment in an ambulatory care unit:
do training and responsibility conflict? Radiology
1991; 181:857861
7. Strub WM, Vagal AA, Tomsick T, Moulton JS.
Overnight resident preliminary interpretations on
CT examinations: should the process continue?
Emerg Radiol 2006; 13:1923
8. Tieng N, Grinberg D, Li SF. Discrepancies in in-
terpretation of ED body computed tomographic
scans by radiology residents. Am J Emerg Med
2007; 25:4548
9. Wechsler RJ, Spettell CM, Kurtz AB, et al. Ef-
fects of training and experience in interpretation
of emergency body CT scans. Radiology 1996;
199:717720
10. Wysoki MG, Nassar CJ, Koenigsberg RA, Novel-
line RA, Faro SH, Faerber EN. Head trauma: CT
scan interpretation by radiology residents versus
staff radiologists. Radiology 1998; 208:125128
11. Cooper VF, Goodhartz LA, Nemcek AA Jr, Ryu
RK. Radiology resident interpretations of on-call
imaging studies: the incidence of major discrepan-
cies. Acad Radiol 2008; 15:11981204
12. Yoon LS, Haims AH, Brink JA, Rabinovici R,
Forman HP. Evaluation of an emergency radiolo-
gy quality assurance program at a level I trauma
center: abdominal and pelvic CT studies. Radiol-
ogy 2002; 224:4246
13. Mann FA, Danz PL. The night stalker effect:
quality improvements with a dedicated night-call
rotation. Invest Radiol 1993; 28:9296
14. Freed HA, Shields NN. Most frequently overlooked
radiographically apparent fractures in a teaching
hospital emergency department. Ann Emerg Med
1984; 13:900904
15. Williams SM, Connelly DJ, Wadsworth S, Wilson
DJ. Radiological review of accident and emergen-
cy radiographs: a 1-year audit. Clin Radiol 2000;
55:861865
16. Itri JN, Kang HC, Krishnan S, Nathan D, Scanlon
MH. Using focused missed-case conferences to
reduce discrepancies in musculoskeletal studies
interpreted by residents on call. AJR 2011;
197:928; [web]W696W705
17. Rhea JT, Potsaid MS, DeLuca SA. Errors of interpre-
tation as elicited by a quality audit of an emergency
radiology facility. Radiology 1979; 132:277280