 TRAUMA

An interobserver reliability comparison

between the Orthopaedic Trauma
Association’s open fracture classification and
the Gustilo and Anderson classification
A. Ghoshal, Aims
N. Enninghorst, To evaluate interobserver reliability of the Orthopaedic Trauma Association’s open fracture
K. Sisak, classification system (OTA-OFC).
Z. J. Balogh
Patients and Methods
From John Hunter Patients of any age with a first presentation of an open long bone fracture were included.
Hospital, New South Standard radiographs, wound photographs, and a short clinical description were given to
Wales, Australia eight orthopaedic surgeons, who independently evaluated the injury using both the Gustilo
and Anderson (GA) and OTA-OFC classifications. The responses were compared for
variability using Cohen’s kappa.

Results
The overall interobserver agreement was ĸ = 0.44 for the GA classification and ĸ = 0.49 for
OTA-OFC, which reflects moderate agreement (0.41 to 0.60) for both classifications. The
agreement in the five categories of OTA-OFC was: for skin, ĸ = 0.55 (moderate); for muscle,
ĸ = 0.44 (moderate); for arterial injury, ĸ = 0.74 (substantial); for contamination, ĸ = 0.35 (fair);
and for bone loss, ĸ = 0.41 (moderate).

 A. Ghoshal, MBBS, BMedSci,
Orthopaedic Registrar in
Training, Department of
Traumatology, Division of
Surgery
 K. Sisak, MD, PhD, Trauma
Fellow, Department of
Traumatology, Division of
Surgery
John Hunter Hospital, Lookout
Road, New Lambton Heights
NSW 2305, Australia.
 N. Enninghorst, MD, PhD,
Staff Specialist Orthopaedic
Trauma, Department of
Traumatology, Division of
Surgery
 Z. J. Balogh, MD, PhD,
FRACS, Director of Trauma,
Department of Traumatology,
Division of Surgery
John Hunter Hospital and
University of Newcastle,
Lookout Road, New Lambton
Heights NSW 2305, Australia.
Correspondence should be sent
to Z. J. Balogh; email:
Zsolt.Balogh@hnehealth.nsw.go
v.au
©2018 The British Editorial
Society of Bone & Joint
Surgery
doi:10.1302/0301-620X.100B2.
BJJ-2017-0367.R1 $2.00
Bone Joint J
2018;100-B:242–6.
Conclusion
Although the OTA-OFC, with similar interobserver agreement to GA, offers a more detailed
description of open fractures, further development may be needed to make it a reliable and
robust tool.
Cite this article: Bone Joint J 2018;100-B:242–6.
Multiple systems currently exist for classify-
ing open fractures, of which the Gustilo and
Anderson (GA)1 system is probably the most
popular. Since its introduction in 1976,1 it has
been adopted worldwide, not least due to its
use in stratifying the risk of infection based on
the classification. In their original study,
Gustilo et al1 reported infection rates of 44%
in their retrospective series and 9% in their
prospective series for Type III open fractures.
A further study by Gustilo et al2 recom-
mended subdivision of Type III fractures into
three subtypes, in order of worsening progno-
sis, based on the varied severity and prognosis
of Type III open fractures. They reported
infection rates of 0% for Type I fractures,
2.5% for Type II fractures and up to 52% for
Type IIIb and IIIc fractures.1,2,3 Although the
classification originally described only open
tibial fractures, it has since been applied to
other anatomical sites. Despite its clinical use,
the definitions of injury characteristics in the
system are imprecise and can lead to variabil-
ity in interpretation, resulting in suboptimal
reliability and reproducibility.4 As a result,
over time the classification has been modified
without validation, thereby creating even
more inherent variability.
To be scientifically sound and justify wide-
spread use, fracture classifications need to be
reliable, reproducible, responsive, clinically
relevant, and valid. In 2010, the Classification
Committee of the Orthopaedic Trauma Associ-
ation (CCOTA) therefore proposed a new clas-
sification system for open fractures, in order to
overcome some of the shortcomings of the GA
classification.5 The Orthopaedic Trauma Asso-
ciation Open Fracture Classification (OTA-
OFC) comprises five factors (Fig. 1): injury to
skin, muscle or arteries, bone loss, and con-
tamination.5 Each of these may be rated mild,
moderate, or severe.
Jones et al6 suggested the classification may
have a role in the assessment of limb viability

242 THE BONE & JOINT JOURNAL

 INTEROBSERVER RELIABILITY COMPARISON BETWEEN THE OTA’S OPEN FRACTURE CLASSIFICATION AND GA CLASSIFICATION 243

Fig. 1

The Orthopaedic Trauma Association Open Fracture Classification. This figure was first pub-
lished in the following study: Evans AR, Agel J, DeSilva GL, et al. A new classification scheme
for open fractures. J Orthop Trauma 2010;24:457-464. Reproduced with the permission of
Wolters Kluwer Health, Inc.
in open fractures, reporting that an increased severity in each
of the five categories was associated with amputation or
increased disability at two years in salvaged limbs. Agel et al7
studied 356 patients with open fractures who underwent
early amputation, vacuum-assisted closure, more than two
debridements, or placement of antibiotic beads following
an open fracture. They demonstrated a relationship
between the OTA-OFC and the type of treatment used to
treat an open fracture. The strongest predictor of vacuum-
assisted closure use was the severity of skin injury; multiple
debridements (> 2) was best predicted by the severity of
skin and muscle injury; bone loss was the strongest predic-
tor of antibiotic bead placement; and skin injury, contami-
nation, and arterial injury were the strongest predictors of
early amputation. More recently, Hao et al8 demonstrated
the superiority of the OTA-OFC over the GA system in pre-
dicting treatment outcomes of infection, need for soft-tissue
coverage and need for limb amputation in patients with
open long bone fractures. In their cohort of 512 patients,
the skin injury component of the OTA-OFC was shown to
be an independent predictor of limb amputation and a
score ≤ 10 increased the odds of successful limb salvage.
Although the OTA-OFC offers a more detailed descrip-
tion of open fractures, it must be noted that the classifica-
tion was originally developed out of expert consensus
opinion and is yet to be fully validated. Hao et al8 therefore
advised caution in the interpretation of their results. The
study by Jones et al6 only investigated the OTA-OFC in the
lower limb. Agel et al9 also presented only a small number
of cases, mainly in the lower limb (three tibial shaft, one
distal tibia pilon, one distal humerus, and one ankle malle-
olar fracture), and reported limitations with their video
presentation methodology. The videos were recorded at the
time of the initial debridement, six hours post-injury, and
so gave little information on whether the system is an accu-
rate or useful tool for classifying open fractures at the time
of initial presentation to the Emergency Department. A
comparative inclusion of the corresponding GA classifica-
tion score would have been useful. As the main authors
were members of the CCOTA, they were likely to have con-
tributed to the development of the OTA-OFC. This might
have led to some unintended bias in fracture selection and
classification and thereby limit the general applicability of
the OTA-OFC system.
Other novel classification systems have also been pro-
posed previously to overcome the well-reported limitations
of the GA classification. Rajasekaran et al10,11 published
the Ganga hospital score in 2006, reporting superior inter-
and intraobserver reliability for prediction of amputation,
nonunion, and infection when compared with the GA clas-
sification. It failed to gain widespread popularity due to its
limited applicability to only GA IIIa and IIIb open tibial
fractures. Garnavos et al12 claimed improved inter- and
intraobserver agreement over the AO/Müller long bone
fracture classification system13 using the new Garnavos
Classification System. Although the system proved simple,
reliable, and memorable for classification of tibial frac-
tures, no comparison with the GA system is available in the
published literature and its use in other long-bone fractures
remains unclear.
The aim of this study is to evaluate interobserver reliabil-
ity of the OTA-OFC, when used by orthopaedic surgeons of
varying levels of experience.
Patients and Methods
Our 18-month prospective observational study was con-
ducted at a Level I Trauma Centre between January 2011
and July 2012 and had ethical approval from the local
review board. Patients of any age with a first presentation
of an open long bone fracture were included.
Initial assessment was performed in the Emergency
Department and routine treatment commenced, including
intravenous antibiotics and tetanus prophylaxis. Details of
patient demographics, the mechanism of injury and the
neurovascular status of the limb were documented. Routine
digital radiographs of the fractured limb were obtained. A
five-megapixel digital camera (Olympus Australia Pty. Ltd,
Notting Hill, Australia) was used by the investigators to

VOL. 100-B, No. 2, FEBRUARY 2018

244 A. GHOSHAL, N. ENNINGHORST, K. SISAK, Z. J. BALOGH

Fig. 2

Interobserver reliability (Cohen’s kappa) for the five categories and three levels of severity in the Ortho-
paedic Trauma Association Open Fracture Classification.

obtain a minimum of two photographs of the wound from
different angles. The fracture was then irrigated with saline
and clean dressings and an appropriate splint applied. The
patient was then admitted under the care of the on-call
orthopaedic surgeon for further management.
The investigators selected the two photographs that best
highlighted the characteristics of the wound for each
patient. The radiographs and photographs, along with the
study number, the patient’s age, the patient’s gender, and a
short description of the mechanism of injury and limb neu-
rovascular status, were presented in a standardized format
to eight orthopaedic trauma surgeons: four consultants,
two senior training registrars, and two junior registrars.
None of the surgeons were affiliated with the OTA. All
eight participants had previous experience in using the GA
classification and none had previously used the OTA-OFC
system. Copies of the GA and OTA-OFC classifications
were provided for reference. Treatment information, post-
operative radiographs, and details of final wound closure
or coverage were withheld. Each surgeon was requested to
evaluate independently the injury according to the GA and
OTA-OFC classification systems.
The responses were compared for variability using kappa
coefficients, as described by Cohen, generating values rang-
ing from -1 (no agreement) to 1 (complete agreement), with
0 representing agreement equivalent to that obtainable by
chance alone.14 Interpretation of the kappa values was per-
formed according to the guidelines of Landis and Koch,15
with kappa values > 0.80 considered excellent, 0.79 to 0.60
substantial, 0.59 to 0.40 moderate, 0.39 to 0.20 slight, and
< 0.20 poor.
In total, 91 open long bone fractures in 87 consecutive
patients aged 43.01 years (7 to 90, standard deviation 22)
were included, consisting of seven of the humerus, 22 of the
radius and/or ulna, 15 of the femur and 47 tibial fractures.
In all, 57 patients were male and 30 female.
Results
Mechanism of injury varied from low-force trauma and falls,
to high-energy injuries caused by road traffic accidents. The
overall interobserver agreement was ĸ = 0.44 for the GA clas-
sification and ĸ = 0.49 for the OTA-OFC, reflecting moder-
ate agreement (0.41 to 0.60) for both classifications. The
agreement in the five categories of OTA-OFC was: for skin,
ĸ = 0.55 (moderate); for muscle, ĸ = 0.44 (moderate); for
arterial injury, ĸ = 0.74 (substantial); for contamination,
ĸ = 0.35 (fair); and for bone loss, ĸ = 0.41 (moderate). The
senior and junior surgeons differed only in the bone loss cat-
egory (ĸ = 0.52 (moderate) vs ĸ = 0.33 (fair)). All other cate-
gories had similar kappa scores (Fig. 2).

Follow us @BoneJointJ THE BONE & JOINT JOURNAL

 INTEROBSERVER RELIABILITY COMPARISON BETWEEN THE OTA’S OPEN FRACTURE CLASSIFICATION AND GA CLASSIFICATION
Discussion
The ability of orthopaedic surgeons to use classification
systems reliably and reproducibly has already been shown
to be variable.16-20 Our study showed only moderate inter-
observer agreement between surgeons using both classifica-
tions systems. The OTA-OFC does not, therefore, represent
an absolute improvement in agreement over the GA classi-
fication, but its similar agreement is notable in the context
of its increased complexity.
The level of agreement using the GA system seen in this
study was like that reported by Brumback and Jones,4 who
demonstrated a mean interobserver agreement of 60%. Mod-
erate interobserver agreement levels were also observed by
Horn and Rettig’s21 retrospective analysis of postoperative
photographs.
The OTA-OFC skin, muscle, and bone loss categories all
demonstrated moderate levels of agreement, with most dis-
agreement occurring in the contamination category. The
substantial agreement seen in the arterial injury category
was expected due to the detail of information given in the
clinical history. This contrasts with the results reported by
Agel et al,9 who demonstrated moderate to excellent levels
of agreement across all five categories.
The strengths of the study were its prospective nature,
large number of open fractures from different anatomical
sites, varying mechanisms of injury, diversity of age, bal-
ance of male and female patients, and the variable levels of
experience of the rating surgeons. High-quality orthogonal
digital photographs were used to capture detail of the open
fractures. The fractures were also classified at the time of
presentation to the Emergency Department and not during
the first operative debridement. We acknowledge that
although this is not the validated means of using the GA
system, it is what is commonly practiced across many cen-
tres worldwide. This helped recreate a more realistic picture
of everyday clinical practice when using both these classifi-
cations. Classification bias was minimized through the par-
ticipation of surgeons not affiliated with the OTA. A
prospective database was also developed specifically for
this study (including details of patient name, date of birth,
hospital identification number, age, gender, mechanism of
injury, fracture radiographs, photographs of wound, and
open fracture classification according to both the GA &
OTA-OFC systems), avoiding retrospective notes review to
ascertain key clinical information. To the authors’ knowl-
edge, this study is also the only one in English literature to
have used both the GA and OTA-OFC systems to classify
prospectively a large number of open fractures in a variety
of anatomical sites and compare the interobserver reliabil-
ity between both systems.
Limitations of the study included the absence of an
intraobserver analysis and a subgroup analysis by surgeon
grade. The scoring surgeons were also not involved in the
initial debridement and could therefore not assess the
extent of muscle damage and contamination first-hand.
VOL. 100-B, No. 2, FEBRUARY 2018
245
The involvement of plastic surgeons in fracture classifica-
tion may also have been beneficial.
In conclusion, the new proposed OTA-OFC showed
moderate agreement between eight surgeons, regardless of
experience, when used to classify 91 open fractures at dif-
ferent anatomical sites. This does not represent an improve-
ment over the widely used GA classification. Although the
OTA-OFC offers a more detailed description of open frac-
tures with similar interobserver agreement, further
development may be needed to make it a reliable and
robust tool.
Take home message:
- This paper gives a reliable independent validation of the
OTA-OFC against the most commonly used system.
- It will help orthopaedic surgeons to understand the strengths and limi-
tations of the OTA-OFC classification.
- It may also facilitate improvement of the OTA-OFC classification.
References
1. Gustilo RB, Anderson JT. Prevention of infection in the treatment of one thousand
and twenty-five open fractures of long bones: retrospective and prospective analyses.
J Bone Joint Surg [Am] 1976;58-A:453–458.
2. Gustilo RB, Mendoza RM, Williams DN. Problems in the management of type III
(severe) open fractures: a new classification of type III open fractures. J Trauma
1984;24:742–746.
3. Gustilo RB, Gruninger RP, Davis T. Classification of type III (severe) open fractures
relative to treatment and results. Orthopedics 1987;10:1781–1788.
4. Brumback RJ, Jones AL. Interobserver agreement in the classification of open
fractures of the tibia. The results of a survey of two hundred and forty-five orthopae-
dic surgeons. J Bone Joint Surg [Am] 1994;76-A:1162–1166.
5. Evans AR, Agel J, DeSilva GL, et al. A new classification scheme for open frac-
tures. J Orthop Trauma 2010;24:457–464.
6. Jones CB, Castillo RC, Carlini AR, et al. Validation of the OTA Open Fracture Clas-
sification with data from a prospective cohort study of limb-threatening tibia frac-
tures. Paper presentation #117 at 28th annual meeting of the Orthopaedic Trauma
Association, October 3-6, 3012, Minneapolis, MN, 2012. http://ota.org/education/
search-abstracts/#|ota12|otapa|OTA1209117.htm (date last accessed 14 November
2017). (Abstract).
7. Agel J, Rockwood T, Barber R, Lawrence Marsh J. Potential predictive ability of
the Orthopaedic Trauma Association open fracture classification. J Orthop Trauma
2014;28:300–306.
8. Hao J, Cuellar DO, Herbert B, et al. Does the OTA Open Fracture Classification pre-
dict the need for limb amputation? A retrospective observational cohort study on 512
patients. J Orthop Trauma 2016;30:194–198.
9. Agel J, Evans AR, Marsh JL, et al. The OTA open fracture classification: a study of
reliability and agreement. J Orthop Trauma 2013;27:379–384.
10. Rajasekaran S, Naresh Babu J, Dheenadhayalan J, et al. A score for predicting
salvage and outcome in Gustilo type-IIIA and type-IIIB open tibial fractures. J Bone
Joint Surg [Br] 2006;88-B:1351–1360.
11. Rajasekaran S, Sabapathy SR, Dheenadhayalan J, et al. Ganga hospital open
injury score in management of open injuries. Eur J Trauma Emerg Surg 2015;41:3–15.
12. Garnavos C, Kanakaris NK, Lasanianos NG, Tzortzi P, West RM. New classifi-
cation system for long-bone fractures supplementing the AO/OTA classification.
Orthopedics 2012;35:709–719.
13. Kellam JF, Audigé L. Fracture classification. In: Rüedi TP, Buckley RE, Moran CG,
eds. AO Principles of Fracture Management. Second ed. Stuttgart and New York:
Thieme, 2007;69–85.
14. Cohen JA. A coefficient of agreement for nominal scales. Educ Psychol Meas
1960;20:37–46.
246 A. GHOSHAL, N. ENNINGHORST, K. SISAK, Z. J. BALOGH
15. Landis JR, Koch GG. The measurement of observer agreement for categorical data.
Biometrics 1977;33:159–174.
16. Burstein AH. Fracture classification systems: do they work and are they useful? J.
Bone and Joint Surg [Am]. 1993;75-A:1743–1744.
17. Frandsen PA, Andersen E, Madsen F, Skjødt T. Garden’s classification of femoral
neck fractures. An assessment of inter-observer variation. J Bone Joint Surg [Br]
1988;70-B:588–590.
18. Papa J, Myerson M, Girard P. Salvage, with arthrodesis, in intractable diabetic
neuropathic arthropathy of the foot and ankle. J Bone Joint Surg [Am] 1993;75-
A:1056–1066.
19. Shaw WW, Breidenbach WC, Jupiter JB, Weiland AJ. Symposium: manage-
ment of type III extremity fractures. Contemp Orthop 1992;25:183–210.
20. Thomsen NOB, Overgaard S, Olsen LH, Hansen H, Nielsen ST. Observer varia-
tion in the radiographic classification of ankle fractures. J Bone Joint Surg [Br]
1991;73-B:676–678.
21. Horn BD, Rettig ME. Interobserver reliability in the Gustilo and Anderson classifica-
tion of open fractures. J Orthop Trauma 1993;7:357–360.
Follow us @BoneJointJ
Author contributions:
A. Ghoshal: Collecting and analyzing the data, Writing the paper, Classifica-
tion of open fractures.
K. Sisak: Editing the paper, Classification of open fractures.
N. Enninghorst: Collecting and analyzing the data, Writing and editing the
paper, Classification of open fractures.
Z. J. Balogh: Editing the paper, Classification of open fractures.
Acknowledgements:
The authors would like to thank the following investigators for their partici-
pation in the classification of open fractures: Nerida Butcher (MBBS); Laszlo
Toth (MD, PhD); David Dewar (MBBS, FRACS); Edward Wellings (MBBS,
FRACS).
Funding statement:
No benefits in any form have been received or will be received from a
commercial party related directly or indirectly to the subject of this article.
This article was primary edited by P. Page and first proof edited by G. Scott.
THE BONE & JOINT JOURNAL