Development and Validation of a Prediction Model for Prehospital Triage for Trauma Original Investigation Research

method for updating prediction models. Stat Med. systems. J Am Coll Surg. 2003;197(5):717-725. doi:
2017;36(28):4529-4539. doi:10.1002/sim.7179
23. Janssen KJM, Moons KGM, Kalkman CJ,
Grobbee DE, Vergouwe Y. Updating methods
improved the performance of a clinical prediction
model in new patients. J Clin Epidemiol. 2008;61(1):
76-86. doi:10.1016/j.jclinepi.2007.04.018
24. R Development Core Team. R: a language and
environment for statistical computing [serial
online]. Vienna, Austria: R Foundation for Statistical
Computing; 2016. https://www.r-project.org/about.
html. Accessed October 31, 2018.
25. van Laarhoven JJ, Lansink KW, van Heijl M,
Lichtveld RA, Leenen LP. Accuracy of the field triage
protocol in selecting severely injured patients after
high energy trauma. Injury. 2014;45(5):869-873.
doi:10.1016/j.injury.2013.12.010
26. Hamada SR, Gauss T, Duchateau FX, et al.
Evaluation of the performance of French
physician-staffed emergency medical service in the
triage of major trauma patients. J Trauma Acute
Care Surg. 2014;76(6):1476-1483. doi:10.1097/TA.
0000000000000239
27. Dinh MM, Bein KJ, Oliver M, Veillard AS, Ivers R.
Refining the trauma triage algorithm at an
Australian major trauma centre: derivation and
internal validation of a triage risk score. Eur J
Trauma Emerg Surg. 2014;40(1):67-74. doi:10.1007/
s00068-013-0315-1
28. Rubenson Wahlin R, Ponzer S, Skrifvars MB,
Lossius HM, Castrén M. Effect of an organizational
change in a prehospital trauma care protocol and
trauma transport directive in a large urban city:
a before and after study. Scand J Trauma Resusc
Emerg Med. 2016;24:26. doi:10.1186/s13049-016-
0218-3
29. Champion HR, Sacco WJ, Copes WS, Gann DS,
Gennarelli TA, Flanagan ME. A revision of the
Trauma Score. J Trauma. 1989;29(5):623-629. doi:
10.1097/00005373-198905000-00017
30. Ocak G, Sturms LM, Hoogeveen JM, Le Cessie
S, Jukema GN. Prehospital identification of major
trauma patients. Langenbecks Arch Surg. 2009;394
(2):285-292. doi:10.1007/s00423-008-0340-4
31. Vassar MJ, Holcroft JJ, Knudson MM, Kizer KW.
Fractures in access to and assessment of trauma
Epidemiol. 2009;62(1):5-12. doi:10.1016/j.jclinepi.
10.1016/S1072-7515(03)00749-X
32. Chang DC, Bass RR, Cornwell EE, Mackenzie EJ.
Undertriage of elderly trauma patients to
state-designated trauma centers. Arch Surg. 2008;
143(8):776-781. doi:10.1001/archsurg.143.8.776
33. Ichwan B, Darbha S, Shah MN, et al.
Geriatric-specific triage criteria are more sensitive
than standard adult criteria in identifying need for
trauma center care in injured older adults. Ann
Emerg Med. 2015;65(1):92-100.e3. doi:10.1016/j.
annemergmed.2014.04.019
34. Teasdale G, Maas A, Lecky F, Manley G,
Stocchetti N, Murray G. The Glasgow Coma Scale at
40 years: standing the test of time. Lancet Neurol.
2014;13(8):844-854. doi:10.1016/S1474-4422(14)
70120-6
35. Wong TH, Krishnaswamy G, Nadkarni NV, et al.
Combining the new injury severity score with an
anatomical polytrauma injury variable predicts
mortality better than the new Injury Severity Score
and the Injury Severity Score: a retrospective
cohort study. Scand J Trauma Resusc Emerg Med.
2016;24:25. doi:10.1186/s13049-016-0215-6
36. Lefering R, Huber-Wagner S, Nienaber U,
Maegele M, Bouillon B. Update of the trauma risk
adjustment model of the TraumaRegister DGU™:
the Revised Injury Severity Classification, version II.
Crit Care. 2014;18(5):476. doi:10.1186/s13054-014-
0476-2
37. Scheetz LJ. Comparison of type and severity of
major injuries among undertriaged and correctly
triaged older patients. J Emerg Med. 2012;43(6):
1020-1028. doi:10.1016/j.jemermed.2011.09.036
38. Schoell SL, Doud AN, Weaver AA, et al.
Development of a time sensitivity score for
frequently occurring motor vehicle crash injuries.
J Am Coll Surg. 2015;220(3):305-312.e3. doi:10.
1016/j.jamcollsurg.2014.11.022
39. Cole TB. Global road safety crisis remedy
sought: 1.2 million killed, 50 million injured annually.
JAMA. 2004;291(21):2531-2532. doi:10.1001/jama.
291.21.2531
40. Leeflang MM, Bossuyt PM, Irwig L. Diagnostic
test accuracy may vary with prevalence:
implications for evidence-based diagnosis. J Clin
2008.04.007
41. Barnes J, Hassan A, Cuerden R, Cookson R,
Kohlhofer J. Comparison of injury severity between
AIS 2005 and AIS 1990 in a large injury database.
Ann Adv Automot Med. 2009;53:83-89.
42. American College of Surgeons Committee on
Trauma. Resources for Optimal Care of the Injured
Patient 2006. Chicago, IL: American College of
Surgeons; 2006.
43. Hedges JR, Feero S, Moore B, Haver DW, Shultz
B. Comparison of prehospital trauma triage
instruments in a semirural population. J Emerg Med.
1987;5(3):197-208. doi:10.1016/0736-4679(87)
90179-X
44. Newgard CD, Nelson MJ, Kampp M, et al.
Out-of-hospital decision making and factors
influencing the regional distribution of injured
patients in a trauma system. J Trauma. 2011;70(6):
1345-1353. doi:10.1097/TA.0b013e3182191a1b
45. Emerman CL, Shade B, Kubincanek J. A
comparison of EMT judgment and prehospital
trauma triage instruments. J Trauma. 1991;31(10):
1369-1375. doi:10.1097/00005373-199110000-
00009
46. Fries GR, McCalla G, Levitt MA, Cordova R.
A prospective comparison of paramedic judgment
and the trauma triage rule in the prehospital
setting. Ann Emerg Med. 1994;24(5):885-889. doi:
10.1016/S0196-0644(94)70207-1
47. Newgard CD, Kampp M, Nelson M, et al;
WESTRN Investigators. Deciphering the use and
predictive value of “emergency medical services
provider judgment” in out-of-hospital trauma
triage: a multisite, mixed methods assessment.
J Trauma Acute Care Surg. 2012;72(5):1239-1248.
doi:10.1097/TA.0b013e3182468b51
48. Harmsen AM, Giannakopoulos GF, Moerbeek
PR, Jansma EP, Bonjer HJ, Bloemers FW. The
influence of prehospital time on trauma patients
outcome: a systematic review. Injury. 2015;46(4):
602-609. doi:10.1016/j.injury.2015.01.008

Invited Commentary

Optimizing Prehospital Trauma Triage—A Step Closer?

Jason S. Haukoos, MD, MSc; Eric M. Campion, MD; Peter T. Pons, MD

Identifying patients with severe injuries in the prehospital
setting remains the first step in a series of interventions that
aim to reduce trauma-related morbidity and mortality. In
2011, the Centers for Disease Control and Prevention and the
American College of Sur-
Related article page 421
geons Committee on Trauma
partnered to revise and up-
date the Field Triage Decision Scheme (FTDS) with the goal of
providing a structure to decision making by paramedics when
determining appropriate destinations for patients with
injuries.1 Recent research suggests that the sensitivity of the
FTDS is lower than previously described, particularly for vul-
nerable populations (eg, elderly people), and lower than the
target of 95% (ie, 5% undertriage rate).2-4
In this issue of JAMA Surgery, van Rein and colleagues5
report the results of a large-scale observational study with
the goal of developing an instrument to improve field
trauma triage. The authors evaluated 43 candidate variables
using severe injury (ie, an injury severity score of more than
15) as the outcome. The final model included age, systolic
blood pressure, Glasgow Coma Scale score, injury mecha-
nism, and injury characteristics (ie, a penetrating injury to
the head or torso, signs or symptoms of the head or neck, or
expected injury to the thorax or to ≥2 body regions) with a
reported sensitivity of 88.8% and specificity of 50% from
the derivation cohort. A comparable calibration and dis-
crimination of the model was reported for a separate exter-
nal cohort.

jamasurgery.com (Reprinted) JAMA Surgery May 2019 Volume 154, Number 5 429

© 2019 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ by a Universidad Del Rosario User on 10/30/2019

 Research Original Investigation Development and Validation of a Prediction Model for Prehospital Triage for Trauma

While laudable, developing a highly sensitive tool with-
out sacrificing specificity is challenging, and this study
must be assessed in the context of several limitations. While
the authors report a comparable model performance
between the derivation and validation portions of this
study, the sensitivity of the tool remains unclear when
applied to an external cohort. Moreover, while the reported
sensitivity improved the baseline rate of undertriage
(78.4%) using a triage scheme specific to the Netherlands, it
is only marginally improved compared with a recent
population-based evaluation of the FTDS and still falls short
of the targeted 95% threshold.2
Using injury severity as an outcome is common but its sole
use constrains the interpretation of results. Approaches to
trauma triage should also be assessed by using the need for
emergent intervention and critical resource use.6,7 Further, the
investigators appeared to use an adult population for whom
emergent return to the hospital occurred, which may have
excluded patients with more occult injuries; among those in-
cluded in the validation cohort, only 165 (2.4%) had an injury
severity score of more than 15, which appears lower than the
percentage reported from other population-based trauma tri-
age studies.2 Unfortunately, this potential for selection may
have artificially increased the tool’s specificity by excluding
patients who did not meet the triage criteria.
Using prehospital data also provides a unique challenge in
that they are collected during a short window when caring for
patients with acute injuries or illness and it often results in large
numbers of missingness, as seen in this study. While the au-
thors used quality missing data methods (ie, multiple imputa-
tion), they assumed the data were missing at random, which is
an important assumption of this approach. Unfortunately, this
assumption is often violated, especially when data are col-
lected from patients with acute injuries. Although using mul-
tiple imputation in this context is reasonable, the results must
be interpreted with caution.
In summary, it remains unclear how well this tool will
perform when applied prospectively as a part of routine care.
This new tool, while possibly a step closer, must be inte-
grated into prehospital care and rigorously evaluated among
a broader, unselected trauma population (including children)
and shown to be at least equivalent when directly compared
with current standards, including the FTDS. Ultimately,
large-scale, pragmatic comparative effectiveness and imple-
mentation research is needed to understand how best to
accurately identify patients who require specialized trauma
care and we may have reached the limits of field triage using
common prehospital data inputs. Novel physiologic mea-
surement or diagnostic technologies may now be required to
further improve field triage accuracy.

ARTICLE INFORMATION
Author Affiliations: Department of Emergency
Medicine, Denver Health Medical Center, Denver,
Colorado (Haukoos, Pons); Division of Paramedic,
Denver Health Medical Center, Denver, Colorado
(Haukoos, Pons); Department of Emergency
Medicine, University of Colorado School of
Medicine, Aurora (Haukoos, Pons); Department of
Epidemiology, Colorado School of Public Health,
Aurora (Haukoos); Department of Surgery, Denver
Health Medical Center, Denver, Colorado
(Campion); Department of Surgery, University of
Colorado School of Medicine, Aurora (Campion).
Corresponding Author: Jason S. Haukoos, MD,
MSc, Department of Emergency Medicine, Denver
Health Medical Center, 777 Bannock St, Mail Code
0108, Denver, CO 80204 (jason.haukoos@dhha.
org).
Published Online: February 6, 2019.
doi:10.1001/jamasurg.2018.4764
Conflict of Interest Disclosures: Dr Haukoos is
funded in part by grants R01AI106057 and
R01DA042982 from the National Institutes of
Health. No other disclosures are reported.
REFERENCES
1. Sasser SM, Hunt RC, Faul M, et al; Centers for
Disease Control and Prevention (CDC). Guidelines
for field triage of injured patients:
recommendations of the National Expert Panel on
Field Triage, 2011. MMWR Recomm Rep. 2012;61(RR-
1):1-20.
2. Newgard CD, Zive D, Holmes JF, et al; WESTRN
investigators. A multisite assessment of the
American College of Surgeons Committee on
Trauma field triage decision scheme for identifying
seriously injured children and adults. J Am Coll Surg.
2011;213(6):709-721. doi:10.1016/j.jamcollsurg.2011.
09.012
3. Newgard CD, Fu R, Zive D, et al. Prospective
validation of the National Field Triage Guidelines for
Identifying Seriously Injured Persons. J Am Coll Surg.
2016;222(2):146-58.e2. doi:10.1016/j.jamcollsurg.
2015.10.016
4. Newgard CD, Richardson D, Holmes JF, et al;
Western Emergency Services Translational
Research Network (WESTRN) Investigators.
Physiologic field triage criteria for identifying
seriously injured older adults. Prehosp Emerg Care.
2014;18(4):461-470. doi:10.3109/10903127.2014.
912707
5. van Rein EAJ, van der Shuijs R, Voskens FJ, et al.
Development and validation of a prediction model
for prehospital triage of trauma patients [published
online February 6, 2019]. JAMA Surg. doi:10.1001/
jamasurg.2018.4752
6. Boatright DH, Byyny RL, Hopkins E, et al.
Validation of rules to predict emergent surgical
intervention in pediatric trauma patients. J Am Coll
Surg. 2013;216(6):1094-1102, 1102.e1-1102.e6. doi:
10.1016/j.jamcollsurg.2013.02.013
7. Haukoos JS, Byyny RL, Erickson C, et al.
Validation and refinement of a rule to predict
emergency intervention in adult trauma patients.
Ann Emerg Med. 2011;58(2):164-171. doi:10.1016/j.
annemergmed.2011.02.027

430 JAMA Surgery May 2019 Volume 154, Number 5 (Reprinted) jamasurgery.com

© 2019 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ by a Universidad Del Rosario User on 10/30/2019