State of the Art Review
Otolaryngology–
The State of the Otolaryngology Match:
A Review of Applicant Trends,
‘‘Impossible’’ Qualifications,
and Implications
Sarah N. Bowe, MD1, Cecelia E. Schmalbach, MD2,
and Adrienne M. Laury, MD3
Sponsorships or competing interests that may be relevant to content are dis-
closed at the end of this article.
Abstract
Objective. This State of the Art Review aims (1) to define
recent qualifications of otolaryngology resident applicants by
focusing on United States Medical Licensing Examination
(USMLE) scores, Alpha Omega Alpha (AOA) status, and
research/publications and (2) to summarize the current liter-
ature regarding the relationship between these measures
and performance in residency.
Data Sources. Electronic Residency Application Service, National
Residency Matching Program, PubMed, Ovid, and GoogleScholar.
Review Methods. Electronic Residency Application Service
and National Residency Matching Program data were ana-
lyzed to evaluate trends in applicant numbers and qualifica-
tions. Additionally, a literature search was performed with
the aforementioned databases to identify relevant articles
published in the past 5 years that examined USMLE Step 1
scores, AOA status, and research/publications.
Conclusions. Compared with other highly competitive fields
over the past 3 years, the only specialty with decreasing
applicant numbers is otolaryngology, with the rest remaining
relatively stable or slightly increased. Additionally, USMLE
Step 1 scores, AOA status, and research/publications do not
reliably correlate with performance in residency.
Implications for Practice. The consistent decline in applications
for otolaryngology residency is concerning and reflects a need
for change in the current stereotype of the ‘‘ideal’’ otolaryngol-
ogy applicant. This includes consideration of additional selec-
tion measures focusing on noncognitive and holistic qualities.
Furthermore, otolaryngology faculty should counsel medical
students that applying in otolaryngology is not ‘‘impossible’’ but
rather a feasible and worthwhile endeavor.
Keywords
otolaryngology, match, applicant, medical student, qualifica-
tions, USMLE, AOA, research, publications
Head and Neck Surgery
1–6
Ó American Academy of
Otolaryngology—Head and Neck
Surgery Foundation 2017
Reprints and permission:
sagepub.com/journalsPermissions.nav
DOI: 10.1177/0194599817695804
http://otojournal.org
Received September 29, 2016; revised December 16, 2016; accepted
January 26, 2017.
E
valuation of the Electronic Residency Application
Service (ERAS) match data indicates a decreasing
trend in otolaryngology applicant numbers over the
past 4 years (2014, n = 489; 2015, n = 442; 2016, n = 367;
2017, n = 352).1 At a single institution, approximately 80%
of third- and fourth-year medical students rated matching in
otolaryngology–head and neck surgery as either ‘‘impossi-
ble’’ or ‘‘near impossible.’’2 While an excess of applicants
per slot presently exists, if this rate of decline continues, the
field of otolaryngology will experience a shortfall of appli-
cants within the next 2 years.
Given the medical student opinions and recent match
trends, this State of the Art Review aims (1) to define
recent qualifications of otolaryngology resident applicants
focusing on United States Medical Licensing Examination
(USMLE) scores, Alpha Omega Alpha (AOA) status, and
research/publications and (2) to summarize the current liter-
ature regarding the relationship between these measures and
performance in residency.
1
Department of Otolaryngology–Head and Neck Surgery, Massachusetts
Eye and Ear Infirmary, Boston, Massachusetts, USA
2
Department of Otolaryngology–Head and Neck Surgery, Indiana
University, Indianapolis, Indiana, USA
3
Department of Otolaryngology–Head and Neck Surgery, San Antonio
Uniformed Services Health Education Consortium, Ft Sam Houston, Texas,
USA
The views expressed herein are those of the authors and do not reflect the
official policy or position of Brooke Army Medical Center, the US Army
Medical Department, the US Army Office of the Surgeon General, the
Department of the Army, the Department of Defense, or the US govern-
ment.
Corresponding Author:
Sarah N. Bowe, MD, Department of Otolaryngology–Head and Neck
Surgery, Massachusetts Eye and Ear Infirmary, 243 Charles St, Boston, MA
02114, USA.
Email: DrSarahNBowe@gmail.com
2 Otolaryngology–Head and Neck Surgery
Methods
A computerized PubMed, Ovid, and GoogleScholar database
search of the English literature was conducted with the fol-
lowing search terms: (medical student OR resident OR resi-
dency) AND (criteria OR selection OR match) AND
(success or performance), in combination with or separate to
(USMLE) OR (Alpha Omega Alpha) OR (publications). In
accordance with the State of the Art Review guidelines, the
databases were searched to include articles from January 1,
2010, to December 31, 2015. Reference lists of studied
papers were hand-searched to identify further relevant works.
In brief, the search identified 198 potentially relevant
articles. After accounting for duplicates and papers not
available in English, abstracts were reviewed by 2 authors
(S.N.B., A.M.L.). Sixty-two articles were pulled for full
review. Manuscripts were included if they evaluated any of
3 applicant variables (ie, USMLE, AOA, research), provided
a measurable aspect of resident performance as defined by
the study (eg, faculty ranking, faculty rating, number of
publications, future academic appointment), and presented a
correlation between them. To increase the number of avail-
able studies, articles from any medical or surgical specialty
were included. Additionally, key historical (published prior
to 2010) landmark articles from the field of otolaryngology
were included secondary to the limited number of studies
meeting the traditional inclusion criteria for a State of the
Art Review. Furthermore, manuscripts were excluded if
they duplicated data from another included study or if they
were a review article, editorial, or letter to the editor.
Ultimately, 13 articles fulfilled these criteria, the majority
of which were retrospective cohort studies.2-14
All available ERAS data were reviewed dating back to oto-
laryngology’s initial participation in 2006. In addition, 2007-
2016 National Residency Matching Program otolaryngology
data were analyzed for the years available.15 Information
included number of applicants per year, average USMLE Step 1
score per applicant, percentage of AOA members per applica-
tion cycle, and number of publications per applicant (Table 1).
Institutional Review Board approval was not required for this
study, which did not involve human subjects.
Discussion
United States Medical Licensing Examination
According to the 2016 National Residency Matching
Program data, the mean USMLE Step 1 score for an otolar-
yngology candidate was 248, compared with 233 for all spe-
cialties (Figure 1).15 Review of the literature, however,
does not reveal that achieving such high scores on this
examination guarantees success in residency. Daly et al
were unable to demonstrate that USMLE scores correlated
with otolaryngology faculty rankings of resident success.3
The authors did note a relationship between USMLE score
and .1 peer-reviewed publication and higher year 2 in-
training scores during residency. Furthermore, it is well
documented that USMLE Step 1 scores within the range of
Table 1. Otolaryngology Applicant Trends and Qualifications:
2007-2016.1,3
2007 2009 2011 2014 2016
Applicants (US 1 IMG), n 554 554 490 572 416
USMLE Step 1 scorea 238 240 243 248 248
Alpha Omega Alpha, % 39.0 36.9 41.6 38.9 44.7
Publications, abstracts, presentationsa 4 4.1 5.1 6.1 8.4
Abbreviations: IMG, international medical graduate; USMLE, United States
Medical Licensing Examination.
a
Mean per applicant.
Figure 1. Average United States Medical Licensing Examination
(USMLE) Step 1 scores of applicants in otolaryngology versus all
specialties: 2007-2016.3
200 to 220 imply a very high likelihood of passing board
examinations in many fields, including general surgery,4
anesthesia,5 and obstetrics/gynecology.6 Unfortunately, due
to data limitations, a similar correlation between USMLE
scores and American Board of Otolaryngology pass rates is
not feasible without extensive efforts to acquire individual
information from each residency program.7 This USMLE
Step 1 score average of 248 is publicly available and fre-
quently quoted by prospective applicants. Consequently, the
test sets up a potential self-selection bias whereby medical
students without a USMLE score near this average are dis-
couraged from applying. In addition, otolaryngology pro-
gram directors often assign a USMLE filter as the initial
review for the hundreds of residency applications received.
It is recognized that ‘‘the more competitive the resident dis-
cipline (eg, orthopedic surgery, radiation oncology, derma-
tology, ophthalmology, and otolaryngology) the higher the
USMLE Step 1 score needed to pass through the filter.’’16
Therefore, applicants are essentially being filtered by their
USMLE step scores at 2 levels—first by themselves and
second by the otolaryngology programs. However, this State
of the Art Review failed to identify convincing studies to
support the concept that USMLE scores correlate with resi-
dency success. In contrast, there is evidence to support
Bowe et al
Figure 2. Percentage of applicants with Alpha Omega Alpha status
in otolaryngology versus all specialties: 2007-2016.3
adequate board passage rates with a much lower licensing
examination score. Therefore, by enacting this practice, the
diversity of the otolaryngology applicant pool may be unne-
cessarily reduced by eliminating potentially excellent clini-
cians with a below-average Step 1 score (Figure 1).
Alpha Omega Alpha
Another factor touted as a reliable predictor of a top-
performing resident is election into the AOA Honor
Society. Otolaryngology has an exceptionally high percent-
age of AOA applicants as compared with all other special-
ties. For instance, in 2016, 44.7% of otolaryngology
applicants were AOA members, as opposed to 17.3% in all
specialties (Figure 2). The relationship between AOA
membership and resident performance, however, is not
straightforward. Daly et al found that AOA membership
was highly predictive of a resident ranking in the top third
of his or her otolaryngology class based on faculty assess-
ment. In addition, those individuals had a higher rate of
receiving an academic appointment following residency.3
For orthopedic surgery, Raman et al showed that AOA
membership strongly correlated with high global evaluation
scores.8
However, a number of studies have refuted the value of
AOA status in determining residency success. Alterman et
al showed that AOA membership was not a significant
factor in predicting successful completion of a general sur-
gery residency.4 Additionally, Chole et al found no signifi-
cant correlation between faculty rating scale and AOA
status among otolaryngology residents.9 In a study from the
Mayo Clinic, first-year internal medicine residents were
evaluated with a multisource professionalism assessment,
with results compared with a variety of ERAS application
variables. It reported no correlation between AOA status
and high rankings on this professionalism assessment.10
Selection to AOA is often tightly linked to cognitive
achievements during medical school. As a result, positive
correlation is expected between AOA and cognitive metrics,
such as passage of national board examinations on first
3
Figure 3. Average number of publications, abstracts, and presenta-
tions for applicants applying in otolaryngology versus all specialties:
2007-2016.3
attempt.17,18 However, in terms of a broader picture of resi-
dent performance, there are many more inconsistencies with
the predictive capability of AOA membership.
Research/Publications
‘‘Medical students are getting the message that experience
in research is very important.’’2 Currently, the average
number of abstracts, presentations, and publications by a
medical student matching into otolaryngology is 8.4
(Figure 3). It appears that an increasing number of appli-
cants are taking additional years to accomplish research
initiatives with the hopes of improving their match success.
Specifically, at the Boston University School of Medicine,
the number of students incorporating an extra year of
research into their curriculum increased from 1 in 2005 to
14 in 2014.2 Overall, the data are controversial regarding
whether doing research as a medical student has any rela-
tionship with residency success. In support, Bhat et al iden-
tified that a minimum of 5 publications/presentations
correlated with a resident’s presence in the top one-third of
her or his emergency medicine class.11
Alternatively, several studies have challenged this find-
ing. While Calhoun et al concluded that research experi-
ences during medical school played a significant role in
matching into otolaryngology, a follow-up study revealed
that the number of publications did not correlate with resi-
dent performance.12 Tolan et al showed no correlation
between publications/research and successful fulfillment of
the ACGME core competencies throughout general surgery
residency.13 Furthermore, no significant difference was
shown with regard to urology in-service scores or clinical
evaluations when stratified by medical student publication
numbers.14
Furthermore, studies have suggested that applicants may
even misrepresent their research experience to bolster their
applications. Baker and Jackson reviewed the literature and
found astoundingly high numbers of misrepresentation in
4 Otolaryngology–Head and Neck Surgery

Figure 5. Percentage change in applicants from the previous year

Figure 4. Percentage change in applicants since 2006 in otolaryn- in otolaryngology.1
gology versus all specialties.1

multiple specialties, including imaging fellowships (16%),

pediatrics (19.7%), emergency medicine (20%), and gastro-
enterology (34%).19 With regard to otolaryngology, a
Canadian study found that as many as 12% of publications
were misrepresented by 23% of applicants.20 Additionally, a
study from the University of Pittsburgh evaluated 432 publi-
cations by 173 otolaryngology applicants in 2010. It identi-
fied that 9.8% of applicants who reported a publication
misrepresented that claim. Univariate analyses revealed that
being an international medical graduate, older age, having a
lower USMLE Step 1 score, and increasing time between
medical school and residency were variables independently
associated with this misrepresentation.21 These findings of
misrepresentation may in turn call into question the positive
correlation between research and class rank as identified by
Bhat et al.11 This State of the Art Review reveals that
research experience to include publications has become an Figure 6. Percentage change in number of applicants from 2011
across the top-4 most-applied-to specialties.1
inherent expectation in a medical student’s application but
that the value of such research remains questionable with
respect to determining residency success.
regard to number of applications per applicant for the past 3
Recent Match Trends consecutive years—dermatology, orthopedics, and urology.
The downstream effects of the aforementioned medical student Figure 6 illustrates the percentage change in number of appli-
‘‘requirements’’ of high USMLE scores, AOA membership, cants since the 2011 cycle for the top-4 specialties. Over the
and research may already be apparent in review of the most past 3 years, the only specialty that appears to be declining is
recent 2016-2017 ERAS data. This study evaluated otolaryn- otolaryngology, with the rest remaining relatively stable or
gology residency application trends since the transition to the slightly increased. This finding further supports the concept
ERAS in 2006. Figure 4 shows the percentage change in oto- that fewer medical students are applying to otolaryngology
laryngology applicant numbers versus all specialties. Based on even when compared with the other highly competitive
these findings, it appears that otolaryngology has hit an all- specialties
time low in the number of residency applications (352 US
applicants in 2017). Further data analysis identifies that the Implications for Practice
downswing in application numbers has been developing over Otolaryngology is an exceptionally unique field composed of
the past 3 years in otolaryngology. Figure 5 illustrates the collegial, diligent, and humanistic surgeons. It is recognized as
percentage change in number of applicants from 2012 to 2017. a highly competitive surgical subspecialty reserved for medical
Beginning in the 2014-2015 cycle, otolaryngology began to students with high board scores, AOA membership, and a cur-
experience a more significant decrease in applicants than had riculum vita boasting numerous publications, presentations,
been documented in ERAS data history. To evaluate if this and research projects. Consequently, 80% of current medical
trend was occurring in other highly competitive specialties, we students cite matching into otolaryngology as either ‘‘nearly
compared otolaryngology to the other top-3 specialties with impossible’’ or even ‘‘impossible.’’2 However, this State of the
Bowe et al
Art Review failed to identify reliable positive correlations
between these metrics and successful residency performance.
Given the declining number of otolaryngology residency appli-
cations that has the potential to reach a shortfall within 2
years, a change in the ‘‘ideal’’ otolaryngology resident is
required. A possible cause for the declining number of oto-
laryngology residency applications over the recent years is
related to the steep scholastic ‘‘requirements’’ that are
widely known and circulated among medical students.
Unfortunately, there is inconsistent support that this intimi-
dating compilation of high USMLE scores, numerous pub-
lications/research, and AOA status corresponds to resident
success in practice. Consequently, the findings summarized
in this State of the Art Review identify the potential for
change in the way in which (1) medical students view the
field of otolaryngology and (2) residency programs ini-
tially filter applicants. For example, medical students with
USMLE scores in the 220-230 range, 3 publications, and
top one-third status of their class but not AOA are cur-
rently under the impression that their applications are not
good enough to even attempt applying in otolaryngology.
Therefore, otolaryngologists in academia need to look at
alternative ways in which their programs can reach this
student cohort, encourage them to apply, and have them
truly believe that this endeavor is realistic and worthwhile.
Intelligence and surgical dexterity are highly valued by
otolaryngologists, and the specialty embraces curiosity,
integrity, and empathy. However, these same traits are
easily missed by focusing more explicitly on the scholastic
achievements of medical student applicants. For instance,
despite the data revealing that most applicants possess high
USMLE scores, AOA membership, and substantial research,
nearly all otolaryngology programs have had residents
requiring remediation for a variety of reasons. A 2015
survey of otolaryngology program directors indicated that
90% of programs have been challenged with problematic
residents. The authors identified unprofessional behavior,
insufficient medical knowledge, and poor clinical judgment
as the most prevalent issues requiring formal remediation.22
Although remediation is not as detrimental to a program as
a resident’s resignation/removal, it still takes a significant
toll on the resources of a department. Bhatti et al cited gen-
eral counseling, frequent feedback sessions, and assignment
of a mentor as the corrective strategies most commonly uti-
lized to assist the remediating resident.22 Therefore, faculty
are often required to devote substantial time and training to
the troubled resident—time potentially taken away from the
teaching and development of the other residents. This per-
sistent need for remediation serves to further highlight the
shortcomings of the current application screening process,
which often emphasizes cognitive attributes.
In summary, this State of the Art Review, investigating
the recent qualifications of otolaryngology resident appli-
cants and the associated validity of these metrics, highlights
the need to adjust the screening criteria beyond the tradi-
tional USMLE scores, AOA membership, and research
metrics. Additional measures focusing on the noncognitive
5
and holistic qualities of an applicant should be included. For
example, instruments such as situational judgment tests23,24
and personality assessments25 could be additional areas of
exploration in the early phase of the selection process.
Furthermore, it is imperative to shed the stereotype of
matching into otolaryngology as being ‘‘impossible.’’ Open
dialogue between otolaryngology program directors and the
medical students at their home institutions is warranted to
successfully make this change. Likewise, it will also be
important to encourage a greater breadth of applicants. Last,
an evidence-based evaluation of the various criteria for oto-
laryngology resident selection and training outcomes is
required to improve the current system. Ultimately, as a
subspecialty, we want to train the best otolaryngologists
possible—not simply enough applicants to fill all the train-
ing spots and residents who will just pass their boards. To
achieve this, further consideration must be given to identify
the qualities that we value and how best to assess them.
Author Contributions
Sarah N. Bowe, substantial contribution to design; acquisition,
analysis, and interpretation of data; drafting and revising; final
approval; accountability for all aspects of work; Cecelia E.
Schmalbach, substantial contribution to design; analysis and inter-
pretation of data; revising; final approval; accountability for all
aspects of work; Adrienne M. Laury, substantial contribution to
design; acquisition, analysis, and interpretation of data; drafting
and revising; final approval; accountability for all aspects of work.
Disclosures
Competing interests: Cecelia E. Schmalbach, AO North
America—honoraria.
Sponsorships: None.
Funding source: None.
References
1. Electronic Residency Application Service. Statistics: 2017 pre-
liminary statistics as of 12/12/2016 (ERAS 2016). https://
www.aamc.org/services/eras/stats. Accessed December 2,
2016.
2. Kaplan AB, Riedy KN, Grundfast KM. Increasing competi-
tiveness for an otolaryngology residency: where we are and
concerns about the future. Otolaryngol Head Neck Surg. 2015;
153:699-701.
3. Daly KA, Levine SC, Adams GL. Predictors for resident suc-
cess in otolaryngology. J Am Coll Surg. 2006;202:649-654.
4. Alterman DM, Jones TM, Heidel RE, Daley BJ, Goldman
MH. A predictive value of general surgery application data for
future resident performance. J Surg Educ. 2011;69:513-518.
5. Dillon GF, Swanson DB, McClintock JC, Gravlee GP. The
relationship between the American Board of Anesthesiology
Part 1 Certification Examination and the United States
Medical Licensing Examination. J Grad Med Educ. 2013;5:
276-283.
6. Armstrong A, Alvero R, Nielsen P, et al. Do US Medical
Licensure Examination Step 1 scores correlate with Council
on Resident Education in Obstetrics and Gynecology in-
6 Otolaryngology–Head and Neck Surgery
training examination scores and American Board of Obstetrics
and Gynecology written examination performance? Military
Med. 2007;172:640-643.
7. Puscas L. Otolaryngology resident in-service examination
scores predict passage of the written board examination.
Otolaryngol Head Neck Surg. 2012;147:256-260.
8. Raman T, Alrabaa RG, Sood A, et al. Does residency selection
criteria predict performance in orthopaedic surgery residency?
Clin Orthop Relat Res. 2016;474:908-914.
9. Chole RA, Ogden MA. Predictors of future success in otolar-
yngology residency applicants. Arch Otolaryngol Head Neck
Surg. 2012;138:707-712.
10. Cullen MW, Reed DA, Halvorsen AJ, et al. Selection criteria
for internal medicine residency applicants and professionalism
ratings during internship. Mayo Clin Proc. 2011;86:197-202.
11. Bhat R, Takenaka K, Levine B, et al. Predictors of a top per-
former during emergency medicine residency. J Emerg Med.
2015;49:505-512.
12. Calhoun KH, Hokanson JA, Bailey BJ. Predictors of residency
performance: a follow-up study. Otolaryngol Head Neck Surg.
1997;116:647-651.
13. Tolan AM, Kaji AH, Quach C, Hines OJ, de Vergilio C. The
Electronic Residency Application Service application can pre-
dict Accreditation Council for Graduate Medical Education
competency-based surgical performance. J Surg Educ. 2010;
67:444-448.
14. Grewal SG, Yeung LS, Brandes SB. Predictors of success in a
urology residency program. J Surg Educ. 2013;70:138-143.
15. National Resident Matching Program. Main residency match
data. http://www.nrmp.org/match-data/main-residency-match-
data. Published 2016. Accessed September 21, 2016.
16. Prober CG, Kolars JC, First LR, Melnick DE. A plea to reas-
sess the role of United States Medical Licensing Examination
Step 1 scores in residency selection. Acad Med. 2015;90:1-4.
17. Shellito JL, Osland JS, Helmer SD, Chang FC. American
Board of Surgery examinations: can we identify surgery resi-
dency applicants and residents who will pass the examinations
on the first attempt? Am J Surg. 2010;199:216-222.
18. Kenny S, McInnes M, Singh V. Associations between resi-
dency selection strategies and doctor performance: a meta-
analysis. Med Ed. 2013;47:790-800.
19. Baker DR, Jackson VP. Misrepresentation of publications by
radiology residency applicants. Acad Radiol. 2000;7:727-729.
20. Sater L, Schwartz JS, Coupland S, Young M, Nguyen LH.
Nationwide study of publication misrepresentation in appli-
cants to residency. Med Educ. 2015;49:601-611.
21. Beswick DM, Man LX, Johnston BA, Johnson JT, Schaitkin BM.
Publication misrepresentation among otolaryngology residency
applicants. Otolaryngol Head Neck Surg. 2010;143:815-819.
22. Bhatti NI, Ahmed A, Stewart MG, Miller RH, Choi SS.
Remediation of problematic residents: a national survey.
Laryngoscope. 2016;126:834-838.
23. Hojat M, Erdmann JB, Gonnella JS. Personality assessments
and outcomes in medical education and the practice of medi-
cine: AMEE guide No. 79. Med Teach. 2013;35:e1267-e1301.
24. Patterson F, Prescott-Clements L, Zibarras L, et al. Recruiting
for values in healthcare: a preliminary review of the evidence.
Adv in Health Sci Educ Theory Pract. 2016;21:859-881.
25. Patterson F, Zibarras L, Ashworth V. Situational judgment
tests in medical education and training: research, theory, and
practice: AMEE guide No. 100. Med Teach. 2016;38:3-17.