Otolaryngol Clin N Am

40 (2007) 1347–1357

Certification and Maintenance

of Certification in Otolaryngology–Head
and Neck Surgery
Robert H. Miller, MD, MBAa,b
a
American Board of Otolaryngology, 5615 Kirby Drive,
Suite 600, Houston, TX 77005-2444, USA
b
Department of Otolaryngology–Head and Neck Surgery,
Baylor College of Medicine, Houston, TX, USA

The mission
The mission of the American Board of Otolaryngology (ABOto) is to assure
that, at the time of certification and recertification, diplomates certified by the
ABOto have met the ABOto’s professional standards of training and knowl-
edge in otolaryngology–head and neck surgery.

History
Shortly after the turn of the twentieth century, the American Academy of
Ophthalmology and Otolaryngology established two committees to explore
the concept of certification in these two specialties. The result of the delib-
erations was the development of the American Board of Ophthalmology
in 1916 and the American Board of Otolaryngology (ABOto) in 1924. Ini-
tially, 465 otolaryngologists were invited to receive certification, and 354
were certified [1]. Through 2007, 16,989 otolaryngologists have been certi-
fied by the ABOto. In addition to certifying individuals, the ABOto set
the standards for and accredited otolaryngology residency programs until
1953, at which time the Accreditation Council for Graduate Medical Edu-
cation assumed this function.
The certification process has evolved over time. Initially, the certification
examination consisted of a written histopathology examination, a practical
examination with real patients, and an oral knowledge examination, which
was waived for ‘‘experienced’’ practitioners. The first written examination

E-mail address: rmiller@aboto.org

0030-6665/07/$ - see front matter Ó 2007 Elsevier Inc. All rights reserved.
doi:10.1016/j.otc.2007.07.010 oto.theclinics.com
1348 MILLER

required essay-type answers but is currently in a multiple choice question

format. The practical examination became the oral examination, which in
its early format permitted a fair amount of individual examiner discretion
in its administration. Currently, the oral examination consists of highly
structured protocols that measures an examinee’s knowledge more effec-
tively and consistently.
The written examination is considered a qualifying examination because
a candidate must pass it before having his or her score on the oral certifying
examination considered. If a candidate passes the written examination but
fails the oral examination, he or she has 3 years to pass the oral examination
and become certified before having to retake the written examination. Both
examinations are administered annually in Chicago.

Examination development and process

The examination development process is detailed, costly, and time con-
suming. The items (questions) for the two multiple choice question examina-
tionsdthe written and otolaryngology training examinations (in-service
examination)dare generated by the Task Force for New Material, which
consists of 36 item writers. Item writers are selected from a pool of individ-
uals nominated by all of the otolaryngology specialty societies and by the
ABOto directors and senior examiners; they serve 3-year terms. Item writers
participate in the annual item writers’ workshop, during which they learn
the nuances of writing effective multiple choice questions.
After the item is generated, it goes through a thorough and extensive ed-
iting and vetting process, which ensures that the medical information is ac-
curate and refines the question wording and syntax. Finally, all new items
are ‘‘field tested’’ on either the otolaryngology training examinations or
written examinations. The statistical performance of each new item is care-
fully reviewed by the ABOto directors and a psychometrician. Items are
evaluated as to whether they are too easy or too hard and how well an in-
dividual item’s performance correlates with how well the people who se-
lected the correct answer did on the test as a whole. Items that do not
meet the ABOto’s standards are either revised or discarded.
The items that are used for scoring on the otolaryngology training exam-
inations and written examinations are selected from a pool of successfully
field-tested items based on the examination blueprint, which is available
on the ABOto Web site (www.aboto.org). The directors select the best items
from the pool based on several statistical and other parameters. The proto-
cols used in the oral and neurotology examinations are written by the oral
examiners, which include the directors, senior examiners, and guest exam-
iners. Each protocol is carefully reviewed and edited several times by the ap-
propriate specialty-specific group before its use.
The results of all of the exams are analyzed by a psychometrician, who
identifies any items that have an unusual statistical performance. These
 CERTIFICATION AND MOC 1349

items are reviewed by the examination committee, and if the question is not
thought to be valid, it is discarded from the examination and not used in
scoring. The written and otolaryngology training examination scores are ad-
justed for annual variations in the ‘‘hardness or easiness’’ of the examination
so that scores are consistent over time. Similarly, the oral examination
scores are adjusted for variations in examiner leniencies and variations in
the test severity for the 2-day examination period and year-to-year
variations.
Before 1999, the passing score was set by failing a fixed percentage of ex-
aminees. In 1999, the ABOto switched to a criterion referenced standard, in
which the directors participated in a psychometric standard setting exercise
that determined a fixed pass/fail point. Theoretically, it is possible to have
a 100% pass rate. This standard-setting exercise is repeated approximately
every 5 years to ensure its validity.

Health care quality milestones

A few historical events have had a major influence on health care quality
in the United States. Perhaps one of the most important was the 1910 Flex-
ner report on medical school education [2]. At the time there were many pro-
prietary schools, many of which did a poor job of educating medical
students. Commissioned by the Carnegie Foundation, the report summa-
rized US medical schools and their curricula and provided recommenda-
tions on how to improve the quality of physician education. The
establishment of medical specialty boards in 1916 was also a milestone be-
cause it not only provided a mechanism for evaluating physicians but also
ultimately provided a mechanism for residency training accreditation. The
public values board certification as a measure of quality [3].
Although quality of care has been an important subject for many
years, the Institute of Medicine reports on quality of care in the United
States have advanced this issue rapidly in many sectors, including the
government, health care providers, insurers, purchasers of insurance,
and public interest groups [4,5]. As a consequence, many groups are ex-
ploring methods to improve all levels of the health care system. As an
example, two organizations that have instituted quality improvement pro-
grams are the Joint Commission on Accreditation of Healthcare Organi-
zations for hospitals and the National Committee for Quality Assurance
for health plans. Similarly, the two groups that have the main responsi-
bility for overseeing the quality of individual physicians are the state
medical licensing boards and their umbrella organization, the Federation
of State Medical Boards and the American Board of Medical Specialties
(ABMS), which is the umbrella organization for the 24 medical specialty
boards. The ABMS and the Federation of State Medical Boards are ac-
tive in the invigorated national health care quality improvement
movement.
1350 MILLER

Board certification continuum

Many people view the board certification process as simply passing the
board examinations at the end of residency. The ABOto, however, views
the certification process as a continuum that begins when residents are se-
lected for training. The selection process is an important screen to identify
the brightest, most capable medical students for otolaryngology–head and
neck surgery training. Within the first few months of training, the new res-
idents must register with the ABOto, which obtains and verifies information
about the residents’ previous training.
The ABOto views the program director as a critical component of the cer-
tification process because the program director, along with the faculty, has
the most exposure to the residents’ knowledge, skills, and behavior over the
5-year residency period. Based on the Accreditation Council for Graduate
Medical Education’s six competencies (Box 1), the board examinations
have strengths in the measurement of medical knowledge and patient care
(with the exception of surgical skills), but the program director is in
a much better position to evaluate the other four competencies (Table 1).
At the end of training, the chairperson and program director must attest
that the candidate is of ‘‘high moral character and worthy of examination
by the ABOto’’ and is acceptable for the examination process.
Before 2002, all ABOto diplomates received timeless certificates, which
means that the individual was certified forever unless the diplomate commit-
ted a crime or some adverse action were taken by the state licensing boards.
Unless the diplomate committed an act that would be deemed egregious
enough to result in decertification, the diplomate would continue to be cer-
tified although he or she had not been evaluated by the ABOto since the date
of initial certification. Unless there was some local review by, for example,
the diplomate’s hospital, a diplomate’s practice was not evaluated for re-
maining current on the latest medical developments or for quality of care.
As the interest and pressure to improve health care quality increased, it
became clear that physicians needed more scrutiny to ensure they were prac-
ticing high-quality medicine. As a consequence, the ABMS and its member
boards determined that a new approach was needed to ensure that certified

Box 1. The Six Competencies

1. Medical knowledge
2. Patient care
3. Interpersonal and communication skills
4. Professionalism
5. Systems-based practice
6. Practice-based learning and improvement
 CERTIFICATION AND MOC 1351

Table 1
Ability to evaluate competencies in residents
Competency ABOto examinations Program director
Medical knowledge þþþ þþþ
Patient care þþþa þþþ
Interpersonal and communication skills þþ þþþ
Professionalism þ þþþ
System-based care þ þþþ
Practice-based learning and improvement þ þþ
a
Excludes surgical technique.

physicians were maintaining the ability to practice high-quality medicine.

No longer could it be acceptable to provide lifetime certification without pe-
riodic assessment. After careful study, the ABMS developed the Mainte-
nance of Certification program (MOC) to address the needs for health
care quality improvement.
All otolaryngology diplomates (both primary and subspecialty) certified
in 2002 and thereafter receive 10-year time-limited certificates. To maintain
certification and have the certificate renewed, all diplomates with time-lim-
ited certificates are required to participate in MOC.

Maintenance of certification
MOC is a quality improvement program designed to improve physicians’
practice of medicine. Although the various boards are taking somewhat dif-
ferent approaches to its implementation, all MOC programs consist of four
parts:
1. Professional standing
2. Continuing education and self-assessment
3. Cognitive expertise
4. Performance in practice

Professional standing
The ABOto requires that all MOC participants possess a valid ABOto
certificate. The participant also must have unrestricted licenses to practice
medicine in all states in which he or she practices. The diplomate also
must have privileges at a hospital or ambulatory surgery center. If the par-
ticipant does not have privileges, he or she must attest that the privileges
were not lost because of an adverse action by the facility.
In the past, it was difficult for the ABOto to routinely obtain information
about adverse actions taken against its diplomates. Occasionally, the Amer-
ican Medical Association or some other source would notify the board if
one of its diplomates had been involved in unprofessional activity. In
1352 MILLER

2004, however, the Federation of State Medical Licensing Boards estab-

lished the Disciplinary Alert Notification Service (DANS), which has facil-
itated transfer of important information to each of the ABMS specialty
boards. Through DANS, the ABOto receives regular reports on any adverse
action taken by hospitals, state licensing boards, and others against ABOto
diplomates. It is important for the ABOto to obtain this information, which
covers a wide variety of actionsdfrom restrictions on privileges at hospitals
to loss of licensure. Most of these offenses are minor in nature and do not
reflect professional misconduct or evidence of poor practice.
Although severe adverse actions against board-certified otolaryngologists
reported through the disciplinary alert notification service are uncommon,
the ABOto has taken action against these diplomates, including the most se-
vere penalty: decertification. The ABOto’s Credentials/Ethics Committee re-
views the details of the sentinel event and obtains letters of explanation from
the diplomate. Only after careful analysis does the Credentials/Ethics Com-
mittee make a recommendation to the full ABOto board of directors, which
is the final arbiter for these actions. It is important for the ABOto to enforce
these standards because it maintains the importance and value of certifica-
tion by the ABOto.

Continuing education and self-assessment

Critical to practicing high-quality otolaryngology–head and neck surgery
is staying up-to-date on the latest developments in the specialty. One way of
achieving this goal is to participate in continuing medical education (CME)
programs. To encourage this activity, the ABOto requires that all MOC par-
ticipants earn category 1 CME credits as a component of Part II of MOC.
The current requirement is that each participant earn as many CME hours
as are required to maintain his or her state medical license. These require-
ments vary from state to state, and a few states do not have CME require-
ments as part of licensure. For individuals in states that do not have a CME
requirement, the ABOto requires a minimum of 15 hours of category 1
CME credits. Sixty percent of CME credits for all MOC participants
must be related to the specialty of otolaryngology–head and neck surgery.
Diplomates who are subcertified in neurotology are required to have 60%
of their CME credits in either neurotology or otology.
Self-assessment of one’s performance as an otolaryngologist–head and
neck surgeon is a critical component of a quality improvement program.
There are various approaches to self-assessment, and the ABOto is explor-
ing several options. The first approach is through a patient simulation, in
which the participant manages a patient with a given medical condition.
These modules will be Internet-based and will simulate real patient encoun-
ters in which the participant must manage the patient. Various high-quality
visuals, including imaging studies, histopathology, and patient photographs,
enhance the patient management protocol, in which the participant can
 CERTIFICATION AND MOC 1353

choose multiple options regarding evaluation and management. Rather than

a linear progression through the program, decisions made by the participant
result in different paths, some of which are not viable. After completion of
the module, the participant is given feedback on which of the decisions were
wise and which were not. References are provided to encourage the partic-
ipant to study in areas of weaknesses, although the participant may choose
products from other sources to supplement the module. The module is not
pass/fail but is intended to assist the participant in identifying areas of
strengths and weaknesses. The participant may be asked to repeat the mod-
ule after he or she has had an opportunity to review the subject.
In addition to specialty-specific modules, the ABOto plans to incorporate
in MOC more generic modules from other sources that cover such topics as
patient safety, communication, and other subjects. Another potential option
is the use of surgical simulations as they are developed. It is anticipated that
the MOC participant will complete modules several times during the 10-year
MOC cycle as more modules become available. These modules will be devel-
oped in partnership with the specialty societies, academic departments of
otolaryngology–head and neck surgery, and perhaps individuals who have
a particular interest and expertise in this type of material. Other options
for self-assessment, including participation in interactive sessions at meet-
ings, are also being explored.

Cognitive expertise
At the end of the 10-year MOC cycle, each participant will be required to
take and pass a multiple choice question examination. The computer-based
examination will be administered in testing centers throughout the country
so that most participants will be able to complete Part III in their own
community.
Otolaryngology–head and neck surgery is a diverse specialty, and many
otolaryngologists tend to focus on subspecialty areas, although there is no
formal recognition of many of these subspecialties. For example, it would
be difficult for someone who practices head and neck surgery to remain cur-
rent on all aspects of otology. An otologist might find it difficult to answer
questions regarding facial plastic surgery. To address this situation, the
ABOto has determined that the Part III examination consist of two mod-
ules, both of which must be passed to renew a diplomate’s certificate. The
first test component is termed the fundamentals module, and it consists of
questions on material that all otolaryngologists should know. Some of the
topics that could be included in the core module are ethics, fluid and electro-
lytes, antibiotics, anesthesia, and patient safety, among others. The second
test component is a specialty-specific module in various areas within otolar-
yngology. The participant selects a test module based on the focus of his or
her practice. The areas that are planned to be available are listed in Box 2.
The neurotology and sleep medicine modules will be available to individuals
1354 MILLER

Box 2. Part III subspecialty areas

General otolaryngology
Head and neck surgery
Otology
Allergy
Pediatrics/bronchoesophagology
Laryngology
Rhinology
Facial plastic surgery
Neurotologya
Sleep medicinea
a
For individuals subcertified in these areas

who have been subcertified in these areas. Holders of these subspecialty cer-
tificates will take the examination in their respective area, which will renew
not only their subspecialty certificate but also their primary certificate.
The Part III examination will be available to MOC participants 3 years
before the expiration date of the certificate so that the individual has three
opportunities to pass the examination before the certificate expires. The first
examination is scheduled to be available in 2010 for diplomates whose cer-
tificates expire in 2012. Upon passing the examination and completing all
four MOC components, the successful participant receives a document
that indicates that his or her primary certificate has been renewed for an-
other 10-year period, during which the MOC cycle is repeated. Other than
the neurotology and sleep medicine modules, passing the Part III examina-
tion does not imply any particular expertise in the subspecialty area. Rather,
the primary certificate is renewed.

Performance in practice
Perhaps the most important component of MOC and the most difficult to
implement is Part IV: performance in practice. The intent of this component
is to measure a participant’s quality of practice. Part IV includes the out-
comes of the knowledge, ability, and skills as they apply to the management
of patients. The plan is to develop measures of these outcomes that are ac-
curate, reliable, and implemented with a minimum of intrusion. All of the
ABMS boards are working diligently to develop these measures. Although
a challenging project, the measures that will be developed for MOC may
be used in other programs that are being implemented by other organiza-
tions to measure the quality of health care. For example, Part IV quality
measures could be used in pay-for-performance programs and vice versa.
 CERTIFICATION AND MOC 1355

Other components of Part IV include Consumer Assessment of Health-

care Providers and Systems (CAHPS), which many health systems and large
health care organizations already use. The program is a means for patients
to provide feedback about their health care. The CAHPS program consists
of a standardized, validated questionnaire given to patients at the time of
a visit. Patients complete the questionnaire at their convenience using
a touch-tone phone or the Internet. Groups that have used this system
have found it helpful in improving patient satisfaction, which includes com-
munication and quality of care. As part of Part IV of MOC, CAHPS will be
available to smaller practices that otherwise may not be able to take advan-
tage of the program.

Who is required to participate in Maintenance of Certification?

In 2002, the ABOto began issuing only 10-year time-limited certificates
with the requirement that the individual participate in MOC. Currently,
1482 certified otolaryngologists participate in MOC, including all primary cer-
tificate holders certified in 2002 and thereafter and all neurotology and sleep
medicine certificate holders. The Board of Directors of the American Board
of Otolaryngology has voluntarily agreed to participate in MOC in support
of the program. The Federation of State Medical Boards is in the process of
developing a maintenance of licensure program, which parallels MOC in
many ways [6]. Although early in development, the Federation of State Med-
ical Boards envisions that all physicians (regardless of whether they are certi-
fied by a specialty board) will need to participate in maintenance of licensure.
Board-certified physicians who participate in their respective board’s MOC
program will meet the maintenance of licensure requirement, however. It is
likely that all licensed physicians may be required to participate in some sort
of maintenance of licensure/certification program in the future.
Finally, any otolaryngologist–head and neck surgeon with a timeless cer-
tificate can voluntarily participate in MOC. Internists who have participated
in the American Board of Internal Medicine MOC program have found it
valuable [7–9]. Voluntary participation does not jeopardize a timeless certif-
icate holder’s certificate, and although MOC in otolaryngology–head and
neck surgery is still in its infancy, the ABOto will be offering more compo-
nents in the not-too-distant future. We hope that many in our specialty find
these various components to be useful as a value-added program and volun-
tarily participate, as do the ABOto directors.

Summary
Based on its mission, the fiduciary responsibility of the ABOto is to the
public. When examining an issue, the ABOto directors view the debate
from several perspectives, but in the end, what is best for the publicdour
1356 MILLER

patientsdguides the decision-making process. The issue of quality of health

care always has been important, as indicated by the milestones of quality im-
provement that have occurred over the years. The issue is rightfully prom-
inent on the radar screen nationally. All the groups in health care are
stepping up to the plate, including physician groups such as the American
Board of Otolaryngology.
When I get on a plane, I feel comfortable that the pilot is well trained and
has undergone a vigorous ‘‘certification’’ process. Each pilot is tested in
a simulator at least annually and is observed by a Federal Aviation Admin-
istration investigator while actually making a flight. The industry also pro-
motes a culture in which ‘‘errors’’ can be reported on a nonpunitive basis,
which further adds to safety. Because the airline industry, in conjunction
with the Federal Aviation Administration, has made commercial aviation
one of the safest means of travel, aviation is sometimes mentioned as a po-
tential model for health care. Although an attractive consideration, it is not
clear that all aspects of their safety system are possible or practical in health
care. The important point, however, is that people in the airline industry
have collaborated to develop a system that has been successful. There is
no reason why we physicians cannot do the same.
Some physicians become defensive when the question of quality of care is
raised. Most otolaryngologist–head and neck surgeons practice high-quality
medicine on most patients most of the time; however, we all know that there
are a few practitioners to whom we would not refer a patient or a family
member because of questions of practice. These few ‘‘bad apples’’ exist,
and everyone agrees that they should be identified, given an opportunity
to improve their practices, and removed from practice if unsuccessful.
Fortunately, few practitioners fit into this category. Most, if not all, of us
could do a better job of staying current within the field, being more consis-
tent in managing patients, or improving patient safety by reducing errors
such as incorrect prescriptions or wrong site surgery. It is in this arena
that quality improvement programs, such as MOC, can improve the prac-
tices of our fellow otolaryngologists. This is the raison d’eˆtre of MOCd
not to penalize practitioners but to raise the standard of care by ensuring
that participants are aware of and knowledgeable about current information
in the field, are attentive to patient safety, and strive to improve patient
communication, among other attributes. It is hoped that MOC can provide
a mechanism or process to achieve this laudable goal.
The ABOto recognizes that MOC requires additional expenditure of time
and money. The ABOto is sensitive to these issues and is trying to imple-
ment a program that is cost effective, minimally intrusive, and meaningful.
MOC is a program in evolution. Considering that it is less than 5 years old,
MOC already has evolved into a value-added program. Physicians in some
specialties have seen a reduction in malpractice premiums by participating in
MOC. It seems natural that the same benefits would accrue in our specialty
as the otolaryngology MOC program matures. Some health care insurance
 CERTIFICATION AND MOC 1357

companies are interested in using MOC as one measure of performance for

pay-for-performance programs, which makes participation in MOC and
pay-for-performance more efficient.
When we were in medical school and residency, our performance was re-
viewed regularly in the form of tests, rotation evaluations, and other mech-
anisms. Ultimately, we passed the ABOto examination and became certified
but without any subsequent formal evaluation to ensure that we were stay-
ing current on the latest medical knowledge. Most certified otolaryngolo-
gists provide excellent care. The national movement to improve the
quality of health care mandates a program to ensure that we remain
up-to-date, however, which is the reason for MOC. If we physicians do
not address the health care quality issue, someone else in the form of the
government or another nonphysician group will. The thrust of the MOC
program is a continuous quality improvement program. It is not intended
to be punitive but rather serve as a stimulus for all of us to stay current
in our practices. Our patients deserve no less.

References
[1] Cantrell R, Goldstein G. The American Board of Otolaryngology 1924–1999. Houston (TX):
The American Board of Otolaryngology; 1999.
[2] Flexner A. Medical education in the United States and Canada. New York: Carnegie Foun-
dation for the Advancement of Teaching; 1910.
[3] Brennan T, Horwitz R, Duffy D, et al. The role of physician specialty board certification sta-
tus in the quality movement. JAMA 2004;292(9):1038–43.
[4] Kohn L, Corrigan J, Donaldson M, editors. To err is human: building a safer health system.
Washington, DC: Institute of Medicine; 2000.
[5] Committee on Quality of Health Care in America, Institute of Medicine. Crossing the quality
chasm: a new health system for the 21st century. Washington, DC: Institute of Medicine;
2001.
[6] Steinbrook R. Renewing board certification. N Engl J Med 2005;353(19):1994–7.
[7] Brennan T. Recertification for internists: one ‘‘grandfather’s’’ experience. N Engl J Med 2005;
353(19):1989–92.
[8] Baron R. Personal metrics for practice: how’m I doing? N Engl J Med 2005;353(19):1992–3.
[9] Batmangelich S, Adamowski S. Maintenance of certification in the United States: a progress
report. J Contin Educ Health Prof 2004;24(3):134–8.