Академический Документы
Профессиональный Документы
Культура Документы
®
Certified (OTC )
JOB ANALYSIS
Conducted on behalf of
by
April 2005
Acknowledgments
We would like to thank the individuals who provided invaluable assistance throughout the conduct of the
NBCOT Orthopaedic Technologist Certified® (OTC®) Job Analysis for the National Board for Certification of
Orthopaedic Technologists.
Jeff Virgo, Chairman of the Certification Committee provided invaluable guidance throughout the study.
i
TABLE OF CONTENTS
METHOD.............................................................................................................................................................. 3
CONDUCT OF PLANNING MEETINGS ................................................................................................................... 3
CONDUCT OF THE JOB ANALYSIS MEETING. ...................................................................................................... 3
FINAL REPORT OF THE JOB ANALYSIS MEETING. .............................................................................................. 3
TASK FORCE MEETING DOCUMENTATION ............................................................................................ 4
TEST DEFINITION FOR THE NBCOT ORTHOPAEDIC TECHNOLOGIST CERTIFIED™ EXAMINATION .................. 4
REVIEW OF THE 1995 ROLE DELINEATION STUDY ............................................................................................ 4
DEVELOPMENT OF TEST SPECIFICATIONS FOR THE NBCOT OTC® EXAMINATION ................ 5
EXAMINATION SECTION WEIGHTING ................................................................................................................. 5
TEST OBJECTIVE WEIGHTING ............................................................................................................................. 5
SUMMARY AND CONCLUSIONS................................................................................................................... 6
APPENDIX B ....................................................................................................................................................... 7
APPENDIX C ..................................................................................................................................................... 11
ii
Executive Summary
The National Board for Certification of Orthopaedic Technologists commissioned a Job Analysis through
Thomson Prometric in early 2003. As described by NBCOT, the Orthopaedic Technologist Certified® (OTC®)
certification serves the public interest by developing, administering, and continually reviewing a certification
process that reflects current standards of competent practice in orthopaedic technology thereby protecting the
public by assuring that individuals are at the appropriate level of competency required for practice.
A job analysis is designed to obtain descriptive information about the tasks performed in a job and the
knowledge needed to adequately perform those tasks. The purpose of the NBCOT OTC® Job Analysis was to:
Review and update the list of the tasks and knowledge related to work performed by NBCOT practitioners;
Develop test specification recommendations for the NBCOT OTC® Examination.
NBCOT will use the results of the study to ensure that its OTC® Examination and certain other initial and
renewal certification requirements continue to be based on validated tasks and knowledge important to the
work performed by NBCOT practitioners.
The Job Analysis consisted of a Task Force Committee meeting where the current testing domain was
examined, updated, and a test specification completed. The successful outcome of the Job Analysis was
dependent on the excellent information provided by NBCOT practitioners.
The NBCOT Orthopaedic Technologist Certified® (OTC®) Examination 1995 Role Delineation Study was
the basis for the 2003 Job Analysis.
A Task Force Committee comprised of NBCOT professionals reviewed and revised:
1) the Test Definition Document
2) the 1995 Role Delineation Study’s tasks and knowledge
3) the test specification
A listing of the Task Force Committee is contained in Appendix A.
Summary.
The Job Analysis process allowed for input from a representative group of NBCOT professionals and was
conducted within the guidelines of professionally sound practice. The results of the Job Analysis can be used by
the NBCOT Board to develop a new version of the NBCOT OTC® Examination, as well as for other purposes
such as curriculum development, communication of examination requirements to potential test candidates, and
for item development.
A job analysis refers to procedures designed to obtain descriptive information about the tasks performed on a
job and/or the knowledge, skills, or abilities thought necessary to adequately perform those tasks. The specific
type of job information collected for a job analysis is determined by the purpose for which the information will
be used.
For purposes of developing licensure and certification examinations, a job analysis should identify important
activities, knowledge, skills, and/or abilities.
The use of a job analysis (also known as practice analysis, role delineation, role and function study) to define
the content domain is a critical component in establishing the content validity of licensure and certification
examinations. Content validity refers to the extent to which the content covered by an examination overlaps
with the important components of a job (tasks, knowledge, skills, or abilities).
A well-designed job analysis should include the participation of a representative group of subject-matter
experts who reflect the diversity within the job. Diversity refers to regional or job context factors and to
NBCOT Orthopaedic Technologist Certified® (OTC®) Job Analysis 1
subject-matter expert factors such as experience, gender, and race/ethnicity. Demonstration of content validity
is accomplished through the judgments of subject-matter experts.
The Standards for Educational and Psychological Testing1 (1999) (The Standards) is a comprehensive
technical guide that provides criteria for the evaluation of tests, testing practices, and the effects of test use. It
was developed jointly by the American Psychological Association (APA), the American Educational Research
Association (AERA), and the National Council on Measurement in Education (NCME). The guidelines
presented in The Standards, by professional consensus, have come to define the necessary components of
quality testing. As a consequence, a testing program that adheres to The Standards is more likely to be judged
to be valid and defensible than one that does not.
As stated in Standard 14.14,
“The content domain to be covered by a credentialing test should be defined clearly and justified in terms
of the importance of the content for credential-worthy performance in an occupation or profession. A
rationale should be provided to support a claim that the knowledge or skills being assessed are required for
credential-worthy performance in an occupation and are consistent with the purpose for which the
licensing or certification program was instituted…Some form of job or practice analysis provides the
primary basis for defining the content domain… (p.161)
The NBCOT OTC® Examination Job Analysis was designed to follow the guidelines presented in The
Standards and to adhere to accepted professional practice.
1
American Educational Research Association, American Psychological Association, National Council on Measurement in Education. (1999). The
standards for educational and psychological testing. Washington, DC: American Psychological Association.
T
his section describes the activities of the 2003 NBCOT Job Analysis.
Several planning meetings were held with Jeff Virgo, Chairman of the Certification Committee for NBCOT,
and Thomson Prometric Test Development staff. In these meetings the overall strategy for the Job Analysis was
discussed, the Test Definition Document, its purpose and constituent parts was explained, and an initial review
of the draft of the Test Definition Document was completed.
Thomson Prometric staff sent a pre-meeting mailing to the Task Force that included a document describing the
Job Analysis and the role of the Task Force Committee, an agenda, and procedures for reviewing and revising
the tasks and topics. The Task Force Committee members were advised they would be reviewing the 1995
NBCOT Role Delineation Study and updating that document during the meeting.
Activities conducted during the one-day in-person meeting included reviewing and revising the:
Thomson Prometric staff prepared this final report to document the process steps and outputs of this Job
Analysis.
T
he documentation from the Task Force Meeting is explained and presented in this section.
The Standards recommend that the test should begin with a very clear definition of its purpose, the audience
intended, and the scope of the test.
“The test developer should set forth clearly how test scores are intended to be interpreted and used. The
population(s) for which a test is appropriate should be clearly delimited, and the construct that the test is
intending to assess should be clearly described.” (p. 17)
“The purpose(s) of the test, definition of the domain, and the test specifications should be stated clearly so
that judgments can be made about the appropriateness of the defined domain for the stated purpose(s) of the
test and about the relation of items to the dimensions of the domain they are intended to represent.” (p. 43)
Thomson Prometric created a Test Definition Document for the NBCOT OTC® examination. This document
clearly delineates all of the necessary elements of the examination and is used throughout the process to ensure
that the domain described and the test items created meets the intended purpose of the examination definition.
Prior to the conduct of the Job Analysis workshop with NBCOT, a template of the Test Definition Document
was sent to Jeff Virgo. Jeff completed the document and then Thomson Prometric reviewed it prior to the
workshop.
During the workshop the Test Definition Document was reviewed by the entire group of subject-matter experts
and finalized after some minor changes. The Test Definition Document is found in Appendix B.
After the subject-matter experts approved the Test Definition Document, the next activity was to review the
tasks and knowledge for the test.
The 1995 Role Delineation Study was approximately ten years old at the time of this work with the NBCOT. In
that ten years very little had changed in the industry and NBCOT felt that there would be few major changes to
the testing domain resulting from the 2003 Job Analysis.
The subject-matter experts reviewed the tasks and knowledge for the 1995 Role Delineation Study, made
changes as needed, deleted some tasks and knowledge and added a few new elements that have changed in the
past years.
The one major change was the elimination of all of Section 2: Practice Management. The majority of candidate
complaints about the current test were about this section. The information in this section generally is specific to
an individual practice, and these practices can vary significantly region to region.
The finalized 2003 Job Analysis for the NBCOT Orthopaedic Technologist Certified® (OTC®) examination is
provided in Appendix C.
T
est specifications were developed during the workshop by the subject-matter experts present. Thomson
Prometric uses a test blueprint template to provide a structured exercise for the subject-matter experts to
create the finalized test specification. This exercise consists of two parts: a section weighting and a test
objectives weighting.
As a preliminary step to finalizing the test specification, the participants each propose a percentage of the
examination that should be applied to each of the sections. The group average is then reviewed and a final
section weighting agreed to by the group. The test section weighting provides the overall weighting by domain
(section). The test specification is then weighted at the test objective level and provides the final specification
for item writing.
The following chart shows the group average and proposed group percentage to allocate to each section.
Consensus
Group Group
Section / Name Average Average
Section 1: Assessment 30.00 40%
Section 2: Practice Management (Section deleted) 0 0
Section 3: Casting, Splinting, and Orthopaedic Appliances 41.42 40%
Section 4: Traction 14.29 10%
Section 5: Surgery 14.29 10%
TOTAL 100.00 100.0%
The next step in the weighting process was to review each of the test objectives and assign the number of test
items that will be written to each of the objectives. This was done through a group activity with the facilitator
asking if the “weight” listed for each test objective properly represents the importance that should be assigned
to each objective in the final form of the examination.
The group reviewed each test objective in the test specification and a number was assigned for total test items
to be written to that objective. The total of the items to be written equals 150, the current number of test items.
The finalized test specification for the OTC® Examination is provided in Appendix D.
he NBCOT Orthopaedic Technologist Certified® (OTC®) Examination Job Analysis was conducted to:
T 1) Validate the test definition for the NBCOT Orthopaedic Technologist Certified® (OTC®) Examination.
2) Review and validate the tasks and topics important in the work performed by NBCOT practitioners.
3) Create a test specification to will be used to develop a new version of the OTC® Examination.
The results of the study can be used as a foundation of validated information for the development of a new
version of the OTC® Examination.
Please first review the test purpose and then in the table below provide a description of the skills and/or
knowledge of each candidate type.
Type of Candidate Description of Candidate
Clearly Qualified Can work without supervision in all aspects of the Orthopaedic Technologist
field as defined by the 2003 NBCOT Orthopaedic Technologist Certified®
(OTC®) Job Analysis.
Can supervise other Technologists.
Can teach or instruct other Technologist.
Borderline Qualified Can work with limited supervision in all aspects of the Orthopaedic
Technologist field as defined by the 2003 NBCOT Orthopaedic Technologist
Certified® (OTC®) Job Analysis.
Does not supervise other Technologist.
Does not teach or instruct other Technologists.
Informed but Clearly Must be supervised by a clearly qualified Technologist in all aspects of the
Not Qualified Orthopaedic Technologist field as defined by the NBCOT Orthopaedic
Technologist Certified® (OTC®) Job Analysis.
Test Modules:
Which of the following methods will be used for this test?
NBCOT Orthopaedic Technologist Certified® (OTC®) Job Analysis 9
Examination delivered and scored by computer
To include hot start tools such as calculator, Excel, etc.
To include simulations of actual environment
To include lab performance in actual environment through intranet/Internet
Essay with human scorers
Live performance with human scorers
Oral with human questioners
Lab performance with human scorers
Paper-pencil examination
Will this test point have more than one separately scored module?
Yes
No
If so, how will the modules be organized?
The candidate must pass each module in order before passing the test.
The candidate must pass all modules in any order before passing the test.
The candidate must achieve a combined passing score across all modules before passing the test. Test
Stakeholders:
What persons or organizations will be served by this test? Specify stakeholder representatives who will be
involved in the test content development and objectives creation.
Person or Organization Representative
NBCOT All Board Members
NAOT General Membership
Athletic Trainers Individuals
X-Ray Technologists Individuals
OPA/PA-C Individuals
2
Note: The numbering of the sections changed due to the removal of what was formerly Section 2: Practice Management.