A Practical Guide To Teacher Education Evaluation

A Practical Guide to
Teacher Education Evaluation

Evaluation in Education and
Human Services Series
Editors:
George F. Madaus, Boston College, Chestnut

Hill, Massachusetts, U.S.A.
Daniel L. Stufflebeam, Western Michigan
University, Kalamazoo, Michigan, U.S.A.
Previously published books in the series:

Kelleghan, T., Madaus, G., and Airasian, P.:
The Effects of Standardized Testing
Madaus, G. (editor):
The Courts, Validity, and Minimum Competency Testing
Brinkerhoff, R., Brethower, D., Hluchyj, T., and Nowakowski, J.:
Program Evaluation, Sourcebook/Casebook
Program Evaluation, Sourcebook
Program Evaluation, Design Manual
Madaus, G., Scriven, M., and Stufflebeam, D.:
Evaluation Models: Viewpoints on Educational and Human
Services Evaluation
Hambleton, R., and Swaminathen, H.:
Item Response Theory
Stufflebeam, D., and Shinkfield, A.:
Systematic Evaluation
Nowakowski, J.:
Handbook of Educational Variables: A Guide to Evaluation
Stufflebeam, D.:
Conducting Educational Needs Assessments
Cooley, W., and Bickel, W.:
Decision-Oriented Educational Research
Gable, R.:
Instrument Development in the Affective Domain
Sirotnik, K., and Oakes, J.:
Critical Perspectives on the Organization and Improvement of
Schooling
Wick, J.:
School-Based Evaluation: A Guide for Board Members,
Superintendents, Principals, Department Heads, and Teachers
Worthen, B., and White, K.:
Evaluating Educational and Social Programs
McArthur, D.:
Alternative Approaches to the Assessment of Achievement
May, L., Moore, C., and Zammit, S.:
Evaluating Business and Industry Training
Abrahamson, S.:
Evaluation of Continuing Education in the Health Professions
Glasman, N., and Nevo, D.:
Evaluation in Decision Making: The Case of School
Administration
Gephart, W., and Ayers, J.:
Madaus, G., and Stufflebeam, D.:
Educational Evaluation: Classic Works of Ralph W. Tyler
Gifford, B.:
Test Policy and the Politics of Opportunity Allocation: The
Workplace and the Law
Gifford, B.:
Test Policy and Test Performance
Mertens, D.:
Creative Ideas for Teaching Evaluation
Osterlind, S.:
Constructing Test Items
Smith, M.:
Evaluability Assessment
edited by
Jerry B. Ayers
Mary F. Berney
.....
"
Kluwer Academic Publishers
Boston/Dordrecht/London
Distributors for North America:
Kluwer Academic Publishers
101 Philip Drive
Assinippi Park
Norwell, Massachusetts 02061 USA
Distributors for ail other countries:

Kluwer Academic Publishers Group
Distribution Centre
Post Office Box 322
3300 AH Dordrecht, THE NETHERLANDS
Library of Congress Cataloging-In-Publication Data
A Practical guide to teacher education evaluation / edited by

Jerry B. Ayers and Mary F. Berney.
p. em. - (Evaluation in education and human services
series)
Includes bibliographical references.
ISBN-13: 978-94-010-7634-0 e-ISBN-13: 978-94-009-2512-0

001: 10.1007/978-94-009-2512-0
1. Teachers-Training of-United States-Evaluation.

Ayers, Jerry B. II. Berney, Mary F. III. Series.
LB1715.P66 1989
370'.71 '0973-dc20 89-36851
CIP
Copyright © 1989 by Kluwer Academic Publishers
Softcover reprint of the hardcover 1st edition 1989
All rights reserved. No part of this publication may be reproduced,

stored in a retrieval system or transmitted in any form or by any
means, mechanical, photocopying, recording, or otherwise, without
the prior written permission of the publisher, Kluwer Academic
Publishers, 101 Philip Drive, Assinippi Park, Norwell,
Massachusetts 02061.
Contents
Contributing Authors ix
Acknowledgments xiii
1
Introduction 1
J. T. Sandefur
2
Background for Teacher Education Program Evaluation 3
Jerry B. Ayers and Mary F. Berney
3
The Accreditation Plus Model 13
Jerry B. Ayers, William J. Gephart, and Paul A. Clark
4
Selection and Evaluation of Knowledge Bases
for Teacher Education Programs 23
Roger S. Pankratz
5
Quality Controls in Teacher Education
Programs 43
William E. Loadman
6
Testing for Admissions 49
Lawrence Rudner
vi
7
Evaluating Field-Based Experiences in Teacher
Education 69
Nancy L. Zimpher
8
Assessing Student Performance Outcomes in
Teacher Education Programs 85
Joyce R. McLarty
9
Assessment of Faculty in Teacher Education Programs 103
John A. Centra
10
Use of Mail Surveys to Collect Information for
Program Improvement 117
Jerry B. Ayers
11
Follow-Up Evaluation of Teacher Education
Programs 131
James R. Craig
12
Evaluating the Structure of the Education Unit 151
Edell M. Hearn
13
Physical Facilities Evaluation in Teacher
Education Programs 169
Mary F. Berney
14
Evaluating Financial Resources for Teacher
Education Programs 177
Robert L. Saunders
vii
15
Evaluation of Library Resources for a Teacher
Education Program 195
Edward D. Garten
16
Models and Modeling for Teacher Education Evaluation 211
Mary F. Berney and William J. Gephart
17
Implementation of Evaluation Results 219
William L. Rutherford
18
Elements of Law as They Relate to Teacher
Education Evaluation 237
Joan L. Curcio
19
We Can Get There from Here 251
Mary F. Berney and Jerry B. Ayers
Appendix 255
Author Index 259
Subject Index 263
NCA1E Standards Index 269

CONTRIBUTORS
Jerry B. Ayers is Director of the Center for Teacher Education Evaluation,

Associate Dean, and Professor of Education, College of Education, Tennessee
Technological University. Fonnerly he was associated with the University of
Georgia and Lenoir-Rhyne College. He holds the Ed.D. in Science Education
from the University of Georgia. He is the author of over 100 published papers
and the co-editor of Teacher Education Evaluation. His current research
interest is in the development and implementation of models for the evaluation
of teacher education programs.
Mary F. Berney is a researcher with the Center for Teacher Education

Evaluation and Assistant Professor of Educational Administration, College of
Education, Tennessee Technological University. Fonnerly she was a researcher
and student retention coordinator at the University of Maine at Machias,
Coordinator of the Center for the Computer in Education at the University of
Dayton, and a high school teacher of English. She holds the Ed.D. in
Educational Leadership from Western Michigan University. She has been
involved in a number of research studies and is the author of numerous papers
and reports. Hel current research interests are in the areas of program evaluation
and the role of women in higher education.
John A. Centra is Professor and Chair of the Higher Education Program,

Syracuse University. Formerly affiliated with Michigan State University,
SUNY College at Buffalo, and Alfred University, he holds the Ph.D. from
Michigan State University with major emphasis in counseling and psychology.
He is a consulting editor for several journals and has served as a consultant to
over 150 colleges, universities, and other organizations and is the author of over
75 publications including Evaluating Teaching for Tenure and
Promotion. His current research interest is in the area of faculty evaluation for
improving instruction.
Paul A. Clark is Director of Teacher Education and Professor of Education at

Milligan College. During the past 25 years he has held a variety of posts at
Milligan. Prior to joining the faculty at Milligan he was associated with several
institutions in Kentucky. He holds the Ed.D. from the University of Kentucky
and completed post doctoral work at the University of Tennessee. During 1987-
88 he served as a visiting professor in the Center for Teacher Education
Evaluation and helped develop the Accreditation Plus Model. His current
research is on the evaluation and improvement of teacher education programs in
small liberal arts colleges.
James R. Craig is Professor of Psychology, College of Education and

Behavioral Science, Western Kentucky University. He formerly served on the
faculty at Drake University. He holds the Ph.D. from Iowa State University
x
with a major in experimental psychology. For the past three years he has been
involved with the development and field testing of assessment devices for use in
the evaluation of teachers in the state of Kentucky. He is the author of over 75
papers and three books including Methods of Psychological Research.
His current research interest is in the area of teacher evaluation.
Joan L. Curcio is Assistant Professor, Department of Educational Leadership,

University of Florida. She served as a visiting professor at Virginia Polytechnic
Institute and State University and Adjunct Professor at Corpus Christi State
University. She received her Ed.D. from Virginia Polytechnic Institute and State
University where she was involved in a study of the legal rights and
responsibilities of school teachers. She has made numerous presentations and is
sought after as a consultant on legal issues affecting the schools. Her current
research interest is in this area.
Edward D. Garten is Director of University Libraries and Professor, The

University of Dayton. Formerly he was associated with the libraries at
Tennessee Technological University, Northern State College, and Moorhead
State University. He holds the M.L.S. from Kent State University and the
Ph.D. in Higher Education Administration from The University of Toledo.
Additionally, Garten is an Adjunct Professor in the School of Library Science,
Kent State University, where he teaches both library foundations and library
management. He is a prolific author and has been involved in a variety of
accreditation studies for regional associations and for professional organizations.
His current research interest is in the use of technology to improve library
services.
William J. Gephart served as Chief of Staff at The Evaluation Center,

Western Michigan University, Kalamazoo, Michigan; Senior Researcher with
the Center for Teacher Education Evaluation,Tennessee Technological
University; and Director of the Center on Evaluation, Development, and
Research at Phi Delta Kappa. He has been a faculty member at The Ohio State
University and the University of Wisconsin-Milwaukee. He received the Ph.D.
from The Ohio State University in 1965. He has authored or edited over 200
articles, books, and meeting presentations on research methodology and
evaluation. He is co-editor of Teacher Education Evaluation.
Edell M. Hearn is Dean and Professor of Curriculum and Instruction, College

of Education, Tennessee Technological University. Formerly he served on the
facuIty of The University of Tennessee-Knoxville and The University of
Tennessee-Martin. He received the Ed.D. from The University of Tennessee-
Knoxville with an emphasis in curriculum and instruction, and has served in a
variety of roles with the National Council for Accreditation of Teacher
Education. He is widely sought as a consultant for matters related to
Xl
accreditation and the governance of teacher education programs. His current

research interest is improving programs for the preparation of teachers.
William E. Loadman is Professor and Coordinator of Measurement,

Evaluation, and Research in Teacher Education, College of Education, The Ohio
State University. He was formerly at Indiana University, the Michigan State
Department of Education, and with the Lansing Michigan Public Schools. He
received his Ph.D. in Educational Psychology from Michigan State University
in 1971. His current research interests are applied research and evaluation
methodology with content focus on educational assessment of students and
programs.
Joyce R. McLarty is Director, Assessment Programs, Test Development

Division, of the ACT Programs in Iowa City, Iowa. Formerly she served as
Assistant Commissioner, Tennessee State Department of Education; Director,
State Testing Evaluation Center, Tennessee State Department of Education;
Research Associate, Riverside (CA) School District; Research Associate,
Neuropsychiatric Institute (UCLA); and Evaluation Specialist with a variety of
projects. She holds the Ph.D. from the University of California, Los Angeles,
with specialization in research and evaluation. She has been published widely
and made numerous presentations at professional meetings. Her current research
interest is in improved methods for assessment
Roger S. Pankratz is Associate Dean for Instruction and Professor of

Education, College of Education and Behavioral Sciences, Western Kentucky
University. He is also Director of the Center for Excellence in Teacher
Education and Professional Development at Western. Formerly he was a faculty
member at Kansas State Teachers College and Tabor College, and he has been a
secondary teacher of science and mathematics. He holds the Ph.D. from The
Ohio State University with major emphasis in science education. He has been
associated with a number of research projects in higher education and has been
published widely. His current research interest is in the development and
evaluation of a knowledge base for teacher education programs.
Lawrence M. Rudner is founder and President of LMP Associates and

Director of the ERIC Clearinghouse on Tests, Measurement, and Evaluation.
Previously he was a Senior Associate with the U. S. Department of Education,
Office of Educational Research and Improvement, and the National Institute of
Education; Senior Research Scientist at the Gallaudet University Research
Institute; and a mathematics teacher in Massachusetts. He holds the Ph.D. in
Psychology from Catholic University with emphasis in evaluation and research
methods. He is the author of numerous publications and presentations at
national meetings. His current research interest is in improved means for the
testing and assessment of teachers.
xii
WiUiam L. Rutherford is Director of Placement and Associate Professor of

Curriculum and Instruction, College of Education, The University of Texas-
Austin. Formerly he was associated with the Research and Development Center
for Teacher Education at The University of Texas, the University of Chicago,
and Tarkio College. He is the author of 15 books or chapters in books, a variety
of papers in professional journals, and numerous monographs. He received his
Ed.D. from North Texas State University. His current research interest is in the
utilization of educational innovations.
J. T. Sandefur is Dean and Professor of Education, College of Education and

Behavioral Sciences, Western Kentucky University. Formerly he served as Dean
of the Graduate School at Western, as an administrator and faculty member at
Kansas State Teachers College, and as a teacher and principal in the Kentucky
schools. He received the doctorate from Indiana University. He is a past
president of the American Association of Colleges for Teacher Education and was
the recipient of the 1988 Edward C. Pomeroy Award for contributions to teacher
education. He has worked actively to improve the education of teachers through
his various roles with the National Council for Accreditation of Teacher
Education. He has been published widely and is sought as a consultant on
matters of accreditation and program evaluation.
Robert L. Saunders is Dean Emeritus, College of Education, Memphis State

University. He served as Dean and Professor of Educational Administration at
Memphis State from 1970 until 1988. Formerly he was associated with Auburn
University and has been a high school science teacher and principal. He is the
author of numerous papers and monographs and three books. He received his
Ed.D. from Auburn University. During his tenure in higher education he served
as President of the American Association of Colleges for Teacher Education and
was actively involved in the accreditation movement, serving in various
capacities with the National Council for Accreditation of Teacher Education. In
retirement he is active as an educational consultant.
Nancy L. Zimpher is an Associate Professor of Education, Department of

Educational Policy and Leadership, College of Education, The Ohio State
University. She completed the Ph.D. at The Ohio State and has been associated
with the institution for the past 16 years in such roles as director of field
experiences and of the undergraduate program in teacher education. Her research
emphasis is in t/le professional development of teachers from their recruitment
and selection for teacher programs through their preservice, induction, and career
development in the profession. She has also studied leadership in schools,
colleges, and departments of education; program evaluation and assessment; and
the design and development of teacher education programs. She has published
widely in these areas.
Acknowledgments
The editors wish to thank a number of people for assisting us in the completion of
this project. Our staff, support personnel at Tennessee Technological University, the
authors, the publisher's staff, and our families contributed to the effort in various
ways. We were blessed with such an abundance of excellent material from the
authors that we had to make difficult choices about what had to be omitted from this
final version, but we want to take this space to make public our thanks to the people
whose assistance made it possible for us to complete this book.
Joni E. Johnson typed the greater part of both the draft and the final version.
She takes pride in having learned to trick the computer into producing what we
wanted rather than what it thought we needed and we are happy that she did. We also
appreciate her constant quest for perfection and her cheerful, professional attitude.
Graduate assistants Lori A. Birdwell, P. Christine Sibert, Boolie S. Stephens,
and Teresa A. Thompson served ably as proofreaders, typists, researchers, and
indexers. They represent the best of the new generation of professional educators and
we were fortunate indeed to have their assistance on this project.
John E. James, Joni. E. Johnson, and Sandra K. Miles each provided some of
the gmphics for this text; we gmtefully acknowledge their expertise.
Patricia Eaves, Sharon A. Heard, and Edith A. Young, support staff in the
College of Education, also helped with numerous editing chores and their willingness
to take on the additional burden is appreciated.
Joel Seber and Carl W. Owens provided technical assistance. Dr. Owens was
most generous in sharing his office and his equipment as well as his time.
Linda Mulder, Jean Moore, and Roger Jones of the Tech library provided
assistance in checking references and compiling the Appendix to the text.
Special thanks go to Mark Gregory of Inacomp Computers in Nashville,
Tennessee. Mr. Gregory loaned a Macintosh computer and word processing package
to the staff in the Center for Teacher Education Evaluation for the production of the
final copy of this book.
While we assume responsibility for the appearance and content of the final
product, we gmtefuIIy acknowledge the painstaking proofreading done by Sharon
Heard and James Harper.
We thank the authors, not only for producing such excellent pieces initially, but
for their patience with our deadlines and our editing of their work as we first
expanded, then reduced the size of each chapter. The trends and issues described in the
papers are those which educators face daily; the proposed solutions or approaches are
practical and worthy of serious considemtion. Working with these authors has been a
positive educational experience for us.
Zachary Rolnik and his staff were very patient with our constant questions and
requests. We are gmteful for the professional assistance we received from them.
To Mary N. Ayers and James Harper, we can only say, "We hope the next one
will be easier." We do appreciate your support, and that of everyone else who was
involved in the project.
1
INTRODUCTION
J. T. Sandefur
Western Kentucky University
American's ability to compete in world markets is eroding. The

productivity growth of our competitors outdistances our own.
The capacity of our economy to provide a high standard of living
for all our people is increasingly in doubt. As jobs requiring little
skill are automated or go offshore and demand increases for the
highly skilled, the pool of educated and skilled people grows
smaller and the backwater of the unemployable rises. Large
numbers of American children are in limbo--ignorant of the past
and unprepared for the future. Many are dropping out--notjust out
of school--but out of productive society.
These are not my words. They are a direct quote from the Executive
Summary of the Carnegie Forum Report on Education and the Economy entitled
A Nation Prepared: Teachers for the 21st Century (p. 2, 1986).
This report was motivated by four purposes:
1. To remind Americans, yet again, of the economic challenges pressing us on

all sides;
2. To assert the primacy of education as the foundation of economic growth,

equal opportunity and a shared national vision;
3. To reaffirm that the teaching profession is the best hope for establishing
new standards of excellence as the hallmark of American education; and
4. To point out that a remarkable window of opportunity lies before us in the

next decade to reform education, an opportunity that may not present itself
again until well into the next century.
Although the Carnegie Report was published in 1986, 1984 may well be
remembered as the year of the "Reports on Education" and the year that initiated
2
what some are now beginning to call the educational reformation. Following
years of increasing public concern about the quality of education of America's
youth, the nation was galvanized to action by a series of reports, chief of which
was the report entitled A Nation at Risk: The Imperative for
Educational Reform (1983). That report confirmed the public's conviction
that education was in desperate need of reform. The report, which was brief,
succinct, and well written, made effective use of emotion-laden words and
phrases. For example, the title, "A Nation At Risk." brought even the most
complacent to attention. Repeated reference to "the rising tide of mediocrity" and
the statement, "If an unfriendly foreign power had attempted to impose on
America the mediocre educational performance that exists today, we might well
have viewed it as an act of war" (p. 5) brought the public's concern to a fervent
pitch. As a result, the report is seen to be the capstone of an educational reform
movement and the impetus for states to legislate and mandate all sorts of
educational reforms. The result in many states was legislation to test both
students and teachers, to increase the length of the school day, to cut out frills
and to stress basic skills, to develop career ladders for teachers, to limit athletics,
to develop beginning teacher programs, and to initiate or implement dozens of
other reforms.
As a result of the emphasis on evaluation by both accreditation agencies and
the so-called reform movement, universities preparing teachers have eagerly
sought assistance in developing and implementing evaluation programs of their
graduates. For years much of that assistance has come from Tennessee
Technological University under the able leadership of Dr. Jerry B. Ayers and his
staff. The leadership continues with the publication of the book, A Practical
Guide to Teacher Education Evaluation.
The content of the book is highly appropriate to the needs of universities.
For example, the knowledge base of teacher education is a primary concern of
institutions preparing teachers. Personnel want to know how it is identified,
explicated and implemented. Other primary concerns of teacher education covered
include evaluation issues of admissions, field experiences, student performance
outcomes, surveys, follow-up programs, faculty and structure of the governance
unit. These and other significant topics have been covered by recognized experts
in teacher education evaluation.
There can be no doubt but that the book will be warmly received by the
teacher education community. The editors and authors should be commended for
their contribution to the improvement of teacher education.
REFERENCES
Carnegie Forum on Education and the Economy. (1986). A nation

prepared: Teachers for the 21st Century. New York: Author.
National Commission of Excellence in Education. (1983). A nation at

risk: The imperative for educational reform. Washington, DC:
U. S. Government Printing Office.
2
BACKGROUND FOR TEACHER EDUCATION PROGRAM

EVALUATION
Jerry B. Ayers and Mary F. Berney

Tennessee Technological University
The need for improved evaluation of teacher education programs has been well
documented over the past two decades. In the past five years, most states have
mandated substantial reforms in teacher education progmms. These reforms have
included:
o strengthening the general (liberal arts) education for prospective teachers,
o developing of five year programs leading to initial licensure,
o developing alternative routes to licensure for liberal arts gmduates, and
o developing and implementing total field base programs operated within a

school district
Recent work by the Southern Regional Education Board indicated that too
little program evaluation was implemented to show if these changes in teacher
preparation were really making a difference (SREB, 1988). States such as
Florida, Virginia, Oklahoma, Louisiana, West Virginia, and Georgia are
examining ways to evaluate teacher education. However, there is a dearth of
practical methods to accomplish the needed evaluations in a systematic and
ongoing manner.
Daughdrill (1988) recently pointed out that assessment and evaluation are
doing more for higher education than any other development in recent history.
The Carnegie Foundation (1982) emphasized a need for institutions of higher
education to "reaffirm and strengthen self regulation." Other national
commissions and scholars echoed this stance. Evaluation is a key to the reform
process. This book is designed to meet the evaluation needs of institutions of
higher education relative to improving programs and the needs of society for
mandated accountability.
4
Staff at the Center for Teacher Education Evaluation at Tennessee

Technological University devoted nearly three years to studying various models
and means of evaluating teacher education programs. The result of this effort
was the Accreditation Plus Model (Ayers, Gephart, & Clark:, 1988) based on the
use of professional judgment to examine programs. The Model is described in
Chapter Three.
This book provides the reader with a selection of ways to evaluate all
aspects of a teacher education program in both a formative and summative
manner. The basic paradigm for evaluation is the Accreditation Plus Model.
Applying the model to formative aspects of a program results in improved
programs for the preparation of teachers. Applying the model in the summative
stages of a program provides the answer to the question of how good a particular
program is for the preparation of teachers.
Accreditation (or program approval) is the vehicle for this evaluation
process. To achieve accreditation or approval, however, a program must undergo
a variety of separate evaluations. This book can be an aid in the process of
establishing an ongoing system of evaluation. If an institution already has a
system for evaluation in place, the materials will provide a base for improving
these processes.
The remainder of this chapter focuses on how to use the book to help meet
accreditation and approval standards through the use of evaluation. Particular
attention is placed on the use of formative evaluation as an aid to meeting
approval standards. Emphasis throughout the book is on the evaluation of four
or five year programs for the preparation of teachers for initial licensure in grades
K through 12. The basic plans contained in the book could be used, with
modifications, for the evaluation and subsequent accreditation of advanced
programs for the preparation of teachers.
This opening chapter departs somewhat from those found in traditional
books on program evaluation in that the last section contains essential
information for the evaluation of the evaluation system.
Throughout the development of this book, the editors have used as a guide
the statement of J. T. Sandefur (1982), leading authority on teacher education
program evaluation, a past President of the American Association of Colleges
for Teacher Education, and a past Chair of the National Council for Accreditation
of Teacher Education. He pointed out that there was a need for more work on the
development of
effective, functional systems that can be used to evaluate graduates of

teacher preparation programs and to provide developmental feedback that
can be used for program development and improvement.
BACKGROUND FOR THE BOOK
The basic tenet of the Accreditation Plus Model is that if a program is

accredited or approved, it is a quality program. To become accredited, an
institution must meet a given set of standards or guidelines. In order to meet
these standards, specific evaluation tasks must be undertaken (e.g., follow-up of
5
graduates after they enter the profession). By undertaking other evaluation

processes (e.g., an examination of the system of governance that controls a
teacher education program), an institution can determine if a given set of
standards or criteria for approval will be met.
National, state, regional, and local accreditations or approval agencies
govern teacher education programs. The accreditation or approval standards at the
national, state, and regional levels are well defined and readily available.
Approval processes by individual governing boards or (e.g., Regents, Board of
Trustees, or Board of Visitors) vary greatly and can be difficult to obtain.
Generally, if a program meets national, state, or regional requirements it will
meet those of governing boards. This chapter will concentrate on using the
Accreditation Plus Model at the first three levels.
National Accreditation
The National Council for Accreditation of Teacher Education (NCATE) is

currently the only organization approved by The Council on Postsecondary
Accreditation (COPA) of the U. S. Department of Education to accredit teacher
education programs. NCA TE has been in existence for over three decades and
has developed a rigorous set of standards for the approval of teacher education
programs. Recently the standards and the total structure of NCA TE were
redesigned (Gollnick & Kunkel, 1986). The revised standards became effective
with all institutions seeking initial accreditation or reaccreditation after 1987
(NCATE,1987).
Prior to building a practical approach for the application of the Accreditation
Plus Model, the NCATE pre-conditions and standards were examined in detail to
ascertain all of the evaluations that would be needed to meet the standards for
accreditation. A variety of evaluations and assessments are needed to fully meet
the intent of the NCA TE accreditation process. Chapters 4 through 15 are
devoted to specialized topics, while Chapters 16 through 18 supplement the
main evaluation process.
The National Association of State Directors of Teacher Education and
Certification (NASDTEC) developed a set of standards for the approval of teacher
education programs. The NASDTEC, although national in scope, has not been
approved by COPA. Several states utilize the NASDTEC standards for the
approval of teacher education programs so the NASDTEC standards were
examined in depth to determine types of evaluations that might be required
beyond those of the NCATE standards. No evaluations beyond those required of
NCATE were required by NASDTEC.
Regional Accreditation
Regional accrediting associations are an outgrowth of the efforts of the

University of Michigan to develop a system of accrediting high schools or
certifying that high schools were preparing students who were qualified for
admission to higher education. From this beginning in the 1890s, the North
Central Association of Colleges and Schools developed. By the 1930s six
6
regional accrediting associations for both schools and colleges were in operation.
These six COPA recognized regional accrediting associations include:
o Middle State Association of Colleges and Schools

o New England Association of Schools and Colleges
o North Central Association of Colleges and Schools
o Northwest Association of Schools and Colleges
o Western Association of Schools and Colleges
o Southern Association of Colleges and Schools
Regional accrediting agencies are responsible for the approval of a total

institution and do not specifically address particular programs such as teacher
education. However, a teacher education program at a given institution must
meet all of the criteria of the regional accreditation agency, otherwise the
accreditation of the total institution could be in jeopardy.
The standards, manuals, and guides for each of the six regional accrediting
associations were examined in depth to locate evaluations that might be needed
to ensure accreditation. These additional needs were included in the particular
relevant chapters of this book.
State Approval
Some type of approval process for teacher education programs exists in each
of the fifty states. The Center for Teacher Education Evaluation staff studied
guidelines for the evaluation and approval of teacher education programs from all
fifty (Ayers, 1988). Six states employed the NCATE standards, and seven states
used NASDTEC standards. Several states use a combination of standards. For
example, in Tennessee, there is a state approval process for teacher education
programs; however, those institutions that are NCATE approved are
automatically approved by the State of Tennessee. It is anticipated that more
states will adopt the NCATE standards (or some modification) for the approval
of teacher education programs in the near future.
The standards for approval of teacher education programs by states that did
not employ NCATE and/or NASDTEC guidelines were examined in depth to
determine the needs for formative and summative evaluation. No additional
evaluation needs were found in these guidelines. In some instances, a state
might require a particular test (Le., California requires students being admitted to
formal study in teacher education to perform at a particular level on the
California Basic Skills Test).
The Program Accreditationl Approval Process
The process is basically the same whether an institution is seeking program

accreditation/approval at the national, state, or regional levels. At the present
time NCATE encourages joint visits and joint accreditation process with other
COPA approved agencies or with state departments of education.
7
Planning is the key to success in conducting a program

accreditation/approval process. Planning should be ongoing and continuous.
[See chapters by Craig and Rutherford for additional details.]
Accreditation agencies and state departments of education have developed
handbooks of standards and other documents needed to plan for accreditation or
approval. These documents should be carefully reviewed and followed
throughout the process of seeking accreditation/approval. The implementation
of a systematic plan of evaluation of all components of the teacher education
program will lead to a continuous self-study process.
After the self-study has been completed, a team of external assessors will
visit the campus and validate their report At that point it will be necessary to
have available background reports and information that will support the claims
of the institution. The external team will prepare a report on the institution and
possibly (depending on the agency) make a recommendation for
accreditation/approval of a program. This report will be reviewed by the central
staff of the accrediting agency or a board of advisors or directors. A
recommendation will then be made to accredit/approve a teacher education
program and reports will be given to the unit
CONTENTS OF THE BOOK
Part II is a practical guide to the evaluation of programs for the preparation

of teachers. It was pointed out by Hord, Savage, and Bethel (1982) that many
models have been developed for evaluating teacher education programs; however,
little has been done to develop means of applying these models of evaluation
problems. This book addresses some of the concerns that have been expressed in
the literature. It provides a practical framework for implementing the
Accreditation Plus Model. In Chapters 4 through 15 a separate set of variables
related to the evaluation of a program for the preparation of teachers is examined
in depth. Following is a summary of the major areas of emphasis.
Knowledge Base
The selection of a knowledge base has become a primary issue in the past
several years. The 1987 NCATE standards emphasize the development of a
knowledge base to guide the operation of a teacher education program. To this
end, Chapter 4 provides a description of how to select a knowledge base that can
become the foundation for program change and improvement.
Evaluation of the knowledge base is described in Chapters 4 through 11.
Chapter 4 provides specific information and suggestions for the evaluating the
knowledge base that was used in a particular teacher preparation program.
Quality Controls in Teacher Education Programs
Throughout a teacher education program there is a need for quality controls

to ensure that all aspects are at an acceptable standard. Chapter 4 provides an
8
overview of the techniques that can be used to detennine if an acceptable level of

quality is being maintained in the various aspects of the program.
Candidates for Teacher Education Programs
The selection of students for admission to teacher education programs is one

of the most crucial elements in the total process. A variety of techniques can be
used to select students. The most common techniques include the use of
standardized tests to measure basic skills in verbal and quantitative areas. Beyond
these basic techniques, interviews, personality tests, rating devices, and
recommendations can be used to examine the qualities of preservice teachers.
Laboratory and Field Experiences
Laboratory and field experiences are among the most important aspects of
the preparation of future teachers. For this reason, emphasis was given in
Chapter 7 to the various methods for evaluating different types of laboratory
situations (e.g., evaluation of students observing in the classroom to the level of
student teaching in which an individual assumes the role of the teacher for an
extended period).
Outcomes Assessment
The assessment of perfonnance outcomes in teacher education is a major

issue in many states. One of the most common means of assessing student
outcomes in teacher education programs is the administration of the National
Teacher Examinations. The infonnation presented in Chapter 8 can be combined
with that contained in Chapters 4, 11, and 16 to develop a plan for detennining
the value of the teacher preparation program.
Follow-up Evaluation
Mail follow-up studies are widely used to gather evaluation data on teacher
preparation programs. Chapter 10 includes techniques to use in the development
of questionnaires specific for a given program. True follow-up evaluation in
teacher education requires the use of observation instruments in the classrooms
of graduates. Chapter 11 includes a description of various techniques that can be
used to effect studies of the follow-up of teacher education graduates. Follow-up
is a key to improving teacher education programs and for providing the needed
feedback for program development.
Governance of Teacher Education Programs
An integral part of fonnative evaluation is an examination of the structure

of the education unit. The education unit, which is frequently referred to in this
book as the teacher education program, controls all teacher education on the
9
campus of an institution of higher education. Chapter 12 provides an avenue to

evaluate the locus of control of the teacher education programs.
Program Resources
Resources are essential to the operation of a teacher education program.

Chapter 9 addresses evaluation of the faculty of a teacher education program.
The chapter includes an examination of the three major areas of activity (i. e.,
teaching, research, and service) and suggested instruments for conducting
evaluations. The physical, financial, and library resources are all essential to the
operation of a teacher education program. Generally a Dean, Chair of Education,
or the individual responsible for assessment on a college campus has not received
training in how to evaluate physical facilities, financial resources, or library
resources. Chapters 13, 14, and 15 were prepared for use by the non-specialist in
evaluating resources. These chapters will provide the non-specialist with
essential information for working with the institutional financial officer(s),
physical plant specialist(s), or institutionallibrarian(s) in evaluating the teacher
preparation program.
Supplemental Aids for Evaluation
Part III of the book was designed to provide supplemental information about
a continuous system of formative and summative evaluation. The "Plus" part of
the Accreditation Plus Model was designed to provide additional information
about a teacher education program that was beyond the normal evaluations
conducted as a part of a continuous accreditation study. Chapter 16 provides a
description of classic models that can be used in evaluation and examples of the
use of selected models. The use of evaluation information for program
improvement is essential. Chapter 17 provides a guideline for using evaluation
information for program improvement. The chapter focuses on faculty
involvement and provides a checklist for using information for program
improvement. Chapter 18 includes a summary and discussion of some of the
major legal questions that an evaluator will be confronted with in conducting
various types of studies. The information is of particular use to those
individuals who do not have an extensive background in legal affairs. Chapter
19 provides an overview of the future. It includes a brief critique of Chapters 1
through 18 and examination of future developments in the field of teacher
education program evaluation.
USE OF THE BOOK
The book was designed to be used either in the development of a complete

evaluation plan or to supplement existing plans. It is suggested that the
following activities take place in the order shown.
1. Review the materials contained in Chapters 3 and 4.

10
2. A key to initial planning is faculty involvement. Review Chapters 9 and

17.
3. In order to become familiar with (or review) the legal status and
requirements related to program evaluation, review Chapter 18.
4. Review the existing evaluation plans for the teacher education program, the
types of data available from existing sources, and additional data needed
(Ewell & Lisensky, 1988; Jones, 1988).
5. Examine the administrative structure of the teacher education unit. Review

Chapter 12 in order to evaluate the structure of the unit.
6. Systematically review Chapters 4 through 15 in order to determine where

gaps may exist in the evaluation procedures for the teacher education
program or to select a complete system for evaluating each of the major
components. This needs to be accomplished based on the available
resources.
7. Determine if there are any additional areas of evaluation that have not been
met. Review Chapter 16 on models and modeling. This will serve as a
base for the development of additional evaluation tools.
8. Develop a system to evaluate the evaluation process. Review the last

section of this chapter.
Meeting Accreditation Standards
Although the materials contained in this book were built around NCATE
standards, due emphasis was given to regional accreditation standards and to the
program approval requirements of the various states. If an institution follows
the evaluation procedures contained in this book, the program for the preparation
of teachers should meet the accreditation/approval standards of any agency.
EVALUATION OF THE EVALUATION
Evaluations must be evaluated. The Standards for Evaluation of

Educational Programs, Projects, and Materials, compiled by the Joint
Committee on Standards for Educational Evaluation, can be used to evaluate an
evaluation. Each of the 30 standards described in the book represents one of four
attributes: utility, feasibility, propriety, and accuracy. Daniel Stufflebeam, now
chair of the committee, said that the members believe that the order represents
the hierarchy of importance of the standards. Every effort must be made to
ensure that any evaluation will be utilized, that it will be practical and cost-
effective, that it will be conducted ethically, and that it will be technically
accurate in design and in the way it is conducted. Meta evaluation or
11
evaluations of the evaluation, are conducted along the same lines as any other
evaluation. The standards were written as guiding principles and
contain cautions and warnings against the past mistakes of evaluators,

and identify practices generally agreed to be unacceptable. And they
propose procedures established as effective and ethical (Joint
Committee, 1981, p. 9).
Each is presented in the same format
Descriptor, or name (e.g., Standard Al Audience Evaluation)

Standard, the definition written as a statement of "should"
Overview, the explication of the standard, including rationale
Guidelines, procedural suggestions, not mandates
Pitfalls, lists of difficulties or mistakes already made that are not easily
recognized
Caveats, trade offs that may be necessary in some applications; mistakes
based on overzealous applications of standards
Illustrative Case, shows how the standard might be applied, describes
setting, instances in which the standard is not met and examples of
corrective actions; usually based on real cases
Analysis of the Case, a discussion which suggests alternative
approaches, notes weaknesses, and serves as a summary
The standards, taken together, represent a working philosophy of evaluation,

an attempt to foster understanding of the process of evaluation and to ensure the
conduct of useful, practical, ethical, and accurate evaluation ..
SUMMARY
Each of the chapters described here can be used alone or in any combination
that will be of greatest benefit to the user. Someone who has experiences with
accreditation may choose to begin with an overview of the changes in the
NCA TE Standards. A person or committee with less experience may want to
read through the chapters in sequence. Individuals wishing to look at aspects of
a program which were not previously evaluated should begin with those, by
topic, or perhaps with the chapter on models to see what precedents exist.
Above all the contributing authors and the editors want this to be a practical
guide to program evaluation for teacher educators.
REFERENCES
Ayers, J. B. (1988). Review of state guidelines for the approval of

teacher education programs. Unpublished manuscript, Tennessee
Technological University, Center for Teacher Education Evaluation.
12
Ayers, J. B., Gephart, W. J., & Clark, P. A. (1988). The Accreditation Plus
Model. Journal of Personnel Evaluation in Education, 1, 335-
343.
Carnegie Foundation for the Advancement of Teaching. (1982). The control
of the campus. Princeton, NJ: Princeton University Press.
Daughdrill, J. H. (1988, January 27). Assessment is doing more for higher
education than any other development in recent history. The Chronicle
of Higher Education, 34(20), A52.
Ewell, P. T., & Lisensky, R. P. (1988). Assessing institutional
effectiveness. Washington: Consortium for the Advancement of Private
Higher Education.
Gollnick, D., & Kunkel, R. (1986). The reform of national accreditation. Phi
Delta Kappan, 68, 310-314.
Hord, S. M., Savage, T. M., & Bethel, L. J. (1982). Toward usable
strategies for teacher education program evaluation. Austin,
TX: The University of Texas, Research and Development Center for
Teacher Education.
Joint Committee on Standards for Educational Evaluation. (1981). Standards
for evaluations of educational programs, projects, and
materials. New York: McGraw-Hill.
Jones, D. W. (Ed). (1988). Preparing for NCATE: Criteria for
compliance: external evaluation. Chicago: North Central
Association of Colleges and Schools.
National Council for Accreditation of Teacher Education. (1987). Stall dards,
procedures, and policies for the accreditation of profe!lsional
education units. Washington: NCATE.
Sandefur, J. T. (1982). Teacher education's evaluation of graduates: Where are
we going and how do we know when we get there? In S. M. Hord, T. V.
Savage & L. J. Bethel (Eds.). Toward Usable Strategies for
Teacher Education Program Evaluation. Austin, TX: The
University of Texas, Research and Development Center for Teacher
Education.
Southern Regional Education Board. (1988, November). State-level
evaluation of teacher education programs in the SREB states.
Atlanta: Author.
3
THE ACCREDIT ATION PLUS MODEL
Jerry B. Ayers and William J. Gephart

Paul A. Clark
Milligan College
Much of the blame for the present condition of education in the nation has been
placed on teacher education. Teacher education is in need of revision and reform.
However, there is a paucity of knowledge about what the content of teacher
education programs should be and about the relationship between preparation
programs/knowledge base and effective teacher performance. Basic research and
evaluation data by institutions preparing teachers must be collected and analyzed
in order to overcome the problems associated with teacher education reforms.
The value of systematic evaluation in improving teacher education programs
cannot be underestimated. The standards of such groups as the National Council
for Accreditation of Teacher Education, regional accreditation associations, and
state departments of education require that teacher education programs be
accountable for their products, i.e., the graduates of the programs. To that end,
systematic formative and summative evaluations must be undertaken.
A variety of models can be used to evaluate various aspects of teacher
education curricula. There is, however, a dearth of comprehensive evaluation
models. The central mission of the Center for Teacher Education Evaluation is
the development, refinement, and field testing of evaluation models and
materials. To remediate that condition, the Center staff developed the
Accreditation Plus Model. This Model is a viable entity that can be used as a
vehicle for the evaluation of teacher education programs. The model has become
the basis for a practical approach to the evaluation and improvement of teacher
education programs that is outlined in this book. An article describing the
Model was published under the same title as this chapter in 1988 in the Journal
of Personnel Evaluation in Education, 1, 335-343. The article is
reprinted as the remainder of this chapter with an updated schematic of the model.
Evaluation of education has an extensive history. It dates to work done by
the University of Michigan in starting the North Central Association of
Colleges and Secondary Schools. The association evaluated secondary schools to
14
help the University make its matriculation decisions (circa 1895). From the tum
of the century to the mid-1950s, diverse factors evolved four general approaches
to educational evaluation. Those four are described by Madaus, Stufflebeam, and
Scriven in "A Historical Overview" (pp. 3- 22) of their book, Evaluation
Models. Those evaluation forms were:
I. The educational objectives focus: curriculum change was being pushed in

ways that made attention to educational objectives of central importance.
2. New developments in testing, especially norm-referenced instrumentation, as

it evolved for the classification of problems in the military. That gave us
the technology for measuring effects of the new curricula.
3. The accreditation or professional-judgment approach: an effort to determine

the worthiness of educational agencies, products, and personnel.
4. The use of field experiments to determine the quality and generalizability of

the new curricula.
Madaus, Scriven, and Stufflebeam made this statement
The foundation of accreditation (no. 3 above) or the professional-

judgment approach to evaluation can be traced directly to the
establishment of the North Central Association of Colleges and
Secondary Schools in the late 1800s. The accreditation
movement did not, however, gain great stature until the
1930s when six additional regional accrediting
associations were established across the nation. Since
then the accrediting movement has expanded tremendously
and gained great strength and credibility as a major means
of EV ALUA TING the adequacy of educational institutions
(p. 6; emphasis added).
The mission of the Center for Teacher Education Evaluation (CTEE) is the
improvement of teacher education in the State of Tennessee and elsewhere. A
central objective in the accomplishment of that mission is the development of a
way or ways to improve teacher education evaluation. As the Center's activities
got under way, the Center staff began a literature search to find applicable
evaluation models or forms of evaluation.
Finding references to "evaluation models" or approaches was not difficult.
In a few months about 40 references to evaluation models were found. The
Center's problem changed. We no longer sought bibliographic references to still
another model. Rather, we sought ways of applying specific evaluation models
to improvements in the education of teachers.
In years past we have chided evaluation theorists about proliferation of
evaluation models. The Center for Teacher Education Evaluation realizes that if
the proliferation of evaluation models is counterproductive when others do it, it
15
is equally abhorrent when we do it. That set the stage. Which of these
evaluation approaches could (1) help us understand the evaluation process central
to our work, and (2) help us reduce the number of extant evaluation models?
The "dimensions of value" useful in assessing the quality of individual
evaluation models include: (a) complete versus incomplete models, (b) old
versus new models, (c) mature versus immature models, and (d) isomorphic
versus analogous models. At the same time we can conceive of types of models.
Here we focus on verbal, graphic, physical, and mathematical modeling. A
model (among other things) is a representative of the entity being represented. It
stands in for some thing. My right hand is a representation of my left hand.
For every part in my left hand there is a corresponding part in my right hand.
There is only one item, handedness, that keeps my left hand from being an
isomorphic model of my right hand. The latter is its mirror imagery character.
We sought a model that will help us understand the evaluation of teacher
education. We would like a complete, whole model, one that has some extended
history, and thus is aged and mature. The Center's charge is to improve teacher
education by improving our ability to evaluate teacher education programs. We
sought a model which has program evaluation at its core.
The recognition of "accreditation" as an evaluative approach helped delineate
and redirect the Center's task. Accreditation is a form of evaluation. It assists
people in making informed decisions in situations in which the relative worth of
competing options is difficult to measure. Madaus, Scriven, and Stufflebeam
speak to that point in their historical overview. Others who state or imply that
accreditation is a form or model of the evaluation process include R. Travers, W.
Webster, E. House, R. Stake, R. Floden, J. Sanders, and B. Worthen, whose
work on the subject are included in the Madaus, Stufflebeam, and Scriven text
cited previously.
A careful examination of the accreditation standards for NCATE (National
Council for Accreditation of Teacher Education) is impressive. NCATE's
standards are clustered in five categories:
o Knowledge Base for Professional Education;

o Relationship to the World of Practice;
o Students;
o Faculty; and
o Governance and Resources.
There are a total of 18 standards and 94 criteria for compliance distributed

within those five categories. There are also 10 "precondition" items that have to
be accomplished before a school can embark on the accreditation process. Any
teacher education agency that is in compliance with all those standards deserves
its hard-won accreditation. And more than that, any teacher education unit that
follows the accreditation approach to evaluation would be continually involved
in planning for change and improvement
NCATE went into a redesign program in 1986. Prior to the redesign
NCATE procedures had a distinct summative evaluation nature. Before that
time, NCATE's operation focused on the decision to accredit, to accredit with
16
provisions. or to deny accreditation. The redesign shifted NCA TE to a slightly

(albeit significant) more formative evaluation character. This was accomplished
by instituting 10 preconditions and annual reports on selected variables. These
components. if followed to both the letter and spirit of the law. would produce
data-driven teacher education units that had the potential for continually
improving their programs.
The emphasis above focused on accreditation at the national level. This
should not be perceived as a negation of state. regional. and occasionally local
accreditation. As will be shown later. the administrative personnel of an
education agency needs information useful in determining the accreditation level
with which to start.
The Center for Teacher Education Evaluation has developed a model for the
evaluation of teacher education programs. It is an amalgamation of the
accreditation model for its core and "use-tailored evaluation plans" for evaluative
needs not met by accreditation. Because of that structure the Center staff referred
to it as the Accreditation Plus Model. And. almost before one could tum around.
a visitor to the Center used the appellation "A-Plus."
The Accreditation Plus Model
Accreditation is referred to by some as a "true" form of evaluation because it

assesses the worth of a school. college, or university's programs and personnel
to determine whether they are meeting minimum standards. In the process, it
can suggest avenues for change and improvement for teacher education programs.
The accreditation handbooks (state, regional, and national) include serious
evaluation criteria worthy of compliance. CTEE's staff proposes their use to
represent thoroughly the quality of any college or university teacher education
program.
Accreditation, as it is now practiced, is a passive process and a crisis
activity. It is done at the direction of the accrediting association and done to the
applying school. Accreditation, at the national level, generally has seven
components: a handbook of standards and procedures, a self-study, a team of
external assessors, a site visit, a report on the institution, review of that by a
panel of experts, and a final report and decision (see Madaus, Stufflebeam &
Scriven, 1983, pp. 249-250).
The CTEE staff, by following the NCATE redesign, will make
Accreditation Plus an active, continual, and formative process. This will be
done by: writing evaluative questions related to the 18 NCATE standards and
their 94 compliance criteria; delineating the data needs to answer those questions
and collecting and interpreting those data; by the continual collection of data for
annual NCATE reports; and by the same treatment applied to the ten
preconditions in the NCATE redesign. If these data are collected and interpreted
on a regular basis, the staff of a teacher education program will have the data
needed in planning for continuous improvement and change. This more active
evaluation stance should produce a more useful and higher quality evaluation.
This in tum may cause the institution to reach out for information beyond the
usual accreditation information.
17
At this point the education unit has a decision to make. If the education
unit is satisfied with the information generated via the accreditation process, then
that documents compliance. The design and planning work is done. What is left
is implementation and monitoring. The Accreditation core of the model has
been accomplished.
If, however, additional evaluative questions exist and, if the education unit
wants those items informed, "use-tailored evaluation" procedures will be planned
and implemented. This is the place for the Plus aspect of the Accreditation Plus
Model. And, this is the time to turn to the 40 or so extant evaluation models in
search of evaluation tools and techniques that will produce the desired evaluative
findings. The call here is for an informed eclecticism in the assembly of
evaluational procedures that will meet the additional evaluative needs. An
application of A-Plus will meet both the accreditation compliance information
and needs not handled in the accreditation evaluative information.
The subtle, unwritten policy that bigger programs are better is not the
position of the CTEE. The Accreditation Plus Model will be designed to be
helpful to serious small colleges, or agencies who want to improve their
evaluation expertise and practices. The assumptions of this model are
summarized below.
1. The NCATE standards, policies, and procedures, if fully employed in the

design and operation of a department, school, or college of education, would
guide the development of a quality teacher education program.
2. Thorough attention to and compliance with NCATE's 10 preconditions, the

compiling and filing of specified annual reports, the preparation of a serious
and thorough self-study, and a carefully conducted site visit will provide an
abundance of data on which a teacher education program can base continual
changes and improvement.
3. The atmosphere of accreditation is, but should not be, that of a crisis.
Threat is detrimental to productive change. The individuals and agencies
who direct accreditation should take all the steps necessary to change that
climate. Accreditation can and should be a team affair, a force for the future,
not a spectre from the past.
As the Center for Teacher Education Evaluation operates, every opportunity

will be seized to make Accreditation Plus a formative evaluation process. And
conversely, wherever possible the Center will take deliberate actions to reduce
negativity.
Components of The Accreditation Plus Model
The components of the A-Plus Model and their general relationships are
presented in Figure 1. Concurrent with this writing Center staffers are
developing flow charts that will show the work to be done in applying
Accreditation Plus. The sequence of presentation of the components of the
THE ACCREDITATION PLUS MODEL
o @ COMPLETION OF THE PROGRAM GUARANTEES The Ultimate Criteria:
PROGRAM El.EA\ENTS THAT THE CANDIDATE WIll HAVE. 9 PUPIL GROWTH OUTCOMES
1. Candidate selection 1. Earned 0 mojor in a recognized field of study 1. Academic growth
2. Program (e.g., physics, history, language arts); 2. Physicol, social. and emotional growth
3. Staff 2. Attained a liberal arts education; 3. Moster)" of language arts skills
4. Candidate outcomes 3. Mastered educational theory. principles, 4. Mathemoticolliteracy
5. EdJcational unit methods, and practices; 5. Learning skills
6. Follow-up 4. Performed according to standards in a set of
- 6. Productive citizenship
7. Pupil outcomes monitored field experiences; and
5. Obtained a provisional license to teach.
o 0
1 9 EVAWATION OF THE EVAWATION--
9 I. SEIKT THE LEVELCS) OF THE JOINT STANDARDS COMMITTEE
TEACHER EDUCATION PROGRAM
--A new one ACCREDITATION SOUGHT.
--A developing one II. UST THE EVAWATIVE QUESTIONS /
THAT STRUCTURE ACCREDITATION
AT THE SEIKTED LEVELCS).
@
,
EXISTENCE OF A TEACHER EDUCATION III. ESTABUSH DATA GENERATION AND
ACCREDITATION PLUS
PROGRAM NECESS;ATES ACCREDITATION REPORTING PROCEDURES.
IV. PLAN FOR IMPROVEMENT AND
APPLY ACCREDITATION PWS

I CHANGE.
Q
ADDmONAl EVAWATIONAL NEEDS
+
/
--One-time needs
--Ongoing evaluation needs
KNOW THE ACCREDITATION AGENCIES
o ' /
Accreditation o +
Levels National [ State Regional
I I ECLECTIC EVAWATION DESIGN
NS N.
o
UNDERSTAND THE ELEMENTS OF THE
Informed eclecticism 1
o Notional tD RESOURCES.
ACCREDITATION PROCESS. Clusters of extant evaluation models
51
State Handbook on standards, self stUdies. including systems models, goal based
teams of experts, site visits. reports @ / ~ models, naturalistic models, formative!
on unit, reviews by panel. reports on FORMATIVE SUMMATIVE summotive models, and others; and
Regional
decision EVALUATION EVAWATION Proven evaluation tools
....,. 1. TH ADa,.d..., .............

19
model has little consequence. The Accreditation Plus Model has been presented
to numerous audiences. Each started with a different component. There is some
logic for starting with the Ultimate Criteria (#1) and the Second Ultimate
Criteria (#2), Pupil Growth Outcomes and Teacher Candidate Outcomes. The
components of The Accreditation Plus Model are as follows:
1. Pupil growth outcomes. The focus here is on the elementary and

secondary school students who will be taught by the next class of beginning
teachers. We expect pupil growth in
1.1 Academic areas
1.2 Physical, social, and emotional domains
1.3 Language arts, reading, writing, speaking, and listening skills
1.4 Skills in mathematical problem-solving
1.5 Learning how to learn, and
1.6 Development of the skills needed to become productive citizens.
2. Teacher education candidates will have satisfactorily

2.1 Earned a major in an area of specialty of the particular institution
2.1.1 Such as physics, mathematics, history, home
economics, etc. (for future secondary teachers)
2.1.2 For elementary education candidates a major in language
arts sets here.
2.2 Completed the sequence of courses that defines the particular school's
liberal (or general) education.
2.3 Completed courses on
2.3.1 Educational theory, principles, methods, and procedures
for teaching.
2.4 Completed a carefully monitored set of field experiences
2.4.1 Starting with learning to observe and measure teaching
using several measurement systems
2.4.2 Progressing through roles of teacher aide, tutor,
supervised teaching, and beginning teacher.
2.5 Attained a provisional teaching certificate.
3. It moves next to the elements of teacher education programs.

3.1 Teacher education candidates.
3.2 Those candidates will work through a program defined by the staff as
the courses and experiences that qualify the teacher to graduate and to
receive provisional certification to teach.
3.3 A staff whose academic background prepares them to conduct courses
as prescribed by the educational unit policy and regulating personnel.
3.4 Candidate outcomes (See components 2.1-2.5).
3.5 An educational unit is the administrative and subject matter home of
members of the staff.
3.6 Follow up the studies engaged in to determine the program's quality,
and changes in the program that need exploring.
3.7 Pupil outcomes (See components 1.1-1.6).
20
4. An application of the Accreditation Plus Model starts with either a new

or a continuing teacher education program.
4.1 It starts with a mandate--thou shalt be accredited
4.2 It calls for expanded knowledge about the accreditation process and
agencies.
4.2.1 There are four levels of accreditation:
4.2.1.1 The national level (NCATE),
4.2.1.2 State accreditation agencies, some of which
are related with the national agencies.
4.2.1.3 Regional agencies--seven agencies that
assess program quality and focus on the total
scope of school's offerings.
4.2.1.4 Local agencies--some places in which
accreditation by local agencies is required
before permission to function is granted.
4.3 There are other indirect accreditation approval agencies, i.e.,
NASDTEC, COPA.
4.4 A decision is made here, a choice of level or combination of levels is
selected for the evaluation.
4.5 The conduct of the self-study including the development of the
evaluation program, and field testing it.
4.5.1 List the level(s) of accreditation sought.
4.5.2 List the evaluative questions that make up the
evaluation.
4.5.3 Establish data generation, analysis, interpretation, and
reporting procedure.
4.5.4 Plan for improvement and change.
4.6 The general elements of an accreditation effort:
4.6.1 A handbook of standards,
4.6.2 A self-study,
4.6.3 A team of external assessors,
4.6.4 A site visit,
4.6.5 A report on the institution,
4.6.6 Review by a panel of experts,
4.6.7 Final report and decision.
5. Application of the Joint Committee Evaluation Standards to

the evaluation plans and procedures.
6. If the teacher education unit is satisfied with accreditation compliance only,

the planning is finished. If there is a need and desire for evaluation beyond
the accreditation compliance, an additional set of evaluation plans will be
drawn up to provide the needed evaluation findings.
6.1 Categories of evaluation models are listed as resources and/or
reminders of the extent of our knowledge about the evaluation
process.
21
6.2 Eclecticism in the design of the evaluation procedures is urged. (We

know a lot about measuring techniques, about the objectives attended
to, about sampling procedures, etc. We should apply informed
eclecticism in choosing proven techniques and procedures for the Plus
side of Accreditation Plus.
6.3 Again, the Joint Committee standards for program evaluation will be
used to assess the quality of the selected techniques, tools, and tactics.
7. There are two general evaluative approaches:

7.1 Summative evaluation used to inform decisions about the relative
quality of program options.
7.1.1 Decisions about competing products.
7.1.2 Staffing decisions about assignment, promotion,
retention, and dismissal.
7.1.3 Summative evaluation is usually done by someone to
something or someone.
7.2 Formative evaluation is used to inform decisions about how a
program, a product, or a staff performance can be improved.
7.2.1 Formative evaluation is a process to be used by the
personnel seeking to improve performance.
7.2.2 Formative evaluation is designed and conducted by the
staff member seeking improvement in performance.
Summary
Evaluation of teacher education became a major thrust of a Center for

Excellence in teacher education when the Tennessee Higher Education
Commission (THEC) established the project in 1986. A major accomplishment
in the project was realized when the Accreditation Plus Model was amalgamated
for the project's evaluation approach. A-Plus! Thorough study of existing
evaluation approaches were convincing. No one approach seemed capable of
doing the task. At the same time it seems ludicrous to create a new evaluation
approach to add to the existing 40 or so models.
As an evaluation form, Accreditation Plus has a long history. It goes back
to 1895 when staff members of the University of Michigan tried to determine the
quality of secondary schools to decide whose graduates ought to be readily
admitted into the University of Michigan. At the start (in the late 1890s),
accreditation focused on program evaluation. The evaluation need in the work of
the Center for Teacher Education Evaluation was the same--the quality
assessment of education programs.
The idea of eclecticism moves in the same general direction. By recognition
that accreditation is not the be all and end all in evaluating teacher education
programs, the Center keeps open the use of any and all of the existing models in
an application of the Accreditation Plus Model. Eclecticism means selecting
what appears to be the best in various doctrines, methods, or styles. It also
means being composed of elements drawn from the various sources. In selecting
22
A-Plus as the evaluation approach for the project, we capitalize on a long and
effective history supplemented by the best of the current crop.
Figure 1 represents the evaluation of teacher education using the
accreditation approach. It describes the model's components and in some places,
the relationships of those elements. The Center's staff is working to detail
further the elements of the accreditation approach to evaluation. Questions and
comments are welcomed as we move further with this work.
REFERENCES
Cronback, L. J. (1963). Course improvement through evaluation. Teacher's

College Record,64, 672-83.
Floden, R. E. (1980). Flexner, accreditation, and evaluation. In Educational
Evaluation and Policy Analysis, 3(2), 35-46.
House, E. R. (1978). Assumptions underlying evaluation models.
Educational Researcher, 7(3), 4-6.
Madaus, G. F., Scriven, M. S., & Stufflebeam, D. L. (1983). Evaluation
Models: Viewpoints on Educational and Human Services
Evaluation. Boston, MA: Kluwer Nijhoff Publishing.
Scriven, M. (1983). Evaluation ideologies. In Evaluation Models:
Viewpoints on Educational and Human Services Evaluation,
Boston, MA: Kluwer Nijhoff Publishing, 229-260.
Stake, R. E. (1973). Program evaluation, particularly responsive evaluation.
In Evaluation Models. Boston, MA: Kluwer Academic Publishers.
Stufflebeam, D. L., et aI. (1971). Educational Evaluation and
Decision Making. Itasca, IL: F. E. Peacock Publishers.
Travers, R. M. W. (1983). How Research Has Changed American
Schools. Kalamazoo, MI: Mythos Press.
Worthen, B. R. & Sanders, 1. R. (1987). Educational Evaluation:
Theory and Practices. New York: Longman.
4
SELECTION AND EVALUATION OF KNOWLEDGE BASES

FOR TEACHER EDUCATION PROGRAMS
Roger S. Pankratz
Accreditation is referred to in the previous chapter as a "true form" of evaluation

in teacher education because it requires an institution to address a set of
minimum standards set by the profession. The first category of the revised
National Council for Accreditation of Teacher Education Accreditation (NCATE)
standards is "Knowledge Bases for Professional Education." The first Standard, I.
A., states:
the unit ensures that its professional education programs are based on
essential knowledge, established and current research findings, and
sound professional practice. Each program in the unit reflects a
systematic design with an explicitly stated philosophy and objectives.
Coherence exists between (1) courses and experiences, and (2) purposes
and outcomes (NCATE, 1987).
Additional specificity is provided in the following statement:
the unit ensures that its professional education programs have adopted a
model(s) that explicates the purposes, processes, outcomes, and
evaluation of the program. The rationales for the model(s) and the
knowledge bases that undergird them are clearly cited along with goals,
philosophy, and objectives (NCATE, 1987).
These excerpts from the Knowledge Base standards, and the remaining four
standards and 22 criteria, have prompted many teacher educators who are
responsible for developing institutional reports for accreditation to ask:
1. What, specifically, are knowledge bases for professional education

programs?
2. What are appropriate responses to the knowledge base standards?

24
3. How can faculty be involved in the selection and evaluation of knowledge

bases?
4. What resources are available for addressing the knowledge base standards?
5. How can faculty know if the knowledge bases that have been selected and/or
developed are adequate?
It is the purpose of this chapter to present some practical responses to these

five questions. The reader should be reminded, however, that what follows is
only one set of responses that could be given to these five key questions. Other
alternatives exist that could be equally helpful and professionally acceptable.
What are Knowledge Bases for Teacher Education?
Philosophical Definitions
Most scholars of teacher education would agree that the concept of a

knowledge base for teaching is broad and somewhat elu.sive. After a re-
examination of several classic texts on knowledge bases for teacher education,
Gideonse (1989) concluded that "a controversy exists respecting even the concept
of knowledge bases." In the allegory "Illinois Smith and the Secret of the
Knowledge Base," the Wise One in the Temple of Knowledge tells his visitor:
It (the Knowledge Base) exists, my son, but it is not a thing. It exists

in your head and your heart. The knowledge base is what your
scholarship validates or challenges. It is your intellectual heritage; it is
all the things you, as a professor use to spread enlightenment, to offer a
vision (Wisniewski, 1988, pA).
Shulman (1987) has delineated three forms--propositional knowledge, case

knowledge, and strategic knowledge--as important elements of the knowledge
base for teacher education. Valli and Tom (1988), in addressing the adequacy of
knowledge base frameworks in teacher education, provide the following
definition:
by knowledge base we mean the entire repertoire of skills,

information, attitudes, etc., that teachers need to carry out their
classroom responsibilities (p. 5).
Clearly there is no such entity as "the one knowledge base for professional
education." Rather, there are many knowledge bases that exist in many different
forms. Furthermore, knowledge bases are not static but rather, they expand as
ongoing research, scholarship, and experience constantly contribute to our
understandings. This is the philosophical side of the knowledge base definition.
Those faculty or staff members selected to produce an institutional report for
25
accreditation will be, however, interested in a much more practical definition.
Furthermore, anyone who has been briefed on the requirements and processes of
the accreditation process knows that the knowledge base for professional
education at each institution must be a well-defined and documented entity that
can be evaluated by members of a Board of Examiners relative to a set of
published standards and criteria. Thus, for those whose lot or honor it is to
provide leadership in selecting, evaluating, and describing program knowledge
bases in an institution, the following operational definition is offered.
An Operational Definition
For the immediate purpose, knowledge bases for professional education

programs are defined as:
that collection of documented researches, theories, and wisdom of

practice which contains the knowledge program faculty believe is
essential for graduates of the program to make competent professional
decisions.
Embedded in this definition are several important concepts. The first of

these is the limitation to professional education or pedagogy. The
knowledge base resulting from the liberal arts and sciences and from an academic
major or minor is important but is not the focus of Standard I. A. Second,
knowledge bases are program specific.
Though elements of knowledge bases may be common to several programs,
the total set of research, theories, and wisdom of practice for any given program
is unique. A third important idea in this definition is that a knowledge base is
represented by a collection of source documents embracing all types of
media that can be communicated and evaluated. Fourth, it is the program
faculty who decide through collaboration what is and is not included in the
knowledge bases for professional education. Fifth, the concept of essential
for competent professional decisions limits what is included to the
essentials for competence. This concept forces program faculty to decide what is
most important and what can be addressed in the life space of the program. This
is the knowledge base for the novice graduate in four-year programs. Graduate
programs will require knowledge bases essential for higher levels of
responsibility and performance.
Four Essential Elements that Address Standard I. A.
The text of the NCATE Standard I. A. and the first two criteria include the
following items as expectations for programs in a teacher education unit:
o Essential knowledge
o Established and current research findings
o Sound professional practice
o Systematic design
26
o Explicitly stated philosophy
o Goals and objectives
o Coherence between experiences and outcomes
o Adopted models
o Purposes
o Processes
o Evaluation
o Rationales for models
o Knowledge bases that undergird
o Scholarly inquiry
o Theory development
Many of these items suggest criteria for the knowledge base source
documents that will contain the essential knowledge base for professional
education programs (i.e., essential knowledge, established and current research,
sound professional practice, scholarly inquiry, and theory development).
However, these source documents cannot exist in a vacuum nor can they be
selected at random. They are based on an "explicitly stated philosophy" and they
are selected according to a "systematic program design." Goals, objectives,
purposes, processes, and evaluations flow from a philosophy, and knowledge
base source documents support program objectives and evaluation processes.
Adopted models show the relationships between the elements of the program and
how the knowledge base sources "undergird" program elements.
Thus, a minimum of four essential elements is recommended for inclusion
in a program knowledge base document that is designed to address Standard I. A.
in an institutional report: (a) Program Philosophy and Assumptions, (b)
Program Outcomes and Evaluation Processes, (c) a Program Model Based on an
Organizing Theme, and Cd) Knowledge Base Source Documents. Each of these
four elements can be developed through collaborative processes and should be
treated as four separate developmental tasks by a program faculty. Each of the
four will be further delineated below by (a) stating the task to be achieved, (b)
providing a rationale, (c) suggesting a process for achieving the task, and (d)
describing the resulting product that could become part of a program portfolio
or an institutional report in addressing Standard I. A.
Program Philosophy and Assumptions
The Task: To develop a set of belief statements and assumptions that

will guide the development of the curriculum and the selection of program
knowledge bases.
Rationale. A professional preparation program is more than a collection

of courses. According to Standard I. A., "Each program in the unit reflects a
systematic design with an explicity stated philosophy ... coherence exists
between (a) courses and experiences, and (b) purposes and outcomes." This
requirement implies there must be general faculty agreement on key issues that
will shape the program, and on beliefs that will translate into program design.
27
There should be assumptions about constraints that must be recognized and
incorporated into the program design.
Suggested Process. It is necessary to obtain faculty input, discussion,

and eventually consensus on important issues, beliefs, and assumptions that will
guide program development and implementation. This can be done by posing a
series of key questions and then facilitating input, debate, discussion, and general
agreement on responses to these questions. Examples of important issue
questions to ask are:
I. What should we assume are the most important purposes of schools and
schooling for the children our graduates will teach (e.g., the development of
cognitive knowledge and skills, thinking skills, social skills, cultural
values, self concept, etc.)?
2. What do we believe are the most important role(s) our graduates should be
prepared to perform in their professional work place (e.g., technical expert,
organizer/manager, counselor model/leader, decision maker, etc.)?
3. What do we believe about future economic and social conditions, directions

of schooling, and the training of teachers in our state and in the nation?
Examples of important design questions are:
1. What do we believe about which person(s) within the university or college,

should have responsibility for the governance of teacher education?
2. What beliefs inform our understanding of the contributions of various

academic units to teach teacher education?
3. What do we believe about the role of general, liberal, or academic studies

that support professional education?
4. What do we believe about the importance and function of clinical and field
experiences in our preparation program?
5. What do we believe about the role of research in our teaching preparation

program?
6. What do we believe about assessment and evaluation as essential elements

in our program?
7. What do we believe about the characteristics and qualities of students who

should be admitted to the program?
8. What do we believe about the qualities, expectations, and roles of program

faculty?
28
Four examples of questions related to program constraints are:
1. What are the limitations of the program with respect to length in years,
credit hours of professional education, and resources?
2. What state policies and mandates will place significant constraints on the
program (i.e., state guidelines, exit testing, statewide internship, etc.)?
3. What are the guidelines of learned societies that have been adopted that
require specific processes and/or outcomes?
4. Are there unique populations being served or unique features of students in

the preparation program that require adaptation in program design?
Answers to each of these questions should affect the curriculum design, and
all have implications for selecting the appropriate knowledge bases. "Issue"
questions (1) and (2) regarding the assumed purpose of schooling and key roles
for which graduates will be trained are of special significance in shaping the
philosophy of the program. For example, if a program faculty assumes that the
primary purpose of schooling is to develop self-concept and that the most
important role of the teacher is that of a counselor, the program and the
knowledge base that supports it will be very different from one for a program
based on the assumption that cognitive knowledge and skills should be the
cenL'al purpose of schools and the role of the beginning teacher is that of a
technical expert. In considering questions (1) and (2) above, ample time should
be allowed for facuIty input, debate, and discussion to develop ownership in key
elements of the program philosophy. Where time is not a factor and where
faculty like to be original and creative, planners might allow faculty to generate
faculty responses to these questions. But, if planners generate a number of
responses to each important issue, design, or constraint question and then
facilitate discussion and agreement on the most acceptable response(s),
ownership can usually be achieved in a relatively short time frame.
Final Product. The above process is designed to produce (a) a series of

faculty beliefs and/or assumption statements that have implications or program
design, (b) a narrative that responds to the types of questions posed, or (c) a
combination of belief/assumption statements and narrative. Whatever the
format, the resulting product will serve as the program philosophy statement and
will provide a unique perspective for the preparation of teachers. This
philosophy statement will provide guidance for the development of program
outcomes and the knowledge bases that will support them.
Program Outcomes and Evaluation Processes
The Task: To develop a series of program outcomes that graduates will

be expected to demonstrate and suggested processes for evaluating these
outcomes.
29
Rationale. The text of the NCATE Standard I. A. states that
each program in the unit reflects a systematic design with an explicitly

stated philosophy and objectives: coherence exists between (1) courses
and experiences, and (2) purposes and outcomes.
This statement implies that program objectives and outcomes are influenced
by the philosophy and must show a direct relationship, a "coherence," with the
curriculum. Program outcomes also provide an organizer for selecting and
developing the program knowledge base. Suggesting an evaluation process
and/or the instrumentation for each outcome helps to further define outcome in
the minds of planners and to remind them that each outcome must be framed in a
manner that can be subjected to assessment and an accountability system.
Suggested Process. The philosophy/ belief/assumption statements

developed in Task # 1 should be used as a mental framework and program faculty
should use it to list the most important program outcomes they believe
graduates of the program should be able to demonstrate, along with a suggested
evaluation process (e.g., paper and pencil test, written product, structured
interview, observation, portfolio product, etc.). Examples of generic teaching
functions that have been used as organizers to suggest program outcomes are:
o Analysis of content
o Analysis of student needs
o Diagnosis of learning problems
o Planning curriculum
o Planning instructional strategies
o Implementing instruction
o Managing student behavior
o Managing materials and resources
o Evaluating student progress and providing feedback
o Evaluating instruction
o Communicating with students
o Communicating with parents
o Communicating with peers
Initially, faculty should work individually to generate lists of performance

outcomes and evaluation processes. The next step is to combine the individual
lists, discuss, refine and try to reach consensus on a master list for the program.
This task should be planned over time with several work sessions to allow
ample time for input, debate, refinement, adoption, and ownership of the final
product. Faculty should be reminded that every program outcome has direct
implication for the selection and/or development of the program knowledge base.
Final Project. The process described above should produce a list of 10-
30 program outcomes statements, each with a suggested evaluation process that
30
embraces the entire realm of knowledge, skills, dispositions, and/or
competencies that a graduate of the program needs to demonstrate in order to
function as a fully prepared professional. The numbers 10-30 represent an
arbitrary recommendation. Experience has shown that too few outcomes
statements fail to provide the delineation of areas of knowledge, skills,
dispositions, etc. needed for program design. On the other hand, with too many
outcomes statements there is the danger of losing sight of the key performance
areas and the major performance foci for which the graduate is being prepared.
Following are examples of four possible program outcomes statements and
accompanying suggested evaluation processes developed for a middle school
teacher professional preparation program.
1. Plan a series of lessons that include appropriate learning outcomes, learner

activities, and assessment techniques, given a curriculum and student data.
Suggested evaluation processes for these lessons include (a) a written
application test demonstrating lesson planning skills and (b) the
examination of lesson plans prepared for student teaching.
2. Provide appropriate responses to learners during lesson development.

Classroom observation with instrumentation is the suggested evaluation
process.
3. Select and use behavior management techniques that encourage self-

discipline and maintain mutual respect between teachers and students.
Suggested evaluation processes include (a) a written application test and (b)
classroom observation with instrumentation.
4. Provide feedback on learning progress to students and their parents. A

structured interview using an observer-teacher dialogue instrument is the
suggested evaluation process.
The four examples listed above are performance oriented. The specific types
of behaviors or performance criteria that would be acceptable are dependent to a
degree on the program philosophy. The more developed the evaluation processes
and instrumentation, the more clearly program performance outcomes will be
defmed and specified.
A Program Model and Organizing Theme
The Task: To obtain faculty agreement on an organizing theme and to

design a program model that related the various program elements to each other
and to the knowledge base.
Rationale. Criterion 1 of Standard I. A. states, "The unit ensures that its

professional education programs have adopted a model(s) that explicates the
purposes, processes, outcomes, and evaluation of the program." If a program is
to reflect "systematic design" and to have "coherence" between the various
31
elements, it must be conceptualized as a whole entity, not just a collection of
courses. Furthermore, the program should have a central focus or an organizing
theme. Short (1987) has described the need for a unifying concept or theme in
the following statements:
The use of design criteria in dev~loping curriculum plans for teacher
education is based on the assumption that a program should exhibit conceptual
integrity and structural unity .... By selecting a single overarching concept,
pervasive metaphor, fundamental principle, or underlying theme, planners can
more readily control the consistency and integrity of meaning conveyed
throughout the plan than would be possible without the use of such a device.
Such a single, dominant idea serves not only a semantic control function; its use
has practical value as well. When different groups of planners work separately
on different components of a plan, it is very easy to create language in anyone
component that is at odds with what is created in another. If, however, some
overarching concept has been identified for the whole plan, it is easier to make
decisions and express them in language consistent with the controlling concept
(p.8).
Whereas the organizing theme contributes to conceptual integrity and
structural unity, a conceptual model shows how the different parts of the
program interrelate and the contribution of each part to the whole. An
organizing theme and program model also provide guidance in the selection and
evaluation of a program knowledge base. If one assumes that all program
elements should be supported by a knowledge base, a program model will serve
as a tool to point out possible gaps in the knowledge base as well as the
linkages between sub-units of the knowledge base and program elements.
Suggested Process. The selection of an organizing theme and the

development of a program model both require a highly sophisticated process of
synthesizing a complex set of ideas. Experience has shown that neither can be
developed to the satisfaction of an entire program faculty over a short period of
time. It can, however, be a process that is begun with the use of several faculty
group exercises and then worked on and changed over time until the appropriate
theme or model emerges for which there is general support.
A group exercise that has been used with success to help a program faculty
develop their first version of an organizing theme employs the faculty's response
to the question, "What is(are) the most important role(s) for which our graduates
should be prepared?" Generic roles such as the teacher as counselor, as expert
performer, as moral educator, as organizer/manager, as decision maker, and as
model/leader usually emerge (pankratz & Galluzzo, 1988; Short, 1987). By
examining the list of program performance outcomes developed in the previous
task and the most important perceived roles for program graduates, faculty can
select an initial organizing theme or unifying concept which serves as a focus for
ongoing discussion and often healthy frustration until a more satisfactory
organizing theme is developed and/or discovered.
Likewise, a group exercise can be used to generate drafts of a program model
or initial conceptualizations of how the various elements of the program
interrelate. The first step is to have a program faculty or sub-groups of program
32
faculty generate lists of program elements such as (a) key teaching functions or
program outcomes (see task #1); (b) key knowledge base domains (e.g.,
general/liberal studies, specialty studies, foundations studies, pedagogy); (c)
clinical and field experience elements; (d) program sequence elements (e.g.,
knowledge development, skill development, application, integration, etc.); and
(e) evaluation processes. Each faculty group should then use the temporary
organizing theme and the set of program elements they generated to design a
graphic representation showing the interrelationships of program elements to the
organizing theme. While at first this exercise may appear to planners and to the
faculty as an effort of futility, interest and serendipitous results have accrued.
The competitive spirit, i.e., which group can come up with the "best" model,
has developed faculty ownership in the process. Often creative faculty come up
with excellent ideas for a program model that identifies a conceptual scheme that
eventually grows into the final product. For some this exercise has created a
degree of conceptual dissonance and frustration over the inability to see
relationships clearly. This frustration also can have a very positive effect in
forcing faculty to look further for unifying themes and integrating structures.
Final Product. The result of this complex process is (1) an organizing

theme for the program that serves as a conceptual focus for all program efforts
and provides a vehicle for structural unity among elements of the program, and
(2) a program model or graphic representation that shows the 'interrelationships
among and between the various elements of the program including how sub-
units of the knowledge base support program elements and how the organizing
theme provides conceptual and structural unity to the program. The final version
of the program model and organizing theme must have faculty consensus. The
use of a systems analysis approach to develop a graphic model is fairly standard.
Knowledge Base Source Documents
The Task: To select and/or evaluate a set of knowledge base source

documents from research theory and the wisdom of practice that contain the
essential knowledge required by program graduates to function as competent
professionals at the level for which they have been prepared.
Rationale. Standard I. A. states, "The unit ensures that its professional

education programs are based on essential knowledge, established and current
research findings, and sound professional practice." Criterion #2 of Standard I.
A. further requires that "The knowledge bases used in professional education are
broad and include the traditional forms of scholarly inquiry as well as theory
development related to professional practice." To ensure that the essential
knowledge base exists, the set of source documents that contain the essential
knowledge must be identified, evaluated, and communicated to students in the
program.
Suggested Process. The process of locating, selecting, and evaluating

knowledge base source documents is facilitated by a set of topical organizers.
33
One method of organizing a knowledge base is to identify major performance
categories from the list of program outcomes and evaluation processes developed
in Task #2. Such a list of categories was developed by Western Kentucky
University faculty and is presented below. It should be noted that this list is
organized according to performance areas that will be evaluated during student
teaching and the internship year. Missing are the foundations knowledge
domains (i.e., social, philosophical, historical, legal, etc.).
Performance Areas for Selecting and Organizing Knowledge Base

Documents
o Assessment and diagnosis

o Curriculum development and planning
o Implementation of instruction
o Evaluation of student progress
o Management of resources
o Communication with students and parents
o Professional responsibilities
Gideonse (1989) discusses the knowledge bases in terms of liberal education,

intellectual foundations, subject matter, and professional knowledge. He states,
quite simply, that teachers must be liberally educated and well-versed in the
intellectual foundations of their profession. They must also understand their
content area and how to teach that content to students with a variety of learning
styles and abilities. They must also develop the professional knowledge and
skills that enable them to function in a school setting; this information includes
methods of evaluating student learning, ethical considerations, various
instructional techniques and more. Still another approach to organizing
knowledge base documents and the program knowledge base is to use these three,
more traditional categories of knowledge: foundations, curriculum and
instruction, and research. In the summer of 1986 at a Teacher Education
Workshop sponsored by the North Central Association of Colleges and Schools,
six teacher educators worked to develop a Taxonomy of the Knowledge Base for
Professional Studies (Bernal, Cleary, Connelly, Gerard, Kryspin, & Nicodemus,
1988). The taxonomy, presented in Table 1, provides 23 knowledge areas for
organizing knowledge base documents. With the more "traditional knowledge"
classification it may be easier to locate knowledge base documents, since much
of the professional literature is already organized according to the topics in the
taxonomy. As with the "performance area" classification described earlier, the
knowledge base is more closely associated with actual teaching functions and
behaviors and may be more useful in the development and implementation of
appropriate teacher behaviors.
Once performance areas, knowledge domains, or a taxonomy of professional
studies has been developed, faculty should identify those knowledge source
documents that contain essential knowledge for graduates. Knowledge source
documents can originate from at least three general sources: (a) research, (b)
theory development, and (c) the wisdom of professional practice. These
34
classifications of knowledge base sources have been found to be helpful
organizers both for the faculty in identifying and locating documents and for
students who are users of the knowledge. Table 2 is recommended for use with
faculty to begin the process.
The first step in the identification of knowledge base source documents is
simply brainstorming authors and/or titles of documents. The second step is to
consult program outlines, course syllabi, and other knowledge base resources as
described later in this chapter to develop a list of complete bibliographical
references of knowledge source documents. The third step in the process is to
locate and assemble all documents recommended for the "essential knowledge"
list of source documents so they can be examined and evaluated by program
faculty. Fourth, a judgment must be made by the faculty whether to include the
book, chapter, journal article, report, or other document in the essential
Table 1
A Taxonomy of the Knowledge Base for Professional Studies
Foundations Multicultural Education

(Social, Historical. Psychological, Urban Education
and Philosophical) Social Problems
School Law
Public Policy
History of American Education
Professional Ethics
Learning Theory
Human Development
Educational Psychology
Philosophy of Education
Curriculum and Instruction Methods of Teaching

Mentoring as a Model
Curriculum Development
Reading and Language Arts
Home and Community
Exceptional Children
Educational Technology
Classroom Management
School Organization
Research Evaluation
(Assessment Technique Tests and Measurements
Tests and Measurements) Assessment
(Bernal, Cleary, Connelly, Gerard, Kryspin, & Nicodemus, 1988)

35
knowledge list. The criteria for inclusion is simply a faculty judgment based on
the program outcomes and evaluation processes for graduates--whether this
document contains essential knowledge that all graduates of the program should
incorporate into their repertoire of knowledge. Not selecting a document does
not preclude it from being a required part of a professional course or on the
required reading list of a course. It should also be made clear that the "essential
knowledge list of source documents" is routinely updated.
Once the program faculty have agreed on the set of knowledge base source
documents they believe contain essential professional knowledge for graduates of
their program, a final step will be to develop a literature review of the key
concepts, principles, and recommended practices contained in the documents in
each performance area or knowledge domain. A quality literature review no
doubt requires substantial resources, especially faculty time. However, if a
Table 2
Identification of Knowledge Base Source Documents
Performance Knowledge Research Theory Wisdom of

Areal Base Source Source Professional
Knowledge Categories Documents Documents Practice Source
Domains Documents
(Examples)
Planning
Instruction
Evaluation
Management of
Student Behavior
program faculty are serious about identifying and explicating 'the essential
knowledge base for their students and about communicating this knowledge base,
then it must be in a form that can be accessed and communicated with ease.
Final Product. At a minimum, the final product of this task should be a

bibliographical list of knowledge base source documents that is considered by the
program faculty to contain the essential knowledge required by graduates of the
36
program faculty to contain the essential knowledge required by graduates of the
program. The bibliographical list should be organized by performance
area/k:nowledge domain and classified as a research, theory development, or
wisdom of practice document.
Beyond the minimum, the final product of this task should be a complete
literature review of the documents in each performance area/k:nowledge domain
summarizing the key concepts, principles, or recommendations for practice
contained in the documents.
How Should Faculty be Involved in the Development of Program

Knowledge Bases: The Recommendations of a Staff Dean
The new NCATE standards in the Knowledge Base Category require faculty
collaboration in the design of curriculum. Furthermore, the intent of the
knowledge base standards is to induce all teacher educator units to develop new
structures, processes, and behaviors that require total and ongoing involvement
of program faculty. It is a task that must be achieved if each program in the unit
is to reflect a "systematic design" with all the requirements listed in Standard I.
A. The pattern many institutions used under the old standards, appointing a
series of faculty committees to independently write parts of a self-study, is
simply not practical under the new accreditation guidelines. Teacher education
unit administrators must find ways to involve all program faculty in the
knowledge base development effort. While every program faculty has unique
characteristics and while some strategies work better in some situations than
others, there are some principles of operation and some leadership strategies that
appear to facilitate faculty involvement and participation in most curriculum
development efforts.
As a staff dean who has worked with a significant number of program
faculties to develop knowledge bases and who is presently orchestrating the
knowledge base development effort of ten program faculties on his own campus,
the author makes the following suggestions from wisdom of practice. First, the
initiative leadership, and authority for the knowledge base evaluation, selection,
and/or development must come from the dean and/or department chair's office. It
must be made clear to program faculty from the outset that the leadership of the
unit expects all faculty to participate in this developmental effort and that the
effort has a high priority. Also, it must be understood by all that those faculty
who have been given leadership responsibilities in the development process
assume these under the authority and full support of the unit administration.
Second, it is the dean's and/or department chair's responsibility to locate and
provide support and incentives for faculty involvement in the evaluation,
selection, and development of program knowledge bases. Even though unit
administrators may argue that budgets are tight and have been pared to the bone,
this author, with more than 23 years in teacher education, has never known an
education dean or department chair that did not have the ingenuity to locate
resources for what he or she regarded as highest priority. If the unit
administrators believe in the new NCA TE standards that have knowledge bases
as the centerpiece, they should show evidence of their commitment to these
37
professional standards through support and incentives for faculty participation.
In practical terms, this may mean providing food services for faculty work
sessions. It may mean finding resources for an overnight faculty retreat. It may
mean locating and providing technical support services and outside expertise to
provide faculty development activities. It also may mean regarding faculty who
make special contributions to the development process by paying them as
consultants or providing release time or some other professional incentive that is
valued. The key is to demonstrate to faculty that their professional contribution
and involvement are prized by the administration. The amount of faculty
participation and commitment to a development effort a dean can buy with
$10,000 is amazing if used judiciously to show faculty that their efforts are
appreciated.
Third, use overnight retreats and away-from-campus concentrated work
sessions to conduct faculty training or to obtain faculty input on key issues
related to knowledge base development. The completion of the four tasks
described earlier in this chapter and their products requires a concentrated effort by
the total faculty that is difficult to achieve near telephones, offices, and students.
It is critical that an initial workshop where the four elements of knowledge bases
described earlier are introduced, where key issues are discussed, and where the
foundation of a knowledge base is determined be done in a setting away from
normal routines. While full-day work sessions away from campus are
successful, planners at Western Kentucky University prefer two half-day
overnight retreats that include opportunity for interaction among faculty as well
as structured workshop activities.
Fourth, use small work groups to process faculty input from total faculty
involvement sessions (i.e., workshops and retreats) and to organizationally
transform these into products that can be studied, discussed, and refined. At
Western Kentucky University ten program faculty work groups of four or five,
each with a designated leader, have the responsibility to organize data,
information, and ideas from knowledge base development workshops and produce
drafts of program knowledge base documents with four elements:
I. Philosophy,
2. Outcomes and evaluation processes,
3. Program theme and model, and
4. Knowledge base sources.
The first draft document will be worked through, refined, and modified with
full faculty participation at the second faculty retreat session spaced four to six
months from the first faculty workshop. Leaders of each of the ten small work
groups will be paid a modest consulting fee for their professional contributions
and will be held accountable for the production of the drafts that will become the
program knowledge base document containing the four elements described.
[Additional suggestions relative to coordinating the involvement of everyone are
provided by Rutherford].
38
How Adequate is our Program Knowledge Base?: Three Sets of
Criteria
As demonstrated to Illinois Smith by the Wise One in the Temple of

Knowledge, the knowledge base is never complete; it is constantly growing and
being refined. In that sense, no program knowledge base is ever totally adequate
nor can a collection of source documents ever represent a fully completed
product. There are, however, milestones in the pilgrimage toward completeness
that help us to understand how far we have come. In an attempt to establish
some milestones of progress, three sets of criteria are presented. The first set is
very practical and represents, in the author's estimation, the very minimum for
any professional education program. The second set is labeled professional
because it uses concepts and terms from the NCATE Standard I. A. to judge
adequacy. The third set, multiple criteria, describes a framework suggested
by Valli and Tom (1988) to help program faculty reach for a higher level of
thinking and conceptualization in evaluating, selecting, and developing a
program knowledge base.
Practical Criteria. Based on the author's earlier somewhat narrow

operational definition of a knowledge base, program faculty who want to judge
the adequacy of their existing knowledge base or a newly developed knowledge
base might simply ask themselves the following questions. Does it provide the
essential knowledge that will enable graduates to make informed professional
decisions about their behavior regarding key teaching functions identified (i.e.,
planning, implementation, instructional evaluation, management of student
behavior, etc.)? Also, is the knowledge base provided in a format that graduates
can easily relate knowledge to those teaching functions they must perform every
day in the professional work place? The concepts "essential" and "easily relate"
are most important. If their response is no or not sure to either of these
questions, the knowledge base development process has not yet passed the bare
minimum adequacy test.
Professional Criteria. Assuming the practical criterion is met, a

program faculty may want to tum their attention to the ideas embedded in the
Knowledge Base Standard I. A. of NCATE. Terms like essential knowledge;
broad, established, and current research; traditional forms of scholarly inquiry;
and theory development related to professional practice convey the most
important concepts professional educators who designed the new NCA TE
Standards believe were important. These must be present in a program
knowledge base to be judged "adequate" to meet accreditation standards. In
practice, a program faculty might judge the adequacy of their collection of
knowledge base source documents with respect to Standard I. A. by asking the
following question:
For each performance area/knowledge domain identified:
o Are the documents of both classic and current research?

o Are important theory and scholarly inquiry documents included?
39
o Are key documents that convey what the program faculty consider essential
wisdom of practice present?
Even though some performance areas or knowledge domains may not have key
documents in all three of the above categories, whether knowledge documents do
or do not exist in all categories should be addressed in the evaluation process.
Multiple Criteria. Valli and Tom (1988) propose that a knowledge base
framework must embody five characteristics for it to adequately inform the
practice of teaching and teacher education. Use of these five characteristics will
prompt a program facuIty to go beyond what is essential and practical for the
beginning graduate. These characteristics are for those program faculty that are
reaching for the knowledge base that fully address the needs of the master teacher
and seasoned practitioner at all levels. According to Valli and Tom (1988) this
higher level knowledge base framework must meet the following requirements:
o It must include knowledge derived from all relevant scholarly traditions.

o It must present competing views of teaching and schooling.
o It must show relationships between technical and normative aspects of
teaching.
o It must be useful and accessible to practitioners, and
o It must encourage reflective practice (p. 6).
The authors of these requirements have labeled these five characteristics as: (a)
the scholarly criterion, (b) the multiplicity criterion, (c) the relatedness criterion,
(d) the usefulness criterion, and (e) the reflectivity criterion.
Using the description of these five characteristics suggested by Valli and
Tom (1988), a program faculty might ask themselves the following questions
regarding the adequacy of their know ledge base:
Scholarly Criterion. Do the source documents contain knowledge that

was derived from scholarly pursuits of research, theory development, and wisdom
of practice? Is inquiry from the traditional academic disciplines included as well
as craft knowledge (wisdom of practice) derived from a systematic analysis of
teaching practice?
Both classic and current documents should be included. The important
concept to keep in mind when judging the adequacy of the program knowledge
base is whether the source documents contain knowledge derived from systematic
and logical inquiry or from mere opinions fueled by biases that have little to do
with scholarly pursuits.
Multiplicity Criterion. Do the source documents of the knowledge

base contain competing theories, perspectives, and explanations of major issues
in education and the functions of teaching?
Even though a professional preparation program may emphasize a consistent
philosophy or focus for professional practice, graduates need to understand that
there are fierce debates over competing theories and models of teaching and
40
learning. They also need to understand the relationship between these competing
views and how the program theme or focus fits into the array of knowledge
perspecti ves.
Relatedness Criterion. Is the knowledge contained in the source

document presented in a manner that shows the philosophical, social, and/or
demographic context in which it was derived?
Knowledge derived from scholarly pursuits is not value or context free.
Research results have demographic contexts. Theory development is based on a
belief system. Wisdom of practice is synthesized from experiences in a social
and cultural environment. These contexts need to be described along with the
knowledge that has been derived in them so users of this knowledge can make
the appropriate application to practice.
Usefulness Criterion. Is the knowledge base organized and presented

in a format that facilitates a direct application to concrete teaching situations?
This criterion provides a rationale for organizing the knowledge base source
documents according to teaching functions or performance areas so the
application to practice is obvious. Also, the recommendation earlier to develop
a literature review and summary of source documents by performance area would
help to key the knowledge base to specific decisions a teacher must make and the
alternative actions that follow.
Reflectivity Criterion. Is the knowledge base presented in a manner

that encourages thoughtfulness about schooling practices and about effective
teaching behavior?
This criterion implies that competing views should be presented as well as
conflicts in research findings. The knowledge base should provide a commentary
on what we know, what we think we know, and what we need to know.
Practitioners using the knowledge base should be able to act with confidence
about those principles of teaching and learning that have been firmly established
but have a disposition of reflectivity and inquiry regarding those areas of practice
where there is less certainty or perhaps a void of organized knowledge.
SUMMARY
New professional standards for the evaluation of teacher education curricula

prescribed by the National Council for Accreditation of Teacher Education require
that programs are based on essential knowledge, established and current research
findings, and sound professional practice. This chapter addresses five areas of
concern often expressed by teacher education unit administrators and program
faculty. These include: (a) a definition of program knowledge bases, (b)
essential elements of a knowledge base document, (c) faculty involvement in
evaluating and selecting a knowledge base, (d) resources for evaluating and
selecting a knowledge base, (d) resources for evaluating and developing
knowledge bases, and (e) criteria for judging the adequacy of knowledge bases.
41
While most scholars of teacher education would agree that the concept of
knowledge bases for teaching is broad and all inclusive, a more narrow
operational definition is needed for curriculum evaluation and unit accreditation.
Thus, a program knowledge base is operationally defined as that collection of
source documents which program faculty believe contains the essential
knowledge required for graduates to make informed, competent, and professional
decisions in the educational work place. Based on the language of the NCATE
Knowledge Base standards and criteria, four elements are recommended as
essential in describing a program knowledge base: (a) a program philosophy, (b)
program outcomes and evaluation processes, (c) an organizing theme and
program model, and (d) an organized compendium of knowledge base source
documents. The developmental tasks, processes, and final products for each of
the four elements are presented from a practical perspective. Based on the
experiences of the author, several concrete suggestions are offered for involving
faculty in evaluating and selecting program knowledge bases. Unit leadership
and support are described as an important ingredient to faculty involvement in
the knowledge base development process. Selected resources for knowledge base
evaluation and development at the local level are recommended. The three
categories of resources included in the selected list are: (a) seminal documents
useful for orienting faculty to the concept of knowledge bases for professional
education; (b) workshops that have been designed to assist faculty in evaluating
and selecting program knowledge bases; and (c) sources that contain
bibliographies of research, theory development, and wisdom of practice related to
the professional knowledge base for teaching. Finally, three sets of criteria are
presented for judging the adequacy of a program knowledge base. The first set is
practical and considered as a minimum. The second set is based on the
professional standards and criteria of NCA TE. The third set is more
comprehensive and suggested multiple criteria that go beyond the minimum
requirements. Included in these more advanced criteria are the key concepts of
scholarships, multiple perspectives, relatedness, usefulness, and reflectivity.
REFERENCES
Bernal, E., Cleary, M., Connelly, M. J., Gerard, M. L., Kryspin, J., &
Nicodemus, E. (1988). In D. W. Jones (Ed.). A taxonomy of the
knowledge base for professional studies. Knowledge Base for Teacher
Education. Muncie, IN: Ball State University.
Brouillet, F. B., Marshall, C. R., & Andrews, T. E. (1987). Teaching and
learning in the affective domain: A review of the literature.
Olympia, W A: Professional Education Section, Office of the Department of
Public Instruction.
Brouillet, F. B., Marshall, C. R., & Andrews, T. E. (1987). Teaching and
learning in the cognitive domain: A review of the literature.
Olympia, WA: Professional Education Section, Office of the Department of
Public Instruction.
42
Gideonse, H. D. (1989). Relating knowledge to teacher education:
Responding to NCATE's knowledge base and related standard.
Washington, DC: AACTE Publications.
Howey, K. R., & Zimpher, N. W. (February, 1988). A workshop on
program change and assessment in teacher education. Presented
at the meeting of the American Association of Colleges for Teacher
Education, New Orleans, LA.
Knowledge base for the beginning teacher internship program.
(1989). Frankfort, KY: The Kentucky Department of Education, Capitol
Plaza Towers.
Mitzel, H. (Ed.). (1982). Encyclopedia of education research (5th ed.).
New York: Collier Macmillan.
National Council for Accreditation of Teacher Education. Standards,
procedures, and policies for the accreditation of professional
education units. (1987). Washington, DC: Author.
Pankratz, R. S., & Galluzzo, G. R. (1988). Designing a knowledge base for
teacher education programs: A developmental workshop for program
faculty. Bowling Green, KY: Western Kentucky University.
Reynolds, M. C. (Ed.). (1989). Knowledge base for the beginning
teacher. New York: Pergamon Press.
Short, E. C. (1987, July/August). Curriculum decision making in teacher
education: Policies, program development, and design. Journal of
Teacher Education, 2-12.
Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new
reform. Harvard Education Review, 57(1), 1-22.
Smith, D. C. (Ed.). (1983). Essential knowledge for beginning
educators. Washington, DC: American Association of Colleges for
Teacher Education, Clearinghouse on Teacher Education.
Valli, L., & Tom, A. R. (1988). How adequate are the knowledge base
frameworks in teacher education? Journal of Teacher Education,
39(5),5-12.
Wisniewski, R. (1988). Illinois Smith and the secret of the knowledge base.
Journal of Teacher Education, 39(5), 2-4.
Wittrock, M. C. (Ed.). (1986). Handbook of research on teaching,
Third Edition. New York: Collier-Macmillan Publishers.
5
QUALITY CONTROLS IN TEACHER EDUCATION

PROGRAMS
William E. Loadman
The Ohio State University
Quality control refers to a system under which a preservice teacher education

program can be examined with respect to a set of standards. Quality control has
been a concern in professional education for decades. Teachers have been under
public scrutiny from their admission into education programs through their
subsequent placement in schools. Teachers and students aspiring to become
teachers have been subjected to various quality control programs, in an effort to
predict who will be a good teacher as well as to assess their knowledge, skills,
attitudes, and values. Teacher preparation standards have been promulgated and
enforced. These various and diverse efforts reflect a growing concern for quality
education. To date our efforts at quality control have been largely unsuccessful.
The attention which is focused on education serves both to expose
weaknesses in practices and procedures and to facilitate change and progress.
Education clearly needs to improve its practices, Quality control is one tool that
can be used to assist in this effort. The approach outlined below can help
improve teacher education programs. As Rutherford has noted elsewhere in this
text, it is important to begin any major change effort with the intent to succeed.
1. Commit to implement, and use results of the quality control mechanism.
2. Allocate resources to accommodate the development and subsequent

implementation of the quality control mechanism.
3. Delineate goals and intended outcomes of the teacher education program(s).
4. Identify and delineate criteria and standards for various program aspects.
5. Examine internal (e.g., local program standards) and external standards (e.g.,
national associations) to consolidate and identify uniqueness to assist in
reduction of duplication of efforts.
44
6. Develop a quality control mechanism through the involvement of all levels

of program participants, including administrators, faculty, staff, students,
and other groups having vested interest
7. Decide which aspects of the program to examine, when to conduct such

inquiry, who will conduct the inquiry, and how it will be conducted.
8. Identify key indicators of quality, the means to efficiently obtain

information about the indicator, and the timing and person{s) responsible for
collection of information.
9. Develop and/or obtain necessary instrumentation to collect information.
10. Establish a workable plan to implement quality control, being sure to

include timelines, responsibilities, and alternative courses of action.
11. Establish clear working agreements among potentially affected parties,

before implementation, of data collection for the quality control effort.
12. Maintain rationality relative to the level of effort, speed of implementation,

and use of information resulting from quality control activities.
13. Implement with appropriate timelines, personnel responsibilities, and

accompanying guidelines.
ISSUES IN QUALITY CONTROL FOR PRESERVICE

PROFESSIONAL EDUCATION
Eight issues must be considered in the development of quality control

systems at the local level. The issues are purpose, locus of control, focus,
personnel, standards, criteria, assessment, and implementation. Each is discussed
in the following section.
Purpose
What is the purpose of a quality control system? Generally, one stated

purpose of such a system is to maintain and ensure an adequate level of quality
relative to the preparation of teachers and to protect the public good. Another
stated purpose is the contribution to the improvement of practice.
Unfortunately, most systems which are now in place tend to result in the
generation of large amounts of data that are seldom used to improve practice or
modify programs. What is needed is a clear set of guidelines for the collection,
analysis, and use of data about programs and faculty members.
Are the purposes of the quality control effort clearly delineated and not in
competition or conflict with each other? Vague or inexplicitly stated purposes
lead to confusion and can diminish trust. The system which seeks to be all
45
things to all people for all situations sometimes results from poorly stated
purposes.
Locus of Control
Who is responsible for the quality control system? Control can reside with
State Departments of Education, legislatures, accrediting agencies,
universities/colleges, and practicing teachers. If care is not taken, this can lead
to duplication of effort, frustration of those under the mandates, and a lack of
cohesion and forward movement. Frequently the impetus for quality control is
exerted from an external source and is improved on at the local level.
Focus
On what does the quality control system focus? Several potential foci exist.
The focus could be on the quality of the preservice program. If that is the case,
major considerations are the definition of "program" and consensus on the
operational definition of "quality." For such topics as program improvement and
judgments about the program, program graduates are one focal point and their
performance on the job one measure of quality. Other possible foci include
faculty capabilities and performance, and student know ledge and skills at the
conclusion of the program. Some systems operate on multiple foci and it is not
uncommon to find systems where the foci are in conflict (Le., formative
evaluations versus program judgment or summative evaluations).
When and where does one focus the quality control system? There are
several options: pre-enrollment, during the program, at exit from the program,
and after graduation. The ideal system is part of an on-going monitoring and
improvement effort. [See chapters by Ayers, Craig, and Rutherford for more
information.]
Personnel
Are those persons most directly affected by the quality control effort
involved in deliberation about the system? If they are not, there is likely to be
little ownership and hence little support for the system or for using the data
which result from its application. [See chapters by Craig and Rutherford.]
Who conducts the quality control process? Many efforts are conducted by
persons within the organization under scrutiny. Their findings may be subjected
to validation by an external team. The findings of an external team frequently
carry sanctions for non-compliance with a set of standards and a seal of approval
for compliance. Positive efforts are more likely when those involved in the
enterprise are initiating and developing the effort.
Standards
Are there standards upon which the quality control system is based? There
are several sets of standards, including those promulgated by accrediting agencies
46
(e.g., National Council for Accreditation of Teacher Education, NCATE); state

departments; regional associations (e.g., Southern Association of Colleges and
Schools); professional societies; and local boards. These standards have differing
focal points and use different indicators of quality, and are sometimes in direct
conflict with each other. Staff in the Center for Teacher Education Evaluation
conducted a study and found that if a teacher education program meets the
NCATE standards, it will more than likely meet or exceed the standards of other
national, state, regional, or local accrediting bodies.
Criteria
Are there clearly specified criteria for each of the standards? In general, the
answer is "No." Some criteria can be deduced easily from standards while other
standards yield only grudgingly what are speculative criteria at best. The level of
specificity differs within as well as across sets of standards.
Is there agreement among professionals about the operationalization of the
criteria? No. There is, for example, a great debate over what constitutes
pedagogical knowledge. The limited specificity and lack of agreement on
standards compounds the problem of developing criteria.
Are the criteria which have been used frequently comprehensive in scope?
No. For example, few existing systems for assessing teacher quality assess oral
communications skills, whether the person is comfortable with children, or the
person's philosophy about teaching, yet these variables are integral to th({
teacher's performance.
Assessment
What are the most common means of collecting information for quality
control in preservice professional education? These are presented in full detail in
the other chapters of this book. Rudner discusses the evaluation of candidates
wishing to enter teacher education programs. Zimpher describes the evaluation
of field and laboratory experiences. McLarty provides a detailed listing and
description of instruments which are commonly used to measure outcomes.
Ayers and Craig discuss mail follow-up and follow-up evaluations, respectively.
Centra discusses faculty evaluation. Heam describes governance and Pankratz
discusses the selection and evaluation of the knowledge base. Checklists and
guidelines accompany many of the chapters.
Implementation
From where do the resources to develop and implement a quality control

system come? Most often they come from an internal allocation, too often at
the expense of program resources. This places quality control in direct conflict
with programs, which is, of course, the antithesis of the desired situation.
What are the typical steps in the development and implementation of a
Quality control system mpreservice professional education?
47
o Delineate purpose
o Establish program goals
o Detennine program standards
o Detennine appropriate criteria for standards
o Obtain necessary instrumentation
o Develop mechanism and procedures for implementation
o Establish timelines for implementation
o Select focus for pilot effort
o Select and train team members
o Pilot test the system
o Revise system as appropriate
o Implement other aspects of system over time
o Compare perfonnance to standards and make judgments about
compliance/non-compliance
o Obtain external validation of findings
o Evaluate quality control system
EXPERTS WHO SHOULD SERVE ON QUALITY CONTROL

PANELS FOR TEACHER EDUCATION
Several options for panel composition exist; they are dependent on the nature of
the task to be accomplished.
Standard Setting Panel
o business community representative(s)

o lay citizen representative(s)
o local educational agency administrator(s)
o local school board representative(s)
o state department representative(s)
o teacher educator(s)
o teacher union/teacher representative(s)
o university/college administrator(s)
Mechanism/procedure Development Panel
o measurement/assessment professional(s)
o local education agency administrator(s)
o practicing classroom teacher(s)
o quality control staff (if available)
48
Quality Control Team
o external professionals (university and public schools)

o quality control staff (if available)
Quality Control Oversight Committee
o comprised of representatives from above Standard Setting Panels
Professional Directions
The profession of education faces many challenges with respect to quality

control. Colleges and universities are therefore caught having to address
multiple sets of standards. Educators must work to establish a better and more
generally accepted set of standards to represent the professior. At this point, our
methodological sophistication relative to quality control is grounded more firmly
than our resolve to conduct and use the information resulting- from quality
control efforts.
SUMMARY
It is important that teacher education programs have sound quality control

systems. Those systems should be agreed upon by key stakeholders and should
begin with admission to the teacher education program. Rather than "ending"
with the follow-up of graduates, the system should include the use of graduates'
feedback to improve the teacher education program. Additional information
about quality controls is found in Hearn's chapter of this book. Additional
information about assessment instruments is also contained within other
chapters. Faculty and Staff members at the organizations involved in program
evaluation listed in the Appendix to this book can provide assistance in
establishing or evaluating quality control systems.
6
TESTING FOR ADMISSIONS
Lawrence M. Rudner
LMP Associates and American Institutes for Research
In many Schools, Colleges, and Departments of Education (SCDE), admissions

testing is yet another administrative burden that appears to be of minimal use.
In others, admissions testing has proven to be a valuable tool that has raised the
quality of teacher education candidates and has made teacher education more
rewarding and more satisfying. The proper use of weII-constructed and validated
tests can lead to the admission of students who are clearly qualified for
instruction. This chapter addresses a range of issues surrounding admissions
testing. More specifically, this chapter describes:
o some of the pressures on SCDE concerning admissions testing and
o some of the issues regarding admissions test selection and use.
The chapter also provides the foIIowing practical tools for those involved in
admissions testing:
o test evaluation criteria,
o a listing of sources of information about standardized tests,
o a description of several tests used by SCDE, and
o a summary of some current admissions requirements.
PRESSURES RELATIVE TO ADMISSIONS
Just as school superintendents are expected to provide a capable teacher for

every classroom, teacher preparation institutions are expected to graduate adequate
numbers of individuals capable of filling those positions. The SCDE are
expected to maintain high standards regardless of enroIIment fluctuations and
regardless of teacher demand.
50
Reform Reports
The most visible of the calls for recruiting talented and committed people
into the profession have stemmed from the reform reports of the 1980s. Reports
such as A Nation at Risk, (U. S. Department of Education, 1983);
Conditions of Teaching, (Feistritzer, 1983); Excellence in Our
Schools, (National Education Association, 1982); Tomorrow's Teachers,
(Holmes Group, 1986); and A Call for Change in Teacher Education,
(National Commission for Excellence in Teacher Education, 1985) have re-issued
the call for improved teacher education. The number and range of concerned
organizations appears to be at an all time high.
These reports postulate that if better candidates were recruited into teacher
education programs, better qualified teachers would graduate and pre-college
students would learn more. While recognizing the host of social realities,
ranging from lack of parental involvement in education to the decline in
enrollments in Teacher Education, these reports present a compelling case to
state, community and college policy makers. These policy makers have, in tum,
placed new pressures on SCDE.
NCATE
The 1987 National Council for Accreditation of Teacher Education

(NCATE) Standard concerning admissions and recruitment reads: "The unit's
admission procedures encourage the recruitment of quality candidates and those
candidates represent a culturally diverse population." In order to comply with
this standard, the following criteria were set:
39. Incentives and affirmative procedures are used to attract candidates with
potential for success in schools.
40. Applicants from diverse economic, racial, and cultural backgrounds are
recruited.
41. A comprehensive system, which includes more than one measure, is used
to assess the personal characteristics, communications, and basic skills
proficiency of candidates preparing to teach (p.44).
Not all SCDE are bound by the NCATE standards; non-NCATE institutions
appear to be equally concerned with recruitment and equity. A survey of 161
institutions showed similar responses between the two types of institutions on
items concerning admissions testing (Kapel, Gerber, & Reiff, 1988).
State Departments of Education and State Legislatures
As of April 1988, 26 states have implemented admissions testing programs

and 44 have implemented certification testing mandates (Eissenberg & Rudner,
1988; see Table 1.). These mandates are often viewed as one of several ways to
51
improve teacher quality and the quality of American education. As Arkansas
Governor Bill Clinton stated, "To those who feel insulted by the test, I can only
reply that it is a small price to pay in exchange for [a tax increase] and for the
contributions the testing process would make in our efforts to restore the
teaching profession to the position of public trust and esteem it deserves."
(Clinton, 1983). While state teacher testing programs may enhance the image of
the profession, Rudner (1988) has argued that the low standards set by some state
testing programs preclude any meaningful impact. In some states, the mandates
are having a direct impact. Admissions testing standards have lead to higher
caliber students. They have lead to a re-analysis of educational programs and
have resulted in an altered set of skills presented to students. State certification
testing is also having an impact on admissions. Since it dOes little for the
image of a school, college, or department of education to admit students who are
not able to obtain certification once they graduate, some institutions are
restricting admissions to students possessing the basic skills expected of
graduates.
ISSUES REGARDING ADMISSIONS TESTS
Issues regarding admissions test selection and use include supply and
demand, test content, basic skills and teacher effectiveness, cut scores, and types
of tests. Each is discussed in the following section.
Supply and Demand
Teacher testing grew during the late 1970s and early 1980s--an era of open
college admissions, surplus graduates in educatiolJ, and declining student
enrollments in professional education programs. During this era there was
approximately one vacancy for every two teacher education graduates.
Educational organizations and many SCDE could afford tougher standards.
Testing as a means of selecting the most capable made a great deal of sense. In
many parts of the country, however, the era of surplus has ended. School
districts are beginning to experience increasing numbers of vacancies and SCDE
are experiencing declines in enrollments. The 1985 annual survey of college
freshmen (Astin, 1985) reported that only 5% of college freshman were interested
in becoming teachers. Further, students with the greatest academic talent were
the least likely to choose teaching as a career. Yet it was during this era that
teacher testing programs were first implemented. In order to meet the demand for
teachers, very low standards were set. Rudner and Eissenberg (1988) pointed out
that the average passing scores on the National Teacher Examinations Basic
Skills examinations were set by the states to be 3 to 5 standard errors of
measurement below the scores recommended by standard setting panels. The
standard setting panels were comprised of testing and measurement experts who
indicated the score marginally qualified individuals would be expected to receive.
Traditionally, enrollment in teacher education programs has reflected the trends of
supply and demand. Shortly after periods of teacher surplus, enrollment declined,
Table 1
Summary for Testing for Admissions to Teacher Education Programs
State Test 1 Passing Scores
Alabama 1) SAT OR ACf 1) 745 OR 16

2) AELPT 2) 70%
Arizona 1) APE 1) 80%
2) PPST 2) no cut score
Arkansas PPST to be set
California CBEST Total of 123
Colorado SAT or CAT or 820 or 19 or
2.5 GPA (on 30 hours) 75%
Connecticut CONNCEPT M=70%. R=71%. W=75%
Delaware PPST R=175.M=175. W=172
Florida SAT or ACf 40%
Georgia Regents'Test Passing Score
Kentucky ACf-COMP 19
Louisiana NTE GK=644. CS=645
Mississippi ACT-COMP or PPST Varies by IRE
Missouri SAT or ACf 800 or 18
Nebraska PPST R=170. M=171. W=170
Nevada PPST R=169. M=169.W=170
New Mexico Basic Skills Screening Varies by IHE
North Carolina NTE CS=636. GK=631
North Dakota SAT or ACf Varies by IRE
Oklahoma PPST Cut scores TBD
Oregon CBEST Total of 123
South Carolina SCEE M=40/56. R=45/56.
W=3.0 out of 4.0
Tennessee PPST M=169. R=169. W=I72
Utah Varies by IHE
Vermont Varies by IRE
Virginia SAT 835
Washington WPCf and/or SAT or 80 and/or 700 or 16
ACT
West Virginia PPST. ACf-COMP. M=I72. R=I72.
IHE listening test W=I71. 17 & pass. score
Wisconsin PPST R=175.M=173. W=174
Wyoming CAT 70%
1ACf =American College Testing; AELPT= Alabama English Language

Placement Test; APE=Arizona Proficiency Test; CAT=California Achievement Test;
CBEST= California Basic Educational Skills Test; CONNCEPT=Connecticut
Competency Examination for Prospective Teachers; NTE=National Teacher
Examinations; SCEE=South Carolina Entrance Examination; PPST=Pre-
Professional Skills Tests; WPCf=Washington Pre-College Test.
53
and shortly after periods of teacher shortages, enrollment increased. If the trends
of the past few years are any indication, interest in teaching as a profession is
once again on the rise. The 1987 survey of college freshman noted that the
number planning to become teachers has increased to 7% - an increase of 40%
over 1985 (Astin, 1987). This increased interest could spell an opportunity for
meaningful admissions testing for many schools. The goals and claims for
improved quality through testing could be realized with large numbers of
candidates.
Test Content
While basic skills tests are the most prevalent form of admissions tests,
interest in measuring other cognitive skills is also increasing. Kapel, Gerber, and
Reiff (1988) noted that most institutions consider social/emotional fitness in
teacher education admissions and that several, although not many, use formal
standardized instruments. Other forms of testing such as evaluation forms,
observation checklists, personality inventories, and biographical information
forms are also used. Marshall, Sears, and Otis-Wilborn (1988) provided an
excellent evaluation of the literature pertaining to admissions test content.
Selection of test content must be based on self-examination of a program
and what is expected of its students:
1. Is knowledge of basic skills required for certification? If so, then the

program personnel must either impart these skills to their students, or be
sure the students have these skills prior to admission.
2. Is Social maturity an intended or unintended by-product of a program? If

not, then enrollment should be restricted along these lines.
3. Does it matter if students enroll who are unsure about their commitment to
the profession? If so, then a career maturity inventory may be a necessary
prerequisite.
Basic Skills and Teacher Effectiveness
Most schools limit their formal testing to an assessment of basic skills

(Kapel, Gerber, & Reiff, 1988). At these schools, students who lack
fundamental skills that should have been acquired by 12th grade (if not by 7th
grade) are excluded from teacher education, or they are admitted on a provisional
basis. The assumption that these basic skills are a necessary condition for
successful teaching makes a great deal of sense and has withstood judicial
scrutiny. The logic, however, only holds so far. While individuals who lack
basic skills probably will not be able to impart many skills to students, it does
not necessarily follow that individuals with high scores on a basic skills test
will make better teachers. In fact Shim (1965), and more recently, Taylor and
Miller (1985) have shown academic ability to be not or even inversely related to
teaching success as judged by school principals. Schlechtly and Vance (1981),
54
Pugach and Raths (1983), and Sykes (1983) have independently concluded that
there is no evidence linking measures of academic ability to teacher effectiveness.
The lack of evidence has significant implications with regard to teacher
recruitment. Attracting the brightest is probably not the best way to recruit the
most capable. Beyond a certain level, raising passing scores will probably not do
much for quality. Other types of measures, such as measures of commitment to
teaching or psychological fitness, are needed.
Cut Scores
The selection of passing scores, or cut scores, is almost as important as the

selecting of content. Selecting a level that is extremely low will result in
virtually everyone passing. If this low level is also indicative of low skills,
then the testing program will become a testing program in name only. Such
testing practices are a waste of everyone's time and money. If everyone is
admitted, then why bother with the test? Selecting a score that is extremely high
will provide an impressive standard, but can also be self-defeating. If virtually
no one passes, then again, there is little need for the testing program (or the
school for that matter). Cut scores are often adjusted downward in the name of
protecting applicants who actually have the ability but fail the test. Adjusting
the scores downward reduces the frequency of these "false negatives." The flip
side of the coin, however, is that it also increases the frequency of false positives
- applicants who actually do not have the ability but pass the test. Passing
scores are also frequently adjusted downward. While this helps enrollment and it
protects against false negatives, it may result in a lower caliber of student and it
means that more students are going to become false positives - unable
individuals who pass. By protecting the marginally qualified, SCDE as well as
state departments of education, are defeating the purposes of the testing program.
Cut scores should reflect the ability expected of the marginally qualified
applicant who will succeed in the program. This score can be established using
a formal study panel and a statistical approach to standard setting. If legal
challenges are not a major concern, passing scores used in other SCDE can be
used as a basis for establishing local passing scores.
Regardless of how the scores are established, they should be evaluated and
adjusted periodically. The caliber of students and program demands will change
over time. It should also be emphasized that the admissions test should not be
the sole criterion for admission. Factors affecting performance and factors related
to program success not covered by the test should be considered.
Type of Test: Commercial or Locally Developed
A decision must be made whether to use a commercially available test or to

develop one locally. Antes (1985) points out the following advantages and
disadvantages of commercially and locally developed admissions tests:
55
Table 2
Advantages and Disadvantages of Commercially
and Locally Developed Tests
COMMERCIALL Y DEVELOPED TESTS
ADVANTAGES DISADVANTAGES
convenience -- test development cost to the student can be

activities are not needed relatively high
scoring can be done by the local needs are not so well
company served
test security can be the vendor's content validity relative to
responsibility the local test use can be
normative national date are available lower
reliability and other technical
features are well documented
manuals are available
limitations and controversies are
known (through critiques)
LOCALL Y DEVELOPED TESTS
ADVANTAGES DISADVANTAGES
lower in cost national norms are not

local needs are better served available
higher reliabilities are often considerable time must be
found invested in test develop-
norms are more relevant to the ment
local situation security may be lacking
technical details may not
be as well known
Practical Tools
The "practical tools" provided in this section include: (a) test evaluation
criteria, (b) a listing of sources of information about standardized tests, (c) a
description of several tests used by SCDE; and (d) a summary of some current
admissions requirements.
Test Evaluation
The Standards for Educational and Psychological Testing

established by the American Educational Research Association, the American
56
Psychological Association, and the National Council on Measurement in
Education (1985), are intended to provide a comprehensive basis for evaluating
tests. The standards of greatest interest in evaluating tests for possible use as
admissions instruments are listed below. Following the list are descriptions and
sample questions.
o Test coverage and use
o Appropriate samples for test validation and norming
o Reliability
o Predictive validity
o Content validity
o Construct validity
o Test administration
o Test reporting
o Test and item bias
Test coverage and use. There must be a clear statement of recommended

uses and a description of the population for which the test is intended. The
principal question to be asked is whether it is appropriate for the intended
purposes and students. The use intended by the test developer must be justified
by the publisher on technical grounds. The user must then evaluate intended use
against the publisher's intended use and the characteristics of the test.
Questions to ask are:
1. What are the intended uses of the test?
2. What types of interpretations does the publisher feel are appropriate?
3. Are foreseeable inappropriate applications identified?
4. For whom is the test designed?
5. What is the basis for considering whether the test is applicable to my

students?
Appropriate samples for test validation and norming. The

samples used for test validation and norming must be of adequate size and must
be sufficiently representative to substantiate validity statements, to establish
57
appropriate norms, and to support conclusions regarding the use of the
instrument for the intended purpose.
The individuals in the norming and validation samples should be representative
of the group for which the test is intended in terms of age, experience and
background.
1. How were the samples used in pilot-testing, validation and norming chosen?
2. Are they representative of the population for which the test is intended?
3. How is this sample related to the population of students?
4. Were participation rates appropriate?
5. Can I draw meaningful comparisons between my students and these

students?
6. Was the number of test-takers large enough to develop stable estimates with
minimal fluctuation due to sampling errors?
7. Where statements are made concerning subgroups, is the number of test-

takers in ea('h subgroup adequate?
8. Do the difficulty levels of the test and criterion measures (if any) provide an
adequate basis for validating and norming the instrument?
9. Are there sufficient variations in test scores?
10. How recent was the norming?
Reliability. The test must be sufficiently reliable to permit stable

estimates of individual ability. Fundamental to the evaluation of any instrument
is the degree to which test scores are free from various sources of measurement
error and are consistent from one occasion to another. Measurement error will
always contribute adversely to an individual's score and will lower the reliability
of the test.
Different types of reliability estimates should be used to estimate the
contributions of different sources of measurement error. Inter-rater reliability
coefficients provide estimates of errors due to inconsistencies in judgment
between raters. Alternate-form reliability coefficients provide estimates of the
extent to which individuals can be expected to score the same on alternate forms
of a test. Of primary interest in selecting or constructing an admissions test are
estimates of internal consistency. These account for error due to content
sampling, usually the largest single component of measurement error.

58
1. Have appropriate types of reliability estimates been computed?
2. Have appropriate statistics been used to compute these estimates? (Split

half-reliability coefficients, for example, should not be used with speeded or
power tests as they will produce artificially high estimates.)
3. What are the reliabilities of the test for different groups of test-takers?
4. How were they computed?
5. Is the reliability sufficiently high to warrant the use of the test as a basis for
making decisions concerning individual students?
Predictive validity. The test must adequately predict academic

performance. In terms of an achievement test, predictive validity refers to the
extent to which a test can be appropriately used to draw inferences regarding
achievement. Empirical evidence in support of predictive validity must include a
comparison of performance on the test being validated against performance on
outside criteria.
A variety of measures are available as outside criteria. Grades, class rank,
other tests, teacher ratings, and other criteria have been used. Each of these
measures, however, has its own limitations.
There are several ways to demonstrate the relationship between the test being
validated and subsequent performance. Scatterplots, regression equations, and
expectancy tables should be considered in addition to correlation coefficients.
1. What criterion measure(s) have been used in evaluating validity?
2. What is the rationale for choosing this measure?
3. Is this criterion measure appropriate?
4. Is the distribution of scores on the criterion measure adequate?
5. What is the basis for the statistics used to demonstrate predictive validity?
6. What is the overall predictive accuracy of the test?
7. How accurate are predictions for individuals whose scores are close to cut-
points of interest?
Content validity. The test content must reflect whatever is being

studied (e.g., a behavior or an academic subject). Content-related evidence
establishes that the relevant content domain is represented in a test while content
outside the domain is not included.
59
Content validity will often be evaluated by an examination of the plan and
procedures used in test construction. The test development procedure must
follow a rational approach to ensure appropriate content. That process should
ensure that the collection of items would be representative of appropriate skills.
I. Is there a clear statement of the universe of skills represented by the test?
2. What is the basis for selecting this set of skills?
3. What research was conducted to determine desired test content and/or evaluate
it once selected?
4. Were the procedures used to generate test content and items consistent with
the test specifications?
5. What was the composition of expert panels used in content validation?
6. What process was used to elicit their judgments?
7. How similar is this content to the content I am interested in testing?
Construct validity. The test must measure one or more aspects of a

trait or theory (construct). Construct validity refers to the extent to which a test
measures a construct or trait that was postulated to explain observable behavior.
Intelligence, self-esteem, and creativity are examples of such psychological
traits.
Evidence in support of construct validity can take many forms. One
approach is to demonstrate that the items within a test or sub-test are inter-
related and therefore measure a single construct. Inter-item correlation and factor
analysis are often used to demonstrate relationships among the items.
Another approach is to demonstrate that the test measures what one would expect
it to measure. One might expect a measure of creativity to show a greater
correlation with a measure of artistic ability than a measure of scholastic
achievement.
I. Is the conceptual framework for each tested construct clear and well-founded?
2. What is the basis for concluding that the construct is related to the purposes
of the test?
3. Does the framework provide a basis for testable hypotheses concerning the
construct?
4. Are these hypotheses supported by empirical data?

60
Test administration. Detailed and clear instructions outlining
appropriate test administration procedures must be provided. Statements
concerning the validity of a test for an intended purpose and the accuracy of the
norms associated with a test can only generalize to testing situations which
replicate the conditions used to establish validity and obtain normative data.
Test administrators need detailed and clear instructions in order to replicate
these conditions. All test administration specifications, such as instructions to
test takers, time limits, use of reference materials, use of calculators, lighting,
equipment, assigning seats, monitoring, room requirements, testing sequence,
and time of day, should be fully described.
1. Will test administrators understand precisely what is expected of them?
2. Do the test administration procedures replicate the conditions under which

the test was validated and nonned?
3. Are these procedures standardized?
Test reporting. The methods used to report test results, including scaled
scores, subtest results and combined test results, must be described fully along
with the rationale for each method. Test results should be presented in a manner
that will help schools, teachers, and students make decisions that are consistent
with appropriate uses of the test. Help should be available for interpreting and
using the test results.
1. How are test results reported to test-takers?
2. Are they clear and consistent with the intended use of the test?
3. Are the scales used in reporting results conducive to proper test use?
4. What materials and resources are available to aid in interpreting test results?
Test and item bias. The test must not be biased or offensive relative to
race, sex, native language, ethnic origin, geographic region or other factors.
Test developers are expected to exhibit a sensitivity to the ethnographic and
demographic characteristics of test-takers, and steps should be taken during test
development, validation, standardization, and documentation to minimize the
influence of cultural factors on individual test scores. Tests do not yield
equivalent mean scores across population groups. If they did do so, one could
inappropriately assume that all groups have had the same educational and cultural
experiences. Rather, tests should yield the same scores and predict the same
likelihood of success for individual test-takers of the same ability, regardless of
group membership.
61
1. Were reviews conducted during the test development and validation process
to minimize possible bias and offensiveness?
2. How were these reviews conducted?
3. What criteria were used to evaluate the test specifications and/or test items?
4. What was the basis for these criteria?
5. Were the items analyzed statistically for possible bias?
6. What method or methods were used?
7. How were items selected for inclusion in the final version of the test?
8. Was the test analyzed for differential validity across groups?
9. How was this analysis conducted?
10. Does the test predict the same likelihood of success for individuals of the
same ability, regardless of group membership?
11. Was the test analyzed to determine the English language proficiency required
of test-takers?
12. Is the English proficiency requirement excessive?
13. Should the test be used with individuals who are not native speakers of
English?
Sources of Test Information
Finding the right test for a particular purpose can be quite difficult. The
evaluator must identify a variety of potentially useful tests, collect and review
technical materials, and identify and evaluate the practical considerations. This
section is designed to help with the first step--identifying useful instruments.
Books which contain lists of available instruments, reviews, and online
information retrieval systems are described below. These descriptions are taken
from the ERIC Digest Finding Information on Standardized Tests
(Rudner & Dorko, 1988).
A vailable Tests
The following books contain basic, non-evaluative information about a wide

range of available tests.
62
Mitchell, James V. Jr. (ed.), Tests in Print III (TIP III): An Index to
Tests, Test Reviews, and the Literature on Specific Tests.
Buros Institute of Mental Measurements, University of Nebraska Press, 901
North 17th Street, Lincoln, Nebraska 68588-0520, (402) 472-3581, 1983,
714 pages.
Keyser, Daniel J., and Sweetland, Richard C. (eds.), Tests: A

Comprehensive Reference for Assessment in Psychology,
Education, and Business (2nd ed.). Test Corporation of America,
Westport Publishers, Inc., 330 W. 47th Street, Kansas City, Missouri
64112, (816) 756-1490, 1986, 1,296 pages.
The Educational Testing Service Test Collection Catalog,

Volume I: Achievement Tests and Measurement Devices.
Oryx Press, 2214 North Central Avenue, Phoenix, Arizona 85004-1483,
(800) 457-6799, 1986,296 pages.
Krug, Samuel E. (ed.), Psychware: A Reference Guide to Computer-

Based Products for Behavioral Assessment in Psychology,
Education, and Business. Test Corporation of America, Westport
Publishers, Inc., 330 W. 47th Street, Kansas City, Missouri 64112, 1984,
816 pages.
Pletcher, Barbara P., Locks, Nancy A., Reynolds, Dorothy F., and Sisson,
Bonnie G. A Guide to Assessment Instruments for Limited
English Speaking Students. Santilla Publishing Company, New
York. Out-of-print. Available through ERIC Document Reproduction
Service, 3900 Wheeler Avenue, Alexandria, Virginia (800) 227-3742,1977,
223 pages.
Test Reviews
Several books provide in-depth, candid reviews of available tests. The best-
known are:
Mitchell, James V. Jr. (ed.), The Ninth Mental Measurement

Yearbook. Buros Institute of Mental Measurements, The University of
Nebraska-Lincoln, 135 Bancroft Hall, Lincoln, Nebraska 68588-0348, (402)
472-3581, 1985, 2,002 pages. The Yearbooks, published periodically
since 1932, are a comprehensive source of factual and evaluative information
about commercially available tests.
Keyser, Daniel J., and Sweetland, Richard C. (eds.), Test Critiques. Test
Corporation of America, Volume I, 1985, 800 pages; Volume II, Test
Corporation of America, Westport Publishers, Inc., 330 W. 47th Street,
63
Kansas City, Missouri 64112, (816) 756-1490, 1985, 872 pages; Volume
III, 1985, 784 pages; Volume IV, 1986, 768 pages; Volume V, 608 pages.
Online Information Retrieval Systems
Identifying and searching test information can be quite simple for people
who have access to the online database system managed by Bibliographic
Retrieval Services (BRS), 1200 Route 7, Lantham, New York, 12110, (800)
345-4277.
The Educational Testing Service File (ETSF)
(ETS) Test Collection, Educational Testing Service, Rosedale Road, Princeton,

N. 1. 08541, (609) 921-9000.
Mental Measurements Yearbook Database (MMYD) (See complete

address under "Test Reviews".)
Other Sources for Testing Information
Other sources for testing information are described in:
Fabian, Emily, and O'Brien, Nancy. Testing Information Sources for

Educators. ERIC Clearinghouse on Tests, Measurement and Evaluation,
American Institutes for Research, 3333 N. Street, NW, Suite 200,
Washington, DC 20007, (202) 342-5060. Report TME-94, 1987, 61
pages.
Crosby-Muilenburg, Corryn. Psychological and Educational Tests: A

Selective Annotated Guide. ERIC Document Reproduction Service,
1988,35 pages. (ED 293896)
Test Descriptions
Following are descriptions of some tests used by SCDE to aid in education

program admissions decisions. Their inclusion here is not an endorsement.
Rather, they are listed as an aid for starting a search.
These instruments should be considered in conjunction with an evaluation of
the publisher's intended purpose and the user's intended purpose. It must be
emphasized that no test should ever be used as the sole criterion for admissions
decisions. Tests are neither perfectly valid nor perfectly reliable.
64
Basic Skills Tests
PPST - The Pre-Professional Skills Test, developed by the

Educational Testing Service, Rosedale Road, Princeton, N. J. 08541, is the
most commonly used teacher education admissions test. The test measures basic
proficiency in reading, writing, and mathematics. NTE Core Battery, the
Educational Testing Service, Rosedale Road, Princeton, N. J. 05841 - Most
often used in certification testing, the NTE Core Battery is used by some schools
for admissions. The Core Battery covers the communication skills of listening,
reading and writing; the general knowledge of social studies, mathematics,
literature and fine arts, and science; and the professional knowledge of teaching.
Only the communication skills and general knowledge tests are used in
admission testing.
MAPS Multiple Assessment Programs and Services,
developed by the College Board, 888 Seventh Ave. Box 886, New York, N.Y.
10101, contains the Descriptive Tests of Language and Mathematics Skills.
Language skills cover reading comprehension, vocabulary, logical relationships,
usage, and sentence structure. Mathematics covers arithmetic, elementary
algebra, intermediate algebra, and functions and graphs.
CBEST - California Basic Education Skills Test, developed by
the Educational Testing Service, Rosedale Road, Princeton,N. J. 08541, service
for the CBEST Program, provides a general measure of basic proficiency in
reading, writing, and mathematics. This is a secure test administered 6 times a
year.
Nelson-Denny Reading Tests, 8420 Bryn Mawr Ave., Chicago, IL
60631, - published by the Riverside Publishing Company, this test was designed
to assess student achievement and progress in vocabulary, comprehension, and
reading rate. Testing time is 35 minutes. Thirty-five test booklets cost $22.17.
SAT - The Scholastic Aptitude Test, by Educational Testing
Service, Rosedale Road, Princeton, N. J. 08541, (609) 771-7588, developed for
the College Board, is a widely used college admissions test. College entrance
scores on this instrument are used by some colleges to evaluate teacher education
applicants. Achievement Tests, also available through ETS, are hour-long
subject-specific tests. Cost is $20.00 and testing dates are the same as those for
the SAT.
ACT Assessment, by the American College Testing Program, ACT
Registration, P. O. Box 414, Iowa City, IA 52243, is one of the two popular
college admissions tests. College entrance scores on this comprehensive
assessment of academic achievement are used by some colleges to evaluate
teacher education candidates. Separate scores are available for English usage,
mathematics usage, social studies reading, and natural sciences reading.
Commitment To Teaching
SDS - Self Directed Search, by Psychological Assessment, Inc., P.

O. Box 98, Odessa, FL 33556. This instrument yields six interest scores:
realism, investigative, artistic, social, enterprising, and conventional.
65
CMI· Career Maturity Inventory, by CTBIMcGraw-HiIl, Del Monte
Research Park, 2500 Garden Road, Monterey, CA 93940. This instrument
assesses a student's maturity regarding career decisions. One scale, based on the
student's attitudes, provides scores for decisiveness, involvement, independence,
orientation, and compromise.
SPV • Surveyor Personal Values, published by Science Research
Associates, Inc., 155 North Wacker Drive, Chicago, IL 60606. This
instrument provides six scores: practical mindedness, achievement, variety,
decisiveness, orderliness, and goals orientation. Testing time is 15-20 minutes.
SIV • Surveyor Interpersonal Values, by Science Research
Associates, Inc., 155 North Wacker Drive, Chicago, IL 60606. This
instrument measures six values by assessing what is important in relation to
others. The values covered are support, conformity, recognition, independence,
benevolence, and leadership. Testing time is 15 minutes.
Psychological Measures
16 PF • Sixteen Personality Factor Test, published by the

Institute for Personality and Ability Testing, Inc., Test Services Division, P. O.
Box 188, Champaign, IL 61820. This instrument was originally published in
1949. The 16 PF provides 16 primary factor scores including: emotional
stability, assertiveness, conscientiousness, shyness, tough-mindedness, group
dependence, tension, anxiety, neuroticism, and rigidity. Testing time is 45-60
minutes. A Spanish version is available.
CPI California Psychology Inventory, by Consulting
Psychologists Press, Inc., 577 College Ave., Palo Alto, CA 94306. This
instrument assesses 18 socially desirable behavioral tendencies, such as
dominance, sociability, self-acceptance, self-control, tolerance, flexibility, and
intellectual efficiency. This 45-60 minute test is also available in Spanish,
Italian and German.
TAT • Thematic Apperception Test, by Harvard University Press,
79 Garden Street, Cambridge, MA 02138. This instrument assesses personality
through projective techniques. The test seeks to measure temperament,
emotional maturity, creativity, sense of reality, and other personality traits.
Testing time is 1 hour. A trained examiner is required.
Interview Forms
Interviews are often seen as an effective, yet informal, method of obtaining

data and information. The most effective interviews, however, are those in
which the interviewer has identified the information he or she wishes to elicit
ahead of time. Good interviewers often have an interview form they fill out with
the applicant.
Most interview forms are locally developed and locally used instruments.
There is little documentation on these forms and they are not easily obtained.
Some impressive forms used to gather data on program expectations,
commitment to teaching and career choice can be found in Sears, et.al. (1986).
66
These instruments were developed as part of a longitudinal project related to
teacher recruitment alternately known as the Brackenridge Internship Teaching
Program and the Bridenthal Internship Program.
Measures of Learning Style
Teacher education can benefit from a good match between student learning
style characteristics and program attributes. Not surprisingly, measures of
learning style characteristics are being considered in teacher education (Van Cleaf
and Schade, 1987). Caution is urged as these measures are often plagued with
low test-retest reliabilities and classification instability.
Myers Briggs Type Indicators, by Consulting Psychologists Press,
Inc., 577 College Avenue, Palo Alto, CA 94306. Frequently used in business
settings, the MBTI examines preferences for extroversion or introversion,
sensing or intuitive perception, thinking or feeling judgment, and judgment or
perception.
Kolb Learning Style Inventory, by McBer and Company, 137
Newberry Street, Boston, MA 02116.
SUMMARY
This report outlined some of the issues surrounding admissions testing and
provided practical information. Sources of information about tests were
identified, several tests were described, criteria for evaluating tests were discussed,
and some current testing practices were identified.
Neither the author nor the staff at the Center for Teacher Education
Evaluation advocates the indiscriminate use of any test. Likewise, no specific
instrument or criterion is recommended for use in candidate selection. The
development of a defensible set of admissions criteria is a necessary step in the
overall process of improving teacher education programs.
Admissions testing can prove to be useful for screening applicants. Before
adopting a testing program, however, testing needs and goals should be carefully
identified. Tests should then be evaluated against those needs and goals. Once
the test is selected, meaningful cut-scores should be established.
Before the needs and goals of an admissions testing program can be
identified, the needs and goals of the teacher education program itself must be
identified and prioritized, and a plan developed for addressing them. The
Accreditation Plus Model for teacher education evaluation was developed in
response to the understanding that the design of an educational program cannot
be a linear or sequential task. Rather, it is possible to begin the process at any
one of a number of points and move through all phases of the design process.
67
REFERENCES
American Psychological Association, American Educational Association, & the

National Council on Measurement in Education. (1985). Standards for
educational and psychological tests and manuals. Washington,
DC: American Psychological Association.
Antes, R. L. (1985). Teacher education admission testing:
Development and implementation. Bloomington, IN: Indiana
University. Coalition of Teacher Education programs. (ED 272 498)
Astin, A. (1985). The American freshman. Los Angeles, CA: American
Council on Education and the University of California at Los Angeles.
Clinton, B. (1983). Address before the special session of the Arkansas General
Assembly, Little Rock, AR.
Eissenberg, T. E. & Rudner, L. M. (1988). The status of state teacher
testing programs-1988. Washington, DC: American Institutes for
Research, ERIC Clearinghouse on Tests, Measurement and Evaluation,
Digest
Feistritzer, C. (1983). The condition of teaching. Princeton, NJ:
Carnegie Foundation for the Advancement of Teaching.
Holmes Group. (1986). Tomorrow's teachers. East Lansing, MI.
Kapel, D. E., Gerber, P., & Reiff, H. B. (1988). Psychological and
social/emotional fitness for teacher education students: A
national survey of current practices. Paper presented at the annual
meeting of the American Educational Research Association, New Orleans,
LA.
Marshall, J. D., Sears, J. T., & Otis-Wilborn, A. K. (1988). Recruitment
and induction of "quality" students into teacher education: A
case study. Paper presented at the annual meeting of the American
Educational Research Association, New Orleans, LA.
National Commission for Excellence in Teacher Education. (1985). A call
for change in teacher education. Washington, DC: U.S.
Government Printing Office.
National Education Association. (1982). Excellence in our schools.
Washington, DC. (ED 246 046)
Pugach, M. C. & Raths, J. (1983). Testing teachers: Analysis and
recommendations. Journal of Teacher Education, 34(1), 37-43.
Rudner, L. M. & Dorko, K. (1988). Finding information about
standardized tests. Washington, DC: American Institutes for Research,
ERIC Clearinghouse on Tests, Measurement and Evaluation, Digest.
Rudner, L. M. (1988). Teacher testing--an update. Educational
Measurement: Issues and Practices, 7, 1, 16-19. "
Rudner, L. M. & Eissenberg, T. E. (1988). Standard setting practices in
state teacher testing programs. Paper presented at the annual
meeting of the National Council on Measurement in Education, New
Orleans, LA.
Schlechty, P. C. & Vance, S. V. (1981, October). Do academically able
teachers leave education? Phi Delta Kappan, 63, 106-112.
68
Sears, J. T., et al. (1986). A longitudinal study of a demonstration
project related to the recruitment, preparation, and retention
of highly qualified persons for the teaching profession: The
Brackenridge interns--the first year. Unpublished manuscript. (ED
272 454)
Shim, c. (1965). A study of the cumulative effects of four teacher
characteristics on the achievement of elementary school pupils. Journal
of Educational Research, 59,(1), 33-34.
Sykes, G. (1983, March). Teacher preparation and the teacher workforce:
Problems and prospects for the 80's. American Education, 23-29.
Taylor, G & Miller, P. (1985). Professional coursework and the practicum:
Do good students make good teachers? Canadian Journal of
Education, 10, 32, 105-120.
U.S. Department of Education. (1983). A nation at risk. Washington,
DC: U.S. Government Printing Office.
Van Cleaf, D. & Schade, L. (1987). Student teacher learning styles: Another
dimension of reform. Teacher Education and Practice, 4, 1, 25- 24.
7
EV ALUA TING FIELD-BASED EXPERIENCES IN

TEACHER EDUCATION
Nancy L. Zimpher
The Ohio State University
Contemporary descriptions of the uses and abuses of field experiences in the

preservice education of teachers abound. These are found in historical
documentations of the role of student teaching and field experiences in teacher
education (Mead, 1930; Curtis & Andrews, 1954; Stratemeyer & Lindsey, 1958;
and Andrews, 1964) and from resources cited in the current literature. A recent
Educational Resources Information Center (ERIC) search (Zimpher, 1987)
generated 1,852 entries from 1966 to the present from the combined categories of
"student teaching" and "field experiences." A series of "state of the art" reviews
of field experiences and student teaching also informs our understanding of both
the nature of these experiences and of how teacher educators have studied these
experiences. A review of these analyses provides a brief overview of issues and
concerns related to field experience. This chapter offers recommendations
regarding the organization and offering of programs of field experience in teacher
education and a mooel for designing appropriate evaluation mechanisms for field
experiences appropriate to the Center for Teacher Education Evaluation's
Accreditation Plus Model.
The chapter begins with a brief review of the literature surrounding the
evaluation of field-based, or student teaching, experiences which is necessary in
order that the recommendations and model which follow will be seen in their
proper context. Following the literature review, a design for the evaluation of
field and laboratory experiences is presented.
REVIEW OF THE LITERATURE
Trends and Issues in Field Experiences
In a comprehensive analysis of the nature of field experiences in teacher

education, Mcintyre (1983) organized literature relative to the nature of field
experiences around four variables:
70
1. Stages through which students pass in adjusting to their responsibilities;
2. Role relationships between and among cooperating teachers, university

supervisors, and field experience or student teachers;
3. Attitude changes in students in field experience and student teaching; and
4. Socialization of education students through the use of field experiences.
Mcintyre (1983) drew from two analyses of the developmental stages

through which student teachers pass (Caruso, 1977 and Sacks & Harrington,
1982). These phases, in both instances, range from dichotomous feelings of
anxiety and euphoria at the beginning of a field placement through ultimate
separation from student teaching and field experience which is categorized as loss
and relief. A second continuum offers a characterization of phases from
anticipation through mastery. This latter set of stages is similar to Fuller and
Bown's (1975) analysis of the survival concerns of beginning teachers in their
initial field and student teaching experience and their ultimate ability to move
toward concerns more focused on instructional methodology and ultimately to
the learning needs of students and the outcomes of instruction. Mcintyre also
documented classical concerns with regard to the role relationships of
cooperating teachers and university supervisors on the instructional and
professional development of field experience students and student teachers. He
cited the Yee study (1969) in which cooperating teachers were believed to have
more influence than university supervisors and other studies which criticized
university supervisors for lacking leadership potential (Diamonti, 1977) and for
the absence of effect on attitude of university supervisors (Morris, 1974).
Attitude changes as variables. Many other scholars have documented

more positive influences of university supervisors as coaches (Friebus, 1977)
and as critics in the absence of critical feedback from cooperating teachers
(Zimpher, deVoss, & Nott, 1980). These role relationships are contrasted to the
absence of control over classroom events felt by field experience students and
student teachers as documented by Tabachnick, Popkewitz, and Zeichner (1979-
80). Attitude changes will not be discussed in this chapter because a brief review
of the research on the subject would simply highlight the number of studies
presented by Zeichner (particularly in 1978) and others about the ideological
orientation of cooperating teachers toward teaching and the degree to which the
effects of perspectives about teacher education programs are counted by the
strength of these ideological influences from cooperating teachers. The fact that
attitude changes are related to the socialization of teachers into the profession is
well documented by Lortie (1975) and Zeichner (1980).
Research on Field Experiences
This review, however brief, of the many issues related to field experiences
can be viewed in the context of the synthesis pieces prepared on early field
71
experience and field experience in general by Applegate (1987) and Zeichner

(1987). Applegate presented an analysis of the comprehensive nature of field
experiences:
Whether the field experience is for observation, participation, tutoring,

small group, or whole class instruction; whether the experience is one-
day, once a week, or semester-long; whether the experience begins early
in the student's preparation or begins when the student enters a
professional semester of preparation; whether the experience is
supervised, organized, or standardized does not appear to have certain
substantial impact (p. 90).
Applegate (1987) analyzed multiple studies which documented the nature of

field experiences and she discovered over 50 different variables that affect the
analysis of early field experiences. She postulated a set of constructs about field
experiences that are important to this analysis, including the complexity of the
relationships and the critical importance of the individuals involved and their
perceptions about the likely success of a field experience. Further, she noted that
what happens as students engage in field experiences goes far beyond the
development of teaching attitudes and skills, extending to the degree of
responsibility held by critical actors. There is great confusion about role among
cooperating teachers, university supervisors, and field experience students;
however, all of the actors seem to find the experiences valuable, the process of
communication is difficult, the absence of parity is troublesome, and the likely
resolution of problems is too often rather circular.
This analysis can be embedded in a threefold analysis of field experience by
Zeichner (1987) in which the nature of field experiences is acknowledged to have
an ecological perspective. This perspective compares relationships among field
experience program features, the nature of the settings in which field experiences
occur; and the role relationships and perspectives of the people involved in field
experiences. Zeichner also registers some concern about the fact that our prolific
studies of the field experiences become interventions to the nature of field
experiences (Tabachnick & Zeichner, 1983). In an analysis of the nature of
research relative to field experiences, researchers have made reference to the
largely atheoretical nature of our work (Shutes, 1975) and to the need for new
theoretical developments in studying the processes and outcomes of teacher
education programs and field experiences (Turner, 1975) in which field
experiences play an extremely important role. The research has been somewhat
fragmented, failing to recognize the antecedents and consequences of one
member's behavior on all others in the experience (Zimpher, deVoss, & Nott,
1980) or to associate the context of student teaching with the actions of
individuals within this context (Zeichner, 1987).
Perhaps most promising in the literature that exists on field experience and
student teaching is the evolution of a conceptualization of field experiences that
is grounded in expectations for what "ought to be" the nature of field
experiences. This conceptualization is largely an outgrowth of Dewey's (1904)
distinction between the ways in which "the habits of the teacher as a teacher may
72
be built up" (p. 15). This conceptualization differentiates among the habits that
help the teacher become thoughtful and alert as a student of teaching as opposed
to those which make the teacher immediately proficient in terms of technique,
but not necessarily reflective about teaching. As a consequence, Dewey
distinguished the outcomes in field experiences related to a conception of field
experience as an apprenticeship (having a utilitarian character) or field
experiences as laboratory (fostering personal inquiry and reflection). Zeichner
(1983) drew on this set of distinctions to present a continuum of alternative
paradigms of field experience in teacher education. She contrasted an
instrumental perspective of learning to teach with designing field experiences for
purposes of reflective thinking as epitomized by Dewey's distinction of the
laboratory approach. Zeichner referred to this approach as "the inquiry-oriented"
laboratory paradigm. This conceptual approach to field experiences provides a
background for an analysis of field experiences as programmatic experiences,
which follows.
Program Attributes of Field Experiences
In this section of the chapter, conceptualizations of the potential

programmatic features of field experiences are referenced to help the reader focus
on the nature of field experiences and the subsequent assessment of these
experiences.
Conceptualizations of Field Experiences. One conceptualization

of field experiences already alluded to is the ecological view of field experiences
from the framework proposed by Zeichner (1987). A second conceptualization
offered by Griffin (1987) derived from three major studies conducted through the
former University of Texas Research and Development Center on Teacher
Education. These three studies addressed:
1 a multi-site, multi-method investigation of preservice clinical teacher

education experiences;
2 a quasi-experimental inquiry into the effects of leadership training for

teaching personnel with regard to teacher effectiveness and development; and
3 a study of state-mandated teacher induction programs.
From these three different aspects of teacher education, Griffin derived a set
of "program features" that could define the nature of field experience programs.
A third conceptualization is drawn from recently completed case studies of

six midwestern elementary teacher education programs by Howey and Zimpher
(1989), which includes an analysis of the nature of courses and pedagogical
experiences, the opportunities for teachers in training to learn to teach, and the
extent to which programs of teacher education exist. From their analysis, they
derived a set of promising "program attributes" which they believed characterized
73
the distinctive elementary teacher education programs which they studied. Drawn
from these conceptualizations, four critical features in the design, conduct, and
study of programs of field experiences are presented below.
Critical Features in Field Experience Programs
To understand and review these four clusters of program attributes one must
understand why it is being proposed that field experiences be looked at more
programmatically. In part, the need for a more programmatic approach to field
experience stems from deficiencies in teacher education programs as identified in
follow-up studies. Responding in a number of these studies (Drummond, 1978;
deVoss, 1978, 1979, 1980; & Loadman, 1983), students noted deficiencies in
their own teaching ability which they attributed to a lack of treatment of those
competencies in their teacher education programs. Particularly noted were those
technical problems related to classroom discipline, classroom management, the
ability to effectively motivate students, and interactions with parents and other
members of the school community. In an analysis of research on teacher
education, Koehler (1985) discussed a series of problems associated with
beginning teaching which were identified by students who had recently graduated
from teacher education programs. Students attributed their deficiencies to a lack
of effective treatment of these concerns in their teacher education programs. As
Koehler observed, these are issues which are clearly placed in the curricula
typical of teacher education programs, but which appear to present content in the
form of answers to questions not yet asked or posed by students. This led
Koehler to observe that in teacher education we experience a "feed forward
problem" wherein we give students information which they, out of context, have
little motivation to learn and then when in context (that is, in the classroom),
cannot recall, or in some instances claim they never covered in their program.
Carter and Koehler (1987) posited that what would be more helpful to
students in the acquisition of knowledge about teaching might be "event
structured knowledge" wherein knowledge about teaching and learning could
become integrated by prospective teachers who acquire knowledge and skills in
the context of classroom events and then process the effects or problems
encountered in the utilization of these skills to approximate utilization of the
skill in the classroom setting. Educators must begin to develop a rationale for
thinking more coherently abQut field experiences. Field experiences must be
integrated with total programs of teacher education, and it is to this integration
that the four program attribute clusters are directed in the next section.
Integration of Field Experiences Into Teacher Education

Programs
The first program feature is identified by Griffin's (1987) notion that

programs are, or should be, purposeful, articulated, analytic, and reflective. This
is parallel to Howey and Zimpher's (1989) belief that programs of teacher
education should be designed around clear and shared conceptions of teaching and
learning. In her treatment of conceptualizations for teacher education programs,
74
Barnes (1987) made the case for designing programs and goals for programs
against conceptualizations to achieve desired program outcomes. In this sense
the conceptual framework presents the "program's assumptions, philosophy and
research base and outlines the implications of that knowledge for teaching" (p.
14), and then builds throughout the program repeated themes which extend,
illustrate and elaborate the conceptual framework. Thus, a critical feature of field
experiences is that these field experiences link in a purposive way to the central
conception of the teacher education program. This informs students, cooperating
teachers, and university supervisors of the critical and over-riding
conceptualization of teaching and learning being adhered to in the program.
Such a conceptualization could be, as Griffin proposes, an analytic and reflective
orientation. A host of other conceptions about teaching and learning could be
fostered either simultaneously or with one having priority over another.
IIIustrations of these conceptions could be, metaphorically, the notion of teacher
as: (a) executive manager, (b) artisan, (c) decision maker or problem solver, or
(d) reflective practitioner.
The second feature critical to the development of field experience programs
relates to contextual issues. Griffin (1987) and Zeichner (1987) posit that field
experiences must be attended to within the context of the setting, or be what
Griffin calIs "context-sensitive." The selection of a field site, the nature of
teaching and learning in that environment, the climate of the classroom, the
nature of the student population, the perspectives of the staff, the philosophy of
leadership embodied by the principal and other teachers, and the conditions of
schooling in a particular school district, city or culture, impact on the nature of
the field experience. Thus, in an analysis or assessment of field experiences, the
evaluator must take into account dimensions which describe the nature of the
field experience from a contextual or ecological perspective.
The third program feature that programs should be organized around are clear
goals and objectives. The underlying assumption must be that one goal of the
program is to create within students dispositions toward the acquisition of
knowledge about teaching and the subsequent utilization of that knowledge.
This is accomplished, not exclusively from a skill orientation, but through a
disposition to use what we know about teaching in order to be productive in the
classroom. Griffin (1986) acknowledges that programs should be built around
the knowledge base on teaching. The Howey and Zimpher (1989) treatment of
this concept focuses on the extent to which programs of teacher education,
including field experiences, are designed with knowledge of the empirical bases
for understanding teaching and learning. These include the research on teaching,
the research on effective schooling, and the research on a host of mediating
variables that impact on the nature of teaching and learning in schools. The
degree to which field experiences focus on a clear empirical or knowledge base
for teaching should be a part of the analysis, evaluation, and assessment of the
nature of these field experiences in teacher education programs.
Finally, and importantly, programs are (a) developmental and ongoing
(Griffin, 1987) and (b) articulated and reiterative (Howey and Zimpher, 1989).
Throughout the construction of the articulated and integrated programs (in tenns
of individual courses, blocked experiences, and laboratory and clinical
75
experiences), there occurs a series of related concepts or ideas. Integration occurs

between the didactic program and the field experience program through
movement from:
I. simple to more complex understandings,
2. observations to more intentional field experiences,
3. the familiar to the less-familiar experiences and contexts, and
4. less complex understandings of techniques of teaching to more complex

notions of teaching and learning.
This is a planned repetition rather than program redundancy and would allow an
evaluator to look at the nature of field experiences and look for the reiterative
and reinforcing concepts that are ongoing and developmental throughout the
program.
These four program attributes or features provide a backdrop for a further
analysis of the nature of field experiences with an eye toward how best to most
effectively evaluate these experiences.
A DESIGN FOR THE EVALUATION OF FIELD EXPERIENCES
The design for the evaluation of field experiences proposed in this section of
the chapter draws heavily on two sources:
1. a documentation and assessment system for teacher education programs

designed and implemented at The Ohio State University (Zimpher &
Loadman,1985);and
2. an analysis of different orientations to teacher competency (Zimpher &

Howey, 1986), which could undergird an evaluation design.
Models and Approaches
There are a series of principles or assumptions about the nature of

evaluation (documentation and assessment) systems which might be helpful to
individuals who are developing systems to evaluate field experiences. The first
of numerous approaches to the design of evaluation systems are those which
look particularly at input and output systems wherein the input is usually
defined as program directives and the output as student attainment of these
objectives. A second popular model is that of competency based programs
which identify exactly those skills which are necessary and effective for
practitioners and converts those skills into tasks which are subsequently
embedded in the program and by which students are evaluated. A third design
for program evaluation is the use of follow-up studies as a summative evaluation
for the design of teacher education programs. In this instance, evaluative and
76
formal judgments are made about teacher candidates after they graduate from
teacher education programs and become practicing teachers. Each of these
systems has limitations, primarily with regard to the absence of more formative
data that would inform how students experience programs and how programs and
students are changed as programs progress. The system proposed herein is
multifaceted, and requires cumulative data gathering and analysis throughout the
student's matriculation in the program and takes a practice-oriented
developmental form. In this instance the system is organized against a set of
expectations, as follows:
o System findings must be the result of multiple and triangulated data inputs
and analyses.
o The system must stress description as well as evaluation.
o The system must contain both formative and summative elements.
o The system must provide for sequential and longitudinal data collection,
analysis, and usage.
o The system must provide for maximum student input.
o The system must have cross-group validity, simplicity, and manageability.
o The system must be legally responsible (Zimpher & Loadman, 1985, p. 11-
12).
This model encourages the multidimensional evaluation of field experiences,

which calls for data collection at various points in the program. This allows for
assessment of the developmental stages of the students' growth, a grounding in
the nature of the program, and the interaction of the program and the student.
This design implicitly fosters the notion that as programs and students interact,
both the students and the programs change accordingly. It is the obligation of
the evaluator to come to understand the nature of those individual and program
changes. Such a system is designed against a matrix of data components which
would be collected intermittently from the earliest field experience, to
observation experience attached to general and special methods, and throughout
the student teaching and/or internship experience. The components of the
program are divided into four parts and explicated below:
Component I: Descriptors. Component I contains demographic

information and descriptive data about schools attended, courses taken, degrees
received, and placement. The data provides a baseline which allows an easily
accessed profile on student progress as well as a comparative picture of students
both within and across departments.
77
Component II: Assessment. Component II contains all the system's

assessment instruments, both objective and subjective. Objective data include
student's test history, such as ACT/SATs, university math and English
placement tests, and the National Teacher Examinations. In keeping with the
holistic approach of the SIS, such records will always be displayed in the light
of other academic measures such as grades and class standing, and also in relation
to more triangulated and qualitative data generated through assessment
instruments designed for this component
Component III: Narrative. Component III consists of descriptive and

analytical materials written by the teacher candidate, advisor, college
instructors/supervisors, and cooperating teachers. These data, which will be
gathered at appropriate points throughout the teacher candidate's participation in
the program, will complement the data available in Components I and II. The
accounts should be a rich source of information about the teacher candidate's
experience and developing pedagogical style.
Component IV: Context. Component IV consists of descriptive

statements about the context, or the "experiential environment" in which teacher
training occurs. This material includes information about the classroom and
field settings which the student has experienced. For further information on the
operationalization of this assessment system, the reader is referred to the
monograph in which this framework is explicated, wherein sample
instrumentation is provided for a series of evaluations proposed within the
system (Zimpher & Loadman, 1985, p. 14-21).
What appears to be missing in this conceptualization is a clear explication
of the nature of teacher competence which would undergird both the central
conceptualization of field experiences and a system for evaluating these
experiences. Zimpher and Howey (1986), proposed a four-part framework as a
conceptualization for thinking about teacher competency as technical, clinical,
personal, and critical. Accordingly, each of these views of teacher competence
constitutes a total framework for looking at the nature of field and clinical
experiences. The framework is reproduced here for several reasons. First, the
nature of field experiences and the practices acquired by teachers are intended to be
multidimensional. Teaching is no more exclusively technical than it is
exclusively personal and stylistic. This framework encourages students,
cooperating teachers, university supervisors, and program designers to look at
the multifaceted and complex nature of teaching and learning. It also cautions
those who would design evaluation systems to look more comprehensively at
the nature of teacher competence as exhibited or occasioned by practice in field
experiences. These four aspects or perspectives on teacher competence are
described briefly below:
The conception of teaching inherent in technical competence
conceptualization is essentially that good teaching is determined by advanced
knowledge of what is to be learned, how it is to be learned, and the criteria by
which success is to be measured. Becoming technically competent involves
mastering strategies or methods of instruction and applying specific teaching
78
strategies to the selection and organization of the curriculum and structure of the
classroom. The knowledge base which undergirds the field and clinical
experiences from this perspective is a set of technical guidelines derived from
empirical knowledge about teaching and from a host of studies about effective
teaching. [See Pankratz's chapter for additional information on this topic.]
Clinical competence refers to problem solving and inquiry in the
classroom and provides a perspective for students in field experiences to think
about classroom activity through a problem solver or clinician framework. This
perspective of reflective action relies in part on personal theories and the
integration of theoretical and practical knowledge.
Personal competence emerges from:
1. the developmental concerns of teachers,
2. the personalization of teaching, and
3. know ledge of self actualization including:

a. self-awareness,
b. identity information, and
c. the interpretative capacities of self-confrontation, values clarification,
interpersonal involvement, and small group processes.
This perspective allows careful examination of the growth and development

of students as they engage in field experiences. It also requires the students
themselves to be reflective about their own personal and professional
development as well as that of the students whom they teach.
The critical competence is characterized by critical theorists whose
conceptions of teaching and learning relate to the social conditions of schooling.
Field-experience students can exercise their roles as inquirers and reflective
practitioners to become change agents. These newest proponents of change can
act to make school experiences more productive instead of simply repeating and
reflecting the existing conditions.
The four conceptualizations, or competencies, form a two-by-two matrix. It
is possible to view the competencies on a continuum from more simple to more
complex. For instance, from the technical competence perspective, individual
skill level could be attained in questioning techniques or the use of wait time.
More complex systems of instructional competence are those such as a system
for classroom management. The clinical perspective begins with the recording
of individual reflections in journals, logs, and critical event forms and moves to
a more complex reflectivity as the study moves toward inquiry. Ultimately, this
system of problem-solving is integrated into the critical perspective as complex
structures for ongoing action research in teachers' classrooms. These four
components are part of a four-fold evaluation system:
1. Program evaluation designs for field experience allow for the collection of
data incrementally throughout the field experience program, from earliest
experiences through the completion of student teaching and internship.
79
2. Data collection moves from data which are largely documentary and
descriptive to data which are analytical and competency oriented, and
ultimately to narratives which reflect the rich ecology of classrooms in
which field experiences occur and acknowledge the context-sensitive nature
of field experiences.
3. Teacher competence as explicated in field experiences can be represented by a

mUlti-perspective framework of teacher competencies including those that
are technical, clinical, personal, and critical in nature.
4. Finally, conceptions of teacher competence move from simple, more

practical notions of technique and competence to more elaborate and highly
complex instructional systems and critical inquiry.
A Categorical Analysis of Evaluation Materials
This section of the chapter is organized against the conceptual framework for
teacher competence explicated above including technical, clinical, personal, and
critical competence. Provided here are lists of representative texts and materials
that could be used by students in self assessment of growth and development
relative to teacher competence and also used as vehicles for formative and
summative program evaluation of field experiences. Included are:
o Inventories,
o Questionnaires,
o Cases useful for individual analysis and assessment, and
o Self-inventories.
These materials can be viewed as a data base for assessing student progress in
field experiences and a basis for an inquiry-oriented and reflective disposition
toward teaching among teacher candidates in field experiences. The letters
following each citation refer to the competence(s) the reference addresses (i.e.,
TC refers to Technical Competence, CLC refers to Clinical Competence, CRC
refers to Critical Competence, and PC refers to Personal Competence).
Sources of Self-Inventories
Berliner, D. C. (1985). Laboratory settings and study of teacher

education. Paper presented at the annual meeting of the American
Educational Research Association, Chicago, IL. (CLC).
Contains a series of laboratory experiences which are best used in on-campus
simulations. It provides a framework for thinking about the nature of field
experiences in teacher education.
80
Boyan, N. J., & Copeland, W. D. (1978). Instructional supervision

training program. Columbus, OH: Charles E. Merrill Publishing
Company. (CLC).
Describes methods of classroom observations, including systems that can be
constructed by pre service teachers and their supervisors. It is useful for
introducing students to classroom observation.
Carr, W., & Kemmis, S. (1986). Becoming critical: Education,

knowledge, and action research. London: The Falmer Press.
(CRC).
Presents conceptualizations of research paradigms and a structure for action
research in the classroom.
Cohn, N., Kottkamp, R., & Provenzo, E. (1987). To be a teacher:

Cases, concepts, observation guides. New York: Random House.
(TC, PC).
Uses a case study approach and contains many inventories, observation guides,
and other materials useful for analyzing the technical competence of a teacher.
Curwin, R. L., & Fuhrmann, B. S. (1975). Discovering 'your teaching

self: Humanistic approaches to effective teaching. Englewood
Cliffs, NJ: Prentice Hall. (TC, PC).
Looks at instructional method and patterned behaviors in the classroom.
Includes inventories, vignettes, worksheets, and other useful tools.
Haysom, J. (1985). Inquiring into the teaching process: Towards

self evaluation and professional development. Toronto: The
Ontario Institute for Studies in Education. (CRC).
Focus is on appraising the teaching process, particularly actions of teachers and
pupils, and on the process for implementing and monitoring classroom change.
Provides a structure for classroom action research.
Holly, M. L. (1984). Keeping a personal-professional journal.

Victoria: Deakin University. (CLC, PC).
Provides guidance for reflecting on events in classroom practice.
Hopkins, D. (1985). A teacher's guide to classroom research. Stony

Stratford, England: Open University Press. (CRC).
Another description of classroom action research, this is a practical guide for
teachers who want to examine the nature of teaching.
Kemmis, S., & McTaggart, R. (1982). The action research planner.

Victoria: Deakin University Press. (CRC).
This can be viewed as a companion piece to the Carr and Kemmis text. It is an
action research planner, a practical guide for the conduct of action research.
81
The Network, Inc. (1987). A compendium of innovative teacher

education projects. Andover, MA: The Network, Inc. (CLC).
A compendium of 29 projects sponsored by the U.S. Department of Education,
Office of Educational Research and Improvement. The project descriptions
include inventories and other instruments.
Sanford, J. P., & Emmer, E. T. (1988). Understanding classroom

management: An observation guide. Englewood Cliffs, NJ:
Prentice Hall. (TC)
Intended for use as an assessment vehicle to provide guidance in classroom
management, this book contains observational inventories to use in practicing
management skills.
Shulman, J. H., & Colbert, J. A. (1988). The mentor teacher case book.
San Francisco: Far West Laboratory for Educational Research and
Development, ERIC Clearinghouse on Educational Management and
Educational Clearinghouse on Teacher Education. (CLC)
Shulman, J. H., & Colbert, J. A. (1988). The intern teacher case book.
San Francisco: Far West Laboratory for Educational Research and
Development, ERIC Clearinghouse on Educational Management and
Educational Clearinghouse on Teacher Education. (CLC)
The two Shulman and Colbert casebooks present a series of cases about
individual practice and experience as well as guided analyses of the cases,
inventories, and discussion questions relative to the cases. They provide
descriptions of practice which can be used for charting progress of field-
experience students.
Zimpher, N. L., & Rieger, S. R. (1988). Using research knowledge to

improve teacher education: Implementation of an induction
program for inquiring professionals. Washington, DC: Office of
Educational Research and Improvement. (CLC)
Contains a reflectivity packet which includes a collection of instruments directed
at problem-solving, conference reporting, critical incidents, and other self-
reporting activities. The instruments could be adapted for evaluative purposes.
REFERENCES
Andrews, L. (1964). Student teaching. New York: Center for Applied

Research in Education.
Ashburn, E., & Fisher, R. (1984). Methods of assessing teacher
education students: Conference proceedings. Normal, IL:
Illinois State University and Washington, DC: American Association of
Colleges for Teacher Education.
Applegate, J. (1987). Early field experiences: Three viewpoints. In M.
Haberman and J. Backus (Eds.), Advances in teacher education.
Norwood, NJ: Ablex Publishing Company, 3, 75-93.
82
Barnes, H. L. (1987). The conceptual basis for thematic teacher education

programs. Journal of Teacher Education,34(4), 13-18.
Carter, K., & Koehler, V. (1987). The process and content of initial years of
teaching programs. In G. Griffin and S. Millies (Eds.), The First Years
of Teaching: Background Papers and a Proposal. Chicago:
University of Illinois-Chicago.
Caruso, J. (1977). Phases in student teaching. Young Children, 33(1), 57-
63 (ERIC FJ # 172 317).
Curtis, D., & Andrews, L. (1954). Guiding your student teacher.
Englewood Cliffs, NJ: Prentice Hall.
deVoss, G. (1978, 1979, 1980). Technical reports: Institutional
follow-up studies. Columbus, OH: The Ohio State University,
College of Education.
Dewey, J. (1904). The relation of theory to practice in education. In The
third yearbook of the National Society for Scientific Study
of Education. Bloomington, IL: Public School Publishing.
Diamonti, M. (1977). Student teacher supervision. Educational Forum,
41(4),477-486 (ERIC FJ # 163 579).
Drummond, R. (1978). 1976 follow-up of 1970-76 college of
education graduates. Orono, ME: University of Maine at Orono.
Friebus, R. (1977). Agents of socialization involved in student teaching.
Journal of Educational Research, 70(5), 263-268 (ERIC FJ# 168
755).
Fuller, F., & Bown, O. (1975). Becoming a teacher. In K. Ryan (Ed.), The
seventy-fourth yearbook of the National Society for the
Study of Education, Part II. Chicago: University of Chicago Press.
Griffin, G. (1986). Clinical teacher education. In J. Hoffman and S. Edwards
(Eds.), Reality and reform in clinical teacher education. New
York: Random House.
Griffin, G., Hughes, R., Defino, M., & Barnes, S. (1981). Student
teaching: A review. Austin: Research in Teacher Education Program,
R&D Center for Teacher Education, The University of Texas at Austin.
Howey, K., & Zimpher, N. (1989). Profiles of preservice teacher
education: Inquiry into the nature of programs. Albany:
SUNY Press.
Koehler, V. (1985). Research on preservice teacher education. Paper
presented at the National Conference on Teacher Education, Research and
Development Center for Teacher Education, University of Texas at Austin.
Loadman, W. (1983). Technical report: Instructional follow-up
studies. Columbus, OH: The Ohio State University, College of
Education.
Lortie, D. (1975) Schoolteacher: A sociological study., Chicago:
University of Chicago Press.
McIntyre, D. (1983). Field experience in teacher education.
Washington, DC: Foundations for Excellence in Teacher Education and the
ERIC Clearinghouse on Teacher Education.
83
Mead, A. (1930). Supervised student teaching. New York: Johnson

Publishing.
Morris, J. (1974). The effects of the university supervisor on the perfonnance
and adjustment of student teachers. Journal or Educational Research,
67(8), 358-362, (ERIC EJ # 097 038).
The Network, Inc. (1987). A compendium or innovative teacher
education projects. Andover, MA: The Network, Inc.
Sacks, S., & Harrington, G. (1982). Student to teacher: The process of
role transition. Paper presented at the annual meeting of the American
Educational Research Association, New York.
Shutes, R. (1975). Needed: A theory of teacher education. Texas Tech
Journal or Education, 2, 94-101.
Stewart, D. (1988). Materials on evaluation of teacher education programs in
the ERIC database. Journal or Teacher Education, 34(4), 23-27.
Stratemeyer, F., & Lindsey, M. (1958). Working with student teachers.
New York: Bureau of Publications, Teachers College, Columbia
University.
Tabachnick, B., Popkewitz, T., & Zeichner, K. (1979-80, Winter). Teacher
education and professional perspectives of student teachers. Interchange
on educational policy, 10(4), 12-29 (ERIC EJ # 233 075).
Tabachnick, B., & Zeichner, K. (1983). The impact or the student
teaching experience on the development of teacher
perspectives. Madison, WI: Wisconsin Center for Education Research.
Turner, R. (1975). An overview of research in teacher education. In K. Ryan
(Ed.), The seventy-fourth yearbook or the National Society
for the Study or Education, Part II. Chicago: University of Chicago
Press.
Yee, A. (1969). Do cooperating teachers influence the attitudes of student
teachers? Journal of Educational Psychology, 60(4), 327-332,
(ERIC EJ # 006 820).
Zeichner, K. (1978). Student teaching experience: A methodological
critique or the research. Paper presented at the annual meeting of the
Association of Teacher Educators, Las Vegas (ERIC ED # 166 145).
Zeichner, K. (1980). Myths and realities: Field-based experiences in preservice
teacher education. Journal or Teacher Education, 31(6).
Zeichner, K. (1983). Alternative paradigms of teacher education. Journal of
Teacher Education, 34(3), 3-9.
Zeichner, K. (1987). The ecology of field experience: Toward an understanding
of the role of field experiences in teacher development. In M. Habennan and
J. Backus (Eds.), Advances in Teacher Education. Norwood, NJ:
Ablex Publishing Company, 3, 118-150.
Zimpher, N. (1987). Current trends in research on university supervision of
student teaching. In M. Habennan and J. Backus (Eds.), Advances in
Teacher Education: volume 3, pp. 118-150. Norwood, NJ: Ablex
Publishing Company.
84
Zimpher, N., deVoss, G., & Nott, D. (1980). A closer look at university
student teacher supervision. Journal of Teacher Education, 31(4),
11-51.
Zimpher, N., & Howey, K. (1986). Adopting supervisory practices to different
orientations of teaching competence. Journal of Curriculum and
Supervision 2(2), 101-127.
Zimpher, N., & Loadman, W. (1985). A documentation and assessment
system for student and program development (Teacher Education
Monograph No.3). Washington, DC: ERIC Clearinghouse on Teacher
Education.
8
ASSESSING STUDENT PERFORMANCE OUTCOMES IN

TEACHER EDUCATION PROGRAMS
Joyce R. McLarty
American College Testing Program
The performance of graduates is the single most important thing one can assess
in a teacher education program. It is good to have well-qualified faculty. It is
good to have a low student-to-faculty ratio. It is good to have well-equipped
facilities, a well-stocked library, and access to good student-teaching situations.
It is good to have well-qualified incoming students and a well-designed, carefully-
articulated instructional program. But none of that matters if the graduates
produced do not have the skills and abilities to become good teachers.
All the attention to input and process in the world cannot guarantee good
outcomes. All it can do is improve the likelihood of obtaining them.
Therefore, it is essential to attend to student outcomes, to observe them and
document them, and to use the information to improve the performance of future
students. No assessment is more critical to the success of a teacher education
program.
This chapter offers an approach to developing a student performance
assessment which is tailored to the individual institution's circumstances and
goals. The approach begins with identification of the goals the institution has
for its graduates, continues through selection of assessment strategies focused on
these desired outcomes, and moves to selection or development of assessment
instruments. The closing discussion addresses the question of risks and the
attribution of performance outcomes. Throughout the chapter, the implications
of both theoretical and practical concerns are noted.
GOAL IDENTIFICATION
The first step in assessing student performance outcomes is to clearly

identify the program's goals. All programs have goals. Not all programs have
written goals. If a clearly-articulated set of program goals does not exist, this is
the time to develop them. If the program's goals have not been reviewed for
some time, now is the time to reconsider them. If the program's goals are
implicit rather than explicit, this is the time to develop explicit goal statements.
86
Clarify Internal Goals
Harris (1986, p. 13) suggests that goal development begins with three key
questions:
o What do you implicitly expect of all students and graduates in terms of

knowledge, skills, attitudes, and behaviors?
o What achievements do you implicitly expect of graduates in each major

field?
o What profiles of your alumni do you have, or can you develop, in terms of
achievements as career accomplishments, life-styles, citizenship activities,
and aesthetic and intellectual involvements?
By analyzing the responses of faculty, students, and other groups of interest

to such questions as these, it is possible to clarify implicit goals.
Reviewing the recent research literature on effective teaching and effective
teachers may also be helpful in clarifying and articulating program goals. The
Institute for Research on Teaching (Porter and Brophy, 1988) recently
summarized eleven characteristics of effective teachers. They picture effective
teachers as semi-autonomous professionals who:
o are clear about their instructional goals,
o are knowledgeable about their content and the strategies for teaching it,
o communicate to their students what is expected of them--and why,
o make expert use of existing instructional materials in order to devote more

time to practices that enrich and clarify the content,
o are knowledgeable about their students, adapting instruction to their needs

and anticipating misconceptions in their existing knowledge,
o teach students metacognitive strategies and give them opportunities to

master them,
o address higher--as well as lower-level cognitive objectives,
o monitor students' understanding by offering regular appropriate feedback,
o integrate their instruction with that in other subject areas,
o accept responsibility for student outcomes, and

87
o are thoughtful and reflective about their practice.
While these characteristics may not apply to students exiting teacher

education programs, they do offer some perspectives on desired long-term
outcomes for teacher education graduates.
Scriven (1988) provides a description of the knowledge and abilities he feels
it is reasonable to expect of teachers. He argues that a list of job specifications
(duties) provides the only proper basis for evaluating teachers for personnel
decisions. The nine dimensions and the subtopics within them provide many
clues for areas in which it may be appropriate to evaluate teacher education
outcomes. These areas are briefly summarized below:
1. Knowledge of duties
a. those specified elsewhere in this listing
b. applicable school laws and regulations
c. school expectations
2. Knowledge of school and community
a. community expectations
b. community context and environment
3. Knowledge of subject matter
a. subject specialization
b. literacy skills
4. Ability to provide instructional design
a. course design based on curriculum requirements and student
characteristics
b. selection and creation of instructional materials
c. competent use of available resources
d. evaluation of course, materials, and curriculum
e. knowledge of the needs of special students
f. ability to use human resources
5. Ability to gather information about student achievement
a. testing skills
b. grading knowledge including grading process and grade allocation
6. Providing information about student achievement
a. to students
b. to administrators
c. to parents, guardians, and other appropriate authorities
7. Classroom skills
a. communication skills
b. management skills
i. discipline (control of classroom behavior)
ii. achievement (coverage of required content)
iii. emergencies (e.g. fire, first aid)
8. Personal characteristics
a. professional attitude
b. professional development
88
9. Service to the profession

a. knowledge of professional issues
b. professional ethics
c. helps beginners and peers
d. works on projects for other professionals
e. research
There are other potential sources of goal statements. Meta-analytic studies,

such as the one conducted by Guskey and Pigott (1988), specify a number of
appropriate and inappropriate performance outcomes for teacher education
graduates. The literature on current and pressing problems experienced by
recently graduated teachers is another source of goal statements. One example is
seen in the descriptions by Olsen and Rodman of the problems faced by new
teachers who are assigned to teach high-risk students in the inner cities
(Education Week, J~me 22, 1988).
Whether the goal statements result from introspection regarding implicit
goals already present in the program or a conscious choice to pursue new goals,
perhaps selected from the literature, it is critical to the success of program
outcomes assessment to identify and clearly articulate the outcomes to be
measured.
Identify External Goals
In addition to conducting a review of the teacher preparation program's

performance outcome goals, it is also important to become informed about any
external performance outcome requirements which may be applicable. Is there
some particular standard that graduates, either individually or as a group, must
meet? Some states, for example, require teacher education students to pass
specific tests, usually of subject area knowledge, pedagogy, or basic literacy
skills. The exact wording of the external requirements may be critical in
determining whether external performance requirements apply to a particular
teacher education program.
While some of these requirements, such as passing tests, are performance
requirements, others are process requirements. These apply to the process of
assessing performance outcomes but generally do not dictate either the specific
outcomes to be assessed or the performance standards to be applied to them. For
example, the National Council for Accreditation of Teacher Education (NCATE),
Precondition Number 6 (1987, p. 26) reads:
The unit assesses the academic and professional competencies of

education students at exit from all programs at all levels through
multiple evaluation methods.
To meet this precondition, institutions must provide a "list of assessment

measures used to evaluate the academic and professional competence of
professional education graduates" (p. 27, Precondition 6.1) and "summary
report(s) of competency assessment outcomes for at least the past three years" (p.
89
27, Precondition 6.2}. Once the preconditions are met, NCA1E has additional
standards and criteria which apply to program completion. These are reproduced
below. Process requirements such as these should be considered in the design of
the outcome assessment process, but are not themselves outcome requirements.
Standard ITI. D: Completion of Program
The unit ensures that the academic and professional competence of education
students is assessed prior to granting recommendations for certification and/or
graduation.
Criteria for Compliance:
50. Prior to making recommendations for certification and/or graduation,

education students must be proficient in communication skills and their
teaching or specialty fields. Students also must be able to demonstrate
skills for effective professional practice.
51. Evaluation systems that assess the academic and professional competence
of students include multiple sources of data (such as standardized tests,
course grades, and performance in classroom or school settings).
52. The applkation of a published set of criteria that specify acceptable levels
of performance for exit from all professional education programs is
monitored (1987, p. 46).
Selecting Goals for Assessment Purposes
Not all program goals are amenable to formal assessment. Many legitimate
and critically important goals are simply beyond the scope of current assessment
technology. While it is important to identify a broad spectrum of desired
outcomes for the program, not all of these will be appropriate assessment
targets. Consider, for example, the list of goals in Table 1. This list could have
been produced by many teacher preparation programs. It is a mixture of
informational (knowledge), attitudinal, and skill-oriented goals. Some are stated
broadly; others are somewhat more narrow. Some are clearly covered within the
span of the instructional program while others may better be thought of as
entrance requirements or as skills to be developed after program exit (e.g., during
the teaching internship). As currently worded, many of the goals do not appear
to be measurable.
90
Table 1
Sample Performance Goals for Teacher Education Graduates
1. Speaks and writes correct English.

2. Behaves ethically.
3. Shows genuine affection for students.
4. Can plan a lesson appropriate to student and subject area.
5. Knows the necessary subject matter.
6. Can provide appropriate direct instruction to students.
7. Knows questioning techniques.
8. Can lead a class discussion.
9. Knows how to monitor students' seatwork.
10. Knows how to evaluate student homework and provide appropriate
feedback to students.
11. Can develop, administer, score and grade student tests.
12. Knows how to talk with parents.
13. Runs a safe and orderly classroom.
14. Knows assertive discipline.
15. Knows the basics of school law (student rights).
16. Helps all students to develop and maintain a positive self-image.
17. Can select or prepare appropriate instructional materials.
18. Is knowledgeable about child development and child psychology.
19. Understands the role of the school in society.
20. Helps students develop higher-level thinking skills.
21. Supports positive self-images for students.
22. Provides collegial support for other teachers.
23. Can select or develop appropriate instructional support materials (visual
aids, handouts).
24. Shows enthusiasm for teaching and for the subject area.
25. Pursues continuing self-education.
26. Is sensitive to the multi-cultural backgrounds of students.
It is necessary to select those goals which are deemed suitable for

assessment and to restate the goals so that they can be measured. The first step
in this process is to evaluate each goal with respect to the following criteria:
o Importance. Assessment is generally too difficult and expensive to be

undertaken for the sake of goals which are not critical.
o Measurability. Most goals can be restated so that at least some aspects

of them are measurable in ways that are appropriate and ethical.
Unfortunately this restatement sometimes changes the goal so much that it
becomes trivial or even inappropriate. Goals chosen for assessment should
be measurable in a form which retains their character and importance.
91
Once individual goals have been selected, the set of goals should be
reexamined to ensure that the balance of coverage of goal areas is appropriate. It
is not uncommon to unintentionally focus on measuring those aspects of
program outcomes which are most easily measured. It is important to determine
what is not being assessed as well as what is being assessed, and to ensure that
the emphasis portrayed in the set of assessment goals is an acceptable
representation of the program's desired outcomes.
Prioritize Goals
Since the program's goals are likely to exceed the program's resources for
assessing them, it may be necessary to establish priorities among the goals
selected for assessment. Focusing on the characteristics of the individual goals
and the probable benefits of the assessment data to the program can be helpful in
determining priorities among the performance outcomes. The following
questions may help to focus priority issues:
o What student performance outcomes do the assessment data address? How

important are these outcomes to the program? To the student?
o How unique will the data be? Will they duplicate or reinforce existing
information, or will they make a unique contribution?
o What decisions will be informed by the data? When? How critical are they?
o Are there any requirements that dictate this type of assessment? What are
they?
o How difficult and costly will the assessment data be to collect?
o Is this goal truly considered an outcome of the program or should it really

have been assessed on entrance? If it should have been assessed on entrance,
is it important to assess it as a goal anyway?
The answers to these questions should help to identify the benefits of

assessing each outcome. These benefits, together with information on the
feasibility of the assessment (e.g., costs, resources required), should help to
establish priorities among the assessment goals. Information on feasibility may
be tentative, however, until the assessment strategy is selected.
SELECTION OF ASSESSMENT STRATEGIES
The first step in designing an assessment requires consideration of how best

to assess student attainment of the desired outcomes. A good place to begin is
to ask "Who would know?" For example, "Who would know whether a student
has knowledge of the appropriate subject area?" "Who would know whether the
student can speak and write correct English?" "Who would know whether the
92
student behaves ethically?" Some possible answers to the question "Who"

include: the student, faculty members in education or other departments, peer
teachers, the student's pupils and their parents, and supervising teachers.
For each potential source of information (the "who"), there may be many
methods of obtaining the requisite information (the "how"). For example,
information about a student's knowledge of a particular subject might be
obtained by looking at courses taken and grades received in that subject
(transcript analysis); asking a faculty member or a supervising teacher in that
subject area who knows the student (questionnaire, rating form, letters of
reference, interview); testing the student on the subject matter (written test, oral
test, structured observation); asking the student (questionnaire, interview); or
observing the student (structured or unstructured observation). Each of these
methods can be expected to yield somewhat different information. A faculty
member in the subject area may have knowledge of and may value different
aspects of the subject than does the supervising teacher. A test may emphasize
still other aspects as well as introducing factors relating to test-taking skills.
Grades may reflect course expectations and the performance of otQer students as
well as the student's own subject-area knowledge. This is why using multiple
sources of information in assessing student performance outcomes is generally
recommended: each source contains certain biases. Although tests and grades are
usually thought of first as sources of information about student knowledge, and
ob!w.rvation is usually thought of first as the source of information on skills and
attitudes, there are many other possibilities which can be considered. Previously
unconsidered alternatives may be identified by listing as many alternatives as
possible for each goal.
SELECTION AND DEVELOPMENT OF INSTRUMENTS
Once sources of information have been identified, it may be useful to also

identify resources available for designing and conducting the assessment process.
While some types of assessment may be readily developed and implemented by
the faculty, staff and students of the program, others require professional
expertise which may be beyond the resources of many programs. Before
determining the process by which the assessment is to be conducted, it is wise to
survey the resources available. Resources may include individuals who can offer
advice, publications containing useful information, organizations or companies
with relevant products or services, and other institutions and agencies with
experience in similar efforts. While it is beyond the scope of this chapter to list
all of the possible resources, names and addresses for some of them are provided
in the Appendix to this text.
The extent and nature of the available resources, together with the needs and
wishes of those responsible for the assessment will jointly determine the nature
of the assessment itself. At this stage, two critical questions must be answered:
o What type(s) of assessment instrumentation will be used?
o Where will it (they) come from?

93
The types of assessment instrumentation refer to ways of collecting data, ranging

from counting courses completed by graduates, to administering tests, to
conducting observations.
The possible sources of the instrumentation include: selecting a currently
available instrument, having an instrument modified or developed by an external
contractor specifically for this application, or developing instrumentation
locally. The advantages and disadvantages of these options are discussed below.
Types of Instrumentation
A number of general types of instrumentation may be used in conducting

performance outcome assessments: data records, tests, observations,
questionnaires, rating scales, portfolios, interviews, and pupil-growth outcomes.
Strengths and weaknesses of these types of instruments may be evaluated in the
following categories:
o Availability. How readily are instruments or models of them available

for consideration?
o Feasibility. How difficult or costly is it likely to be to develop this type

of instrument?
o Credibility. How likely is this type of instrument to be viewed as

appropriate to the purpose?
o Reliability. How reliable is the data resulting from this instrument

likely to be (with respect to common, irrelevant sources of score variation)?
o Lack of Intrusiveness. How easily can this type of instrument be

administered without interfering unduly with the educational program?
o Security. (if appropriate) How suitable would this type of

instrumentation be in high-risk situations (where security and attribution of
source are critical)?
Each type of instrument can be evaluated in terms of its strength or

weakness relative to the above mentioned categories. For example, data records
generally have the advantage of being available, feasible, and unobtrusive, but
they are appropriate to a very limited range of goals. Tests are perhaps the most
developed form of instrumentation, but a good test can be difficult and costly to
develop. The use of student performance outcomes is especially problematic
because of the wide variety of potential approaches and the difficulty of
attributing student performance to teacher effectiveness. Appendix B summarizes
key characteristics, advantages, and disadvantages of some of the types of
instruments frequently considered. There are, of course, many variations on
these general and data collection approaches other than those described which
94
may be considered. It is beyond the scope of this chapter to describe and provide
directions for constructing the many types of instrumentation a teacher
preparation program may wish to consider. Fortunately, there are many
references and experts available to provide assistance.
Currently Available Instruments
Harris (1986) provides an extensive list of currently available assessment

instruments along with evaluations and descriptions of some active assessment
programs. Although his article is now several years old, most of the
information remains accurate. Some new instruments have, however, been
developed. In addition to the College Outcomes Measures Program (COMP),
"perhaps the most widely used general education battery" (Harris, 1986, p. 18),
and the Proficiency Examination Program (PEP), the American College Testing
Program, ACT, now offers the Collegiate Academic Assessment Program
(CAAP). This battery consists of multiple-choice tests in English,
Mathematics, Reading, Science Reasoning and Critical Thinking, and an essay
component, all focused at about the college sophomore level. In addition to the
Graduate Records Examinations (GRE) and the Defense Activity for Non-
Traditional Education Support (DANTES) programs, the Educational Testing
Service, ETS, now offers the Academic Profile (AP) which covers Reading,
Writing, Critical Thinking, and Using Mathematical Data, and 15 major field
achievement tests at the college level. ETS is also in the process of
redeveloping its National Teacher Examinations (NTE) program; the new edition
of this program is scheduled for introduction in 1992. Information on these
instruments should be available from the respective publishers.
Assessment Efforts Which May Have New Instruments
Ongoing development efforts, usually within the context of research or of

actual assessment programs, also yield a steady flow of new instruments and
approaches. Marchese (1987) provides descriptions of a number of institutional
assessment programs in the context of his six approaches to assessment:
1. the assessment center,
2. assessment as learning,
3. assessment as program monitoring,
4. assessment as student learning and growth,
5 assessment as standardized testing, and
6. the senior examiner.

95
The AAHE Assessment Forum, to which Marchese's work relates, seems

likely to remain a good source of information on current student assessment
programs in higher education settings, as are the New Interstate Teacher
Assessment and Support Consortium and the Teacher Assessment Project
(TAP). The Council of Chief State School Officers has recently assumed the
leadership of the Project. Another source of prototype work in teacher
assessments is the states. A number of states, mostly in the South, have
developed teacher assessment programs. Some of these focus on beginning
teachers, others on more experienced ones. Many involve the use of classroom
observations. Current information on which states are involved in teacher
assessment and what they are doing may be available from the Education
Commission of the States (ECS) or, for states in the South, from the Southern
Regional Education Board (SREB). Their addresses are provided in the
Appendix to this text.
Instrument Development. In addition to standardized assessment

instruments provided by not-for-profit organizations such as ACT, the College
Board, and ETS, and prototype efforts such as those being developed by TAP,
there are organizations, including ACT and ETS, which specialize in developing
teacher assessment materials on a contract basis. lOX, which has worked
primarily with minimal competency tests in basic skills areas, and National
Evaluation Systems (NES), which has prepared an extensive bank of test items
in the various subject areas for which teachers are likely to be certified, are
probably the best known. A third organization, Measurement Incorporated (MI),
which specializes in essay tests, has also done work with teacher assessment. If
ready-made instruments which meet assessment needs are not available, and the
institution does not have the required resources internally, it may be possible to
contract with an individual or organization to develop appropriate
instrumentation. While standardized instruments have the advantage of being
available quickly and may provide appropriate comparison data (norms), they
probably will not match program goals as well as an individually tailored
instrument. On the other hand, developing instrumentation can sometimes be a
frustratingly difficult and expensive task. The decision to buy, borrow, or
develop instrumentation should be made only after a careful consideration of:
1. The cost and degree of match to program goals of available instruments,
2. The level of difficulty, available resources and level of expertise for

developing instruments in-house, and
3. The degree of risk associated with using the instrument.
Risks
So far, the focus of this chapter has been on the use of information from
performance outcomes assessment to improve the performance of future students.
A common approach is to provide faculty with feedback so that they can
96
improve the selection of and instructional programs for students. This is

generally a low-risk enterprise. If the performance outcomes are not what the
faculty would wish, efforts are made to improve the relevant program
components. Not all uses of performance outcomes information are of a low-
risk nature, however. Information from such assessments may be used for such
high-risk purposes as determining whether students should graduate, whether
faculty should be tenured or whether schools of education should be accredited.
The hallmark of a high-risk situation is that a decision in which someone could
get hurt will be made based, at least in part, on the data. The greater the
dependence on the data and the greater the potential harm, the higher the risk
level. Less-than-ideal standards of instrument development or selection are never
desirable, but in a low-risk situation it is often possible to take an experimental
approach; to try a new approach on a pilot-test basis. In a high-risk situation, it
is essential to meet the highest possible standards. Refer to the Standards for
Educational and Psychological Testing (American Educational Research
Association, American Psychological Association, and National Council on
Measurement in Education 1985) for some critical guidelines. [Rudner's section
on Test Evaluation incorporates some of these guidelines.]
Selecting Assessment Instruments
Selecting Primary Instruments. While consideration of the

characteristics of assessment instruments discussed above (availability,
feasibility, credibility, reliability, security and lack of intrusiveness) may be
helpful in selecting general types of instrumentation, the selection of specific
instruments for use may require the evaluation of additional factors. Consider,
for example, how the data will be used.
o Will it be descriptive (e.g., a summary of responses), or is some score

required? If there is to be a score, how is the answer key to be determined?
(Note: This is not usually difficult for a test but for an interview or
observation it can be challenging.)
o On what scale will the data be reported, and how will that be determined?
o Will normative data be needed?
o Will data need to be linked longitudinally, or will multiple cross-sectional

samples be required?
o What about data aggregation? Will it be necessary to combine information

across items, across individuals, across multiple administrations of the same
or parallel forms, across instruments?
o If aggregation is required, how will missing data be handled? Will it be

difficult to distinguish missing data from other categories of non-response?
97
o If data must be aggregated across different scales, how will the conversion be
handled (equipercentile, z-score)?
Some instruments are appealing from a content and credibility standpoint, they
may prove intractable under some implementation requirements. Table 2
provides a sample performance outcome assessment plan which incorporates
standardized testing where a minimum competency criterion is required.
Table 2
Sample Performance Outcome Assessment Plan
Performance Outcomes Methods of Assessment
1. Knowledge of subject area Standardized test (required by state)

Meets course requirements
Grade point average
2. Knowledge of school law, Test developed by faculty

child development, direct
instruction, test development,
assertive discipline, methods
of self-improvement
3. Skills in reading and writing Standardized minimum

English competency test
(if not passed on entrance)
4. Skills in speaking Faculty-developed classroom

English, maintaining an orderly observation
classroom, interacting with Rating by supervising teacher
students, providing direct
instruction
5. Positive affect toward all Faculty-developed classroom

students observation
Two faculty-developed instruments; a test over a variety of knowledge areas, and

a classroom observation over a variety of skill areas are also included. Because
both context information and quality control are expensive, it may be necessary
to review the costs and benefits of the performance outcomes assessments in
light of the importance and costs of monitoring and documenting assessment
quality and of collecting data on context variables.
98
Monitoring the Assessment
A critical factor in ensuring the accuracy and credibility of the assessment is

to monitor the adequacy of the assessment itself. It is important to ensure and to
document that instruments are reliable and valid, and that the data are accurate. If
there is any way the objectivity of raters, scorers, or observers could be
questioned, it may be essential to provide verification, perhaps by having an
outside, neutral person re-rate or otherwise validate at least a sample of the data.
If there are concerns about whether students may express the attitudes they think
are desired for fear of lowered grades, it may be necessary to have questionnaires
or ratings completed anonymously. Table 3 provides some possible quality
assurance approaches. Regardless of the exact approach chosen, it is critical to
anticipate potential problems and concerns and to address or circumvent them.
Selecting Secondary Instruments. Although the primary assessment

instruments, those which are focused on the performance outcome goals,
generally receive the greatest attention and resources, consideration of how data
are to be used will often suggest the inclusion of additional instruments focusing
on context variables. These secondary instruments provide information which
may be critical in order to put the performance assessment results into an
appropriate context. Consider, for example, the potential need for data on
background and characteristics of incoming students, program completion rates,
and funding levels. Consider also any historical data which might aid
interpretation of current performance levels. Have there been recent changes in
the student population, faculty, program, institution, or community? Some of
them may be relevant to interpreting outcomes.
Some types of data which may be useful in interpreting and utilizing the
primary outcome information are presented below.
Student Data
Age (returning student?)
Sex
Race/ethnic background
Is English the primary language?
Previous educational experience (transfer student?)
Previous work experience (any teaching?)
Subject area/grade level specialization
Specific field experiences (supervising teacher, student grade level(s), subject
areas)
Courses taken, grades received
Current work, if any (hours per week) or other outside obligation
Program Data
Faculty (level, training, experience in course taught)
Texts used
Field experiences offered (type, duration, adequacy of supervision and
feedback)
Table 3
Possible Quality Assurance Efforts
Outcome Instrument Quality Control
Subject-area knowledge Course grades (faculty) Ask student to verify; compare with transcript
Questionnaire Ask multiple faculty about the same student.

(faculty) Verify faculty members' knowledge of the student
Test (student) Calculate reliability for each sub-content area; compare

to course grades and faculty questionnaire to validate
Transcript (student) Compare with grades, questionnaire and test.
English-speaking Oral/Written Retest student and compare scores; ask faculty from
and writing Test (student) English or ESL department to score. Compare with
observation. If scored by multiple observers, calculate
inter-rater consistency.
Questionnaire Verify faculty qualification to rate, provide training,

(faculty) compare to test results
Direct-instructional Classroom observation Use trained observers, use two at the same time and
skills (student) compare scores. Relate scores to pupil gains in
learning
100
Financial resources (adequacy)

Educational resources (library, media center, computer center)
Attribution of Performance Outcomes
Many times in assessing performance outcomes, the only interest is in the

nature and extent of the outcome itself. If graduates can meet requirements with
respect to knowledge of the subject area, for example, it may not be essential to
determine when and how they acquired that knowledge. If students can provide
direct instruction to their pupils, perhaps it doesn't matter where they learned the
skill. Often, however, questions arise regarding where and how students acquired
(or failed to acquire) outcomes of interest. The topic of attribution seems to
come up when the outcomes are not as desired and it becomes important to
identify the source of the problem. This investigation may be complex. If a
student has acquired particular knowledge or a skill, it may be possible to
identify when and where it was acquired. However, if a student has failed to
acquire the knowledge or skill, he or she did not acquire it anywhere. Generally,
the source of the problem is simply attributed to whatever experience(s) were
supposed to have imparted the relevant knowledge or information. This is likely
to be an over-simplification. The situation can be made clearer by comparing
the experiences of students who have achieved the outcome with those of
students who have not. It may be that the difference is fairly obvious; that there
is some specific instruction or experience that those who did not achieve the
outcome lacked. It is more likely that the situation is complex. The students
who fell short of the desired outcome may have lacked some enabling or
prerequisite skill needed to profit from the instructional experiences they were
provided. Motivation may have been lacking, or the students' preferred learning
style may be poorly matched to the instructional approach. Determining that the
instruction was ineffective may simply describe the situation. It may not
explain anything.
Berk (1988) offers 50 possible reasons why student achievement cannot be
used to conclude that instruction was effective. There are at least as many
reasons why a lack of student achievement cannot be used directly to conclude
that instruction was ineffective. Alternative reasons must be systematically
identified, explored, and ruled out. If it is important to attribute measured
outcomes to the instructional programs, it will be necessary to plan ahead to do
so. First, consider pretesting the students on any of the outcomes it is
reasonable to think they may have acquired before entering the instructional
program. If, for example, the pedagogical knowledge test consists of questions
one might answer on the basis of previous experience as a student or of common
sense, it may be difficult to show that the program caused the students to master
pedagogical knowledge. For skills that are expected to be cumulative, several
assessments at different times throughout the program may provide convincing
evidence of a learning trend. For example, students could be videotaped while
teaching once each semester. Expert teachers who do not know the students
could be asked to arrange the videos for each student from the one showing the
least teaching skill to the one showing the most, and comparing the arrangement
101
with the chronological sequence. Of course, if the teaching sequence filmed is

not standardized (Le., contains many confounding variables), it may be necessary
to examine large numbers of observations to detect any trends. Collecting
additional observations of each student or combining data for many students to
arrive at program-wide results are two methods of improving the reliability of
this type of data.
CONCLUSION
Many reasons are given for avoiding the assessment of program outcomes.
The following arguments are among the most frequently cited:
o It costs too much; the money would be better spent on instruction.
o The really important goals cannot be measured anyway.
o The faculty/staff/students already know what the results would be, so it is a

wasted effort
o Outcomes data are un interpretable because of the large number of interacting

variables involved in education.
o The data will be misinterpreted and misused.
o Teaching is too complex a process to be evaluated based on its outcomes; as

long as the process is appropriate, education programs should not be held
accountable for the results.
Most of these arguments have merit. Assessment, especially good

assessment, is often expensive, and even the best assessment cannot document
all of education's important goals. Program participants may already be aware of
many of the program's strengths and weaknesses. Outcomes data may be
complex and difficult to attribute to a particular program component. Data can
be misinterpreted and misused. Although these problems are real, they are not
unique to outcomes assessment. Rather they are objections to and characteristics
of all types of assessment. With good planning, the difficulties can generally be
avoided or at least substantially reduced. The last argument, however, is
fallacious. The fact that teaching is a complex process is one of the most
important reasons for evaluating it based on its outcomes. It is extremely
difficult to determine whether any process has been appropriate without
knowledge of its outcomes. The more complex the process, the greater the
likelihood that outcomes information will be essential for determining its
success.
102
REFERENCES
American Educational Research Association, American Psychological

Association, and National Council on Measurement in Education. (1985).
Standards for educational and psychological testing.
Washington, DC: American Psychological Association.
Berk, R. A. (1988). Fifty reasons why student achievement gain does not mean
teacher effectiveness. Journal of Personnel Evaluation in
Education, 1, 345-363.
Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives. NY:
David McKay Company, Inc.
Guskey, T. R. & Pigott, T. D. (1988). Research on group-based mastery
learning programs: a meta-analysis. Journal of Educational
Research, 8(4), 197-216.
Harris, J. (1986). Assessing outcomes in higher education. In C. Adelman
Ed.), Assessment in American Higher Education (pp. 13-31).
Washington, DC: U. S. Department of Education.
Marchese, T. J. (1987). Third down ten years to go. AAGE Bulletin, 40
(4), pp. 3-8.
McLarty, J. R. & Rakow, E. (1986, April). Low inference in the
evaluation of Tennessee's Career Ladder teachers. Paper
presented at the meeting of the National Council on Measurement in
Education, San Francisco, CA.
National Council for Accreditation of Teacher Education. (1987). NCA TE
standards, procedures, and policies for the accreditation of
professional education units. Washington, DC: NCATE.
New Interstate Teacher Assessment and Support Consortium (1988, Spring).
The Consortium Report, p. 1.
Olsen, L. (1988, June 22). The unbalanced equation. Education Week, 19,
20, 22, 23, 26.
Olsen, L. & Rodman, B. (1988, June 22) In the urban crucible. Education
Week,27-33.
Porter, A. C. & Brophy, J. (1988). Synthesis of research on good teaching:
insights from the work of the institute for research on teaching.
Educational Leadership, 45(8), 74-85.
Rodman, B. (1988, June 22). A bold step from the ivory tower. Education
Week, 21, 23.
Scriven, M. (1988). Duty-based teacher education. Journal of Personnel
Evaluation in Education, 1(4), 319-334.
Teacher Assessment Project (1987, January). Teacher Assessment Project
News, 1.
9
ASSESSMENT OF FACULTY IN TEACHER EDUCATION

PROGRAMS
John A. Centra
Syracuse University
There are numerous approaches to faculty evaluation. This chapter presents

relevant research findings, references to instruments, and specific criteria and
methods for evaluating teaching, research, and service.
What is Evaluated?
Teaching, research, and service are the major functions of higher education,
but the emphasis assigned to each of these in evaluating the performance of
individual faculty members varies across institutions. Expectations and the
relative value accorded each function also vary situationally. At some
institutions, scholarly productivity is clearly a faculty member's primary
responsibility; at others excellence in teaching is weighted more heavily. A fair
evaluation system begins with communicating what will be evaluated and what
criteria will be used to make judgments. In this chapter, the evaluation of a
faculty members' teaching will be considered as a function of: (a) student
learning, (b) student evaluations of faculty, (c) self-evaluations, (d) colleague or
peer ratings, and (e) evaluations by committees and administrators.
The National Council for Accreditation of Teacher Education (NCA TE)
Standard IV. D. reads, "The unit implements a faculty evaluation system to
improve faculty teaching, scholarly and creative activities, and service" (1987, p.
48). For this reason, research and service will be discussed, although in less
detail than teaching.
Evaluation is used to direct the improvement of performance and to guide
personnel decisions. Evaluations which are conducted to improve performance
are called formative; evaluations which are made for personnel decisions are
called summative. Planners of teacher education programs must be concerned
with formative evaluation. That is not to say that evaluation for personnel
decisions plays no role in teacher education programs, but program improvement
is the cornerstone of the Accreditation Plus Model and thus is the focal point for
this chapter.
104
Multiple sources of information must be included in any evaluation

decision to ensure accuracy and fairness. The weight of the evidence accumulated
through triangulated sources goes farther toward providing a basis for a fair
decision than does the use of single indicators. The use of a variety of
evaluation methods will result in greater improvement in teaching because
different methods will be effective with different types of teachers and will
expose weaknesses in different areas of instruction (Centra, 1979).
ASSESSING F ACUL TV PERFORMANCE
Faculty members may be evaluated in terms of student learning. They may

be evaluated in terms of their scholarship, research, and service by students,
department chairpersons, or deans. They may be evaluated by committees, and
they may evaluate themselves. Each of these data sources is discussed below.
Assessing Student Learning
Student learning is one measure of a faculty member's performance. In order

to consider this issue more thoroughly, consider the role that assessment of
student learning mayor may not play in evaluating teaching. The role of theory
in the definition and evaluation of good teaching should not be overlooked.
Likewise, views of faculty members, students, administrators, and alumni must
be considered. Similar descriptions of good teaching held between and across
those groups when members of each were surveyed. Effective communications
skills; favorable attitudes toward students; demonstrated knowledge of the
subject; and good organization of the content and the course are the four top-
ranked attributes of good teaching identified by Wotruba and Wright (1975).
These attributes are frequently reflected in student and faculty teacher rating
questionnaires and they form the basis for evaluation systems (Centra, et aI.,
1987). Thus, when research studies demonstrate that colleague ratings and
student ratings produce similar results, it is probably due, at least in part, to the
existence of a common definition of "good teaching." In short, consensus has
played a major role in identifying qualities of effective teaching and in evaluating
individual performance.
Student learning is considered by some to be the fairest and most objective
method for assessing teacher effectiveness. Information on what and how much
students have learned in a course is critical for course and instructional
improvement; however, the practical and psychometric problems inherent in
assessing student learning make it a difficult measure to use for summative
evaluations. Too many factors can affect end-of-course student outcomes. Prior
knowledge is one such factor; pre-existing differences such as motivation level
are another. Ideal research conditions are rarely found in college classrooms, so
it is difficult to isolate that student learning which can be directly attributed to a
single instructor. Some people, however, believe that the use of criterion-
referenced tests will make it possible to judge teachers according to the
proportion of students who reach a desirable level of performance.
105
Evidence of what students are learning is necessary to focus improvement in

both selection and delivery of content Assessments of student progress during a
course allow the instructor and the students to make adjustments. Frequent
testing is especially critical when the subject matter is cumulative.
Student Evaluation
Although research evidence supporting the use of student evaluations dates

back over fifty years, studies done in recent years have been able to resolve a
number of major critical issues. The findings of this recent research have
contributed to the use of faculty evaluations in personnel decisions. Initially,
such evaluations were used by individual teachers for course and instructional
improvement, but as the positive research evidence mounted, and as the need for
"objective" information on classroom teaching has increased, student evaluations
have been recommended or required for summative purposes.
Those who use student rating forms for formative or summative purposes
have relied on at least three approaches to collecting the information. At some
institutions, personnel have developed sets of items or have adapted forms
developed elsewhere. Occasionally individual faculty members or departments
will develop and administer their own forms. A second approach is to adopt one
of the systems developed at another institution. Several large institutions have
developed instruments or systems to evaluate instruction. One such system
features a computerized catalogue of items that allows instructors to select items
appropriate to their teaching style. Purdue's Cafeteria System and the Instructor
and Course Evaluation System (ICES) developed at the University of Illinois are
examples of the computer-based catalogue approach. The ICES system includes
over 400 items classified by content (e.g., course management, instructional
environment) and by level of specificity (global, general concept, specific).
Additional information on these computer-based systems and other forms is
included in the Appendix. A third choice is to select a commercially available
form. The IDEA form, developed at the Center for Faculty Evaluation and
Development at Kansas State University, and the SIR form, developed at
Educational Testing Service, are the two best known and widely used forms.
Research Evidence on Student Evaluations
Reliability and Stability. Studies have shown that the internal

consistency of ratings, that is, the extent of student agreement on ratings within
a class, is at an acceptable level providing enough students have made ratings
(Feldman, 1977; Centra, 1979). The reliability coefficients are typically close to
.90 for twenty student raters, and just above .80 for fifteen raters. For personnel
decisions, some studies indicated that the number of courses needed to provide a
sound judgment of an individual's teaching effectiveness is five or more,
assuming at least fifteen raters in each class (Gilmore, Kane, & Naccarato,
1978). Another aspect of reliability is the stability of ratings over time. Marsh,
Overall, and Kessler (1979) studied ratings of former students in a longitudinal
design and found that students' ratings at the end of courses and one year after
106
completing their degrees (several years after completing the courses) were quite
stable. Marsh (1977) found agreement between recent graduates' and current
students' evaluations of the same instructors teaching the same courses.
Dimensions. Hundreds of factor analyses of instruments on rating items

have been conducted over the years and the factors that emerged depended on the
items included. Certain factors, however, have been found repeatedly. These are:
1. course organization/planning,
2. student-instructor interaction (sometimes divided into individual and group

interaction),
3. communication skills, or presentation clarity,
4. workload, or course demands,
5. examinations/grading,
Other factors, less frequently identified, include:
6. student involvement,
7. assignments/readings, and
8. instructor enthusiasm/dynamism.
These factor-analytic studies not only support the construct validity of

student evaluations, but the factors have also been useful in summarizing item
responses for teacher or administrator use.
Potential Biasing Effects. The question of whether student

evaluations reflect teacher behavior or whether they reflect extraneous course,
teacher, or student variables has been addressed in numerous research studies. A
bias would be present if any of the variables studied had a significant effect on
students' ratings but did not also affect other indicators of teacher effectiveness.
Among the variables studied relative to potential bias are:
o Instructor rank,
o Instructor or student personality.
o Instructor or student gender,
o Student college year,
o Class size,
o Expected (or actual) grade,
o Reason for taking a course,
o Purpose of ratings, and
o Academic discipline.
107
Reviews of these studies by Doyle (1975), McKeachie (1979), Centra (1979),

Murray (1985), Aleamoni (1981) and others indicate relatively small biasing
effects. Instructors of small classes, those having fewer than 15 students,
generally receive slightly higher ratings but one might argue that they also can
provide a better teaching environment (i.e., more individual attention) and hence
produce better learning as well as better ratings.
A major concern of faculty members is the influence of grades on ratings
and the possibility that students will reward easy-grading teachers with higher
ratings. Although there are probably instances when this does occur, there is no
evidence that "leniency bias" produces a systematic and constant effect that has
any practical significance (Marsh, 1984).
The "Dr. Fox studies" also investigated a potential biasing effect on student
evaluations--specifically whether instructor expressiveness had an over-riding
influence on ratings. The initial study by Naftulin, Ware, and Donnelly (1973)
discredited the validity of student ratings by demonstrating that an expressive
teacher was rated highly in spite of the inaccurate content being taught
("educational seduction" was their term for the finding). Later, more
sophisticated studies (Abrami, Leventhal, & Perry, 1981; Marsh & Ware, 1982)
found that variations in expressiveness and content were reflected in ratings of
teacher enthusiasm and organization respectively. Students learned more when
both expressiveness and content were present.
Validity: Multisection Studies. A criterion-related approach to the

validity of student evaluations uses an assessment of student learning at the end
of the course and relates this to ratings that students give the teacher. Higher-
rated teachers should also have better student achievement results. Multisection
courses with common final examinations have been employed by researchers to
investigate the relationships, with mean student ratings and mean student
achievement in each course used as the units of analysis. Ideally, as in two
studies conducted at a Canadian university, students are also assigned at random
to each section (Sullivan & Skanes, 1974; Centra, 1977). The results of these
and some 70 multisection validity studies summarized in a meta-analysis by
Cohen (1981) were supportive: correlations were in the .40 to .50 range for
many of the factors as well as for the global evaluation of teaching. Slightly
lower correlations for the Faculty-Student Interaction factor and near-zero
correlations for the Workload or Course Demands factor were also found. These
results, along with those which reported agreement between trained observers'
ratings of teacher behavior and student ratings of the same behavior, provide
strong support for the validity of student evaluations (Murray, 1985; Erdle &
Murray, 1986).
Utili ty . The utility of student evaluations, how useful they are in

improving instruction, and whether they provide useful information in personnel
decisions, is also a critical issue. The theoretical underpinning for expecting
change in teacher behavior is provided by dissonance or imbalance theories
(Heider, 1958; Festinger, 1957). Specifically, if the feedback provided by
108
students is new and is valued by teachers, teachers can be expected to change

their behavior. Cohen's (1980) meta-analysis and Marsh's (1984) review of
feedback studies concluded that student evaluations do provide teachers with
feedback that can lead to improvement, albeit modest, in instruction. But when
augmented by an effective intervention, such as an external consultant, the
improvement has been greater. Murray (1985) found that improvement was
more likely if training procedures focused on specific, observable behaviors
rather than generalities. Research on the importance of student evaluations in
tenure and promotion is more limited. Leventhal, et. a1. (1981) and Salthouse,
McKeachie, and Lin (1978) studied student ratings at research universities and
found that student evaluations had little effect on personnel decisions over and
above a department chair's report. The extent to which the student ratings may
have influenced department chairs was not ascertained. A later study by Lin,
McKeachie and Tucker (1984) found that a combination of numerical data and
student comments on teaching effectiveness did influence decisions when
candidates had moderate research productivity.
In summary, much of the research evidence supports the use of student
evaluations for both formative and summative purposes. Based on much of this
research, Centra (1979) developed a set of guidelines that should be observed in
using these evaluations.
Self Evaluations
A faculty member's description of his or her teaching and research activities

is an important part of any personnel review. The question is wheth(;r self-
evaluations, as opposed to self-descriptions, should have any weight in an
activity report. Another question is whether self-evaluations can contribute to
improvement in performance. The evidence points to a negative response to the
first question and a positive reply to the second.
Self-ratings of overall teaching effectiveness differed significantly from
colleague, administrator, or student ratings, in a study conducted at a small
college by Blackburn and Clark (1975). In that study, all but self-ratings showed
substantial overlap. Several studies have compared self-ratings of teaching with
student ratings. Centra (1973) found relatively low correlations between the two
(about .21), but Marsh, Overall, and Kessler (1979) and Marsh (1982) found a
correlation of about .45. Marsh reported generally small differences in mean
responses between student ratings and facuIty self-ratings, a finding that also
conflicted with the Centra (1973) study. One explanation for the variation in
findings is that ratings by students in a previous semester may have caused
convergence during a later semester when the Marsh studies were conducted.
Braskamp and Caulley (1978) found that student ratings improved after a
semester and self-evaluations may have become more realistic.
While these findings could cause one to question the use of self-ratings for
salary, tenure, or promotion purposes (in fact, under such conditions, self-ratings
might be even more inflated), the same studies included findings that supported
their use in formative applications. Self- ratings or self-analysis could be useful
in conjunction with consultations or in helping faculty members select the most
109
appropriate materials or workshops for teaching improvement. An example of a

self-evaluation that can be used in this way appears in Centra (1979).
Colleague Evaluations
Colleague assessments are essential in judging research and scholarship

performance but how they should be used in judging teaching is more uncertain.
When colleague evaluations are based entirely on classroom observations, they
will likely have low inter-rater agreement. Faculty members give each other
generally high ratings and do not always concur in what they observe (Centra,
1975). Training and an extensive observation schedule would increase the
reliability but it is not realistic to expect faculty members to spend the time
required for these activities. When colleague evaluations are based on something
other than, or in addition to, classroom observations, they tend to have better
reliability and to correlate reasonably well with student evaluations (Blackburn &
Clark, 1975; Guthrie, 1954; Maslow & Zimmerman, 1956).
Committee Evaluations
Root (1987) studied ratings by a six-person executive committee that had

undergone some basic training in order to establish a common understanding of
the ratings. The committee agreed substantially in their ratings of faculty
members on teaching, research, and service, with the strongest inter-rater
reliabilities in research (.97) followed by teaching (.90) and service (.90). The
correlations between ratings in teaching, research, and service are low (.19 for
research x teaching), indicating that performance in each area tends to be
independent. An alternative explanation is that compensatory adjustments were
made by the raters--they would rate someone high in one area to compensate for
a low rating in another area, thus spreading out salary increments (for which the
ratings were used). The Root study was based on only one year's data so more
years of data are needed, but the results may encourage colleges to use ad hoc
committees of colleagues to judge teaching, as some now do. The committees
should be expected to base their judgments on more than just classroom
observations (i.e., course materials, reports from current and ex-students, and the
like).
In the interest of going beyond student rating forms, evaluation system
designers need to consider ways of collecting reliable and valid assessments from
colleagues. Examples of an outline for a colleague observation guide and a
classroom observation worksheet appear in Centra (1979).
Summary on the Assessment of Teaching
All outcomes can be measured either formatively or summatively.

Formative evaluation is used mainly for instructional and course improvement,
while summative evaluation is used for tenure, salary, and promotion
considerations. Formative evaluation of student learning is very important for
assessing student progress and for adjusting instruction. Summative evaluation
110
of student learning is necessary, but is difficult to apply since one needs the right
situation and the proper controls as described previously. Fonnative evaluation
of student ratings can be useful for some teachers, but the changes may not be
overwhelming and it is not always evident what to do about poor ratings. With
summative evaluation of student ratings there is the possibility of some bias, so
one needs proper controls for collecting and interpreting data. Summative
evaluation also uses accumulated ratings across courses and years. Self-analysis
can be useful and video/audio feedback helpful with formative self-evaluation,
but with summative self-evaluation, self-ratings are not very useful, and an
activities report is essential, as are materials submitted by the teacher.
Formative colleague ratings can be helpful since they provide informal feedback,
whether based on classroom visits or not, but they depend largely on the skill
and knowledge of the colleagues. Summative colleague ratings of classroom
practices tend to be biased and unreliable; however, peer evaluations of
knowledge of subject, course outlines, texts, and student performance, could be
useful periodically. Formative alumni ratings about the curriculum and other
college experiences can be useful in program and institutional evaluation. On
the other hand, summative alumni ratings are difficult to obtain in a systematic
and reliable manner. They correlate highly with ratings by current students, so
in most instances they would not add much new information.
Assessment of Research
The importance given to research and scholarship in evaluating faculty

members will, of course, depend on institutional type and purpose. Moreover,
the definition of acceptable performance varies not only by institution but by
discipline.
The criteria for evaluating research and scholarship performance include both
quantity and quality dimensions. Examples of the quantity dimensions include
the number of:
o articles in professional journals,

o books as sole or senior author,
o monographs or book chapters,
o papers at professional meetings, and
o books as junior author or editor.
The quality of scholarly research is generally judged by peers at the

institution, peers at other institutions, and by various administrators. Honors or
awards from professional organizations, grants, and serving as a referee or editor
of a professional journal are also considered.
A relatively new method of assessing quality is to note the number of times
a person's publications have been cited in subsequent literature. The citation
indices published through the Institute for Scientific Information (Indices in the
Sciences, Social Sciences and Humanities) allow a systematic account of
citations (Garfield, 1979). Although principally used for literature searches, the
indices are being increasingly used by personnel committees, particularly when
111
tenure or upper level promotions are being considered. Cole and Cole (1967) and
Braxton and Bayer (1986) identified three shortcomings of citations as
performance indicators:
1. Citation style varies by discipline,
2. Citations may be critical rather than positive, and
3. The significance of the work may not have been recognized by the author's
colleagues.
Another way of rewarding quality work is to give extra weight to articles

which are published in highly rated journals. Braxton and Bayer (1986) discussed
various ways in which journals might be weighted, including the use of peer
ratings and an "impact" factor based on the average number of citations received
by the journal.
Quality and qu;mtity of research and scholarship are related but the extent of
the relationship indicates that each should be considered separately in assessing
performance. In studies where citation counts, as a measure of quality, were
related to the number of articles published, Cole and Cole (1967) found a
correlation of .72 for a sample of physicists. Slightly lower correlations were
reported for other disciplines by Meltzer (1956), and by Schrader (1978) who
noted a correlation of .60 for a sample of psychology Ph.Ds.
Since quality may not always be reflected in a publication count, other ways
of assessing the significance of a person's work are necessary. Success is often
related to continuity in research, as indicated by a number of articles (or, in some
instances, a book) dealing with a particular topic or problem. Such continuity
does not mean repetition of subject matter or narrowness of interests, but rather
that the person has explored related problems within a particular area.
Professional peers are undoubtedly best able to judge the continuity element
of a colleague's work as one part of overall quality. In evaluating research in
specialized areas scholars at other institutions are probably essential. Survey
results suggest that peers at other institutions are being increasingly called on for
their opinions. If conducted in a confidential manner, these assessments can be
more objective than those of colleagues within an institution who, because of
friendships or rivalries, may not always be objective. Enough colleagues should
be surveyed to prevent undue weighting of a single unusually high or unusually
low assessment In eliciting judgments from colleagues, a request for a general
estimation is less apt to help than asking specific questions, in particular,
questions that touch on the significance of the candidate's accumulated efforts.
Summary of the Assessment of Research
A tally of publications and presentations, alone, should not serve as the sum
total of a person's scholarly endeavors. Boyer (1987) remarks that measures of
scholarly activities could include a facuIty member being asked to author or
review a text book or react to a recent development in his or her field.
112
Continuing scholarship, keeping current in the field, should be recognized for

improving both teaching and research.
Assessment of Service
The third area of faculty evaluation is service and as with research, the
weight service carries in an evaluation will vary with the institution. Miller
(1987) defines professional service as activities "such as participating or holding
office in professional associations and societies and to professional status as
viewed by oneself and by others" (p. 65). Public service, he notes, includes
"applied research, consultation and technical assistance, instruction, products, and
clinical work or performance" (p. 66).
In a recently released book, the Joint Committee on Standards for
Educational Evaluation describe the Propriety Standard for service orientation as
the promotion of "sound education principles, fulfillment of institutional
missions, and effective performance of job responsibilities, so that the
educational needs of students, community, and society are met" (1988, p. 21).
The book addresses personnel evaluation in terms of the utility, feasibility,
propriety, and accuracy.
Summary of the Assessment of Service
Faculty and administrators in each institution must agree on what

constitutes service and on what types of service they want to consider in the
evaluation or accreditation process. The aforementioned book should be required
reading by all administrators and faculty members. It contains concise
descriptions, case studies, and analyses of those studies which will be useful in
establishing and clarifying evaluation criteria.
CONCLUSION
The basic principles of faculty evaluation are no different from those which
guide assessment in other fields. This paper has focused on the need to base
assessments on multiple sources of research-proven valid information.
Assessment information can and should also be used to help faculty overcome
weaknesses and build on strengths. For personnel decisions it is also necessary
to review the evidence at several levels and to follow legal and ethical procedures.
Additional information about legal issues is found in Curcio's chapter at the end
of this book and readers will certainly want to read Rutherford's chapter on
utilizing evaluations.
APPENDIX
Information About Selected Student Rating Systems
The Instructional Assessment System (lAS) presently consists of six

distinct forms, each tailored to provide diagnostic information for a broad course
113
type (large lecture, small lecture-discussion, seminar, problem solving, skill

acquisition, and quiz sections). Each form has three sections. Section 1 contains
four global evaluative items whose major purpose is normative. Section 2
contains eleven items designed to provide diagnostic information. Section 3
contains seven items designed to provide information to students as well as
being diagnostic. Sections 1 and 3 contain items common to all forms.
Contact: Gerald M. Gilmore, Educational Assessment Center, PB-30, University
of Washington, Seattle, Washington 98195.
The Instructor and Course Evaluation System (ICES) is a computer-

based system whereby faculty can select items from a catalogue of over 400
items classified by content (course management, student outcomes, instructor
characteristics and style, instructional environment, student preferences, and
settings) and by specificity (global, general concept, and specific). Global and
general concept items are normed by rank of instructor and required-elective
status whereas specific (diagnostic) items, recommended for course improvement
purposes, are not normed.
Contact: Dale C. Brandenburg, Coordinator, Instructor and Course Evaluation,
307, Engineering Hall, Urbana, Illinois 6180l.
Purdue's Cafeteria System consists of four FORTRAN computer

programs, a 200-page operations manual, a computer-managed catalogue
containing 200 diagnostic items, and a norm library. Cafeteria can be installed
for local operation easily on virtually any computer that has FORTRAN
capability and it functions equally well as a sheet or card-based system. Cafeteria
supports both administrative and instructional improvement processes.
Contact: J. O. Derry, MRC, ENAD 402, Purdue University, W. Lafayette,
Indiana 47906.
The Student Instructional Report (SIR) program includes a machine

scorable answer sheet with 39 questions, plus space for responses to ten
additional questions that may be written locally. SIR covers such areas as:
Instructor-Student Interaction, Tests and Exams, Course Organization, Student
Interest, and Course Challenge. Student responses are presented as percent
responding to each alternative to each item, item means, percentile equivalents of
the means, and scores on six factors. Comparative data, based on national use of
SIR, are available separately for two-year colleges and for four-year colleges and
universities. Comparative data for approximately 31 academic disciplines are
included.
Contact: Nancy Beck, Student Instructional Report, Educational Testing
Service, Princeton, New Jersey 08540.
The Instructional Development and Effective Assessment (IDEA)

System has as its criterion of teaching effectiveness students' ratings of progress
on course goals selected as important or essential by each course's instructor.
No one model of effective teaching is implied. Beyond offering comparisons
with all instructors in the comparison group, comparisons are made with other
114
courses of similar size in which students report similar levels of motivation.

Where students report unsatisfactory progress on a teacher's goals and also report
instructor's infrequent use of teaching methods which are related to their progress
ratings, then the computer-prepared report identifies teaching strengths and
weaknesses.
Contact: William Cashin, Center for Faculty Evaluation and, Development,
Kansas State University, 1627 Anderson Avenue, Box 3000, Manhattan, Kansas
66502.
REFERENCES
Abrami, P. C., Leventhal, L., & Perry, R. P. (1982). Educational seduction.

Review of Educational Research, 52, 446-464.
Aleamoni, L. M. (1981). Student ratings of instruction. In J. Milliman (Ed.),
Handbook of teacher evaluation. Beverly Hills, CA: Sage, 110-
145.
Blackburn, R. T., & Clark, M. J. (1975). An assessment of faculty
performance: Some correlates between administrators, colleagues, students,
and self-ratings. Sociology of Education, 48(2), 242-256.
Boyer, E. L.(1987). College: The undergraduate experience in
America. New York: Harper & Row.
Braskamp, L. A., & Caulley, D. (1978). Student rating and instructor
self-ratings and their relationship to student achievement.
Urbana-Champaign: University of Illinois, Measurement and Research
Division.
Braxton, J. M., & Bayer, A. M. (1986 June). Assessing facuIty scholarly
performance. In J. Crewell (Ed.), Measuring Faculty Research
Performance, in New directions for institutional research,
13(2), 25-42.
Centra, 1. A. (1979). Determining faculty effectiveness. San
Francisco: Jossey-Bass.
Centra, J. A. (1977). How universities evaluate faculty performance:
A survey of department heads. (GREB Research Report No. 75-5BR.
Princeton, NJ: Educational Testing Services. (ERIC ED 157-445).
Centra, J. A. (1977). Student ratings of instruction and their relationship to
student learning. American Educational Research Journal,14(1),
17-24. Centra, J. A. (1975). Colleagues as rater of classroom instruction.
Journal of Higher Education, 46(3), 327-338.
Centra, J. A. (1973). Self-ratings of college teachers: A comparison with
student ratings. Journal of Educational Psychology, 65(3), 395-
401.
Centra, J. A. (1973). Effectiveness of student feedback in modifying college
instruction. Journal of Educational Psychology, 65(3), 395-401.
Centra, J., Froh, R., Gray, P., & Lambert, L. (1987). Evaluating teaching
for tenure and promotion. Syracuse University: Center for
Instructional Development.
115
Cohen, P. A. (1981). Student ratings of instruction and student achievement:

A meta-analysis of multisection validity studies. Review of
Educational Research,51(3), 281-309.
Cohen, P. A. (1980). Effectiveness of student rating feedback for improving
college instruction: A meta-analysis of findings. Research in Higher
Education, 13(4), 321-341.
Cole, S., and Cole, J. R. (1967). Scientific output and recognition: A study in
the operation of the reward system in science. American Sociological
Review, 32,(3), 377-399.
Doyle, K. O. (1975). Student evaluation of instruction. Lexington,
MA: Lexington Books.
Erdle, S. & Murray H. G. (1986). Interfaculty differences in classroom teaching
behaviors and their relationship to student instructional ratings. Research
in Higher Education, 24(3), 115-127.
Feldman, K. A. (1977). Consistency and variability among college students in
rating their teachers and courses; A review and analysis. Research in
Higher Education, 6(3), 223-274, 277.
Festinger, L. (1957). Theory of cognitive dissonance. Stanford, CA:
Stanford University Press.
Garfield, E. (1979). Citation indexing: Its theory and application
in science, technology, and humanities. New York: John Wiley &
Sons.
Gilmore, G. M., Kane, M. T., & Naccarato, R. W. (1978). The
generalizability of student ratings of instruction: estimation of teacher and
course components. Journal of Educational Measurement, 15(1),
1-13.
Guthrie, E. R. (1954). The evaluation of teaching: A progress
report. Seattle: University of Washington Press.
Heider, F. (1958). The psychology of interpersonal relationships.
New York: John Wiley & Sons.
Joint Committee on Standards for Educational Evaluation. (1988). The
personnel evaluation standards: How to assess systems for
evaluating educators. Newbury Park, CA: Sage Publications, Inc.
Leventhal, L., Perry, R. P., Abrami, P. D., Turcotte, S. J. C., & Kane, B.
(1981). Experimental investigation of tenure-promotion in
American and Canadian universities. Presented at the meeting of
the American Educational Research Association, Los Angeles.
Lin, Y., McKeachie, W. J., & Tucker, D. G. (1984). The use of student
ratings in promotion decision. Journal of Higher Education, 55(5),
583- 589.
Marsh, H. W. (1984). Student evaluations of university teaching:
Dimensionality, reliability, validity, potential biases, and utility. Journal
of Educational Psychology, 76(5), 707-754.
Marsh, H. W. (1982). Validity of students' evaluations of college teaching: A
multitrait-multimethod analysis. Journal of Educational
Psychology, 74(2), 264-279.
116
Marsh, H. W., Overall, J. u., & Kessler, S. P. (1979). Validity of student

evaluations of instructional effectiveness: A comparison of faculty self-
evaluations by their students. Educational Psychology, 71(2), 149-
160.
Marsh, H. W., & Ware, J. E. (1982). Effects of expressiveness, content
coverage, and incentive on multidimensional student rating scales: New
interpretations of the Dr. Fox effect. Journal of Educational
Psychology, 74(1), 126-134.
Maslow, A. H., & Zimmerman, W. (1956). College teaching ability, scholarly
activity, and personality. Journal of Educational Psychology, 47,
185- 189.
McKeachie, W. J. (1979). Student ratings of faculty: A reprise. Academe,
384-397.
Meltzer, L. (1956). Scientific productivity in organizational settings. Journal
of Social Issues, 12(3), 32-40.
Miller, R. I. (1987). Evaluating faculty for promotion and tenure.
San Francisco: Jossey-Bass.
Murray, H. G. (1985). Classroom teaching behaviors related to college
teaching effectiveness. In J. C. Donald and A. M. Sullivan, (Eds.) Using
research to improve teaching. New directions for teaching
and learning 23, (pp. 21-34). San Francisco: Jossey-Bass.
Naftulin, D. H., Ware, J. E., & Donnelly, F. A. (1973). The Doctor Fox
lecture: A paradigm of educational seduction. Journal of Medical
Education, 48(7), 630-635.
Root, L. (1987). Fac llty evaluation: Reliability of peer assessments of
research, teaching and service. Research in Higher Education, 26(1),
71- 84.
Salthouse, T. A., McKeachie, W. J., & Lin, Y. G. (1978). An experimental
investigation of factors affecting university promotions decisions. Journal
of Higher Education, 49(2), 177-183.
Schrader, W. B. (1978). Admissions test scores as predictors of
career achievement in psychology. GREB No. 76-1R. Princeton,
NJ: Educational Testing Services.
Sullivan, A. M., and Skanes, G. R. (1974). Validity of student evaluation of
teaching and the characteristics of successful instructors. Journal of
Educational Psychology, 66(4), 84-90.
Wotruba, T. R., & Wright, P. L. (1975). How to develop a teacher-rating
instrument: A research approach. Journal of Higher Education,
46(6), 653-663.
10
USE OF MAIL SURVEYS TO COLLECT INFORMATION FOR

PROGRAM IMPROVEMENT
Jerry B. Ayers
The survey is one of the oldest research techniques used in the social sciences and
education. Survey data are generally gathered by use of one of three techniques
(or some combination of the three). These three methods are mail surveys,
personal-interview surveys, and telephone surveys (Kerlinger, 1967). The mail
survey is widely used to gather follow-up data for improvement of teacher
education programs (Adams & Craig, 1981; Ayers, 1979; Ewell & Lisensky,
1988; Isaac & Michael, 1981; Villene & Hall, 1981). Craig addresses the
broader spectrum of follow-up evaluation in the following chapter; therefore, the
major focus of this chapter will be on the design, development, and use of mail
follow-up surveys.
Follow-up Surveys
Follow-up surveys are one means of gathering descriptive information about

teacher education programs that can be used to improve them. Data are collected
in the form of physical counts and frequencies about attitudes and opinions (Isaac
& Michael, 1981; Kerlinger, 1967; Oppenheim, 1966; Babbie, 1973). This
information can then be used to answer specific questions about the graduates of
a program, to identify or describe the graduates' perceptions of the program, and
to aid in the process of redesign of the teacher education program. According to
Isaac and Michael (1981, p. 128) surveys of the graduates of a teacher education
program should be systematic and carefully planned to ensure appropriate content
and coverage, representative of the population studied, objective to ensure that
the data are observable and precise, and quantifiable, yielding data that can be
expressed in numerical terms.
Advantages of Mail Surveys
Some of the more important advantages of using mail surveys to gather

information for program follow-up are that they: (a) can serve as a basis for
118
collecting essential demographic information, (b) are inexpensive, (c) can be

completed at the leisure of the respondent, (d) can be anonymous if desired, and
(e) can reduce interviewer bias. Babbie (1979), Berdie and Anderson (1974),
Erdos (1983), and Isaac and Michael (1981) are among the best sources of
information on this topic
Limitations of Mail Surveys
As with any endeavor there are limitations and disadvantages. In some

instances mail surveys are totally inappropriate and at other times their
usefulness is limited. Some of the major disadvantages and limitations of mail
surveys for program follow-up are listed by Babbie (1979), Berdie and Anderson
(1974), Erdos (1983), and Isaac and Michael (1981).
Mail Follow-up Instruments
This section describes the development of mail questionnaires that can be

used in conducting follow-up studies of the graduates of a teacher education
program. Also included is a description of commercially available instruments
and the National Database. Similar techniques and instruments can be used with
follow-up studies of graduate as well as undergraduate programs.
Development of a Follow-up Questionnaire
Numerous books have been written on the subject of developing a follow-up

questionnaire (Sudman & Bradburn, 1984; Sudman, 1985; Lockhart, D. C.,
1984). This section will present only a few reminders to use in constructing a
questionnaire for use in follow-up evaluation. The Appendix includes a checklist
of items for use in questionnaire construction and use.
Once the returns are in, it is too late to wonder what would happen if--? A
review of pertinent ERIC documents reveals numerous examples of follow-up
questionnaires that can serve as examples. The work of Edwards (1957) will aid
in the construction of attitude measures and various organizations have
developed useful instruments for follow-up (Ewell, 1983; Ewell &
Lisensky, 1988). Also, the faculty of the teacher education program should
be asked to contribute questions for consideration for inclusion in the
instrument. Faculty input into the instrument will aid in making sure that
the information that is collected will be utilized. [See Rutherford elsewhere in
this text for further information.]
Design Constraints
The ideal questionnaire is one that an individual wants to complete. If the

questionnaire does not appear difficult and time-consuming to complete, the
chances of its being completed and returned are higher. The printed page must be
neat in appearance and the directions must be clear. Erdos (1983) provides
sample layouts for questionnaires.
119
Erdos (1983) also points out that a questionnaire in a 7 by 10 inch fonnat is

optimal. The fewer the pages, the higher the percentage of return. Although
there is no real proof, it is generally felt that the upper limit for successful
returns is a questionnaire of four to eight pages.
Generally, paper in light colors, such as ivory and light buff, is
recommended. The paper stock should be pleasant to look at and thick enough
to be printed on both sides. Professional printing will enhance the appearance of
the instrument and that should in turn improve the percentage of returns.
Microcomputers with laser printers can be used to prepare professional quality
questionnaires that can be used directly for printing.
Pilot studies of the instrument are essential. One of the best ways of
developing good objective questions is to administer the instrument to a sample
of subjects who are representative of the population that will complete the
questionnaire. The subjects can be asked to make notes on the instrument and
suggestions for improving questions, fonnat, and the general appearance of the
instrument. Isaac and Michael (1981), Erdos (1983), and Babbie (1979) provide
detailed directions for conducting pilot studies of instruments.
Commercially Available Instruments
Several organizations have produced instruments which are general in nature

and also provide services for conducting follow-up evaluations via the mail.
The National Center for Higher Education Management Systems (NCHEMS)
has developed a series of questionnaires that can be used to survey students as
they progress through their collegiate program and after they have graduated
(Ewell, 1983). The Educational Testing Service and the American College
Testing program provide instruments and services that can be used in conducting
follow-up studies of college graduates. The instruments from these
organizations are described in the Appendix to this chapter.
National Database
Many teacher education faculty and institutional researchers rely on their

own survey instruments when conducting follow-up studies of their graduates.
There are, of course, advantages to doing so, but one national project merits
some consideration. In the winter of 1988, the National Center for Research on
Teacher Education at Michigan State University, the College of Education at The
Ohio State University, and a group of evaluators representing ten teacher
education institutions began the task of developing an instrument that could be
used across a number of institutions (Loadman, Brookhart, & Freeman, 1989).
Freeman (1988) prepared a compendium of follow-up items by analyzing survey
instruments used at 18 different institutions. He collated and edited the items
from these surveys, and developed new questions to address deficiencies suggested
by reviewers. Pilot tests of the instrument were made at The Ohio State
University (Loadman, Brookhart, & Freeman, 1989), at Tennessee Technological
University (Birdwell, 1989) and at several other locations. Based on the various
120
pilot tests, the instrument was revised and long and short versions were
generated.
The instrument requests graduates to provide information in six broad areas
including: ratings of preservice program quality, knowledge and understanding of
program content, adequacy and source of development of teaching, employment
history, background information, and perceptions of the goals and
responsibilities of teachers
Data Collection Via Mail Follow-Up Questionnaires
This section provides an overview on collecting data by means of a follow-

up questionnaire. The topics include some of the more important considerations.
Mailing Process
It is imperative that mail survey instruments be accompanied by a cover

letter which identifies the sender and the purpose of the survey and which solicits
the support of the recipient. Many factors enter into the construction of a
successful letter. Erdos (1983) provides a list of 22 elements to include in the
letter which accompanies a mail survey packet
Questionnaire distribution. Data collection is described in detail by

Babbie (1973). Distribution of the questionnaire can most logically be
accomplished through use of the postal system. There is some debate over the
use of first class and bulk rate options. First class is more expensive, but it is
also more flexible. Letters will be forwarded by the postal service and those that
cannot be delivered will be returned. In order to use bulk-rate, a special permit
must be purchased (usually institutions already have such permits) and a
minimum of 250 pieces of mail must be sent The permit must be printed on
each envelope and the envelopes must be arranged in zip code order and tied in
bundles. The U. S. Post Office has complete information on bulk-rate mailing.
Return envelopes. There are three options for return postage on

questionnaires: (a) the respondent provides postage, (b) the researcher provides a
stamped envelope, and (c) the researcher provides a business reply envelope. The
first is an extremely poor choice and will result in a very low return rate on
completed questionnaires. Research has shown there is little difference in the
return rate between stamped envelopes and business reply envelopes (Erdos,
1983).
To use a business reply envelope, a special permit must be purchased and
printed on each envelope. Most institutions have such permits .. Business reply
rates are those of first-class postage plus a surcharge for each piece of returned
mail (check with the U. S. Post Office to determine the current rate). If the
researcher can expect a high return from the initial mailing, it may be less
expensive to affix a stamp to each reply envelope.
It is essential that the rate of return of questionnaires be monitored. This
information will be valuable in determining when to make additional mailings
121
and can also provide a preliminary indicator of where problems might lie in the
completion of the questionnaire. In the future, the questionnaire could be
modified in such a way that returns can be increased.
Number or Mailings
There is no rule for determining when to make a second mailing of the

questionnaire. Researchers generally consider a 50 percent return adequate, 60
percent good, and 70 percent very good (Babbie, 1973; Isaac & Michael, 1981).
Typically the researcher will make a second mailing after the rate of return slows
to the point that only two or three questionnaires are being received in a day. A
third mailing may be necessary to increase the rate of return to an acceptable
level.
Respondents and Non-respondents
The issue of increasing the rate of return for questionnaires and how to deal
with non-respondents has been frequently discussed in the literature. McKillip
(1984) advocated the application of attitude theories to the return of mailed
questionnaires. He has explored four theories of attitude measurement for
increasing the rate of return of mail follow-up questionnaires. Altschuld and
Lower (1984) {escribed some factors that have increased the return of mail
follow-up questionnaires to 96%. Hogan (1985) found little difference in the
results of follow-up questionnaires that had high and low response rates. Boser
(1988) explored the problem of whether respondents are representative of the
total group. The results of her work indicated that there were no differences
between respondents and non-respondents.
Data Processing and Analysis
Data processing and analysis are dependent upon the use that will be made of
the results of the survey. Therefore, the original objectives of the survey need to
be examined to determine the specific type of analysis needed. The Appendix of
this chapter includes a suggested checklist for preparing a final report
The most widely used statistic derived from surveys is frequency counts
(counting the replies and presenting them in tabular form). Data is tabulated
either by hand or by a computer. Frequency counts serve as a base for all other
data analysis.
A complete set of descriptive statistics should be calculated for the data.
Correlations across various data sets may also be of value. If data have been
collected from previous years or from other programs or institutions, inferential
statistical techniques may be used. A variety of standard research texts designed
for the social sciences and education can provide assistance in this area (Isaac &
Michael, 1981; Erdos, 1983; Babbie, 1973; Kerlinger, 1967).
Data analysis via the computers is common. Microcomputers provide easy
access to computing capabilities that will allow for constructing frequency
distributions and other descriptive statistics. A number of programs are available
122
for use on mainframe computers and microcomputers. Probably the most

common packages are the Statistical Package for the Social Sciences
(SPSS) published by McGraw Hill and the Statistical Analysis System
(SAS) published by the SAS Institute Incorporated. Both are available for use
on mainframe, mini, and microcomputers.
Reporting of Results
Reporting the results of mail follow-up studies is one of the most important
aspects of follow-up evaluation, yet frequently it is not given sufficient
attention. In order for the information to be useful in program improvement and
in redesign efforts, it must be put into the hands of the faculty and administrators
in a concise form. The typical report should contain a description of the purpose
of the survey, the method employed, a summary of the findings, conclusions,
recommendations, a summary of the tabulations and calculations, and a copy of
the survey instrument.
Other information can be included at the discretion of the researcher. Copies
of the report and all of the original data sheets should be kept in order to develop
a longitudinal study of the graduates of the institution. Longitudinal studies are
of importance in improving teacher education programs.
Use of Follow-up Results
The active support of faculty and administrators in the teacher education

program is crucial to the successful use of follow-up survey results. [See
Rutherford for information on implementing innovations.] Martin (1987) and
Davis and Davis (1986) report that the results of follow-up studies are utilized to
improve teacher education. The ERIC collection contains many examples of
program changes and improvements that have been made as a result of the
studies.
Employers as Resources of Information
Employers of teacher education program graduates are a valuable source of

information for program improvement. Mail surveys can be of assistance in
collecting this information. Prior to soliciting information it is advisable to
secure the permission of the graduate. The instrument construction and use
techniques already described can be used to create a mail questionnaire to be used
with principals and supervisors. Sandefur (1970) describes a four-item
instrument that includes questions about the graduates' subject matter
competence, their relationships with students, the appropriateness of
assignments, and their academic expectations and overall classroom effectiveness.
The National Database project (Loadman, Brookhart, & Freeman, 1989) is
beginning work on the development of a mail follow-up survey instrument for
use with principals that will be available in 1990.
123
Direct Survey Techniques
No chapter on surveys would be complete without mentioning direct survey

techniques. These techniques involve direct contact with the respondent and are
frequently used in tandem with mail follow-up surveys. These surveys take two
fonns: direct face-to-face contact between the interviewer and the respondent, and
contact through the telephone. The reader is referred to such sources as Isaac and
Michael (1981), Babbie (1973), Kerlinger (1967), and Davis and Davis (1986)
for additional details.
Face-to-Face Surveys
Two approaches are available in face-to-face interviews--individual

interviews and group interviews. Each technique has its own advantages and
limitations.
Individual interviews are characterized as being personal; they pennit in-
depth free responses; they are flexible; and they allow the interviewer to obtain
impressions from gestures, voice tone, and facial expressions. Individual
interviews, however, can be very costly and are vulnerable to personality
conflicts. Other limitations are that interviews require the use of individuals
who have had extensive training, and it may be difficult to summarize interview
findings. Group interviews are more economical and efficient, reflect group
behavior and consensus and can stimulate brninstonning. According to Isaac and
Michael (1981) the major limitations of interviews include they may intimidate
an individual, foster confonnity, and can be manipulated by an influential
member of the group. To be fully effective, face-to-face interviews must be well
planned. Because of the cost factors, large scale interviewing is probably not a
viable source of follow-up infonnation for many teacher education programs.
But, the technique can be used on a limited basis to probe particular areas of
concern.
Telephone Surveys
Telephone surveys are widely used in place of face-to-face interviews. In

general, the results obtained by the two methods are comparable (Isaac &
Michael, 1981). Telephone surveys of graduates are a viable method for
obtaining information about graduates' perceptions of their professional
prepamtion programs (Davis & Davis, 1986).
Telephone surveys have severnl advantages aside from being less costly than
face-to-face interviews. The interviewer can probe the answers of the respondent,
the survey can be conducted both at night and the daytime, unlimited callbacks
are possible, the respondents are more at ease in their own homes, and the
availability of a Wide Area Telephone Service (W ATS) will make it possible to
serve a large number of individuals (Isaac & Michael, 1981; Babbie, 1973).
There are certain disadvantages to the use of the telephone in survey
research. The most common problems include lack of availability of accurate
telephone numbers, the telephone survey can be viewed as an intrusion into the
124
home, rules out the advantages of the face-to-face interview, and the logistics of
the situation can be difficult.
SUMMARY
Surveys are one of the most widely used techniques in education and the
social sciences to gather data for program change and improvement. The mail
survey is probably the most common technique of gathering information from
graduates in order to obtain self-reports of their perceptions of the teacher
education programs in which they were trained. The mail survey allows the
collection of a set of data which ranges from frequency counts of occurrences of
events to attitudes and opinions about a particular program. This information,
in turn, can be used
o to answer questions that have been raised about the program,

o to solve problems that may exist,
o to assess needs and set goals,
o to determine whether specific objectives have been met,
o to establish baselines against which future comparisons can be made, and
o to analyze trends and changes.
This chapter provided suggestions for the development and use of various
type:> of instruments, a bibliography of references, information on commercially
available instruments, and a checklist to be used in survey work. The
information in this chapter will be of use in the development and
implementation of mail follow-up surveys.
APPENDIX
Instruments for Use in Follow-up Evaluation
ACT Evaluation/Survey Service (ESS). The ESS is a collection of

12 instruments designed to be used in surveying students before entrance into
college, while they are enrolled, and after they leave. ACT, Evaluation/Survey
Service, P. O. Box 168, Iowa City, IA 52235. Or call 309-337-1102.
Graduate Program Self-Assessment Service (GPSAS). The

GPSAS is a program sponsored by the Council of Graduate Schools in the
United States and the Graduate Record Examinations Board. Three instruments
are available at, respectively, the Master's and Doctoral program levels.
Instruments are available at each level for faculty, students, and alumni. The
alumni questionnaires would be of most value in follow-up assessment work. In
excess of 500 per fiscal year, the cost is reduced to $1.00 per instrument plus
$100 per summary report. Graduate Program Self-Assessment Service,
Educational Testing Service, Princeton, NJ 08541-6013. Or call 609-734-5966.
125
Student-Outcomes Information Services (SOlS). The SOlS is a

collection of instruments that can be used to gather information about students at
various stages during and after their college careers. NCHEMS Publications
Department, P. O. Drawer P, Boulder, CO 80302. Or call 303- 497-0390.
National Database for Teacher Education Follow-up Studies.

The National Database for Teacher Education Follow-up Studies is a joint
project that involves representatives from at least 15 institutions across the
United States that have been actively involved in mail follow-up evaluation
work for a number of years. Dr. William E. Loadman, 356 Arps Hall, 1945
North High Street, The Ohio State University, Columbus, OH 43210. Or call
614-292-3239.
Compendium of Items for Follow-Up Surveys of Teacher

Education Programs (Freeman, 1988). This document was an outgrowth of
a review of the survey instruments used by 18 selected institutions. The
document includes a variety of questions that have been used by institutions to
collect information from graduates. National Center for Research on Teacher
Education, 516 Erickson Hall, Michigan State University, East Lansing, MI
48824-1034. Or call 517-355-9302.
CHECKLIST FOR EVALUATING MAIL FOLLOW-UP

SURVEYS
This section presents a set of criteria which can be used in the evaluation of
mail follow-up surveys. The criteria were extracted from a variety of sources and
are divided into several sections including survey design, survey instrument, data
gathering, data processing, and reporting. Each section can be used as a separate
checklist of items that should be carried out in order to conduct a mail survey
that will provide useful data for program improvement and redesign.
Survey Design
1. What is the purpose of the survey?
2. Will a single instrument accomplish the purpose of the survey?
3. What is the population to be surveyed?
4. Will a sampling scheme be needed or will the total population be surveyed?
5. If a sampling plan is used, is it defensible?
6. Is the survey going to be made at an appropriate time of the year?
7. Was a pilot study of the instrument conducted?

126
8. Are adequate resources available to accomplish the survey?
Survey Instrument
1. Is the questionnaire of an appropriate length?
2. Does the instrument include the appropriate demographic questions that will
allow for continuing contact with the subjects?
3. Does the layout of the instrument encourage clear answers?
4. Are all of the questions written such that the respondent can be expected to
answer without guessing?
5. If open-ended questions are used, are they clear and unambiguous?
6. Does any question concern confidential information?

7. Can the answers on check questions limit or bias information?
8. Is the questionnaire free of bias?
9. Is the title free of bias?
10. Does the instrument avoid any ambiguity?
11. Are the directions clear and the instructions helpful in getting precise
answers?
12. Was the instrument field tested?
13. Did the teacher education faculty have input into the questions' construction?
14. Did the teacher education faculty critique the instrument?
15. Was the instrument edited by one individual?
16. Was the instrument printed in an attractive manner?
17. Were appropriate stationary and envelopes used for mailing?
18. Was the total instrument of such a length that it could be completed in a
reasonable time by the respondents?
19. Do the questions reflect the objectives of the teacher education programs?
127
Data Gathering
1. Were the instrument and accompanying letters accurate and truthful?
2. Was a business reply envelope or self-addressed stamped envelope enclosed?
3. Were second and third mailings made to increase response rate?
4 . Was an attempt made to readdress and remail undeliverable questionnaires?
5. Is the percentage of return acceptable (i.e., at least 60 percent)?
6. Do the returned instruments appear to be representative of the population?
7. Are all of the questions on the instruments completed?
8. Is one individual responsible for accumulating and logging in all of the

completed questionnaires?
Data Processing
1. How will the data from the questionnaire be processed?
2. Is there a coding plan for analyzing the information obtained?
3. Is the coding plan in written form?
4. Who is responsible for the coding of all data?
5. Were consistency checks made on coding the data?
6. How were the calculations (e.g., percentages, means, and medians) made?
7. Were consistency checks made on the calculations?
Reporting
1. Who is responsible for preparing the final report?
2. Who is responsible for the final editing of the report?
3. Does the report list the date of publication, title of the project, and the
sponsoring agency?
4. Does the report include a copy of the original questionnaire?
5. Does the report describe clearly the objectives and limitations of the study?
128
6. Does the report describe clearly the methodology used in the study?
7. Are there comparisons with results obtained from other studies?
8. Is there an executive summary of the finding of the study?
9. Are the analysis and recommendations clear?
10. Are the tabulations for all questions included in the report?
11. If the answer to question lOis no, why were some results omitted ?
12. Are all relevant cross tabulations shown?
13. Are all tables clear and readable, with appropriate titles?
14. If charts are used, are they correct and relevant?
15. If projections are shown, are they justified and correct?
16. Is the report useful to the faculty responsible for revising teacher education
programs?
17. Is the report useful to administrators and governing board members in

making decisions relative to program changes?
REFERENCES
Adams, R. D., & Craig, J. R. (1981). A survey of undergraduate teacher

education evaluation practices. In S. M. Hord & R. D. Adams (Eds.),
Teacher education program evaluation, 1981: Theory and
practice (pp. 9-28). Austin, TX: The University of Texas, Research and
Development Center for Teacher Education.
Altschuld, J. W., & Lower, M. A. (1984, March). Improving mailed
questionnaires: Analysis of a 96 percent return rate. New Directions
for Program Evaluation, 21, 5-18.
Ayers, J. B. (1979, April). Follow-up studies: Providing the thrust
for program improvement. Paper presented at the meeting of the
American Educational Research Association, San Francisco, CA.
Babbie, E. R. (1973). Survey research methods. Belmont, CA:
Wadsworth Publishing Co.
Babbie, E. R. (1979). The practice of social research. Belmont, CA:
Wadsworth Publishing Co.
Berdie, D., & Anderson J. (1974). Questionnaires: Design and use.
Metuchen, NJ: Scarecrow Press.
129
Birdwell, L. A. (1989, February). Teacher attitudes toward their

preparation program and their profession. Paper presented at the
meeting of the Eastern Educational Research Association, Savannah, GA.
Boser, J. A. (1988, November). Teacher education follow-up surveys:
Are the respondents representatives of the groups? Paper
presented at the meeting of the Mid-South Educational Research
Association, Louisville, KY.
Davis, T. M., & Davis, J. F. (1986, Winter). Telephone surveys of graduates
can improve professional preparation programs. College Student
Journal, 20(4), 335-336.
Edwards, A. L. (1957). Techniques of attitude scale construction.
New York: Appleton-Century-Crofts.
Erdos, P. L. (1983). Professional mail surveys. Malabar, FL: Robert
F. Krieger Publishing Co.
Ewell, P. T. (1983). Student-questionnaires: An implementation
handbook. Boulder, CO: National Center for Higher Education
Management Systems.
Ewell, P. T., & Lisensky, R. P. (1988). Assessing institutional
effectiveness. Washington: The Consortium for the Advancement of
Private Higher Education.
Freeman, D. J. (1988). Compendium of items for follow-up surveys
of teacher education programs. (Technical Series 88-1). East
Lansing: Michigan State University, National Center for Research on
Teacher Education.
Hogan, R. R. (1985, Fall). Response bias in student follow-up: A comparison
of low and high returns. College and University, 61(1), 17-25.
Isaac, S., & Michael, W. B. (1981). Handbook in research and
evaluation. San Diego: EdITS Publishers.
Kerlinger, F. N. (1967). Foundations of behavioral research. New
York: Holt, Rinehart and Winston.
Loadman, W. E., Brookhart, S. M., & Freeman, D. J. (1989, March).
Developing a national database for preservice teacher
education follow-up studies. Paper presented at the meeting of the
American Association of Colleges for Teacher Education, Anaheim, CA.
Lockhart, D. C. (Ed.). (1984). Making effective use of mailed
questionnaires. San Francisco: Jossey-Bass.
Martin, O. L. (1987, November). A score plus one year of graduates'
perceptions of their teacher education program. Paper presented
at the meeting of the Mid-South Educational Research Association, Mobile,
AL.
McKillip, J. (1984, March). Applying attitude theories to the return of mailed
questionnaires. New Directions for Program Evaluation, 21, 77-
87.
National Council for Accreditation of Teacher Education. (1970). Standards
for accreditation of teacher education. Washington: Author.
Oppenheim, A. N. (1966). Questionnaire design and attitude
measurement. New York: Basic Books.
130
Sandefur, J. T. (1970, September). An illustrated model for the

evaluation of teacher education graduates. Washington: American
Association of Colleges for Teacher Education.
Sudman, S. (1985, June). Mail surveys of reluctant professionals.
Evaluation Review, 9, 349-360.
Sudman, S., & Bradburn, N. (1984, March). Improving mailed questionnaires
design. New Directions for Program Evaluation, 21, 33-41.
Villene, M., & Hall, B. (1981). Dispelling myths through systematic follow-
up of teacher education graduates. Teacher Education, 16, 34-38.
11
FOLLOW-UP EVALUATION OF TEACHER EDUCATION

PROGRAMS
James R. Craig
Follow-up evaluations or studies of teacher education programs are based on the

premise that teacher training produces competencies, defined as program
objectives, that can be observed in the behaviors of teacher education graduates.
Effective follow-up begins with the design and implementation of systems that
provide accurate information regarding the graduates' behaviors. Effective
systems provide timely, adequate, and appropriate feedback to program faculty
and administrators. It is crucial to note that effective follow-up studies involve
more than just the design of data collection systems. Follow-up evaluation is
NOT research-oat least according to the traditional meaning of the term research
(Sommer,1977). True, the conduct of a follow-up study incorporates research
techniques, but it also includes a set of additional processes and procedures that
are not required when engaging in research. An effective follow-up study requires
an understanding of:
o The social context in which the follow-up is being conducted,

o The purposes associated with the follow-up evaluation, and
o The manner in which the follow-up evaluation data will be used.
When these factors are taken into account, and only then, can truly effective
follow-up evaluation systems be designed and implemented. To ignore these
factors is to squander resources and produce data that will be of minimal use, if it
is used at all. To incorporate these factors is to enhance the likelihood that
follow-up evaluation will provide meaningful and timely feedback that will be
used to create more effective and efficient teacher education programs.
A BRIEF HISTORY OF FOLLOW·UP EVALUATION STUDIES
Follow-up evaluation of teacher education programs has been conducted for

some time and has utilized a variety of procedures. The emphasis on follow-up
132
studies in teacher education in the last two decades can perhaps be traced to the
1968 revision of the standards by the National Council for Accreditation of
Teacher Education (NCA TE). The new standards emphasized follow-up
evaluation. Sandefur's (1970) monograph on follow-up evaluation has been
widely used for teacher education evaluation. His model was based on an
outcome-oriented and competency-based approach. The continued emphasis on
follow-up evaluation has been reinforced recently by the NCATE; its revised
standards (1987) call for follow-up evaluation as a criterion for compliance. The
widespread use of follow-up evaluation in teacher education programs was
documented by Adams and Craig (1981) in a survey of 445 respondent
institutions affiliated with the American Association of Colleges for Teacher
Education (AACTE). Adams and Craig reported that 86% of the sample
indicated that they conducted follow-up evaluations of their programs using
questionnaires mailed to their graduates. [Refer to the previous chapter for
specific information on mail follow-up studies.] Interviews and direct
observations were also reported as being used, but much less frequently. Despite
widespread practice, however, the conduct of follow-up evaluation studies in
teacher education has been criticized.
After reviewing 26 evaluation studies, Katz, Raths, Mohanty, Kurachi, and
Irving (1981) raised issues about the validity and the usefulness offollow-up data
based on surveys of graduates. Katz, et aI., (1981) questioned the selection of
survey participants, the representativeness of the sample, the conclusions drawn
from the follow-up data, and the timing of follow-up data collection. They
reported what they believed to be sampling bias in response rates and obvious
and global recommendations for change which probably would not and, given the
vague and general nature of the recommendations, could not be addressed by
program faculty. Thus, Katz et al. concluded that under current conditions,
there is little reason for conducting follow-up studies, especially when using
questionnaires. They suggested that enough rival hypotheses and explanations
can be generated from an evaluation report to render it virtually useless for
program development. Adams, Craig, Hord, and Hall (1981) agreed with Katz et
al. that practice in follow-up evaluation is narrowly conceived. However,
Adams et al. went on to argue that by focusing on follow-up questionnaires
onl y, Katz and her colleagues presented a distorted view of the variety of methods
actually employed in the evaluation of teacher education programs--other
procedures (e.g., employer interviews, direct observation) have been used, albeit
much less frequently. Perhaps the most important lessons to be learned from the
many years of practicing follow-up studies in teacher education is that the social
context in which the follow-up is being conducted is one of the primary
determiners of its form and substance.
THE SOCIAL CONTEXT
The social context within which follow-up evaluation is conducted sets the
parameters that frame decisions relative to resources, operational procedures, and
133
other evaluation issues. In particular, four aspects of the social context associated
with follow-up evaluations must be considered and understood in order to design,
implement, and operate effective follow-up evaluation systems. These are
described in the following section.
The Political Environment
Follow-up evaluation in teacher education is conducted within a political

atmosphere wherein different groups and individuals "relate to each other from
different positions of power, influence, and authority" (Banner, Doctors, &
Gordon, 1975, p. 2). The reality of the interplay among the various players is
that, for political reasons, the purpose of follow-up evaluation may be less than
ideal and, therefore, the follow-up evaluation may be less effective and less
useful than it might otherwise be. An understanding of the purpose of the
follow-up evaluation is, therefore, crucial.
Personal/Professional Relationships
The personal and professional relationships between and among program

operators and program evaluators, if, in fact, they are different people, are critical
to the operation of a follow-up evaluation system. Ideally, program operation
and follow-up evaluation should be relatively independent but should function
together to create, operate, and refine the best program possible. When program
operation and follow-up evaluation are not independent, objectivity may be lost
and the follow-up evaluation may reflect only personal values rather than
accurate assessments. On the other hand, if the two are totally independent,
operational data may never get to evaluation people and follow-up information
may never get to program people. The two truly go hand in glove and directly
affect the manner in which the follow-up evaluation data are used.
Program Changes over Time
Teacher education programs change over time and, therefore, what was
appropriate and acceptable for follow-up evaluation at one time may not be at
another. Laws are passed, new research is published, facuIty retire, etc. Any or
all of these may necessitate a revision of the program of study and/or the
development of new courses. Such changes typically necessitate changes in the
design and/or content of the follow-up evaluation procedures. Unlike research
designs which attempt to maintain constancy of conditions throughout the
course of the data collection process, evaluation designs must be flexible and
adaptable to meet ever changing program conditions and information needs.
134
Values of the People Involved
Perhaps the most important aspect of the social context which influences
follow-up evaluation is the values held by the various audiences involved in the
evaluation process. Briefly, values are positive and negative feelings that people
have and project onto other people, objects, and situations (Williamson,
Swingle, & Sargent, 1982). As such, values are relatively permanent cognitive
structures that have a pervasive effect on an individual's behavior. In the context
of follow-up evaluations, values are a major force in the formation, operation,
and evaluation of programs (Wortman, 1975) and in the interpretation of
evaluation and program outcomes (Gorry & Goodrich, 1978). Because values are
relatively resistant to change, they tend to determine the direction and operation
of a program regardless of the nature and extent of the evaluation information
made available. This means that while both administrators and faculties aspire
to follow the Dogma of the Immaculate Perception (Kaplan, 1964) and try to
conduct value-free evaluation, in reality it is not possible. Recognition of the
prominent role that values play in the evaluation process provides a basis for
understanding how "non" evaluation factors influence the purpose, use, and
design of follow-up evaluations.
PURPOSES
Properly designed follow-up evaluations ask: the questions which need to be

answered to inform our thinking. In this context, there are at least four positive
purposes for conducting follow-up evaluation in teacher education:
1. Accountability,
2. Improvement,
3. Understanding.
4. Knowledge production.
Accountability
Accountability refers to conducting a follow-up evaluation to meet external

accreditation standards. Historically NCATE has included a standard on
evaluation. Most state approval processes and regional accreditation associations
also require that approved and accredited programs routinely collect evaluation
data for decision-making. In their 1981 study of program evaluation practice,
Adams and Craig did not conclude that evaluations were conducted primarily for
accountability purposes. Given, however, the fact that almost 400 institutions
reported conducting evaluations and also the general absence of evaluation reports
in the literature, they concluded that concerns about accountability are the
primary motivators which lead teacher educators to collect evaluation data from
their programs.
135
Improvement
Ideally, follow-up evaluations should be conducted to improve programs--to

inform stakeholders (Le., program administrators and faculty) of the strengths
and weaknesses of the program in all respects. The information needed by the
stakeholders to make informed decisions about programs is not necessarily
different from the data needed to meet the accountability purpose (e.g.,
systematically collected teaching performance data, students' performances on
paper-and-pencil tests). Faculty members are in the best position to make
program adjustments based upon a follow-up evaluation designed around the
purpose of improvement, if they value improvement and if they accept the
follow-up data as being appropriate and meaningful.
Understanding
A potentially more useful purpose for conducting program evaluation

studies is understanding. The more clearly the stakeholders understand the
experiences which the preservice teachers have and the more ownership they have
in the evaluation process, the more capable they will be of making program
decisions. Because one of the greatest obstacles to the use of evaluation data is a
feeling of detachment by the stakeholders, developing a broad-based
understanding of the program becomes an essential step in facilitating ownership
of the follow-up evaluation process. Thus, while the methods used to gather
follow-up data to address this purpose may not be different from the evaluation
methods used for other purposes, the way in which the follow-up plan is
designed and implemented is as important to the use of evaluating data as is the
resultant data and recommendations.
Knowledge Production
Perhaps the loftiest purpose for conducting follow-up evaluation is to

contribute to the knowledge base about teacher education. Little is known about
teacher education as a result of evaluation studies; there is an historic tendency
for evaluation reports to be used to meet accountability concerns and be
discarded. [Refer to Eisner's and Baker's articles relative to this topic in the
March, 1984 Phi Delta Kappan for their perspectives on the relationship
between research and practice.] There has been no systematic attempt to publish
evaluation findings and, thereby, to build a body of literature on evaluation
practice in teacher education. This is not to say that follow-up evaluation data
does not exist The College of Education at Tennessee Technological University
has been conducting longitudinal studies of teacher education graduates for some
twenty years and has used Sandefur's basic model for the past 15 years. The
findings from these studies are contributing to the knowledge base in teacher
education evaluation. Understandably, generalizing from an evaluation at one
institution to other settings is difficult at best but the goal should be to develop
136
a comprehensive knowledge base where the relationships among contexts,

inputs, processes, and products are thoroughly examined (Galluzzo, 1986).
USE OF FOLLOW-UP DATA
Follow-up evaluation is not "method bound" but rather is bound by the

social context within which it is conducted and the data are used (Chelimsky,
1987a; Mathis, 1980; Patton, 1988). Ideally, follow-up evaluation data are used
to make objective decisions regarding program development, modification,
operation, effectiveness, and the like. However, follow-up data are not always
used in such a direct, straightforward manner (patton, 1978, 1988; Chelimsky,
1987b). Part of the problem in determining the nature and extent of use lies
with the definition of use.
Use Defined
The definition of use typically advanced has been that use occurs when
follow-up data are directly employed in objective, observable ways in program
modification and operation (Cohen, 1977; Mathis, 1980; Patton, 1978).
However, such a definition does not take into account the fact that decision
making often is based more on the values held by the decision makers than on
the information available to them (Harper & Babigan, 1958). The use offollow-
up data in program decision making is almost always diffuse in nature and is not
always directly observable (patton, 1978). Use of follow-up data is an iterative
process which focuses on the assimilation of evaluation information into the
decision making process (Craig & Adams, 1981).
Follow-up evaluation systems can be structured to use the data they provide
by building in mechanisms to increase the systematic inclusion of follow-up
data in program decision making (e.g., Akpom, 1986; Covert, 1987; Craig &
Adams, 1981; Nowakowski, 1985). First and foremost, policy makers,
administrators, and program personnel at all levels must be actively involved in
the organization and implementation of the follow-up evaluation. Involvement
begins with the chief administrative officer responsible for the program but
extends to all individuals who are involved in the program and who have a stake
in its operation. The particular procedures by which this involvement is
accomplished can vary (see Havelock & Lindquist, 1978) but the outcome should
be that everyone has the opportunity to conduct a preliminary overview of the
evaluation and an analysis of its various ramifications. One way this may be
accomplished is detailed below. In addition, the individuals involved should have
regular and consistent contact with the follow-up data through such means as
reports, presentations at faculty meetings, and problem solving activities based
upon the follow-up (Freeman, 1987). A follow-up evaluation in which useful
and usable data are collected and which meets the criterion of internal logic is
doomed to failure unless a knowledge utilization strategy is built into the system
during the planning and implementation processes.
137
Design of Follow-up Evaluations
The design of follow-up evaluation systems in teacher education is almost

always limited to site-specific, idiosyncratic models developed for and
implemented by individual institutions. These recommendations are based in
part on experience and in part on theory and recognize that the design and
implementation of any follow-up evaluation system must be situation-specific.
Systematic follow-up must be a part of the design for any comprehensive teacher
education program. The general follow-up evaluation design outlined below and
summarized in Figure 1 is based on three premises:
1. The follow-up evaluation system is being instituted to provide relevant,

meaningful information that is to be systematically included in program
decision making.
2. All individuals responsible for program planning, implementation, and

operation should be involved in the evaluation effort.
3. The follow-up evaluation syste.m must operate within, and not apart from,
the organizational framework that currently exists.
Initiation and Verification
The impetus for implementing an evaluation system may come from

outside the organization or from within. Regardless of the source or the position
of the initiator, the initiation and the verification of a follow-up evaluation must
ultimately begin with the dean, director, or chair person of the unit responsible
for the program. Without that individual's understanding of and commitment to,
the evaluation of a program, the follow-up effort will not be successful.
Role of the Chief Administrative Officer in the Evaluation

Process
Individually (or perhaps with the assistance of an experienced evaluator), the

dean; director or chair of the unit should conduct a preliminary overview of the
evaluation and an analysis of its various ramifications. The intent is to identify
the dean, director, or chair of the unit's perceptions of the present institutional
and program circumstances; the ideal proglam; the need to conduct a follow-up
evaluation; the possible options for implementing the follow-up evaluation; the
possible program implications that might be suggested by different follow-up
evaluation data; and the resource restrictions within which the follow-up
evaluation must be conducted. The objective is NOT to have the chief
administrative officer establish the form and substance of what the follow-up
evaluation should be but to provide him or her with an understanding of how the
Initiation and Verification
Planning
Implementation
Feedback and Utilization
Figure 1. Key components of a follow-up evaluation design

in teacher education
139
follow-up evaluation process may operate. This process will allow the realistic
setting of the parameters within which the follow-up evaluation must function
(e.g., the budget), develop a commitment to the follow-up evaluation process,
and identify possible program decisions that could result.
If, at that point, the decision is still to institute a follow-up evaluation
system, then the preliminary review should be repeated with those individuals
within the organization who serve as the primary decision makers for the
programs to be evaluated. The decision making team should be identified by the
chief administrative officer and should be composed of individuals who
collectively facilitate the establishment of specific program goals, functions, and
operational procedures as well as those who are responsible for decisions
regarding program modification and continuance (e.g., an assistant dean, a
department chair). The same considerations addressed by the chief administrative
officer should be addressed by the decision making team. This group should also
determine the procedures for the creation of the planning and evaluation team.
Planning
The planning and evaluation team is the primary initiation and operational
force in a follow-up evaluation effort. The team determines the details of the
follow-up evaluation (e.g., the types of data to be collected) and is responsible
for communicating to all concerned the form and substance of the various
activities. So that the team can function effectively, it should be composed of
both formal and informal program leaders and be limited to no more than ten
members. The exact composition of the team will be specific to each situation
and is determined by such things as the number of program staff, the leadership
style of the chief administrative officer, budget restrictions, etc. The planning
and evaluation team should be charged with accomplishing four tasks:
1. Creating a follow-up evaluation plan,
2. Creating a communication system,
3. Designating a follow-up evaluation manager, and
4. Identifying a follow-up evaluation advisory group.
Each of these tasks is discussed below.
Creating a Follow-up Evaluation Plan
The planning and evaluation team should create a viable, flexible, workable
follow-up evaluation plan that:
o specifies the evaluation data required to make decisions regarding the present;
140
o rates the importance of the various data consistent with the teacher education
program goals and objectives and the current knowledge regarding effective
teaching;
o identifies and lists the possible sources of data;
o evaluates the possible data sources and data collection procedures in light of
access, cost, developmental time, staff development needs, time delays, etc;
o prioritizes the possible data sources and collection procedures in terms of the
follow-up evaluation data required and related resource restrictions; and
o describes the available data collection procedures selected and/or adapted, new
procedures developed, and the data collection training required, if any.
The planning and evaluation team should consider the following types of
evaluation data:
Knowledge--(general, professional and subject matter)

Teaching Behavior/Skills
Attitudes
Perceptions of preparation
No evaluation decision should be made based on a single source of data. A

useful evaluation will result when each source is considered--singly and in
combination.
Knowledge
The knowledge possessed by teacher education graduates is commonly

considered a product of a program. Tests such as the National Teacher
Examinations do not always capture the students' gain in general knowledge over
the four year college experience.
The attempt by one institution to measure the knowledge possessed by
teacher education students prior to, during, and after the program has been
reported by Galluzzo (1984). NTE scores indicated no significant differences in
gain of knowledge of general studies content over the four years of a teacher
preparation program. The data did show significant differences in mean scores on
the professional education component of the NTE, suggesting that students
attend college to become something, if not necessarily liberally educated.
The main difficulty with using knowledge as an indicator (as did Galluzzo)
is that general standardized tests are used to measure what students are expected to
know as a result of completing a teacher education program. The problem is
that the degree to which the test is consistent with the content taught in the
141
courses a student takes is an unknown. There is no reason to believe that the

NTE (or any other standardized test) is a content valid measure of either general
or professional education. Therefore, if knowledge is to be employed as a
follow-up measure, program faculty have to specify the particular knowledge the
students should possess as a result of their teacher education preparation. They
must also specify or develop the appropriate knowledge assessment instrument
on that basis. Institutions have traditionally resisted preparing such measures
because of the cost involved in preparing a test, and because faculty are unable to
reach consensus as to what the measures should include (i.e., what a prospective
teacher needs to know). The most promising work in this area lies ahead.
Alternative methods of assessing knowledge must be developed before
measurement of cognitive attainment becomes routine in follow-up evaluation.
At this time, beginning teachers are being inappropriately tested and teacher
educators are not building convincing arguments about the shortcomings of
current practice. Currently, follow-up evaluations which employ tests are
determined by what tests are available, rather than by what is to be evaluated.
Teaching Behavior/Skill
That a prospective teacher will attain skills is an expectation in all teacher

education programs. Expectations of teaching skills, however, are not
always clearly articulated. The systematic assessment of teaching skill by
teacher education programs is an unorganized practice and is being replaced
by internships and performance assessments of beginning teachers
(Galluzzo, 1987). Teaching skill assessment systems address selected
behavioral outcomes established for graduates of teacher education programs.
Several direct observation assessment systems are available. Each has
strengths and weaknesses and each requires training in its use before it can
be employed. Five systems which have demonstrated utility are listed
below. Each is described in the Appendix of this chapter.
1. Classroom Observation Record (Ryans, 1960),
2. Classroom Interaction Analysis System (Adams & Sandefur, 1976),
3. Teacher Performance Assessment Instrument (Ellett & Capie, 1981),
4. Classroom Observation Keyed for Effectiveness Research (Coker & Coker,

1982), and
5. Kentucky Career Ladder Classroom Observation Instrument (Galluzzo,

Roberts, & Taylor, 1988).
142
Teacher Attitudes
Attitudes represent a broad spectrum of dispositions that can be useful

indicators in a follow-up evaluation system. What and how students think about
the professional, technical, student, procedural, and intellectual aspects of their
craft can provide important understandings of a preparation program and its
effectiveness. Teacher educators have long argued that better teachers are not
only good technicians, they also possess certain attitudes. Demonstrating the
relationship between attitudes and ability has historically been a demanding and
inconclusive effort. The instruments used often do not have adequate predictive
validity (Borich, 1977) and have typically been reduced to global measures that
offer little insight into program effects (Zeichner & Tabachnick, 1981). Part of
the problem can be directly traced to the inability of faculty to agree upon a set
of attitudes to address. Thus, instruments such as the Minnesota Teacher
Attitude Inventory (Cook, Leeds, & Callis, 1951) and the Minnesota
Multiphasic Inventory (Hathaway & McKinley, 1970) remain the most overused
instruments in the study of program effects, and yet they yield very little
information which teacher educators can use to improve their programs.
Knowledge, behavior, and attitudes are equal and legitimate expectations
from any instructional interaction (Berliner, 1976). Studies of teacher
preparation have neglected to link affective variables to cognitive or behavioral
variables. Yet, by examining skills and knowledge in relationship to attitude,
evaluators can create a more complete description of the expectations of their
programs. An understanding of how changes in students' attitudes or
dispositions explain other findings is essential to understanding the outcomes of
a teacher education program. Deciding which attitudes are worth examining is a
program faculty decision; "how best to measure" those attitudes is a design
decision.
Perception of Preparation
Students' responses to survey questionnaires usually have been considered a

measure of their attitude toward and/or knowledge about their preparation. The
survey of graduates' perceptions of their preparation is probably the most
understood and frequently used data in teacher education follow-up evaluation.
As the concept of attitude is undergoing reconceptualization, however, responses
to survey questions are probably best characterized as perceptions (Katz, et al.,
1981). It is important to note that the perceptions of beginning teachers appear
to be very unstable during the first years of teaching. Perceptions stabilize
somewhere around the fifth year of teaching (Adams, 1987). Therefore, one
should not overreact, either positively or negatively, to graduates' perceptions of
their preparation programs. Rather, those perceptions should be treated as
another piece of evaluation information that must be used in conjunction with
other follow-up data to build a complete evaluation picture.
143
Creating a Communication System
In creating a follow-up evaluation plan, the planning and evaluation team

should also create an effective, operational communication system to function
along with the follow-up evaluation system. The communication system should
be used during the development of the follow-up evaluation plan to ensure the
involvement of all individuals in planning and modifying the follow-up
evaluation system. If implemented properly, the system should function after
the follow-up evaluation plan is operational to follow-up data to all concerned
faculty and administrators for use in program decision making.
Designating a Follow-up Evaluation Manager
A follow-up evaluation manager should be designated. The manager should

serve as the team'~ executive officer and be responsible for the daily tasks
associated with the implementation and operation of the follow-up evaluation
plan. The manager should have the needed technical evaluation skills and be a
capable group facilitator.
Identifying a Follow-up Evaluation Advisory Group
An advisory group comprised of practitioners, teachers, and administrators

should be established. Their input is necessary to provide the perspectives of the
school administrators and practicing teachers because administrators hire the
program graduates and teachers can share expertise gained on the job. Others
who may be important members of the advisory team are university educators in
areas other than teacher education (e.g., content areas, data processing).
Implementation and Data Collection
Implementation and data collection reflect the activities associated with the
operation of the follow-up evaluation plan. Briefly, the follow-up evaluation
plan established by the planning and evaluation team should be specified to the
extent that procedures for data collection are established. These include selecting
instrumentation and data collectors, establishing procedures for selecting
participants, identifying data management systems, etc. The follow-up plan
should be implemented and operated by the evaluation manager under the
direction of the planning and evaluation team.
One important aspect of the follow-up evaluation system is an active,

effective communication system that is continually operative from the beginning
of the follow-up evaluation effort. It is important to realize that the generation,
dissemination, and utilization of evaluation data is NOT a lock-step, linear
144
sequence (Havelock & Lindquist, 1978). Follow-up evaluation of teacher

education programs is conducted within a social/political environment and the
key to its successful operation is continual involvement of all concerned
individuals through an ongoing exchange of ideas and values coupled with the
evolution of a commitment to make the educational program the best it can
possibly be. The form that any particular communication system may assume
in a given instance will be peculiar to that situation. Regardless, the emphasis
should be on the effective use of follow-up evaluation data in program decision
making. As follow-up data become available, the data should be systematically
reviewed, assessed, and interpreted. Evaluation data should be analyzed and
presented in a form that is both consistent with the program objectives and
readily understood by the users of the information. A formalized process must
be established by which these data are reviewed by all staff and applied in making
program decisions. The program change process should be documented and the
follow-up evaluation system reviewed in light of any changes. instituted.
CONCLUSION
Involvement and "evolvement" are the keys to the development of follow-up

evaluation systems where follow-up data are effectively used. Policy makers,
administrators, and program personnel at all levels must be actively involved.
They all should participate in the determination and operation of an evaluation
system if it is to be one where the data collection and analysis is valid and one
where the data are seriously and regularly considered in making programmatic
decisions. Second, it should be realized that follow-up evaluation must be
viewed as iterative in nature with both the follow-up evaluation processes and
products evolving over time. The system should reflect current concerns but
also be flexible enough to adapt to new and changing needs. If follow-up
evaluation systems are to provide useful information for program decision
making, then they must be included in the planning process so that they might
be adapted for their most effective and efficient use in meeting the needs of the
education profession.
APPENDIX
The Classroom Observation Record. The Classroom Observation

Record (Ryans, 1960) is a medium inference observation instrument which
employs a summary rating scale used to assess 18 characteristics of teacher
behavior and four characteristics of student behavior. The Classroom
Observation Record yields three major teaching factors (Shiek & Adams, 1978):
(1) Factor A which reflects organized, confident, and systematic classroom
behavior; (2) Factor B which reflects empathetic, understanding, and adaptable
classroom behavior; and (3) Factor C which reflects stimulating, original, and
alert teaching behavior.
145
The Classroom Interaction Analysis System. The Classroom

Interaction Analysis System (Adams & Sandefur, 1976) is a low inference
observation instrument based on a modification of Flanders' interaction analysis
(Flanders, 1970). The system is composed of twelve categories of teacher talk,
three categories of student talk, and four nonverbal behavioral categories. When
using the system, the observer records a numerical value corresponding to a
specific category every three seconds or every time the category changes.
The Teacher Performance Assessment Instrument. The Teacher

Performance Assessment Instrument (Ellett & Capie, 1981) is a high inference
observation instrument composed of four broad behavioral categories of
competency statements. The categories are termed instruments and are used to
observe and assess: teaching plans and materials, classroom procedures,
interpersonal skills and professional standards. A fifth category or instrument,
termed student perceptions, is not part of the observation system and is a 30-
item paper and pencil rating scale completed by students. In using the four
observational categories or instruments the observer rates from 1 to 5 (with 5
being the highest rating) a number of indicators contained with each category or
instrument.
The Teaching Plans and Materials (TPM) instrument consists of 15
indicators of a teachers competence in planning instruction and selecting
procedures. It is divided into five sections as follows:
1. Plans instructions to achieve selected objectives. This section

contains five items relating to the planning of objectives, teaching
procedures, content, materials of assessment and instruction at varying
levels.
2. Organizes instruction to take into account individual

differences among learners. This section consists of three items
relating to organizing instruction by considering capabilities, learning
styles, and rates ofleaming.
3. Obtains and uses information about the needs and progress of

individual learners. This section has two items concerning the use of
evaluation materials as well as communication with learners about their
needs.
4. Refers learners with special problems to specialists. This

section contains three items relating to using student records, identifying
learners who need specialists, and using information to assist learners with
their problems.
5. Obtains and uses information about the effectiveness of

instruction to revise it when necessary. This section uses two
146
items relating to obtaining information about instruction and revising

instruction as needed.
The Classroom Procedures (CP) instrument assesses the teacher's cognitive

interaction with learners, skill in organizing and presenting instruction, and skill
in classroom management. This instrument involves 20 behaviors and is
divided into the following six areas:
1. Uses instructional techniques, methods and media related to

the objectives. This section contains three items concerning methods,
aids, and materials.
2. Communicates with learners. This section involves five items

dealing with giving directions (explanations), using learner responses,
providing feedback, and using written and oral expression.
3. Demonstrates a repertoire of teaching methods. This section

consists of three items relating to the implementing of learning activities,
demonstrating variety in methods, and demonstrating ability to use large and
small groups as well as individual instruction.
4. Reinforces and encourages learner involvement in

instruction. This section provides four items dealing with involving
students in lessons.
5. Demonstrates an understanding of the school subject being

taught. This section has two items relating to helping students to see the
importance of subject matter and demonstrating knowledge of subject
matter.
6. Organizes time, space, materials and equipment for

instruction. This section uses three items concerning routine tasks,
instructional time, and learning environment.
The third instrument involves Interpersonal Skills (IS) and ten behaviors.
This instrument assesses the teacher's characteristics as they are related to climate
in, and the teacher's ability to manage, a classroom. The instrument is divided
into three areas as follows:
1. Demonstrates enthusiasm for teaching and learning and the

subject being taught. This section contains three items relating to
communicating enthusiasm, stimulating interest, and conveying the
impression of knowing what to do and how to do it.
147
2. Helps learners develop positive self-concepts. This section uses

three items dealing with demonstrating warmth, sensitivity, and empathy.
3. Manages classroom interactions. This section has four items dealing

with providing feedback on behavior, promoting interpersonal relationships,
maintaining classroom behavior, and managing disruptive behavior.
The fourth instrument involves Professional Standards (PS) and assesses the
teacher's relationship with colleagues, the acceptance of professional
responsibilities and efforts to improve professional skills. The instrument is
divided into two areas as described below:
1. Meets professional responsibilities. This section contains four

items dealing with working with colleagues, following procedures,
demonstrating ethical behavior, and performing extra-instructional duties.
2. Engages in professional self development. This section has two

items dealing with participating in professional activities, and sharing
professional ideas and materials.
The Classroom Observation Keyed for Effectiveness

Research. The Classroom Observation Keyed for Effectiveness Research
(Coker & Coker, 1982) is a low inference observation instrument composed of
two sections. Section A is a two dimensional mati ix where the cells in the
matrix represent specific types of teacher/student illteractions. The teacher
dimension of the matrix is divided into three components of teaching behavior:
presenting, questioning, and responding. Each of these is in tum divided into
subcategories regarding more specific teacher/student behaviors. Section B
consists of 33 items relating to student behaviors (e.g., teases, shows pride) and
86 items relating to teacher behavior divided into the areas of methology, affect
and control, verbal, and grouping, each with a number of subdivisions.
The Kentucky Career Ladder Classroom Observation

Instrument. The Kentucky Career Ladder Classroom Observation Instrument
(GallUZZO, Roberts, & Taylor, 1988) is a combination low and medium inference
observation instrument composed of four sections. In the first section, the
observer records information regarding the characteristics of the teacher (e.g.,
years of teaching experience), the class (e.g., number of students with learning
disabilities), etc. In the second section the observer codes the development of
subject matter that occurs during the lesson (e.g., gives examples). In the third
section the observer codes the frequency of certain teacher behaviors (e.g.,
questioning techniques, etc.) as it occurs during the course of a lesson. After the
completion of the observation, the observer uses the fourth section to make
higher inference judgments regarding the teacher's behavior that was just
observed (e.g., degree of involvement, enthusiasm).
148
REFERENCES
Adams, R. D. (1987). Follow-up studies of teacher education graduates. In M.

Haberman & Backus, J. M. (Eds.).(1987). Advances in Teacher
Education, Volume 3. Norwood, NJ: Ablex Publishing Company.
Adams, R. D., & Craig, J. R. (1981, February). A survey of
undergraduate teacher education evaluation practices. Paper
presented at the meeting of the American Association of Colleges of Teacher
Education, Detroit. In S. M. Hord & R. D. Adams (Eds.). Teacher
Education Program Evaluation, 1981: Theory and Practice.
Austin, TX: The Research and Development Center for Teacher Education,
The University of Texas at Austin.
Adams, R. D., Craig, J. R., Hord, S. M., & Hall, G. E. (1981). Program
evaluation and program development in teacher education: A response to
Katz et al. Journal of Teacher Education, 32(5), 21-24.
Adams, R. D., & Sandefur, J. T. (1976, November). Factors influencing
the perceived problems of first year teachers. Paper presented at
Mid South Education Research Association, New Orleans, LA.
Akpom, K. (1986). Planning program evaluation to meet management
information needs. Evaluation Practice, 7(4), 35-37.
Banner, D. K., Doctors, S. I., & Gordon, A. C. (1975). The politics of
social program ev duation. Cambridge, Mass: Ballinger.
Berliner, D. C. (1976). Impediments to the study of teacher effectiveness.
Journal of Teacher Education, 27(1), 5-13.
Borich, G. (1977). The appraisal of teaching: Concepts and
process. Reading, MA: Addison-Wesley.
Chelimsky, E. (1987a). The politics of program evaluation. In D. S.
Cordray, H. S. Bloom, and R. J. Light (Eds.), New Directions for
Program Evaluation: Evaluation Practice in Review (Number
34, p. 5-21). San Francisco: Jossey- Bass.
Chelimsky, E. (1987b). What have we learned about the politics of program
evaluation? Evaluation Practice, 8(1), 5-21.
Cohen, L. H. (1977). Factors affecting the utilization of mental health
evaluation research findings. Professional Psychology, 8(4), 526-
534.
Coker, J. G., & Coker, H. (1982). Classroom observations keyed for
effectiveness research. Atlanta: Georgia State University/Carroll
County Teacher Corps Project.
Cook, W. W., Leeds, C. H., & Callis, R. (1951). The Minnesota teacher
Attitude inventory. New York: The Psychological Corporation.
Covert, R. W. (1987). Ways of involving clients in the evaluation process.
Evaluation Practice, 8(4), 83-87.
149
Craig, J. R., & Adams, R. D. (1981, February). Use-oriented evaluation.

Paper presented at the meeting of the American Association of Colleges of
Teacher Education, Detroit. In S. M. Hord & R. D. Adams (Eds.),
Teacher education program evaluation, 1981: Theory and
practice. Austin, TX: The Research and Development Center for Teacher
Education, The University of Texas at Austin.
Eisner, E. W. (1984). Can educational research inform educational practice?
Phi Delta Kappa, 65 (7), 447-452.
Ellett, C., & Capie, W. (1981). The teacher performance assessment
instrument. Athens: Teacher Assessment Project.
Flanders, N. A. (1970). Analyzing teaching behavior. Reading, MS:
Addison-Wesley.
Freeman, D. (1987, October). Issues and strategies for using program
evaluation findings in teacher education at Michigan State
University. Paper presented at the American Evaluation Association,
Boston.
Galluzzo, G. R. (1984, March). An evaluation of a teacher education
program. Paper presented at the American Educational Research
Association, New Orleans.
Galluzzo, G. R. (1986). Teacher education program evaluation: Organizing or
agonizing? In J. D. Raths & L. G. Katz (Eds.) Advances in Teacher
Education, Volume 2. Norwood, NJ: Ablex.
Galluzzo, G. R. (1987). Assessment of the teaching skills of beginning
teachers. In L. M. Rudner (Ed.), What's happening in teacher
testing. Washington: U.S. Department of Education.
Galluzzo, G. R., Roberts, R. A., & Taylor, K. H. (1988, March).
Development of a research-informed classroom observation
instrument: Lessons learned. Paper presented at the meeting of the
American Educational Research Association, New Orleans, LA.
Gorry, G., & Goodrich, T. J. (1978). On the role of values in program
evaluation. Evaluation Quarterly, 2, 561-571.
Harper, D., & Babigan, H. (1958). Evaluation research: the consequences of
program evaluation. Mental Hygiene, 55, 151-166.
Hathaway, S. R., & McKinley, J. C. (1970). The Minnesota
multiphasic inventory. New York: The Psychological Corporation.
Havelock, R. G., & Lindquist, J. (1978). A conceptual framework increasing
the impact. In J. Lindquist (Ed.), Increasing the import of social
innovations funded by grantmaking organizations. Battle Creek,
MI: The W. K. Kellogg Foundation.
Kaplan, A. (1964). The conduct of inquiry: Methodology for
behavioral science. San Francisco: Chandler.
Katz, L., Raths, J., Mohanty, C., Kurachi, A., & Irving, J. (1981). Follow-up
studies: Are they worth the trouble? Journal of Teacher Education,
32(2), 18-24.
150
Mathis, W. (1980). Evaluating: The policy implications. Phi Delta Kappa

CEDR Quarterly, 13(2), 3-6,22.
National Council for Accreditation of Teacher Education (1987). Standards
for the accreditation of teacher education. Washington: Author.
Nowakowski, J. (1985). Evaluation for strategy setting. Evaluation
Practice, 6(4), 57-61.
Patton, M. Q. (1978). Utilization-focused evaluation. Beverly Hills,
CA: Sage.
Patton, M. Q. (1988). The evaluator's responsibility for utilization.
Evaluation Practice, 9(2), 5-24.
Ryans, D. G. (1960). Characteristics of teachers. Washington, D.C.:
American Council on Education.
Sandefur, J. T. (1970). An illustrated model for the evaluation of
teacher education graduates. Washington, D.C.: American
Association of Colleges for Teacher Education.
Shiek, D. A., & Adams, R. D. (1978). Factor score interpretation of
the classroom observation record. Bowling Green, KY: Western
Kentucky University, Office of Educational Research, Technical Report.
Sommer, R. (1977). No, not research. I said evaluation! APA Monitor,
8(4), 1-11.
Williamson, R. C., Swingle, P. G., & Sargent, S. S. (1982). Social
psychology. Itasca, IL: Peacock.
Wortman, P. M. (1975). Evaluation research: A psychological perspective.
American Psychologist, 30, 562-575.
Zeichner, K. M., & Tabachnick, B. R. (1981). Are the effects of university
teacher education "washed out" by school experience? Journal of
Teacher Education, 32(3), 2-11.
12
EVALUATING THE STRUCTURE OF THE

EDUCA TION UNIT
Edell M. Hearn
The organization and administration of teacher education programs has become

an important issue on many college and university campuses because
controversy exists over which unit within an institution should be responsible
for teacher education. For example, in many institutions students desiring to
become secondary teachers must enroll in a College of Arts and Sciences, while
at others they enroll in a School, College, or Department of Education (SCDE).
There is no one system of organization and administration that can be uniformly
applied or said to apply across all of the 1,250+ institutions in the United States
which prepare teachers.
The curriculum for teacher education varies across institutions and between
programs. There is, however, a need to apply certain quality controls and to
prove that certain standards have been met. One way of ensuring that the
graduates of a teacher education program meet a high level of performance is
through the implementation of a system of governance which is controlled by
those who can administer the program effectively. The control or governance of
teacher education programs is of concern to the accreditation/approval agencies
for teacher education programs.
The National Council for Accreditation of Teacher Education (NCA TE),
specifically requires that an institution have a governance system that meets
certain compliance criteria. The regional accrediting associations have general
standards for the governance of institutions of higher education. The absence of
a strong governance system in a teacher education program, for example, could
jeopardize the total accreditation of the institution. The approval processes for
teacher education programs vary from state to state. Some states have adopted
the NCATE standards, while others have developed their own standards and
processes for use in the review and approval or accreditation of teacher education
programs. A review of the regional standards for approval of institutions and the
state processes for approval of teacher education programs led to the conclusion
that if a unit met the NCATE standard on governance of teacher education
programs, it would have met the standards imposed by the other agencies (Ayers,
1988). This chapter focuses on governance as defined by NCATE and on ways
152
to analyze and determine whether a given system meets the intended standards.
NCA1E has defined governance as follows:
Governance addresses responsibility for basic policy development,

program initiation, on-going evaluation, leadership and coordination
with other campus units, the maintenance and support of all
professional programs, selection and evaluation of faculty, and fiscal
matters. Governance establishes ultimate accountability for the quality
of programs in professional education and the quality students who are
graduated from professional programs (1987, p. 56).
The definition of governance provided by NCA1E allows for some latitude

in interpretation and implementation. This chapter discusses governance
structures for both basic (undergraduate) and advanced (graduate) programs.
Included here are illustrations of three governance systems which are in place at a
small, a moderate size, and a large institution. The governance systems at these
institutions have been reviewed on one or more occasions and have been judged
to meet the NCA1E standards. There are, of course, other possible governance
structures. The questions contained in the Appendix to this chapter will aid
faculty at a given institution in determining if their governance system meets the
NCA1E standards. .
NCATE STANDARD ON GOVERNANCE
Governance is a major component of the NCA1E standards as they were

revised in 1986. Under the revised system governance is a separate standard
under Category V entitled "Governance and Resources." Standard V. A:
Governance, includes nine criteria for compliance. The standard states:
The governance system for the professional education unit ensures that
all professional education programs are organized, unified, and
coordinated to allow the fulfillment of its mission (NCA1E, 1987, p.
49).
Category V includes Standard V. B: Resources. The standards on Resources and

Governance are very closely related. The issue of resources is important in the
operation of a teacher education program. The editors of this text considered it
so important that they devoted a chapter to the evaluation of three specific types
of resources. [Refer to the chapters by Berney for Physical Facilities Evaluation,
Saunders for Evaluation of Financial Resources for a Teacher Education
Program, and Garten for Evaluation of Library Resources for a Teacher Education
Program].
Although some changes in the redesigned standards were semantic in nature,
the most striking ones concerned the membership of the unit and the experience
that the majority of the faculty must have had in the public schools. Reference
is made to an "administrative body" being acceptable to carry out the
responsibilities of the unit. Although no mention is made of a council,
153
commission, or committee, it is assumed that these could form all or part of the
administrative body if they were officially recognized by the institution.
As in the past, although governance or its synonym may not be used
directly in the other standards, the likelihood of many other standards being met
or not met depends to a great degree upon the governance system. When
governance is weak it is likely that one or more other weaknesses in the total
teacher education program may be in evidence. Following are some hypothetical
examples of problems that could occur:
1. Professional courses in methods of teaching may be taught by individuals

outside of the professional education unit. Control of the content and
methods of instruction will not likely be under the person responsible for
teacher education. The faculty may be lacking in experience in grades K-12.
2. The knowledge base controlling the teacher education program may have
gaps, or there may be a lack of coordination of the program.
3. Faculty may have joint appointments in two departments. Such matters as

tenure and salary may be arrived in an awkward manner and usually not to
the satisfaction of the faculty member(s).
4. Selection cf faculty may be more difficult because the decision making

process may involve two departments. There will not be a fixed person to
whom the faculty will be responsible.
5. When a faculty member is not fulfilling a contractual obligation, the

situation is difficult if not impossible to remedy. Who has the authority to
correct the situation and how?
6. Courses in teacher education may be spread across several departments but

no one is really in charge or has the authority over the content of courses or
when they will be offered.
7. Students may not be differentiated by major field of study. For example, the
courses of study within a program for the preparation of an industrial
chemist are essentially the same as those for an individual who wishes to
teach chemistry in grades 7-12. There is, however, a need for the individual
who aspires to be a teacher to take specialized courses to meet state licensure
requirements and to pursue work in pedagogy and other areas of professional
education.
8. There may be few, if any, interdisciplinary courses in which prospective

teachers can enroll.
9. Faculty who teach in general education and also in the specialized major
areas of a teacher education program are reluctant to become involved in the
154
public schools. They do not see it as a part of their job and are sometimes
not rewarded for it when an effort is made to become involved.
10. The existence of multiple programs in a single specialty at an institution

(e.g., early childhood in Education and child development in Home
Economics) may be a weakness unless there is careful coordination among
the units involved.
11. Separate autonomous programs may exist in which students receive little
preparation in professional education. These programs are most often
offered through a College of Arts and Sciences in such areas as Music, Art,
Health and Physical Education, and Psychology.
12. Advisement may be frustrating and complex for both faculty and students.
A strong system of governance must exist for the other NCATE standards to
be met. The governance system must be in place, along with a knowledge base
and a system for quality controls, before such activities as follow-up evaluations
of graduates can be effectively implemented. It is important therefore, that the
unit give special attention to this fact. One must not forget that a system for
evaluation is a necessity. It is a rather simple process of accountability that
calls for the institution, based upon the best research available, to plan a
program(s), explain what program(s), implement the program(s), and prove that
the goals and purposes of the program(s) have been met. The crucial element of
any program is how well the graduates have performed. Without a proper
governance structure this information is not likely to be obtained.
Historically, teacher education has been faced with the responsibility of
effecting a balanced program between disparate elements. On the one hand it has
been important to have the cooperation of the arts and sciences faculty for both
general education and subject matter preparation while at the same time having
the support of the practitioners in the field to provide clinical experiences and
supervision. It has been necessary for the professional faculty to work
effectively with both groups in trying to blend both elements (general and
clinical education) into the teacher education programs. This program-making
responsibility and authority is central to the policy decisions governing teacher
education in the institution. Program development is no easy task and the
burden sometimes resides with teacher education faculty.
There is no one system of governance that will meet the needs of all
institutions. It should be kept in mind that governance must be tied to the
Professional Education Unit, which is described as
the college, school, department or other administrative body within the

institution that is primarily responsible for the preparation of teachers
and other professional education personnel. Not all of the programs for
the preparation of school personnel need to be administratively located
within the unit. However, the NCATE standard on governance requires
155
that all professional education programs are organized, unified and

coordinated by the unit (NCATE, 1987, p. 58).
Governance Structures for Teacher Education Programs
Accreditation is an important issue to everyone involved in teacher education

programs. A question that frequently arises is "What should be the governance
structure for a given teacher education program?" There are several ways to
approach the problem of studying the governance structure. Any institution that
is preparing for an accreditation study should make contact with other like
institutions that may have recently undertaken such an activity. There is
nothing wrong in sharing written and oral material related to any aspects of
accreditation. It is a good idea to capitalize on the work that others may have
done relative to governance. Additionally, consultants who have had
considerable experience with accreditation can be called upon to provide
assistance. Individual(s) outside the institution should be called upon to read and
react to the different documents related to governance. This can be very helpful.
The visiting team members will likely be strangers to the campus being
considered. Others always have a view of some things differently than the
campus faculty and staff perceive them.
There appear to be at least three distinct types of governance structures in
teacher education programs that meet accreditation standards. These three
structures are based, in part, on the size of the institutions (Le., small, moderate,
and large institutions). Generally small institutions include private colleges
with enrollments of less than 2,500 students. Moderate size institutions include
both public and private colleges and universities with enrollments up to 12,000
students (generally these institutions have a separate school or college of
education). The large universities include those with enrollments exceeding
12,000 students and are generally multi-purpose institutions (N.B., the smaller
land-grant institutions would probably fit into this category).
The remainder of this section includes examples and a description of the
organization and governance structures of three teacher education units. These
examples are based on real institutions that have met the NCATE standards.
College X is a small independent liberal arts college that is affiliated with an
organized church. College Y is a moderate-sized state supported regional
institution, and College Z is a large multi-purpose state supported complex
institution which is oriented toward graduate teacher education programs and
contract research work. The three schemes outlined are only examples and are
limited in scope and comprehensive descriptions. They are meant to serve only
as "possibilities." Other governance structures are possible.
The checklists and questions related to the compliance criteria for the
standard on Governance contained in the Appendix to this chapter will aid in the
process of determining if a given structure meets the NCATE standard. The
checklist and questions are not inclusive, but should be of help in determining if
an institution is in compliance. No claim is made that by answering the
questions satisfactorily an institution will achieve accreditation. That decision is
156
within the realm of professional judgment. The checklist and questions can be
applied to all types of institutions irrespective of size or mission.
Governance Structure for Small Institution (College X).
College X is a small, liberal arts institution with an enrollment of 552 full-

time and 34 part-time undergraduates. The enrollment in undergraduate
professional education programs is 96. The total college faculty consists of 52
individuals, with six full-time and four part-time in professional education. The
College offers three undergraduate and no graduate programs.
The teacher education faculty of the College serves as the governing unit.
All policies affecting teacher education are channeled through this group. The
unit is responsible to a curriculum committee as are all programs within the
institution. The curriculum committee is actually an overseer of all programs of
the institution and depends upon the teacher education unit to approve and
recommend any changes in teacher education programs. The teacher education
unit is chaired by the Director of Teacher Education who is responsible to the
academic dean of the College. In cases where the curriculum committee differs
with the unit of teacher education on an issue, the matter is resolved by an
appeals board which is established and chaired by the academic dean of the
institution. Figure 1 shows how governance decisions reflecting teacher
edu( ation are made at the College. Figure 2 shows the administrative structure
for the teacher education program for College X.
It has been clearly established in writing that the Director is the individual
responsible for the teacher education programs of the college. This person
reports to the academic dean, who in turn reports to the President of the
institution.
Governance Structure for Moderate Size Institutions (College V).
College Y is a moderate-sized, regional, public institution. The institution

is comprised of five colleges and schools with a total enrollment of 8,572
students. This includes 7,577 undergraduate students and 995 enrolled in various
master's and educational specialist's programs. The institution has a faculty of
410. The College of Education has the third largest enrollment with 2,489 full-
time and 238 part-time students. Teacher education has a 93 member facuIty. In
addition to the teacher education programs of the College of Education, the
College of Business Administration offers work leading to teacher certification in
Business Education. Students enroll in the Department of Business Education
and Office Administration in that College.
The institution uses an all-institutional approach for the undergraduate
teacher education programs. The unit for teacher education is the Teacher
Education Committee. All policies and program requirements must be acted
upon by this committee. The College of Education is the organizational unit
responsible for carrying out the policies of the Teacher Education Committee.
The Dean of the College of Education is the person designated by the
President to be in charge of and the spokesperson for teacher education. This
157
individual is responsible for implementing the policies of the Teacher Education

Committee. Although most students who are preparing to be teachers major in
the College of Education, some may choose to major in an area in the Arts and
Sciences (e.g., chemistry, English, or history). These students, however, must
meet all of the requirements of the teacher education program.
I PRESIDENT I
.... - -- -l APPEALS
COMMITTEE
I
COLLEGE
CURRICULUM
COMMITTEE
TEACHER EDUCA-
TIONFACULTY
(The Unit)
Figure 1. Governance Structure for Teacher Education

at College X.
The Teacher Education Committee establishes policies for the teacher
education programs of the University and provides a whole institution approach
to the preparation of teachers. The work of this body is reviewed by the
Academic Council which has representatives from all of the academic units of
the institution and serves as an advisory body to the Chief Academic Officer.
The graduate education programs at this institution are decentralized which
means many policies and procedures are determined at the College level. Within
the College of Education, the Graduate Teacher Education Committee serves in
the same capacity at the graduate level as does the institution's Teacher Education
Committee. The work of the Graduate Teacher Education Committee is subject
to review, however, by the Graduate Executive Committee of the institution.
This body includes representatives from all units offering graduate work. The
work of the Graduate Executive Committee is subject to review also by the
Academic Council. The Dean of the Graduate School is responsible for
administering general policies related to all graduate students, while the Dean of
158
the College of Education, is specifically responsible for implementation of all

policies related to graduate students enrolled in work related to teacher education.
I PRESIDENT
ACADEMIC
DEAN
DIRECTOR OF
TEACHER
EDUCATION
I I
PROGRAM IN PROGRAM IN PROGRAM IN
EARLY CHILD- ELEMENTARY SECONDARY
HOODEDUC. EDUCATION EDUCATION
Figure 2. Administrative Structive for Teacher Education

at College X.
Figure 3 shows an illustration of how governance decisions reflecting teacher
education are made at the institution. Figure 4 shows the administrative
structure responsible for the implementation of policy related to teacher
education programs at College Y.
Governance Structure for a Large Multi-purpose Institution

(College Z).
College Z is a large, public, multi-purpose institution (a university) with a

major emphasis on graduate teacher education programs and research. There are
ten colleges with total enrollment of 16,250 students of whom 4,100 are
enrolled in graduate programs. The faculty includes 898 full-time and 92 part-
time individuals.
The College of Education is the unit responsible for both the undergraduate
and graduate teacher education programs. The undergraduate programs are limited
to secondary education with only 12 teaching fields. Graduate programs are
offered at the master's, 6th year/specialist and docrorate levels. There are 680
ACADEMIC
COUNCIL
I
I 1
TEACHER EDUCA- GRADUATE
TION COMMITTEE EXECUTIVE
(The Unit) COMMITTEE
- - - -- DEAN, COLLEGE
OF
EDUCATION
- - Indicates
coordinating
relationships.
CURRIC ~UM COLLEGE OF TEACHER EDUCA-

COMMITTE ESOF EDUCATION TION GRADUATE
OrnE t CURRICULUM COMMITTEE
COLLE ~S* COMMITTEE
*Would include other Colleges such as

Business Administration and Arts and Sciences.
Figure 3. Governance Structure for Teacher Education at College Y.

IPRESIDENT I
I
VICE PRESIDENT
FOR
ACADEMIC AFFAIRS
I
DEAN, COLLEGE DEAN,COLLEGE
OF BUSINESS OF - - Indicates a Coordinating Relationship
ADMINISTRATION - - - - - - - -- EDUCATION
I
I I
DIRECTOR OF ASSOCIATE
r--------- SUPPORT SER- DEAN FOR
I VICES & LAB- ADMINISTRA-
I ORATORY EXP. TION
I
I
I I
DEPT. OF BUS. DEPT. OF DEPT. OF DEPT. OF SCHOOL DEPT. OF LAB.
EDUC. & OFFICE LEISURE ART & MUSIC SERVICES C& I SCHOOL
ADMINISTRATION EDUCATION EDUCATION -PERSONNEL
--
Figure 4. Administrative Structure for Teacher Education at College Y.

161
undergraduate teacher education majors and 2,120 students working toward

graduate degrees. The College of Education faculty is composed of 220 full-time
and 26 part-time individuals.
The unit for teacher education at this institution is the College of Education.
There is an inter-university committee on teacher education that exists because of
teacher education programs outside of the College of Education. This
committee, however, reports directly to the Dean of the College of Education
who is responsible for all of teacher education at the University. All students
who are admitted to teacher education meet the same requirements as those of the
College of Education. Within the College of Education there are several
committees that report directly to the Teacher Education Committee of the
College of Education. Figure 5 shows an illustration of how governance
decisions reflecting teacher education are made at the institution. The
administrative structure responsible for the implementation of policy related to
teacher education programs at College Z is similar to that shown in Figure 4 for
College X.
Summary on Governance
In some way the standards of all five NCATE categories relates to

governance. The Introduction to the NCA TE standards for the Accreditation of
Professional Education Units states:
The focus of these standards is on the overall quality of the professional

education unit, [college, school, department or other administrative
body within the institution that is primarily responsible for the
preparation of teachers and other professional education personnel] and
not on individual programs in the institution. However, the unit's
policies, procedures, governance, administration, staffing, and other
resources should ensure that programs are accountable. In responding to
these standards, institutions should indicate how the unit ensures that
the standards are met. For example, institutional reports should make
clear in responding to standard I. A how the unit ensures that programs
are based on established and current research findings and sound
professional practice (NCATE, 1987, p. 35).
It is clear that if governance is out of place many of the other standards will
be weak or not met. All of the standards are important but the two most critical
standards in the minds of most educators are those of governance (control) and
evaluation (program and product [student] at entrance and after entering the
profession).
In examining the governance of a teacher education program, special
attention should be given to the NCA TE criteria for compliance with the
Standard V. B: Resources. Particular attention should be given to the criteria
since they are closely related to governance and control of the teacher education
program and have been sources of problems in meeting accreditation standards at
some institutions. The criteria are presented following the Figure.
CURRICULUM
COUNCIL*
(Faculty Senate)
COLLEGE OF
EDUCATION
COLLEGE OF INTER-UNIVERSITY
SUBJECT MATIER
EDUCATION COMMITTEE ON
DEPTS. OF OTHER t- - -
DEPTS. TEACHER EDUCATION
COLLEGES
(The Unit)
- - Indicates coordinating relationships

FACULTY OF
TEACHER EDUCA-
TION PROGRAMS *The Graduate School delegates the develop-
ment of policy in academic matters to the
Graduate Council of the Faculty Senate.
Figure 5. Governance Structure for Teacher Education at College Z.

163
77. Adequate number of faculty, cooperating teachers and other field-based

supervisors to support the program;
79. A maximum faculty to student practicum ratio of 1 to 18;
89. Long-range plan for facilities; and
92. An identifiable and relevant media and materials collection for use by
students and faculty (NCATE, 1987).
The role the Unit and governance plays in regard to the purely quantitative
standards of accreditation is also critical. For example, NCATE standards require
that students achieve a 2.5 grade point average for admission to the study of
professional education, that students spend a minimum of ten weeks full-time in
student teaching, and that the maximum teaching loads for full faculty at the
undergraduate level not exceed 12 semester hours and 9 semester hours for faculty
teacQing at the graduate level.
Irrespective of the specificity that is contained in the standards, there is
much professional judgment to be made in regard to whether they are met. This
is as it should be! Application of the questions to the compliance criteria in the
Appendix of this chapter should be helpful in establishing an excellent system of
governance. In tum the likelihood of an institution being successful in meeting
accreditation standards is greatly enhanced.
APPENDIX
QUESTIONS TO AID IN DETERMINING IF THE NCATE

ST ANDARD ON GOVERNANCE HAS BEEN MET
Governance of basic (usually undergraduate) and advanced (graduate) teacher

education programs is a primary concern of institutions seeking NCA TE
accreditation. The materials presented below are concerned directly with meeting
the NCATE standard on Governance. However, the questions and checklist will
be of value in evaluating the overall structure of the program irrespective of
whether an institution is seeking to meet the NCATE standards. Standard V. A
on Governance reads:
The governance system for the professional education unit ensures that
all professional education programs are organized, unified, and
coordinated to allow the fulfillment of its mission (NCA TE, p. 49).
In order to meet this standard, an institution must demonstrate that it has

fulfilled certain compliance criteria. Following is a listing of the nine
compliance criteria related to Governance and questions that can be answered in
order to evaluate the degree to which the standard has been met. The compliance
criteria were taken from the NCATE standards (1987).
164
Criteria 68. The goals of the professional education unit are congruent with
the institution's mission.
1. What is the mission of the institution?
2. What evidences exist that indicate that the mission of the institution is in
harmony with the goals of the professional education unit?
3. If the mission of the institution and the goals of professional education are
not in harmony, what is the major rationale?
4. How did the institution determine that the goals of the professional
education unit and the missions of the institution were congruent?
Criteria 69. The unit effectively carries out its responsibility and discharges
its authority in establishing and implementing appropriate policies for
governance, programs, admission and retention of education students, and faculty
selection and development in professional education.
1. Do the minutes of appropriate committees, other groups and other valid

evidences indicate that the unit has established and implemented appropriate
policies for governance?
2. Are there indications (in writing) that the control of programs lie with the
unit of teacher education?
3. Has the institution adopted admission and retention standards that reflect the
carrying out of other NCATE criteria for compliance?
4. What types of activity have been provided in the past three years for faculty
development in professional education?
5. How much financial support has been given to professional development

activities in relation to other units of the institution?
6. How does faculty selection in professional education take place?
7. What are the procedures and processes that are followed in the
employment of new faculty?
Criteria 70. The unit effectively carries out its authority in making decisions
affecting professional education programs.
165
1. How does the unit determine if it is effectively carrying out its

responsibility in discharging its authority in matters related to the
professional education program?
2. Does the central administration of the institution actively support the

education unit in carrying out its responsibilities?
its authority for identifying and using appropriate resources for professional
education.
1. What resources such as computers, audio-visual materials, and special

laboratories are available in teacher education?
2. Is there a person(s) charged with the responsibility of identifying and

developing resources for professional education?
3. How do the resources available for teacher education compare with those
available with other divisions of the institution?
4. What evidence is there that the students actually make effective use of the
resources provided?
its authority in developing and maintaining appropriate linkages with other
units, operations, groups, and offices within the institution and with schools,
organizations, companies and agencies outside the institution.
1. By what means does the teacher education unit assure working relationships
with schools, organizations, companies and agencies outside the institution?
2. What has been the result of the activities conducted with schools,
organizations, companies and agencies outside of the institution within the
past three years?
3. In what way is the institution organized to assure appropriate relationships

between teacher education and other units?
4. Are there problems that exist between teacher education and other units at
the institution?
5. If Question 4 was answered "Yes," what problems, if any, have been dealt
with in the last three years and how?
166
6. Is the teacher education unit considered to be one that demonstrates that

education is truly the responsibility of the total profession recognizing that
there are different roles for individuals in the profession ranging from pre-
school or kindergarten through the college/university level?
7. Is there respect for each of the varying roles of faculty in the institution to
the extent that there is a high degree of cooperation and a spirit of learning
from one another?
8. Is teacher education considered to be very much field oriented or is emphasis

on educational theory dominant?
Criteria 73. The unit has, and regularly monitors, a long-range plan.
1. Has the teacher education unit had adequate input into the long-range plans
of the institution?
2. What documentation is available to show the input teacher education had

into the long-range plans of the institution?
3. Does teacher education have a specific long-range plan?
4. Describe the process used to develop long-range plans in the teacher

education unit
5. How, by whom and how often are the long-range plans of the institution
monitored?
6. How, by whom and how often are the long-range plans of the teacher
education program monitored?
Criteria 74. An officially designated professional educator administers the

professional education unit
1. Who is the officially designated person for administering the professional

education unit?
2. By whom was the person in Question 1 appointed and how was it made
official?
3. Is the individual recognized throughout the institution as the person who is

responsible administratively for the professional education unit?
167
Criteria 75. A systematic plan ensures the involvement of teachers, education

students and other education professionals in the unit's policy-making and/or
advisory bodies for the purpose of recommending requirements and objectives for
professional education programs.
1. What evidence is there that assures that the plan is systematic for ensuring
the involvement of teachers, education students and other education
professionals in those policy-making and/or advisory bodies recommending
requirements and objectives for the professional education programs?
2. By whom were the individuals selected and what was the process?
3. What evidence is there that various individuals' presence on these various

bodies have made a difference, that is, influenced changes that have been
made in the past three years?
4. Has any of the above bodies, on their own initiative, made specific
recommendations that have been considered, approved, or rejected by the
governing unit?
Criteria 76. Policies in the unit guarantee due process to faculty and students.
1. Does the institution have a person responsible for monitoring and

determining if its affirmative action goals are being met?
2. Who is the person responsible for monitoring the affirmative action goals
of the institution?
3. Is there equity regarding salary, working conditions, and other factors among
faculty in teacher education and other faculty in the institution?
4. What is the appeals procedure used by the institution if a student feels that
due process has been violated or he/she has been treated unfairly?
5. What is the appeals procedure used by the institution if a faculty member

feels that due process has been violated or he/she has been treated unfairly?
6. Have there been instances of the due process procedures being utilized during
the past three years?
7. If the answer to Question 6 was yes, what were the conditions?

168
REFERENCES
Ayers, J. B. (1988). Review of state guidelines for the approval of

teacher education programs. Unpublished manuscript, Tennessee
Technological University, Center for Teacher Education Evaluation.
Collins, E. R. (1971). 12th Charles W. Hunt Lecture. The impossible
imperatives: Power, authority and decision making in
teacher education. Washington: American Association of Colleges for
Teacher Education.
National Council for Accreditation of Teacher Education. (1979). Standards
for the accreditation of teacher education. Washington: NCA TE.
National Council for Accreditation of Teacher Education. (1987). Standards,
education units. Washington: NCATE.
13
PHYSICAL FACILITIES EVALUATION IN TEACHER

EDUCA TION PROGRAMS
Mary F. Berney
Educational facilities have come a long way from the days when the primary
demand was for some structure to protect the students and teachers from the
elements (Castaldi, 1987). The facility is now considered an integral part of the
instructional process, and evaluation of physical facilities is a component in the
accreditation processes established by most national, state, regional, and local
accrediting agencies. In its 1986 revision of standards, the National Council for
Accreditation of Teacher Education (NCATE) placed added emphasis on the
evaluation of resources. Standard V. B: Resources includes criteria for
compliance under Personnel Resources; Funding Resources; Physical Facilities;
and Library, Equipment, Materials, and Supplies. Similar standards can be found
in state policy guidelines for teacher education program approval as well as in
the guidelines for other agencies at the national, regional, and local levels.
This chapter is not intended to make physical facilities experts of the reader,
or to replace more detailed facilities evaluation guidelines that might exist for
individual institutions. It calls attention to steps that can be taken to incorporate
facilities evaluation into the routine planning and evaluation processes of an
educational unit. The reader is asked to keep in mind that facilities planning can
refer both to the construction, or anticipated construction, of a new facility and
to the continual upkeep of an existing one. Much of what is written on the
topic of facilities assumes new construction, but the principles can be applied to
the renovation of existing facilities as well. As with any form of evaluation, the
evaluation of physical facilities should not be relegated to crisis status, or
become a task which is conducted by tht unit only in its self-study for an
accreditation visit. If evaluation is to be used to direct improvement, evaluation
must occur routinely.
This chapter will address the evaluation of physical resources with an
emphasis on existing facilities. Financial and Library Resources are covered in
separate chapters, and although specific information about those topics will not
170
be included here, some overlap is to be expected. A checklist which includes

questions relative to all five NCATE criteria is provided at the end of the chapter.
Facilities Evaluation in Teacher Education Programs
Much of what is written about educational facilities evaluation refers to

elementary and secondary education facilities rather to than those in higher
education institutions, but there is much to be learned from studying this
literature. It is true, for instance, that "periodic evaluation of educational
facilities is essential to the task of the educational facility planner" whether the
facility be an elementary school or the university which offers a teacher
education program (Hill, 1984, p. 8). Hill also observed that
the data base for the facility planner must include valid, reliable,
objective information concerning the level of adequacy of existing
facilities to provide for (1) meeting current ... program requirements,
(2) effective and efficient utilization of space, (3) a structurally and
mechanically sound physical plant, and (4) a healthy and safe
environment for building occupants (p. 8).
Those four points resemble the criteria set forth by the various state and regional
agencies and approval boards which address teacher education programs.
Educational Specifications
Educational specifications, also called "building programs," are prepared to

aid the architect in understanding the intended function of the building that is to
be remodeled or built. Certain aspects of educational specifications can be used
as evaluation criteria for existing facilities, and it must be recalled that these are
more often found in references to elementary and secondary schools, but that they
are generally applicable to higher education facilities as well. Educational
specifications are written by the educator(s) to describe to the architect the
educational activities that the facility should accommodate. Glass (1986)
described a process model for the development of educational specifications for
elementary and secondary schools which would be applicable, with minor
modifications, to higher education facilities. For the technologically inclined,
Lilley (1985) described the process of using a microcomputer to generate
educational specifications.
Foldesy (1985) encouraged faculty involvement in the development of
educational specifications. References to the need for faculty involvement in an
evaluation effort can be found throughout this text. [See Rutherford, in
particular, regarding involving faculty in innovations.] Castaldi (1987, p. 142)
stressed that educational specifications must "serve as the link between the
educational program and school facilities." The specifications should not consist
of drawings, and should be free from jargon and vaguely worded statements.
171
Educational specifications must describe, as clearly and concisely as possible; (a)

the educational program; (b) numbers of students and the size and type of each
necessary space; and (c) environmental factors, such as lighting requirements and
climate control. More specifically, within these three areas, it will be necessary
to consider seven aspects of facilities planning which were pointed out by Wiley
(1987, p. 3):
o the educational activities to be housed in the facility,

o the people involved,
o furniture and equipment,
o how the program will be implemented,
o time of day/night areas will be used,
o preference of staff for teaching methods and materials, and
o curriculum emphasis.
Where possible, an evaluator or the evaluation committee should refer to the

existing educational specifications for the facility when conducting an
assessment. The specifications, and assistance in interpreting them, should be
available from the institution's physical plant.
The NCATE Criteria
For ease of discussion and to eliminate unnecessary repetition, Criterion 85

will be presented separately, as will Criterion 88. The remaining criteria will be
presented and discussed as a group.
85. Facilities are accessible to individuals with disabilities
"The design of the educational facility should facilitate and promote the
freedom of movement of all handicapped students and provide for their
participation in as many regular activities" as possible (Castaldi, 1987, p. 141).
We cannot discriminate against the physically handicapped by the design of our
facilities. Access to buildings, and within them, to restrooms, offices,
classrooms, and other instructional spaces must be provided. Beyond that,
however, constant monitoring is required to ensure that facilities remain safe and
accessible to everyone.
In addition to providing access to the facilities, teacher educators must
provide instruction designed to address the needs of handicapped learners. The
educational program must offer students who major in Special Education training
and research activities which are designed to develop and enhance their skills in
working with handicapped learners. Human development, psychology, and
motor skills laboratories are among the specialized facilities that should be
provided and evaluated.
172
88. Facilities accommodate technological needs in professional

education
Garten touches on this matter in his discussion of library facilities and it

overlaps with the broader area of specifications that will be addressed in the
discussion of Criteria 86, 87, and 89 as well. The most common problems
associated with this criterion are related to the electrical system within a building
or an entire facility. The lack of sufficient numbers of grounded electrical outlets
in offices, classrooms, laboratories, and other instructional spaces is a real
problem for units which are housed in older buildings. The absence or shortage
of outlets either places artificial limits on the number and placement of
microcomputers and other equipment or necessitates an extensive and expensive
rewiring. The undesirable "quick fix" is the use of power strips to extend the
number and location of outlets.
The increasing use of micro-teaching and related laboratory experiences
places another demand on the facility. Lighting, ventilation, sound insulation,
furnishings, and space for teachers and learners are all items to be considered as
these laboratories are planned for and constructed in new or existing facilities.
In addition to the taping areas described above, the unit will most likely
require its own instructional media center where students and staff can prepare
audio and/or visual aids, and preview films or tapes. Again, adequate ventilation
and sound-proofing are necessary conditions.
Programs such as music, art, special education, and others which require
special equipment and facilities must be supported with appropriate space and
resources. If a music education major is offered, for example, a listening library
should be available, as should practice rooms and the appropriate print and non-
print materials to meet the program's goals. The programs which are offered by
a teacher education program vary and the evaluation plans for each must vary
accordingly.
86. For each professional education program, faculty have

office, instructional, and other necessary space
87. The facilities are well maintained and functional
89. An institutional long-range plan for renovation/updating of

physical facilities (i.e., additions and replacements) has
been developed
These three criteria are addressed primarily through the educational

specifications and are closely inter-related. A functional, well maintained facility
will have adequate space. Long range plans will exist for the institution as a
whole and for the educational unit in particular. Once again, the best advice to
the evaluator is to consult the specifications and physical plant personnel to
173
detennine areas of strength and weakness. The institution's Facilities Planner

should be able to provide infonnation relative to Criterion 89.
The key to success relative to both Criterion 87 and Criterion 89 is
preventative maintenance. When small problems are not allowed to mushroom,
when facilities are constantly examined and necessary repairs made, the facility
will remain functional. Routine replacements, repairs, and upgrading of such
high-ticket items as roofs and cooling systems must be planned for, with funds
allocated to do the job(s) properly at the right time(s). It is unwise, and yet not
uncommon, to divert those funds into other projects because the money "is just
sitting there and we need a .... " When it comes to facilities maintenance, it is
imperative that the institution establish and adhere to long-range plans. This is
not to preclude the possibility of change; on the contrary, in a well-run system,
the plan itself will be monitored and updated as new infonnation dictates the need
to do so.
When plans are made by groups of stakeholders rather than by single
individuals the chances are greater that everyone's needs for space will be
addressed and that suitable compromises can be reached when the space has truly
run out. There must be office space to house the administrators, staff, and
faculty for each program offered by the educational unit, but it is important to
consider the differences between needs and wants, especially relative to offices
and personal pet projects.
When the unit's evaluation committee meets, its members must be prepared
to appraise the physical facilities of the unit honestly and, for accreditation
purposes, with the criteria described above in mind. The following guidelines
are compiled from a variety of sources, and are intended to serve as discussion-
starters or to fill in any gaps that may exist in the unit's own evaluation fonns.
GUIDELINES FOR PHYSICAL FACILITIES EVALUATION
These guidelines are by no means inclusive but they can serve as the
starting point for an evaluation. Many state standards include guidelines for
facilities evaluations in their standards for program approval and faculty at other
institutions might also be good sources of additional points for a checklist.
Garten provides a detailed set of evaluation guidelines which covers print
materials in libraries. In the interest of conserving space, that infonnation will
not be repeated in this chapter.
Site
1. Is the facility easily accessible to all students, faculty, staff, and visitors in
tenns of:
Location
Parking
Handicapped Access
2. Is the site free from traffic hazards?
174
3. Is the site free from distracting noises?

4. If the unit occupies space in more than one building, are all buildings
equally accessible to faculty and students?
5. Is the site landscaped attractively?
Energy Conservation
1. Are heating and cooling systems inspected regularly and kept in good repair?
2. Is the heating and cooling system controlled by an automatic timer?
3. Are windows properly sealed?
4. Are there curtains and blinds or shades for all windows?
5 Do the exterior doors fit properly?
6. Are there leaks or drips in the plumbing?
Building and Equipment Safety
1. Are routine fire and other safety inspections conducted?

2. Is the wiring safe and adequate for instructional and administrative needs?
3. Is the roof sound?
4. Is the foundation sound?
5. Is there any peeling paint?
6. Is there any loose flooring?
7 Are power strips being used where there are too few outlets?
8. Are telephone and electrical cords protected and kept out of the traffic
pattern?
9. Are exits posted and lighted?
10. Do outer doors open outward?
11. Are outer doors kept unlocked during the hours the building is expected to
be open?
12. Are equipment carts sturdy and well-balanced?
13. What provisions are there for storing and disposing of volatile materials?
14. What provisions exist for waste removal?
Instructional Spaces
1. Are classrooms and laboratories furnished appropriately for their intended

function?
2. Are lighting and temperature controls easily adjusted for comfort?
3. Are the acoustics appropriate for the room's intended use?
4. Is there sound-insulation between classrooms?
5. Is the furniture comfortable?
6. Is there adequate display space in each classroom?
7. Are chalk and chalkless boards conveniently placed in each room?
8. Are the boards in computer laboratories chalkless?
9. Is there adequate space for demonstrations or other special activities?
175
10. Are video-taping rooms available?
Audio-Visual/Microcomputer Center
1. Is there such a center for the educational unit?

2. Does the center house computers, video equipment, and other technological
aids to teaching and learning?
3. Is the number of microcomputers adequate to support the number of courses
requiring computer use?
4. Is there a reasonable ratio of printers and plotters to computers?
5. Is there a variety of administrative and instructional software for previewing
and using with classes?
6. Are the major computer brands, or those which are used in the schools
where graduates expect to be placed, represented in the center?
7. Is the personnel able to provided assistance in using equipment and
software?
Handicapped Access
1. Are all buildings barrier-free? This includes libraries, the student center,
classrooms, and administrative offices.
2. Are water fountains accessible to wheelchair-bound persons?
3. Are access ramps and elevators provided in parking areas as well as
buildings?
4. Is the furniture and equipment arranged so as not to create barriers?
5. Is parking made and kept available for handicapped persons?
6. Are restrooms designed to provide access?
Administrative Areas
1 Are these areas accessible to everyone?

2. Are there private areas for conferences and meetings?
3. Is there adequate computer access?
Flexibility for Expansion
1. How readily can the facility be expanded or upgraded?

2. Is the wiring accessible?
3. Is there physical space for building expansion or for the construction of
additional buildings?
4. Can interior spaces be adapted without major reconstruction?
176
Ease of Maintenance/Housekeeping
1. Are the building exteriors constructed of materials that withstand weather

and use?
2 Are floors and walls easily cleaned and refinished?
3. Are routine inspections conducted and potential problems corrected?
4. What budgetary provisions exist for replacing roofs?
5. What provisions are made for routine replacement of worn flooring, stair
treads, etc.?
Restroorns
1. Are restrooms located on each floor and/or wing?

2. Are they accessible by handicapped persons?
3. Are they kept clean?
4. Are they well-ventilated and well-lighted?
5. Are they kept supplied with soap, towels, and toilet tissue?
Public Areas
1. Are directories within each building prominently displayed and kept current?
2. Are public areas well-lighted, well-ventilated, and pleasant in appearance?
3. Do distinct non-smoking areas exist and are the regulations enforced?
REFERENCES
Castaldi, B. (1987). Educational facilities: Planning, modernization, and

management, (3rd. ed.). Boston: Allyn and Bacon, Inc.
Foldesy, G. (1985, October). A paradigm for teacher involvement in the
development of educational specifications. Paper presented at the Northern
Rocky Mountain Educational Research Association. (ERIC Document
Reproduction Services No. ED 267 021)
Glass, T. E. (1986). Educational specifications: A blueprint for the future
program. CEFP Journal, 24(1), 4-13.
Hill, J. C. (1984). Performance-based evaluation of physical facilities. CEFP
Journal, 22(2), 8-12.
Lilley, H. E. (1985). Computerizing educational specifications. CEFP
Journal, 23(1), 14-16.
National Council for Accreditation of Teacher Education (1987). Standards,
education units. Washington, D. C.: Author.
Schlotfeldt, J. (1986). Telecommunications and computer requirements in the
development of educational specifications. CEFP Journal, 24(1), 21-23.
Wiley, A. L. (1982). Building design and the architect: An instructional
module. (ERIC Document Reproduction Services No. ED 252191)
14
EV ALUA TING FINANCIAL RESOURCES FOR TEACHER

EDUCATION PROGRAMS
Robert L. Saunders
Memphis State University
An interesting paradox exists relative to the financial resources of schools,

colleges, and departments of education (SCDEs). Anytime two or more teacher
educators, particularly department heads or deans, get together, the discussion
invariably turns to the subject of adequate financial support, both for the existing
program(s) and for planned or necessary future programs. That SCDEs are
underfunded is a widely held belief, supported often in meetings, both formal and
informal, and in the literature. Despite this widely held and deeply felt belief
among teacher educators, there is little provision for evaluating the financial
adequacy of SCDEs, either in terms of institutional processes and procedures or
in terms of state, regional, and national accreditation processes. There are several
reasons for this paradoxical condition. Many of the approximately 1200
institutions which offer teacher education programs spend too little time on
developing short, medium and long range goals and translating them to action
plans, and on the identification of financial resources needed to attain goals.
Even when such planning programs are present, SCDEs often find themselves
with insufficient leverage to influence the allocation of resources and
consequently fare poorly in the face of powerful leverages by other professional
schools within the institution. Similarly, in most states the processes and
policies for program approval virtually ignore the financial resources matter,
except occasionally when oblique references are made to "adequate faculty,"
"sufficient clinical experiences," and other kinds of subjective and high inference
measures and requirements. The situation is similar but in some respects even
worse concerning regional accreditation requirements which are institutionally
focused and provide SCDEs little opportunity to document financial deficiencies
and little opportunity to correct them through the accreditation process. One
rarely hears of an institution experiencing problems with a regional accrediting
association because of financial inadequacies in the school, college or department
of education.
178
The situation vis-a-vis national accreditation is not much better, even

though the accreditation focus is on programs. The National Council for the
Accreditation of Teacher Education (NCATE) has in its new redesign provisions
for evaluating the financial resources of applicant SCDEs, but the orientation is
largely qualitative rather than quantitative. Thus the "eye of the beholder"
syndrome is inevitable, despite NCATE's latest efforts to minimize the problem
by use of a board of examiners, intensive training sessions for board members,
and worksheets and checklists. Still the subjective nature of the evaluation
remains, as evidenced by the presence of such words as "adequate," "necessary to
carry out the mission of the program" (without any required effort to evaluate the
quality of the mission), "functional facilities," "necessary supplies," and so
forth. The paradox bears repeating: On the one hand there is strong belief among
teacher educators that SCDEs are seriously underfunded. On the other hand, the
profession has been unsuccessful in devising ways through institutional
procedures, state approval systems, or regional and national accrediting
mechanisms for documenting the condition and drawing attention to it by
decision and policy makers. The purpose of this chapter is to help improve that
condition by (a) documenting, more fully than done above, the condition of
underfunding of SCDEs, (b) explicating and summarizing from regional and
national accrediting associations the policies and provisions for evaluating the
financial adequacy of SCDEs, (c) analyzing the literature of research needed to
operate quality programs in teacher education, and (d) suggesting instrumentation
which will enable teacher education administrators and others to evaluate the
financial adequacy of teacher education programs in a manner consistent with the
Accreditation Plus Model being developed at Tennessee Technological University
through its Center for Teacher Education Evaluation.
THE CONDITION OF FINANCIAL RESOURCES FOR

TEACHER EDUCATION
The literature on teacher education is replete with research findings and

statements from researchers and knowledgeable authorities to the effect that
teacher education programs historically have been inadequately funded. This
condition is undoubtedly linked to the problem of the low status accorded teacher
education programs, historically and now.
The problem of inadequate funding dates to the inception of normal schools
which were subsequently transformed into state teachers colleges. As state
teachers colleges became state colleges, teacher education became a component of
a larger entity and, often, soon lost ground as the state college added programs
(business, engineering, law, etc.) with greater appeal and which were capable of
exacting more leverage in decisions about the allocation of funds within
institutions. The funding situation worsened when many state colleges took on
"research university" missions and developed a full range of professional
programs, including graduate programs leading to the doctorate. Teacher
education programs which began as components of a university (chiefly
programs for secondary teachers and administrators) fared no better than their
counterparts which began as normal schools.
179
Summarizing the reform efforts in teacher education for the past half
century, Bush (1987) asserts that, historically, teacher education has been
economically impoverished, receiving much less funding than parent institutions
allocate for other professional fields. Bush blames this funding inadequacy for
the fact that past efforts at reform have not resulted in big changes.
Lanier and Little (1986) cite the findings of several researchers which
support the conclusion that "the record of financial support for teacher education
is low" (p. 556). Institutional analyses conducted by Clark and Marker (1975)
support the earlier point that the low funding of teacher education is attributable
in part to the prestige factor, noting that "teacher training is a low prestige, low
cost venture in almost all institutions of higher education " p. 57).
A consistent pattern of apparent underfunding of teacher education was
found by Peseau and Orr (1980). The title of their research findings, "The
Outrageous Underfunding of Teacher Education," has been repeated many times
in discussions about teacher education. Virtually all the findings from a
longitudinal study of teacher education funding in 63 leading institutions in 37
states support the premise that teacher education programs are indeed poorly
supported. For the year 1979-80, for example Peseau (1982) reported that
SCDEs received "only about 65% as much as for a public school student and
only 50% as much as the average cost per undergraduate student in all university
disciplines" (p.14).
One reason, among others, for the underfunding of teacher education,
according to Peseau and Orr (1980), is found partially in state funding formulas
which typically place teacher education with undergraduate programs of low
complexity. Exacerbating this problem are two related conditions: (a) In many
states, funds are allocated on historical patterns whict~ were built on traditional
assumptions that were unfavorable to teacher educl:ltion (Temple & Riggs,
1978); and (b) When university administrators reallocate funds derived from state
formulas they give less to teacher education and more to programs that, in their
judgment, deserve or need a higher level of support (peseau & Orr, 1980). The
lack of leverage which SCDEs have in influencing the institutional reallocation
of state-derived funds was noted by an External Review Committee for the
Improvement of Teacher Education in Georgia (1986).
Nutter (1986) described three negative funding situations found in teacher
education which contribute to the general evidence concerning the depth and
scope of current underfunding. The first situation is Starving Institutions.
Some, perhaps many, of the country's approximately 1200 institutions with
teacher education programs have declined from historically mediocre institutions
into fiscal (and intellectual) poverty, unable to support adequately any of their
programs. The second situation is seen at Research-Oriented Institutions. In
some large, well-supported, research-oriented institutions, teacher education
programs tend to be ignored at the expense of more "academically respectable"
endeavors. Finally, the third situation is Institutions. In some institutions, the
attitudes of central administration ranges from lukewarm to hostile. Some
administrators regard teacher education as an embarrassment, a peripheral
activity, perhaps a necessary nuisance and certainly not something on which to
spend much money. As pointed out by Monahan, Denemark, Egbert, Giles and
180
McCarty (1984), inhospitable institutions can be found even among those with
strong funding bases and with well-developed and highly respected programs in
other fields.While citing the needs for stronger financial support by state and
national governmental agencies, and from philanthropic foundations, Howsam,
Corrigan, Denemark and Nash (1985) use strong language in identifying funding
inadequacies at the institutional level:
Higher education has exploited teacher education for its own

interests, while granting it low status, misplacing it
organizationally and programmatically, and seriously underfunding
it. Even the vigorous development effort of the last 10 years has
not produced much change; teacher education still sits on the
academic street comer, tin in hand, begging for the capital to
market its product (p. 57).
The report of the National Commission for Excellence in Teacher

Education (1985) asserts that "a not-so-benign neglect has contributed to the
problems in teacher education" (p. 23). The report cites three reasons why this
basis for allocating resources to teacher education is "simply inadequate" (p. 21).
The three reasons are given later in the section titled" A Rationale for Evaluating
Beyond Accreditation Standards. "The above citations represent only a sampling
of those available; more are included in the references. However, the citations
commented on above clearly support the assertions that teacher education is not
sufficiently funded to achieve its mission as a professional program and that its
funding level is inequital,le relative to other professional programs in colleges
and universities.
CURRENT APPROVAL AND ACCREDITATION PROVISIONS

FOR EVALUATING THE FINANCIAL RESOURCES FOR
TEACHER EDUCATION UNITS
Current program approval procedures and standards at the state level as well
as regional and national accreditation standards are of insufficient help to teacher
education administrators and others trying to evaluate the adequacy of financial
resources given teacher education programs. This shortcoming lends credence to
the importance of the Accreditation Plus Model being developed by the Center
for Teacher Education Evaluation at Tennessee Technological University.
In this section, a summary analysis is given of how state, regional and
national approval and accreditation procedures address the matter of financial
resources.
Program Approval Procedures at the State Level
It was beyond the scope of this chapter to identify and analyze the program
approval policies and procedures for the various states. Some generalizations are
possible, however, based on the general available knowledge about such
181
programs which is disseminated in professional meetings, discussion with

colleagues, education articles, and so forth.
Some states, Ohio and New York, to name two, have standards that address
directly and forthrightly the matter of financial resources. The vast majority of
the state approval mechanisms address primarily curricular and programmatic
matters, student teaching and clinical experiences, library holdings and media
capability, and faculty. All of these components have implications for financial
resources, or course, but they are indirect and subjective. An institution cited for
an inadequate number of faculty (rather unlikely in most states, given the
absence of precise standards in this regard) can use such information for
evaluative purposes. Similarly, relative strengths and weaknesses in the financial
adequacy of a teacher education program can be ascertained occasionally by
carefully analyzing reports of site visits. But such efforts are likely to be
sporadic, fragmented, and lacking in comprehensiveness.
Regional Accrediting Associations
The six regional accreditation agencies (Middle States, New England, North
Central, Northwest, Southern and Western Associations of Colleges and
Schools) have clearly stated standards which address the financial resources of
institutions; however, the standards are institutionally focused and these are of
limited used to those seeking to evaluate the financial resources for teacher
education programs.
It is true, of course, that teacher education programs benefit when parent
institutions are in compliance with the standards for resources. Like other
programs within a given institution, a teacher education program benefits from
being in an institution that has "a history of financial stability" and where
"financial planning and budgeting are ongoing, realistic and based upon
institutional educational objectives" (Western Association of Colleges and
Schools, 1988, p. 82). Moreover, in addition to being institutionally oriented,
the standards emphasize processes and procedures (e.g., budget control,
purchasing and inventory control, refund policies, cashiering, etc.).
It is safe to say, then, that standards used by the regional accrediting
associations were not designed and are not intended to be used as instruments for
evaluating the financial resources of teacher education programs--or any other
specific program area. An exception to this generalization might be when a
program within an institution is so poorly funded that this condition prevents
the institution as a whole from meeting the standards of overall financial
sufficiency (generally found in the preamble).
National Accreditation
National accreditation is the mechanism by which evaluations are made of

the professional education unit responsible for the preparation of K-12
professional educators within institutions. The National Council for the
Accreditation of Teacher Education (NCA TE) is the only accreditation agency
182
approved for this purpose by the Council on Postsecondary Accreditation

(COPA) and the U.S. Department of Education.
NCATE's standards, policies and procedures are more useful than state
approval programs and regional accreditation in evaluating fmancial resources for
teacher education programs, primarily because of two factors. First, NCATE is
program specific, whereas regional accrediting agencies evaluate institutions as
whole entities. Second, the standards on resources (V. B.) contain compliance
criteria that in two instances are quantitative in nature and several others are
somewhat usable as "measuring" devices. Criterion 77, for example, requires at
least three full-time faculty who have earned doctorates in the respective field of
specialization for each advanced degree program. An additional quantitative
measure is Criterion 79, which limits to 18 the number of full-time equivalent
students to one full-time equivalent faculty member in the supervision of
practicum experiences.
NCATE's standard on resources covers four areas (personnel; funding;
physical facilities; and library, equipment, materials, and supplies), and includes
18 criteria for compliance. Even so, there are several aspects of financial
resources needed by SCDEs to achieve their mission that are addressed
inferentially and obliquely by the NCA TE standards and criteria, and, in some
instances, not at all. Again, this reinforces the need for the Accreditation Plus
Model, i.e., the need for ways to evaluate the financial resources for teacher
education programs beyond accreditation standards. Needed are more specificity,
measuring devices, and instruments beyond those available in state, regional, or
national accrediting mechanisms. An attempt to provide these is made in the
following section.
A RATIONALE FOR EVALUATING BEYOND

ACCREDITATION STANDARDS
The standards and criteria of accreditation agencies, both regional and

national, are insufficient for use as a complete and comprehensive basis for
evaluating the financial resources of teacher education programs. Providing a
base for a more complete and comprehensive evaluation, along with a rationale,
is the purpose of this section.
Funding and Budgeting Policies, Rationale and Procedures
Realistic evaluation of financial resources for teacher education must

address aspects of funding and budgeting that typically go unaddressed by state
approval systems as well as by regional and national accrediting agencies.
The fundamental problem in this area of concern is the one referred to
earlier: the erroneous assumption that teacher education is an inexpensive
program not needing funding amounts equal to those granted other professional
program areas. This problem often exists both at the state level (Temple and
Riggs, 1978) and at the institutional level (Orr and Peseau, 1979). Overcoming
this problem at either level is difficult, perhaps impossible when overall
183
resources are limited and when a long- standing historical pattern of inequity
exists.
One way, perhaps the best way, to approach the problem would be to
reconceptualize teacher education as a clinically based instructional mode (like
medicine, veterinary medicine, nursing, and clinical psychology) and remove it
from the lecture-discussion mode (like history and English). Smith (1980)
makes this point forthrightly: "Without it [a new rationale and formula for
financial support], a clinical program is impossible, for clinical work
accompanying courses in pedagogy requires at most a ratio of 10 students per
instructor" (p. Ill).
A reconceptualization of teacher education is unlikely if attempted in a
vacuum, however. For it to occur, as Clark and Marker (1975) wrote, teacher
education must in fact remove itself from the "classic mold of undergraduate
lecture courses where students end up being taught to teach by being told how to
teach" (p. 57). Kerr (1983) extends this notion by suggesting that this
fundamental change is not likely to occur so long as teacher education remains
an undergraduate program.
Encouraging in the reconceptualization of teacher education along the line
of a clinically-based instructional mode is the cost model for clinical teacher
education developed by a legislative task force in Florida (peseau, Backman &
Fry, 1987). The unique activities of clinical preparation were identified and
applied to actual cost data gathered from nine teacher education programs.
Recommending that teacher education programs should be about two-thirds
clinical (and then translating that to one-third clinical instruction and two-thirds
clinical practice), a budget comparison was made on the basis of 33% classroom
instruction, 22% clinical instruction, and 44% clinical practice. After applying
weights of 1.5, 2.0, and 2.5, these calculations resulted in "an increase in
indirect program costs of 106% for the nine undergraduate programs and an
overall (undergraduate and graduate) forecast budget increase of 47%" (p. 33).
Support for the reconceptualization of teacher education as a clinically oriented
professional model was provided by the report of the National Commission for
Excellence in Teacher Education (1985) referred to earlier. "At least three
factors," the report states,
cause teacher education programs to require special funding beyond

that for traditional lecture courses: (1) At each step of laboratory
and field experience, a teacher candidate must have individual
supervision and guidance. (2) Specialized equipment is needed, such
as video machines for analysis and critique of a candidate's
performance in campus clinical settings and in schools, micro-
computers for instructing students how to use technology in both
simple and complex learning tasks, and laboratories for producing
and using slides, transparencies, and other teacher aids. (3) The
faculty required in quality teacher education programs, faculty who
have skills and experience in teaching, research and supervision, are
in high demand in other positions. Incentives are needed to attract
and keep such faculty in teacher education. For these reasons,
184
supporting teacher education on the same basis as liberal arts

lecture courses is simply inadequate (p. 21).
Another way to improve the rationale by which funding decisions are made
would be the use of the consent of peer institution comparisons (peseau, 1988).
Dependent upon the availability and analysis of quantitative data on resources and
productivity variables, this concept can help teacher education administrators by
providing less biased justification for needed additional resources, that is,
comparative quantitative information on resources and productivity of similar
institutions. Peseau (1988) notes that the peer identification process does not
preclude comparisons with other institutions of perceived better quality. The
concept of peer institution comparisons is not unlike the concept of peer
program comparisons, suggested several places throughout this chapter.
Correcting the funding rationale would enable SCDEs to engage in budgeting
processes with more confidence, optimism, and success than is currently the
case.
Personnel Resources
SCDEs must have faculty with sufficient expertise and 'in sufficient
numbers to enable teacher education units to achieve their tripartite mission of
instruction, research and service. The tripartite, three-fold mission is
fundamental, the raison d'etre of a professional school as contrasted with an
academic school, a research unit, or a service agency as separate and disjointed
entities.
Funding should be in accordance with the gestalt of this three-fold mission.
Funding most often is based on the instructional function alone, resulting in
either severe overloading of faculty or diminished roles in research and service
(Orr and Peseau, 1979). Central administrators tend to require teacher education
units to generate their own research dollars through externally-funded grants.
This practice often leads faculty into research activities of questionable relevance
and utility to K-12 schools, creating a public relations problem. Research that is
highly relevant to schools, as noted by Howsam, et al. (1985), is not unlike
the complementary research and development functions of agriculture schools and
extension programs. Faculty need time to engage in research and development
activities in collaboration with local education agencies. They also need
financial support, along with assigned work time, for the research activities,
even when external funds are unavailable.
The professional service role is endemic to teacher education units. Failure
to provide professional service to schools and other appropriate agencies is a sure
way to earn the labels of aloofness, unresponsiveness, and indifference, the
consequence of which can have a disastrous impact on recruiting programs,
settings for field-based experiences for students, and other collaborative efforts.
Howsam, et al. (1985) provide an instructive and useful way to
conceptualize both the research and professional service roles - and their
interdependence - of teacher education programs. Noting that the most
fundamental of the purposes of a university is "the pursuit of valid knowledge,"
185
and that "all the basic activities of a university are directly concerned with the
search for valid knowledge, with its presentation and dissemination, and with its
use (emphasis added)" (p. 57), the authors suggest three continua as ways of
conceptualizing the search for new knowledge and the development and use of
that knowledge. The three continua are shown in Figure 1.
Search Application Use

FUNCfIONS ON THE VALID KNOWLEOOE CONTINUUM
Search Application Use

Researcher Developer Practitioner
PERSONS ON THE VAUD KNOWLEDGE CONTINUUM
Search- Applications- Use-

Researcher Developer Practitioner
INTERESTS ON THE VAUD KNOWLEDGE CONTINUUM
Figure 1. Valid Knowledge Continuum

Note. From Educating A Profession, Second Edition (pp. 57 -58)
by R. Howsam, D. Corrigan, G. Denemark & R. Nash, 1985, Washington,
DC: American Association of Colleges for Teacher Education.
The implications of these schemata are that SCDEs must, in addition to

their instructional role, (a) have faculty with the expertise needed to perform
effectively in the application/developer role and (b) have this responsibility
accepted as part or all of the assigned work loads of certain designated faculty.
Performing the application/developer role would enable the SCDE to meet its
responsibilities in both applied research and professional service.
Salaries for faculty in SCDEs must be sufficient to attract and retain the
expertise needed for the unit to achieve its three-fold mission. Faculty salaries
must be competitive with those paid for comparable expertise and
responsibilities in other education organizations, not just K-12 schools, and
186
other professional program areas within the institutions. SCDEs are often
penalized in this regard. Central administrators are quick to apply the concept of
"market sensitivity" in negotiating salaries for faculty in such areas as business,
engineering, law, and medicine. In applying the concept to teacher education,
however, they tend to think of classroom teachers as the basis of comparison,
and often even that comparison is misread. It is not uncommon to find teacher
education faculty in curriculum and instruction departments, for example, with
salaries less than what they would have if they transferred to K-12 classroom
positions. Faculty in departments of administration and supervision, as a further
example, often receive salaries lower than those in positions being held by their
students (principals, supervisors, superintendents, etc.). If the concept of market
sensitivity is valid in program areas such as law, medicine, engineering and
business, it is also valid in teacher education.
Salaries of teacher educators should be commensurate with and equitable
with salaries in other program areas. To set them lower is to give credence to the
widespread belief that state funding authorities and central administrators see less
value in teacher education programs and are willing to relegate these programs to
second class, low prestige status. Faculty should be sufficient in number and
expertise to perform the clinical, laboratory experience programs of the teacher
education unit. As noted earlier, Smith (1980, p. 111) recommends a ratio of 10
students per instructor in the clinical program. The same ratio is required in the
clinical supervision of nursing interns (Tennessee Board of Nursing, 1988).
NCATE (1987) requires a ratio of one full-time faculty member for 18 full-time
equivalent students in practicum experiences. A ratio of five students as the
equivalent of a three-hour course is recommended by the Council for the
Accreditation of Counseling and Related Educational Programs (1988).
Resources should be sufficient to preclude the need for faculty having these
responsibilities assigned because their scheduled courses failed to make.
Sufficient faculty qualified and available for supervising clinical and laboratory
experience would obviate the need for graduate assistants and part-time faculty
(often employed at the eleventh hour) to provide this instruction in a program
component which students consistently say is the most important of all.
Programs for faculty development should be operated with sufficient funds
to permit them to be ongoing, viable and effective. It is not enough to say that
such programs should be "at least at the level of other units in the institution"
(Standard V. B: 80, NCATE, 1987). Meeting this standard would be
meaningless if the institution had a weak or non-existent program for faculty
development, by no means a far-fetched possibility given the state of the art in
this matter in higher education generally. More meaningful standards for faculty
development programs would include the presence of official plans, procedures
policies regarding faculty development leaves, budgeted funds for the program,
the dissemination of information regarding eligibility, application procedures,
kinds of developmental leaves possible and preferred, and clearly stated
expectations for accountability.
Funds for faculty travel beyond the travel involved specifically in faculty
development leave programs should be available and accessible. Funding should
be sufficient to permit each faculty member to attend professional meetings at
187
the state and national level, annually and biannually respectively, as a minimum.
Cost sharing between the institution and the faculty member should be
encouraged. The judicious use of discretionary funds or cost sharing
arrangements and for exceptionally productive faculty would be advisable.
SCDEs should be funded sufficiently to enable them to employ
practitioners from the field as part-time, adjunct faculty. NCA TE properly
prohibits the overuse of part-time faculty and graduate assistants teaching in the
professional program (Standard V. B: 81). The other side of that coin, however,
is that SCDEs increasingly need clinical, field-based, practitioners to play
essential roles in the education of teachers (External Advisory Committee on
Teacher Education and Certification in Mississippi, 1989). Greater use of
practitioners in establishing practicum and internship programs and supervising
students in such experiences seems to be gaining force. To expect such
programs to function with the meager (and professionally insulting) stipends
characteristic of current student teaching programs is to be naive and short-
sighted. Needed are bolder, more imaginative (and more expensive), approaches
such as the one proposed by Goodlad (1983):
it will be necessary, I think, to try one more time to provide

monetary incentives, in this case for a few promising networks
each comprising several key demonstration schools and a key
university (p. 315).
Goodlad predicts that without efforts of this type and the substantial funds
necessary to mount such initiatives,
future attempts to improve teacher education--and, subsequently,

our schools--are doomed to repeat the puny, inadequate efforts of
the past (p. 318).
Reports of the Carnegie Forum on Education and the Economy (1986) and
the Holmes Group (1986) recommended significantly different approaches to the
way students gain clinical and internship experience and the increased use of
"proven teachers" or mentors. These recommendations, as Goodlad's, would
require substantial increases in funds allocated teacher education programs to
cover, among other costs, the employment of mentoring teachers and increased
time allocation of university faculty.
Some institutions and some state agencies are moving toward the
employment of K-12 educators who are assigned responsibility for specialized
instructional methodology, incorporating this highly criticized instructional
component into extended and intensified internships. Should this movement run
full course, significant amounts of new funds would be necessary, at least until
reductions in faculty size occur as a result of the shifted responsibility.
No aspect of teacher education is more seriously underfunded than the area
of support personnel. SCDEs with adequate numbers of secretarial and clerical
personnel, technicians, artists, computer programmers, and media specialists are
clearly exceptions. It is not uncommon to find as many as 12 or 15 facuIty
188
being supported by a single secretary, faculty members having to produce their

own media presentations (often of low quality), no specialized support personnel
in the increasingly important world of computers and video machines, and so
forth. Attempting to fill these voids with untrained, transient work-study
students and graduate assistants (even GAs are unavailable in the many
institutions without graduate programs) is a exercise in futility. A cursory
examination of support personnel available in education programs in non-school
organizations (e.g., training programs in business and industry, the defense
department, and in instructional television programs) reveals a level of support
which is equally vital to teacher education. A realistic assessment of the
adequacy of financial resources must take these needs into account
Physical Facilities
NCATE has five compliance criteria for use by vlsltmg teams in

evaluating this aspect of financial resources available to an applying SCDE.
The five criteria are relevant and clearly stated. Collectively they should enable a
visiting team to evaluate rather successfully an institution's physical facilities
insofar as the NCATE standards are concerned. It must be said, however, that
the criteria are minimal in nature and require the visiting team to develop their
own specifics and substandards. For example, Criterion 87 states that "The
faciUties are well-maintained and functional," and Criterion 86 requires the
presence of "office space, instructional space, and other space necessary to carry
out the unit's mission." The lack of specific expectations in these criteria may
prove troublesome to NCATE examiners attempting to evaluate the adequacy of
a teacher education unit's physical facilities.
More specificity is needed, as are comparative standards and judgments.
For example, classroom, laboratory, and office space should be comparable in
quality, quantity, and condition of maintenance to other program units within the
institution. Ideally, the teacher education program should be housed in a single
facility, and in no case in more than two or three sites. Programs can function
with faculty fragmented and scattered across campus in six or eight separate
buildings, but such conditions make it difficult to carryon the important
functions of collegiality, student access to faculty for advising, discussion, etc.
Standards are needed which reflect these functions of facilities. [Additional
information on this topic can be found in Berney's chapter on Facilities
Evaluation.]
Library, Equipment, Materials, and Supplies
Again, the criteria used by NCA1E do not go far enough and thus require
examiners to furnish the specifics, to flush out the nuances, and tease out
dysfunctional and restricting circumstances and conditions. An Accreditation
Plus Model should contain standards that cover these nuances, dysfunctions and
restrictions. No longer can teacher education be regarded as an inexpensive
program, not requiring complex and expensive technology, laboratory
equipment, and abundant supplies. The truly effective and sophisticated
189
programs in teacher education do require expensive technology, such as micro-

teaching capability, multimedia rooms, closed circuit television capability,
satellite reception hardware, desk-top publishing and other media production
capabilities, portable television cameras, and so forth. Unfortunately, the
absence of these capabilities when funding formulas were developed signaled a
lack of need for them and mitigated subsequent efforts to include them in future
funding decision.
It has been difficult to establish branch libraries for teacher education
comparable to those found routinely in engineering schools, law school, and
medical schools, to cite only three examples. In their place, teacher education
units have settled for curriculum laboratories; many subsequently became media
or resource centers. Most such centers fall short of what is needed to support
faculty and students in a truly viable professional school.
The evaluation of library holdings for teacher education should include
holdings in cognate, collateral and teaching field content knowledge. Too often
these fields are ignored, yet they comprise the greater portion of the preparation
program for students.
Students preparing to teach need opportunity to learn about school
textbooks and other teaching and learning materials relevant to their future
assignments. Main libraries usually are reluctant to house such materials,
reinforcing the need for branch libraries and media centers in the teacher education
unit. Funds should be available for the procurement of such materials, for their
being made accessible to students, and updated periodically. [For further
information, see Garten on the Evaluation of Library Facilities.]
A CHECKLIST FOR EVALUATING FINANCIAL RESOURCES
This section sets forth a set of criteria which can be used in the evaluation
of financial resources for teacher education programs. The criteria were extracted
from citations in the above sections. In some instances the criteria were
developed from 32 years' personal experiences in evaluating financial resources as
an administrator of teacher education programs.
The criteria go beyond those found in state approval systems and in
regional and national accrediting agencies. In some instances, however, the
criteria have embedded in them the standards used in state, regional and national
accrediting/approval systems. For each statement the user can indicate the degree
to which each statement is characteristic of the teacher education program being
evaluated, by circling the appropriate number from "1" (the statement is not at all
true) to "4" (the statement is always true).
Funding Procedures, Policies and Rationale
1234 Decision making about fund allocation reflects recognition that

teacher education has a threefold mission (instruction, research,
and professional service).
190
1234 Decision making about fund allocation reflects recognition that

in teacher education the instructional component requires
clinical, practicum, and internship experiences for students.
1234 Funds allocated to the teacher education unit are equitable with
those allocated to other program areas within the institution,
based on whatever unit measure (size, scope and depth of
program, etc.) is used in the allocation process.
Personnel
1234 There are sufficient budgeted line positions for faculty to enable
the unit to achieve the threefold mission of instruction,
research, and professional service.
1234 Policies governing faculty workloads permit faculty

assignments to be made in each of the three areas of instruction,
research, and professional service.
1234 Faculty workloads (courses taught, number of preparations,

number of students taught, etc.) in teacher education are
equitable with those in other professional program areas, e.g.,
law, engineering, business, etc.
1234 Faculty are sufficient in number and expertise to staff the

clinical, field-based experience programs for students.
Assignments in the supervision of student-teaching interns do
not exceed the ratio of six students as the equivalent of a three
hour course or 18 students for a full-time equivalent faculty
member. The number of graduate students being supervised in
practica does not exceed five for the equivalent of a three hour
course.
1234 Salaries of faculty in teacher education are "market sensitive,"

that is, competitive with and equitable to salaries they would
command in other educational organizations.
1234 Salaries of faculty in the teacher education programs are

equitable to and comparable with salaries of faculty in other
professional programs within the institution and in other
similar institutions (with allowances for level of preparation,
years of experience, and professional rank).
1234 Budgeted positions are available for secretarial and clerical

support (beyond the work-student level), and are adequate in
number to provide at least one full-time secretary for each five
full time faculty.
191
1234 Budgeted positions for technical support personnel (media

technicians, computer programmers, artists, etc.) are available
in sufficient number and types to enable the unit to make
maximum use of state-of-the-art technology. The unit has at
least one media technician, one computer programmer, one
media designer, and one library/materials coordinator.
1234 Budgeted funds are available in sufficient amounts to enable the

unit to employ work-study students and graduate assistants
sufficient in quality and quantity to provide support services
necessary for the program and commensurate with their
qualifications. Graduate departments have a ratio of one GA
per faculty (exclusive of externally funded projects) plus two for
department-wide responsibilities.
1234 Employment policies permit and funds are available for hiring
qualified adjunct, part-time faculty to augment the regular
faculty, exceeding the traditional arrangements for evening and
weekend courses by using such persons as clinical professors,
supervisors of interns and as mentors.
1234 Funds are available and institutional policies permit the unit to
operate a structured and on-going program of faculty
development. Included are provisions for professional leave and
sabbaticals sufficient to enable all faculty to be eligible for such
a program at least once during each seven-year period. Funds
are sufficient to preclude colleagues having to take on overloads
while a facuIty member is on professional development leave.
1234 Funds and policies permit faculty to attend and participate in

professional meetings regularly, being able to attend a state
meeting at least once a year and a national meeting at least
every other year.
1234 Discretionary funds are available and can be used to fund faculty
travel beyond the above level to accommodate faculty of
exceptional productivity and prestige.
Physical Facilities
1234 Faculty and staff in the teacher education unit have office space,
instructional space and other space that enables the unit to
achieve its goals and objectives at a high level of attainment and
are, also, comparable in size, maintenance and functionality to
192
spaces available to other professional program units within the

university.
1234 The teacher education unit is housed in a single complex or in

sufficiently small number of complexes so as to be conducive
to collegiality and student-faculty accessibility and interaction.
1234 Facilities support the technological operation of the unit and

make possible the use of state-of-the-art educational technology
and media. Functional space is available in the unit for micro-
teaching, media production, multi-media presentations,
mediated instruction and other uses of educational technology.
Library, Equipment, Materials and Supplies
1234 Library holdings for teacher education programs are of adequate

scope, breadth, depth and currency to support the program at a
high degree of success and in these regards are comparable to the
holdings provided other professional program areas within the
institution.
1234 A branch library and/or a learning resource center which

contains a branch library available in the teacher education
program and is funded, staffed and otherwise supported
comparable to branch libraries and special collection centers in
other professional program areas.
1234 Both the branch library and the learning resource center are
staffed sufficiently to enable students involved in late-evening
and weekend courses to have full access.
1234 There are strong holdings in the cognate, collateral, and

teaching field content knowledge needed by students preparing
to be teachers. Education faculty participate in the selection of
these library holdings.
1234 K-12 textbooks and other teaching and learning materials

relevant to students' future roles are available and accessible,
including evenings and weekends.
1234 The unit has state-of-the-art, specialized equipment (such as

video machines, computers, desk-top publishers, micro-
teaching machines, etc.) which are adequate in number,
maintenance, and accessibility to permit students and faculty to
make full use of these important devices and materials.
193
1234 The unit has telephone and other communication capability

sufficient to permit each faculty to have privacy and engage in
extended professional conversation without penalizing
colleagues.
1234 Word processing machines, transcriber equipment, and personal

computers are available and in sufficient supply to support the
professional needs of all faculty and staff.
REFERENCES
Ayers, J. B., Gephart, W. J., & Clark, P. A. (1988). The accreditation plus
model. Journal of Personnel Evaluation in Education, 1, 335-
348.
Bush, R N. (1987). Education reform: lessons from the past half century.
Journal of Teacher Education, 38 (3), 13-19.
Carnegie Forum on Education and the Economy. (1986). A nation
prepared: Teachers for the 21st century. New York: The
Carnegie Foundation.
Clark, D. L. & Marker, G. (1975). The institutionalization of teacher
education. In K. Ryan (Ed.), Teacher Education. (74th Yearbook of the
National Society for the Study of Education, Part 2). Chicago, IL:
University of Chicago Press, 53-86.
Council for the Accreditation of Counseling and Related Educational Programs.
(1988). Accreditation procedure manual and application.
Washington, DC: American Association of Counseling Department.
External Review Committee for the Improvement of Teacher Education in
Georgia. (1986). Improving undergraduate teacher education in
Georgia. Atlanta, GA: The University System of Georgia Board of
Regents, 14-15.
External Review Committee on Teacher Education and Certification in
Mississippi. (1989). Recommendations for the improvement of
teacher education and certification in Mississippi. Jackson, MS:
State Board of Education and Board of Trustees of Institutions of Higher
Learning, 14-15.
Goodlad, J. I. (1983). A Place Called School. New York: McGraw-Hill
Book Company.
Holmes Group (1986). Tomorrow's teachers: A report of the
Holmes group. East Lansing, MI: The Holmes Group.
Howsam, R, Corrigan, D., Denemark, G., & Nash, R (1985). Educating a
profession, (2nd ed.). Washington, DC: American Association of
Kerr, D. H. (1983). Teaching competence and teacher education in the U. S. In
L. S. Shulman and G. Sykes (Eds.), Handbook of teaching and
policy, New York: Longman., 126-149.
194
Lanier, J. E. & Little, J. W. (1986). Research on teacher education. In M.

Wittrock (Ed.). Handbook of Research on Teaching, (3rd ed.). New
York: Macmillan Publishing Company, 556-557 ..
Manning, M. L. & Swick, K. J. (1984). Revitalizing teacher education: fiscal
and program concerns. Action in Teacher Education, 6 (3), 76-79.
Monahan, W. G., Denemark, G., Egbert, R., Giles, F. & McCarty, D.. (1984).
Fiscal conditions and implications in selected schools and
colleges of education in comprehensive universities, 1982-
83. Association of Colleges and Affiliated Private Universities. (ERIC
Document Reproduction Service No. ED 234 046).
National Commission for Excellence in Teacher Education (1985). A call for
change in teacher education. Washington, DC: American Association of
National Council for the Accreditation of Teacher Education. (1987). NCATE
standards, procedures, and policies for the accreditation of
professional education units. Washington, DC: Author.
Nutter, N. (1986). Resources needed for excellent teacher preparation programs.
In T. J. Lasley (Ed.). Issues in teacher education, Vol II.
Washington, DC: ERIC Clearinghouse on Teacher Education.
Orr, P. G. & Peseau, B. A. (1979). Formula funding is not the problem in
teacher education. Peabody Journal of Education, 57,61-71.
Parkay, F. W. (1986). A school-university partnership that fosters inquiry-
oriented staff development. Phi Delta Kappan, 67, 386.
Peseau, B. (1982). Developing an adequate resource base for teacher
education. Journal of Teacher Education, 33(4), 13-15.
Peseau, B. (1985). Resources allocated to teacher education
programs. (Resource Paper Commissioned by the Commission on
Excellence in Teacher Education). Washington, DC: American Association
of Colleges for Teacher Education.
Peseau, B. (1988, November). Quantitatively-based peer program identification.
Paper presented to the Mid-South Educational Research Association,
Louisville, KY.
Peseau, B., Backman, C., & Fry, B. (1987). A cost model for clinical teacher
education. Action in Teacher Education, 9(1), 21-34.
Peseau, B. & Orr, P. (1980). The outrageous underfunding of teacher education
(Special issue). Phi Delta Kappan, October, 1980, 100-102.
Sanders, T. (1985). Improving the quality of teachers and teaching.
Education Digest, 51, 22-25.
Smith, B. O. (1980). A design for a school of pedagogy. Washington,
DC: U. S. Department of Education.
Temple, C. M. & Riggs, R. O. (1978). The declining suitability of the
formula approach to public higher education. Peabody Journal of
Education, 55(4), 351-357.
Tennessee Board of Nursing. (1988). State Board Administrative Rule,
1001.007. No. 2C Nashville: Author.
Western Association of Schools and Colleges. (1988, January). Handbook of
accreditation. Oakland, CA: Author.
15
EV ALUA TION OF LIBRARY RESOURCES FOR A

TEACHER EDUCATION PROGRAM
Edward D. Garten
The University of Dayton
Central to this chapter is the assumption that each college or university library
system--and the education library or collection within those systems--is unique
and therefore should determine its own criteria for performance and evaluation.
Any evaluation should be undertaken within the framework of the college or
university's mission and goals. While recognizing that all academic libraries are
unique and individual. this chapter is. occasionally. prescriptive and will refer to
specific standards developed by the Association of College and Research Libraries
(ACRL). a division of the American Library Association (ALA). All
suggestions within this chapter are intended to assist non-librarians who are
responsible for determining priorities and evaluating the performance of a library
which supports a teacher preparation program. While the standards and
guidelines noted in this chapter cannot be stated as absolutes applicable to all
college and university libraries. such standards and guidelines do set forth a
process by which expectations may be established. Further, they suggest the
topics that must be addressed in some fashion during any evaluation of a library
which supports a teacher education program.
Underlying Assumptions
This chapter and any evaluation of librafy facilities as they support teacher
education programs are based on three assumptions which are described below.
Centrality of the Education Library to the Program. The

education library. its staff. and collections are of central importance to the
college's teacher education programs. The education library, together with its
parent library. is an organic combination of staff. collections. and physical
facilities whose purpose is to assist users in transforming information into
knowledge. The insights and skills gained from the resources of the education
library are central to the attainment of many. if not most. teacher education
goals. The ways in which information is collected. housed. accessed. and
distributed within the college or its extension programs will determine the level
196
and success of scholarship and research afforded both students and faculty. It is
assumed that the academic library which supports a teacher education program
has a mission statement which has been reviewed by appropriate senior
university officials and that this mission statement is congruent with the
mission statement of the School, College, or Department of Education (SCDE).
Library evaluators should be certain that a library mission statement exists.
That statement should be current and widely distributed on campus. If teacher
education is an important mission of the institution, then that should be reflected
at some point in the library's mission statement. In a larger university, which
has many colleges and schools, and in particular a school or college of education,
it is important that the university library have a specific undergraduate mission
statement. In October 1987, The ACRL published guidelines for the
development of such a mission statement. Teacher education program evaluators
must obtain their academic library's statement, if it exists, and make judgements
on that statement's adequacy.
The Significance of the Total Investment in the Library. The

library represents one of the largest accumulations of capital on any campus.
Moreover, libraries provide added value because they are used for all learning and
research processes. Libraries are not luxuries; nor are they peripheral to the
programmatic efforts of teacher education.
Technological Change. The pace of technological change has

eliminated any concept of isolation and self-sufficiency within libraries.
Evaluators of library services must realize that the library exists within a
complex information world. While libraries will not abandon their traditional
roles as collectors and conservators, increasingly they will add new roles, as
facilitators and processors of information, which must be recognized and
assessed.
ACADEMIC LIBRARY STANDARDS: A REVIEW
Several emerging trends relative to academic library evaluation are worthy of

note.
o There is a movement toward standards which address an institution's needs

rather than absolute ideals. This development suggests needs assessment,
analysis of resources, and coordination with the goals and objectives of the
larger institution, as well as coordination with various programs such as
teacher education. Standards are increasingly reflective of commitments
toward user services.
o User-oriented evaluation processes emphasize outputs or the actual services

the patron receives. More resources do not necessarily improve services or
the overall value of a library. An evaluation of effectiveness is an
evaluation of user satisfaction and efficiency of resource allocation. This
197
presupposes the existence of stated objectives against which a service or

program can be measured.
o Normative standards are being developed. This allows a total library, or a

sub-area, such as an education library, to compare itself with other libraries
similar in purpose, size, staff, collections, and facilities. This suggests a
process by which a library can develop standards appropriate to its purpose
and clients. .
o There is a need to address standards on a continuous, revisionary fashion.

Academic libraries are in a continuing state of development and flux, a fact
recognized in this trend.
Regional Accreditation Commissions: Common Criteria
The policy statements and the self-study handbooks of accreditation

commissions for postsecondary education contain statements regarding the
centrality of the library to the educational institution. They list specific criteria
or standards for libraries, including the following:
1. The library/leaming resource center should be of central and critical

importance to the institution as a primary learning resource.
2. The expanded role of the library to include non-print materials of all

varieties and to serve as a learning development center for students and
faculty is acknowledged and encouraged.
3. The library should be directly related to the institution's mission and its
programs of instruction.
4. The institution should have its own library or collection of resources and
while cooperative relationships and arrangements with other institutions are
encouraged, the institution's own library should have the resources to
support its programs.
5. Library collections should be housed in an adequate facility which has

sufficient space to support the needs of students and faculty. The library
should be open during hours convenient to the needs of users and materials
should be organized for easy access and use. There should be appropriate
supporting equipment for the use of both print and non-print materials.
Competent staff should be employed.
6. The library should be managed by qualified professionals who are supported

in their work by an adequate number of clerical and technical staff.
Professional librarians should be accorded faculty status.
198
7. The chief administrative officer of the library and the professional staff are
responsible for administering the total program of library services within the
college or university. An advisory committee of faculty and students should
assist them in the planning, utilization, and evaluation of library resources.
8. The library budget should include all expenditures related to the operation
and maintenance of the library. Sufficient funds should be provided to
support a sound program of operation and development, with planning for
subsequent years and a systematic program for removing obsolete resources.
9. If the institution has off-campus educational programs, it should provide

adequate library resources to serve the students.
10. Services of the library should be evaluated regularly to determine the

library's effectiveness through the nature and extent of its use. Circulation
and acquisition statistics should be maintained for all library resources. User
surveys should be conducted periodically for use as primary evidence of a
library's effectiveness. The library should periodically examine its own
statement of purposes and provide an assessment of its strengths, concerns,
and plans to remedy concerns identified, evidence of effectiveness, and plans
for the future.
A number of national, state, and regional accreditation documents exist and

will be of interest to anyone involved in evaluation. They generally include an
overview of the library's function and criteria relative to services, collections, and
staff.
American Library Association Standards
The majority of standards and guidelines which have been published by the
ALA have been developed and promulgated by ALA's ACRL. The guidelines
and standards have been developed and tested through extensive participation by
librarians, administrators, and others involved with higher education. These
documents are impressive because they were prepared by professionals who are
dedicated to the ideal of superior libraries which are related to the college or
university'S mission, are adequately staffed, and are continuously evaluated.
Perhaps more importantly, each of the statements has been developed and refined
through a meticulous process that includes development of an initial draft by a
committee; circulation of the draft for review and revision; development of a
final document that is approved by the Board of Directors of the ACRL; and
finally, publication and wide circulation of the standards. The documents are
well-written and can be understood and appreciated by non-librarians. They
include introductions, definitions, explicit standards with supporting
commentaries, and supplementary checklists or quantitative analytical techniques
to assist in the application of standards.
The guidelines and standards published by ACRL (this also can be said of
the regional accreditation bodies' documents) are standards which focus mainly on
199
inputs or process criteria. Until recently the guidelines rarely addressed outcomes
or evidence of effectiveness. To the credit of both, however, ACRL and the
regional accreditation bodies have begun to recognize the need for more tangible
outcomes information and are developing better ways to obtain concrete evidence
of effectiveness.
American College Research Libraries
Perhaps the most valuable and comprehensive standards document for use by
the evaluator is ACRL's "Standards for College Libraries, 1986," a revised
version of the 1975 standards. The 1986 version reflects the integration of new
technologies into libraries. Because all of the standards which are found in this
document are germane to the specialized education library or the education
resources which are part of the larger college or university library, evaluators
especially are urged to read these standards with care. Addressing the questions
raised by each standard is critical to a fair and comprehensive assessment of
library support.
In May 1986, ACRL released the "Guidelines for Audiovisual Services in
Academic Libraries: A Draft." Evaluators of libraries and learning resource
centers which support teacher education programs will want to familiarize
themselves with this document as it offers excellent guidance relative to media.
Microcomputer use has increased within both general academic libraries and
specialized education libraries. While no guidelines for evaluating
microcomputer and software support for teacher education programs have been
written by the ALA or the ACRL, guidelines and standards do exist. See, for
example, Cohen's (1983) list of evaluative criteria for software. The ACRL's
"Administrator's Checklist of Microcomputer Concerns in Education Libraries"
is another suggested resource.
In "Guidelines for Extended Campus Library Services," ACRL recognized
the tremendous growth in off-campus instruction. This document reflects the
concern that librarians and accreditation agencies share about the quality of off-
campus education: Are library facilities at off-campus sites equivalent to those
on-campus? Numerous other issues are raised in the "Guidelines" and evaluators
are urged to review this paper with some care.
Library Materials. P. Heller and B. Brenneman (1988) list some useful

criteria for writing (or evaluating) library handbooks. These cover such topics as
content, writing, style, and format. Evaluators will want to obtain copies of all
resource guides, specialized bibliographies, and other hand-outs which the library
makes available to students enrolled in teacher education programs. These
documents should be analyzed to determine if they meet the needs of the
students.
Critical Areas in Evaluation
Budget. Most libraries are heavily dependent on yearly base funding from
the parent institution for operating expenses, including personnel and
200
acquisitions expenditures. Private colleges and universities, to a greater degree

than public institutions, typically have endowment sources upon which they can
draw to supplement base acquisitions funding, but for the most part, libraries
continue to be heavily dependent on internal allocations.
The operating budget of the library must be appropriate to the mission of
the library within the college or university and there should be evidence that the
budget is developed interactively by the university and the library following the
accepted budgetary practices of the institution. There should be some
mechanism in place to allocate fairly the book and periodical budget. This
mechanism will vary across institutions, but it should be one which has gained
consensus among a broad spectrum of campus constituents. Teacher education
faculty, like their counterparts across campus, should be aware of the factors that
drive any allocation scheme used.
There should be evidence of at least a small yearly capital expenditure budget
available for minor investments in building improvements, renovations, or for
adding new services. The library is responsible for the ongoing maintenance of
its resources and services. This includes not only provision for replacement of
equipment and library materials, but also keeping adequate statistics and other
measures to determine whether the standards of service are being met
Human Resources. In order to achieve their goals, academic libraries

are dependent on human resources skilled in the knowledge-based disciplines.
The library should be staffed so as to meet the institution's expectations, and the
expectations of each department or school. The number of staff required is
determined by the range of programs offered, the number of buildings in which
services are rendered, and the hours during which services are offered. While
there are no absolute requirements as to number of staff, clearly the level of
service is determined by the availability of staff.
The staff should include professional librarians, support staff, and clerks
(often students) to provide services at the appropriate levels. The staff should be
able to meet the academic needs of the college or university and to provide
appropriate management support.
Collections. Library programs should be developed with the primary

goal of selecting, collecting, organizing, and providing access to all varieties of
information for users. The library staff should select and acquire materials to
support the teacher education program, as well as all other academic programs at
the institution. Collection management and development includes not only
purchasing for retention, but also leasing; removing obsolete materials;
providing access to other collections; including planned resource sharing and
cooperative storage; and providing electronic access to regional and national
databases. Evaluators should explore the nature of the cooperative resource
agreements which are in place.
The collections should be extensive enough to support the academic
programs offered by the teacher education program, recognizing that there are
instances where reliance can and should be placed on access to resources
elsewhere, rather than on ownership. It should be noted that while there are no
201
definitive standards or guidelines which would suggest a foundational or core

education collection, librarians historically have tended to rely heavily upon
disciplinary-developed bibliographies which are reflective of the range of courses
and specialties in the field. Collection development librarians should
communicate with faculty in the various specialty areas within the school,
college, or department of education to gain consensus on the value of particular
specialty bibliographies for collection development. Any attempt to delineate a
"core collection" to serve education students, however, will be heavily influenced
by the range of program and specialty areas offered at the institution.
As with all areas of the collection, there should be provision for adequate
funding of the education collection to ensure that both current and future
programs will be supported by the library.
Access to Programs and Services. The library should develop and

maintain policies and procedures for ensuring optimal access to its own
collections and to resources available elsewhere. The catalog and other records
should tell the user what is owned, where it is located, and how to find it.
Access no longer means simply enabling users to find desired materials. With
the development of information transfer techniques, many users are now able to
conduct their bibliographic research outside the library. In such instances,
providing access implies the delivery of information, whether in printed or
electronic format, by the library to the user at the user's location. This process
should be reflected in the policies and procedures of the library and if there are
particular or unique provisions made for education faculty and students they
should be noted.
The library should check collection availability regularly and periodic
inventories should be conducted. In addition, the library has a responsibility to
explain its services and resources. Thus, the library should provide directions for
accessing and information about all resources and services which are specifically
directed toward the teacher education programs as well as related materials which
may be found in different collections.
Information Transfer. There are two forms of information transfer: the

physical transfer of documents and facsimiles of documents, and the electronic
transfer of information. The evaluator will want to know if the library
participates successfully and efficiently in the full range of local, regional, and
national programs for interlibrary loan, telefacsmile, and document delivery. The
rules and conditions applying to these programs should be available to users of
these services and the library should be prepared to facilitate direct transfer to the
user of information available from all the erucation-specific or education-related
databases or by referral to other agencies caJ: able of meeting this need.
Measuring Achievement In Library Services
All mechanisms used to evaluate library performance should include

identifiable outcomes, both qualitative and quantitative, and should provide
appropriate feedback. The evaluation process should be continuous rather than
202
"one time," though it must also fit into any process established by the
institution. The goal for performance evaluation, of course, is to arrive at the
clearly stated set of expectations which can be matched against the resources
needed.
There is no single way of measuring achievement, thus a variety of
procedures and methods should be explored. All activities for performance
review should, within economic and political constraints, provide a setting
within which an open, supportive assessment can occur. Inevitably,
comparisons will be made with libraries in other colleges and universities.
Although such comparisons are difficult to avoid, they should be made carefully.
Any performance study should be aided by appropriate quantitative measures and
should never be based solely on subjective measures. Finally, any performance
evaluation requires that the responsibility for the evaluation be clearly assigned;
that the procedures to be followed are understood fully by the participants; and
that the goals are stated and defined clearly. Resources exist for securing data and
other performance appraisal information and should be consulted for useful
instrumentation and methodology.
Finally, the Evaluative Criteria which follow suggest a range of questions
that explore ways libraries have gathered data for performance evaluation. The
evaluator will want to assess what the library is currently doing to obtain
performance data and perhaps recommend studies or methodologies which will
obtain useful data for future review.
APPENDIX
Basic Library Resources for Support of Teacher Education

Programs
Dictionaries
Good, C. V. (Ed.). (1973). Dictionary of education (3d ed.). New York:

McGraw-Hill.
Hawes, G. R. & Hawes, L. S. (1982). The concise dictionary of
education. New York: Van Nostrand.
Page, G. T., Thomas, J. B., & Marshall, A. R. (1977). International
dictionary of education. London: Kogan Page; New York: Nichols.
Rowntree, D. (1982) A dictionary of education. Totowa, New Jersey:
Barnes and Noble.
Encyclopedias
Dejnozka, E. L. & Kapel, D. E. (1982) American educators'

encyclopedia. Westport, CT: Greenwood Press.
Mitzel, H. E. (Ed. in Chief). (1982) Encyclopedia of educational
research (5th ed.). New York: Free Press.
Reynolds C. R. & Mann, L. (Eds.) (1987). Encyclopedia of special
education. New York: John Wiley and Sons.
203
The international encyclopedia of education: research and

studies. (1985). Oxford and New York: Pergamon Press.
Knowles, A. S. (Ed. in Chief). (1977). The international encyclopedia
of higher education. San Francisco: Jossey-Bass.
Monroe, P. (1911-1913) A cyclopedia of educations. New York:
Macmillan (1968 reprint available from Gale).
Kurian, G.. (Ed.). (1988). World education encyclopedia. New York:
Facts on File Publications.
Thesauri
Houston, J. E. (Ed.). (1987). Thesaurus of ERIC descriptors. Phoenix,

AZ: Oryx Press.
Thesaurus of Psychological Index Terms. (1988). Washington, DC:
American Psychological Association.
Indices to Education
Current Index to Journals in Education. (1969). Phoenix, AZ: Oryx

Press. (Available as CD-ROM products).
Education Index. (1932). New York: Wilson.
Resources in Education. (1975). Washington, DC: U. S. Department of
Health, Education, and Welfare. (Available as CD-ROM products).
Directories
Accredited Institutions of Postsecondary Education: Programs,

Candidates. (1976/77). Washington, DC: American Council on
Education,.
Cabell, D. W. E. (Ed.). (1984). Cabell's directory of publishing
opportunities in education. Beaumont, TX: Cabell Publishing Co.
Directory of American Scholars: A Biographical Directory.
(1982). New York: Jaques Cattell Press/Bowker.
Directory of Education Associations. (1980/81). Washington, DC: U.
S. Government Printing Office.
Miller, E. E. (1985). Educational media and technology yearbook.
Littleton, CO: Libraries Unlimited, Inc.
NEA Handbook. (1987/88). Washington, DC: National Education
Association of the United States.
Patterson's American Education. (1904). Mount Prospect, IL:
Educational Industries, Inc.
T.E.S.S.: The Educational Software Selector. (1984). New York:
Teachers College Press.
Statistics, Annuals, Yearbooks
The Condition of Education. (1975). Washington, DC: USGPO.

204
Digest of Educational Statistics. (1962). Washington, DC: USGPO

Gerald, D. E. (1985). Projections of education statistics to 1992-93.
Washington, DC: USGPO
Gutek, G. L. (Ed). (1985). Standard Education Almanac. Chicago:
Marquis Who's Who, Inc.
Yearbook of School Law. (1950). Danville, IL: Interstate.
Handbooks
Burks, M. P. (Ed.). (1988). Requirements for certification: For

elementary schools, secondary schools, junior colleges.
Chicago: University of Chicago Press.
Buros, O. K. (Ed.). (1985) Mental measurements yearbook (9th
edition). Highland Park, NJ: The Gryphon Press.
Buros, O. K. (Ed.). (1961). Personality tests and reviews: Including
an index to the mental measurements yearbooks. Highland Park,
NJ: The Gryphon Press.
Buros, O. K. (Ed.). (1974). Tests in print: A comprehensive
bibliography of tests for use in education, psychology, and
industry. Highland Park, NJ: The Gryphon Press.
Buros, O. K. (Ed.). (1974). Tests in print II: An index to tests,
tests reviews, and the literature on specific tests. Highland
Park, NJ: The Gryphon Press.
Goddard, R. E. (1988). Teacher certification requirements: All fifty
states. Lake Placid, FL: Teacher Certification Publications.
Hoover, K. H. (1982). The professional teacher's handbook: A guide
for improving instruction in today's middle and secondary
schools (3d ed.). Boston: Allyn and Bacon.
Kauffman, J. M. & Hallahan, D. P. (Eds.). (1981). Handbook of special
education. Englewood Cliffs, NJ: Prentice-Hall, Inc.
Mitchell, J. V. (Ed.). (19830. Tests in Print III: An index to tests,
test reviews, and the literature on specific tests. Lincoln, NE:
Buros Institute of Mental Measurements, University of Nebraska-Lincoln.
Office of Educational Research and Improvement. (1987). What's
happening in teacher testing: An analysis of state teacher
testing practices. Washington, DC: USGPO.
Wittrock, M. C. (Ed.). (1986). Handbook of research on teaching (3d
ed.). New York: MacMillan Publishing Company.
Sources of Grant Information
Annual Register of Grant Support. (1969). Los Angeles: Academic

Media.
Coleman, W. E. (1984). Grants in the humanities: A scholar's
guide to funding sources ( 2d ed.). New York: Neal-Schuman.
The Complete Grants Sourcebook for Higher Education. (1980).
Washington, DC: American Council on Education.
205
Directory of Research Grants. (1975). Phoenix, AZ: Oryx Press.

Foundation Center. (1975n6). The Foundation Center Source Book.
New York: Foundation Center.
Foundation Directory (10th ed.). New York: Foundation Center.
Foundation Grants Index. (1972). A Cumulative Listing of
Foundation Grants. New York: Foundation Center.
Lefferts, R. (1982). Getting a grant in the 1980s: How to write
successful grant proposals (2d ed.). Englewood Cliffs, NJ: Prentice-
Hall.
Guides to Educational Research
Berry, D. M. (1980). A bibliographic guide to educational research

(2d ed.). Metuchen, NJ: Scarecrow Press.
Kennedy, J. R. (1979). Library research guide to education:
Illustrated search strategy and sources. Ann Arbor, MI: Pierian
Press.
Woodbury, M. (1982). A guide to sources of educational
information (2d ed. completely rev.). Arlington, VA: Information
Resources Press.
Bibliographies
Bibliographic Guide To Education. (1979). Boston: G.K. Hall.

Drazen, J. G. (1982). An annotated bibliography of ERIC
bibliographies, 1966-1980. Westport, CT: Greenwood Press.
Durnin, R. G. (1982). American education: A guide to information
sources. Detroit: Gale.
EVALUATIVE CRITERIA FOR USE BY EVALUATORS

SEEKING TO ASSESS LIBRARY SERVICES AND
RESOURCES SUPPORTIVE OF TEACHER EDUCATION
PROGRAMS
This resource is intended to serve as the first tool in the mutual department
or school of education/librarian assessment of library services supportive of
teacher education programs. It is not meant to be used alone; rather, it or the
alternative list of questions which follow on the next document are intended to
be used in a process of discussion and services description.
The questions which follow are suggested as one means of reaching a proper
description and assessment of the library which supports the teacher education
program. This guide and the more procedural and methodological one which
follows include many of the measures for evaluation now commonly accepted
and practiced within academic libraries.
206
Budget Adequacy
1. Are the budgetary resources sufficient to support current activities and to

provide for future development?
2. Does the budget support the purchase of or access to the necessary range of
library materials in the field of education?
3. Does the budget support appropriate numbers and kinds of staff
specialization for the programs offered?
4. Does the budget provide adequate support for other library operating
expenses?
5. Does the budget provide adequate support for new programs and
innovations?
6. Does the process by which the budget is developed allow for appropriate
consultation, especially by designated members of the department or school
of education?
Collection Adequacy
1. Is there a written policy for managing the library collections and, in

particular, is the development of the education collection described in the
collection development policy?
2. Does this policy address issues of user satisfaction?
3. Is there provision for considering change in academic programs and needs?
4. What basis is used for determining collection levels and sizes, especially
with respect to the education collection?
5. Is there evidence of areas of undersupply?
6. Is there evidence of areas of oversupply?
7. Does the collection match the academic programs and specialties offered by
the department or school of education?
8. Is the collection growing and responding to change at an appropriate rate?
Access and Availability
1. Are the policies governing access and use of the collections clearly stated
and are they readily available?
2. Are the bibliographic records adequate and accurate?
3. Are the collections properly housed?
4. How readily can the library provide materials now owned?
5. Is the staff provided for technical services and other collection-related
services sufficient for the task?
Preservation and Conservation
1. Does the library have proper environmental controls?

2. Does the library have an emergency plan in place?
3. Does the library budget have adequate provision for the preservation and
repair of damaged, aged, and brittle books?
207
4. Does the library have adequate safeguards against loss, mutilation, and theft?
Adequacy of Services
1. What range of services is offered and over what ranges of times?

2. Are the locations where services are offered adequate to the purpose?
3. What statistics and other measures of quality and quantity are maintained?
4. Is the size and distribution of public service staff adequate for the numbers
and kinds of users?
Adequacy of Buildings and Equipment
1. Are the buildings sufficient to house staff and collections?

2. Are the buildings adequately maintained?
3. Are there appropriate space plans?
4. Is there appropriate provision for use by the handicapped?
5. Is the range, quantity, and location of equipment adequate to the programs
offcred?
6. Is the equipment adequately maintained?
7. Is there budgetary provision for upgrading, repair, and replacement of
equipment?
8. Is there evidence of planning for the use of new and improved technologies?
MEASURES OF LIBRARY PERFORMANCE: SOME

QUESTIONS
The evaluator should use these questions as a guide for discussion with
those librarians and curriculum materials specialists responsible for providing the
services and resources which support the college or university'S teacher
preparation programs.
As a caution, it should be noted that not every academic library does all of
the following evaluative activities. Libraries may find some evaluative practices
more relevant than others. Additionally, and somewhat unfortunately, many
libraries simply may not have adequate staffing to afford the time to conduct
many of the evaluative activities noted here. However, several of the activities
are considered more important than others.
Reference Service Evaluation
1. Does the library maintain statistics on inquiries received and does it

maintain them by discipline, e.g., education?
2. Does the library maintain statistics on the proportion of questions answered
correctly?
3. Does the library conduct a periodic test of the reference staffs performance
by using a test set of questions?
4. Can the evaluator(s) obtain times to observe reference staff performance
unobtrusively, particularly staff serving teacher education students?
208
5. Does the library analyze reference use patterns?

6. Does the library survey patrons on their evaluation of the personal
assistance available for finding information?
Collection Evaluation
1. Do members of the library staff compare collection size with accepted

standards?
2. Are standard education bibliographies compared against the education
collection?
3. Do members of the library staff compare expenditures on the collection in
relation to the total institutional budget?
4. What type of formula for allocation is used by the library?
5. How does the allocation for resources to support teacher education programs
compare with allocations for other program areas within the college or
university?
6 Does the library staff periodically study distributio'l of funds for the
collections by individual subject field?
7. Does the library staff periodically compare the collection, especially in
education and related areas such as psychology and sociology, against
holdings of other institutions?
8. Does the library on occasion use subject specialists who examine the
collection and assess its adequacy?
9. Does the library survey patrons on their evaluation of the adequacy of the
collection?
Catalog Evaluation
1. Does the library staff periodically interview catalog users?

2. Does the library staff periodically distribute self-administered questionnaires
to users after catalog use?
3. Does the library staff observe catalog use unobtrusively?
4. Does the staff monitor computerized catalog use statistics on-line?
5. Does the staff survey patrons on their use of the catalog as an information
finding tool?
Materials Use Evaluation
1. Does the library maintain statistics on circulation of materials outside the

library? Can these statistics be obtained for education?
2. Does the library maintain statistics on the use of materials within the
library? Can these statistics be obtained for education?
3. Does the library conduct an analysis of circulation records?
4. Does the library periodically analyze its borrowing policy/privileges?
5. Does the library study materials accessibility (difficulty or delay in
obtaining materials)?
6. Does the library study materials availability (chance of obtaining materials)?
209
7. Does the library test document delivery success rate by use of any accepted
documents delivery test?
8. Does the library analyze the proportion of interlibrary loan requests which
are satisfied?
9. Does the library assess the time required to satisfy interlibrary loan requests?
10. Are education students and faculty members satisfied with the quality of
interlibrary loan services?
Bibliographic Instruction Evaluation
1. Does the library measure the effectiveness of bibliographic instruction by a

pre- and post-test study?
2. Does the library survey patrons on bibliographic instruction (how well it is
presented, how important it is to patrons, what can be done to improve it,
etc.)?
Physical Facilities Evaluation
1. Does the library periodically study its facilities and, in particular, if there is
a separate education library, are these facilities periodically studied? If such a
study is done, are physical arrangements of materials, service points,
furniture, and equipment evaluated?
2. Does the library analyze the use of space for stacks and seating by
comparison with accepted standards?
3. If there is a separate education library, does seating and stack space compare
favorably with facilities provided in other areas of the library?
4. Does the library periodically survey patrons on their evaluation of
surroundings, e.g., environmental conditions, attractiveness, etc.?
Patron Use Evaluation
1. Does the library maintain statistics on the number of patrons entering the
library?
2. Does the library measure average time patrons spend in the library?
3. Does the library measure patron use by hourly counts in individual areas of
the library?
4. If there is a separate education library, is this done here as well?
5. Does the library compare hours of service with those of similar libraries?
6. Does the library periodically survey patrons on types of materials used in
the library including personal materials?
User Needs/Satisfaction Evaluation
1. Does the library periodically distribute self-administered questionnaires to

users to assess needs/satisfaction?
2. Does the library periodically interview users to assess needs/satisfaction?
210
3. Does the library analyze feedback from library committees or academic

departtnentliaisons?
4. Does the library utilize library users' suggestion boxes?
5. Does the library request diary-keeping of a sample of library users,
describing library services needs/use over a period of time?
On-Line Bibliographic Searching and Information Retrieval
1. Does the library maintain use statistics? Are they divided by discipline
including education?
2. Has the library ever requested users to indicate which retrieved
citations/items were relevant?
3. Has the library studied searcher performance by comparing a search against
standard searches conducted solely for the purpose of evaluation?
4. Does the library survey patrons on their use of the on-line search services to
obtain information?
REFERENCES
Cohen, V. B. (1983). Criteria for the evaluation of microcomputer course ware.

Educational Technology, 23(1), 9-14.
Heller, P. & Brenneman, B. (1988, February). A checklist for evaluating
your library's handbook. College and Research News, 78-80.
16
MODELS AND MODELING FOR TEACHER EDUCATION

EVALUATION
Mary F. Berney and William J. Gephart

The year 1973 marked the beginning of what Madaus, Stufflebeam, and Scriven
refer to as the "age of professionalization" (1983, p. 15) in the field of
evaluation. In the 1970s, eight journals devoted partially or entirely to
evaluation were established, universities began offering graduate programs in
evaluation, and centers for the study of evaluation were established. Educators
and legislators called for accountability and more evaluation models were
developed. A recent count yielded over fifty extant evaluation models. How
does this proliferation of models and theories about modeling affect the routine
operation of teacher education programs? Most decisions can be traced to the use
of some model. Understanding the models and the rolf of empirical evidence in
decision-making will enhance the ability to make good decisions. This chapter
is written for use by individuals who have addressed the questions relative to
accreditation for their units but who face additional questions about program
evaluation and improvement. The emphasis is on models and how they can be
applied to improve teacher education programs. Readers who find this chapter
too elementary will want to go directly to the cited sources for more
sophisticated treatments of the content. Background information, definitions,
and suggestions for choosing the appropriate model(s) for a given situation are
provided.
Definitions of Terms
The following terms will guide the discussions in the remainder of the
chapter. The first of the definitions has already been seen in the second chapter,
but bears repeating here. "A model ... is a representative of the entity being
represented" (Ayers, Gephart, & Clark, 1988, p. 336). Stufflebeam and Webster
define educational evaluation as a study "that is designed and conducted to
assist some audience to judge and improve the worth of some educational object"
(1983, p. 24). That definition, as the authors observe, is common, despite the
fact that it is not often adhered to in practice. It is also important to remember
212
another point Stufflebeam has made, that evaluations are conducted to

improve, whereas research is conducted to prove. Thus evaluation is an on-
going process and not the crisis-driven event that it so often becomes, especially
as personnel prepare for accreditation.
Value is at the heart of "evaluation" and plays a correspondingly important

role in defining, utilizing, implementing, and understanding the evaluation
process, but the distinction between values and preferences must be made and
kept clearly in mind. Worthen and Sanders (1973) warned that "valuing and
choosing should be distinguished, if for no other reason than to prevent
educational evaluators from turning away from answering difficult value
questions and moving toward less threatening activities . . . that are not
intrinsically evaluative" (p. 24). A related and equally difficult term is worth.
The Joint Committee on Standards for Educational Evaluation defines worth as
"the value of an object in relationship to a purpose" (1981, p. 156). It must be
noted that not all evaluators agree as to the role of values and worth in
evaluation studies. The definitions and the stance which undergird this chapter
have been presented by Stufflebeam and others.
Evaluations are often classified as either formative or summative.
Patton (1986) cites Scriven's distinction between the two: collecting
information to be used primarily for developing or improving programs is a
formative activity and making judgments about the overall quality of a program
is a summative activity. Evaluators must also consider the various audiences
or constituents who have some stake in the proceedings.
These definitions apply to all models and will be useful to hold in mind
while reading the following sections which describe particular models.
Origins of Accreditation Plus
The Accreditation Plus Model was developed by Ayers, Gephart, & Clark
(1988) and is based in part on the professional judgment model of evaluation and
includes the use of other extant models as necessary to answer evaluation
questions which are not addressed by the accreditation process. The latter part is
the "Plus." In the process of developing the model, its creators searched the
literature and found reference to some fifty extant evaluation models. They are
not all relevant to educational evaluation, or more specifically, to the evaluation
of teacher education programs, but some types of evaluations are and will be
discussed later in this chapter.
The Accreditation Plus Model was designed for use by those seeking
approval or accreditation from some national, state, or regional agency. A
survey of states revealed that in most, if a teacher preparation program meets the
standards set forth by the National Council for the Accreditation of Teacher
Education (NCATE), it will also meet or exceed the standards for other national,
state, or regional approval processes. So its creators chose the accreditation
process, the professional judgment approach to evaluation, as the basis of the
model.
213
Professional Judgment
Worthen and Sanders (1973) describe professional judgment as a general

approach to evaluation which has often been used by evaluators and which "is
exemplified by the use of experts to produce professional judgments about a
phenomenon being observed" (pp. 126-127). They add that examples can be
found in, among other processes, "visits by accrediting agencies to secondary
schools and universities" and further derme the accreditation process as consisting
of "standards against which schools or universities are measured" which are
"generally arrived at through the collective judgments of persons seen as
possessing expertise about secondary or higher education "(p. 127).
The passage continues with a description of the accreditation process and
concludes with the statement that "the main emphasis of the professional
judgment approach to evaluation is that of application of presumed expertise to
yield judgments about quality or effectiveness" and "in accreditation, per se, self-
study and the maintenance of defined standards are also formalized" (p. 127).
Stufflebeam and Webster (1983) identified as the main advantages of
accreditation/certification studies that they aid "lay persons in making informed
judgments about the quality of educational institutions and the qualifications of
educational personnel" (p. 32). The difficulties inherent in such an approach are
"that the guidelines of accrediting and certifying bodies typically emphasize the
intrinsic and not the outcome criteria of education" and that the "self-study and
visitation processes ... offer many opportunities for corruption and inept
performance" (p. 32).
Ayers, Gephart, and Clark recognized that particular weakness in the use of
the accreditation model alone. They further recognized that if and when
evaluation becomes an integral part of teacher education programs there will
most likely be questions which the accreditation process is not designed to
answer. That understanding prompted them to add the "Plus," the
recommendation that users choose an informed eclectic approach to determine
which other model(s) would answer those questions and apply the model(s)
accordingly.
Other Evaluation Models
Accreditation is derived from professional judgment and the "Plus" comes

from the application of other models. What are those models? A related
question might well be, and how many of them really exist? Evaluators differ
on the answer to that. For instance, within a single collection of papers on the
topic of evaluation models (Madaus, Scriven & Stufflebeam, 1983), Stake
identified two models and nine approaches; Stufflebeam and Webster identified
thirteen approaches to evaluation, or types of study classified into three general
approaches, and House described eight models. Elsewhere, Herman, Morris, and
Fitz-Gibbon (1987) describe seven models of evaluation, Patton (1986) identifies
thirty separate types of evaluation, and Worthen and Sanders (1973) describe
eight. It would not be difficult to find still other models, approaches, or types of
214
evaluation described in the literature. The point of this is not to debate

nomenclature but to make it clear that even the experts vary on what is and is
not an evaluation "model."
Stake (1983) identifies the purpose, key elements, emphasized purview,
protagonists, case examples, risks, and payoffs for each model and approach.
Stufflebeam and Webster (1983) describe each of thirteen approaches in terms of
advance organizers, purpose, source of questions, main questions, typical
methods, pioneers, and developers. They classify approaches as being political,
questions-oriented, or values-oriented and observe that of the three, only values-
oriented studies are true evaluations because these are "designed primarily to
assess and/or improve the worth of some object" (p. 24). These include
accreditation/certification studies, policy studies, decision-oriented studies, client-
centered studies, and connoisseur-based studies. House (1983) identifies the
person(s) with whom the model is most closely associated; identifies major
audiences of the evaluation; the area(s) on which the model presumes consensus;
describes data collection methodology; identifies expected outcomes; and poses
some typical questions for each of eight approaches. Tables which depict this
information are presented in the original source. Patton (1986) provides
questions for each of the 30 evaluation types he lists in Appendix II of his book.
The section which follows contains pertinent information about nine models for
or approaches to evaluation.
EV ALUA TION MODELS OR APPROACHES
Of systems analysis House writes, "In this approach one assumes a few
quantitative output measures, usually test scores, and tries to relate differences in
programs to variations in test scores" (p. 46). Patton (1986) further describes
systems analysis as looking "at the relationship between program inputs and
outputs" (p. 68). Efficiency is the desired outcome and the model is used to
answer the question, "What combination of inputs and implementation strategies
will most efficiently produce desired outcomes?" (patton, 1986, p. 68). Other
questions include, "Are the expected effects achieved? What are the most efficient
programs? (House, 1983, p. 46). Cost benefit analysis, linear programming, and
computer-based information systems are used to generate answers.
Another model or approach is objectives-based. Ralph Tyler originated
this approach, which is described by its name. Program objectives are written in
terms of "specific student performances that can be reduced to specific student
behaviors" which are "measured by tests, either norm-referenced or criterion-
referenced" (House, 1983, p. 46). The obvious question addressed by this model
is, "Are the students meeting the objectives?" Test scores and measures of
discrepancy are sources of answers to that question. It is possible, but somewhat
dangerous, to use this approach to evaluate teacher productivity. It has been
stressed throughout this text that no single approach should be used to evaluate
anything and as Centra points out elsewhere in this text, there are too many
intervening variables associated with any measure of student learning to use it as
a single, fair indicator of teacher performance.
215
Stufflebeam is the major proponent of the decision-making approach.

His CIPP (Context, Input, Process, Product) model is widely recognized.
Stufflebeam defines evaluation as "the process of delineating, obtaining, and
providing useful information for judging decision alternatives" (Worthen and
Sanders, 1973, p. 129). Decision-making looks at effectiveness and thus asks,
"Is this object effective?" Another questions is, "What parts are effective?"
(House, 1983, p. 46). Decision-makers are a key source of both questions and
answers in this model.
The goal-free model is most often associated with Scriven. Goal-free
evaluation, according to Patton (1986, p. 112) "essentially means gathering data
on a broad array of actual effects and evaluating the importance of these effects
in meeting demonstrated needs." In short, the evaluator(s) know nothing about
the goals of the program that is being evaluated, by choice, because that
eliminates the possibility of looking only for stated objectives and thus missing
other important outcomes. The question associated with this model is, "What
are all effects?" (House, 1983, p. 46). Sources of answers to that question
include consumers, society, and the evaluator.
The art criticism model of evaluation has been forwarded by Eisner and is
derived from art and literary criticism. The evaluator is a critic who has the
experience and training to judge an educational program. One question which
can be answered by this model is, "What attributes distinguish this object from
others of similar type?" Another is, "Would a critic approve this program?"
(House, 1983, p. 46). Critics and authorities are the sources of information.
The adversary or advocacy-adversary approach is conducted along quasi-
legal lines, often including a trial by jury. Arguments for and against the
program, generally the continuation of a program, are presented. The approach
is most commonly associated with Owens, Levine, and Wolf. (House, 1983, p.
46). Questions include, "Which of two competing programs will maximize the
achievement of desired outcomes at the more reasonable cost? Sources of
answers are legislators, policy boards, and members of the jury.
The transaction model "concentrates on the educational processes... uses
various informal methods of investigation and has been increasingly drawn to the
case study as the major methodology" (House, 1983, p. 47). Stake writes that
transaction is used to "provide understanding of activities and values" (1983, p.
304). The question House (1983, p. 48) poses is, "What does the program look
like to different people?" Sources of answers are the clients and the audience.
Classroom observation, case studies, and interviews are among the commonly
employed methodologies.
The discrepancy model compares performance with a previously
determined standard. This approach is much like the behavioral approach in that
it can be used to measure gain. Once the discrepancy has been identified,
however, the application of other models is usually necessary. Provus was one
the proponents of this model. He suggested the use of teams, with the evaluator
serving as a team member who is external to the object of the evaluation. This
model can be used to answer the question, "Which students achieved which
objectives?" (Stufflebeam & Webster, 1983, p. 38). Program staff are sources of
questions and answers.
216
Utilization-focused evaluations can include all other types of

evaluations. They serve to answer the question, "What information is needed and
wanted by decision makers, information users, and stakeholders that will actually
be used for program improvement and to make decisions about the program?"
(Patton, 1986, 347). The subject of Patton's book, utilization-focused, or
oriented, evaluations emphasize the use, utilization, of evaluation findings by
program personnel and other decsion-makers. In the chapter which follows this,
Rutherford discusses the need to implement evaluation findings and provides
suggestions for doing so.
Staff in the Center for Teacher Education Evaluation, as well as many other
evaluators, advocate an informed eclecticism in choosing the model(s) for use in
evaluating a specific program. Informed means that the decision to choose one
model over another has been well and carefully researched; eclecticism suggests
that a variety of approaches or models might be used successfully. In the
remainder of this chapter some typical evaluation questions which are outside the
scope of accreditation will be presented and the processes by which the model(s)
chosen to address each question were selected will be outlined.
The "Plus": Alternative Approaches to Evaluation'
Some typical evaluation questions were presented earlier in the descriptions

of models. Three questions and suggested alternative responses are presented in
this section to serve as examples and starting points for discussion. Good
planning is essential to the success of any evaluation effort The time spent in
developing a comprehensive checklist and series of worksheets will be time
saved by personnel who don't have to stop and wait for the evaluator or some
other person to direct the next step. The process of developing such tools will
also uncover tasks that might otherwise have been overlooked.
Question One
Which design for the preparation of teachers is more effective? One

approach to answering the question is to apply the systems analysis model,
asking the question, "What combination of inputs and implementation strategies
will most efficiently produce desired outcomes?" (Patton, 1986, p. 68).
Stufflebeam's CIPP model also makes use of systems analysis characteristics and
he provides some useful insights into the relationship between evaluation and
the change process, some of which will be discussed here. When the systems
approach or the decision-making approach is used evaluation becomes an integral
part of a program. When evaluations are an integral part of a program they will
be vital in stimulating and planning change. The information that is collected in
a systematic process can be used to judge, or decide whether the program is
effective and whether decisions which were made relative to the program are
defensible.
The adversary or adversary-advocacy model could also be used to address the
question of which design is more effective. In that instance, two competing
judges (or teams) would choose, or be assigned, sides and would prepare and
217
present cases promoting the adoption or continuance or discontinuance of a

program. One of the drawbacks to the use of this model is its cost. If used to
choose a design and more than one option existed, the process would have to be
repeated for each option. A strength of the model is that it does tend to promote
full disclosure, but the costs and benefits must be weighed carefully. This
approach is time-consuming and should be undertaken only by trained personnel.
Question Two
What are alternative approaches to judging the quality of teachers in the

classroom? The behavior objectives approach is often used, and often misused.
If the objectives are set for the teacher's performance rather than for the students'
performance, the measure will be of how well the teacher meets the objectives.
The decision-making and systems analysis approaches can be used to judge the
quality of teachers.
Transaction analysis could be used, providing that everyone is alert to the
danger of relying too heavily on subjective perceptions formed from classroom
observations. Peer evaluations, also somewhat risky for being subjective, are
another means of evaluating classroom performance. The Joint Committee on
Standards for Educational Evaluation recently published standards for personnel
evaluation (1988). The standards "require that evaluations be proper, useful,
feasible, and aCI~urate" (p. 5) and provide a number of answers to the question
which begins this paragraph.
Question Three
Is student progress in the teacher education program satisfactory? One of the

more common approaches to answering this question is the application of the
discrepancy model whereby actual performance is compared with standards.
Student grades and test scores would be compared with department or unit
objectives to identify any gaps or discrepancies. The discrepancy approach
resembles the behavioral objectives approach in that respect. It can be applied to
entire programs as well as to individual performances. The use of the
discrepancy approach presupposes the existence of a set of clearly written
standards against which performance can be compared. If those are not available,
a different approach should be considered. The behavioral objectives approach
also requires clearly written objectives. The goal-free approach could be used to
measure student progress. It would be expensive to apply but could provide an
abundance of objective information on which decisions could be based. As with
the adversary approach, this requires, if not trained evaluators, personnel who are
willing to work without written objectives.
SUMMARY
Models, particularly professional judgment and those others which

comprise the "Plus" of the Accreditation Plus Model, have been described in this
chapter. Some common evaluation questions and approaches to answering them
218
were presented. The discerning reader will have noticed that there is rarely one
best approach or answer where evaluation is concerned. Some authors, it is true,
will advocate rigid adherence to a single approach, but that perspective is not
shared by the authors of this chapter or the editors of the book. The informed
eclecticism described earlier is more than just a catchy phrase. As evaluation is
integrated into a program and is utilized for decison-making, the questions that
arise will dictate the use of a variety of approaches or models.
REFERENCES
Ayers, J. B., Gephart, W. J., & Clark, P. A. (1988). The accreditation plus
model. Journal of Personnel Evaluation in Education, 1, 335-
343.
Herman, J. L. , Morris, L. L., & Fitz-Gibbon, C. T. (1987). Evaluator's
handbook. Newbury Park, CA: Sage Publications, Inc.
House, E. R. (1983). Assumptions underlying evaluation models. In G. F.
Madaus, M. S. Scriven, & D. L. Stufflebeam (Eds. ), Evaluation
models: Viewpoints on educational and human services
evaluation, (pp. 45-64). Hingham, MA: Kluwer Academic Publishers.
Joint Committee on Standards for Educational Evaluation. (1981). Standards
for Evaluations of Educational Programs, Projects, and
Materials. New York: McGraw-Hill Book Co.
Madaus, G. F., Stufflebeam, D. L., & Scriven, M. S. Program evaluation: a
historical overview. (1983). In G. F. Madaus, M. S. Scriven, & D. L.
Stufflebeam (Eds.), Evaluation models: Views on educational
and human services evaluation (pp. 3 - 22). Hingham, MA: Kluwer
Academic Publishers.
education units. Washington, DC.: Author.
Patton, M. Q. (1986). Utlization·focused evaluation (2nd ed.).Beverly
Hills, CA: Sage Publications, Inc.
Stake, R. (1983). Program evaluation, particularly responsive evaluation. In
G. F. Madaus, M. S. Scriven, & D. L. Stufflebeam (Eds.) Evaluation
models: Views on educational and human services
evaluation, (pp. 287-310). Hingham, MA: Kluwer Academic
Publishers.
Stufflebeam, D. L. & Webster, W. J. (1983). An analysis of alternative
approaches to evaluation. In G. F. Madaus, M. S. Scriven, & D. L.
Stufflebeam (Eds.) Evaluation models: Views on educational
and human services evaluation, (pp. 23-43). Hingham, MA:
Kluwer Academic Publishers.
Worthen, B. R. & Sanders, J. R. (1973). Educational evaluation:
Theory and practice. Worthington,OH: Charles A. Jones Publishing
Company.
17
IMPLEMENTATION OF EV ALUA TION RESULTS
William L. Rutherford
The University of Texas at Austin
Evaluation is but one step in a process intended to raise an organization to a

maximum level of efficient and effective performance. If evaluation is to have
the greatest possible impact on the organization the findings must be fully and
appropriately implemented. The purpose of this paper is to consider some of the
links and problems that occur in the movement from evaluation to
implementation and to offer suggestions for enhancing the probability that
evaluation findings from the Accreditation Plus Model (as well as any others)
will actually be used to improve programs. A set of guidelines for readers to
follow in implementing evaluation results is keyed to the major headings and
appears following the text.
The story of a teenager and his interactions with his father regarding the
acquisition of his first automobile seems to portray some problems of getting
evaluation data into practice. After some discussion the father agreed he would
purchase a car for the son when the son met certain conditions. These included a
variety of things from improved school performance, earning some money
himself, and regular school attendance to cutting his long hair. After some
months the teenager had met all the conditions except for cutting his long hair,
which he cherished. To get around this condition the youngster developed what
he thought was a totally defensible position. He explained to his father that
ancient scholars always wore their hair long. The father responded by pointing
out that the scholars walked almost everywhere they went. This story illustrates
three important points.
1. The same information may be interpreted and used quite differently by two
parties.
2. Individuals typically interpret available data in a manner that serves their

best interest. Often this means they will try to fit new information into the
old way of doing things so as to minimize the change required of them.
220
3. The movement from evaluation to acceptance and use of the information can
easily result in conflict rather than change if steps are not taken to prevent
it.
Recognizing and respecting some guiding principles can facilitate the

utilization of evaluation results and ensure that implementation of the results is
positive and productive.
LINK EVALUATION AND IMPLEMENTATION
Too often teacher education evaluation activities are conducted by one person
or group and when the work is completed they hand the fmdings over to another
person or group. Most institutions would deny that the division between
evaluation and implementation is that pronounced but many faculty and
administrators would concur. When this dichotomy occurs it greatly reduces the
probability that evaluation outcomes will be implemented.
Involve Those Responsible For Implementation
All persons or groups who will be expected to use or respond to evaluation

findings should have input into the evaluation process. This includes faculty and
adrrinistrators who will be expected to implement the findings as well as those
who will be responsible for guiding the implementation effort. Students should
also be involved if they will be in any way influenced by implementation of the
findings.
There are several reasons why this involvement is important:
1. The collective ideas of all these persons will certainly provide for a broader
knowledge base upon which to make decisions than would the ideas of just a
few persons.
2. When the individuals feel they had a sincere, meaningful opportunity for
input and influence on the evaluation and implementation activities they
will have a better understanding of what transpired and why. With this
understanding there should be a few surprises or shocks at findings and
recommendations that emerge.
3. Such involvement carries the potential for enhancing acceptance and

implementation of findings.
Seek Evaluation Outcomes That Address Meaningful Needs
Those who are to use or be influenced by evaluation outcomes are in the

best position to know what kind of information they want or need and this is
another reason for widespread involvement Suppose, for example, one thrust of
an evaluation effort is to determine how well prepared first year teachers feel for
teaching reading and mathematics. If asked, methods course instructors would
221
likely state that they did not have time in one class to fully develop in their
students proficiencies in all aspects of these subjects. Therefore, general
impressions or even quantitative ratings of general statements of proficiency
would have little implementation value. But, if the instructors identified specific
teacher proficiencies they felt they had taught, and if evaluators could determine
how well prepared the teachers felt in those specific areas and why they felt that
way, the faculty would have information upon which they could act in revising
courses.
Use Evaluation to Determine Necessary Changes
To be the most effective, evaluation and implementation should be closely

linked. Evaluation does not begin when the first tests are administered or the
first survey instruments are placed in the mail. Nor does it end when the final
data have been analyzed and prepared for distribution. The process of evaluation
begins when the need for evaluation is conceived and emerges as a verbal or
written description, no matter how informal. Collection, analysis and reporting
of data clearly are the heart of evaluation but they are not the whole of it. After
the reporting is done the process should continue to determine if and how the
information was used and how the content and process of evaluation could be
improved to increase their impact on practice.
Were Changes Implemented and Did They Make Differences?
Assume that an evaluation effort reveals certain weaknesses in the teacher

education program. Recommendations for what should be done to overcome the
weaknesses could be made and that aspect of the evaluation considered completed.
If, however, the recommendations are not accepted as being useful or usable by
the faculty, the evaluation effort has not been successfully completed. On the
other hand, if the recommendations are implemented faculty have a responsibility
to continue the evaluation to determine if the recommendations resulted in the
anticipated outcomes.
DEVELOP AN UNDERSTANDING OF IMPLEMENTATION
Three important points must be understood about the implementation of

evaluation results. Implementation is a process which requires time for
personal acceptance. Each of these points is discussed below.
Implementation Is a Process
More often than not implementation of change is treated as an event rather

than a process that occurs over time (Hall & Hord, 1987; Hord, Rutherford,
Huling-Austin, & Hall, 1987). Change is treated as an event when it is
introduced into the organization with little or no preparation of the intended users
(persons who are expected to change their practice) and there is no facilitation or
monitoring of the change effort. Simple changes may be made easily with
222
minimal preparation or lead time. When the change is complex, however, it

may require much preparation and active support for an extended period.
Allow Time for Change
Should a Dean decide to make a minor change in the form used by faculty to
report final grades, that is a simple change and might be accomplished by a
printed directive (i.e., a memo). A complex change would be the directive,
"Every methods course must incorporate a practicum component so that students
can apply theory and methods as they learn them." Faculty will have many
uncertainties about their own role in the new process and what is expected of
them. Certainly there will have to be a complete and radical revision of course
and grading procedures to accommodate the expectations of the practicum
component. If time allocations for courses are not increased faculty will be faced
with the irritating task of eliminating content they believe to be important.
Should time allocations be increased without increasing teaching load credit
faculty will surely be unhappy. In addition to these problems there will be the
need to arrange for the space, time, materials, students, transportation, and other
variables necessary for conducting the practica and these arrangements will have
to involve public or private schools. In short. this is a highly complex change.
Research has shown that to effectively implement a complex change such as
this one may take several years even under ideal conditions (Hord, Rutherford,
Huling-Austin, & Hall, 1987; Hall & Hord, 1987; Fullan, 1982). If the change
effort is not properly supported and facilitated or if individuals are asked to
implement more than one complex innovation. At the same time, the rate and
effectiveness of implementation will be reduced. (An innovation is any change
that is introduced into an organization.)
Change is Personal
When discussing implementation we often speak of changing a program or a

department or a unit (e.g. student teaching) within a school, or college, or
department of education. Innovations, in most instances, require changes in
individual human beings before there will be program or organizational change.
Individuals should be the focal point of implementation and it should be
recognized that each one responds to and experiences change in a personal way.
To be effective, facilitation of change must attend to individuals and their
concerns rather than treating them as anonymous groups such as "a program" or
"the methods people" or "field supervisors."
To be convinced of the individual differences in adults that might influence
change one has only to review the many theories of adult development. Erikson
(1959), Loevinger (1976), Kohlberg (1973), and Sheehy (1976) are just a few of
the theorists who have attempted to explain or describe various stages of
development that adults experience. Most of these theories relate developmental
stages to age or some general events or situations in life.
223
Stages or Concern
One theory focuses specifically on the stages adults pass through when they
are faced with making a change. The stages of concern in this theory operate
independent of age or other situations in life. In fact, unlike other theories of
development, this one holds that individuals recycle through the stages of
concern with each new change they experience. The Stages of Concern theory
proposes seven stages a person might pass through with each change effort
(Hord, Rutherford, Huling-Austin, & Hall, 1987; Hall & Hord, 1987; Hall,
George & Rutherford, 1977; Newlove & Hall, 1976). The seven stages and a
brief description of each are shown in Figure 1.
Stages Brief Description
(; Refocusing There is concern to do something that will

work even better than the present program
5 Collaboration Concern is about relating what I am doing

with what others are doing
4 Consequence Concern is about the effects of the

innovation on students
3 Management Trying to handle all the materials, time and

logistics of the innovation is the focus of
concern
2 Personal Concerns are related to the influence the

innovation will have on me personally and
professionally
1 Informational There is concern to know more about the

innovation
oAwareness There is no awareness of the innovation or

its possible implementation so there is no
concern about it
Figure 1. Stages or Concern About the Innovation
Although Stages of Concern are seen as developmental, individuals need not

progress through all stages. Also, the intensity of concerns a person will have
at each stage will vary according to the complexity or demands of the innovation
as perceived by the individual. (Remember, reality is what a person perceives it
to be, not necessarily what actually exists.) There are two important points to
capture regarding stages of concern:
224
1. Individuals move through the stages at different rates and

with differing intensity of concerns at each stage; and
2. Movement from one stage to another can be facilitated but

cannot be forced.
Attempts to force movement may impede the development process and, in turn,
implementation efforts.
It is commonly assumed that people are resistant to change, but the theory
of concerns suggests that there is an explainable (and predictable) reason behind
the seeming reluctance to change and thus that people may not be cantankerous
or uncooperative in personal efforts to thwart change.
For a brief insight into the way concerns may influence individual response
to change, consider the matter of faculty evaluation at Steady State College (a
fictitious institution that was created for this book). There is already a rating
instrument in use for obtaining student evaluations of faculty teaching and a
change is proposed in the design of the instrument. Faculty would surely be
concerned about what kinds of changes were being proposed and why (Stage I,
Information). They would also want to know if the modifications in the
instrument would affect in any way decisions to be made relative to their tenure,
promotion and salary (Stage 2, Personal). If the modifications in the instrument
and its use are minor these concerns should not be very intense and could be
facilitated rather easily.
Now suppose that evaluation results suggest a completely new system of
evaluating teaching, one that involves supervisor observations and ratings of
performance. Without a doubt informational (Stage 1) and Personal (Stage 2)
concerns will be quite intense throughout the faculty and responding to these
concerns will require skill, effort and time. Also, Management concerns (Stage
3) will likely become quite intense as faculty try to modify their teaching to
satisfy the requirements of the assessment procedure.
An instrument that is easy to administer and score is available for measuring
Stages of Concerns about an innovation (Hall, George, & Rutherford, 1977).
With this instrument it is easy to collect data that can be applied directly to
implementation. Systematic attention to the personal element in change is
essential and the use of a model such as stages of concern could be the key to
success. Readers are advised to contact the authors for further details about the
instrument.
STEPS TO SUCCESSFUL IMPLEMENTATION
Designate a Change Facilitator
Too often the following scenario happens: from evaluation, the need for a
change is identified and this need is conveyed to a dean or chairperson who then
presents the problem to those persons who will be expected to make the change.
That group is then given a semi-formal charge to implement the needed changes
or a committee will be formed to generate recommendations for how it should be
225
done. What is missing here is the understanding that serving as a change

facilitator is much more than being the person in charge of the unit where the
change is to take place. A change facilitator must fIrst understand all the
ramifications of implementation for individuals and the organization (many of
which are addressed in these principles). In addition to knowledge of change
theory and process, facilitators must have those personal and professional skills
that enable them to effectively work with users. Some of these skills are:
1. being a good listener and communicator;
2. having patience;
3. accepting individual differences in response to the innovation;
4. assisting and supporting rather than forcing; and
5. being willing and able to collect, analyze, and use data that inform the
implementation process.
Finally, the change facilitator must be given the time, resources and support
necessary to carry out the required responsibilities. It has already been
emphasized that the facilitator must be formally identified and assigned the
responsibility. Once this is done the facilitator must be given the administrative
support and the time needed to carry out the responsibilities. This support and
time allocation must be provided throughout the change process, not just at the
beginning.
Few, if any, institutions can afford a full-time change facilitator. This
means the task will likely be given to someone among the faculty or staff,
someone who already has a full work load. If this person is not chosen
carefully, given some training for the role (if not already trained) and given an
ample allocation of time to do the work of change facilitator, both the facilitator
and the change effort are likely to be less than successful.
Provide Administrative Support
Most institutions of higher education continue to be organized hierarchically

and certain individuals by virtue of position, have authority and power that
others do not have. Deans and department heads are obvious examples. When
attempting to implement an innovation it is extremely important that the effort
have the clear and certain backing of the highest authority fIgure who has the
power to influence what does or does not happen. (Hall & Hord, 1987; Fullan,
1982; Huberman & Miles, 1984).
Suppose the elementary education faculty members at Steady State College
decide they wish to employ a new set of standards for entry into the elementary
program. Without the support of the dean of the school of education this change
will probably never occur. Administrative sanction and support is equally, if not
more, important in the case of complex changes than in less complex changes.
226
Consider the situation where evaluation reveals the need for a better blend of
theory and practice in the preservice program. Based on this information a
faculty planning committee recommends earlier field experiences for all students
and the addition of practicum components to many education courses.
Implementation of this very complex change will involve and influence many
people in significant ways and may require additional resources or reallocation of
resources. For all these reasons such an innovation will require that the chief
administrator give it full support and that such support be evident to all.
To be supportive does not mean the chief administrator must be the primary
change facilitator for the innovation. It does mean, however, that administrators
must make clear and evident their support of the change and that they monitor
closely how implementation progresses. Saying, "Let's do it," appointing a
facilitator, and then leaving implementation to take its course is not being
supportive. If the chief administrator is not interested enough in the innovation
to commit his or her own time and effort it is unlikely faculty will have any
greater commitment.
A note of caution is in order here. While administrative support is
essential, it is not sufficient. For implementation to succeed all the qualities of
effective change must be in place along with the support. Also, support should
not be confused. It is improbable that an administrator can cause a faculty or
staff to change through the application of power or force. Resistance to and
resentment of the innovation and the administrator will be the most immediate
outcome of such an attempt.
Intend To Succeed
This principle is closely related to the one above. Organizations that have
experienced successful change are more likely to attempt and to be successful
with implementation of the next change. The reverse is also true. The key
implication of this principle is that no implementation effort should be initiated
unless those in key leadership positions have every intention of it succeeding and
they plan to work toward that end. It is unforgivable for leaders to permit the
initiation of a change if they will not support it actively or, even worse, if they
subvert the effort through inaction or negative actions.
A corollary to this implementation principle is the evaluation principle,
"Plan to use it." Money and effort should not be expended on evaluation
activities that are not intended to inform future actions. Granted, some
evaluation activities are undertaken to meet the requirements of state and national
regulatory agencies but these too can be structured to serve the needs and
interests of the individual institution.
Make Clear What Change Is Expected
A common virus among implementation plans is ambiguity. All too often

an innovation is designed and described in a manner that is so general that it is
unclear. Those who are to implement the change do not know exactly what or
how they are to change and there is no clear vision of what the innovation will
227
look like if implemented as desired. Ambiguity is a barrier to change (pullan,

1982; Waugh & Punch, 1987).
Strengthening the Knowledge Base: An Example of Change
To illustrate the principle let us look at Steady State College, an institution

that has found through evaluation that the knowledge base of teaching needs to
be strengthened. A plan is developed to base those changes on the report from
the Carnegie Forum on Education and the Economy (Shulman & Sykes, 1986).
This work identifies eight categories of knowledge ranging from GenerallLiberal
Education to Foundations of Professional Understanding. The categories and
their descriptions are quite useful but they are not specific enough for direct
transfer to implementation.
Under the Foundations category is the subtopic of professional ethics. Just
what does that mean to a particular institution? What information/
experiences/requirements should be added to the program? How is the addition to
be made? What is expected of the individuals(s) who is charged with delivering
this portion of the knowledge base? To reduce the ambiguity surrounding the
innovation of a strengthened knowledge base, these questions, and others, must
be asked and answered about all elements of the innovation and it should be
done before implementation. [For additional information on selecting a
knowledge base, see Pankratz in this book].
"Innovation configuration" is a term used to describe the procedure for
identifying and specifying an innovation before implementation (Hord,
Rutherford, Huling-Austin & Hall, 1987; Heck, Stiegelbauer, Hall & Loucks,
1981; Hord & Hall, 1987). This procedure not only leads to clarification of the
innovation but it illustrates ideal, acceptable, and unacceptable uses of the
various components of the innovation.
Identify Differences for Current Practice
Innovation configuration also establishes a set of expectations that are clear

to and that facilitates common understanding between users and facilitators. This
enhances the efficiency and effectiveness of both groups as they work to
accomplish the desired changes. All too often during implementation users and
facilitators spend much of their time just trying to understand what the
innovation means and what they are to be doing. This not only slows
implementation, it also generates much frustration and often misdirects the entire
effort.
When innovation configuration is employed one problem will likely
emerge. Users will complain that it makes use of the innovation too restrictive
and this complaint will be valid. The greater specificity for use, the less
flexibility users will have. However, with careful planning there should be
reasonable freedom for users to act even though what they are to do is fairly
prescribed. There is no ideal solution to this problem but, acceptable balance
between innovation clarity and certainty and user flexibility is a reasonable
expectation.
228
Treat Change as A Systemic Process
Any change that is made in one part of the teacher education "body" will
influence other parts of the "body." A change in admission standards may affect
class size and even some teaching assignments. Introduction of new technology
with the intent of increasing teaching effectiveness may create a need for an
equipment maintenance unit that competes for available dollars. Administrative
reorganization designed to expedite program efforts may result in communication
blocks and create new obstacles to efficient management.
Watch For Ripples and Mushrooms
These ripple effects are inevitable, but action can and should be taken to
keep the ripples from becoming waves. This can be done by first taking a broad
view of the proposed change, one that anticipates all possible influences on
individuals and the organization. When this has been done then steps can be
taken to reduce or eliminate negative ripples and maybe even convert them into a
positive force.
Even when initial anticipation is skilled and thorough, once implementation
is underway unanticipated influences are still likely to occur. Most often a
number of small incidents will accumulate until they form what has been termed
a mushroom (after the atomic bomb cloud) by some (Hall & Hord, 1987).
When this occurs the consequences for the implementation effort may be
positive or negative but more often than not they are negative. Change
facilitators must be alert to any suggestion that a mushroom is beginning and
take whatever action they deem necessary to control it before it interferes with
implementation.
EVALUATING IMPLEMENTATION
If change efforts are evaluated at all, typically, they are evaluated only in
terms of their outcomes. While assessment of outcomes is something that
should occur, alone it is inadequate and may be misleading. Before evaluating
outcomes two other critical assessments must be made:
1. It must be determined how many of those individuals who are supposed to

be using the innovation are actually doing so; and
2. It must be determined whether the innovation is being used in an acceptable

manner.
The importance of these types of evaluative data and the ways they might be
collected will be discussed in the paragraphs that follow.
229
Assessing Degree or Innovation Use
In almost every change effort there will be a portion of the intended users
who do not actually use the new program or procedures (Individuals who are
expected to make a particular change to use the innovation and do not are termed
non-users. Hord, Rutherford, Huling-Austin, & Hall, 1987; Hall & Hord,
1987). There may be many reasons why individuals do not implement an
innovation as was intended. These can include lack of understanding of what is
expected, inadequate resources, inability to make the change or outright refusal.
These reasons can be very informative when making decisions about how to best
facilitate implementation but the important point for evaluation is that the
presence of these non-users should not be ignored.
To get a sense of the importance of the use/non-use issue, consider again the
hypothetical innovation of adding a practicum to all methods courses at sse.
Assume that three years after the innovation was introduced, a large group of
students (and former students) participate in an evaluation to determine what
influence the new methods courses had made. A comparison was made of their
responses with those of earlier graduates who had not taken practicum methods
courses. The outcomes indicated no difference between the two groups. This
could have led to the conclusion that the practicum component was ineffective.
But additional information showed that during the first year of implementation
only 35 percent of the methods courses included a practicum. During the second
year this number increased to 55 percent and by the end of the third year it had
risen to 75 percent (when acceptable use of an innovation is the criterion for
implementation these percentages are not atypical for complex innovations).
Given the fact that many of the students classified in the practicum group for the
follow-up evaluation had not even been in a practicum methods course, it would
have been grossly misleading to draw any conclusions about the differences
between practicum and no-practicum students. Simply stated, one cannot
accurately evaluate innovation outcomes without knowing how the innovation
has been used.
One procedure for assessing use of a change or innovation is called Levels of
Use of the Innovation (Hord, Rutherford, Huling-Austin, & Hall, 1987; Hall &
Hord, 1987; Loucks, Newlove & Hall, 1975). Eight discrete levels of use are
proposed. These levels range from "no knowledge of the innovation" and
"effective use of it" to "active searching for innovation improvements." This
procedure can identify those individuals who have not made the expected change
(non-users); those who are using the innovation but in disjointed, uncertain
manner; those who are using it routinely; and those who have gone beyond the
routine to do things to increase the effectiveness of the innovation.
Levels of Use information is most valuable when it is used as a part of
formative evaluation rather than as a summative procedure. Data from Levels of
Use can and should be used throughout the implementation process to inform
and guide the efforts of the change facilitator in personalizing support and
assistance.
230
Levels of Use of the Innovation
Level VI Renewal. State in which the user re-evaluates the quality of

use of the innovation, seeks major modification of or alternatives to present
innovation to achieve increased impact on clients, examines new developments
in the field, and explores new goals for self and the system.
Level V Integration. State in which the user is combining own efforts

to use the innovation with related activities of colleagues to achieve a collective
impact on clients within their common sphere of influence.
Level IV -B Refinement. State in which the user varies the use of the
innovation to increase the impact on clients within immediate sphere of
influence. Variations are based on knowledge of both short- and long-term
consequences for clients.
Level IV-A Innovation. Use of the innovation is stabilized. Few if

any changes are being made in ongoing use. Little preparation or thought is
being given to improving innovation use or its consequences.
Level III Mechanical Use. State in which the user focuses most effort
on the short-term, day-to-day use of the innovation with little time for reflection.
Changes in use are made more to meet user needs than client needs. The user is
primarily engaged in a stepwise attempt to master the tasks required to use the
innovation, often resulting in disjointed and superficial use.
Level II Preparation. State in which the user is preparing for first use
of the innovation.
Level I Orientation. State in which the user has acquired or is

acquiring information about the innovation and/or has explored or is exploring
its value orientation and its demands upon user and user system.
Level 0 Non-Use. State in which the user has little or no knowledge of the
innovation, no involvement with the innovation, and is doing nothing toward
becoming involved.
Assess How the Innovation is Used
Before it can be decided whether individuals are using an innovation it must

first be established what constitutes acceptable use of the innovation. To
illustrate the essential nature of this information assume that three years after the
practicum courses were introduced at Steady State College the following five
patterns of use (innovation configurations) were found among the faculty:
231
Configuration 1 The entire practicum is conducted on the university

campus with students teaching small peer groups
followed by feedback from peers.
Configuration 2 Each week three students from class go into a public
school and teach a lesson of approximately 15 minutes
which is video taped by a fellow student. These tapes
are viewed and critiqued by the entire class during on-
campus class time.
Configuration 3 Students are expected to spend at least 90 minutes per
week for four consecutive weeks tutoring one pupil.
The students must make their own arrangements for this
tutoring activity. There is no on-site evaluation of the
tutoring.
Configuration 4 Every Tuesday and Thursday for ten weeks the entire
methods class goes to a public school where they teach
small groups of pupils (3-5). There is some general
supervision by university faculty.
Configuration 5 After the first three weeks of the semester the methods
class regularly meets in a public school where the
students teach individuals or small groups under
frequent faculty supervision and under clearly stated
guidelines for instructional performance.
Obviously, these configurations represent very different approaches to the

inclusion of practicum experiences into methods courses, so it would be
reasonable to assume that their respective impacts on students would be
different. Consequently, before a meaningful evaluation of innovation outcomes
can be conducted it must first be decided if all, some, or none of the classes have
a practicum experience that meets the original intentions for the innovation
(ideally these intentions were made clear before implementation began through
the development of an innovation configuration).
Which instructors of methods classes are using the innovation and which
ones are not? The answer to this question is not meant to be punitive or to
restrict academic freedom, but until there is an answer it would be a waste of
time, effort, and money to attempt to evaluate the outcomes of this (or any
other) innovation. Before there is any attempt to evaluate the outcomes of the
innovation, there must first be an evaluation of the implementation process to
determine how many users and non-users there are. These figures must then be
included in any data measuring outcomes.
A VOID THESE RISKY ASSUMPTIONS
Change efforts are often influenced by certain assumptions. Many of these

assumptions will be specific to the institution and its particular situation. There
are, however, four assumptions that seem to be common regardless of the
situation. If they are accepted unquestioningly, they can lead to action (or
inaction) that impedes implementation.
232
Participation Develops Ownership
When individuals feel they have had a meaningful role in designing or

selecting the innovation it is true they will probably have a greater sense of
ownership and acceptance of it. Attempts will frequently be made to get as
many of the intended users as possible involved in some way in the change
effort. Certainly this. is commendable but it is risky to assume that mere
involvement creates ownership.
If involvement is to have a positive impact on potential users it must be
meaningful, significant participation. Individuals must feel that they really can
influence the innovation and implementation plans. All too often a subgroup
will do all of the in-depth planning and design and the larger group of intended
users is involved only in information sessions with no real opportunity to offer
suggestions. Under these conditions the larger group will not feel really
involved and will have no sense of ownership of the innovation. In fact, they
may be angered by what they consider to be lack of involvement and interest in
their ideas. Building ownership requires much more than superficial
engagement.
Attitude Change Must Precede Behavior Change
As a change effort gets underway there will be grumbles, groans and puzzled
musings about what is going to happen. These utterances will often reflect what
seems to be a negative attitude in the school or college relative to the
impending change. The negative attitude may not be as real or widespread as it
is perceived to be but a typical response from administrators or facilitators is to
set in motion some actions or efforts to modify these attitudes. The underlying
assumption is that until attitudes are changed there will be no implementation of
the innovation.
It is humane and proper to be concerned about the attitudes of individuals
but this concern should not override common sense. In the first place, so much
time and energy might be spent attempting to adjust attitudes that the
implementation effort is damaged. Secondly, it is not necessarily correct to
assume that attitudes have to change before behavior will change. Guskey
(1986) makes a strong case for the position that often attitudes change after the
users try the innovation and have evidence that it works. This does not mean
there should be no concern for user attitudes, it suggests that one effective way
to change attitudes is to have a good innovation based on solid findings from
evaluation and coupled with a good implementation effort.
Some Change Is Better Than No Change
At the cognitive level there may be few people who would believe in this
assumption but actual practices suggest considerable acceptance of it. Any time
233
personnel in a school or college set in motion a change in teacher education with

less than total effort and commitment to its full implementation they are
demonstrating acceptance of the assumption.
In truth, some good may come from partial implementation of a curricular
innovation; those who use it may benefit a number of students. But this good
may be offset by negative effects. Partial implementation may well lead to a
fragmented program that leaves students uncertain of what is expected of them
and why, and also to friction among faculty members as they hold to their
respective practices. Additionally, it sends a signal to all personnel that the
administration is not really serious about change and improvement in teacher
education. This will make it even more difficult to succeed with the next change
effort.
Even partially implemented changes require the expenditure of time, effort,
resources and sometimes cash. In short, they cost money and it is not easy to
justify such expense if administrative commitment to change is weak or
inconsistent.
Older Persons Are More Resistant To Change Than Younger

Ones
Right away each reader can probably think of several administrators or

faculty whose behaviors shift from an assumption to a truism. Of course, there
are older individuals who seem reluctant to change but there is no evidence in the
change research that supports this as a universal assumption.
One reason for mentioning this assumption is that when it operates within a
school or college it often means that older faculty are not included as much as
younger ones in planning and decision-making because it is felt they might be
obstructionists. When this happens the self-fulfilling prophecy occurs. Because
they are not included, the older faculty believe there is no interest in their ideas
and concerns so they are not interested in the change effort and thus they appear
resistant.
When older personnel are "shut out," or perceive that they are, there is a loss
of those who should and could provide the best leadership and support for
change. Mter all, in terms of rank and promotion they have the least amount to
risk in committing their time to a change effort. Also, they may have better
developed communication channels to all personnel and quicker access to key
persons and resources within and outside the college of education.
It should also be remembered that those who are older are also more
experienced. If at times they seem to be dragging their feet it may be because
experience has identified for them the smooth roads and rough roads to change
and they are trying to direct the effort onto the smooth roads.
This assumption focuses on age but care should be taken to avoid assuming
that any group or set of people will be resistant to or non-cooperative in a
change process. This will only increase the likelihood of the self-fulfulling
prophecy. Should there be any group that appears reluctant to participate, the
appropriate action is to attempt to facilitate their participation. A voiding or
ignoring them is inadvisable.
234
By now the reader has probably concluded that moving from evaluation to
effective implementation of change in teacher education is not a perfunctory task.
This is an accurate conclusion. Yet, there are no viable alternatives. To avoid
change altogether means there will be no improvements in the teacher education
program. To approach change in a half-hearted way or through a series of short
cuts can create more problems than are solved. By far the best solution is to
determine what is required to effectively implement a change and do it. This
means all who are responsible for the evaluation to implementation process
must communicate and collaborate. The results will be personally and
professionally rewarding for all who are involved.
GUIDELINES FOR IMPLEMENTATION OF EVALUATION

OUTCOMES
Link Evaluation and Implementation
Involve in the evaluation process all persons or groups who will have
any responsibility for implementing findings.
Seek from evaluation outcomes that address meaningful needs.
Seek evaluation outcomes that are useful and practical enough to
implement.
Use evaluation to determine changes to be implemented.
Use evaluation to determine if the changes were made.
Use evaluation to determine what difference these changes made.
Develop an Understanding of the Implementation Task
Allow enough time for implementation of the change to occur. If the

change is complex it can require several years to implement fully.
Remember that any change impacts individuals in a personal way. Be
prepared to recognize and respond to these individual concerns and needs.
Steps to Successful Implementation
Designate a person who has primary responsibility for facilitating the

implementation effort. This person should have an understanding of the
change process and the skill to direct it.
Key administrators must make clear their support for the change
through their actions and allocations of resources.
Begin and pursue the implementation effort with the absolute intention
that it will succeed.
At the outset make it quite clear what the change is that is supposed to
be implemented.
Identify what people are expected to do and how that differs from current
practice.
235
Changes in one part of an organization will likely affect other parts.

Anticipate those "ripple" effects and be prepared for them. Ambiguity
can be dangerous.
Evaluating Implementation
Before attempting to assess the outcomes or impact of a particular

innovation, know to what extent it is actually being used by those who
are supposed to implement it.
Determine if the innovation is being used in an acceptable manner.
Avoid These Risky Assumptions About Change
Participation automatically develops ownership.

Attitude change must precede behavior change.
Some change is better than no change.
Older persons resist change more than younger ones.
REFERENCES
Erikson, Erik H. (1959). Identity and the life cycle. Psychological Issues,
1(1),9-171.
Fullan, M. (1982). The meaning of educational change. New York:
Teachers College Press, Columbia University.
Guskey, T. R. 1986). Staff development and the process of teacher change.
Educational Researcher, 15(5),5-12. .
Hall, G. E. & Hord, S. M. (1987). Change in schools: Facilitating
the process. Albany, NY: State University of New York Press.
Hall, G. E., George, A., & Rutherford, W. L. (1977). Measuring stages of
concern about the innovation: A manual for use of the SoC
questionnaire. Austin: The University of Texas as Austin, Research and
Development Center for Teacher Education (ERIC Document Reproduction
Service No. ED 147342).
Heck S., Stiegelbauer, S. M., Hall, G. E., & Loucks, S. F. (1981).
Measuring innovation configurations: Procedures and
applications. Austin: Research and Development Center for Teacher
Hord, S. M., Rutherford, W. L., Huling-Austin, L. & Hall, G. E. (1987).
Taking charge of change. Alexandria, VA: Association for
Supervision Curriculum Development.
Huberman, A. M. & Miles, M. S. (1984). Innovation up close how
school improvement works. New York: Plenum Press.
Kohlberg. L. (1973). Continuities in childhood and adult moral education
revisited. In P. Baltes and K. Schale (Eds.). Life-span developmental
236
psychology: Personality and socialization. New York: Academic

Press.
Loevinger, J. (1976). Ego development. San Fransisco: Jossey-Bass.
Loucks, S. F., Newlove, B. W., & Hall, G. E. (1975). Measuring levels
of use of the innovation: A manual for trainers, interviewers
and raters. Austin: The Research and Development Center for Teacher
Newlove, B. W. & Hall, G. E. (1976). A manual for assessing open-
ended statements of concern about innovation. Austin: The
Research and Development Center for Teacher Education. The University of
Texas at Austin.
Sheehy, G. (1976) Passage: predictable crises of adult life. New
York: Dutton.
Shulman, L. S. & Sykes, G. (1986). A national board for teaching?
In search of a bold standard. New York: Carnegie Forum on
Education and the Economy.
Waugh, R. F., & Punch, K. F. (1987). Teacher receptivity to systemwide
change in the implementation stage. Review of Education Research,
57(3), 237-254.
18
ELEMENTS OF LAW AS THEY RELATE TO TEACHER

EDUCATION EVALUATION
Joan L. Curcio
University of Florida
What legal considerations (and constrictions) do personnel need to be aware of as

they evaluate teacher education programs? One of the major purposes of this
chapter is to provide some answers to that question, as well as to two others:
What legal principles apply in the arena of teacher education program evaluation
and, is there a checklist of legal issues associated with program evaluation which
could be used as a preventive measure against liability?
The thrust of a teacher education program needs to be of a level of quality
"that fosters competent practice of graduates" and "rigorous standards of academic
excellence" (National Council for Accreditation of Teacher Education, 1987, p.
1). To provide a professional education program of such quality, an institution
should have a system of assessing its program and "product" which is rigorous
and valid and which does not violate the constitutional and federal statutory
rights of those involved in the evaluation process. There is always the potential
for public institutions, and under certain conditions private institutions, while
engaged in state action, to be liable for such violations. When a private
institution acts as an agent of government in performing a task, or performs a
function generally thought to be government's function, or obtains prestige,
resources, or encouragement from government contacts, its actions could be
considered state action which is subject to constitutional restrictions (Kaplin,
1985, p. 18). "Public institutions and their officers are fully subject to the
constraints of the federal Constitution" (Kaplin, 1985, p. 2).
How might it happen that in the process of evaluating a teacher education
program an institution, through its agents, would violate someone's
constitutional or statutory right? It would be useful to look at the strategies to
be used in implementing an evaluation model--particularly where they involve
the assessment or evaluation of performance of students or faculty. An
awareness of the legal issues are associated with such activities as faculty
evaluation, candidate selection, performance outcomes, classroom observation,
and the collection and use of data can go a long way to avoiding legal action
through preventive assurance.
238
POTENTIAL LEGAL ISSUES REGARDING EVALUATION
Perhaps a primary legal concern for an institution, when following a model

of teacher education evaluation, is that of assuring due process. Where an
institution is subject to constitutional limits, its actions cannot be contrary to
rudimentary due process or grounded in arbitrariness (Kaplin, 1985, p. 564).
How the rights of due process might be violated in the due process of evaluation
are explored below.
Fourteenth Amendment
A second legal issue that could emerge is the Fourteenth Amendment

guarantee of equal protection under the laws. Individuals have been shielded by
law, both constitutional and federal statutory, from discrimination on the basis
of race, national origin, sex, age, handicap, and religion. Where an evaluation
strategy includes candidate selection, for instance, we would want to assure that
the methods, techniques, and instrumentation used to assess students as they
enter a teacher education program are not biased or discriminatory, and have a
rational relationship to the purpose of selection.
Section 1983
The federal law which will be used to sue the institution or the individual
administrator who may violate a student's or faculty member's civil rights in
employing given strategies of an evaluation model is 42 U.S.C., Sec. 1983. It
reads in part:
Every person, who under color of any statute, ordinance regulation,

custom or usage of any State ... subjects or causes to be subjected,
any citizen of the United States ... to the deprivation of any rights,
privileges or immunities secured by the Constitution and laws, shall be
liable to the party injured in an action of law, suit at equity or other
proper proceeding for redress.
The defense that a college or university administrator would use to respond

to a charge brought under U.S.C. Sec. 1983 of violating someone's
constitutional rights is that of qualified/good faith immunity (Miles, 1987, p.
21). However, acting merely in good faith with ignorance of what constitutional
rights are to be accorded can also merit personal liability for an administrator
(Miles, 1987, p. 23).
The Buckley Amendment and the Right To Privacy
Another area with potential for violating rights, specifically student rights,
is that of privacy. The privacy of students' educational records is guaranteed
239
through the Buckley Amendment, officially known as the Family Educational

Rights and Privacy Act (FERPA), 20 U.S.C.A., Sec. 1232g. The Amendment
applies to all public or private institutions receiving federal funds from the U. S.
Department of Education or whose students receive such funds and pay them to
the institution. While FERPA may have limited application where the issue is
admissions records, there are other provisions of which we need to be aware in
order not to invade student privacy through use of their records in process of
evaluation. In addition to the Buckley provision, developing right-to-privacy
common law is growing out of the states which could protect students against
practices regarding records that are abusive. While state law on privacy varies
from state to state, an institution which is in compliance with FERPA
regulations is likely to be in compliance with specific state laws on the right to
privacy. There are exceptions, such as in information collection practices and
also in the kinds of records that may be kept, since those two issues are not part
of the Buckley Amendment to any extent. For example, Massachusetts has a
law which considers requests for information, or the making or keeping or
records "concerning certain arrests on misdemeanor convictions of students or
applicants," an unfair educational practice on the part of educational institutions--
those would include public and private postsecondary institutions (Kaplin,
1985, p. 359). Another example comes out of a California case in which
students were found to be protected against collection and retention practices
which are clearly abusive and which collect information concerning private
affairs. In the case, the "court held undercover police surveillance of university
classes and meetings violated the right to privacy because 'no professor or
student can be confident that whatever he may express in class will not find its
way into a police file'" (Kaplin, 1985, p. 360). Individual institutions will want
to check state statutes or counsel to determine if such state law exists.
Defamation
It is unlawful under the law of civil defamation for a person to "publish"

orally or in writing information that tends to harm another's reputation and
expose him to hatred, shame, disgrace, contempt, or ridicule. In order to be
defamatory, the statement must have been communicated to a third party. Where
there is evaluation, there is communication. It is important in the process of
sharing information and communicating about students (and employees as well)
that we are speaking in good faith and without malice, within the scope of one's
authority, and in the interests of the parties involved (Alexander & Alexander,
1985). Under those conditions, we are likely speaking with the "conditional
privilege of fair comment and criticism," a defense against the charge of
defamation (Kaplin, 1985, p. 61).
Certainly there are legal issues that can arise in regard to any activities, rules
or regulations, or actions resulting from teacher education program evaluation
beyond the four focused on above. It would seem, however, given the nature of
the activity--evaluation--that these four are the most common, and the legal
questions likely to surface are these:
240
1. Are there "property interest" issues involved (where the evaluation model
includes faculty evaluation, in particular)?
2. Are there liberty interest issues (where information is being exchanged

and/or documented concerning failures, successes, and/or job potential of
individuals)?
3. Are there related defamation issues?
4. Are there invasion of privacy issues?
5. Are there due process issues, where notice of an action or an opportunity to

be heard is due?
6. Has anyone representing the institution acted in an arbitrary or capricious

manner regarding any aspect of program evaluation, in violation of someone's
rights?
7. Are there discriminatory (14th Amendment, equal protection) issues (race,

national origin, sex, age, handicap, or religion) in selection, for'instance, or
evaluation procedures?
SOME SUPPORTING CASE LAW
There is case law supporting the assertion that there are legal aspects
involved in the process of teacher education evaluation. While it would not be
possible in this brief chapter to include all of the decisions which might shed
light on what legal principles are significant here, there are specific cases--
sometimes landmark cases--which should be examined as indicators of when and
how legal errors can be made as part of an evaluation program.
Admission, Selection, Certification
Courts are disposed not to become entangled in matters, particularly

academic, of educational institutions. When suits do occur involving such
matters, the legal principles on which the case is decided can assist us in
appraising how similar future situations will be judged. The following legal
guidelines come from a 1963 New York case in which a student, claiming
statutory protection, attempted to force a state college to admit him:
Courts may not interfere with the administrative discretion exercised by

agencies which are vested with the administration and control of
educational institutions, unless the circumstances disclosed by the
record leave no scope for the use of that discretion in the manner under
scrutiny . . . (Lesser v. Board of Education of City of New
York, 1963).
241
Particularly in the delicate areas of eligibility of applicants and the determination

of reasonable standards, courts refrain from interference unless a statutory
authority has been abused or a discriminatory practice is involved. The
limitation on governmental discretion, then, is that it be used fairly and not
arbitrarily (Lesser v. Board of Education, 1963). A court's interest or
involvement in a case will end "when it is found that the applicant has received
from the college authorities uniform treatment under reasonable regulations fairly
administered" (Lesser v. Board of Education, 1963). Where admission
standards and selection into a teacher education program are ultimately part of the
program evaluation model, the preventive need is to assure that actions taken as
a part of those tasks do not violate a student's substantive rights to be free from
arbitrary or capricious governmental action.
It is absolutely within the purview of the institution to screen potential
teachers for quality and to provide rigorous preparation. Stiffer requirements for
admission are being adopted across the country, and in fact have been mandated
by at least 27 states. These requirements include minimum grade point averages
and tests of academic ability, followed by programmatic requirements (Darling-
Hammond & Berry, 1988, p. vii). A state, through its agent institution, has the
plenary right to govern in these educational matters, but cannot violate legal
rights in the process. Although a person may not have a property interest in
admission and/or selection into a teacher education program, the lack of a
"common perception of what prospective teachers should know and be able to
do" would caution us to be prudent and judicious in our selection processes.
Where a "property interest" is involved, procedural due process rights accrue;
while the right to remain at a public institution has been identified by the court
as such a right, and suspension (Goss v. Lopez, 1975) and expulsion (Dixon
v. Alabama State Board of Education, 1961) cannot occur without such
due process, "what factors should enter into the standards set for college
admission was within the exclusive province of the college authorities" (Lesser
v. Board of Education, 1963).
Testing Issues
By 1986, 46 of the states had mandates to test teacher competency in subject

matter, basic skills, and/or professional knowledge as requirements for admission
to teacher education and/or certification (Darling-Hammond & Berry, 1988, p.
viii). [Refer to Rudner's and McLarty's chapters for more specific details.]
Whether such tests are used to screen applicants for admission to a program or to
judge certification fitness, legal considerations exist. The concerns relate to the
ability of those who use competency tests to assure their reliability and validity.
Tests which have been shown to produce racially disparate outcomes are grist for
lawsuits under the Fourteenth Amendment, as well as under several federal
statutes and their enabling regulations. Consequently, some states have
delegated the implementation of the tests they have mandated. Other concerns
with legal implications have to do with the appropriateness of the content of
these tests, especially where professional knowledge is being tested (Darling-
Hammond & Berry, 1988). [See Griggs v. Duke Power Co., 1971, for a
242
decision that employment standards and tests that are not significantly related to
job performance are violative of Title VII of the Civil Rights Act of 1964.] The
issue of reliability and validity will be discussed again later in reference to
instruments used to evaluate the performance of students and teachers who are in
the teacher education program.
Dismissal From Teacher Education Programs
When students are evaluated as failing to meet the standards and/or

expectations of a program, and dismissal is a consideration, legal issues again
arise. Although the courts, as mentioned earlier, are reluctant to interfere in
educational matters, particularly the academic aspects, individuals do have
protections which cannot be violated. In a case in point, Board of Curators
of the University of Missouri v. Horowitz (1978), the U. S. Supreme
Court considered what procedures were to be accorded to a student at a state
institution whose dismissal might constitute a deprivation of "'liberty' or
'property' within the meaning of the Fourteenth Amendment." The medical
student's academic performance was in question; her performance was alleged to
be "below that of her peers in all clinical patient-oriented settings." The Court
distinguished such academic dismissals from conduct dismissals, saying that the
former case called for "far less stringent procedural requirements."
Under such circumstances, we decline to ignore the historic judgment of

educators and thereby formalize the academic dismissal process by
requiring a hearing. The educational process is not by nature
adversarial. (Board of Curators v. Horowitz, 1978)
However, an appellate court in New York found, in 1980, that where an

institution itself, in this case a private institution, adopts a regulation or
guideline that establishes a procedure to be followed when suspending or
expelling, "that procedure must be substantially observed. The judicially
reviewable question here then would be: did the institution act in good faith in
the action it took?" (Tedeschi v. Wagner College, 1980). The same court,
in the same year, deciding whether or not it might intervene in the decision of an
educational institution, this time to withhold a diploma on academic grounds,
said,
judicial reluctance to intervene in controversies involving academic

standards is founded upon sound considerations of public policy. In
order for society to be able to have complete confidence in the
credentials dispensed by academic institutions ... it is essential that the
decisions surrounding the issuance of . . . credentials be left to the
sound judgment of the professional educators who monitor the progress
of their students on a regular basis. (Olsson v. Board of Higher
Education, 402 N. E. 2d 1302, 1980).
243
While procedural due process may not be a constitutional requirement in

academic dismissal, institutions and their officials are not totally immune from
judicial scrutiny. Courts have indicated that:
1. They will, in fact, intervene should the institution exercise its power in an
arbitrary, capricious, or irrational manner.
2. The implied contract between the institution and the student demands that it
act in good faith in dealing with students (if the student meets the terms, he
gets what he seeks) (Olsson v. Board of Higher Education, 1980).
Both of these indications have implications for student/program evaluation.
Teacher Dismissal
Student achievement evaluation is not the only component of an evaluation

model; faculty performance is also evaluated. Such evaluation also carries with
it legal implications, especially when the staff enjoys tenure, or expressed
legally, a property interest. Where a property interest resides, so does the
requirement of procedural due process. Dismissal of an allegedly incompetent
faculty member requires a formal procedure with full due process safeguards. To
withstand judicial scrutiny, an evaluation process should include minimal
acceptable teaching standards, a fair system capable of detecting and preventing
teacher incompetencies, and a process for informing faculty of required standards
and for correcting deficiencies (Darling-Hammond, Wise, & Pease, 1983, p.
287). Furthermore, if performance assessments are going to be used to make job
status decisions, then courts will likely require that they bear a direct,
demonstrable relationship to "effective job performance, that is, they must have
proven validity" (Darling-Hammond et a1.,1983, p. 287).
LIBERTY INTEREST
Even when a property interest is not the issue, the Supreme Court
recognized in Board of Regents of State Colleges v. Roth (1972) that
damaging an educator's reputation or ability to be re-employed as a result of an
employment action could violate his/her Fourteenth Amendment liberty rights
(whether or not a faculty member has tenure). In other words, the institution has
to avoid actions which would "stigmatize" the employee. Of course, not all
defamation by a public official would convert to a deprivation of liberty.
According to Roth, unless an employer stigmatizes an employee by a public
statement that might seriously damage his/her standing and community
associations, or foreclose opportunities for his/her employment, liberty interests
are not affected. On the other hand, if comments are made in private, even if
false, they cannot be the basis of a liberty violation (Bishop v. Wood, 1976).
Very recent decisions have looked upon this liberty interest as "liberty of
occupation." To "fire a worker to the accompaniment of public charges that
244
make it unlikely that anyone will hire him for a comparable job infringes his
liberty of occupation" (Colaizzi v. Walker, 1987).
Liberty Interest of Students
Students are also protected constitutionally against the violation of their

liberty interests. Those issues can come into focus in a teacher education
program when students are evaluated as not achieving and are suspended or
dropped from the program, or their potential to pursue a program of studies is
interrupted or damaged in some way. In Dixon v. Alabama State Board of
Education (1961), the Court judged that the expulsion of students in that case
exposed them to harm:
There was no offer to prove that other colleges are open to the
plaintiffs. If so, the plaintiffs would nonetheless be injured by the
interruption of their courses of studies in midterm. It is most unlikely
that a public college would accept a student expelled from another
public college of the same state. Indeed, expulsion may well prejudice
the student in completing his education at any other institution.
(Dixon v. Alabama State Board of Education, 1961).
DISCRIMINATORY ISSUES
Discriminatory issues involving staff and students can arise in regard to

teacher education program evaluation. Claims of discrimination within
institutions have been re"iewed by courts under both the federal Constitution and
civil rights statutes. The areas under which school personnel specifically have
been shielded by law from discrimination include: race, national origin, sex,
age, handicap, and religion. Most of these categories also apply to students
under different statutes. All are protected under the equal protection clause of the
Fourteenth Amendment. What are some of the ways in which a discrimination
suit can be triggered in the process of program and/or performance evaluation?
Policies and regulations which have a discriminatory impact on a certain class of
individuals, failure to maintain consistent standards for all involved, not
having a rational job-related basis for what is said and done, and not acting in
good faith can all have legal implications. Liability can be reduced by acting in
good faith without malice, and sharing informed opinions based on trained
expertise. Courts recognize the need of education officials to exercise judgment
and discretion in carrying out their professional duties.
THE BUCKLEY AMENDMENT
The Family Educational Rights and Privacy Act of 1974 (FERPA), known
as the Buckley Amendment, has carved out a role for the federal government
which is substantial in regard to student records. The Act's implementing
regulations are extensive and thereby represent a "predominant legal consideration
in dealing with student records" (Kaplin, 1985, p. 358). Essentially, the Act
245
ensures the confidentiality of students' records and provides that federal funds will
be withheld from an educational institution which:
1. Fails to provide access for parents or students 18 years or older to the

student's educational records, or
2. Disseminates such record information to third parties without permission

(with certain exceptions).
Students have the right to request that information on record be amended if it is

believed to be misleading, incorrect, or violative of the student's rights
(McCarthy & Cambron-McCabe, 1987, p. 93).
There are exceptions to FERPA requirements of privacy--including certain
public directory information, and a teacher's daily records of student progress.
[However, where private notes are shared, even among those who have an
educational need to know, they become "records" and are subject to legal
requirements (McCarthy & Cambron-McCabe, 1987, p. 94]. Additionally,
federal and state authorities may have access to data needed for auditing and
evaluating federally supported education programs, and composite student
achievement information can be made public, so long as individuals are not
identified. Finally, student records must be revealed if subpoenaed by court
(McCarthy & Cambron-McCabe, 1987, p. 95).
There are some guidelines concerning the content of such records. First,
courts have supported the obligation which schools have to record relevant
student data, even if it is negative--the criterion is accuracy. On the other hand,
libel suits have been brought by students for damages against school officials for
recording and/or communicating false defamatory information about them. The
key then to avoiding such suits is prudent recordkeeping and sharing of accurate
information.
SUMMARY REMARKS
The activities, processes, and interactions that occur around teacher education
program evaluation are subject to legal restraints. Those legal restraints are
constitutional, statutory--federal and state, and even institutional. The people
involved with program evaluation and those who support it administratively need
to be aware of the pitfalls and build in processes to prevent uncomfortable and
expensive legal situations. There is no substitute for the assistance and advice of
a good attorney. However, an attorney may be required less if the items on the
following checklist are noted and given attention.
GLOSSARY OF LEGAL TERMS
ARBITRARY AND CAPRICIOUS - not done according to reason or

judgment; nonrational; tyrannical.
246
DEF AMATORY - libelous, slanderous, harmful to one's reputation; a

communication is defamatory if it tends to harm someone's reputation so as to
lower him or her in the community's estimation or if it deters others from
associating or dealing with him or her.
DUE PROCESS - means fundamental fairness; there are two aspects to due
process:
PROCEDURAL DUE PROCESS - when the rights to life, liberty, or

property are being affected, a person is entitled to be heard, and in order to enjoy
that right, must be notified.
SUBST ANTIVE DUE PROCESS - protection from unreasonable and

arbitrary state action.
FOURTEENTH AMENDMENT - Section 1 (in part). No state shall make

or enforce any law which shall abridge the privileges or immunities of citizens
of the United States; nor shall any State deprive any person of life, liberty, or
property, without due process of law; nor deny to any person within its
jurisdiction the equal protection of the laws.
INV ASION OF PRIVACY - violating the right that someone has to be left
alone, and if he chooses, unnoticed; publicizing someone's private affairs with
which the public has no legitimate concern.
LIBERTY INTEREST - "liberty," as used in the U. S. Constitution,

includes freedom to pursue any lawful trade, business or calling (Black's Law
Dictionary); therefore, a liberty interest rises if an employer stigmatized an
employee and jeopardizes his or her opportunities for future employment or
diminishes the individual's reputation.
PROPERTY INTEREST - "a person's interest in a benefit is a 'property'

interest for due process purposes if there are such rules or mutually explicit
understandings that support his claim of entitlement to the benefit and that he
may invoke at a hearing" (Perry v. Sindermann, 92 S. Ct. 2694).
Note. These definitions were based on Black's Law Dictionary by

Campbell Black, West Publishing, St. Paul, MN, and American Public
School Law (2nd ed.), by Kern Alexander and M. David Alexander, same
publisher.
247
PREVENTIVE LEGAL MEASURES CHECKLIST
Safeguards Against Legal Errors

in Teacher Education Evaluation
1. Are college rules and regulations regarding program evaluation fair

and lacking arbitrariness?
_ _2. Have they been reviewed by counsel, as a preventive measure?
_ _3. Are regulations being fairly and consistently followed?
_ _4. Has the faculty had training in the legal implications of program
evaluation?
_ _5. Has the faculty had training in the legal implications of student
evaluations?
_ _6. Are faculty aware of any liabilities?
_ _7. Does the institution follow a formalized evaluation process of

faculty performance?
8. Are due process procedures followed when a teacher's or student's

property or liberty interests are in danger of being violated?
_ _9. Are appropriate regulations regarding the handling of student

information and student records in place?
_ _ 10. Is the opportunity to appeal concerns about information in a

student's records available?
11. Is an appeal process in place?
12. Does any evaluative criteria used bear a rational relationship to

goals? (For instance, if criteria are used to measure a faculty
member's performance, they should have a reasonable relationship to
that performance.)
13. Are the criteria job-related?
14. Are evaluation instruments valid and reliable?
15. Are they fairly scored and administered?
16. Are evaluators/supervisors trained?

248
17. Do any evaluation procedures or processes have a discriminatory

effect or impact?
18. Are faculty well-qualified and trained?
19. When information is shared, is it spoken "within the scope of legal

authority" (among those with a need to know, at an appropriate
place and time, so that communications are privileged)?
__20. Are faculty and students' privacy rights and confidentiality preserved?
__21. Are opportunities for remediation made available to students (and

faculty)?
__22. Are advisors available?
__23. Is any documentation on students or personnel safety and legally

preserved?
__24. Are processes and procedures in the teacher education evaluation

model being used free of bias and discrimination in regard to
handicap, sex, race, national origin, etc.?
REFERENCES
Alexander, K., & Alexander, M. D. (1985). American public school law

(2nd. ed.). St. Paul, MN: West.
Bishop v. Wood, 426 U.S. 341 (1976).
Black, H. C. (1979). Black's law dictionary. St. Paul, MN: West.
Board of Curators of the University of Missouri v. Horowitz, 435
U.S. 78, 98 S.Ct. 948, 55 L.Ed.2d 124 (1978).
Board of Regents of State Colleges v. Roth, 408 U.S. 564, 92 S.Ct.
2701 (1972).
Colaizzi v. Walker, 812 F.2d 304 (1987).
Darling-Hammond, L., & Berry, B. (1988). The evolution of teacher
policy. Santa Monica, CA: Center for Policy Research in Education,
Rand Corporation.
Darling-Hammond, L., Wise, A. E., & Pease, S. R. (1983). Teacher
evaluation in the organizational context: A review of the literature.
Review of Educational Research, 53(3), 287-288.
Dixon v. Alabama State Board of Education, 294 F.2d 150 (1961).
Goss v. Lopez, 419 U.S. 565, 95 S.Ct. 729 (1975).
Griggs v. Duke Power Co., 401 U.S. 424 S.Ct. 849 (1971).
Kaplin, W. A. (1985). The law of higher education (2nd ed.). San
Francisco: Jossey, Bass.
Lesser v. Board of Education of City of New York, 239 N.Y.S.2d.
776 (1963).
249
McCarthy, M. M, & Cambron-McCabe, N. H. (1987). Public school

law, teachers' and students' rights (2nd ed.). Boston: Allyn and
Bacon.
Miles, A. S. (1987). College law. Tuscaloosa, AL: SEVGO Press.
education units. Washington, DC: Author.
Olsson v. Board of Higher Education, 402 N.E.2d 1302 (1980).
Tedeschi v. Wagner College, 404 N.E. 2d 1302 (1980).
19
WE CAN GET THERE FROM HERE
Mary F. Berney and Jerry B. Ayers

Ever since a chance glance at the title of an article caught the eye of one of the
editors some months ago, it seemed that this chapter could have only one
possible title, the one you see here. The article is titled, "Which Way to
Millinocket?" In it, Gideonse (1987) refers to the punchline of a classic example
of Downeast comedy, which reads, (sans dialect), "you can't get there from here."
It is true that education has been in trouble, heading along the path which is
clearly marked with that punchline. It is possible to improve education, but
educators must assume the responsibility for directing those improvements or
state legislatures and governing boards will take that responsiblity upon
themselves, as has been demonstrated in such states as Texas, Virginia, West
Virginia, and California. Evaluation is a crucial part of the reform effort. We
must begin to implement the findings from our research efforts, particularly the
follow-up studies of teacher education graduates.
A Brief Summary of the Book
For the past eleven months we have collected, edited, written, and discussed
materials which address teacher education evaluation. One of the initial goals of
the project from which the book grew was to create a practical guide that could
be used to evaluate a teacher education program. We believe that we have
succeeded; time and the readers/users will tell us if that is true. One of our
regrets is that we were not able to include all of the excellent materials the
authors sent and that we uncovered in various searches. Rating instruments,
annotations, and some graphics were omitted, but references to the instruments
were provided and the substance of the graphics was put into text. One of our
sources of pride is that the materials contained in this book are current. Indeed,
we received word of one organizational change and a copy of the most recent
edition of a referenced book the day before we sent the pages to the publisher.
That information is included either in this chapter or in the appropriate place
elsewhere in the text. We are not in possession of a crystal ball, but we do want
to make some suggestions and to hazard a prediction at this time.
252
As we attended national, regional, and state conferences and met with teacher
education faculties from other institutions, it soon became apparent that
accreditation, particularly the need to meet the new National Council for
Accreditation of Teacher Education (NCATE) standards, was among the primary
concerns of educators. As the result of that realization, we decided to emphasize
the standards wherever possible throughout the book. This was not done to the
exclusion of the standards and policies of other organizations. We determined
that in most instances, a program that meets the new NCATE standards is likely
to meet or exceed the standards and policies of other accrediting or approval-
granting agencies or associations. This book, then, is intended to help teacher
education program faculty meet accreditation and other accountability needs.
In Chapter 1, J. T. Sandefur observed that efforts such as this book and the
related activities which are sponsored by the Center for Teacher Education
Evaluation at Tennessee Technological University are providing much needed
information relative to teacher education program evaluation. A statement he did
not make was that five years after the beginning of the current educational reform
movement, much remains to be accomplished before evaluations assume their
rightful place in teacher education programs. We will make suggestions relative
to that later in this chapter. In Chapter 2 we stated that institutions of higher
education need to develop and implement evaluations of their teacher education
programs, that these evaluations should encompass both formative and
sumllative techniques, and that they must be implemented at reasonable costs.
Tho~e chapters, along with the description of the Accreditation Plus Model,
provide the background, or setting, for the next twelve chapters.
Pankratz, Loadman, and Hearn in Chapters 4, 5, and 12, discuss the
knowledge base, quality controls, and governance. These areas are the basis of a
teacher education program; without strong systems for governance and quality
controls and the presence of a knowledge base it would be extremely difficult to
ascertain how effective a program is. The remaining nine chapters address
separate aspects of teacher education programs, ranging from admissions criteria
through follow-up studies of program graduates and from resources to faculty
evaluation.
In the last section of the book, Berney and Gephart describe, briefly, a few
of the evaluation models or approaches that could be used to implement the
"Plus" of Accreditation Plus. Rutherford provides excellent suggestions for
implementing the results of an evaluation, and Curcio discusses legal aspects of
evaluation that teacher educators should be aware of. The emphasis of this book
has been on the practical aspects of evaluating teacher education programs. The
recommendations it contains were made by scholars who have been involved in
conducting research and development work in the areas they address. At this
point, let us tum to the future. Where do we, members of the educational
community, go from here?
The Future of Teacher Education Evaluation
By this time it is understood that accreditation agencies and governing

bodies are putting increasing demands on institutions of higher education to be
253
accountable. All fifty states have examined their programs and most have made
changes in the requirements for the preparation and certification or licensure of
teachers. Many times, however, these changes were made without a thorough
investigation, through evaluation, of the effects of the proposed changes on
students in grades K through 12 now and in the years to come.
The on-going efforts to develop a national system of licensure for teachers,
the work of researchers at several universities on improved means for evaluating
teachers in clinical settings, and the increased emphasis on general education in
the curriculum are pointing the way to the future. The policies which are
developed as an outgrowth of this work will direct teacher education in years to
come. We urge that the groups continue to monitor their own activities and that
they apply the Joint Committee Standards to evaluate them for utility,
feasibility, propriety, and accuracy. The Joint Committee recently released a
related set of standards for evaluating personnel. These should also be studied
and applied to faculty evaluations.
The Joint Committee on Standards for Educational Evaluators. (1988). The

personnel evaluation standards: How to assess systems for
evaluating educators. Newbury Park, CA: Sage Publications, Inc.
Additional Details
This section is analogous to an attic. It contains all those things that we

wanted to share, but couldn't place anywhere else and those which arrived too late
to be put elsewhere. They include software for use in evaluating student
teaching, the aforementioned organizational change, and relevant publications.
Technological advancements now include the use of computer-supplements
to handbooks on supervising student teachers. One example is referenced below;
we are not endorsing or evaluating this product; we are simply presenting it for
consideration. A combination of text and computer programs for the university
supervisor, the supervising teacher, the instructional supervisor, and the student
teacher have been prepared. The software is currently available for the Apple IT
series (5 1/4" and 3 1/2" disks), the Apple Macintosh, the Commodore-64, and
the IBM PC (5 1/4" and 3 1/2" disks). The materials are available from the
Sycamore Press, Inc. P. O. Box 552, Terre Haute, IN 47808

(812) 299-2784.
Assessment is indeed a "hot" topic. The April, 1989 issue of Educational

Leadership (Volume 46, issue 7) is devoted to the topic of assessment.
Articles describing aspects of testing in California, Connecticut, Georgia,
Illinois, Iowa, Michigan, and Pennsylvania are included. Also included are an
editorial by Arthur Costa and a conversation between George Madaus and Ron
Brandt on the misuses of testing. Educational Leadership is published by
the Association for Supervision and Curriculum Development. The May, 1989
issue of the Phi Delta KAPPAN (Volume 70, number 9) contains a section
on assessment. Standardized tests are the focus of most of the articles. Henry
254
A. Giroux discusses Educational Reforms and President Bush, offering a number

of pointed and worthwhile suggestions for improving education at all levels.
And speaking of President Bush, another late-arriving bulletin came to us
via electronic mail, the proposed budget of the Office of Educational Research
and Improvement (OERI). Educational reforms have been slowly taking place
since 1984, but American students still finish behind those in most other
countries in academic performance and more colleges and universities are forced
to provide remedial education. Some educators believe that Americans now see
that schools must meet certain individual student needs and prepare to answer
difficult questions about such areas as at-risk students, early childhood programs,
levels of literacy, and school management. To address those questions and
improve the overall quality of education, the OERI is requesting funds to conduct
research relative to the Experiments for Educational Achievement program,
library programs, and school improvement, among other activities.
And finally, we learned recently that The Council of Chief State School
Officers has become the new host organization for the New Interstate Teacher
Assessment and Support Consortium. See the Appendix to the text for the
address.
Additional Evaluation Needs
The operation of schools is directly affected by school services personnel.

We have discovered little evidence of work being conducted in the evaluation of
programs for the preparation of school superintendents, principals, supervisors,
curriculum specialists, librarians, psychologists, and counselors. How should
such programs be evaluated? We do not know at this point, but suggest that the
answers should be sought.
Epilogue
The prediction. If evaluation is conducted in a systematic manner, from

the selection of the knowledge base and governance structure through the
evaluation of faculty, resources, and students (from admissions through their
employment as teachers), it will serve the purpose of improving education. If
this book helps in any way to establish evaluation as an integral part of a teacher
education program, it has served its purpose.
REFERENCE
Gideonse, H. D. (1987). "Which way to Millinocket?", American Journal

of Education, 95(2), 309-313.
APPENDIX
Organizations Involved in Program Evaluation
American Association of Colleges for Teacher Education, One

Dupont Circle, N. W., Washington, DC 20036. (202-293-2450)
American Association for Higher Education, One Dupont Circle,
N.W., Washington,DC 20036. (202-293-6440)
American Association of State Colleges and Universities, One
Dupont Circle, N.W., Washington, DC 20036. (202-293-7070)
American College Testing Program. 2201 North Dodge Street, P. O.
Box 168, Iowa City, IA 52243. (319-337-1000.)
American Council on Education, One Dupont Circle, N.W., Washington,
DC 20036. (202-939-9410)
American Educational Research Association, 1230 17th Street, N.W.,
Washington, DC. 20036. (202-223-9485)
American Psychological Association, 1200 17th Street, N.W.,
Washington, DC 20036. (202-955-7710)
Assessment Resource Center, University of Tennessee, 2046 Terrace
Avenue, Knoxville, TN 37996-3504. (615-974-2350)
Association for Institutional Research, 314 Stone Building, Florida
State University, Tallahassee, FL 32306. (904-644-4470)
Association for the Study of Higher Education, Department of
Educational Administration, Texas A&M University, College Station, TX
77843 (409-845- 0393)
Association of American Colleges, 1818 R St., Washington, D.C.
20009. (202- 387-2760)
Association of Teacher Educators, 1900 Association Drive, Reston, VA
22091. (703-620-3110)
Buros Institute of Mental Measurements. 135 Bancroft Hall,
University of Nebraska-Lincoln, Lincoln, NE 68588-0348. (402-472-
6203).
Center for Excellence in Teacher Education, College of Education and
Behavioral Sciences, Western Kentucky University, Bowling Green, KY
42101. (502-745-4662)
Center for Faculty Evaluation and Development, Kansas State
University, 1627 Anderson Ave., Manhattan, KS 66502. (913 532-6011)
Center for the Study of Evaluation, Graduate School of Education, 4405
Hilgaard Ave., University of California, Los Angeles, CA 90024. (213-
825-4321)
Center for the Study of Testing, Evalluation, and Educational
Policy, Boston College, Boston, MA 02215. (617-353-2000)
Council of Chief State School Officers. 400 N. Capitol St., N. W.,
Suite 379, Washington, D. C. 20001. (202393-8161).
Council on Postsecondary Accreditation, One Dupont Circle, N.W.,
Washington, DC 20036. (202-452-1433)
256
Education Commission of the States, 1860 Lincoln St., Denver, CO

80295. (303-830-3600)
Evaluation Center, Western Michigan University, Kalamazoo, MI 49008.
(616-387-5895)
Education Research, Evaluation and Development Centre, The
University of Lethbridge, 4401 University Drive, Lethbridge, Alberta,
TlK3M4, Canada. (403-329-2424)
Educational Testing Service. Rosedale Road, Princeton, NJ 08541.
(609-921-9000).
Institute for Educational Evaluation, Office of the Associate Dean,
College of Education, Tennessee Technological University, Cookeville, TN
38505. (615- 372-3177)
lOX Assessment Associates. 5420 McConnell Avenue Los Angeles, CA
90066. (213-822-3275)
Measurement Incorporated. 1058 W. Club Blvd., Northgate Mall - Office
#4, Durham, NC 27701. (919-286-3231).
U.S. Department of Education, 400 Maryland Avenue, S. W.,
Washington, D. C. 20202. (202-732-3366).
National Association of State Universities and Land-Grant
Colleges, One Dupont Circle, N.W., Washington, DC 20036. (202-778-
0818)
National Center for Higher Education Managment Systems, Drawer
P, Boulder, CO 80302. (303-497-0390)
National Center for Research to Improve Postsecondary Teaching
and Learning, The University of Michigan, 2400 School of Education
Building, Ann Arbor, MI48109. (313-927-1305)
National Center for Research on Teacher Education, Michigan State
University, 116 Erickson Hall, East Lansing, MI 48824-1034. (517-355-
9302)
National Education Association, 1201 16th St., N.W., Washington, DC
20036. (202-833-4000)
National Evaluation Systems, Inc. 30 Gatehouse Road., P. O. Box
226, Amherst, MA 01004, (413) 256-0444.
Southern Regional Education Board, 592 Tenth St., N.W., Atlanta, GA
30318-5790. (404-875-9211)
Teacher Assessment Project, CERAS 507, School of Education, Stanford,
CA 94305-3084. (415-725-1229).
Teacher Education Center, Memphis State University, Ball Education
Building, Memphis, TN 38152. (901-678-2351)
The College Board. 45 Columbus Avenue, New York, NY 10023-6992.
(212-713-8000).
The New Interstate Teacher Assessment and Support Consortium,
Consortium Liasion Office, CERAS, 402 South, Stanford, CA 94305.
(415-723-4972).
257
Journals and Periodicals of Value in Program Evaluation
AAHE Bulletin, American Association of Higher Education, One Dupont

Circle, N.W., Washington, DC. 20036.
Action in Teacher Education, Association of Teacher Educators, 1900
Association Drive, Reston, VA 22091.
American Educational Research Journal, American Educational
Research Association, 1230 17th St., N.W., Washington, DC 20036.
American Journal of Education, University of Chicago, P.O. Box 37005,
Chicago, IL 60637.
Assessment and Evaluation in Higher Education, School of
Education, University of Bath, Clarerton, Down, BA2 7AY, England.
Assessment Update, Jossey-Bass, Inc., 350 Sansome Street, San Francisco,
CA 94104.
Canadian Journal of Higher Education, Canadian Society for the Study
of Higher Education, 152 Slater, Ottawa, Ontario, KIP 5H3, Canada.
Chronicle of Higher Education, Chronicle of Higher Education, 1333
New Hampshire Ave., N.W., Washington, DC 20036.
College Student Journal, Project Innovation, 1372 Santa Cruz Court,
Chula Vista, CA 92010.
College Teaching, Heldref Publications, 4000 Albermarle St., N .. W.,
Washington, DC 20016.
Education, Project Innovation, 1372 Santa Cruz Court, Chula Vista, CA
92012
Educational and Psychological Measurement, P.O. Box 6856, College
Station, Durham, NC 27708.
Educational Evaluation & Policy Analysis, American Educational
Research Association, 1230 17th St., N.W., Washington, DC 20036.
Educational Measurement, National Council on Measurement in
Education, 1230 17th St., N.W., Washington, DC 20036.
Educational Researcher, American Educational Research Association, 1230
17th St., N.W., Washington, DC 20036.
Higher Education, Kluwer Academic Publishers, 101 Phillip Dr, Norwell,
MA 02061.
Journal of Educational Measurement, National Council on Measurement
in Education, 1230 17th St., N.W., Washington, DC 20036.
Journal of Educational Research, Heldref Publications, 4000 Albermarle
St., N.W., Washington" DC 20016.
Journal of Experimental Education, Heldref Publications, 4000
Allbermarle St., N.W., Washington, DC 20016.
Journal of Higher Education, Ohio State University Press, 1050 Carmack
Rd., Columbus, OH 43210.
Journal of Personnel Evaluation in Education, Kluwer Academic
Publishers, 101 Phillip Dr, Norwell, MA 02061.
Journal of Teacher Education, American Association of Colleges for
Teacher Education, One Dupont Circle, N.W., Washington, DC 20036.
258
New Directions for Higher Education, Jossey-Bass, Inc., 350 Sansome

St., San Francisco, CA 94104.
New Directions for Institutional Research, Jossey-Bass, Inc., 350
Sansome St., San Francisco, CA 94104.
New Directions for Program Evaluation, American Evaluation
Association, Jossey-Bass, Inc., 350 Sansome St., San Francisco, CA
94104.
New Directions for Testing and Measurement, Jossey-Bass, Inc., 350
Sansome Street, San Francisco, CA 94104.
Review of Educational Research, American Educational Research
Association, 1230 17th St., N.W., Washington, DC 20036.
ERIC Clearinghouses of Importance in Higher Education

Program Evaluation
ERIC Clearinghouse on Higher Education, George Washington

University, One Dupont Circle, N.W., Suite 630, Washington, DC
20036. (202-296-2597)
ERIC Clearinghouse on Teacher Education, American Association of
Colleges for Teacher Education, One Dupont Circle, N.W., Suite 616,
Washington, DC 20036. (202-293-7280)
ERIC Clearinghouse on Tests, Measurement, and Evaluation,
American Institutes for Research, 1055 Thomas Jefferson St. N.W., Suite
200, Washington, DC 20007. (202-342-5000)
Author Index
Abrami, P. C., 107 Cambron-McCabe, N. H., 245

Adams, R. D., 117, 132, 136, 141, Capie, W., 141, 145
142, 144-145 Carr, W., 80
Akpom, K., 136 Carter, K., 73
Aleamoni, L., M., 107 Caruso, J., 70
Alexander, K., 239 Cashin, W., 114
Alexander, M. D., 239 Castaldi, B., 169-171
Altschuld, 1. W., 121 Caulley, D., 108
Anderson, J., 118 Centra, J. A., ix, 103-105, 107-108
Andrews, K., 69 Chelimsky, E., 136
Andrews, T. E. 41 Clark D. L., 179, 183
Antes, R. L., 54 Clark, P. A., ix, 4, 13, 211-213
Applegate, 1., 71 Cleary, M., 33
Ashburn, E., 81 Clinton, B., 51
Astin, A. 51, 53 Cohen, L. H., 136
Ayers 1. B. , ix, 2, 3, 13, 117, 151, Cohen, P. A., 108
211-213 Cohen, V .•B., 199
Cohn, N., 80
Babbie, E. R., 117-121, 123-124 Coker, H., 141, 147
Babigan, H., 136 Coker, J. G., 141, 147
Banner, D. K., 133 Colbert, 1. A., 81
Barnes, H. L., 74 Cole, S., 111
Bayer, A. M., 111 Collins, E. R., 168
Beck, N., 113 Connelly, M. J., 33
Berdie, D., 118 Cook, W. W., 142
Berk, R. A., 100 Copeland, W. D., 80
Berliner, D. C., 79, 142 Corrigan, D., 185
Bernal, E., 33 Covert, R. W., 136
Berney, M. P., ix, 3, 169, 211 Craig, 1. R., ix, 117, 131-132, 136
Berry, B., 241 Cronbach. L. 1., 22
Bethel, L. J., 7 Curcio, 1. L., x, 237
Birdwell, L. A., 119 Curtis, D., 69
Black, H. C., 246 Curwin, R. L., 80
Blackburn, R. T., 108, 109
Borich, G., 142 Darling-Hammond, L., 241-243
Boser, J. A., 121 Daughdrill, J. H., 3
Boyan, N. J., 80 Davis, J. P., 122-123
Boyer, E. L., 88 Davis, T. M., 122-123
Bradburn, N., 118 Denemark, G., 179-180, 185
Brandenburg, D. C., 113 Derry, 1. 0., 113
Braskamp, L. A., 108, 111 deVoss, G., 70-71, 73
Brenneman, B., 199 Dewey, J., 71
Brookhart, S. M., 119, 122 Diamonti, M., 70
Brophy, J., 86 Doctors, S. I., 133
Brouillet, P. B., 41 Donnelly, F. A., 107
Brown, 0., 70 Dorko, K., 61
Bush, R., 179 Doyle, K. 0., 107
Drummond, R., 73
Callis, R., 142
260
Edwards, A. L., 118 Hathaway, S. R., 142

Egbert, R., 179 Havelock, R. G. 136
Eisner, E. W., 214 Haysom, J., 80
Eissenberg, T. E., 50 Hearn, E. M., x, 151
Ellett, C., 141, 145 Heck, S., 227
Emmer, E. T., 81 Heider, F., 107
Erdle, S., 107 Heller, P., 199
Erdos, P. L., 118-121 Herman, J. L., 213
Erickson, E. H., 222 Hill, 1. C., 170
Ewell, P. T., 10, 117-119 Hogan, R. R., 121
Feistritzer, C., 50 Holly, M. L., 80
Feldman, K. A, 105 Hopkins, D., 80
Festinger, L., 107 Hord, S. M., 7, 132, 221-222, 225,
Fisher, R., 81 227-229
Fitz-Gibbon, C. T., 213 House, E. R., 15, 213-215
Flanders, N. A, 145 Howey, K. R., 72-77
Floden, R. E., 15 Howsam, R., 180, 184-185
Foldesy, G., 170 Huberman, A. M., 225
Freeman, D. J., 119, 122, 136 Hughes, R., 84
Friebus, R., 70 Huling-Austin, L., 221-223, 227,
Froh, R., 114 229
Fry, B., 183
Fuhrmann, B. S., 80 Irving, J., 132
Fullan, M., 222, 225, 227 Isaac, S., 117-119, 121, 123-124
Fuller, F., 70
Jones, D. W., 10
Galluzzo, G. R., 31, 136, 140-141,
147 Kane, M. T., 105
Garfield, E., 110 Kapel, D. E., 50, 53
Garten, E. D., x, 195 Katz, L. 132, 142
George, A, 223-224 Kemmis, S., 80
Gephart, W. 1., x, 4, 13, 211-213 Kerlinger, F. N., 117, 121, 123
Gerard, M. L., 33 Kerr, D. H., 183
Gerber, P., 50, 53 Kessler, S. P., 105
Gideonse, H. D., 24, 33, 251 Keyser, D. J., 62
Giles, F., 179 Koehler, V., 73
Gilmore, G. M., 105, 113 Kohlberg, L., 222
Griffin, G., 72-44 Kottkamp, R., 80
Glass, T. E., 170 Krug, S. E., 62
Gollnick, D., 5 Kryspin, J., 33
Goodlad, J. I., 187 Kunkel, R., 5
Goodrich, T. J., 134 Kurachi, A, 132
Gordon, A C., 133
Gorry, G., 134 Lambert, L., 115
Guskey, T. R., 88, 232 Lanier, D. H., 183
Guthrie, E. R., 109 Leeds, C. H., 142
Leventhal, L., 107, 108
Hall, B., 117 Levine, M., 215
Hall, G. E., 132, 221-225, 227-229 Lilley, H. E., 170
Harper, D., 136 Lin, Y. G., 108
Harrington, G., 70 Lindsey, M., 69
Harris, J., 94 Linquist, J., 136
261
Lisensky, R. P., 10, 117-118 Nutter, N. 179

Little, 1. W. ,179
Loadman, W. E., xi, 43, 73, 75-77, Oppenheim, A. N., 117
119, 122 Olsen, L., 88
Lockhart, D. C., 118 Orr, P. G., 179, 182, 184
Locks, N. A., 62 Otis-Wilborn, A. K., 53
Loevinger, J., 222 Overall, J. U., 105
Lortie, D., 70 Owens, T. R., 215
Loucks, S. F., 227, 229
Lower, M. A., 121 Pankratz, R. S., xi, 23, 31
Patton, M. Q., 136, 213-216
Madaus, G. F., 14, 16, 211, 213 Pease, S. R., 243
Manning, M. L., 194 Perry, R. P., 107
Marchese, T. J., 94 Peseau, B. A., 179, 182-184
Marker, G., 179, 183 Pigott, T. D., 88
Marsh, H. W., 105, 106, 107, 108 Pletcher, B. P., 62
Marshall, C. D., 41 Popkewitz, T., 70
Marshall, J. D., 53 Porter, A. C., 86
Martin, O. L., 122 Provenzo, E., 80
Maslow, A. H., 109 Pugach, M. C., 54
Mathis, W., 136 Punch, K. F., 227
McCarthy, M. M., 245
McCarty, D., 180 Rakow, E., 102
McIntyre, D., 69-70 Raths, J., 54, 132
McKeachie, W. J., 107, 108 Reiff, H. B., 50, 53
McKillip, 1., 121 Reynolds, D. F., 62
McKinley, J. C., 142 Reynolds, M. C., 42
McLarty, J. R., xi, 85 Rieger, S. R., 81
McTaggart. R., 80 Riggs, R. 0., 179, 182
Mead, A., 69 Roberts, R. A., 141, 147
Metzler, L., 111 Rodman, B., 88
Michael, W. B., 117-119, 121, 123- Root, L., 107
124 Rudner, L. M., xi, 49-51, 61
Miles, A. S., 238 Rutherford, W. L., xii, 219, 221-224,
Miles, M. S., 225 227, 229
Miller, P., 53 Ryans, D. G., 141-144
Miller, R. I., 112
Mitchell, J. Y., 62 Sacks, S., 70
Mitzel, H., 42 Salthouse, T. A., 108
Mohanty, C., 132 Sandefur, 1. T., xii, 4, 122, 132, 141,
Monahan, W. G., 179 145
Morris, 1., 70 Sanders, 1. R., 15, 212-214
Morris, L. L., 213 Sanford, J. P., 81
Murray, H. G., 107, 108 Sargent, S. S., 134
Saunders, R. L, xii, 178
Naccarato, R. W., 105 Savage, T. M. 7
Naftulin, D. H., 107 Schade, L., 66
Nash, R., 180, 185 Schlechty, P. C., 53
Newlove, B. W., 223, 229 Schlotfeldt, 1., 172
Nicodemus, E., 33 Schrader, W. B., 111
Nott, D., 70-71 Scriven, M. S., 14, 16, 87, 211, 213
Nowakowski, J., 136 Sears, J. T., 53
262
Sheehy, G., 111 Tom, A. R., 24, 38-39

Shiek, D. A., 144 Tucker, D. G., 108
Shim, C., 53 Turner, R., 71
Short, E. C., 31 Tyler, R. , 214
Shulman, I. H., 81
Shulman, L. S., 24, 227 Van Cleaf, D., 66
Shutes, R., 71 Vance, S. V., 53
Sisson, B. G., 62 Villene, M., 117
Skanes, G. R., 107
Smith, B. 0., 183, 186 Ware, I. E., 107
Smith, D. C., 42 Waugh, R. F., 227
Sommer, R, 131 Webster, E., 15
Stake, R. E., 213 Webster, W. 1., 211, 213, 215
Stiegelbauer, S. M., 227 Wiley, A. L., 171
Stratemeyer, F., 69 Williamson, R. C., 134
Stufflebeam, D. L., 10, 14, 16, 211, Wise, A. E., 243
213, 215 Wisniewski, R., 24
Sudman, S., 118 Wittrock, M. C., 42
Sullivan, A. M., 107 Wolf, A., 215
Sweetland, R. C., 62 Worthen, D. L., 15, 212-214
Swick, K. 1., 194 Wortman, P. M., 134
Swingle, P. G., 134 Wotruba, T. R., 104
Sykes, G., 54, 227 Wright, P. L., 104
Tabachnick, B. R., 70-71, 142 Zeichner, K., 70-72, 74, 142

Taylor, G., 53 Zimmerman, W., 109
Taylor, K. H., 141, 147 Zimpher, N. W., xii, 69-70,72-77,81
Temple, C. M., 179, 182
Subject Index
Academic Profile, 94 purposes, 89-91

strategies, 91-92
Accreditation as evaluation, 4, 211-
213, 217 Basic skills tests, 64
Accreditation Plus Model, 4-5, 13-22, Buckley Amendment, 238-239, 244

69, 103, 178, 211-213, 217, 219
Carnegie Forum on Education and the
Accrediting process Economy, 1-2,187
national, 5, 20, 181-182
regional, 5-6, 20, 181 Carnegie Foundation for the
state, 6, 20, 180-181 Advancement of Teaching, 3
Admissions Case law on evaluation, 240-244

affective testing, 64-65
cut scores, 54 Case studies in evaluation, 215
cognitive testing, 53
instruments, 61-66 Center for Faculty Evaluation and
interviews in, 65-66 Development, 105, 113-114
legal issues, 240-242
screening, 51-65 Center for Teacher Education
Evaluation, 4, 13-17, 21, 66, 69,
American Association of Colleges for 216
Teacher Education, 132
CIPP Model of evaluation, 216
American Association of Higher
Education Forum, 95 Classroom Interaction Analysis
System, 145
American College Research Libraries,
199 Classroom Observation Record, 144
American College Test, 94 Clinical competence, 78
American College Testing Program, COKER observation instrument, 147

94, 119
College Outcomes Measures Program
American Educational Research (COMP),94
Association, 55, 96
Collegiate Academic Assessment
American Library Association, 195, Program (CAAP), 94
198
Commitment to teaching, 64-65
American Psychological Association,
56 Communication system, 143
Assessment of outcomes Council for the Accreditation of

development, 95 Counseling and Related Education
instruments, 92-97 Programs (CACREP), 186
monitoring of, 98
264
Council of Chief State School professional judgment, 14, 212-

Officers, 95 213, 217
systems analysis, 214
Council on Postsecondary transaction, 215
Accreditation (COPA), 5, 20 utilization focused, 215
Critical competence, 78 External Advisory Committee on

Teacher Education and Certification
Defense Activity for Non-Traditional in Mississippi, 187
Education Support (DANTES), 94
External Review Committee for the
Defamation, 239 Improvement of Teacher Education
in Georgia, 179
Discriminatory issues, 244
Facilities
Dr. Fox Studies, 107 accessibility, 171
energy conservation, 174
Education Week, 88 evaluation of 170, 173-176
fmances related to, 188
Educational Assessment Center for handicapped, 171, 175
(University of Washington), 113 instructional space, 174
maintenance, 172-173
Educational Testing Service, 94, 119 planning of, 172, 175
site, 173-174
Evaluation specifications, 170-171
beyond accreditation standards, technology needs, 172, 175
182
implementation of results, 122, Faculty evaluation
136, 143, 148, 219-236 based on student learning, 103-104
legal issues, 237-249 biasing effects, 106
models, 211-218 by administrators, 104
of faculty, 103-116 by colleagues or peers, 104, 109
of field experiences, 75-81 by committees, 109
of governance, 151-168 by students, 104-108
of libraries, 195-210 dimensions, 106
of financial resources, 177-194 formative, 103, 109-110
of graduates, 91-101 history of, 131-132
of physical facilities, 169-176 of research, 110-111
of students, 53-66 of service, 112
of teaching, 109-110
Evaluation models political environment, 133
Accreditation Plus, 4-5, 13-22, 69, self, 108-110
103, 178, 212, 213, 217, 219 summative, 103, 105, 109-110
adversary, 215 utility, 107-108
art criticism, 215 validity, 107
CIPP, 214
decision-making, 214 FERP A, 238-239, 244
discrepancy, 215
goal free, 215 Field-based experiences
literary criticism, 214 attributes of, 72-73
objectives-based, 214 critical features, 73
evaluation of, 75-79
265
integration into program, 73-75 as a process, 221-222

research on, 70-72 change facilitator, 224-225
evaluation of, 228
Financial resources examples, 227-229
for teacher education, 178-180 guidelines, 234-235
checklist, 189-193 linkage, 220
stages of concern, 223-224
Follow-up evaluation, understanding of, 221-224
accountability, 134
advisory group, 143 Innovation
creating a plan for conducting, evaluation of, 229
139-140 levels of use, 229-230
data collection, 143
design of, 125, 137 Institute for Scientific Information
feedback and utilization, 143-144 (lSI), 110
history of, 131-132
items for follow-up questionnaires, Instructional Assessment System,
118-120, 125 (lAS), 112
instrumentation, 141-142, 144-
147 Instructional Development and
management, 137-139 Effective Assessment (IDEA), 105,
planning, 139-140 113
political environment, 133
purposes, 134 Instructor and Course Evaluation
social context, 132-133 System (ICES), 105, 113
Formative evaluation, 4, 21, 45, 103, Instruments for evaluation

109-110, 212 development of, 92-93, 95
new, 94-95
Fourteenth Amendment, 238 risks, 95-96
selection of, 92-93
Goals of teacher education program, types of, 93-94
85-88
Interview forms, 65, 123
Governance of teacher education
programs, lOX, 95
structure for graduate programs,
152 Joint Committee on Standards for
structure for moderate size Evaluation, 10-11, 20, 112, 212,
institutions, 156-158 217
structure for multi-purpose
institutions, 158-161 Kentucky Career Ladder Classroom
structure for small institutions, 156 Observation Instrument, 147
Graduate Program Self-Assessment Knowledge bases, 7, 23-42, 153

Service (GPSAS), 124 criteria, 38-40
definition, 25
Graduate Record Examination, 94 development of, 26-30, 36-37
essential elements, 25-26
Holmes Group, 50, 187 strengthening of, 227
sources of, 35
Implementation of evaluation results taxonomy, 34
266
Legal issues Methods of teaching, 153

checklist, 247-248
in evaluation, 9, 237-249 Model, definition of, 211
terms, 245-246
National Association of State
Liberty interests, 244 Directors of Teacher Education and
Certification (NASDTEC), 5, 20
Library resources in education
bibliographies, 205 National Center for Higher Education
dictionaries, 202 Management Systems (NCHEMS),
directories, 203 119
educational research, 205
encyclopedias, 202 National Center for Research on
grant information, 204-205 Teacher Education, 119
handbooks, 204
indices, 203 National Commission of Excellence
statistics, 203-204 in Teacher Education, 2, 50, 180,
thesauri, 203 183
Library, evaluation of National Council on Measurement in

collections, 200 Education, 56, 96
criteria for, 205-207
financial resources, 188-189, 199- National Council for Accreditation of
200 Teacher Education, (see special
human resources, 200 index)
information transfer, 201
performance, 207-210 National Database on Teacher
program, 201 Education Follow-up, 119-120
services, 201-202
National Education Association, 50
Library Standards
American College Research National Teacher Examinations
Libraries, 199 (NTE), 51, 94, 140
American Library Association,
198-199 Organization of teacher education
general, 196-197 programs, 151
regional associations, 197-198 Outcomes assessment, 8, 97
Mail surveys Outcomes assessment goals

advantages of, 117-118 identification of, 85, 88-89
checklist, 125-128 importance of, 90
conducting of, 120-121 measurability, 90
design constraints, 118-119 prioritization of, 91
follow-up, 118 selection of, 89
limitations of, 118
National Database, 119-120 PersonaVprofessional relationships,
of employers, 122 133
Measurement Incorporated, 95 Personnel resources, 184-188
Meta evaluations, 10, 88, 108 Phi Delta Kappan, 135

267
Physical facilities, (see Facilities) face-to-face interviews, 123

group interviews, 123
Pre-Professional Skills Test (PPST), telephone, 123
64
Teacher Assessment Project, 95
Proficiency Examination Program, 94
Teacher attitudes, 97
Professional judgment approach to
evaluation, 14, 213, 217 Teacher competencies, 68-88
Program approval procedures Teacher education programs

national level, 3-4, 20, 181-182 assessing student performance, 85-
regional level, 5-6, 20, 181-182 102
state level, 6, 20, 180, 181 candidates for, 49-68
dismissal from, 242-243
Program changes, 122, 133, 136, finances for, 177 -194
138, 143 governance of, 151-168
improvement of, 219-236
Program improvement, 135 quality controls in, 43-46
reform movement, 1-4
Program philosophy, 26-28 resources for, 195-210
Psychological tests, 65 Teacher effectiveness, 53
Pupil growth outcomes, 18 Teacher Performance Assessment

Instrument, 145
Purdue Cafeteria System, 105, 113
Teacher supply and demand, 51
Rights to privacy, 238-239
Telephone surveys, 123
Southern Regional Education Board
(SREB), 3, 95 Tennessee Board of Nursing, 186
Stages of concern, 223-224 Test and testing

basic skills, 53-54, 63-64
Student Evaluations of Faculty, 105 commitment to teaching, 64-65
cut scores, 54
Student Instructional Report (SIR), for admissions, 49-51
105, 113 information sources, 61-66
interviews, 65
Student Outcomes Information issues in, 51
Services (SOlS), 125 item bias, 60
mandated teacher, 51-52
Student performance, 85 psychological measures, 65
reporting, 60
Student teaching, 69 standards for, 55
types of, 54-55
Summative evaluation, 4, 21, 212
U. S. Department of Education, 50
Survey techniques
mail, 120-122 Use of evaluation information, 122,
employers, 122-123 136, 138, 143, 219-236
268
Validity in admission testing

content, 58
general, 56
predictive, 58
construct, 59
Western Association of Colleges and

Schools, 181
Index to Standards of the National Council for Accreditation of
Teacher Education (NCATE)
Preconditions, 5, 8, 15-16, 26-28, III. Students, 18, 50-67

50-67, 85-102, 117-168,
237-245 III. A. Student Admission, 8, 18,
51-55, 163, 240-243
I. Knowledge Base for
Professional Education, III.B. Monitoring Progress, 43-
18, 23-42 48, 69-84, 85-102, 140-
147
LA. Design of Curricula, 7,
23-42 III. C. Advisory Services, 43-
48, 163, 184-188
I.B. Delivery of the
Curriculum, 24-28, 43-48 III. D. Completion of Program,
8, 51-55, 89-103, 131-
I. C. Content of the 150
Curriculum--General
Education, 24-28, 85- IV. Faculty, 103-116, 163,
102, 117-150, 154 184-188, 240-243
I. D. Content of the IV. A. Faculty Qualifications and

Curriculum--Speciality Assignments, 184-188
Studies, 24-28, 85-102,
117-150 IV. B. Faculty Load, 163, 184-
188
I. E. Content of the
Curriculum--Professional IV. C. Faculty Development,
Studies, 24-28, 85-102, 110-112, 163, 184-188
117-150, 153
IV.D. Faculty Evaluation, 103-
II. Relationship to the World 116, 184-188, 240, 243
of Practice, 69-84
V. Governance and
II. A. Clinical and Field-Base Resources, 8-9, 151-194
Experiences, 69-84, 103-
116, 141-147 V. A. Governance, 8, 151-168
II. B. Relationship with V. B. Resources, 9, 152, 163

Graduates, 117-150 Personnel, 163, 184-
188
II. C. Relationships with Funding, 177-194
Schools, 24-28, 43-48, Physical Facilities,
103-116 169-176, 188
Library, 188-189

A Practical Guide To Teacher Education Evaluation

Загружено:

Сведения о документе

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

A Practical Guide To Teacher Education Evaluation

Загружено:

Авторское право:

Доступные форматы

A Practical Guide to

Teacher Education Evaluation

George F. Madaus, Boston College, Chestnut

Previously published books in the series:

Distributors for ail other countries:

Library of Congress Cataloging-In-Publication Data

A Practical guide to teacher education evaluation / edited by

ISBN-13: 978-94-010-7634-0 e-ISBN-13: 978-94-009-2512-0

1. Teachers-Training of-United States-Evaluation.

Copyright © 1989 by Kluwer Academic Publishers

Softcover reprint of the hardcover 1st edition 1989

All rights reserved. No part of this publication may be reproduced,

Author Index 259

Subject Index 263

NCA1E Standards Index 269

Jerry B. Ayers is Director of the Center for Teacher Education Evaluation,

Mary F. Berney is a researcher with the Center for Teacher Education

John A. Centra is Professor and Chair of the Higher Education Program,

Paul A. Clark is Director of Teacher Education and Professor of Education at

James R. Craig is Professor of Psychology, College of Education and

Joan L. Curcio is Assistant Professor, Department of Educational Leadership,

Edward D. Garten is Director of University Libraries and Professor, The

William J. Gephart served as Chief of Staff at The Evaluation Center,

Edell M. Hearn is Dean and Professor of Curriculum and Instruction, College

accreditation and the governance of teacher education programs. His current

William E. Loadman is Professor and Coordinator of Measurement,

Joyce R. McLarty is Director, Assessment Programs, Test Development

Roger S. Pankratz is Associate Dean for Instruction and Professor of

Lawrence M. Rudner is founder and President of LMP Associates and

WiUiam L. Rutherford is Director of Placement and Associate Professor of

J. T. Sandefur is Dean and Professor of Education, College of Education and

Robert L. Saunders is Dean Emeritus, College of Education, Memphis State

Nancy L. Zimpher is an Associate Professor of Education, Department of

American's ability to compete in world markets is eroding. The

1. To remind Americans, yet again, of the economic challenges pressing us on

2. To assert the primacy of education as the foundation of economic growth,

4. To point out that a remarkable window of opportunity lies before us in the

Carnegie Forum on Education and the Economy. (1986). A nation

National Commission of Excellence in Education. (1983). A nation at

BACKGROUND FOR TEACHER EDUCATION PROGRAM

Jerry B. Ayers and Mary F. Berney

o strengthening the general (liberal arts) education for prospective teachers,

o developing of five year programs leading to initial licensure,

o developing alternative routes to licensure for liberal arts gmduates, and

o developing and implementing total field base programs operated within a

Staff at the Center for Teacher Education Evaluation at Tennessee

effective, functional systems that can be used to evaluate graduates of

BACKGROUND FOR THE BOOK

The basic tenet of the Accreditation Plus Model is that if a program is

graduates after they enter the profession). By undertaking other evaluation

The National Council for Accreditation of Teacher Education (NCATE) is

Regional accrediting associations are an outgrowth of the efforts of the

o Middle State Association of Colleges and Schools

Regional accrediting agencies are responsible for the approval of a total

The Program Accreditationl Approval Process

The process is basically the same whether an institution is seeking program

Planning is the key to success in conducting a program

CONTENTS OF THE BOOK

Part II is a practical guide to the evaluation of programs for the preparation

Quality Controls in Teacher Education Programs

Throughout a teacher education program there is a need for quality controls

overview of the techniques that can be used to detennine if an acceptable level of

Candidates for Teacher Education Programs