1

2012-06-22 Open Response to the University of Regina Strategic Plan for Teaching and Learning [SPTL] as drafted by University Committee on Teaching and Learning [UCTL] (February, 2012).

Dear University Committee on Teaching and Learning, Fellow Colleagues, and the Broader University Community: We, the undersigned from the Faculty of Education, endorse the following list of suggestions, comments, and supplementary materials based on: Current practices and scholarship in teaching and learning, including our own 2010 Criteria Document Many discussions within the Faculty of Education (including those made at an open working meeting as response to the University Strategic Plan for Teaching and Learning held on May 24, 2012 with many members of the Faculty of Education), Discussions with the broader university community The UCTL Town Hall of June 21st, 2012.

Preamble: It is reassuring to hear the UCTL state that teaching is of the utmost importance and thus ought to be supported and recognised in criteria documents, performance reviews, and promotion and tenure decisions. Suggested modifications: Here are several suggestions to be incorporated into subsequent drafts of the SPTL:

1. The SPTL document should state at the very onset that teaching, along with research and community service, is one of three pillars of the Universitys mission, as does the Strategic Research Plan (2010-2015).

2. As well, the SPTL Vision section should be reworded to state that The University of Regina will continue to be, and increasingly become, known for the quality of undergraduate and graduate instruction it offers. (p. 1). As it currently reads is potentially insulting to the many good teachers and faculties and departments already recognised as such. 3. Reporting should remain at the faculty and department levels and not centralised in the Provost and AVP (Academic) offices, nor centralised to a university-wide committee. Centralising the responsibility of evaluating teaching represents a diminution of the deans powers as well as faculty autonomy. It should remain the responsibility of each faculty to support teaching and to engage in constructive dialogue with other faculties in order to further such a goal. One possibility might be to have each faculty organize a meeting to get members' input into how they want to be supported to improve their teaching practices, and how teaching should be evaluated / recognized in performance reviews. The Centre for Teaching and Learning should continue to act as a facilitator to dialogue about effective teaching and share scholarship, best practices, and related matters.

4. Teaching assessment and recognition should not be framed through a surveillance lens, but rather as a formative endeavour to be supported by central administration, faculties, and departments. Two examples of such surveillance framing from the draft of the SPTL that ought to be changed are: 1. On page 4 where it states Mentors will visit faculty members classrooms once per year during their probationary period and once every three years thereafter; the choice of classroom will be at the members discretion but the timing will be random. (p.4), and 2. On pages 5-6 where it states the University Committee on Teaching and Learning will be charged with sampling course syllabi and assignments from courses with unusual grade distributions... in cases where the committee believes that instructors are expecting too little or too much of students, it will work with the relevant deans and department heads to effect changes in those expectations (pp.5-6). The latter statement may even be contrary to academic freedom and our Collective Agreement.

5. The logic (and optics) that the Centre for Teaching and Learning (CTL) is better served by a faculty associate, rather than a director, does not reflect the stated importance of teaching to the universitys mission and reputation. Furthermore, the explanation that the CTL is better served by a hands-on faculty associate is not paralleled by the research pillar which has a vice-president, an associate vicepresident, as well as several directors. 6. As supported by the significant body of research and scholarship and as recognised by the Center for Research on Teaching and Learning at the University of Michigan: Just as there is no simple system for evaluating the quality of faculty research, there is no simple system for evaluating the quality of faculty teaching. However, by thinking carefully about the purposes of evaluation, and by crafting multiple methods of evaluation that suit those purposes, one can devise evaluation systems that are reliable, valid, and fair. Equally important, the process of discussing and crafting evaluation systems focuses attention on the practice of good teaching and helps to create a culture in which teaching is highly valued....The most important consideration in teaching evaluation, both for improvement purposes and for personnel decisions, is the use of multiple methods of teaching evaluation involving multiple sources of data.... some sources of data for evaluating teaching [are]: Students, colleagues, and self-reflection... (entire document included in Appendix A and submitted as an exemplary model to be followed) (http://www.crlt.umich.edu/tstrategies/guidelines.php, formatting and bold added) 7. Further to the evidence guidelines presented in point 6, the Faculty of Education has already struggled with, and been successful in, developing evidence of teaching and related commitments. They are articulated on pages 24-25 of our 2010 Criteria Document and included in Appendix B below as further guidelines to be modeled. 8. An important component that should be added to student assessments of faculty teaching ought to be an assessment of the students own learning and learning processes. 9. The university should support innovative teachers and teaching by providing funding for novel approaches to teaching practices, including but certainly not limited to, field learning components, team-teaching designs, open-access initiatives, and other experimental additions and modifications to teaching activities. 10. The SPTL should recognize that teaching and learning is manifest in a variety of ways extending beyond direct engagement with students in classrooms.

11. The SPTL should recognise teaching and learning as a complex, dynamic endeavour involving the interplay among teacher, learner, and subject matter in specific and diverse contexts and over time. The SPTL should state the importance of encouraging ourselves and our colleagues to engage in a diversity of ontological perspectives in conceptualising, enacting, and supporting teaching and learning. 12. Conversations and research exploring the connections among race, gender, and perceptions of good teaching should also be initiated and acted upon.

Many thanks for your continued work in the development of the University of Reginas Strategic Plan for Teaching and Learning.

Respectfully,

Signatures from the Faculty of Education: Name Marc Spooner Jennifer Tupper Andrea Sterzuk Larry Steeves Ann Kippling Brown Charles Seaman Lace Brogden Patrick Lewis Rick Ast Alec Couros Paul Hart Xia Ji Carol Schick Ken Montgomery Warren Wessel Wanda Lyons James McNinch JoLee Blackbear Rod Dolmage Marilyn Miller Fatima Pirbhai-Illich Carol Fulton Cindy Hanson Janice Huber Kathleen Nolan Rhonda Nelson Jerry Orban Michael Cappello Douglas Brown Valerie Mulholland Twyla Salm Position Associate Professor Associate Professor and Associate Dean Associate Professor Associate Professor Professor Associate Professor Associate Professor Associate Professor Sessional Instructor Associate Professor Professor Assistant Professor Associate Professor Assistant Professor Associate Professor Assistant Professor Professor and Dean Assistant Professor Professor and Associate Dean Associate Professor Associate Professor Associate Professor Assistant Professor Associate Professor Associate Professor Sessional Instructor Field Placement Manager Lecturer Assistant Professor Associate Professor Associate Professor

Appendix A Guidelines for Evaluating Teaching Center for Research on Teaching and Learning at the University of Michigan: http://www.crlt.umich.edu/tstrategies/guidelines.php

Guidelines for Evaluating Teaching

Introduction
Just as there is no simple system for evaluating the quality of faculty research, there is no simple system for evaluating the quality of faculty teaching. However, by thinking carefully about the purposes of evaluation, and by crafting multiple methods of evaluation that suit those purposes, one can devise evaluation systems that are reliable, valid, and fair. Equally important, the process of discussing and crafting evaluation systems focuses attention on the practice of good teaching and helps to create a culture in which teaching is highly valued.

Some Principles of Teaching Evaluation

1. Multiple methods. The most important consideration in teaching evaluation, both for improvement purposes and for personnel decisions, is the use of multiple methods of teaching evaluation involving multiple sources of data. 2. Faculty, departmental and school responsibilities. To ensure that the evaluation system adopted is credible and acceptable, faculty members must have a strong hand in its development. Before departments and schools adopt teaching evaluation systems, the faculty members should determine their criteria for effective teaching. Departments and schools can then take responsibility for developing their own evaluation methods and evaluation criteria. Since different disciplines require different methods and settings for instruction, they require different methods and criteria for evaluation. This is also true for interdisciplinary instruction. Teaching evaluation systems can be flexible to accommodate diversity in instructional methods (e.g., lecture, discussion, lab, case study, small group interaction, practicum, studio, field work, clinical work, etc.). To promote compatibility within the university, standards should be reviewed, understood, and accepted by all groups involved in the promotion and tenure review process. 3. Individualizing teaching evaluation. Effective teaching evaluation must be individualized. A uniform system

discriminates against some individuals, so a plan sensitive to individual variation should be developed. A faculty member should provide information about his/her contributions and accomplishments as a teacher on a longitudinal basis over his/her teaching career. Consideration can then be given to changes in emphasis and interest that will naturally occur in an academic career. 4. What may be assessed. Teaching evaluation has as its central element the assessment of the quality of classroom instruction. Since teaching includes activities broader than classroom instruction, evaluation of teaching must assess more than classroom performance. While departments and schools may identify additional items, among the teaching activities that may be assessed are the following: 1. quality, amount, and level of classroom instruction (including shared instruction) 2. development of curricula, new courses, and classroom materials; 3. supervision and mentoring of graduate students, including chairing of dissertations; 4. service on graduate examination and dissertation committees; 5. one-on-one consultation with students, including supervision of independent study and readings courses; 6. supervision of teaching assistants in undergraduate courses; 7. conduct and supervision of laboratory instruction; 8. supervision of undergraduate and graduate research; 9. advising students in the major; 10. supervision of field work; and 11. supervision of clinical and practicum experiences.

Some Sources of Data for Evaluating Teaching: Students, Colleagues, and SelfReflection
1. Students: Multiple Methods 1. End-of-course rating forms and written comments. Generally, students are able to report on the extent to which a teacher appears prepared for class sessions, communicates clearly, stimulates interest, and demonstrates enthusiasm and respect for students; research shows that student responses on these dimensions are valid and reliable. Generally, students are less able to judge the knowledge of the instructor or scholarly content and currency of a course. When using student ratings for personnel decisions and teaching improvement, institutions often include the following among their guidelines: a. Questions about instructors and courses should be relevant. They should fit the instructors and courses being evaluated.

b. Multiple sets of ratings of faculty courses over time should be considered; personnel decisions should be influenced only by ratings from several courses over several terms. c. Because global ratings of the teacher or course tend to correlate higher with student learning than do more specific items, personnel decisions should rely more on global items (e.g., "Overall, this is an excellent course." "Overall, the instructor is an excellent teacher."). d. Comparative data (such as departmental, school, or institutional norms) should be provided so that individual evaluations can be interpreted within a meaningful context. For example, information about course characteristics (e.g., disciplinary field, class size, required/elective, lower division/upper division, etc.) should be considered when reviewing evaluation results. e. When results from student evaluation forms are used in personnel decisions, it is essential that standardized procedures for administering the forms be followed. Procedures should indicate who will distribute, collect and return questionnaires; when the evaluations should take place; and when the evaluation results will be made available. f. Student rating results should be considered in personnel decisions only when most of the students in a class have completed the surveys. g. The use of optional items chosen by the instructor customizes and makes the forms more useful for teaching improvement purposes. h. Rating forms should include open-ended questions so that students can write their own comments. Written comments are particularly helpful in improving classroom performance. i. A knowledgeable colleague or teaching improvement consultant should be available to discuss evaluation results with individuals in order to help them interpret scores, provide encouragement, and suggest teaching improvement strategies. 2. Alumni letters and surveys. Many institutions request information from recent alumni (e.g., those who graduated two years ago and/or five years ago). Alumni have a perspective for evaluating both individual faculty members and the department's program. Alumni have the additional advantage of being able to judge the relevance of course work to their present situation. It should be noted, however, that information from alumni may do no more than agree with present students' assessment of teaching; studies have found alumni ratings of faculty correlate highly with those of current students. 3. Focus-group interviews, exit interviews, and surveys of students. Focus-group interviews and "exit interviews" may be used to provide information about faculty members and courses for personnel decisions and to strengthen a department's program. Interviews can provide a depth and breadth of information, elicit unanticipated responses, and allow for clarification of student satisfaction and concerns. Focus-group interviews, exit interviews, and surveys of graduating students are especially helpful in strengthening a department's program.

4. Mid-course and periodic student feedback. Feedback from students throughout the term is particularly helpful for teaching improvement purposes. Faculty may ask students to provide informal assessments of their teaching effectiveness at mid-semester by means of focus-group interviews with teaching consultants or through the use of student rating forms, especially ones that include open-ended questions. Throughout the term, faculty also may invite students to comment informally -- perhaps by e-mail or by writing short evaluations at the end of a class period. Mid-course feedback should not be used for summative evaluation unless an instructor chooses to include the feedback in a teaching dossier. 5. Evaluation of student learning. Throughout the term, faculty members may act as "classroom researchers," gathering measures of student learning in order to improve their teaching. Faculty may also wish to provide examples of student learning as evidence of their teaching effectiveness for personnel decisions. 2. Colleagues: Peer Review In most institutions, faculty and administrators have relied on student ratings of teaching effectiveness for teaching improvement purposes and for personnel decisions. Now, however, surveys about how teaching is evaluated on college and university campuses demonstrate an increase in use of faculty colleagues as raters of teaching effectiveness. Colleague review of teaching can play as significant a role as does peer evaluation of research. Colleagues who have expertise in the discipline being taught and training in what to observe can provide important evaluative information through classroom visits and review of course materials and instructional contributions. For a faculty member engaged in interdisciplinary instruction, evaluation may involve colleagues with expertise in similar interdisciplinary instruction and/or with expertise in each of the individual disciplines represented by the faculty member. 1. Evaluation of classroom teaching -- Colleagues can provide important evaluative information through classroom visits. In particular, a colleague's observation of such aspects of teaching as appropriateness of materials and methods, breadth and depth of material covered, the relation of such material to the syllabus and goals of the course, and incorporation of recent developments in the discipline can offer a more informed appraisal of the instructor's mastery of content than can students' perceptions. There is consensus that peer observation has enjoyed more success as a strategy for teaching improvement than for personnel decisions. When used for personnel decisions, it is important to have explicit criteria by which colleagues make evaluations. A standardized observation form will yield systematic and comparable data, especially if participating faculty are trained in what and how to observe. The evaluation process is enhanced when, prior to classroom visits, colleagues review the syllabus and course-related materials and discuss course goals and class

10

objectives with the instructor. 2. Evaluation of course materials -- Colleagues can evaluate course materials, such as syllabi, textbooks, handouts, assignments, graded exams, graded papers, etc. In the visual and performing arts, colleagues may evaluate faculty-directed art exhibits, theater and dance productions, musical ensembles, and individual performances when these activities are directly related to a faculty member's instructional activities. Examination by colleagues offers several advantages: It properly uses faculty expertise, can be done in a reasonable period of time, and can be done anonymously (just as is done with peer review of research). It is also appealing because it can be used for both personnel decisions and for teaching improvement purposes. 3. Evaluation of instructional contributions -- Colleagues may be in the most advantageous position to evaluate such teaching-related activities as curriculum development, supervision of student research, participation in colleagues' and teaching assistants' teaching development, articles on teaching in disciplinary journals and other publications, and authorship of textbooks and other instructional materials.

3. Self-Reflection: Teaching Dossiers The development of a teaching dossier (or portfolio) is a method that allows individuals to collect and display multiple sources of information regarding their teaching effectiveness for examination by others. It contributes both to sound personnel decisions and to the professional development of individual faculty members. A dossier is a "factual description of a professor's major strengths and teaching achievements. It describes documents and materials which collectively suggest the scope and quality of a professor's teaching performance" (Seldin, 1991, p. 3). The purpose of the dossier will drive decisions about format and content. The purpose will also guide decisions about what materials will be reviewed and by whom. There is no single prescription for how a teaching dossier should be structured or what specific information it should contain. Each unit will need to decide what is important and relevant. Units might want to consider including information in the following three areas: 1. The background of the faculty member. The dossier may contain reflective statements by the faculty member on the development of and changes in his or her teaching philosophy, strategies, and objectives; efforts to evaluate and improve teaching and changes resulting from having done so; ways in which he or she has kept up with the professional field in areas related to teaching performance; and his or her future teaching goals.

11

2. The environment in which the faculty member works. For example, the faculty member may describe his or her current expectations regarding distribution of effort among teaching, research, and service activities; include a list of classes taught; discuss important details about these classes that may affect teaching, such as class size and the characteristics, abilities, and motivations of the students; and provide a list of other teaching-related responsibilities and accomplishments. 3. Elements regarding the faculty member's teaching process. The faculty member may provide the following: a. samples of teaching materials, such as course syllabi, laboratory assignments, and videotapes of classroom teaching; b. samples of student learning, such as exams, papers, projects, slides of student work, etc.; and c. the faculty member's reflections about the samples of teaching and learning materials. For example, a faculty member may comment on the reasons for curricular revisions; innovations or experiments with teaching methods or course structure; how and why a particular course's syllabus has changed from one year to another; why specific exam questions were chosen or specific assignments suggested; and ways in which students are provided feedback on exams and assignments.

Concluding Remarks
Evaluation of teaching is not a science; there is still much to learn. However, as indicated in this brief set of guidelines, there is already a considerable body of knowledge about teaching evaluation. The academic community has a strong incentive to add to that knowledge since we will not be able to recognize and reward teaching adequately until we craft a better system for evaluating it.

Selected Bibliography
Braskamp, Larry A.; Brandenburg, Dale C.; & Ory, John C. (1984). Evaluating teaching effectiveness: A practical guide. Beverly Hills, CA: Sage. Centra, John A. (1993). Reflective faculty evaluation: Enhancing teaching and determining faculty effectiveness. San Francisco: Jossey-Bass. Cross, K. Patricia; & Angelo, Thomas A. (1993). Classroom assessment techniques: A handbook for college teachers (2nd ed.). San Francisco: Jossey-Bass. Edgerton, Russell; Hutchings, Patricia; & Quinlan, Kathleen. (1991). The teaching portfolio: Capturing the scholarship in teaching. Washington, DC: American Association for Higher Education. Marsh, Herbert W. (1984). Students' evaluations of university teaching: Dimensionality,

12

reliability, validity, potential biases, and utility. Journal of Educational Psychology, 76, 707-54. Seldin, Peter & Associates. (1990). How administrators can improve teaching: Moving from talk to action. San Francisco: Jossey-Bass. Seldin, Peter. (1997). The teaching portfolio (2nd ed.). Bolton, MA: Anker Publishing Company, Inc. back to top CRLT University of Michigan 1071 Palmer Commons 100 Washtenaw Ave. Ann Arbor, MI 48109-2218 Phone: (734) 764-0505 Fax: (734) 647-3600 Email: crlt@umich.edu Copyright 2011 The Regents of the University of Michigan Discrimination Policy Best viewed @ 1024 X 768 UM Gateway | Non-

13

Appendix B

A. Evidence of Teaching and Related Commitments:

May include:

documentation of the number and type of classes taught, new class preparations, the relative difficulty of preparing a class outside of a faculty members normal expertise, and the significant time and creative effort required to design and deliver distance education courses; the results of assessments of student learning; the results of formal student evaluation of the course and instructor; the results of peer evaluation; evidence of participation in faculty development activities; creation of independent studies and directed readings; evidence of sustained mentoring activity to colleagues within the Faculty; documentation of the number of students advised, type of advising, and time spent in University advising sessions; description of unique geographic and social contexts of teaching/learning; integrating international/global competencies in various teaching/learning contexts; documentation reflecting the level of informal advising that occurs in an ad hoc fashion; documentation of undergraduate and graduate supervision; documentation of the number of graduate supervisory committees chaired, types of committees, membership on graduate supervisory committees within and outside of the Faculty; honors and awards for teaching; a. other activities, not mentioned, that are clearly consistent with the scholarship of teaching principles outlined in this document. (These may include, for example, sharing knowledge or understandings in oral presentations, creating relationships between numerous local sectors or countries around global issues such as HIV/AIDS, the development of summer institutes or conferences in response to emergent needs within the local and broader community, and so on.)