TRENT UNIVERSITY

INTERNATIONAL DEVELOPMENT 4220

ASSESSMENT OF DEVELOPMENT PROJECTS
2012-2013

Instructor: Chris Beyers, CC-K3

Secretary: Dana Gee, CC-H11, 748-1339
Office hours: Tuesday 1:00 to 1:50 pm, Thursday 15:00 -15:50

A. GENERAL OUTLINE AND INFORMATION

1. General Description. An examination of methodology in the evaluation of
development projects and programs, and of selected case studies. Case studies
will include Canadian projects for development assistance in the Global South.
Introductory lectures and weekly seminar.

Objectives of the course are:

a. To introduce the techniques and methods required in assessments or

evaluations.
b. To develop a critical understanding of the normative bases and
methodological implications of different approaches to assessment and
evaluation.
c. To provide opportunities and exposure to actual evaluation studies by:
(i) studying evaluation programs implemented by governmental and non-
governmental organizations;
(ii) hosting or visiting development consultants and agencies;
(iii) undertaking to conduct an exercise in the evaluation of an
organization.

FORMAT LECTURE-SEMINARS: TUESDAYS 9:00 to 11:50 a.m., CC

E1.2

FIELD TRIP: All students will be required to participate in a two-day

field trip to Ottawa in early January to visit governmental organizations, non-
governmental agencies and private consultants that are involved in assessing
development projects. Students will be responsible to arrange for lodging,
breakfast and dinner (depending on interest we could arrange for a group to stay
at the International Hostel, which is the least expensive lodging). There is a $75
field trip fee to cover transportation and other expenses, although this fee
may be increased depending on the costs of transport.
 2. ASSIGNMENTS, EXAMINATIONS AND GRADING

Individual students will be expected to make presentations and lead group discussion in
seminars during the first part of the course. Students will complete two short assignments
one during each term related to exercises applying material discussed in class. There
will be a three-hour test during the December exam period.

Students will prepare a Placement Agreement and Ethical Review Form in the first term
in preparation for their placement in the second term. In pairs or groups of no more than
three, students will do an evaluation of a project implemented by a local organization. It
will thus be a joint effort in which students will negotiate with the clients the terms of
reference for the assessment, design the research, conduct the data gathering, prepare the
results and write the final report. The report is to be no longer than 35 pages at 1.5
spacing (excluding references and appendices), beyond which I will not mark.

The design, implementation and report of the evaluation must be carried out within the
terms that apply to the guidelines for research on human subjects; each design must
receive approval by the IDS Program Ethics Committee. The completed Application for
Research on Human Subjects should be submitted to the Instructor by the last week
of the first term, and any revisions must be completed by first week of the Winter
Term at the latest.

Grades for the course will be determined on the following basis:

Presentation ...10%
Participation ...10%
Exercises (2) ......5%, 5%
Test 25%
Placement agreement, ethical review ....5%
Evaluation Assignment ......40%
-Mark is to approximately broken down as follows:

Design .5%
Data Collection and Analysis..20%
Final Report and Reporting to Clients
and Community...15%

Academic Integrity Policy:

Academic dishonesty, which includes plagiarism and cheating, is an extremely serious
academic offence and carries penalties varying from a 0 grade on an assignment to
expulsion from the University. Definitions, penalties, and procedures for dealing with
plagiarism and cheating are set out in Trent Universitys Academic Integrity Policy.
You have a responsibility to educate yourself unfamiliarity with the policy is not an
excuse. You are strongly encouraged to visit Trents Academic Integrity website to learn
more: www.trentu.ca/academicintegrity.
Access to Instruction:
It is Trent University's intent to create an inclusive learning environment. If a student has
a disability and/or health consideration and feels that he/she may need accommodations
to succeed in this course, the student should contact the Disability Services Office (BL
Suite 109, 748-1281, disabilityservices@trentu.ca) as soon as possible. Complete text can
be found under Access to Instruction in the Academic Calendar page 14.

Please see the Trent University academic calendar for University Diary dates, Academic
Information and Regulations, and University and departmental degree requirements.

Last date to withdraw from Fall-Winter term full courses without academic penalty in
2012-13 is February 7, 2012.

4. TEXTS AND OTHER READINGS

Required and recommended readings not available through the librarys electronic
resources will be available to students through the Bata Library reserve system.

B. TOPICS AND READINGS

September 11 - INTRODUCTION

September 18 - WHY DO WE EVALUATE IN DEVELOPMENT?

Projects Presentation: Marjorie MacDonald, TCCBE

Bamberger, Michael, 2000, The evaluation of international development programs: a

view from the front, American Journal of Evaluation, Winter 2000, pp. 95-102
(Available through TOPCAT).
IPDET Handbook, 2009, Module 1: Introduction to Development Evaluation, pp. 12-25,
37-45. Website: http://www.worldbank.org/oed/ipdet/modules/M_01-na.pdf
Schwartzman, Stephen, January 2000, The World Bank and Land Reform in Brazil

Begin reading: Patton, M. 1990, Chapter 1, The Nature of Qualitative Inquiry, in

Qualitative Evaluation and Research Methods, 2nd Ed., Sage, pp. 9-34 (reserve)

September 25 DATA GATHERING TECHNIQUES # 1

Video: The Interview Game: Body Language

Finish reading: Patton, The Nature of Qualitative Inquiry (on reserve)

Patton, M, 1982, Thoughtful Questionnaires in Patton, Practical Evaluation, Sage,

139-159.
B. L. Berg 2001. A Dramaturgical Look at Interviewing in Qualitative Research
Methods for the Social Sciences. Allyn and Bacon, pp. 66-98
October 2 - THINKING METHODOLOGICALLY: ETHICAL AND POLITICAL
CONCERNS

Schwandt, Thomas, 1997, Reading the `problem of evaluation in social inquiry in

Qualitative Inquiry, Vol. 3., #1, pp. 1-16
Bellah, Robert, 1983, Social Science as Practical Reason. In Callahan and Jennings
(eds.), Ethics, the Social Sciences and Policy Analysis, Plenum P, pp. 37-68.

October 9 EVALUATION STRATEGIES # 1

Patton, M, 1990. Chapter 2, Strategic Themes in Qualitative Inquiry in Qualitative

Evaluation and Research Methods, 2nd Ed., Newbury Park: Sage, pp. 35-63.
Stake, R., 2004, Excerpt from Chapter 1, Criterial and Interpretive Evaluation, pp. 1-
23, in Standards-Based and Responsive Evaluation, Sage

Begin reading Patton, M., 1990, Chapter 6, Fieldwork Strategies and Observation
Methods (especially 199-213, 224-238, 250-274), in Qualitative Evaluation and
Research Methods, 2nd Ed., Sage.

October 16 - PLANNING YOUR EVALUATION / DATA COLLECTION

Video: Planning Data Collection, 59 min.

Finish reading Patton, Fieldwork Strategies and Observation Methods

Owen, J M, 1999, Chapter 2, The Nature of Interventions: What We Evaluate, and
Chapter 3, Focusing Evaluation: Evaluation Forms and Approaches in
ProgramEvaluation: Forms and Approaches, Sage, pages 22-62.

October 23 - Reading Break

October 30 - DATA ANALYSIS

Patton, M., 1990, Chapter 8, Qualitative Analysis and Interpretation, in Patton,

Qualitative Evaluation and Research Methods, Sage, especially pages 371-379,
384-400, and 428-432.
Silverman, David, 1993, Chapter 7, Validity and Reliability, in Interpreting Qualitative
Data: Methods for Analyzing Talk, Text and Interaction, Sage, pages 144-170

November 6 - SURVEYS AND QUESTIONNAIRES; INTERVIEWS # 2

(Due: Exercise One and Draft of Placement Agreement)

B. L. Berg, 2001. Focus Group Interviewing, in Qualitative Research Methods for the
Social Sciences. Allyn and Bacon, pp. 111-130
Paul Nichols, 2002, Choosing the sample, in Social Survey Methods: A Fieldguide for
Development Workers, Oxfam, pp. 50-72

November 13 - EVALUATION STRATEGIES # 2

Debate between Schwandt and Lipsey in American Journal of Evaluation.

Spring-Summer 2000, Vol. 21, Issue 2, pages 207-230:
-Lipsey, Mark. Meta-Analysis and the Learning Curve in Evaluation Practice
-Schwandt, Thomas A. Meta-Analysis and Everyday Life: The Good, The Bad,
and The Ugly.
-Lipsey, M. Method and Rationality Are Not Social Diseases.
-Schwandt, T. Further Diagnostic Thoughts on What Ails Evaluation Practice
Weiss, Carol, 1972, Design of Evaluation, in Weiss, Evaluation Research, Prentice
Hall, pages 60-91.

November 20 - THE LOGICAL FRAMEWORK, RBM AND CIDA EVALUATION

POLICIES (Due: Draft Ethical Review Form)

CIDA Reference Guide, 2007. Results-based Management in CIDA: An Introductory

Guide to the Concepts and Principles. In Voluntary Sector Capacity
Development Workshop: Promoting Effective Partnerships with CIDA
Jean Christie, November 2008. Evaluation of Development Action Why? of What?
for Whom? The Challenges of Evaluation, as seen by some Canadian CSOs.

Additionally, refer to:

CIDA, 2004, CIDA Evaluation Guide, Website:
http://www.acdi-cida.gc.ca/CIDAWEB/acdicida.nsf/En/EMA-218132515-PMZ

November 27 - CASE STUDY: SRI LANKA

Geoffrey Cameron and Rachel Yordy, 2006, Making Progress? A Case Study of the
WUSC Plantation Communities Project in Sri Lanka.
CIDA, 2004, CIDA Evaluation Guide

December 4 PRESENTATION OF EVALUATION RESULTS

Morris, Lynn Lyons et al. 1987. How to Communicate Evaluation Findings, Newbury
Park: Sage, pp. 7-45, 77-89.
Boulmetis, John and Phyllis Dutwin, 2005, Evaluation as a Business in The ABCs of
Evaluation: Timeless Techniques for Program and Project Managers, Jossey-
Bass, 167-185
B. TOPICS AND READINGS (continued) - WINTER TERM
(Dates subject to change upon majority class decision)

16-18 January 2013 - OTTAWA FIELD TRIP

We plan to leave from the Sadlier House at 5 pm on Wednesday afternoon, and return to
Peterborough Friday night at around 9 pm. See above for details.

PLACEMENTS:

Having completed the Placement Agreement and Ethnical Review, implementation of

your evaluation plans will begin in the first week back in January. You are encouraged to
be in regular communication with the Trent Centre for Community Based Education
(TCCBE) and myself about the progress of your placement. Please continue to refer back
to the readings in the first term as you progress through the various stages of your
evaluation.

26 February 2013 PROGRESS REPORTING

We will reconvene as a class for three hours, in order to take the opportunity to discuss
your respective placements as a group. Each pair will conduct a 10 to 15 minute
presentation on the progress made in the placement relative to your placement agreement,
after which we will have a general discussion in order to troubleshoot any problems and
highlight common issues. The focus on this session will be on methodological decisions
in your placements in order to complete your projects most effectively.

2 April 2013 FINAL PRESENTATIONS

This will be our last class, and will run an hour longer than the normally scheduled time.
Each pair will present the key findings and recommendations for of their evaluation. You
may see this both as an opportunity to receive any feedback before you submit your
report, as well as a practice run of your final oral presentation to your host organization.

TBA DUE DATE FOR FINAL DRAFT OF YOUR PLACEMENT REPORTS

I will determine the final grade for the report based on this draft. I will also provide
detailed commentary, suggestions, and corrections, which you will need to consider and
incorporate in preparing the final version of the report that will be presented to host
organization. You will need to make three copies of this final, revised report for each of
the three stakeholders the host organization, TCCBE, and myself.

Although I will have determined your final grade for the course, I will not submit
these grades to the Registrars Office until I receive the final, revised version of your
report.
University Undergraduate Degree Learning Expectations

a. Depth and breadth of knowledge

The course

1. Building upon the broad conceptual and empirical foundations of earlier core courses,
IDST 4220Y engenders a focused and critical understanding of the normative bases and
methodological implications of different approaches to assessment and evaluation. This is
the heart of the course. The approach taken is that methods are not merely instruments to
be randomly deployed, but tools for producing very specific kinds of raw data required to
address a conceptual research question, tools that thus need to be custom designed to this
purpose. Students are presented with different research strategies, and the kinds of
epistemologies that underpin them. Challenges arising in the course of a particular
research project are viewed as methodological problems to be dealt with in a principled
way, according to ones starting assumptions about how to best to conduct research.

2. Provides students with a strong foundation in the techniques and methods required in
assessments or evaluations. Students are instructed in practical topics such as: negotiating
terms of reference and ethical standards with evaluation stakeholders and independent
regulatory bodies; data-gathering techniques such as interviews, questionnaires, focus
groups; forms of qualitative and quantitative data collation and synthesis; and how best to
present results.

3. Provides students with opportunities and exposure to actual evaluation studies by: a)
studying evaluation programs implemented by governmental and non-governmental
organizations; b) hosting or visiting development consultants and agencies, and c)
undertaking to conduct an exercise in the evaluation of an organization. This final
component consists of a evaluation placement throughout the second term, and is the
culmination of the course. It enables students to apply what was learned during the first
term in a real world setting.

b. Knowledge of methodologies
Students are presented with a range of advanced methodologies through lectures and
required and recommended readings. Methodology is defined not only as the application
of methods, but as a normative rationale for their use in a sustained research strategy.
Students are required to team up to develop a viable research question on a topic of their
choice, and to develop research instruments consisting of interviews and questionnaires
to address this question. This in turn prepares them for their evaluation placement in the
second term.

c. Application of knowledge
During their 12 week field placement in the second term, students apply the methods
learned in the first term by carrying out an evaluation of a project or program within an
organization in Peterborough, Haliburton, or City of the Kawartha Lakes. This requires
completing a Placement Agreement with their host organization, in which they negotiate
a research plan and terms of reference for their placement. They also go through an
Ethical Review process at Trent.

d. Communication skills
By conducting presentations on assigned reading material and on their research progress
and results (including to the host organization and at the Community Innovation Forum:
Knowledge in Action event), students are able to develop their oral communications
skills. Students prepare a lengthy final report that presents their findings, analyses, and
recommendations in a manner sensitive to the needs of the host organization, and to the
context within which the report will be put to use.

e. Awareness of limits to knowledge

By doing research within an applied setting and under time and resource constraints
students learn first hand about the difficulties in producing valid and reliable information
that is socially useful. Students are encouraged to acknowledge the limitations of their
research, and to frame their ongoing research and presentation of results as constructively
as possible in light of these limitations.

f. Autonomy and professional capacity

The course is specifically directed at providing students with professional experience
towards pursuing a career in monitoring and evaluation, or to provide them with specific
skills relevant to other careers. Students are called upon to take initiative in developing a
suitable research plan with their host organization, to work collaboratively to deliver on
this plan within definite time constraints, and to produce an extensive research report
detailing their findings, analyses, and recommendations. Students are encouraged to
cultivate their skills in time management, task organization, leadership, and social
interaction all of which are critical to the success of students projects.