Values-engaged Evaluation

Jennifer GREENE, University of Illinois at Urbana-Champaign

VALUES-ENGAGED EVALUATION
Webinar Series on Equity-Focused Evaluation Jennifer C. Greene University of Illinois at Urbana-Champaign

Welcome
Importance of this conversation on equity in evaluation Values-engaged evaluation Case examples Discussion

On valuing in evaluation
Evaluation intrinsically involves judgments of quality Evaluation is thus inherently imbued with values But values are rarely named or claimed in evaluation

On valuing in evaluation (cont)

Evaluation that is Utilization-oriented Accountabilityfocused Impact Culturally responsive Democratic Values advanced Use Accountability Causal knowledge about outcomes Context, respect for difference Social justice, equity, voice Structural critique

Critical

On valuing in evaluation (cont)

Where do values show up in evaluation planning and practice? Everywhere Most importantly:
Purpose and audience, and intended use(s) Key evaluation questions Criteria for judging quality Communication and reporting (form and content) The social relational aspects of the evaluators role in context

Values-engaged evaluation
Values-engaged describes the plurality of stakeholder values Values-committed prescribes engagement with particular values inclusion and equity Inclusion of all legitimate stakeholder standpoints Equity of access, participation, and accomplishment for all, especially those least well served in that context

Values-engagement in practice
Evaluation purpose and audience
Enhanced contextualized understanding of promising practices, especially for those underserved Primary audiences of local stakeholders, including intended beneficiaries

Intended uses
Learning Program improvement (greater equity)

Practice (continued)
Key evaluation questions
Appropriate questions on design quality, implementation quality, and outcome attainment (each inclusive of multiple standpoints) Question on equity of program access, experience, and accomplishment
In what ways and to what extent has the program served to advance the interests and well being of those least well served in this context?

Practice (continued)
Criteria for judging quality ~ 3 domains
Quality of program design Contextual power of the program design Advancement of the interests of underrepresented and underserved groups Equity in program access, experiences, and accomplishments

Practice (continued)
Communication and reporting Evaluator role and relationships

Example 1: Hillside High Math Program

Context: Regional high school serving multiple rural areas Purpose: This evaluation aims to investigate the structure and content of the math pathways at Hillside High, with a focus on equity of access to and learning opportunities in mathematics across the whole diversity of the student body, toward deeper program understanding and improvement.

Hillside High (cont)

Evaluation questions: How and with what rationale are math pathways at Hillside High structured? In what ways does the structure support equitable opportunities to learn across the diversity of the schools student population? Criteria: A good public high school math program affords all students meaningful opportunities and instruction needed to succeed.

Hillside High (cont)

Communications and reporting

Lower level math classes

On the whole, students in these classes were disconnected, disengaged and/or disruptive. The atmosphere of the classroom was one of getting through the class with as little trouble as possible. Student disruptions were continuous and constant, and as such, a considerable amount of class time was spent on the teacher making repeated and persistent efforts to discipline acting-out students, telling them to sit in [their] own seats, stop talking, focus and/or pay attention

Higher level/honors math classes In general, the majority of the students in these classes were well behaved, attentive and fully engaged, which in turn brought about interactive and collaborative-competitive class atmosphere. Most students were highly attentive to the days lesson... For the most part, the teacher had no problems eliciting students participation in the class, or getting them to stay on task and complete the assignments

Example 2: WWW Access

Context: A hypothetical technology education program designed to enhance the individual and collective high-tech capacity of rural residents and their villages in selected Central American countries. The program is funded by the Gates Foundation.

WWW Access (cont)

Purpose and audience: Assess service to those most in needed, include diverse perspectives, toward deep program understanding and equitable program enhancements

WWW Access (cont)

Quality criteria considerations:
1. Quality of program design basis in relevant theory and research Contextual power of program design showing up meaningfully in peoples lives Equity of access, experience, and accomplishment

2.
3.

For further information

Greene, J.C., Boyce, A., & Ahn, J. (2011). A valuesengaged, educative approach for evaluating educational programs: A guidebook for practice. University of Illinois. (available in the American Evaluation Association elibrary, http://www.eval.org)
jcgreene@illinois.edu

Discussion .

THANK YOU!