Practice: Develop a theory of action for your program

Key Action: Develop relevant evaluation questions based on your

logic model.

TOOL: Generating Questions for Three Types of Magnet Evaluation

Purpose: Reviewing the three different types of evaluation helps you use the outputs
and outcomes from your logic model to develop the evaluation questions
most relevant to your program needs.

Note: This chart refers to ideas in the website section Set the Stage—
including the tool “Considering Two Types of Evaluation”—to further define
what stakeholders want to know about the magnet program. At the same
time, it addresses critical questions that may emerge from your logic model.

Instructions: 1. Review the different types of evaluation and the sample questions
drawn from various magnet program evaluations and reflect on how
appropriate the questions might be in your context.

2. Record the intended outputs and outcomes from your program’s logic
model and use these components to generate evaluation questions relevant
to your program needs.

1
 Practice: Develop a theory of action for your program
Key Action: Develop relevant evaluation questions based on your
logic model.

Generating Questions for Three Types of Magnet Evaluation

Relevant components Possible questions
Evaluation type Sample evaluation questions
from logic model for your evaluation
Implementation • To what degree are teachers using the strategies acquired in List the outputs (activities
– How is the program the science-based math professional development to plan and participants) from your
being implemented? and deliver lessons? magnet program logic model:
– Are all participants • Are all students receiving program services as planned?
being reached as How often? What is their level of engagement and
intended? satisfaction?
– What is the quality • Was the magnet theme integrated into core subjects as
and extent of often as anticipated? Were magnet-related lessons and
implementation? courses documented and made available to all teachers?
How many new courses were produced?
• Which recruitment activities have been successfully
implemented? Which target populations are being reached
through these activities?
• What is the role of magnet staff in supporting
implementation?
Outcome Evaluation • Do students who receive Limited English Proficiency List the outcomes (short-
– To what extent are services and participate in magnet programs meet academic term to long-term) from your
the program’s goals and language acquisition targets? magnet program logic model:
met? • Do magnet students have more knowledge of marketable
– For whom are they vocations and career pathways?
being met?
• Do recruitment activities create a magnet school applicant
– What are unintended
pool that reflects the diversity of the general student
outcomes?
population in the district? For which schools?
Impact Evaluation • Do students attending magnet schools make greater NOTE: Determine if impact
(a form of outcome achievement gains than similar students attending or rigorous evaluation design
evaluation that uses conventional schools? What is the magnitude of the is feasible in your context
rigorous evaluation difference in achievement gains? before developing impact
techniques to • What is the relative impact of magnet school attendance evaluation questions using
determine the impact) when compared to other relevant factors influencing your logic model.
– What are the net achievement?
effects of the program? • How do perceptions of school life (e.g., student perceptions
– To what extent can of academic expectations, school attachment, school social
changes be attributed support, and family support for school) differ between
to the program? magnet and non-magnet students?

2
Practice: Develop a theory of action for your program
Key Action: Develop relevant evaluation questions based on your
logic model.