Вы находитесь на странице: 1из 9

Evaluation Plan

By: A. Bush, C. Abel, and H. Grimes

I. BACKGROUND

Program Description
Introduction
The Saturn Learning Technology Initiative (SLTI) is a strategic initiative to support
Saturns K-12 schools so that every public school student will graduate from high school,
globally competitive for work and postsecondary education and prepared for life in the 21st
century. The Saturn Learning Technology Initiative (SLTI) has provided all 7th and 8th grade
students and their teachers with laptop computers, created a wireless internet infrastructure in all
of Saturns middle schools, and provided teachers and staff technical assistance and professional
development for integrating laptop technology into their curriculum and instruction. The concept
of the Saturn Learning Technology Initiative (SLTI) began with a vision of the former Governor.
He believed that if Saturn wanted to prepare its students for a rapidly changing world, and
wanted to gain a competitive edge over other states, it would require a sharp departure in action
from what it had done in the past.
In late 1999 a one-time State surplus of general funds provided the opportunity to act
upon his beliefs. He proposed that all middle school students and teachers be provided laptop
computers. In the summer of 2000 the Legislature and Governor convened a Joint Task Force on
the Saturn Learning Technology Endowment and charged the task force with conducting an in-
depth examination of the issues surrounding Governors proposal, and to recommend the best
course for Saturn to follow.
In the spring of the 2002-2003 academic year, the first full implementation phase of the
SLTI began. In this first phase, over 17,000 seventh and eighth graders and their teachers have
received laptop computers, paid for by the initiative. At the same time with the first deployment
of the laptops, Saturn initiated a professional development program to assist teachers in
integrating the laptops into their curriculum and instruction. Teacher training through
professional development was believed to be the leading reason for the successful
implementation of the laptop program.
In 2006 the Department of Education issued a second phase of the initiative to update and
refresh each of the Saturns schools. Then in 2009 the SLTI expanded into high schools giving
the schools the option to participate and ordered more than 70,000 personal learning devices. By
2014 school year the Saturn Learning Technology initiative (SLTI) has giving over 75,000
personal learning devices to schools in the State of Saturn.
While the most visible component of SLTI was providing a personal learning device for
every student and teacher in grades 7th -8th, the Initiative also addressed education, technology
infrastructure, policy, professional development, community engagement, funding, and
organization as necessary components of a workable model for supporting future-ready students.

1
Need

The program is needed in an effort to leverage technology to improve student learning


experiences and to develop proficiency in learning targets from the Saturn Learning Results. All
students and teachers involved, no matter the socio-economic area of the school, will receive
technology and have access to what all these instruments can afford. Teachers will also take part
in numerous professional development opportunities throughout the year and over the summer.
These opportunities are meant to grow teachers' abilities to teach with the aid of technology.

Target Population

80,000 students and teachers in the Saturn school district from grades 7-12. The framework is
intended to support teachers, tech leads, librarians and other school leaders in their efforts to
leverage technology to improve student learning experiences.

Objectives/Goals

The program's objectives/goals are as follows:


To provide free professional learning services.
To implement a three-day summer institute deep dive to offer specific
learning opportunities for teachers' professional growth.
To provide college and career readiness standards to students.
To increase student engagement in the classroom setting.

II. EVALUATION PURPOSE

The purpose of this evaluation is both summative and formative. This evaluation will assess
both the merit and worth of the program and will seek program and organizational improvement.
Millions of dollars are utilized to purchase the 1:1 technology and to supply ongoing professional
development programs to support this program. Assessing the merit and worth will help the
evaluators and stakeholders get answers to questions like what are the effects of the program? In
particular, are teachers integrating best practices and is the quality of student education
increasing? Considering the costs, is having a 1:1 technology program worth it the money being
spent?
The SLTI program seeks to improve the educational attainment and achievement of all
participating students. Assessing the program to seek out areas of improvement and also to
document the areas of strength will be beneficial to all involved so that they may consider ways
to do their jobs better. Having access to the assessment results will provide invaluable
information so that the district and school leaders will know how to continue to effectively run
the program in an effort to meet the goal of increasing student achievement.

2
III. USE OF EVALUATION FINDINGS

Understanding the information your stakeholders need from your evaluation and how the
findings will be utilized are important steps in constructing a sound evaluation plan.
Understanding Stakeholder Needs

Identify what your stakeholders need to learn from the evaluation.

Primary stakeholders interested in the evaluation findings will include teachers, students,
parents, administration and district personnel. Other stakeholders that will have an
interest in the evaluation findings include the state governor, state department of
education, district superintendent and district's school board. Local school partnerships
will also have a stake in the issues being addressed during the evaluation proceedings.

How will the findings be used?


The evaluation findings will be used to assess whether the SLTI program has demonstrated
increased professional growth amongst the teachers who participated in the professional
development service, determine if there are areas of growth and improvement, calculate the
financial return of investment in the program, and to demonstrate the impact on student learning.

What do they need to learn from the evaluation?


The stakeholders need to learn about the areas in the program that are in need of
improvement, the areas that are working well and just need to be maintained. Also, the
additional anticipated costs to implement the program effectively for continuity would be
beneficial information obtained from the evaluation.

Audience Evaluation findings of Utilization of evaluation findings


interest
Teachers Participant program evaluations; survey On-going program improvement
data of change in knowledge, changes in
test scores of participants; Data analysis
of students' assessment (formative and
summative)

Funders Data illustrating changes Determination of future funding


in behavior of the population
served; return on investment
calculations, etc
Students Changes in test scores of participants On-going student improvement

Administration Participant program evaluations; On-going program improvement


survey data of change in knowledge,
changes in test scores of
participants; etc

3
Evaluation Questions and Anticipated Data Sources

In order to effectively evaluate the SLTI, we have formulated evaluation questions based
on the request of the stakeholders. Our first question asks how well prepared and supported are
the teachers to implement the SLTI Program with high educational quality in the school? Are
teachers able to successfully transfer the knowledge and skills gained during professional
development sessions to their lesson development? Answering this question is important in order
to assess the program. To enable the students to reach the long-term outcome of gaining rich
learning experiences to master 21st century skills, the teacher will have to use the knowledge
gained as a result of professional development and apply it to his/her lesson plan. According to
the SLTI program overview, the professional learning pillar is "designed to encourage and assist
teachers' successfully bolstering and broadening classroom practices". If teachers aren't using the
knowledge and skills gained from professional development in their teaching, then the program
is all for naught. We need to evaluate why the skills are not being utilized.
There is specific data needed to answer this evaluation question. The qualitative and
quantitative data needed to analyze classroom implementation are teacher feedback surveys
throughout the school year, the submission and analysis of possible lessons created during or as a
result of professional development sessions, and classroom walk-throughs by administration and
trained evaluators during the implementation of lessons which involve skills obtained from
various professional development sessions.
The next question is: Are the results worth the time and money, especially how efficiently
are the resource being used, does the cost differ from the cost of different locations? This
question is important to answer in order to assess the program for several reasons. Can 1:1
computing to be economically viable, the combined cost of instructional materials and devices
must be no greater than whats currently spent on textbooks? While the cost of materials is
coming down, text books still will be around for a while until the adoption cycle ends. Currently,
schools have to pay a little more for devices and materials to make 1:1 computing economically
viable.
The data needed to answer this question would be an achievement measure, where each
of the states objectives are tested. This type of test is called an objectives-based test, which
exactly measures the objectives of the standards and tied to a success on the measure. We will
use existing data instruments that have specified certain kinds of attainment. This data will be
quantitative data, because it will have a concept of value.
Our last question involves changes in achievement due to the implementation of the
program: Is there an increase in 21st Century technological skills and practices amongst students?
How have these practices affected student engagement and achievement? Are the students
meeting Saturns college and career readiness standards? Answering these questions is important
in order to assess the program because the primary stakeholders (students) have a vested interest
in seeing the program succeed from the beginning to end. Acquiring data that will demonstrate
measurable achievements, as well as highlight areas of weaknesses that may need to be
addressed and/or revised is vital. Other stakeholders, such as parents, teachers, administration,
district personnel, and state officials would all be very interested in the results of this data too.

4
It will be necessary to collect a variety of data to answer this question. Qualitative data
(via interviewing methods/questionnaires) will be utilized to collect data from the stakeholders
(teachers and students). This information is to be gathered via open-ended probes. Questions
formulated for the interviewing process includes, but would not be limited to the 5 W's and H
strategy (who, what, when, where, why and how). Data explicitly investigated will capture the
stakeholders' values, priorities and concerns, if any, regarding the program. Another source
involves a semantic differential scale in order to obtain a summative and measured snapshot of
the stakeholders' attitude/feelings towards the overall program. Observational data on how
technology is being used and integrated in the classroom setting is to be gathered by both the
teacher and evaluator at regular intervals. Data collected from observations of students will be
compared to Saturns college and career readiness standards, in order to determine mastery.

Observatio Focus
Projected/Proposed ns Interviews Groups Surveys
Document/ Educator (Classroom/ (Teacher/ (Student/ (Student/
Data Source Course Eval Tool Institute/ Admin/ Teacher/ Teacher/
Review Results Workshop/ Other) Other) Other)
Other)
Evaluation Question
How well prepared and supported are x
the teachers to implement the SLTI X X X X X
Program with high educational quality
in the school? Are teachers able to
successfully transfer the knowledge
and skills gained during professional
development sessions to their lesson
development?

Are the results worth the time and X


money, especially how efficiently are X X
the resources being used? Does the
cost differ from the cost of different
locations?

Is there an increase in 21st Century X X


technological skills and practices X X
amongst students? How have these
practices affected student engagement
and achievement? Are the students
meeting Saturns college and career
readiness standards?

5
IV. DATA COLLECTION

The evaluator will be responsible for obtaining data using the mixed methods type surveys that
will be used to collect data from the stakeholders; and this will include the 5 W's and How, (format of
questioning using who, what, when, where, why and how) within the surveyed evaluation processes. This
data will be collected throughout the evaluation program (pre, mid and post survey).
The teachers will be responsible for obtaining data using the behavior recording method,
(frequency recording, 2 option format) that will be used to compute quantitative data by counting how
many times they incorporate YouTube and project based learning in their instructional practices. The
teachers' responsibility for frequency recording will also be used to monitor, and collect data on students'
interactions, and the percentage of time that they are actively engaged in the utilization of YouTube and
the other 21st Century practices in the classroom setting will be notated.
Data will be collected by the teachers daily, weekly, monthly, and then every few months using
the previous methods as mentioned above. Teachers will monitor and supervise students use of
technology and a data tool, such as Bright bytes (http://brightbytes.net) will be utilized to collect and
record information in a timely and efficient manner.

Data Collection Plan

Indicator Data Sources/Tools Collection

Increased knowledge Mixed- Methods/Surveys From Who When How

Students Pre, Mid and Post Frequency


stages of the Recording Tool
evaluation (Bright Bytes)
process.

Frequency of usage Behavior Recording Method Teachers Pre, Mid and Post Frequency
stages of the Recording Tool
(Self-Report) evaluation (Bright Bytes)
process.

Increased knowledge Mixed- Methods/Surveys Evaluator Pre, Mid and Post Survey Format
stages of the
evaluation
process.

V. DATA ANALYSIS

The evaluation of the SLTI program will be a Mixed- Method approach with both
qualitative and quantitative assessments. On-line and paper surveys will be given to all the
teachers, students, parents, principals, and superintendents in the Saturn district.

6
Data analyzing will consist of segmenting and coding the data while looking for patterns
and growth. Data summary include using nominal data, naming the data (word counting and
coded categories) of the survey questions. Using a computer program like Survey Monkey will
be a great on-line method for the stakeholders to use due to its simplicity. I will use a frequency
table with coding the categories. Evaluation evidence will be collected using a variety of tools.
Survey data will be collected from teachers, students, parents, principals, and superintendents.
In addition, another quantitative type of data will be a skills attainment or ability test for
determining if knowledge was attained. Interval data that will rank how the program is
successful by comparing test scores. A box plot will be used to get a sense of distribution. The
box plot will show the distance in percentages from 0 to 100% marked off in 25% intervals or
quartiles. Many of the benefits of 1-to-1 laptop programs are not detected by standardized tests,"
Zheng said. "For the many programs whose purpose is to help students be a better 21st-century
citizen, we need to develop and use corresponding measurements."
Classroom observations and will be utilized to see if there is an increase on student
achievement and engagement in 21st century technology skills. A 4-point Likert scale will be
used with 1 Strongly disagree, 2 Disagree, 3 Agree, and 4 Strongly agree.
There are computer programs that will assist in the coding and analyzing data, for
example Vivo and Atlas ti. I will be utilizing the software available through Google Docs as this
website does give the feedback from data in a spreadsheet format. The utilization of these tools
can enhance the accuracy and reliability of data. These spreadsheets and diagrams should help
my stakeholders interpret the data analyzed while creating an interesting and informative
evaluation report.
The results of this program evaluation will be collected and compiled into a written report that
will be of assistance to the SLTI program report will be submitted to the stakeholders.

7
APPENDICES
STUDENT SURVEY

The laptop program is being studied to find out how laptops are being used in your classroom.
Your participation in the survey is voluntary, and your identity and responses will be kept
confidential. The survey asks you to answer some questions about your experience with the
laptop program. Your views about the laptop program are important, and we hope you will take a
few minutes to answer the survey questions. Some of the questions contained in this survey are
closely related to questions asked on previous surveys. Please answer all of the following
questions, as we are attempting to track changes over time.
Thank you for your participation.

Strongly disagree Neither agree agree Strongly


disagree or disagree agree
I consider myself to
be tech savvy?
My technological
abilities are greater or
less than my peers?
My technological
abilities greater or
less than my parents?
My technological
abilities are adequate
for todays
employers?
I visit academic
websites often.

Do you have a personal computer? yes no


Does it have access to the internet?
Do you have a cell phone?
Do you belong to a social network
(e.g. MySpace)?
Do you use the internet for
entertainment?
Do you use the internet for
communication?
Do you use the internet in
academics for general information?

8
Program: SLTI Professional Development- Logic Model

Situation: Students are exposed to a broader curriculum that can motivate them to pursue
secondary education and career readiness skills. The core emphasis is to afford students with
enhanced learning experiences.

Вам также может понравиться