Вы находитесь на странице: 1из 6
Proposal for Evaluation of Determining Instructional Purposes Training Program Submitted to: Far West Laboratory for
Proposal for Evaluation of
Determining Instructional
Purposes Training Program
Submitted to:
Far West Laboratory for
Educational and Research
Submitted by:
Western Pennsylvania
Evaluation Group (WPEG)
October 27, 2015


October 2015, Far West Laboratory for Educational & Research Development (FWL), released a request for proposal (RFP) for evaluation of its Determining Instructional Purposes (DIP) training program. The following document, from Western Pennsylvania Evaluation Group (WPEG) of Pittsburgh, Pennsylvania, is being submitted in response to the Far West Laboratory RFP. Western Pennsylvania Evaluation Group provides professional evaluation and consultation services for organizations with areas of expertise in planning, evaluation, data analysis, and market research. WPEG is offering our services to assist FWL’s decision-making process with regards to determining whether or not resources should be allocated to market the DIP training program package. Additionally, WPEG aims to provide useful information to school administrators seeking to make decision on the purchase and use of DIP training units.

Description of Program

The Determining Instructional Purposes (DIP) training package, developed by FWL, is designed to prepare school administrators and graduate students in educational administration in effectively planning school programming. DIP contains three units each of which include four- to-six modules focusing on the following topics: Unit 1-Setting Goals, Unit 2- Analyzing Problems, and Unit 3- Deriving Objectives. The units, ranging in length from 155 to 259 pages, are intended to be self-contained allowing for use of either one unit or in combination of three units. FWL estimates required training time for Unit 1 and 3 as 10-15 hours with Unit 2 at 12-18 hours. Materials for the units, available exclusively in print form, are available from FWL at a cost of $8.95 per single unit or $24.95 for a set of all three units.

Training courses conducted with unit materials, as per request by program developers, should be overseen by a coordinator that is not required to have prior knowledge of the content area but should have gone through the units either individually or as a part of a training group. A Coordinator’s handbook is included in the training package and contains directions for

coordinating all three units. This handbook is $4.50 per copy.

presented entirely in print form and priced at

Evaluation Method

The primary purpose of Western Pennsylvania Evaluation Group’s (WPEG) evaluation is to determine whether Far West Laboratory (FWL) should commit money and resources to continue to market the Determining Instructional Purposes (DIP) training package. Overall, FWL is interested in making decisions regarding the future use of DIP and the need to keep the package as a part of FWL business expansion. To aid in FWL’s decision-making process, WPEG intends to evaluate the quality and effectiveness of the DIP training package in providing school administrators and graduate students in educational administration the necessary skills to plan effective school programs.

In order to achieve this, WPEG is proposing the use of multiple data sources to evaluate DIP content, delivery of training, and need. Through use of both formative and summative evaluations, WPEG is looking to assess 1) value of content and activities within the modules in

meeting instructional objectives, 2) effectiveness of different delivery modes offered, 3) level of need for the training package. Qualitative and quantitative findings generated from the evaluation will be reported to stakeholders at FWL and other individuals as indicated by FWL.

Facilitated Training Groups

To assess the value of content and activities within the modules and effectiveness of the different delivery modes, four training groups will be utilized. Training groups will consist of 8-10 participants consisting of school administration graduate students from universities as well as school administrators from schools districts located within and surrounding San Francisco, California. Each group will be placed in one of four training workshops which are as follows: 1) Unit 1 comprising of two, six-hour sessions, 2) Unit 2 including four, three-hour sessions, 3) Unit 3 containing three, four-hour sessions, 4) Units 1, 2, & 3 involving five, seven-hour sessions.

When sessions are completed, participants will complete reaction surveys regarding session content and activities. Running this formative assessment will provide specific information which will allow FWL to improve the training for future trainees. Observations by WPEG will be conducted during using the Best Practices for Training Rubric and will occur during one session per training group. The rubric will be used to assess delivery of the training and trainer preparedness. Upon completion of each module, participant mastery of content knowledge will be assessed through completion of a skills assessment questionnaire which will be designed with the assistance of subject-matter experts and designers of DIP. These questionnaires will be administered by training coordinators.

Participants and training coordinators will fill out online exit surveys at the completion of the entire training workshop to provide feedback regarding content and activities housed within the modules as well as delivery mode for the unit. The coordinators survey will also cover the topic of implementation. Coordinators will also be interviewed to gain a more detailed narrative of the DIP training package. Summative evaluations will include surveys focusing on the implementation of DIP skills to plan effective school programs.

Costumer Analysis

Due to FWL seeking to expand the business and being in the decision-making process to determine whether or not the DIP is worthy of monetary investment, it is important to better understand its marketability. To do this, WPEG plans to perform a customer analysis to gauge the level of need for the training through use of a product needs assessments. WPEG will conduct market research and interview FWL marketing personnel on the topics of demographic makeup, buying habits, and other characteristics of the target market. Once market research is completed, WPEG will administer needs assessment questionnaires to school administrators as well as other target audience members revealed by the market research. This questionnaire will be used to identify and weigh the relative importance of factors influencing decisions to purchase DIP and its usefulness to target consumers.

Proposed Task Schedule

Description of Task

Expected Completion Date

To establish an Evaluator’s Program Description (EPD), WPEG will meet with FWL stakeholders, via video conference, to review DIP program objectives, activities, and current evaluation materials/methods.

February 1, 2016

WPEG will develop reaction and exit surveys (used upon completion of each training session and at completion of entire workshop) as well as skills assessment questionnaires (used after each module).

March 7, 2016

WPEG will create questions for the coordinators exit interviews.

March 14, 2016

WPEG will meet with FWL management via videoconference to review surveys, questionnaires, and interview questions and complete the necessary revisions.

March 31, 2016

WPEG will conduct market research, phone or video conference interviews with FWL marketing personnel, and create customer needs assessment (CNA) questionnaire.

April 4, 2016

WPEG will administer the CNA questionnaire to local school administrators and target audience members determined by FWL marketing personnel and market research.

May 4, 2016

Start of four training workshops.

June 13, 2016

WPEG will conduct observations of sessions using the Best Practices for Training rubric.

June 2016

Completion of final Group 4 (Unit 1, 2, & 3) training workshop. With groups 1, 2, & 3 finishing prior to this date.

July 18, 2016

WPEG will perform coordinator exit interviews via phone or video conference.

July 25, 2016

WPEG will organize and analyze observation, survey, and interview data collected during implementation of the training workshops as well as draft the final evaluation report and submit to FWL.

August 2016

WPEG will meet with FWL stakeholders via video conference to provide an oral presentation on their findings, answer questions, and offer recommendations regarding next steps.

September 8, 2016

Project Personnel

Carter Barnett: Senior Evaluation Specialist

Having worked as a K-12 educator for 8 years prior to becoming interested in research and development, Mr. Barnett has over 15 years experience conducting evaluations. Working in partnership with state and federal agencies, non-profit organizations, universities, and K-12 educational settings he has served as Senior Evaluator for WPEG for 10 years. Mr. Barnett has extensive experience in designing and implementing qualitative and quantitative research methods as well as the ability to quickly and efficiently analyze content.

Heather Arenth: Research and Evaluation Assistant

Ms. Arenth, a graduate student in the Master’s program in the Department of Educational Technology at Boise State University, holds a bachelor’s degree in adult education and training with a minor in statistics. She has been working with WPEG for 3 years as an Evaluation Assistant. Ms. Arenth has broad experience designing and implementing a variety of different surveys and questionnaires. Due to her ability to build good working relationships through excellent communication and interpersonal skills, Ms. Arenth is now being utilized more to conduct focus groups and interviews. Ms. Arenth’s background in statistics as well as her passion to produce user friendly data products and reports has allowed WPEG to also offer her skills to also assist the Data Analysis team. Over the past 3 years, Ms. Arenth has improved her ability to collect, organize, analyze, and disseminate information with great detail and accuracy.

Joel Cigfox: Data Analyst

Mr. Cigfox has an extraordinary ability to turn data into information that clients are able to

insightfully use to make valuable decisions. Mr. Cigfox is new to WPEG but with his 20+ years in the field of data analysis and an educational background including a Masters in Data Analysis and Statistics from Carnegie Mellon University he is a welcomed addition to the WPEG team. Over the past 20 years, Mr. Cigfox has skillfully developed and implemented data collection systems and strategies that have helped many organizations, both for profit and non-profit,

optimize statistical efficiency and data quality.

has been displayed with his strong knowledge of and experience with reporting packages,

databases, and programming along with his strong analytical skills. Adelyn Tyler-Paxton: Market Research Analyst

His proven work experience as a data analyst

For the past 5 years, Mrs. Tyler-Paxton has served as Chief Market Research Analyst at WPEG.

She holds a Masters in marketing with a Bachelor’s degree statistics both of which were received

from the University of Pittsburgh.

PNC Financial Services Group, Inc. for 17 years as a Marketing Manager. Her work as a Marketing Manager allowed her to better understand business objectives and the importance of remaining fully informed on marketing trends and competitors marketing research. Mrs. Tyler- Paxton duties with WPEG include interpretation of data, formulation of reports, and consultation. Her ability to compile and analyze statistical market data and provide competitive analyses on companies’ market offerings has proven extremely beneficial to WPEG’s clientele.

Prior to working with WPEG Mrs. Tyler-Paxton worked with



Professional Salaries:

Carter Barnett: 50 days at $375/day


Heather Arenth: 45 days at $275/day


Joel Cigfox: 15 days at $375/day


Adelyn Tyler-Paxton: 20 days at $325/day




Travel and Per Diem

One 4-day round trip: Pittsburgh to San Francisco (includes per diem)




Four day car rental for session observations


Mileage estimated at an average of 75 miles/day




Supplies and Materials

Supplies and Photocopying


Materials (includes 3 coordinator’s handbooks, 10 three unit packages, and 30 single unit packages)


Video Conferencing Services: Join.me conference services (includes cost for FWL and WPEG at $15 per user total $30/month for 6 months)