1

Commission on Teacher Credentialing

Biennial Report 2014

Academic Years 2012-2013 and 2013-14

Institution Sacramento County Office Of Education (SCOE)
Date report is submitted September 15, 2014
Program documented in this report Administrative Services Credential Program
Name of Program SCOE Leadership Institute
Please identify all delivery options
through which this program is offered This program is delivered through a traditional option.
(Traditional, Intern, Other)
Credential awarded Tier I Preliminary Administrative Services Credential
Tier II Clear Administrative Services Credential
Is this program offered at more than one site? Yes
Sacramento County Office Of Education
If yes, list all sites at which
Shasta County Office Of Education
the program is offered
Program Contact Dr. L. Steven Winlock
Title

Executive Director

Phone #

(916) 228-2612

swinlock@scoe.net
E-Mail

If the preparer of this report is different than the Program Contact, please note contact
information for that person below:
Name

Dr. Melissa J. Neuburger

Title

Program Manager

Phone #

916-228-2575

E-mail

mneuburger@scoe.net

SECTION A CREDENTIAL PROGRAM SPECIFIC INFORMATION

PART I Contextual Information


Program Information
The Sacramento County Office of Education (SCOE) Leadership Institute provides credentialing
programs for aspiring and beginning administrators and provides professional development
opportunities to continuing administrators. Program coursework is focused on supporting both
credentialing and the development of leadership skills and utilizes curriculum based on the
California Professional Standards for Educational Leaders (CPSELs).

The preliminary administrative services credential program is a one-year program consisting of
coursework, fieldwork, end-of-course papers, and an end-of-year project and symposium. The
clear administrative services credential program is a two-year program consisting of one-on-
one coaching, workshops, practicums, reflections, and assessments of participant competence
and is based on the California Commission on Teacher Credentialing Guidelines. Candidates
may enter this program at three points throughout the year and begin their work in the CPSEL
course that is scheduled for delivery at that entry point. In addition, the Leadership Institute
has partnerships with National University and Drexel University that support administrators
who are additionally seeking masters or doctorate degrees.

Although the program offers support to administrators beyond the scope of the credentialing
programs, this report will focus only on analysis of the credentialing programs and the common
and program standards that are assessed.

Changes since Last Accrediting Activity

This program started under the umbrella of the Santa Clara Office of Education and received its
first independent accreditation as the SCOE Leadership Institute in February 2011. In 2012-13
the program added the Shasta County Office of Education as a Tier 1 program site and in
2014-15 the program added the Placer County Office of Education as a Tier I program site. In
addition the program began work towards the reorganization of the Leadership Institute into
the SCOE School of Education. This is the first biennial report conducted for the Leadership
Institute.


Program Specific Candidate Information
Numbers of candidates and completers/graduates for two years reported

Site (If multiple sites)
Delivery Option
SCOE Tier I Preliminary
Shasta Tier I Preliminary
SCOE Tier II Clear

2012-2013
Number of
Number of
Candidates
Completers/
Graduates
70
70

2013-14
Number of
Number of
Candidates
Completers/
Graduates
67
65

16

15

23

23

15

41

10

2
SECTION A CREDENTIAL PROGRAM SPECIFIC INFORMATION

PART II Candidate Assessment/Performance and Program Effectiveness Information

Candidate Assessments used to Recommend Candidates for Credentials
a) Please identify and describe the tool(s) used to assess candidates, the data collection
process and the types of data collected

The SCOE Leadership Institute utilizes several primary candidate assessments to recommend a
candidate for a credential. These assessments and measures focus on the successful
completion of coursework and the application of learning modules to instructional
management and school leadership.

Candidate Assessments
Key
Assessment
Tools

Description

Data Collection Process

For Tier I, coursework

consists of course
attendance, fieldwork,
online discussions, and end-
Assessment 1.
of- course papers.
CPSEL
For Tier II, CPSEL work
Coursework
includes the completion of
Completion
courses, applied practicum,
reflections, workshops, and
progress meetings with
coaches.

Standards
Assessed

Each element for each course is graded

on a set rubric that indicates whether
or not the candidate met the standards
for the course. If a candidate does not
meet the standards, revisions of
CS 3, 4, 7, 9
coursework are required until the
standard is met. Data for the
completion of each element is collected
in each course and stored in the central
data system.

Midway through the year, the

candidate submits a project proposal to
the executive director that is either
Tier I candidates are
approved or requires revision. A
assessed on an end-of-year
Assessment 2.
practice of the symposium presentation
project that relates to the
End-of-Year
is required. During the actual
course content for the
CS 3, 4, 9
Project and
symposium, panel members assess the
CPSELS. Each candidate is
Symposium
presentation and provide the candidate
required to present this
with their feedback using a prescribed
project during a symposium.
form. Completion of each element is
collected and stored in the central data
system.

March 2014

3
Candidate Assessments (continued)
Key
Assessment
Tools

Description

Data Collection Process

Tier II candidates assess

their own practices with the
input by assigned Tier II
coaches through the Moving
Leadership Standards into
Assessment 3.
Everyday Work
Descriptions of
Descriptions of Practice
Practice
(DOP) tool developed by
Assessments
WestEd. It asks participants
to rate the level of their
practice across 6 leadership
development program
standards.
Tier II candidates must
complete an exit interview
with their Tier II coaches
and the Leadership Institute
Staff after completion of the
6th CPSEL course. During the
interview the candidate is
Assessment 4.
asked about the learning
Exit Interview
that occurred and the merits
of the program as it relates
to the standard. Coaches are
asked to reflect on the
strengths of the candidates
and areas for continued
development.

Standards
Assessed

This tool is administered three times in

the course of the program: at the onset
of the program; during the middle of
the program; and at the end of the
CS 6, 9
program. Results for each of these
measures is collected online and stored
in the central data system.

Completion of the interview and

associated notes are logged into the
central data system. Information
gathered from the coaches is used
during the interview to discuss
candidate experience and
preparedness. The Institute staff
evaluates the final DOP.

CS 1, 2, 6, 7,
9

Additional Data and Measures Used to Analyze Program Effectiveness and Inform
Programmatic Decision Making
b) What additional information about program effectiveness is collected and analyzed that
informs programmatic decision-making?

In addition to measuring candidate competencies, the SCOE Leadership Institute also assesses
additional information to evaluate and inform the programs effectiveness. The table below
describes the variety of assessments used during the 2012-13 and 2013-14 academic years that
informed programmatic decisions. Data was collected from candidates, coaches, and
Leadership Institute staff and was reviewed throughout each year.

March 2014

4


Program Effectiveness
Measures of
Program
Effectiveness
Assessment 1.
Candidate
Evaluation of
Course
Effectiveness

Assessment 2.
Workshop
Feedback

Assessment 3.
Reflective
Records

Assessment 4.
Mid-Program
Review

Description

Data Collection Process

Tier I candidates complete a course

evaluation at the conclusion of each
of the 6 CPSEL courses using a locally
developed survey tool. It assesses
instructor preparation and
knowledge, applicability of course
content, and resources and support.
Tier II candidates complete a locally
developed online workshop feedback
form at the conclusion of each
workshop they attend. It assesses
presenter knowledge and
effectiveness and course content as
well as the application of the
workshop content to the applied
practicum.
Tier II candidates and coaches
complete a locally developed online
form that asks them to reflect on the
support given and received and the
quality of the candidate-coach
relationship specific to each course.
Tier II candidates complete a locally
developed form that requests
feedback about program quality in
the areas of standards, coaching,
logistics, and program.

Standards
Assessed

The online survey tool is

administered to all participants at
the completion of each course.
CS 2, 4
Data is collected online and stored
in the central data system.

The assessment is online at the

conclusion of each workshop.
Leadership Institute staff
evaluates forms and results are
stored into the central database.

CS 2, 4, 9

This assessment is administered

online at the completion of each
course. Data is collected and
stored in the central data system.

CS 2, 3, 6

After completing one year of the

program, candidates complete the
form to provide overall feedback
CS 2, 3
about the program. The forms are
reviewed by Leadership Institute
staff and are locally stored.

Candidate Assessment and Program Effectiveness Data
c) Include aggregated data from 4-6 assessments that were described in (a) and (b).

Candidate Assessment 1. CPSEL Coursework Completion
Tier I
Tier I candidates are required to complete nine specific elements of the program. These nine
elements are:
Program orientation
Completion of the six CPSEL courses and related coursework
Completion of the end-of-year project and symposium plan
Presentation of the end-of-year project at a symposium
March 2014

5

The table below shows the candidate counts of program element completion by location and
year. For both years, program completion rates were very high. For the 3 candidates that did
not complete the program within the expected year:
Two candidates dropped out of the program after completing only the orientation in
2013-14.
One candidate in 2012-13 had to postpone the completion of the project and
symposium due to severe family circumstances. The candidate is scheduled to complete
the program in the 2014-15 year.

2012-2013
2013-2014
7 of 9
9 of 9
1 of 9
9 of 9
Elements Elements Elements Elements

70
2
65
1
15

23
Grand Total
1
85
2
88

Tier I Candidate Counts: Required

Elements Completed
SCOE
Shasta COE
Percent Completed Program

98.8%

97.8%

Tier I CPSEL End-of-Course Papers

A key component of course completion is the end-of-course paper. If the paper is not approved
upon the initial submission to the instructor, the candidate must continue to resubmit the
paper until the course standard has been met. In the 2012-13 cohort year, 26 candidates (30%)
required a resubmission of an end-of-course paper and in 2013-14 year, 20 candidates (22%)
required resubmission. The table below shows by cohort, location, and course title, the
number of candidates that had to resubmit a paper and the number of resubmissions that were
required. Resubmissions were most often required in the Management for Teaching and
Learning and the Shared Visions of Learning courses. Resubmissions were least often required
in Personal Ethics for Leadership. No resubmissions were required for the Political and Social
Influences course. Reasons instructors gave for declining a paper included:
Omission of references
Omission of implementation plan
Candidate did not address the required number of elements
Sections required clarification as related to the leadership standards and its elements

Tier I: Number of Candidates that Required Resubmission
of the End-of-Course Paper by Course Title

2012-13 Cohort
(N=70)

2013-14 Cohort
(N=67)

Number of Paper Resubmissions

Two

Two

SCOE (Counts of candidates resubmitting papers)

Three

Three

Grand
Total

Management for Teaching and Learning

Personal Ethics for Leadership

Shared Vision of Learning

10

Teaching and Learning Culture

Working with Diverse Families and Communities

March 2014

17

6
Total

12

Shasta COE (Counts of candidates resubmitting papers)

18

38

Management for Teaching and Learning

Shared Vision of Learning

Total

12

19

22

50

Grand Total

Tier II
The table below shows the number of courses completed by Tier II candidates as of August
2014. Although the candidates can enter the program at any of three points during the year
(fall, winter, or spring), each candidate is expected to complete three CPSEL courses each year
to stay on track. Of the 2012-13 candidates that completed zero courses, one dropped out and
the other had to put the program on hold due to medical reasons. Of the 2013-14 candidates
that completed zero courses, all four had begun the first CPSEL course in May of 2014. Overall
since the beginning of the program, 10 candidates from the 2012-13 cohort have completed the
program and received their clear credential.

Tier II Candidates: Number of CPSEL
Courses Completed as of August 2014
0 courses
1 courses
2 courses
3 courses
4 courses
5 courses
6 courses
Grand Total

Program Start Year for Cohort

2012-13
2



2
1
10
15

2013-14
4
13
9
10



36

Candidate Assessment 2. End-of-Year Project and Symposium
The end-of-year project and symposium is an evolving process whereby the district coaches and
faculty mentors work with the Tier I candidates to provide continuous feedback on this element
of the program. This occurs through one-on-one meetings and assessment of the progress
made along the way. At the end of the year, the candidates present their project to a panel of
educational leaders from the region and receive feedback and evaluation on their projects.
Panel members in each of the following required elements assess projects and presentations:
Project vision and expected outcomes
Rationale of need and importance
Evidence of transformational leadership
Implementation artifacts
Progress towards accomplishing the vision
Next steps

March 2014

7
The feedback of these elements (captured through a locally developed form) is provided to
candidates. Of all the candidates that remained in the program in 2012-13 and 2013-14, only
one did not complete the end-of-year project and symposium as noted above and is scheduled
to complete it in 2014-15. A sample of comments from two of the candidates indicated that
the program provided a chance to engage in leadership roles and to act as a mentor to new
teachers. It also afforded the opportunity to learn how to assess and reassess successes and to
value data and use feedback to foster improvement.

Candidate Assessment 3. Descriptions of Practice Assessments
Throughout the duration of the program, Tier II candidates utilize the Moving Leadership
Standards into Everyday Work Descriptions of Practice tool to assess their level of competency
in each of the six program standards measured. Candidates complete this assessment initially
at the beginning of the program, again after completing one year of the program, and lastly at
the end of the program. All participants are required to complete these assessments therefore
the participation rate was 100%.
Scoring rubric used for analyses:
1 = directed toward the standard
2 = approaches the standard
3 = meets the standard
4 = exemplifies the standard
In addition to assessing their level of competency, candidates provided comments on their
growth experience and described the practices they employed. The table below shows the
average scores for each area at each stage of measurement as well as the number of
candidates that grew one or more levels at the mid-point and end-point.

Tier II Description of Practice: Self-Assessments by

Candidates

Initial
Program

Mid-
Program

End of
Program

Number
Increased 1
or More
Levels

Mean
(N =
50)

Std.
Dev.

Mean
(N =
17)

Std.
Dev.

Mean
(N =
10)

Std.
Dev.

Initial
to
Mid

Initial
to
End

1.1 Develop a shared vision

1.2 Plan and implement activities around the vision

2.08
2.18

.900
.825

2.71
2.59

.588
.507

3.00
3.10

.667
.568

11
11

7
7

1.3 Allocate resources to support the vision

Standard 2. Teaching and Learning Culture

1.90

.814

2.76

.562

3.10

.568

13

2.1 Develop school culture and ensure equity

2.2 Guide the instructional program
2.3 Guide the professional growth of staff

2.18
2.24
2.20

.850
.822
.728

2.82
2.88
3.06

.529
.332
.556

3.20
3.20
3.20

.632
.632
.632

10
13
13

6
7
8

2.4 Create and utilize accountability systems

Standard 3. Management for Teaching and Learning

1.98

.769

2.47

.514

3.10

.568

10

Standard 1. Shared Vision of Learning

March 2014

8
3.1 Ensure a safe school environment

2.38

.805

2.88

.600

3.40

.699

3.2 Create an infrastructure to support an effective

learning environment

2.14

.833

2.65

.493

3.20

.632

3.3 Manage the school as a learning-support system

3.4 Maintain legal integrity

2.24
2.34

.771
.982

2.76
3.12

.437
.332

3.20
3.30

.422
.483

8
11

6
6

Standard 4. Working with Diverse Families and Communities

4.1 Collaborate to incorporate the perspectives of
families and community members

2.10

.814

2.71

.772

3.30

.675

11

4.2 Establish and manage linkages between the site and

the larger community context

1.84

.766

2.47

.800

2.80

.632

4.3 Engage and coordinate support from agencies

outside the school

1.86

.700

2.41

.870

2.70

.675

5.1 Maintain ethical standards of professionalism

2.76

.960

3.06

.429

3.60

.516

5.2 Guide sound courses of action using pertinent,

state-of-the-art methods

2.18

.850

2.71

.470

3.20

.632

11

5.3 Model reflective practice and continuous growth

2.66

.823

3.00

.500

3.40

.516

5.4 Sustain professional commitment and effort

Standard 6. Political and Social Influences

2.50

.909

3.12

.600

3.50

.527

6.1 Engage with the policy environment to support

school successes

1.88

.824

2.29

.772

2.70

.483

6.2 Interact with stakeholders

6.3 Incorporate input from the public

1.98
1.88

.742
.718

2.82
2.29

.809
.686

2.80
2.80

.422
.632

13
5

6
6

Standard 5. Personal Ethics and Leadership

Candidate Assessment 4. Exit Interview
After Tier II candidates complete each of the six CPSEL courses and additional program
requirements, they must complete the exit interview to clear their administrative services
credential. The exit interview included a practicum review, a description of practice review,
and a coaching review. Feedback from candidates during the interviews was collected and was
used to assist candidates in reflecting on their program experience and personal and
professional growth. Of the 10 candidates that began the program in fall of 2012, 100% had
completed the exit interview and had completed the program. An analysis of each review area
indicated that:
100% of candidates met the standards in all 6 CPSELs
100% of candidates were rated by the coaches as exceeding the requirements

Overall coaches indicated that candidates demonstrated great growth over the course of the
program, often moving from approaching standards to meeting or exceeding them. Areas
noted for continued growth for candidates included expanding relationships with parents and
the community and also refining their leadership and management skills.



March 2014

9
Program Effectiveness Assessment 1. Candidate Evaluation of Course Effectiveness
After the completion of each CPSEL course, Tier I candidates were asked to rate the course
effectiveness by indicating their level of agreement for each of 10 statements.
Scoring rubric used for analyses:
1 = strongly disagree (was coded as 2 on original survey)
2 = disagree (was coded as 3 on original survey)
3 = agree (was coded as 4 on original survey)
4 = strongly agree (was coded as 5 on original survey)

The participation/completion rate for the workshop feedback forms for all Tier I candidate
groups and locations was 100% for both 2012-13 (N=86) and 2013-14 (N=87). The table below
shows the results by year and location with the responses for agree and strongly agree
combined to demonstrate the percent that agree. Cells highlighted in yellow indicate where
less than 93% of candidates agreed with the statements. Green shaded cells indicate areas
where 100% of candidates agreed with the statements. When computed as averages on the
scale from 1 to 4, the means for each course by location and year ranged from 3.46 to 3.87 and
the standard deviations ranged from .485 to .712.

March 2014

10

Mgmt. for
Teaching

Personal
Ethics

Political
Influences

Shared
Vision

Learning
Culture

Diverse
Families
and Comm.

Year

% Agree

% Agree

% Agree

% Agree

% Agree

% Agree

2012-13

2013-14

97.1%
98.4%
97.1%
98.4%
97.1%
98.4%
97.1%
98.4%
97.1%
98.4%

100.0%
98.4%
100.0%
98.4%
100.0%
98.4%
94.3%
95.2%
94.3%
98.4%

98.5%
100.0%
100.0%
100.0%
100.0%
100.0%
95.5%
98.4%
97.0%
100.0%

97.1%
96.9%
97.1%
96.9%
97.1%
96.9%
97.1%
96.9%
97.1%
96.9%

100.0%
100.0%
100.0%
100.0%
100.0%
98.4%
100.0%
100.0%
100.0%
100.0%

97.1%
96.8%
97.1%
96.8%
97.1%
96.8%
97.1%
96.8%
97.1%
96.8%

2012-13

97.1%

98.6%

97.0%

97.1%

100.0%

97.1%

2013-14

98.4%

98.4%

100.0%

96.9%

100.0%

96.8%

2012-13

2013-14

97.1%
98.4%
97.1%
98.4%

94.3%
96.7%
95.7%
93.7%

97.0%
98.4%
97.0%
100.0%

97.1%
96.9%
97.1%
96.9%

100.0%
98.4%
98.5%
100.0%

97.1%
96.8%
98.6%
96.8%

2012-13

87.1%

97.1%

91.0%

98.6%

94.2%

94.3%

2013-14

98.4%

98.4%

100.0%

90.6%

96.8%

93.7%

Online live chat experience supported my 2012-13

learnings through the translation oftheory
2013-14
into practice.

92.9%

92.9%

92.5%

98.6%

92.6%

94.3%

96.9%

98.4%

100.0%

95.3%

93.7%

95.2%

Shasta COE

Year

% Agree

% Agree

% Agree

% Agree

% Agree

% Agree

2012-13

100.0%
95.0%
93.8%
95.0%
100.0%
95.0%
93.8%
95.0%
93.8%
95.0%

86.7%
100.0%
86.7%
95.0%
86.7%
100.0%
86.7%
100.0%
86.7%
100.0%

100.0%
86.4%
100.0%
86.4%
100.0%
86.4%
100.0%
86.4%
100.0%
86.4%

100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%

93.8%
100.0%
93.8%
100.0%
93.8%
100.0%
93.8%
95.2%
93.8%
90.5%

Tier I Course Evaluations

Sacramento COE
Demonstrated knowledge of the key
instructional components of the course.
Was prepared to teach the course.
Addressed participants' questions in a
professional manner.
Overall, delivered an effective course.
Major concepts were presented that
assisted with course understanding.
Strategies and processes around course
concepts will assist with implementation
of leadership skills.
Resources assisted with the learning of
course concepts.
Mini-session instructor built upon the
understanding of course concepts.
Online Principal-Coach provided guidance
and facilitation that supported the
community chat experience.

Demonstrated knowledge of the key

instructional components of the course.

2013-14
2012-13
2013-14
2012-13
2013-14
2012-13
2013-14
2012-13

2013-14
2012-13

Major concepts were presented that

assisted with course understanding.

2012-13
2013-14

100.0%
95.5%
100.0%
95.5%
100.0%
95.5%
100.0%
95.5%
100.0%
95.5%

Strategies and processes around course

concepts will assist with implementation
of leadership skills.
Resources assisted with the learning of
course concepts.
Mini-session instructor built upon the
understanding of course concepts.
Online Principal-Coach provided guidance
and facilitation that supported the
community chat experience.
Online live
chat experience supported my
March
2014
learnings through the translation oftheory
into practice.

2012-13

100.0%

93.8%

86.7%

100.0%

100.0%

93.8%

2013-14

95.5%

95.0%

100.0%

86.4%

100.0%

95.2%

2012-13

2013-14

100.0%
95.5%
100.0%
95.5%

93.8%
95.0%
100.0%
95.0%

86.7%
100.0%
86.7%
100.0%

100.0%
86.4%
100.0%
86.4%

100.0%
100.0%
100.0%
100.0%

93.8%
100.0%
93.8%
100.0%

2012-13

93.8%

100.0%

93.3%

100.0%

100.0%

93.8%

2013-14

95.5%

95.0%

100.0%

86.4%

100.0%

100.0%

2012-13

10
93.8%

93.8%

86.7%

100.0%

100.0%

87.5%

2013-14

95.5%

95.0%

100.0%

86.4%

100.0%

95.2%

Was prepared to teach the course.

Addressed participants' questions in a
professional manner.
Overall, delivered an effective course.

2013-14
2012-13
2013-14
2012-13
2013-14
2012-13
2013-14

2013-14
2012-13

11
Program Effectiveness Assessment 2. Workshop Feedback
Tier II candidates were required to complete five workshops of their choice in each year of the
program. At the conclusion of each workshop they completed a workshop feedback form that
assessed the quality of the presenter and the workshop. Participants were asked to indicate
their level of agreement to seven statements and provide comments about the workshop and
the application of the workshop to their own work. The table below shows the average score
and descriptive statistics for each statement by workshop year and includes information about
workshop length and diversity.
Scoring rubric used for analyses:
1 = strongly disagree
2 = disagree
3 = neutral
4= agree
5 = strongly agree
Workshop comments indicated that:
The information was valuable and useful
Presenters were very knowledgeable
Opportunities to collaborate with colleagues were appreciated
Comments regarding the applicability of workshop content to participants work generally
indicated that seeing how the presenter applied the information in various settings helped
them to understand the challenges in applying it to their work.

2012-13
Tier II Workshop Evaluations by Candidates

2013-14

Mean

Std.
Dev.

Demonstrated knowledge of key instructional components

4.58

60

Was prepared to teach

4.63

Addressed participants' questions professionally

Overall, delivered an effective workshop

Instructor Effectiveness

Workshop Content

Mean

Std.
Dev.

.591

4.78

189

.509

60

.551

4.75

189

.555

4.67

60

.572

4.77

189

.545

4.55

60

.675

4.70

189

.564

Information and materials were helpful in meeting the

objectives of the course

4.50

60

.676

4.75

189

.555

I feel prepared to implement the workshop content

4.27

60

.778

4.44

189

.724

The content was applicable to the CPSEL and my Applied

Practicum Action Plan

4.50

60

.701

4.57

189

.708

Additional Information
Average Workshop Length
Number of Different Workshops

March 2014

11

3.94 hours

4.74 hours

39

116

12
Program Effectiveness Assessment 3. Reflective Records
Throughout the Tier II program, coaches and candidates reflect on the program support process
at the conclusion of each CPSEL course. The amount of program support is rated across 6
areas. The table below shows the average rating for each CPSEL course by year and by
candidates and coaches.
Scoring rubric used for analyses:
1 = none
2 = minimal
3 = adequate
4= over and above
Program Year
Tier II CPSEL Courses: Overall Average Score for Support
Given or Received
Candidate Reflective Record
Shared Vision of Learning
Teaching and Learning Culture
Management for Teaching and Learning
Working with Diverse Families and Communities
Personal Ethics for Leadership
Political and Social Influences
Coach Reflective Record
Shared Vision of Learning
Teaching and Learning Culture
Management for Teaching and Learning
Working with Diverse Families and Communities
Personal Ethics for Leadership
Political and Social Influences

2012-13

2013-14

Mean

Count

Std.
Dev.

Mean

Count

Std.
Dev.

3.63
3.48
3.31

10
11
9

.47
.85
.94

3.83
3.28
3.47
3.43

4
23
35
46

.33
.68
.54
.62

3.55
3.39
3.39

10
14
12

.48
.52
.51

3.08
3.24
3.09
3.11

2
25
35
46

.12
.49
.47
.53

To assess the strength of the support received, each item on the reflection tool was reviewed
by year. The charts that follow depict these results for candidates and coaches across all CPSEL
courses combined. Also note that coaches are expected to visit the candidate at their site at
least one time per year (this equates to one visit during one of the three CPSEL courses
annually, hence a 33% visitation rate).

March 2014

12

13

March 2014

13

14
Program Effectiveness Assessment 4. Mid-Program Review
After completing one year of the program, Tier II candidates complete a locally developed mid-
program form that requests feedback about the program quality and their experience as a
candidate. This form is designed to capture free-form comments in four key areas using the
following guiding questions:
Program
o How are practicums aligned with elements of standards?
o What knowledge areas will you focus on for remaining CPSELS?
Coaching
o What are some examples of coaching strategies that have supported your
leadership development?
Logistics
o What contacts are being made that support development?
o What areas of coaching would be helpful to further develop leadership?
Support
o What assistance has been provided during the program?
o What area(s) of need would assist the work needed to complete the program?
A total of 10 candidates from the 2012-13 cohort and 14 from the 2013-14 cohort completed
this requirement to date. The responses provided through these forms were reviewed by
Leadership Institute staff and are locally stored. Because a lengthy analysis of all comments
received would not serve the concise nature of this report, a review of a representative sample
from the 2013-14 is provided instead.
Overall, data collected from the mid-program review indicated that:
Program
Candidates emphasized how practical the program work was and how relevant it was to
their work. Goals for the school site they worked at meshed well with the expectations
of the leadership program. The practicums were described as authentic and beneficial
to candidates learning and professional growth.
Coaching
Candidates were very thankful for the support and feedback they received from their
coaches and the validation of their work in the program. Coaching sessions were timely
and relevant and the discussions helped candidates explore options and expand their
perspectives.
Logistics
Candidates agreed that the program logistics were flexible and that the schedules
provided them with sufficient time to complete their tasks.
Support
Overall districts were very supportive of their candidates including the provision of
financial support for the program. Sites appreciated the professional development
activities that candidates brought back to their site and the Leadership Institute was
noted for its overall organization of the program and the availability and responsiveness
of staff.
March 2014

14

15
PART III Analyses and Discussion of Candidate and Program Data

Describe what the analyses of the data demonstrate about your program relative to: a)
candidate competence; and b) program effectiveness.

Overall, results from the candidate and program assessments indicate that the Leadership
institute provides appropriate professional development and support to administrative
credential candidates to enable them to successfully complete the program requirements and
earn the related credentials.

Candidate Assessment 1. CPSEL Coursework Completion

In both the Tier I and Tier II programs, candidates routinely complete their CPSEL coursework
within the expected timeline, indicating that this is not an area in which candidates fall behind
or that the program advances too quickly. A close assessment of the Tier I end-of-course paper
resubmission rates provides evidence that instructors work closely with candidates to ensure
that they understand the elements of the CPSEL course contents and application. Worth noting
is that the resubmission rates decreased in the second year suggesting that the program is
responsive to feedback form candidates. Also worth noting is that resubmissions were most
often required in the Management for Teaching and Learning and the Shared Visions of
Learning courses in both years. Within the structure of the Tier I coursework, this could be
considered a relative weakness among candidates in their course area, albeit a small one. The
fewest resubmissions occurred in the Personal Ethics for Leadership course and none were
required for the Political and Social Influences course which suggests that candidates have
greater skills in these areas.
Candidate Assessment 2. End-of-Year Project and Symposium
Given that all but one Tier I candidate completed this requirement on time, this requirement of
the program could be considered to be appropriately leveled and not an area of concern at the
time of this program review. Feedback from participants supports this conclusion.
Candidate Assessment 3. Descriptions of Practice Assessments
The Tier II descriptions of program assessments show that candidates typically begin the
program at the approaching standards level and end at the meets standards level at the
end of two years. Candidates rate their level of practice as highest in the standard of personal
ethics and leadership (relative strength) and lowest in the standard of political and social
influences (relative weakness). In general, approximately 70% of candidates moved up one or
more levels from their initial assessment of their practice with an average movement of one
level across the duration of the program.

The standard elements in which the smallest number of candidates moved up one or more
levels were: 4.2 Establish and manage linkages between the site and the larger community
context; 5.1 Maintain ethical standards of professionalism; and 5.3 Model reflective practice
and continuous growth. A closer analysis of the data for these items shows that candidates
who initially rated themselves as low had all made a one to two level gain at the conclusion of
the program while those that initially rated themselves as meeting or exemplifying the standard
March 2014

15

16
rated themselves at the same level or one level lower at the conclusion of the program, likely
due to a better knowledge of what exemplifying a standard truly means. This could be
considered a strength in the program whereby candidates are gaining a better understanding of
the leadership standards and expectations as they progress through the program. This also
attests to candidates ability to effectively reassess their learning.

Candidate Assessment 4. Exit Interview
Data from the Tier II exit interview indicated that the candidates all met each standard and
made great growth however the coaches comments regarding areas of significant growth and
areas for continuous growth were general in nature. More specific comments with concrete
examples as well as resources and suggestions for improvement would benefit the professional
growth of candidates and review by program staff. It is also worth noting that each candidate
was rated as exceeding the standards of the program which supports strong candidate
competency. On the other hand, this might be erroneously interpreted by the candidates that
they have reached their full potential as an administrator and that little growth is left to
accomplish. However, given the comments by candidates of their personal and professional
growth, this is likely not a concern. Nonetheless, a rating scale with more levels could be
developed to provide this assurance and refine this assessment of candidate competency.
Program Assessment 1. Candidate Evaluation of Course Effectiveness
Overall, the Tier I courses received high ratings for overall effectiveness. The highest rated
areas were the Learning Culture and the Political and Social Influences across both program
sites and for both years. Political and Social Influences had improved at the Shasta COE site
from the prior year and the online learning component of this course improved at both sites
from 2012-13 to 2013-14. Beyond those areas, the program sites differed. For example, the
Management for Teaching course was rated higher at the Shasta COE site than at the SCOE site
while the rating for the Shared Vision course at the Shasta COE had dropped from 2012-13 to
2013-14. Because different staff teach these courses at the different sites, it would be valuable
to learn what is different about how these courses are being taught. Nonetheless, the ratings
for all courses were very high and support the strength of the program.
Program Assessment 2. Workshop Feedback
At the conclusion of each workshop, Tier II candidates completed workshop evaluation forms.
As evidenced by the data, information from the 2012-13 evaluations was successfully used to
improve all workshop areas for the 2013-14 year. This improvement also coincided with the
increased lengths of the workshops by nearly an hour, addressing the desire by candidates to
go more in-depth during courses. The areas of greatest strength in the 2013-14 year for the
workshops was in the instructors demonstrated knowledge of key instructional components
and usefulness of materials in meeting the objectives of the course. The area of relative
weakness for both years was in the candidates belief that they were prepared to implement
the workshop content, a common theme found in many professional development programs.


March 2014

16

17
Program Assessment 3. Reflective Records
Results for both program years were similar and it was interesting to note that Tier II
candidates reported receiving slightly more support from the program coaches than the
coaches believe they provided. This was most evident in the Management for Teaching and
Learning course which received the highest rating during the 2013-14 year yet received the
lowest rating in the 2012-13 year. Results also indicated that candidates need greater coach
support in the Working with Diverse Families and Communities course.
A comparison of candidate ratings to coach ratings across all CPSEL courses shows that in 2013-
14 a disparity between perceived support offered and received existed across nearly every area
measured. In 2013-14 communication through a variety of mediums was the highest rated area
of support by candidates where 68% of candidates indicated that coaches went over and above
in this area with the next highest support areas identified as assisting with and reviewing the
Applied Practicum Action Plan. Although the area of lowest support would appear to be in the
area of visiting the site, this may not be the case. It is difficult to discern from the structure of
the question whether it is assessing whether a site visit occurred, whether several site visits
occurred, or whether a visit occurred and the coach was particularly supportive during the visit.
The question stem was whether the coach supported the candidate by visiting the site however
this is based on a four point rubric ranging from none to above and beyond. Interpretation
of the data gleaned from this question is inconclusive. It is recommended that this item be
scored on a yes/no rubric in the future.
On an interesting note, it should have occurred that coaches became more experienced with
the program over time and the coaches in the second year and should have been able to
provide better support to candidates, yet coaches indicated that they gave less support in the
second year of the program. Again, it is difficult to interpret whether coaches actually provided
less support or whether they became more efficient in their delivery of it.
Program Assessment 4. Mid-Program Review
Analysis of the mid-program review data confirms the findings of the other program
assessments. Overall the program is efficiently run and the content and format are relevant to
the work of the candidates. Based on the overall information provided by candidates, no
causes for program improvement were discovered however a revised review tool that includes
a rating scale might be able to better discern subtle areas where the program could benefit
from refinement.

PART IV Use of Assessment Results to Improve Candidate and Program Performance

In summary, the information gleaned from the candidate and program assessments indicate
that the Leadership Program is well organized, utilizes appropriate curriculum, and has a
successful candidate completion rate. Although this program received high scores in each of
the assessed areas, there is always room for improvement. Based on information collected
from the candidate and program assessments, a number of proposed changes will be made to
March 2014

17

18
the program, most of which are a refinement of assessment tools used and the increased
sharing of the assessment results to candidates and other stakeholders. The table below
presents the focus areas for improvement and the proposed changes to support improvements.

Data Source
Plan of Action or Proposed Changes Made
Applicable
Program or
Common
Standard(s)
Candidate
To better assess the quality of the coursework that Tier I
CS 1, 2
Assessment 1. candidates complete, the scoring rubric will be changed
from Meets/Does not Meet Standards to a multi-level
CPSEL
rubric reflective of the Description of Practice. This will
Coursework
provide more information to the program staff about
Completion
candidate competency as well as differences seen among
for Tier I
courses or instructors. The revised rubric will be pilot tested
in the 2014-15 year and implemented in the 2015-16 year.
Candidate
Details about the project and symposium and the exit
CS 1,2,6,7,9
Assessments 2 interview will be captured electronically and analyzed
and 4: End-of- annually. In addition, rating scales will be developed and
candidates, faculty, and Institute staff will rate their
Year Project
effectiveness in meeting course elements and program
and
effectiveness. This information will be used to guide
Symposium
program design, services and support.

and Exit
Interview
Program
Assessment 2.
Workshop
Feedback

It was evident that candidates desire more support in

implementing workshop content. To identify the areas in
which implementation is most challenged, a focus group
will be conducted with candidates. Results of the focus
group will provide information about where the program
will need to provide further support to candidates. (e.g.,
workshop design, types of workshops, delivery,
implementation of concepts.)
Program
To better understand how and why candidates cited
Assessment 3. receiving more support in 2013-14 than what coaches
believe they provided, focus groups with coaches will be
Reflective
conducted in 2014-15. Questions will assess how the type
Records
of coaching changed from 2012-13 and 2013-14 and what
the greatest perceived needs of candidates were. Data
from the focus groups will be used to refine the coaching
model and how support is assessed. In addition, site visits
will be coded using a yes/no rubric rather that the existing
four-point scale.
March 2014

18

CS 6,7

CS 6,9

19
Program
Assessment 4.
Mid-Program
Review

To capture more discreet data, a rating scale will be added

to the Mid-Program Review tool that will support the
quantification and summarization of data and analysis of
trends.

Analysis of all
Candidate and
Program
Assessments

In most cases, the assessments collect data from

CS 1,2
candidates, coaches, and instructors. The Leadership
Institute recognizes that formal data must also be collected
from the program staff and the advisory board. Beginning
with the 2014-15 year, biannual formal data will be
collected from these groups and their feedback used to
inform program effectiveness and guide program direction.
One overall goal for the program that will be implemented CS 1,2, 9
in the 2014-15 year will be the increased sharing of
assessment results with stakeholders. At the conclusion of
data collection cycles, results will be compiled and made
available to candidates, coaches, instructors, program staff
and other stakeholders. In addition, the results will be
presented during meetings where feedback will be
promoted.

All Candidate
and Program
Assessments

March 2014

19

CS 9