Evaluation Project

THE STUDENT COMMUNITY BOARD
Program Evaluation
Proposal
Alison Reimel
Office of Student Conduct and Conflict Resolution
Loyola University Chicago
EVALUATION PROPOSAL
2
TABLE OF CONTENTS
Evaluation Plan
Theoretical Framework
Program Context
Mission and Values
History and Significant Changes
Key Characteristics
Program Goals
Stakeholders
Program Resources
Logic Model
Assumptions
External Factors
Purpose of Evaluation
Evaluation Format
Strengths and Weaknesses
Evaluation Criteria
Quantitative Evaluation Approach
Research Design
Survey Instrument
Implementation Plan
Statistical Analysis Plan
Quantitative Results Presentation
Qualitative Evaluation Approach
Focus Group Participants
Focus Group Procedure
Implementation Plan
Moderator
Recording Procedures
Logistics and Scheduling
Analysis Plan
Coding Procedure
Validity
Limitations
Timeline
Budget
Next Steps
References
Appendices
Appendix A: SCB Syllabus
Appendix B: Logic Model
Appendix C: Construct Map and Item Matrix
Appendix D: Pre-test Survey
Appendix E: Post-test Survey
Appendix F: Introduction to Qualitative Method Email

4
5
6
6
8
9
11
12
13
13
14
15
15
16
17
17
18
18
19
21
22
22
23
24
24
25
25
26
26
27
28
28
28
29
30
31
33
35
35
41
42
46
49
52
EVALUATION PROPOSAL
Appendix G: Focus Group Protocol
Appendix H: Consent Form
Appendix I: Coding Rubric
Appendix J: Timeline
Appendix K: Budget
Appendix L: Presentation
3
53
55
57
59
60
61
EVALUATION PROPOSAL
4
Evaluation Plan Introduction
The Student Community Board (SCB) at Loyola University Chicago (LUC) is an

undergraduate student organization sponsored, trained, and advised by LUCs Office of Student
Conduct and Conflict Resolution (OSCCR). SCB exists to hear cases of alleged student
misconduct on a peer-to-peer level, hold student accountable for policy violations, education the
student community on LUCs expectation and standards of behavior, and act as an advocate for
the student voice within the student conduct process. Like any other conduct administrator at
LUC, SCB is responsible for meeting with students, discussing an incident of alleged
misconduct, and determining if any violations of Community Standards occurred. Then, if any
violations did occur, SCB determines the sanctions, or required outcomes, that will aid in the
education, engagement, and behavioral change of those students in violation of LUC policies. In
addition to hearing cases of alleged misconduct, SCB also engages in a yearlong leadership
development experience aimed at increasing their efficacy and ability for leadership.
This project will evaluate the intended learning outcomes of the overall SCB experience,
which includes the skills and capacities necessary to carryout an administrative conduct hearing,
as well as the year-long learning goals of the leadership development experience. It is also
intended to better understand the SCB members experience outside the defined learning
outcomes. This proposal will further describe the context of theoretical framework for conduct
administrator skills and leadership development programs, the larger context of SCB, and the
key characteristic of the program. This information has informed my development of a mixed
methods evaluation plan intended to be implemented next Fall 2015. This plan also discussing
the rationale for each component of the evaluation and data analysis, the timeline for
implementation, and the future impact and steps stemming from the evaluation. Overall, the
EVALUATION PROPOSAL
evaluation was developed considering the resources available to OSCCR and has been designed
to be feasible and practitioner-friendly.
As a student conduct administrator, I know first-hand the skill, competencies, and
practice is takes to become an effective hearing officer. It takes intention and continuous hard
work to continue to develop oneself as an effective conduct administrator. In the case of SCB,
we are expecting that undergraduate students possess and practice the skills necessary for this
role. Lancaster and Waryold (2008) identify many skills for effective practice, including a deep
awareness of self and others, practicing integrity, civility, collaboration, and attentiveness in
conversation. Supported by research these are in alignment with many of the expectations
OSCCR has of SCB functions and member development.
Beyond developing in SCB members the skills necessary be effective conduct
administrators, OSCCR also works to develop in them other skills and values needed for great
leadership. Reflected within the skills identified by Lancaster and Warywold, there are also
components of the Social Change Model of Leadership Development, (SCM) a leadership
development used in many colleges and university, including LUC. Specifically, SCM identifies
seven values necessary for student leadership. These competencies include a consciousness of
vales and beliefs, congruence with those values, conflict with civility, collaboration and
commitment, shared values, and citizenship (Cliente, 2009). Just as many of these are also found
in the essential skills identified by Lancaster and Warywold, they are also valued within the
context of student leadership development.
In developing SCB, OSCCR married all of these concepts in order to create an
experience where in members would develop the skills necessary to be effective in the role of
EVALUATION PROPOSAL
conduct administrator, as well as broader leadership skills that could be used inside and outside
of the SCB program. However, as of the current, OSCCR has no way of knowing whether or not
the program is developing students in this light. Therefore, this evaluation is essential to
ensuring that the SCB program is designed to develop the noted skills, values, and competencies.
Before further discussing the specifics of he evaluation plan, the following sections will more
thoroughly develop the context of SCB.
Program Context
As previously discussed, SCB is housed within the OSCCR, which is considered a
department within the larger LUC Division of Student Development. All of SCBs multiple
working parts are consistent with the mission of the OSCCR and reflective of the growth and
development of the department. Not only is SCB a relatively new component of the OSCCR,
but the student organization has also undergone a significant programmatic change within the
last year of its existence. Before discussing the evaluation plan, it is important to further details
the broader context of SCB, as well as the intricate details of the program that directly reflect that
evaluation design. This section first discusses the larger context for SCB including the mission
and values of OSCCR and SCB, the history of the organization, the key characteristics, and then
the key characteristics and goals of the program, as well as relevant stakeholders.
Mission and Values
First, it is important to understand the larger mission and values of OSCCR, as the
mission and values of SCB reflect the offices larger mission. The Office of Student Conduct
and Conflict Resolution (2014) states:
The Office of Student Conduct and Conflict Resolutions (OSCCR) is dedicated to
providing a safe environment for students by promoting responsible decision making and
EVALUATION PROPOSAL
a focus on self-awareness. We also strive to strengthen relationships between students

and their communities by encouraging students to take ownership of the choices they
have made, enforcing accountability and engaging in open dialogue with community
members.
Consistent with LUCs Jesuit tradition, OSCCR strives to uphold the Jesuit value of cura
personalis, or care for the whole person, which is reflected in the offices values. Specifically,
OSCCR (2014) cites the care, integrity, support, safety, and justice as the values that serve as
cornerstones to their work with students. Care speaks to demonstrating compassionate and
concern for all students in the LUC community. Integrity means modeling and holding students
accountable for their behavior as it relates to the Community Standards. Support refers to
responding to a students needs with dignity and respect. Safety speaks to the offices
responsibility to ensuring the personal safety of students as well as the safety of the larger
community. Lastly, justice is a commitment to a fair, consistent, and impartial process for all
students that honors the uniqueness of their identities.
As a function and component of OSCCR, SCB adopts the mission and values of the
office, but has a unique approach to acting on them in their interaction with students during a
hearing. Unlike any other conduct administrator at LUC, SCB consists of students. Because of
their unique position as peers to the students who come before them in hearing, the angle from
which they discuss misconduct and the values of the office and community is different. They
demonstrate care and support from a place of being equal peers rather than administration. SCB
has the unique opportunity to serve as a student role model for integrity and the Community
Standards. They incorporate justice into their work by ensuring that the student voice is
represented in LUCs conduct process. Though SCB has not always been a part of OSCCR,
EVALUATION PROPOSAL
since their inception they have had this unique way of carrying out the mission and values of
OSCCR.
History and Significant Changes
In order to learn more about the history of SCB, I spoke with stakeholder and director of
OSCCR, Dana Broadnax. It was not until 2005 that OSCCR created SCB out of a call from
students to have a voice in the student conduct process (D. Broadnax, personal communication,
September 18, 2014). At this time, many other institutions had student conduct boards that
functioned to do just this. Additionally, OSCCR staff recognized the opportunity for leadership
development for student members. Though there has not been consistent or detailed record
keeping of the exact ways in which SCB has evolved, current staff are still aware of the large
scale program changes over the last nine years. First, from 2005 to 2012, SCB consisted of of
two student boards of six members with alternate members of SCB who only heard cases if a
member of a board could not be present at a hearing. Members were trained in the on the
conduct process at the beginning of the year and there were monthly professional development
sessions that intended to continue to build skills and knowledge related to conduct hearings.
These sessions topics were chosen based upon the needs or perceived needs of the boards at that
particular time.
Over the course of the past two school years, SCB has undergone a few significant
changes. First, a six more student members were recruited and trained, which resulted in the
addition of a third hearing board within SCB. That year, SCB and OSCCR saw an increase in
interest during the member recruitment period (D. Broadnax, personal communication,
September 18, 2014). They determined that it was feasible to add more students to the
organization and did so by increasing the number of student selected for membership. Most
EVALUATION PROPOSAL
significantly, this school year (2014-2015), the professional development model used in the past
has been drastically altered. Through conversation with office professional staff, members
expressed not finding the monthly professional development sessions particularly useful or
engaging. At the same time, the director of the office expressed interest in increasing the
intentionality and impact of SCB to members development as student leaders on campus.
As a result of this feedback, I, as the current graduate assistant and advisor to SCB, spent
a significant amount of time in the spring of 2014 developing a more intentional developmental
program with the input of current SCB members. Discussed further in the next section, I
incorporated values of SMC to the monthly professional development sessions. Now considered
leadership development sessions, each session is based off of a leadership competency identified
from SCM. Considering the short history of the program, there has been little to no other
significant structural changes that current staff is aware of. With the history of the program in
mind, I now want to discuss the key characteristics of the SCB, as there are many working parts
to the current program.
Key Characteristics
This section is broken down into two basic parts, SCB as an hearing board and the
leadership development curriculum. First, SCB consists of fifteen undergraduate students.
Currently there are four sophomores, six juniors, and five seniors. Students are recruited, apply
for, and selected in January and February and are trained the following August. Hence, no first
year students are members of SCB. Within SCB, there are three hearing boards each consisting
of five members. Each board has a Board Chairperson, who is a returning member of SCB and
has demonstrated the skills and commitment needed to fulfill the role. SCB meets every
Wednesday from four to seven pm. In a given month, three of these Wednesday are dedicated to
EVALUATION PROPOSAL
10
hearing cases of alleged student misconduct and one Wednesday is dedicated to a leadership
development sessions. Therefore, typically, each board hears three cases per month and
participated in one leadership development session.
As stated, members are recruited in the winter months. Current SCB members lead the
recruitment efforts, encouraging students to apply using social media, the winter student
organization fair, and outreach to student organizations and campus partners. Students apply,
then are selected for an individual interview with the advisor and current members. Candidates
with high scores then participate in a group interview where they engage in group activities in
order to evaluate team work and consensus building ability. After this, the final selections are
made and selected candidates are asked to join for the next academic year. Typically, SCB
recruits seven to ten new members, as many members choose to return for a second or third year.
The following August prior to the start of school, participate in a two and a half day training
retreat created and led by OSCCR staff that is intended to educate SCB members on the student
conduct process at LUC, as well as develop the necessary competencies for the role. This is also
an opportunity for community building as an entire organization, as well as with their specific
boards.
As a vehicle for continued development throughout the coarse of the year, the second
basic component of the SCB program is the monthly leadership development sessions that are
attended by all members together. As stated, these sessions are based off of the Social Change
Model. According the Wagner (2009), SCM identifies seven values or competencies necessary
for leadership, including consciousness of self, values congruence, controversy with civility,
commitment, common purpose, collaboration, and citizenship. In employing these values,
people are groups are able to effect positive social change within their communities. With
EVALUATION PROPOSAL
11
feedback from SCB members that they wanted to be more present and engaged with their
campus community, I based each monthly leadership development session off of one of these
values, culminating with the implementation of a SCB-planned campus-wide service event.
As described in the syllabus (see Appendix A), the yearlong experience starts by
members engaging sessions that are focused on the self. More specifically, identifying there won
personal values and the ways in which they are congruent with those values. As SCM indicates,
a deep awareness of self is critical for the engagement with others when working toward creating
positive change (Wagner, 2009). Moving into fall and winter, the sessions focus on group
dynamics such as group conflict, identifying shared values and a common purpose, and
collaboration. In engaging with sessions focused on group values, SCB begins planning what
will be the culminating campus-wide service project at the end of the school year. The last of the
values identified by SCM is citizenship, which is a deep engagement with ones community
(Cliente, 2009). The service event, called Project Citizenship, is a way for SCB to engage
themselves and others with the LUC community in a way that honors the mission and values of
OSCCR.
Program Goals
Both of these program components, the conduct hearings and the leadership development
experience, have specific goals as stated in the SCB syllabus (see Appendix A). The training
retreat is aimed at developing specific outcomes identified by OSCCR as necessary for the role
of hearing administrators. Over the course of the year, the weekly hearings should continue to
develop these skills. What I will call hearing skills for the remainder of the proposal, include the
following competencies: active listening, effective questioning, consensus building, navigating
EVALUATION PROPOSAL
12
group dynamics, accountability to actions, critical thinking and analysis, openness to having
viewpoints challenged, and conflict resolution.
Additionally, the goal of the leadership development sessions are grounded in the values
of SCM. Specifically, once completed with all of the monthly sessions, students should have
developed in the following areas: exploring personal values and beliefs, practicing congruence
with those values, make and follow-through with commitments, develops a common purpose
with other SCB members based on shared values, practice collaboration by planning and
implementing a large scale program, engage in constructive conflict with others, and engage
more deeply with the LUC community. Overall, the hearing goals and the leadership
development goals work together to create a developmental experience rooted in SCM and the
mission and values of OSCCR.
Stakeholders
There are a variety of different groups and people who have an interest in the SCB
program and are invested in the members reaching the above goals. First, SCB members have an
interest in the program, as they are the people who are joining this organization for a given
purpose. That purpose might look very different depending on the member, but all are joining
with the intent to gain knowledge, experience, or build relationships in some way. The OSCCR
and larger Division of Student Development have a stake in the SCB, as SCB acts as conduct
administrators and student developers just as the professional staff. Because of this, an
inconsistent and undeveloped SCB compromises the integrity of the office and its mission, as
well as poses a risk to the larger division and the University. As a result, it is essential that
OSCCR and the larger division are confident that SCB members have the ability and necessary
skills to carry out the work, mission, and values of their office. Lastly, the undergraduate
EVALUATION PROPOSAL
13
students who are documented for possible misconduct and come before the SCB have an interest
in SCB. Students who come before SCB have a right to a fair and just process that is grounded
in the values of the OSCCR and the institution. A strong SCB makes for a consistent and fair
process, as well as a more developmental experience for the student.
Program Resources
In order to carry out the functions of SCB, a lot of OSCCR resources are allotted to the
program. A significant amount of monetary resources are funneled into SCB mostly used for
training and community building activities. SCB also utilizes campus partners, such as Campus
Ministry and Residence Life, to facilitate some of the leadership development sessions. Most
notably, the OSCCR graduate assistant, who is the primary advisor of SCB, dedicated about half
of her time to training, supporting, and advising the student organization. Together, these
resources are necessary to the functioning, health, and overall development of the student
members and whole organization.
Logic Model
In order to thoroughly design the evaluation, a logic model (see Appendix B) was created
to outline and organize the key components of the program. The logic model provides a concise
view of the different elements and how they work together to produce the desired outcomes. The
logic model is broken down into three parts. It first outlines the resources that are put into SCB,
called inputs. The inputs include the staff dedicated to training and advising SCB, the amount of
time they put into the program, the amount of money needed to run the program, and other
necessities for program functioning. The second part of the logic model is the outputs. The
outputs capture what SCB does and who is reached. The outputs include the training retreat,
weekly hearings, monthly leadership development sessions, the campus-wide service event, and
EVALUATION PROPOSAL
14
the recruitment and selection of member. Outputs also include the stakeholders as those reached
as a result of the SCB program. These include SCB members, students who come before SCB,
and office and division staff.
Following the outputs, the last component of the logic model is the program outcomes.
The program outcomes are divided into short term, medium term, and long-term goals. In this
case, the short-term goals refer to the intended outcomes of the training retreat and engagement
in hearings. The medium-term goals place more emphasis on the intended outcomes of the
monthly leadership development sessions. In this case, there are no specific long-term goals,
however it is possible that many of the short-term and medium-term goals continued to be
practiced far beyond students membership on SCB. It is also possible that as a result of the
evaluation, we are able to identify if there are long-term effects and what those might be.
Assumptions
In addition to the inputs, outputs, and outcomes, it is also relevant to discuss the
assumptions that factor into this evaluation. First, we are assuming that members of SCB find
their experience valuable in some way. We assume that students join this organization for a
purpose and are looking to get something out of it. This may look completely different
depending on the student, but overall we still assume that they are joining SCB in hope that it
will add value to their educational experience. We are also assuming that all SCB members are
open and have bought into the newly updated leadership development portion of the SCB
experience. The model also assumes that there will be cases of alleged misconduct appropriate
and challenging enough for the SCB to hearing. Whether or not these assumptions are true can
alter the outcome of the evaluation, so understanding them is key to appropriately measure and
analyze the evaluation data.
EVALUATION PROPOSAL
15
External Factors
The final component of the logical model outlines the external factors that may influence
the evaluation and outcomes of the program. First, it is important to consider SCB members
priorities and commitments outside of SCB, as this can effect how engaged students are with the
skills and experience. In the same light, the level of complexity to the conduct cases heard by
SCB also acts as a external factor. The cases are a primary outlet for SCB members to practice
many of the short-term and medium-term outcomes, such as active listening and consensus
building. Program funding and the amount of time and energy able to be committed by the
professional staff to advise SCB is also an external factor. Specifically, funding determines
much of the training resources for SCB. When the graduate assistant and staff members who
advise SCB become consumed with other aspects of their role, the same amount of attention,
feedback, and development cannot be paid to SCB. All of these external factors contribute to the
level at which outcomes are met. Though they can be hard to control for, it is important to
recognize the potential impact they could have on the outcome of the evaluation.
Purpose of Evaluation
The purpose of this evaluation is twofold. We are looking to evaluate the extent to which
student members of the SCB are meeting the intended outcomes of trainings and leadership
development sessions (see Appendix B), as well as gain an understanding of what students are
more broadly gaining from the SCB experience. Specifically, we are looking to answer the
question: what is the developmental experience of student as a result of membership in SCB?
We are interested in what skills and knowledge they are taking away from it, what they would
like to take away from it, and how we can improve the program in the future. With this in mind,
we will be taking an approach that combines formative and summative evaluation in order to
EVALUATION PROPOSAL
16
evaluate the SCB program. The evaluation is summative in that it is concerned with providing
information that will help us to make decision on program changes and continuation (Fitzpatrick,
1997). Is the also formative in that it will likely provide us with a deeper understanding of the
program, as well as how to improve upon it. The evaluation is intended to evaluate the intended
outcomes, as well as the overall experience, or process, of being an SCB member in order to
continuously improve the SCB experience. The outcomes approach will allow us to determine
whether or not students are meeting the intended learning outcomes of each leadership
development session and hearing skills (see Appendix B), and a process approach will allow us
to better understand the member experience and gain a deeper understanding of development.
Evaluation Format
Based on the purpose of the evaluation, as stated, we plan to use an outcomes-based
approach that again takes into account a summative and formative approach. The outcomes
approach is utilized in order to describe or explore the changes that that occur in participants as a
result of program participation (Fitzpatrick, 1997). Outcomes-bases approaches can be used to
measure specified learning outcomes or unintended outcomes and development from a program.
In this case, an outcomes approach is most appropriate for determining whether or not SCB
members develop in the programs intended outcomes, as well explore the outcomes beyond
these specified outcomes.
In order to measure the program goals, we will utilize a quantitative survey will measure
the development of competencies consistent with the program goals. In order to explore the
development of skills and competencies outside of intended outcomes, we will use focus groups.
These focus groups should aid in gaining a deeper understanding of SCB member development.
It should also allow us to gain insight on which parts of the program are most beneficial to the
EVALUATION PROPOSAL
17
development of the intended learning outcomes, therefore allowing us to make programmatic

changes that will enhance the developmental experience.
It is also important to note that the focus groups will be the primary source of data
collection and analysis for this evaluation. As discussed later, due to the small sample size and
focus on program improvement, the focus groups will provide us with the most valuable and
significant information.
Strengths and Weaknesses
Based on the purpose of the evaluation, the outcomes-based approach that incorporates
both formative and summative aspects of evaluation allows us to the opportunity to deeply
explore the developmental experience of SCB members. However, an inherent weakness to this
evaluation format is the vast difference in each individual students experiences. This
uniqueness may contribute to the inability to attribute the outcomes of the evaluation with
participation in the program, as we cannot control for differences in each members LUC
experience. To address this weakness, the focus group aspect of the evaluation is designed to ask
questions regarding the overall process of the SCB experience. These questions address what
parts of the program are students finding valuable to the development of particular competence,
as well as the pieces are they do not find valuable. This information will be essential to the future
improvement of the program. The last potential weakness of the evaluation format is the
possibility that self-reported skills and development may be difficult to measure, as well as be
skewed by what SCB members know are the program outcomes.
Evaluation Criteria
Given the intent to explore the outcomes of the SCB program, the evaluation approach is
defined by two main criteria, which will be used to guide the assessment of the value of the SCB
EVALUATION PROPOSAL
18
program. Hearing skills development is the first criterion and is guided by the short-term goals
of the program (see Appendix B). Leadership development is the second and is guided by the
medium-term goals of the program (see Appendix B). Typically, it is best practice to utilize
benchmarking, comparisons to other programs and national standards (Fitzpatrick, Sanders, &
Worthen, 2003), but considering that there is very little research specific to student conduct
board programs, we will utilize our own standards based off of necessary skill of student conduct
administrators, as well as the values of SCM.
Quantitative Evaluation Approach
This section will discuss the quantitative evaluation approach and will include key
information with regards to design, methods, and analysis. The components of the quantitative
approach has been informed by the learning goals of the SCB program (see Appendix B) as well
as the overarching purpose of the evaluation. These have been organized in a construct map and
item matrix (see Appendix C), which indicates how each component relates to the program
outcomes. In addition to the design explanation, this section will cover the expected analysis of
the data as well as the reporting of the data. Understanding the design of the quantitative
approach is the first step in ensuring that the evaluation provides the information that is needed
for understanding and improving the SCB experience.
Research Design
With the intent of assessing the learning outcomes of the SCB, the quantitative approach is
designed as a pre-experimental, longitudinal study through the use of a pre-test and post-test
survey. This approach is explained as one that allows an evaluator to study a single group,
without a comparison group, over the course of an intervention (Creswell, 2009). Because of the
small sample size, it is likely that we will reach census, which is defined as participation of the
EVALUATION PROPOSAL
19
entire population receiving the intervention, or in our case, participation in the SCB program
(Wholey, Hatry, & Newcomer, 2010). Aiming for census, all fifteen students will be asked to
complete the surveys. The following section outlines implementation strategies to mitigate
coercive recruitment techniques, as well as how we plan to address the incredibly small sample
size. Since the approach is longitudinal in nature, the students will be pre-tested prior to the
training retreat, and then will be post-tested after the completion of the final leadership
development session. This approach was chosen in order to understand the student
developmental growth as they relate to the intended learning outcomes.
As consistent with a pre-experimental design, this evaluation will not utilize a control or
comparison group. Although a lack of comparison groups makes it difficult to attribute any
change or development to the SCB experience, it is not realistic to use one in this design. It
would take up an unreasonable amount of time to identify a comparison group and control for
other experiences that might contribute to the development of similar developmental outcomes.
It is important to take into account this design limitation when analyzing the data and impact of
the SCB experience. Another limitation of this type of pre-test and post-test survey concerns
inaccurate reporting. Student may sense what the evaluator is looking to find which could skew
the survey results. Fortunately, this will be mitigated and supplemented by the qualitative
approach, which will take place later.
Survey Instrument
Because of the expected high response rate and minimal cost of survey implementation, a
survey is a feasible option for assessing the intended outcomes of the SCB experience. In
attempting to use already existing survey instruments, it was difficult to find any that were
specific to the skills the SCB program is looking to encapsulate. For this reason, the survey
EVALUATION PROPOSAL
20
questions in the pre and post-test were created uniquely for the SCB program and as stated, are
informed by the intended learning outcomes of the experience (see Appendix C). These learning
outcomes are outlined in the syllabus and include but are not limited to listening and questioning
skills, effective communication, consensus building, personal values identification, conflict
resolution, and working in a team.
The pre-test survey (see Appendix D) is comprised of 6 questions, several with multiple
parts, and will take approximately three to five minutes to complete. The survey consists of
questions that ask students to identify their basic demographics including class standing and
standing within SCB. Because of such a limited sample size, participants will not be asked to
provide any other demographic information such as race, gender, religion, etc. This information
will be used to identify whether there is a difference in experience based on these factors. These
questions are in a checklist format. There is also a question regarding the motivation to join SCB
and what students are looking to gain from the experience, which are in Likert scale format. In
addition to these questions, the pre-test includes two other Likert scale questions that include
between multiple statements. These are statements reflected the intended outcomes and are
associated with specific goals represented in the logic model (see Appendix C). The scales range
from 3-point to 5-point in order to avoid neutral responses. Five-point and 4-point scales were
used, with 4-point scales being utilized in order to avoid a neutral response.
The post-test survey (see Appendix E) is comprised of three Likert scale questions, again
consisting of multiple statements. Like the pre-test survey, it is estimated to take four to five
minutes to complete. Because the evaluator is seeking to assess the change in the outcomes used
in the pre-test, the same questions will be asked. Likewise, students will be asked again what
they were hoping to gain from participating in SCB. This question is changed to reflect to what
EVALUATION PROPOSAL
21
extent they feel they have gained the given statements. In addition to the questions from the pretest, there is one additional Likert scale questions that addressed topics that are only applicable
once completing a year of the SCB experience. These include how they felt about a multitude of
different experiences within the SCB program.
Implementation Plan
For both the pre and post-test, the surveys will be distributed as a web-based survey upon
arriving to the training retreat and at the end of the last leadership development session. Students
will be asked to bring their laptop, as it has been confirmed that all student have access to a
laptop device. In both cases, the evaluator will leave the room and student will be asked to
spread out as not to coerce any students into completing the survey. In this case, the web-based
surveys allow for simplified evaluation and storage of the collected data.
Although not ideal, the students will be asked to use their name when completing the
survey as the evaluator needs to be able to march the pre and post surveys wen analyzing the
data. In order to maintain the privacy of the students, upon completion of the survey, the
evaluator will replace their name with a numerical code (e.g. 001) and then repeat the same upon
completion of the post-test.
Although the survey is new and the program has been reimaged and restructured in the past
year, a pilot study of the survey instrument is not feasible considering the already very small
sample size. With only fifteen participants, taking one or two from the sample has the ability to
significantly alter the data. However, other stakeholders and recently graduated SCB members
will be asked for feedback on the survey instruments to ensure that they are comprehensive to the
goals of the program. These stakeholders include the Director and the Coordinator of the
OSCCR.
EVALUATION PROPOSAL
22
Statistical Analysis Plan

Since the quantitative aspect of the evaluation plan is aiming to evaluate the outcomes of
the program, the evaluator will use two different tests to evaluate such outcomes. Specifically,
the evaluator will use central tendency measures and paired sample t-test. First, the descriptive
statistics test will measure the mean, median, and mode of the collected data across the Likert
type scales (Huck, 2012). This type of descriptive data allows the evaluator and stakeholders to
understand how the population as a whole met or did not meet the intended outcomes of SCB.
Additionally, this data will help us to understand the development of members based on their
year in school and role in the organization.
Next, as a precaution to the small sample size, we will conduct a power analysis in order to
determine the sample size needed for statistical significance. As described by McCrum-Gardner
(2009), when a sample size is too small, the data is limited to how significant, and therefore
valuable the findings are. It is our hope that using a power analysis, w can increase the validity
and usefulness of our data. In addition to the power analysis, the qualitative component of the
evaluation will also serve to counter the limitation of this small sample size.
Next, the paired sample t-test will allow for a more thorough understanding of the
development of SCB members based on the intended learning goals. Specifically, this test
determines whether or not there is a statistically significant change in the development of the
SCB members upon completion of the experience. Additionally, testing for change across a span
of time will also allow the evaluator to identify any significant differences in growth regarding
year in school and class rank.
Quantitative Results Presentation
EVALUATION PROPOSAL
23
The results of the quantitative portion of the pan will be compiled into a report that uses
graphs, charts, and written explanations of the evaluation findings. The report will be presented
to and discussed with key stakeholders, including the Director and Coordinator of OSCCR, in
conjunction with the findings of the qualitative portion of the report. The evaluator and named
stakeholders will then give a written report to the Associate Dean of Students who oversees the
OSCCR.
Qualitative Evaluation Approach
In addition to a quantitative approach, a qualitative approach adds depth and breadth to
the overall evaluation of the SCB. Specifically, a qualitative component to the evaluation of
SCB will allow the evaluator to gain an understanding of the process and personal experiences of
SCB members. It will provide the evaluators with additional anecdotal information beyond
whether or not intended learning outcomes are being met. A qualitative approach provides the
opportunity for the evaluators to explore growth and development that occurred outside the
constraints of learning outcomes. It can provide valuable feedback about how to improve the
program, making it more valuable for all stakeholders.
In this case, focus groups will be utilized in order to gather a substantial amount of
information from a small sample population. Focus groups reduce the amount of time and
resources needed for individual interviews, but allows for the same amount of participants to be
involved. The focus groups discussions will be based off of the questions asked in the
quantitative survey, but allow for expansion and deepened perspective from student experience.
The focus groups will also allow the evaluator to further explore themes drawn from the
quantitative evaluation (Schuh, 2009). Overall, the use of open ended and probing questions
EVALUATION PROPOSAL
24
should encourage open and honest conversation, resulting in rich data about the experience of
SCB members.
Focus Group Participants
Focus group participants, similar to the survey population, will be a census population,
meaning all current SCB members will be asked to participate. Considering the small number of
participants, the fifteen students will make up three focus groups of five students. First, the
evaluator will ensure that each of the three student conduct board chairpersons are split up
among the three groups. Then, by using maximum variation sampling, the evaluator can ensure
that there are a variety of experiences represented in each group. The intent is for each focus
group to have a mix of members from each of the three boards, year in school, and number of
years as an SCB member. This way, the evaluator is able to identify themes and trends across
the entire group rather than within individual boards.
SCB members will be asked to participate in the focus groups at the conclusion final
leadership development session. This way the evaluator has a captive audience in order to
explain the value of this portion of the SCB evaluation plan. The evaluator will encourage SCB
member participation by providing a free meal of the groups choosing during the focus group
meeting. However, it will be clearly stated that participation is voluntary, as ensure students do
not feel required to participate. As this is a tight knit group already largely invested in the SCB
program, it is expected that participation in the focus groups will be high. After this meeting,
SCB members will also be sent an email (see Appendix F) with more detailed information about
the focus groups, including a link to a scheduling tool called Doodle. This will allow the
evaluator to find a time and day that works for all members of a given focus group.
Focus Group Procedure
EVALUATION PROPOSAL
25
A Focus group protocol will be used in order to provide structure and direction to the
discussion (see Appendix G). The moderator for each of the three focus groups will be a
member of the Division of Student Development who is familiar with SCB, the purpose of the
group, and the new leadership development component of the program, but in no way directly
involved in the creating of the leadership development curriculum, recruiting, training, or
advising of the SCB. This way, SCB members do not have a pre-existing relationship with the
moderator based on their SCB experience that might bias responses or inhibit honesty. As
Creswell (2009) explained, if the evaluator is serving as the moderator, this could bias the
participants responses. The moderation will begin by giving an overview of the agenda for the
hour and a half discussion and further explaining the larger purpose of the focus group and
evaluation of SCB.
Next, the moderator will review the participants consent form (see Appendix H) and ask
the participants to sign it. Since all of the participants already know one another, there is no need
to do formal introductions. At this point, the moderator will begin the protocol questions.
Implementation Plan
In setting the stage for a focus group discussion, the evaluator has given much thought to
ensure that the participants feel that the environment is suitable for open and honest sharing of
their experiences. Keeping in the mind the purpose of this portion of the evaluation plan, the
following sections outline the key components to ensuring that SCB members are able to share
their thoughts, feelings, reflections, and feedback on the SCB program.
Moderator.
The focus group moderator will be Jen Kosciw, a Residence Life staff member within the
Division of Student Development who is familiar with SCB and its purpose on campus, and the
EVALUATION PROPOSAL
26
intended outcomes of the leadership development curriculum. This choice was made in order to
mitigate the potential pressure that student might feel if the moderator was the evaluator or
another member of OSCCR who they frequently work with. The evaluator recognizes that her
presence during this focus group could skew SCB members answers, as she has a close working
and mentoring relationship with many of them. With this in mind, the moderator will be a
neutral party that has experience in conducting focus groups, asking probing questions, and
setting a comfortable space for open and honest feedback. Since the curriculum is not graded
and all identifying information will be excluded, there should be little to no reason for biased
feedback during the discussion.
Recording procedures.
In order to ensure accuracy the information provided, the recording procedures will be
two-part. First, the discussion will be audio recorded using one of OSCCRs recording devices.
There will also be a graduate assistant within the Division of Student Development taking notes
on body language, tone, volume, how many times each person spoke, and other process-oriented
notes. This will act as supplemental information to the interview recording, providing valuable
information that cannot be provided through an interview transcription. Combined, this method
of recording is meant to capture the full group discussion experience. In addition to being audio
recorded, the recording will then be transcribed in full by the graduate assistant taking process
notes. Although transcribing takes a lot of time, the small number of focus group discussions
makes this task more manageable. Additionally, transcribing ensures that all feedback is
captured accurately and identifying information can be discarded.
Logistics and scheduling.
EVALUATION PROPOSAL
27
As touched on earlier, in order to increase participation and create a comfortable

experience for participants, a Doodle (online scheduling tool) will be used to ensure that the
focus group discussion is held during a time that fits within all participants schedules. The
focus groups will be held during the weeks after midterms and prior to reading week and finals
during the spring semester. This is again in an attempt to increase participation and mitigate
factors that might add stress to students experience.
The groups will be help in the same room as the leadership development sessions. This
way, participants are comfortable and familiar with the space, but do not experience the
evaluator or other OSCCR staff members in the space. This is a space where students are used to
holding deeper discussion and reflections, therefore it might be easier for them to get into a
reflective mindset. As stated, a full meal will be provided during the discussion as an incentive,
therefore the evaluator will try to schedule around a meal-time hour if participants schedules
allow.
Analysis plan
The audio recording transcription will be the primary data source to be coded. Since the
focus group protocol has been split into segments based on the key components of the SCB
program, it is logical that the coding uses the same components. Based off of these components,
an initial coding rubric has been developed (see Appendix I). The coding rubric consists of six
different codes, their meaning, and example themes of topics related to that code. Though the
evaluator is looking to gain a deeper understanding of the information gathered from the
quantitative survey, the questions are only loosely reflective of the specific outcomes. The intent
behind this is to also gather information on the process of SCB, allowing the students to speak to
what feels relevant to them in regards to their experience. Although this is not as focused of an
EVALUATION PROPOSAL
28
approach as the quantitative survey, it is essential look beyond learning outcomes especially in
the pilot year of a major change in the program.
Coding procedure.
Using the six codes from the coding rubric, descriptive coding will be applied to the
transcripts using the comment capability in Microsoft Word. The evaluator will then carefully
apply the codes to the whole transcript. This also allows for the evaluator to apply codes
possibly not included in the six codes included in the coding rubric as a way to accurately
capture any unexpected information that arises during the focus group discussions. After each
transcription is coded, the evaluator to will pull the text from each code into separate documents.
This way, it is easier to then identify themes and concepts within each topic. The evaluator and
another staff member in OSCCR will be responsible for coding each focus group discussion as a
way to ensure inter-rater reliability, which increases the validity and rigor of methods and
findings (Wholey et al., 2010).
Validity.
In addition to ensuring inter-rater reliability, the evaluator will also use member-checking
as a way to increase the validity of the methods and findings (Wholey et al., 2010). Immediately
after the transcripts are coded, a summary of the findings will be drafted and sent to the
respective members of each focus group. Members will be encouraged to provide additional
feedback on the themes and concepts pulled from each discussion. This strategy is employed to
ensure that the evaluators accurately captured intended messaged within each group discussion.
As addressed previously, the focus group facilitator will not be a stakeholder in the SCB
program. This is another attempt to ensure that the student feedback is honest and accurate.
Limitations.
EVALUATION PROPOSAL
29
Though the facilitator and transcriber will not be OSCCR staff, the coders are members
of OSCCR and heavily invested in the program. It is important to address this as a possible
limitation to the qualitative evaluation method. Up until this point in the evaluation, the
evaluator and other members of OSCCR staff have been excluded as a way to ensure student
feedback is not skewed. However, the intent in having the evaluator and OSCCR staff code is in
order for them to know and understand the data on a deep level. Also, the personal relationships
between SCB members may also affect the data. It is possible that their relationships with one
another hinders them from sharing honest feedback during the focus group discussions.
However, by using maximum variation sampling, the evaluator hopes to spread out board
members across groups, minimizing the closer relationship within each group. Lastly, using
pre-determined codes may allow for the evaluator to miss or misidentify themes outside of the
six in the rubric. Knowing this, the evaluator is aware of the ability to create more codes if
necessary.
Timeline
In order to ensure that the evaluation process moves forward, a timeline (see Appendix J)
was created to ensure that all steps of the evaluation are carried out in an timely and efficient
manner. Although it is ideal that a similar evaluation plan be implemented each year, this plan
outlines from July of 2015 to June of 2016, beginning just before the next academic calendar and
ending just after. Beginning in July of 2015, I will continue to work with stakeholders in order
to finalize any last detail or adjustments to the evaluation plan. In August, one day before SCB
goes on their training retreat, the pre-test web survey will be distributed. In early September, I
will analyze the data collected from the pre-test survey using SPSS. From October to April, no
tasks need to be completed for the plan. Once April, the tasks increase significantly. The post-
EVALUATION PROPOSAL
30
survey will be distributed early to mid April, along with the invitation to participate in the focus
group interviews. Focus groups will be held late April as to not interfere with final exams in
May. In May, we will analyze the quantitative data, as well as transcribe, code, and analyze the
qualitative data. Finally, we will pull together the report and sharing it with stakeholders in June.
This leaves time over the summer for OSCCR staff to make changes and improvements to the
SCB program based on the results of the evaluation. Although this timeline is in place in order
to keep the evaluation moving forward, we do have the ability to make adjustments to the
timeline if needed.
Budget
In addition to the timeline, a detailed budgeted is necessary to ensure that the evaluation
is feasible and fiscal responsibility is upheld. The budget (see Appendix K) outlines the
expected total costs, as well as a cost breakdown, for an entire phase of the evaluation. Overall,
the costs are fairly minimal. The majority of spending is in the dinner incentive for focus group
participation. Although this cost is not directly related to carrying out the evaluation procedures,
I do not anticipate that stakeholders, particularly the director of OSCCR, will consider this costs
unreasonable since students are giving up a significant amount of time for the programs benefit.
Additionally, without this incentive, we run the risk of an even smaller sample population, which
in turn further decreases the value of the data.
The higher cost of focus group incentives is countered by the extremely minimal cost of
other resources. Because of the nature of their work, OSCCR already owns an audio recording
device, a cost that may have been significant. Also, as a part of LUC, OSCCR has access to the
statistical analysis software, SPSS, needed to analyze that data. Most notably, OSCCR will not
have any extra cost for the staff resources, as the evaluator and focus group moderator,
EVALUATION PROPOSAL
31
transcriber, and coder are graduate assistants or staff within the division. Overall, the low cost of
the evaluation makes it feasible to continue for many years to come.
Next Steps
As this is the first time SCB will be formally evaluated, there are several future steps to
consider in order to continue developing the SCB program. First and foremost, the results of the
evaluation will be used to inform programmatic changes intended to strengthen the
developmental experience of SCB members. It is essential that the data from the evaluation be
used to create a clear and purposeful plan for improving the organization and student experience.
Given the timeline, this plan can be created over the summer months and implemented come the
start of the 2015-2016 school year.
In looking to the future, we also consider improvements to the evaluation itself. As
stated, the purpose of this first evaluation is determine whether learning outcomes are being met
and to gain a deeper understanding of the student experience on SCB. This is a basic evaluation
aimed at understanding the student developmental experience. With this said, there is
opportunity in future to further develop the evaluation plan in order to account for more of the
complexities within the SCB student experience. For example, it is relevant to understand how
the Board Chairpersons development differs from the general member. It is also necessary to
consider members who return for a second and their year, as their development can be tracked
over the course of multiple years.
Finally, we are interested in considering other means evaluating the SCB program in
order to get a full understanding from stakeholders. Specifically, this evaluation does not
incorporate the experience of students who come before SCB in hearings. This student
perspective is essential to understanding not only how developed the skills are of SCB members,
EVALUATION PROPOSAL
32
but also the impact that SCB has on their peers through interaction in the conduct process.
Overall, the hope is the continue to develop a program in which SCB members are achieving the
desired learning outcomes through developmental experiences including conduct hearing and
leadership development activities, resulting in a positive impact to their Loyola community.
EVALUATION PROPOSAL
33
References
Cilente, K. (2009). An overview of the social change model of leadership development. In S. R.

Komives, W. Wagner, & Associates (Eds.), Leadership for a better world:
Understanding the social change model of leadership development (pp. 43- 78). San
Francisco, CA: Jossey-Bass.
Creswell, J. (2009). Research Design: Qualitative, Quantitative, and Mixed Methods
Approaches (3rd ed). SAGE Publications: Thousand Oaks, CA.
Fitzpatrick, J., Sanders, J., & Worthen, B. (2003). Program evaluation: Alternative approaches
and practical guidelines (3rd ed.) New York: Longman.
Fitzpatrick, J. L., Sanders, J. R. & Worthen, B. R. (1997). Program evaluation: Alternative
approaches and practical guidelines (2nd ed.). New York, NY: Addison Wesley
Longman.
Huck, S.W. (2012). Reading statistics and research (6th edition). Boston, MA; Pearson Maki,
P.L. (2010). Assessing for learning: Building a sustainable commitment across the
institution (Second Edition). Sterling, VA: Stylus.
Lancaster, J. M., & Warywold, D. M. (2008). Student conduct practice: The complete guide for
student affairs professionals. Sterling, VA: Stylus
McCrum-Gardner, E. (2010). Sample size and power calculations made simple. International
Journal of Therapy and Rehabilitation (17)1, 10-14.
Office of Student Conduct and Conflict Resoluion (2014). Mission. Retreived from
http://www.luc.edu/osccr/about/mission/
Schuh, J.H. & Associates (2009). Assessment methods for student affairs. San Francisco, CA:
Jossey-Bass
EVALUATION PROPOSAL
34
Wagner, W. (2009). What is social change? In S. R. Komives, W. Wagner, & Associates (Eds.),
Leadership for a better world: Understanding the social change model of leadership
development (pp. 7- 42). San Francisco, CA: Jossey-Bass.
Wholey, J. S., Hatry, H. P., & Newcomer, K. E. (Eds.) (2010). Handbook of Practical Program
Evaluation (3rd ed). Jossey-Bass: San Francisco, CA.
EVALUATION PROPOSAL
35
Appendix A
LOYOLA UNIVERSITY CHICAGO

OFFICE OF STUDENT CONDUCT AND CONFLICT RESOLUTION
2014-2015
SCB 101: SCB and Social Change

Wednesday (once monthly), 4:00 7:00 pm
Damen Student Center
Purpose
The purpose of this curriculum is to provide a developmental experience for SCB members that focuses
on cultivating and refining personal values, navigating group dynamics, and fostering community
engagement in order to create a positive impact on the Loyola community and beyond.

Curriculum Description
The curriculum will consist of monthly, 3-hour sessions (C-Sessions) that focus on the development and
utilization of one or more of the seven leadership values outlined in the Social Change Model of
Leadership Development. The journey through the curriculum will culminate in a final all-member
project that serves the greater Loyola, Rogers Park, and/or Edgewater community.

The curriculum is based on the Social Change Model of Leadership Development (SCM) which was
derived from the research out of the Higher Education Research Institute and developed by Helen and
Alexander Astin. This theory approaches leadership as a process that is based in values, relationships,
and collaboration which result in positive social change.

The following concepts are key to understanding SCM and how we will engage with it for the remainder
of the school year:

Leadership involves collaborative relationships that lead to collective action grounded in the shared
values of people who work together to effect positive change for the betterment of others, the
community, and society (Cilente, 2009, p. 45).

SCM takes a values-based approach to process of leadership, including both personal and group
values.

SCM is based on six assumptions:
1. Leadership is socially responsible, it impacts change on the behalf of others
2. Leadership is collaborative
3. Leadership is a process, not a position
4. Leadership is inclusive and accessible to all people
5. Leadership is values-based
6. Community involvement/service is a powerful vehicle for leadership
EVALUATION PROPOSAL
36
SCM consists of seven key concepts that individuals and groups should strive to develop. These are also
known as the 7 Cs of Change. All seven concepts are buildable and constantly interacting with one
another.

1. Consciousness of Self
2. Congruence
3. Commitment
4. Controversy with Civility
5. Common Purpose
6. Collaboration
7. Citizenship
Expectations
As a member of SCB, you are expected to attend every C-Session ready to engage and participate, as
well as complete any reading or reflection that could be assigned before each session.

Project Citizenship is the culminating project of the curriculum. It is a collaboration of all SCB members,
campus partners, and outside organizations in order to create an opportunity for Loyola students to
engage in service with the outside community. In being a joint effort, all members are expected to share
responsibility in planning and executing the final project as it is developed over the spring semester.

Course Outline
August 21, 22, & 23
SCB Retreat
Description: SCB Retreat serves as the main training for members role
as conduct administrators for the University.
Facilitator: All OSCCR staff
Intended Outcomes:
-
Gain skills and knowledge necessary to successfully carryout a

conduct hearing including active listening, effective questioning,
consensus building, and critical analysis
Build community within and across boards

August 27, 2014
C-Session : Consciousness of Self

Description: From SCM, we understand that consciousness of self is
crucial to enacting leadership for positive change. This session will
explore personal beliefs, values, and attitudes through intentional
reflection using the OSCCR sanction, Values Workshop.
EVALUATION PROPOSAL
37
Facilitator: Dana Broadnax, Office of Student Conduct and Conflict
Resolution
Intended Outcomes:
- Identify personal values, beliefs, and attitudes
- Identify how and where those values are demonstrated and not
- Gain first-hand experience with Values Workshop as a sanction
as a means of more effectively sanctioning students
C-Session: Congruence

September 17, 2014
Description: SCM would also find that congruence to self-identified

values is a key concept of leadership. This session will prompt a deep
exploration of congruence to values, ethics, and integrity.

Facilitator: Lauren Schwer, Campus Ministry
Intended Outcomes:
- Develop a deepened awareness of where personal values do
and do not align with action, behavior, and role on SCB
- Develop strategies for reducing conflict between values and
behavior
C-Session: Controversy with Civility

October 15, 2014
Description: Within any group whose members hold different values

and beliefs, there will always be a degree of differing opinions. In order
for a group to work together toward positive change, it is important
that we can engage in open, constructive, and productive civil discourse
with one another and those outside our group. This session focuses on
identifying the root of conflict and skills to address it during the
moment. (This session is a part of Conflict Resolution Week)

Facilitator: Jess Landis, Office of Student Conduct and Conflict

Resolution
Intended Outcomes:
- Identify how personal values and experiences can conflict with
others when consensus building on SCB
- Develop strategies and skills for addressing conflict within the
board and students who come before the board

EVALUATION PROPOSAL
38
November 19, 2014
C-Session: Commitment
Description: In order to create positive change within a community,

members of that community must be committed and passionate
about doing so. This session focuses on your commitment and
accountability to The Student Promise and the Community Standards
and creating buy-in to these things within your communities.
Facilitator: SCB Board Chairs
Intended Outcomes:
-
-
Identify difficulties in creating buy-in to the Student Promise

and Community Standards
Reenergize members about the role they play in changing the
University culture around decision-making and community
engagement

January 14, 2015
C-Session: Common Purpose

Description: The SCM suggests that common purpose is essential to
group trust and engagement. Common purpose serves as a shared
energy that can be aimed collectively toward a vision or goal. This
session will focus on developing a common purpose that guides our
efforts toward the final project.
Facilitator: Ali Reimel
Intended Outcomes:
-
-
-
Identify shared goals and values explored throughout fall

semester that relate to The Student Promise
Navigate group dynamics in order to determine a common
purpose for Project Citizenship
Delegate initial responsibilities for Project Citizenship

February 15, 2015
C-Session: Collaboration
Description: Collaboration, both within group and beyond group,
multiplies a groups efforts when working towards a common purpose.
Collaboration capitalizes on the different strengths, backgrounds, and
ideas within a group and requires all to take responsibility and
accountability for the success. During this session, SCB members run
EVALUATION PROPOSAL
39
their own collaborative meeting with the intent to continue advancing
Project Citizenship.
Facilitator: SCB members
Intended Outcomes:
-
-
-
-
Navigate group dynamics

Demonstrate accountability to common purpose
Capitalize on strengths, connections, and relationships of
individuals
Show advancement of creating and implementing Project
Citizenship

March 18, 2015
C-Session: Citizenship (Project Citizenship)

Description: Citizenship occurs when one becomes responsibly
connected to the community in which one resides by actively working
toward change (Cilente, 2009, p.54 ). Project Citizenship is a
community service day hosted by SCB members participating alongside
LUC community members. This day serves as a service opportunity for
students with community service sanctions, as well as a service
opportunity for the entire community.
Facilitator: SCB members
Intended Outcomes:
-
-
-
Implement community engagement project that seeks to

address the root of an issue facing our community
Improve ability to work as a team
Bring together a variety of community stakeholders in order to
work toward positive change in the community

April 15, 2015
C-Session: Change
Description: Change is the ultimate goal of the SCM. It advances the
status quo and ultimately creates a better world. Today serves as a
reflection on the change created by you and SCB as a whole this year.
Facilitator: Ali Reimel and Dana Broadnax
Intended Outcomes:
EVALUATION PROPOSAL
40
-
Answer questions: How did I/we improve the status quo? How
did we create a better LUC community this year? How did I/we
demonstrate comfortability with the ambiguity that
accompanies change? How did I see employ my own values,
skills, and strengths to contribute to the common purpose?
Identify avenues for continued positive change
EVALUATION PROPOSAL
41
Appendix B
SCB Program Logic Model
EVALUATION PROPOSAL
42
Appendix C
Construct Map and Item Matrix
Pre-Test
Logic
Model
Component
SQ
#
Question
Answer
Type
Answer
Option
General
Information
Name
Text box
N/A
Select one
Sophomore
Junior
Senior
Check all
that apply
New member
Returning
member
Board Chair
Likert Scale
Check box
3-point scale
Very important
to Not
important
Likert Scale
Check box
3-point scale
Very important
to Not
important
2
Please indicate you class standing.
Please indicate you role on SCB this

year.
Please indicate how important the
following were in your decision to join
SCB using the below scale.
Please indicate how important it is for
you to gain the following from you SCB
experience
Experience that will enhance my

resume
A way to positively impact my

community
Find a sense of community and support

at Loyola
Building my capacity in communication
Improving listening skills
Working in groups
Improving conflict resolution skills
Engaging in consensus building
How do hold myself and others

accountable
EVALUATION PROPOSAL
Short Term
Outcomes
Medium
Term
Outcomes
How would you rate yourself in the

following skills using the following scale
Listening skills
Effective questioning skills
Overall communication skills
Consensus building
Navigating difficult group dynamics
Accountability
Critical thinking and analysis
Openness to having personal views

challenged
Leadership ability
Conflict resolution
6
a
Since starting at Loyola, how often do

you engage in the following
Explore my personal values and beliefs
Practice congruence with my beliefs
Practice commitment and followthrough
Develop a common purpose with others

based on values
Collaborate with peers to create a

change
Engage in conflict constructively
Participate in service or community

engagement activities
43
Likert Scale
Check box
5-point scale
Major strength
to Major
weakness
Likert Scale
Check box
5-point scale
Very often to
Never
EVALUATION PROPOSAL
44
Post-test
Logic
Model
Component
SQ
#
General
Information
General
Information
Answer
Option
Name
Text box
N/A
Select one
Sophomore
Junior
Senior
Check all
that apply
New member
Returning
member
Board Chair
Likert Scale
Check box
3-point scale
Very much to
Not at all
Likert Scale
Check box
5-point scale
Major strength
to Major
weakness
2
Please indicate you class standing.
Short Term
Outcomes:
Hearing
skills
Question
Answer
Type
Please indicate you role on SCB this

year.
Please indicate the extent to which you

have gained the following from you SCB
experience:
Experience that will enhance my resume
A way to positively impact my

community
Find a sense of community and support

at Loyola
Working collaboratively in groups
Built consensus building skills
How do hold myself and others

accountable
How would you rate yourself in the

following skills using the following scale
Listening skills
Consensus building
EVALUATION PROPOSAL
e
Medium
Term
Outcomes:
Leadership
Dev
Accountability
Openness to having personal views

challenged
Leadership ability
Conflict resolution
45
Since starting at Loyola, how often do

you engage in the following
Practice commitment and follow-through
Develop a common purpose with others

based on values
Collaborate with peers to create a

change
Engage in conflict constructively
Participate in service or community

engagement activities
Likert Scale
Check box
5-point scale
Very often to
Never
EVALUATION PROPOSAL
46
Appendix D
Student Community Board: Pre-Test Survey

You are being invited to complete this survey as a member of the Student Community Board.
Your participation is voluntary, but we hope that you will aid in our evaluation of the Student
Community Board as a way to continue improving members experience.
This survey is the first component of the evaluation procedure. At the end of the school year, you
will be invited to take a follow-up survey. Although you will be asked for your name, the results
of this survey will not be shared with anyone outside of the evaluation team.
Should you have any questions, please contact: Dana Broadnax, Director of Student Conduct and
Conflict Resolution at dbrodnax@luc.edu
EVALUATION PROPOSAL
47
Student Community Board: Student Pre-test Survey

1. Name [text box]
2. Please indicate you class standing (choose one):
- Sophomore
- Junior
- Senior

3. Please indicate your role on SCB this year (choose all that apply):
- New member
- Returning member
- Board Chair

4. Please indicate how important the following were in your decision to join SCB using the
below scale:
1- Very important
2-somewhat important
3- Not important
a) SCB aligns with my career and/or academic interests

b)I was looking for a way to get involved on campus
c) I had been through the conduct process before which prompted my interest in
SCB
d)Other (if you select this option, please explain in the box below)
[text box]

5. How would you rate yourself in the following skills using the following scale:
5- Major Strength
4-Strength
3- Average
2- Weak
1- Major Weakness
a)
b)
c)
d)
e)
f)
g)
h)
i)
j)
Listening skills
Consensus building
Accountability
Openness to having personal views challenged
Leadership ability
Conflict resolution

6. Since starting at Loyola, how often do you engage in the following:
5- Very Often 4- Often 3- Sometimes 2- Seldom
a)
b)
c)
d)

Practice commitment and follow-through
Develop a common purpose with others based on values
1- Never
EVALUATION PROPOSAL
e) Collaborate with peers to create a change
f) Engage in conflict constructively
g) Participate in service or community engagement activities

7. Please indicate how important it is for you to gain the following from you SCB experience:
3- Very important
2-somewhat important
1- Not important
a)
b)
c)
d)
e)
f)
g)
h)
i)

A way to positively impact my community
Find a sense of community and support at Loyola
Working in groups
Engaging in consensus building
How do hold myself and others accountable
Thank you for completing the SCB Pre-test survey!
48
EVALUATION PROPOSAL
49
Appendix E
Student Community Board: Post-test Survey

You are being invited to complete this survey as a member of the Student Community Board.:
Your participation is voluntary, but we hope that you will aid in our evaluation of the Student
Community Board as a way to continue improving members experience.
This survey is a follow- up to the survey you took at the beginning of the school year. Like that
survey, all information you provide will only be shared with the evaluation team.
Should you have any questions, please contact: Dana Broadnax, Director of Student Conduct and
Conflict Resolution at dbrodnax@luc.edu
EVALUATION PROPOSAL
50
Student Community Board: Post-test Survey

1. Name [text box]
2. Please indicate you class standing (choose one):
- Sophomore
- Junior
- Senior

3. Please indicate your role on SCB this year (choose all that apply):
- New member
- Returning member
- Board Chair

4. How would you rate yourself in the following skills using the following scale:
5- Major Strength
4-Strength
3- Average
2- Weak
1- Major Weakness
a)
b)
c)
d)
e)
f)
g)
h)
i)
j)
Listening skills
Consensus building
Accountability
Openness to having personal views challenged
Leadership ability
Conflict resolution

5. Since starting at Loyola, how often do you engage in the following:
5- Very Often 4- Often 3- Sometimes 2- Seldom
a)
b)
c)
d)
e)
f)

Practice integrity and congruence with my beliefs
Make a commitment and follow-through with it
Collaborate with peers to create a change
Engage in organized conflict
Participate in service or community engagement activities
a)
b)
c)
d)

A way to positively impact my community
Find a sense of community and support at Loyola
Built my capacity for effective communication
1- Never

6. Please indicate the extent to which you have gained the following from you SCB experience:
3- Very Much 2-Somewhat
1- Not at all
EVALUATION PROPOSAL
e)
f)
g)
h)
i)
Improved listening skills

Worked collaboratively in groups
Improved conflict resolution skills
Built consensus building skills
How to hold myself and others accountable

Thank you for completing the SCB Post-test survey!
51
EVALUATION PROPOSAL
52
Appendix F
Qualitative Introduction Email
Hi ______,
Recently at our last leadership development session, I talked to you all about participating
in a focus group discussion about your experience as an SCB member. As a valuable member of
SCB, I would highly encourage you to participate in one of the three, one-hour discussion. Like
I said during out meeting, a meal of your groups choice will be brought in for you all.
Your feedback is essential to ensuring that SCB continues to be a valuable and positive
experience for members. Considering all of the changes to SCB this year, we would really like to
know how it went and how it could be better.
The link below will prompt you to select your availability over the next two weeks. The
discussion will be scheduled during a time that works for you and you get free food. What more
is there to ask for!
link
Please let me know if you have any more questions about the focus group.
Best,
Ali Reimel
Graduate Assistant
Office of Stduent Conduct and Conflict Resolution
areimel1@luc.edu
773.508.8905
EVALUATION PROPOSAL
53
Appendix G
Focus Group Protocol
Preliminaries:
Introduction: Hi everyone, my name is __________. My role is at Loyola is _________. I am
really looking forward to our discussion today. For the next hour to hour and a half we will be
talking about your experience as an SCB member over the past year.
Purpose: The purpose of this focus group is to gain a deeper understanding of your experience as
SCB members. I will be providing questions that will guide our conversation, but this is meant to
be a space for you all to provide open and honest feedback about your experience. Please feel
free to share as much or as little as you feel comfortable with. I will be recording this discussion
and then transcribing it, so all of the identifying information will be excluded. This information
will be used to evaluate SCB as a program and leadership development experience. Do you have
any questions so far?
Now, I will direct your attention to the consent form in front of you. Please take a couple minutes
to read it and let me know if you have any questions or concerns. If not, please sign it and pass it
in to me. If you havent already gotten your food or would like more, feel free to get some more
and we will start shortly.
Focus Group Discussion:
First, why did you choose to join SCB?
Segment One (overall experience):
1. Can you share with me your general feeling about your experience on SCB this year?
a. What did you like? What did you not like?
b. How did you feel about the format and structure of SCB?
c. How did you find SCB beneficial?
d. How could SCB continue to be improved in the future?
e. Was there anything you wanted from your SCB experience that you did not get?
f. Was there anything that you gained from SCB that you did not expect?
Segment Two (hearing skills development):
1. What skills were needed to be successful in hearing cases of alleged student misconduct?
2. How were these skills developed over your time in SCB?
a. Did you feel prepared to hear cases of alleged student misconduct?
b. What about the training retreat was most beneficial?
c. How could the training retreat be improved?
3. How will you utilize these skills beyond SCB?
Segment Three (leadership development):
1. What are you general feeling about the leadership development curriculum?
EVALUATION PROPOSAL
54
a. For returning members, how would you compare last year with this year in
regards to the curriculum?
b. What parts of the curriculum did you like?
c. What parts of the curriculum did you dislike?
d. How could the curriculum be improved?
2. What skills or insight did you gain from the leadership development sessions?
a. Was there any session or topic that were extremely beneficial? Not beneficial?
b. How will you utilize these skills beyond your SCB experience?
3. How do you feel that you impacted the LUC and great community?
a. How did you effect change through the hearing piece of the SCB experience?
b. How did you effect change through the culminating service day, Project
Citizenship
Conclusion:
We have reached the end of my questions. Does anyone want to add any feedback, additional
questions, or comments that they would like to add?
Again, thank you for your participation in the focus group. You input and feedback is invaluable
in continuing to improve SCB in the years to come. I hope that you have also found this
conversation to be an opportunity to reflect on your own experience and how you can keep
utilizing all that you have gained in order to continue making a positive impact in your
community. Please feel free to contact me or Ali Reimel if you have further questions.

EVALUATION PROPOSAL
55
Appendix H
Consent Form
Project Title: Student Community Board Focus Group

Evaluator: Ali Reimel
Introduction: You are about to take part in a focus group to help the OSCCR gain a better
understanding of your SCB experience. You are being asked to participate in this focus group
based upon your completion of the follow-up survey conducted.
Please read through the following information carefully. If you have any questions regarding the
information below, please ask the moderator prior to deciding whether to participate.
Purpose: The purpose of this focus group is to gain information about SCB members
experiences.
Procedures: Once you agree to participate in the focus group, you will be asked a series of
questions pertaining to your perspective on SCB. Please respond openly and honestly, as there is
no right answer. Please know that you are not required to respond to any of the questions and
should only respond based on your comfort level
Risk/Benefits: There are no known risks involved in participating in this study. Benefits to
participating in this study may include further improvement to ACB and a deepened
understanding of learned outcomes experienced as a result of participation the student
organization.
Confidentiality: By participating in the focus group, your name will be omitted from any
responses you provide. Any information provided will be compiled in a report and basic themes
will be shared with your prior to finalizing it. The information shared today will only be shared
with OSCCR staff.
Voluntary Participation: Participation in this focus group is voluntary. At any point in the focus
group, you are free to withdraw from participation. Please know that you are not required to stay
for the entire time.
Contacts and Questions: If you have any questions about the focus group, you may contact Ali
Reimel at areimel1@luc.edu
Statement of Consent By signing below, you will be agreeing to the information provided around
the focus group. Additionally, you have had a chance to ask any concerning questions and agree
to participate in the following interview. A copy of this document will be provided for your
records.
_______________________________________
Participant's Signature
EVALUATION PROPOSAL
56
_____________
Date
_______________________________________
Evaluators Signature

EVALUATION PROPOSAL
57
Appendix I
Coding Rubric

Construct Code
Meaning
Theme
HRG
Hearing
CURR
Curriculum
PRO SKL
Hearing Skills
PER SKL
Personal Skills
LDRSHP
COMM
Leadership
Development
Community
POS
Positive Feedback
NEG
Negative Feedback
Comments related to hearings, including feelings,

experiences, and happenings
Comments related to leadership development
sessions and overall curriculum
Comments related to communication skills,
consensus building, conflict resolution, training
retreat, etc
Comments related to values identification,
integrity, etc
Comments related to the Social Change Model,
definitions of leadership, team works, etc
Comments related to SCB impact in the
community, community engagement, etc
Comments related to what they liked or enjoyed
about their SCB experience
Comments about what they did not like or what
could be improved about their SCB experience
EVALUATION PROPOSAL
58
Appendix J
Evaluation Timeline
2015
Task
July
Finalize
evaluation
plain with
OSCCR
Administer
pre-test
(day prior to
training
retreat
Enter pretest data in
SPSS
Aug
Sept
Oct
2016
Nov
Dec
Jan
Feb
March
April
May
Administer
post-test
survey
(immediately
after final
leadership
development
session)
Invite SCB
members to
participate in
focus groups
Enter posttest data

into SPSS
Condcut
focus groups
Run analysis
of
quantitative
data in SPSS
and organize
Listen to and
transcribe
focus group
interviews
June
EVALUATION PROPOSAL
Code and
analyze
qualitative
data
59
Prepare final
report
Share report
with
stakeholders
EVALUATION PROPOSAL
60
Appendix K
Evaluation Budget
Activity
Item
Cost per item
Quantity
Total $
$0.00
30
$0.00
$0 (pre-owned)
$0.00
$12
15
$180.00
Soda
$2.00
$6.00
Plate, cups, utensils
$1.00
15
$15.00
$0
$0.00
$0 (pre-owned)
$0.00
Spiral bound reports
$5
$25.00
Power point
$0
$0.00
Survey
Administration
Google form
Focus Groups
Audio Recording Device
Dinner
Room reservation
Statistical Analysis
SPSS
Reporting
Total
cost:
$226.00
EVALUATION PROPOSAL
61
Appendix L
Presentation
EVALUATION PROPOSAL
62
EVALUATION PROPOSAL
63
EVALUATION PROPOSAL
64
EVALUATION PROPOSAL
65
EVALUATION PROPOSAL
66
EVALUATION PROPOSAL
67
EVALUATION PROPOSAL
68
EVALUATION PROPOSAL
69

Evaluation Project

Загружено:

Сведения о документе

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Evaluation Project

Загружено:

Авторское право:

Доступные форматы

THE STUDENT COMMUNITY BOARD

The Student Community Board (SCB) at Loyola University Chicago (LUC) is an

a focus on self-awareness. We also strive to strengthen relationships between students

development of the intended learning outcomes, therefore allowing us to make programmatic

Statistical Analysis Plan

As touched on earlier, in order to increase participation and create a comfortable

Cilente, K. (2009). An overview of the social change model of leadership development. In S. R.

LOYOLA UNIVERSITY CHICAGO

SCB 101: SCB and Social Change

Facilitator: All OSCCR staff

Gain skills and knowledge necessary to successfully carryout a

C-Session : Consciousness of Self

Description: SCM would also find that congruence to self-identified

Facilitator: Lauren Schwer, Campus Ministry

C-Session: Controversy with Civility

Description: Within any group whose members hold different values

Facilitator: Jess Landis, Office of Student Conduct and Conflict

November 19, 2014

Description: In order to create positive change within a community,

Facilitator: SCB Board Chairs

Identify difficulties in creating buy-in to the Student Promise

C-Session: Common Purpose

Facilitator: Ali Reimel

Identify shared goals and values explored throughout fall

Facilitator: SCB members

Navigate group dynamics

C-Session: Citizenship (Project Citizenship)

Facilitator: SCB members

Implement community engagement project that seeks to

Facilitator: Ali Reimel and Dana Broadnax

Please indicate you role on SCB this

Experience that will enhance my

A way to positively impact my

Find a sense of community and support

Building my capacity in communication

Improving listening skills

Improving conflict resolution skills

Engaging in consensus building

How do hold myself and others

How would you rate yourself in the

Effective questioning skills

Overall communication skills

Navigating difficult group dynamics

Critical thinking and analysis

Openness to having personal views

Since starting at Loyola, how often do

Practice congruence with my beliefs

Practice commitment and followthrough

Develop a common purpose with others

Collaborate with peers to create a

Engage in conflict constructively

Participate in service or community

Please indicate you role on SCB this

Please indicate the extent to which you

Experience that will enhance my resume

A way to positively impact my

Find a sense of community and support

Building my capacity in communication

Improving listening skills

Working collaboratively in groups

Improving conflict resolution skills

Built consensus building skills

How do hold myself and others

How would you rate yourself in the