Академический Документы
Профессиональный Документы
Культура Документы
Table of Contents
Introduction
History of Preview Days
Inputs
Purpose and Assumptions
Format and Structure
The Buddy Program
Higher Education Program Information
Workshops and Panels
Campus Tours
Socials
External Factors
Stakeholders
Evaluation Approach
Quantitative Approach
Population and Sampling
Design and Instrument
Pilot Testing and Implementation
Analysis and Results Presentation
Qualitative Approach
Participants
Interview Protocol
Implementation
Limitations
Budget
Timeline
Next Steps
References
Appendices
A. Logic Model
B. Previous PD Schedules
C. Preview Days 2010 Evaluation Findings
D. 2013 Instrument
E. Survey Instrument
F. Construct Map
G. Email Templates
H. Interview Protocol
I. Interview Consent Form
J. Interviewers Notes
K. Email Templates
L. Initial Codes
M. New GA Compensation and Structure Email
N. Timeline
O. Budget
P. Presentation
3
4
6
7
10
10
11
11
13
13
14
15
16
18
18
18
21
23
24
25
26
27
29
30
31
32
33
34
35
38
46
58
67
73
75
78
79
80
81
82
85
86
87
Specifically for students looking to pursue a degree in student affairs, interview and
outreach programs are one of the ways students look to better understand campus culture and
make a more informed decision on the school where they want to pursue their masters degree.
Depending on the program, these programs are often used as requirements for admission or
consideration of an assistantship. At Loyola University Chicago (LUC), the Preview Days
program is a partnership between the School of Education, Academic Affairs, and the Division
of Student Development (DSD) to help recruit and fill Graduate Assistant (GA) positions across
various functional areas within the university. The GA positions filled during Preview Days
supplement the coursework in various masters programs across the university, including the
Master of Education in Higher Education program (Division of Student Development, n.d.).
Each assistantship allows students the ability to engage in professional development
opportunities, while receiving financial support for their graduate studies (in the form of tuition
remission and a stipend, depending on position). Graduate Assistants work in departments
across the University, including, but not limited to, Student Activities and Greek Affairs, Student
Diversity and Multicultural Affairs, Residence Life, Academic Affairs, and Campus Ministry.
GAs are typically offered ten month contracts, lasting the duration of their academic program
(two years for Higher Education masters students) and work about twenty hours per week, on
average (Division of Student Development, n.d.).
In addition to providing a space to conduct interviews for the various GA position
openings, Preview Days provides an opportunity for the Universitys partners to interact with
potential graduate students and help candidates articulate the Loyola GA experience.
Throughout the two days of the program, participants are invited to attend various lectures,
panels, tours, and socials in order to better understand the institution and the opportunities
available at LUC. Even though the Preview Days program generally caters to Higher Education
masters students, participants from other graduate preparation programs (Social Work, Pastoral
Studies, Business, etc.) are invited to attend as well.
History of the Preview Days Program
Prior to 2006, each department within the Division of Student Development (DSD) and
Academic Affairs used their own process to recruit, interview, and hire Graduate Assistants
(GAs) for their open positions. Due to the lack of consistency among the various departments,
GA candidates were met with a confusing process that required them to submit similar and
redundant application materials for each GA position. Jack McLean, JD, Assistant Vice
President for the DSD, notes that even though many departments shared application materials
and processes, there was little collaboration to create a cohesive process and hiring timeline
(personal communication, September 25, 2014). Driven by requests from individual departments,
the DSD was charged with creating a Preview Days experience that introduced candidates to
Loyola and reflected the changing demographic of the Higher Education masters program
(specifically, increased student interest from outside the Midwest). Under McLeans leadership,
the GA Recruitment and Selection Team was created to help streamline the entire process, which
included planning the components of the Preview Days program.
The original Preview Days program was first introduced in 2006 as the Graduate Intro
Days, which invited GA candidates to Chicago to interview for all open positions in a more
centralized process. As graduate enrollment increased in the Higher Education program and the
DSD expanded their GA program, the Division focused on creating a development plan for their
GAs. This included the creation of the GA Formation Council and the introduction of the GA
orientation, which was used as a supplement to the Intro Days program in 2008. In 2009,
under new leadership, the DSD revised the GA recruitment and selection process to increase
collaboration between the Higher Education program and the various divisional partners in order
to create a more meaningful experience for prospective Graduate Assistant (GA) candidates. It
was at this point that the Division of Student Development (DSD) changed the name to Preview
Days, signaling a renewed focus on introducing GA candidates to the University and the GA
experience. McLean explains that the initial vision for Preview Days was modeled after the
program at Seattle University. In this new model, interviews would be paired with sessions
about the Loyola GA experience and information about moving to and living in Chicago (Jack
McLean, JD, personal communication, September 25, 2014).
Since 2009, the GA Recruitment and Selection Team have worked to create a more
coordinated and centralized process by evaluating and assessing the various components of the
process (see Appendix C and D for a summary of the evaluation findings from 2010 and the
evaluation instrument used in 2013). One of the more significant changes is the move from
paper application materials to a fully electronic system of gathering and evaluating applicant
materials and demographic data (J. Curtis Main, personal communication, September 25, 2014).
Using this new online system, GA candidates are able to submit their applications in one
centralized location and indicate their intention to attend the Preview Days program. On the
administrative end, hiring supervisors (individuals selected by their departments to coordinate
the hiring process for their GAs) are able to gather candidate information and easily identify
possible candidates for their open positions. This also allows for more consistent communication
from the DSD and an easier process for GA candidates to inquire about the assistantship
offerings and Preview Days program. As a result of the Divisions efforts to create a more
streamlined process and training sessions offered by the Higher Education program to help GA
supervisors understand the graduate application process, the DSD has seen an increase in the
number of submitted applications and overall attendance at Preview Days.
Inputs to Preview Days Program
The Preview Days program requires collaboration between DSDs departments and
partners in order to create a positive experience for prospective GAs. As our logic model
(Appendix A) illustrates, the execution of the Preview Days program requires the interaction and
use of four groups of inputs: prospective Graduate Assistants (GAs), time, resources, and
personnel. Since the Preview Days program is planned for the benefit of prospective GAs, it is
important to consider their input and behaviors when considering the many components of the
program. Without the attendance of prospective GA candidates, the Preview Days program
would not achieved its desired goals and outcomes (discussed in the next section).
Jack McLean and J. Curtis Main (personal communication, September 25, 2014) both
expressed the considerable amount of time that goes into the planning and execution of the GA
recruitment and selection process. Beginning in late June, members of the Division of Student
Development (DSD) and the Higher Education program get together to evaluate the process from
the previous year and set internal and external deadlines for the departments and prospective GA
candidates. During this planning stage, the various departments work to define the dates for
Preview Days (usually late February) and the timeline for the planning and execution of the
program. In addition, individual departments set aside time to review applications and plan for
the potential openings in their departments for the next academic year.
The Preview Days program must also take into account the various resources needed to
provide an impactful experience for the potential GA candidates. During the program,
participants are also treated to a selection of food from various Chicago restaurants, which
require the commitment of monetary resources from the Division of Student Development
(DSD). In addition to the cost of food, the DSD must cover the cost of printing all materials
(Preview Days guide, nametags, and handouts) used throughout the program (J. Curtis Main,
personal communication, September 25, 2014).
Finally, the Preview Days program enlists the participation of the personnel within the
DSD in various capacities. In order to plan and execute the entire GA recruitment and selection
process, the DSD relies on two committees: the Graduate Assistant Recruitment and Selection
Team (GARST) and the Preview Days Planning Committee (PDPC). Whereas the GARST
works to plan the entire recruitment and selection process for the DSD, the PDPC works to plan
the various components of the Preview Days program. Both of these committees require the
DSDs personnel to commit time to carrying out the process and work in concert to create a
positive experience for prospective GAs. In addition to serving on one of the planning
committees, Divisional staff and current GAs are also invited in a number of capacities
(presenting, attending a social, leading a campus tour, etc.) to interact with the prospective GAs
throughout Preview Days. Since the Preview Days program is planned in partnership with the
Higher Education program, faculty members are also invited to interact with candidates and
provide an introduction to the Higher Education program (Jack McLean, JD, personal
communication, September 25, 2014). It is assumed that throughout this process, University and
Divisional personnel will be available and willing to dedicate their time and resources to the
execution of this program (Appendix A).
Purpose and Assumptions of the Preview Days Program
Even though the Preview Days program has existed in its current form since 2009, the
program lacks explicit learning outcomes to help guide the planning and execution of the
program (Jack McLean, JD, personal communication, September 25, 2014). When assessing a
specific program, stated learning outcomes are essential to understand the reasons why the
program exists and what the program aims to achieve. As Weiss (1998) explains, when goals
are unclear or ambiguous, more than the evaluation can be affected. Where there is little
consensus on what a program is trying to do, the staff may be working at cross purposes (p. 53).
In the absence of specific learning outcomes for the program, McLean explains that the primary
focus of the Preview Days program is to expose participants to Loyola and the Division of
Student Development (DSD) in order to assess whether or not Loyola will be a good fit for a
candidates graduate studies. The Preview Days program works to achieve this outcome by
partnering with the Higher Education program to expose students to the expectations of the
program, introducing the Jesuit mission of Loyola University Chicago, helping departments
coordinate their outreach to potential candidates, and expose candidates to the diverse pool of
applicants and staff within the DSD. By highlighting these specific components of the Loyola
GA experience, McLean hopes that prospective GA candidates will walk away from Preview
Days having a better understanding of their fit at Loyola as well as the unique aspects of the
DSDs vision for the GA experience.
When assessing the level at which these learning outcomes occur, it is important to
highlight the assumptions that can be made about the DSDs approach to the Preview Days
program (listed in Appendix A). As we will discuss later, there is no formal requirement that
each participant attend all of the programs offered during Preview Days. Candidates may either
be taking part in interviews or choose to not attend various programs. By structuring the
Preview Days program in this way, the Division of Student Development (DSD) assumes that
participants will attend all applicable workshops and sessions during their time at Preview Days.
Various factors may lead to a participants decision to attend a specific session during their
experience, which may change their perspective on the usefulness or impact of the Preview Days
program on their choice to attend Loyola for graduate school. This assumes that participants will
be aware of all the opportunities at Preview Days and will know what sessions to attend in order
to better understand the Loyola Graduate Assistant (GA) experience.
In addition, the program assumes that potential GA candidates will have little to no prior
knowledge about Loyola University Chicago, Loyolas GA experience, and the city of Chicago.
In our examination of the Preview Days schedule and program offerings, discussed later
(Appendix B), many of the sessions are planned and presented to candidates with the intent to
educate candidates about the unique opportunities at Loyola and in Chicago. For example, there
is time set aside in the schedule to talk about the mission and vision of Jesuit education (Jesuit
Education 101: Jesu-WHAT?). When planning this program, the assumption is that the
participants posses little understanding about the Jesuit educational practices. Even though many
of the participants may not have knowledge of the Jesuit education model, those that have prior
knowledge about the topic may not experience the same impact as a result of the session. Using
this framework to guide the participants experiences during Preview Days, there is an
assumption that the programs presented will meet the needs and retain the interest of prospective
students. In its current format, students are not asked prior to Preview Days about the questions
they may have regarding Loyola and the Loyola GA experience. Previous evaluations have been
conducted, focusing on the logistics (communication, flow of the schedule, etc.) and the
programmatic aspects (various sessions) of Preview Days (Appendix D). Absent in previous
evaluations is a focus on the participants overall experience (in relation to the learning
outcomes) as well as an assessment of the information participants wish they would have
10
received during the process. The Division of Student Development (DSD) does use previous
evaluations (see Appendix C) to frame their conversations about individual sessions, however
the evaluations are used to understand how the content should be presented during Preview Days.
Format and Structure of Preview Days
The Preview Days program, in its current form, takes place over two days and is held at
Loyolas Water Tower and Lake Shore Campuses in late February (see Appendix B for previous
Preview Days schedules). Candidates are invited to attend if they are offered an interview by the
hiring supervisors throughout the Division of Student Development (DSD) and are scheduled
according to the number of interviews offered to the candidate. Typically, the Preview Days
program begins with the Higher Education program introduction and dinner on the first day of
the program and ends once the candidates interviews have completed on the second day.
During the second day of the program, interviews are conducted simultaneously with the
programs planned by the Preview Days Planning Committee (PDPC). Over the past three years,
the opportunities available to Graduate Assistant (GA) candidates consist of the buddy
program, Higher Education program information, workshops and panels, campus tours, and
socials, which are spread throughout the two days of Preview Days (see Appendix A).
The Buddy Program
Unlike other institutional programs, candidates are not required to attend the Preview
Days program in order to be considered for a Graduate Assistantship or admittance into the
Higher Education program. For the last few years, PDPC has paired current GAs with applicants
as a way to bridge the gap between individuals who attend the program and those that choose not
to attend. Current GAs are paired with six to ten GA candidates and are instructed to reach out
11
via email and act as a resource if GA candidates have any questions about either the recruitment
and selection process or about Loyola.
Higher Education Program Information
During the first day of the program is the Higher Education Program Overview &
Dinner, which usually takes place at Loyolas Water Tower Campus (Appendix B). During this
session, candidates hear from program faculty about the various components of Loyolas
program and are encouraged to interact with faculty and current students over dinner. Since an
outcome of the Preview Days program is to be introduced to Chicago, the Preview Days
Planning Committee uses meal times as a way to expose candidates to the various food offerings
in Chicago.
In addition to the information session on the first night of the program, Higher Education
students are encouraged to attend an optional class during on the second day of the program. In
the past, students have typically attended the Student Development Theory class, but are asked to
RSVP in advance since there are limited seats in the classroom. This offers candidates another
opportunity to interact with the faculty and understand the academic experience at Loyola.
Since many of the Graduate Assistant (GA) positions do not require students to be
enrolled in the Higher Education program, candidates from other programs (Social Work,
Pastoral Studies, Business, etc.) are invited to interview during the Preview Days program.
These candidates are invited to the activities on the first night of the program, but are not
required to attend and encouraged to reach out to their respective programs of study.
Workshops and Panels
Throughout the Preview Days program, GA candidates are invited to attend various
sessions that help them to understand the Loyola GA experience. Over the years, the overall
12
framework of the sessions has remained unchanged (Appendix B), even though the content
presented may look different from year to year. Starting with the first night of the program,
candidates participate in a panel of current masters students in the Higher Education program,
who speak to their experience with the academic components of the program. The Preview Days
Planning Committee (PDPC) works to gather a diverse set of students who have varying
experiences with the Higher Education program (full and part time, current GAs, postundergraduate experiences, etc.) in order to offer different perspectives about the program. In
addition to the panel on the first night, an additional panel is offered during the second day,
focusing on the Graduate Assistant (GA) experience and the relationship between GAs and
University personnel. During both panel discussions, prospective GAs are encouraged to ask
questions that help to clarify their intent to join Loyola as a GA.
In addition to the panels offered during the Preview Days program, GA candidates are
also invited to attend sessions and workshops about the Jesuit and social justice mission of
Loyola as well as the opportunities available to GAs living in Chicago. Beginning with the
Division of Student Development welcome, usually delivered by the Vice President for Student
Development, candidates are exposed to the Loyola student GA experience. As an introduction
to the Jesuit identity of Loyola and the Jesuit education model, Jesuit Education 101: JesuWHAT? is presented by staff members from Campus Ministry. Since many candidates come to
Preview Days with little knowledge about Jesuit education, this session serves as a way for
students to understand the characteristics of Jesuit education and understand how Loyolas Jesuit
identity may frame their work at the institution. In conjunction with faculty members in the
Higher Education program and divisional staff, candidates are also engaged in a discussion about
social justice and how it informs professional practice at Loyola. Given that the Higher
13
Education program has a focus on social justice, this session has historically allowed candidates
to engage with each other in conversations about social justice and introduces them to the types
of conversations and material covered within the classroom.
Finally, the Moving to Chicago presentation offers candidates an opportunity to learn
about living options in Chicago as well as the opportunities available in the city. Since most
Graduate Assistants (GAs) will live off campus, this session is intended to provide information
about living in the city and is usually presented by GAs and the Division of Student
Development (DSD) staff who live in the neighborhoods surrounding Loyola. All of these
programs are intended to help candidates assess their fit at Loyola as well as educate them
about the mission of Loyola.
Campus Tours
During the Preview Days program, participants have many opportunities to tour the Lake
Shore and Water Tower Campuses. During these tours, led by current GAs, candidates are
exposed to the physical layout of Loyolas campuses and are introduced to the various campus
resources graduate students use frequently. During the tour, candidates are able to see the
various facilities and become somewhat familiar with the campus. This is another opportunity
for participants to interact with current GAs and gain a better feel for the campus environment at
Loyola.
Socials
In addition to the Higher Education Program Overview and Dinner on the first night,
candidates are given the opportunity to interact with undergraduate and graduate students, DSD
staff, and faculty members. In previous evaluations, the Preview Days Planning Committee has
observed that candidates have requested more time set aside to meet with the members of the
14
Loyola community. Throughout the past few years, these social aspects have been included into
meal times, structured settings (Social with Grads and Undergrads), and throughout the
interview process. The Preview Days Planning Committee has worked to make sure candidates
can have their questions answered by staff members and current Graduate Assistants (GAs)
throughout out the process. DSD staff operate a Hospitality Booth throughout the second day
of the program and current GAs are able to volunteer to mingle with candidates in the interview
preparation space to calm nerves and serve as a resource.
External Factors
In addition to the assumptions made by the DSD about the achievement of the learning
outcomes, there may be various external factors, which may lead to obstacles and disruptions in
the execution of Preview Days (Appendix A). Since the program has seen an increase in the
number of candidates from outside the Chicago area, the cost of attendance has been a factor to
which Jack McLean and J. Curtis Main are sensitive (personal communication, September 25,
2014). Many candidates travel via different forms of transportation, which may be cost
prohibitive or highly susceptible to changes in weather. Since Preview Days normally takes
place in late February, candidate travel may also be impacted by the cold and unpredictable
weather in Chicago.
The extent that the Preview Days program impacts a candidate may also be affected by
their previous beliefs and assumptions about Loyolas program. For example, if a candidate
labels Loyola as more desirable than another institution (assuming their process includes
multiple institutions), they may respond more favorably to the experiences during Preview Days.
In contrast, candidates who may be on the fence may look at the Preview Days experience
more critically in order to assess their interest in the Higher Education program and GA
15
16
a formal requirement to hold a GA at Loyola in the graduate program, the Preview Days
experience is an important piece in a students decision to attend Loyola. This is not the only
factor, but must be taken into consideration as we begin to form and create an evaluation plan for
the program (Jack McLean, JD, personal communication, September 25, 2014).
Finally, the personnel in the Division of Student Development who use the process to
recruit GA candidates would also be impacted by an evaluation of the program. Much like the
Higher Education program faculty, the Preview Days experience is an important step in selecting
the most desirable candidates for the open Graduate Assistant (GA) positions. As we mentioned
earlier, the absence of this streamlined and cohesive process caused confusion for GA candidates,
which may have persuaded them to consider other institutions for their graduate studies. If
candidate needs are not taken into account or met, hiring supervisors may not have a reason to
recruit their GAs through this process (or hire a graduate student for the position at all). As we
begin to understand the design of our evaluation, it will be important to consider these
stakeholders throughout the entire process.
Evaluation Approach
Our evaluation will be formative in nature as we seek to provide information that could
be used to make improvements. A formative evaluation is defined as one whose primary
purpose is to supply information for the improvement of a program (Fitzpatrick, Sanders,
Worthen, p. 20, 2010). Based on our conversations with stakeholders, we will be using a
combination of process and outcomes approach to the evaluation of Preview Days. The reason a
process approach is helpful when evaluating Preview Days is due to the purpose of a process
study. Such studies may focus on whether the program is being delivered according to some
17
delineated plan or model or may be more open-ended, simply describing the nature of delivery
and the success and problems encountered (Fitzpatrick et al., 2010, p. 26).
Since we are looking into the effectiveness of Preview Days we must look at the various
panels, workshops, and events that occur throughout the program. Given that a process study is
based on the delivery of a program and also takes into account the success and difficulties of the
program, this approach would be advantageous to the needs of the Preview Days program. An
outcome approach is defined as one that takes into account the changes in an audience as a result
of a program (Fitzpatrick et al., 2010). As we have previously mentioned there are several
learning outcomes that aid in describing the purpose of Preview Days. Since the nature of our
evaluation is focused on the delivery and execution of Preview Days we use the learning
outcome that McLean expressed to help guide our evaluation. The main focus of Preview Days
according to McLean is to paint a picture for participants of the culture here at Loyola and within
the DSD. By providing participants with this, Preview Days hopes to help participants discern
whether or not Loyola is a good fit for their graduate studies. In evaluating Preview Days and
its execution, it is vital that we keep this learning outcome in the forefront as we progress
through the evaluation.
The combination of process and outcomes approaches is appropriate as we seek to
address the effectiveness of the Preview Days program and the value it offers to the Loyola GA
experience. Since the process aspect looks at the delivery of the program, we will be looking at
the influence of individual components in relation to the learning outcomes to expose
participants to Loyola and DSD in order to assess whether or not Loyola will be a good fit for
a candidates graduate studies. As the outcome approach examines changes in an audience of a
program, we will be surveying those individuals who attended Preview Days. This also
18
addresses the learning outcome of exposing participants to Loyola and DSD. The strength of this
dual approach is that the process aspect lends itself to looking at both the individual components
and their execution in relation to Preview Days as a whole. The outcomes aspect of this
approach illustrates the effectiveness of Preview Days and allows to us to see the affect such a
program has on its participants. This way if any changes need to be made we can find out what
exactly needs to be change and how to make the necessary adjustments in order to ensure a better
delivery.
Although both process and outcome approaches provide strengths to this evaluation, there
are also weakness to each aspect. Even though the process approach lends itself for a closer
examination of individual components of the Preview Days program, it does not take into
consideration outside influences and variables that potentially affect each piece. The challenges
of an outcome approach is the willingness participants to speak to their experiences and the
potential for varying answers from year to year that would leave have an effect on any changes
that would need to be made. Taking this into consideration, we will explain the various
components and approaches to our evaluation plan.
Quantitative Approach
Population and Sampling
Our intended population of study will be the candidates in attendance during the Preview
Days (PD) program. We selected this group because the focus of this evaluation entails
observing a change within a selected population as a result of attending a program. By surveying
the candidates, we can begin to evaluate the delivery and effectiveness of Preview Days. Since
Preview Days is designed and executed with the candidates needs in mind, it is also important to
note that they are one of the primary stakeholders. Their input and feedback will provide insight
19
into any changes that need to be made and other areas of improvement. Since Loyola aims to
become more competitive in the search and hiring process of Graduate Assistant (GA)
candidates, the information gained by this survey will prove beneficial.
As a result of surveying the candidates who attend Preview Days, we will conduct census
sampling. On any given year, there are about 50-75 candidates who attend Preview Days. The
reason we have selected census sampling is that we are choosing all the individuals who attend
PD. In this case, since those attending PD will have their email accounts already on file, it will
be easy to reach out to them and send out invitations. Another reason that census sampling is
best serves our evaluation is because the design and implementation of our evaluation focuses on
the experience of the individual participant throughout PD. One of the strengths of using census
sampling is that it allows for researchers to select participants who are able to speak to their
experience, in this case attending PD, and will also be able to answer the questions posed in the
survey surrounding their experience. We define our population as people who attend PD in order
to look at specific outcomes that arise as a result of their attendance. When using census
sampling, one weakness can be the limited ability to make generalizations and conclusions due
to the small scope of the population sample. Additionally, another limitation to our survey is the
assumption that all the participants that are in the intended survey sample will answer the survey
in a timely manner presents a weakness and potential challenge to our evaluation. Our
anticipated response rate is at least 50 percent, which reflects our realistic goal based on previous
response on the assessment for Graduate Assistant professional development sessions. Even
though this number is less than the optimal achievement 70 percent identified by Wholey et al.
(2010), it will still yield results that will allow us to make conclusions about the Preview Days
program.
20
21
including: strongly agree, agree, neither agree or disagree, disagree, and strongly disagree. One
of the strengths of the implementation of a Likert scale is that it allows us to assess general
trends and distribution of the data collected. It also paints a general picture of the level of
agreement respondents had towards the statements we made. Our challenge in using a Likert
scale is that the difference between each choice in the response scale is dependent on the readers
understanding of the scale itself and statement made. Since we are assessing the experience of
PD, we are concerned with the distribution of the responses rather than the average response
given that a numerical value would not have much significance. Another concern of this
approach may be that participants may pick the extremes or the middle of the response scale in
interest of time. We hope to mitigate this concern with the use of a large response rate and
evaluation of the descriptive statistics the survey produces.
We are also utilizing open-ended questions to provide a unique space for participants to voice
their opinion and clarify their responses. Unlike the Likert scale, participants are able to
articulate their response in a more complex and unique way that allows researchers better
understand their experience. Our open-ended questions ask respondents to identify any lingering
questions, experiences that were valuable or could have been improved, and a space for
comments and suggestions. A limitation to using these types of questions is the complexity and
level of detail that different participants may employ. Variety, detail, and the unpredictability of
responses can also be a limitation as it may be a burden on researchers to analyze quickly.
Additionally, to many open-ended questions can lead lower responses rates and incomplete
answers as survey fatigue may set in (Schuh & Associates, 2009). The responses to these openended questions will be compiled and given to stakeholders in order to make more informed
decisions surrounding any changes or improvements to the program.
22
23
linked to the participant. Two limitations to distributing the survey through email is the potential
for an incorrect email being used and user error in accessing the survey.
Analysis and Results Presentation
All the data we gather from the survey be organized and describe. The only statistical
analyses we will be using will be descriptive univariate statistics. These kinds of statistics are
those intended to summarize information on a single variable (Wholey et al., 2010, p.
455). As previously noted, many of the questions are tied to an outcome listed in our logic
model (Appendix F) and will provide interval data. For our demographic and general
information questions we will be analyzing the data using frequency of responses. For our Likert
questions we will be analyzing the data using means and medians to help determine the
frequency and distribution of responses. Along with using means and medians to help describe
the data, we will be using ANOVA and r-square statistics in order to measure the interaction
among variables and the significance of variables to their full experience. In addition to these
statistics we will also run a Chi-square test to determine if there is a statistically significant
difference between the Preview Days experience and their demographic characteristics. We
anticipate using SPSS as our statistical software to translate the data into these measures (which
is available on University computers).
We will be presenting these results in our final report using tables and figures that will
contain both the questions and the corresponding scores. There will also be an executive
summary to help the reader grab a big picture of the results and make it easier for them to
comprehend the results. In the executive summary there will be information containing our
purpose, several points that the reader should take notice of, and our own recommendations as
24
part of the evaluation. Our recommendation will be a result of our own interpretation of the data
collect and organized.
Qualitative Approach
In combination with the survey conducted at the conclusion of the Preview Days program,
we will conduct interviews with participants in order to better understand the achievement of our
outcomes (Appendix A). Utilizing a qualitative approach will allow us to better understand the
experience of candidates and explain phenomena that are presented during the administration of
our survey. Recognizing that our quantitative approach attempts to describe the experience of
Preview Days participants as a whole, our qualitative measure will help identify the nuances of
each students experience with the program. Our qualitative approach will use semi-structured
interviews in order to engage participants in a reflection about their experience and identify their
interaction with specific aspects of the Preview Days experience. Semi-structured interviews
combine directed interview questions with supplemental questions (known as probes) that ask
the participant to further explain their experience (Wholey et al., 2010). This approach allows
the interviewer to engage with the participant and clarify specific experiences that may be
essential to understanding an individuals experience with the Preview Days program.
We identified a semi-structured interview approach for the Preview Days program
because of the ability to conduct interviews easily while being able to have the freedom to ask
follow-up and probing questions to better understand their experience. In contrast to focus
groups, semi-structured interviews are conducted with one participant at a time (Wholey et al.,
2010). Even though focus group interviews allow participants to interact with each other and
speak about shared and individual experiences, semi-structured interviews will allow us to
quickly identify and interview participants without creating a focus group with participants
25
across the country. Using this one-on-one interview format, we will be able to easily schedule
interviews with participants and control for any technical difficulties that may arise from
conducting a focus group via telephone or videoconference.
Participants
The participants for the interviews will be selected from those who complete the survey
administered after the conclusion of Preview Days. While participants are completing the survey,
they will be asked if they would be willing to participate in an interview regarding their
experience. If they select yes, they are asked to provide an email address that they can be
contacted in order to schedule the interview. After the survey closes on March 11th, we will
begin to contact participants in order to schedule interviews for mid-April to mid-May. This
timeframe will allow us to analyze the data and edit our interview protocol, as appropriate. We
anticipate interviewing five Preview Days candidates and will compensate interview participants
with a $10 Amazon.com gift card.
Given that our interviews will last approximately 30 minutes, our sample size was
selected in order to conduct the interviews and analyze the data in order to make changes and
improvements to the next Preview Days program. The size of our sample for our qualitative
approach is a limitation to our assessment. Since we are not interviewing candidates that exhibit
a specific characteristic and relying solely on self-selection, we limit the generalizability of our
findings. However, we hope to use the interviews to not only understand the experience of
candidates during Preview Days, but to bring light to phenomena that may present itself during
the analysis of the quantitative data. Additionally, this evaluation component was chosen in
order to quickly gather information and recommendations and results in order for the Division to
consider any changes to the Preview Days program. If the Division of Student Development
26
wishes to better understand the experience of candidates who exhibit a specific characteristic
(demographic, programmatic, etc.), the qualitative approach would be the place to better
understand those variables. Our goal is to use the interviews in order to enhance the
interpretation of the quantitative data, while providing tangible examples to explain areas of
improvement for the program.
Participants will be selected randomly once the email addresses have been collected from
the survey instrument. Each email will be assigned a random number and will be placed in
ascending order. Once this has been completed, we will begin to contact participants in order of
their assigned number. When participants accept the invitation to be interviewed, we will work
to schedule their interview and send them the consent form (Appendix I), which outlines the
purpose of the interview. Additionally, the consent form will include demographic information
(similar to the information included in the survey instrument), which will be used during data
analysis after the interviews have been conducted. If participants decide to decline the offer for
the interview, we will select the next participant on the randomized list.
Interview Protocol
An interview protocol (Appendix H) will be used to conduct the 30-minute semistructured interviews. The overall structure of the interview will include an introduction, a set of
three foci with probing questions, and a conclusion. Prior to the interviews, we will conduct a
pilot test of our questions with current Graduate Assistants. Much like the quantitative approach,
pilot testing will allow us to make sure that the questions are constructed without any confusion
and will draw out directed responses. During the introduction of the interview, the interviewer
will explain the rationale of the interview and ensure that the participant has electronically signed
and returned their interview consent form (Appendix I). The interviewer will also reiterate that
27
the participants answers will be kept confidential and will not be traced back to their name or
candidate file. Since we are conducting these interviews during the timeframe when
assistantship offers are typically made, it is important that we ensure that the participants can be
honest with their responses, without any effect on their assistantship offer status. This timeframe
does present a limitation, but our goal is to ensure that the information provided during
interviews will not be traced back to their candidate file.
After the interviewer has explained the interview and asked for the participants verbal
agreement for continuing the interview and audio recording, the interviewer will ask the
questions outlined in the protocol. Given that we only have 30 minutes to complete the
interview, we have focused our time for the participant to answer three foci (all open-ended) with
additional probes provided for the interviewer. The interviewer will not be obligated to ask all of
the probing questions, but will be provided a list of probing questions in order to ensure that
participants are providing the desired response. Including the probing questions also allows
participants to provide more focused responses that will aid in our ability to analyze and interpret
the data. Our foci are intended to speak to our outcomes outlined in our logic model (Appendix
A), with specific emphasis on the medium and long-term outcomes. After the interviewer has
asked the three focused questions, they will be instructed to thank the participant and offer to
answer any questions about the interview. Additionally, participants will be asked to confirm
their email so that they can receive their compensation.
Implementation and Analysis
A current graduate assistant in the Division of Student Development (DSD) who is
recruited by the Preview Days Planning Committee will facilitate the interviews for this portion
of the assessment plan. Utilizing a current graduate assistant helps ensure the validity of the
28
responses because they are minimally involved in the hiring process for Graduate Assistants and
also able to relate to participants experiences during Preview Days. In partnership with the DSD,
the selected Graduate Assistant will work to schedule the interviews. All interviews will be
conducted via telephone and the interviewer will be asked to take notes regarding any major
themes or ideas that emerge during the interview (Appendix J). All notes will be collected in
order to be analyzed by the researchers, but will not weigh heavily in the final interpretation of
the data.
All interviews will be audio recorded, saved electronically, and transcribed by a graduate
student who will receive compensation (see budget in Appendix O). Once the interviews have
been transcribed, we will then engage in the process of descriptive coding in order to analyze the
data. Descriptive coding allows researchers to create categories as they examine transcripts,
while using initial codes developed in relation to the outcomes of the program (Wholey et al.,
2010). The initial codes we have developed (Appendix L) relate specifically to the learning
outcomes outlined in the logic model (Appendix A) and will be amended if specific phenomena
appears from the quantitative survey that becomes of interest to the DSD. Utilizing descriptive
coding for the interviews is helpful since this is the first time the DSD will take on a qualitative
approach, which involves coding. Descriptive coding allows the researchers to be flexible with
their coding structures and focus more on the specifics described in the interview. This type of
coding will be time intensive for the researchers doing the coding and those interpreting the
results (Wholey et al., 2010), but will help the DSD create a coding structure for future
interviews. We will also compare the candidates information with their choice to accept of
decline an assistantship using back-end coding once we have compiled all of the transcripts and
information.
29
After the transcripts have been coded, we will comb through the data using pawing in
order to observe the themes in the data as they emerge. As we examine the development and
interaction of the codes and the data, we will be able to look for specific themes and re-analyze
the data to identify other areas where those themes may emerge. Once we have examined the
data from the quantitative and qualitative measures, we will ensure validity through a process of
triangulation. As Wholey et al. (2010) explain triangulation helps to limit the bias that results
from relying on one source of information to make conclusions. Because we are asking
questions in both instruments about the outcomes and goals of the Preview Days program, we
can match the results of the evaluation approaches to identify any inconsistencies. If the data
does result in any inconsistencies, we will flag them and find ways to evaluate those phenomena
in future evaluations (Wholey et al., 2010). In addition to triangulation, we will also perform
member checking with the participants of our interviews. Member checking involves taking
initial interpretations findings, and descriptions back to the participantsto determine whether
they are accurate from the participants point of view (Schuh & Associates, 2009, p. 169). This
will ensure that our observations match the experience of the participants. Once the interviews
have been examined and validated, the conclusions will be paired with the quantitative data,
using direct quotes in order to support the conclusions within the final report.
Limitations
This evaluation plan will allow the Division of Student Development and the Preview
Days Planning Committee valuable information in order to make changes to the Preview Days
program to meet the needs of the Division and assistantship candidates. Even though the intent
of this evaluation is to aid in the execution of the Preview Days program, we must acknowledge
that there are a few limitations to our approach. Since we will be conducting the survey using an
30
online platform, our response level (50 percent) will be lower than what is optimal for
quantitative instruments (70 percent; Wholey et al., 2010). This will limit the generalizability of
our findings; however, we believe the 50 percent response rate will still let us make general
conclusion about the experiences that candidates have during Preview Days. We will hope to
increase our response rate through announcements during Preview Days and multiple reminder
emails after the program.
In addition to the response rate on the survey instrument, our small sample size for our
interviews presents further limitations. We only anticipate interviewing five participants, which
is about less than ten percent of the total participants in the Preview Days program. This size
was selected given that the DSD does not have any current qualitative evaluation structure for
the Preview Days program. Additionally, we are looking to gather results and make
recommendations before planning for the next Preview Days program. Selecting a small sample
size will be realistic and will provide quotes and observations that can be added to the
quantitative results.
Budget
As outlined in Appendix O, our total anticipated cost for this evaluation plan is $350.
The DSD already has access to CampusLabs and printing services, which will help deliver the
survey and print any paper materials and final report. The bulk of our budget will be spent on
the surveys in the qualitative plan. Participants will be compensated for their participation in the
interview with a $10 Amazon.com gift card. We anticipate the total cost of gift cards to be $50
and they will be distributed via email. Since our qualitative plan requires transcription and
coding, we will recruit a current graduate student to complete this step of our plan. Our
estimates indicate that it will take approximately 10 hours for the students to transcribe the two
31
and a half hours of audio recording (audio recorder provided by digital media services).
Additionally, we estimate coding to take a similar amount of time. We will pay our graduate
students $15 per hour, which we believe to be an acceptable incentive to complete the
transcription and coding. The Division may be able to explore options for students to perform
this work as part of an internship or summer project, if they wish to be cost conscious about the
evaluation program.
Timeline
Throughout the evaluation plan, a timeline has been outlined in order to provide further
direction for the evaluation of the Preview Days program. Our specific timeline is further
explained in Appendix N. After the completion of our evaluation plan, we will discuss the
findings with Jack McLean and J. Curtis Main to discuss our findings and understand the
feasibility of the study. We anticipate meeting with Jack and Curtis in December in order to
understand how to implement the evaluation plan within the Division. Once we have met with
Jack and Curtis, we will begin pilot testing our survey and interview protocol in order to
understand if any further changes need to be made to our plan. Based on this feedback, we
anticipate completing our survey instrument at the beginning of February, in anticipation of
Preview Days, which takes place later in the month. Once Preview Days is completed,
participants will be emailed the survey (with subsequent reminders) and be instructed to
complete the survey before March 11.
Once the survey is completed, we will email those participants who indicated that they
were open to being interviewed (after random selection is completed). We will schedule the
interviews and hold the interviews during April and May. The data will be transcribed and
coded after the completion of the interviews. Initial conclusions will be made based on the
32
evaluation results and sent to the participants who were interviewed for member checking. Once
we have received information back from the participants, we will compile the information into
the final report and inform the DSD of our findings and suggestions for changes to future
Preview Days programs.
Next Steps
As was mentioned earlier, we will meet with Jack and Curtis to discuss our findings and
understand the feasibility of our evaluation plan. We will send Jack and Curtis our plan
electronically and anticipate scheduling a meeting face-to-face with them to understand their
general reactions to the plan and any considerations we should take moving forward. Our hope
would be to assist the Division with this evaluation (in any capacity) and extend our help for the
planning and implementation of Preview Days 2015.
If the Division wishes to continue with this evaluation plan, it should consider finding
ways to perform an evaluation of students who are offered interviews but who choose not to
attend Preview Days. This information will allow the Division to assess any barriers to
attendance and better tailor the program to be available to a wider audience. Since the scope of
this evaluation plan is the execution of the existing program, we did not address this area in our
plan. Finally, the Division should also work with Preview Days stakeholders in order to better
target the quantitative and qualitative questions and understand how each office perceives the
Preview Days experience.
33
References
Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods
approaches. Thousand Oaks, California: SAGE Publications.
Division of Student Development (n.d.). Graduate assistantships. Retrieved from:
http://www.luc.edu/studentdevelopment/assistantships/
Fitzpatrick, J. L., Sanders, J. R., Worthen, B. R., & Worthen, B. R. (2011). Program evaluation:
Alternative approaches and practical guidelines. Upper Saddle River, NJ: Pearson
Education.
Schuh, J. H. & Associates (2009). Assessment methods for student affairs. San Francisco, CA:
Jossey-Bass.
Weiss, C. (1998). Evaluation: Methods for studying programs and policies (2nd ed.). Upper
Saddle River, N.J.: Prentice Hall.
Wholey, J. S., Hatry, H. P., & Newcomer, K. E. (Eds.) (2010). Handbook of practical program
evaluation (Third Edition). San Francisco: Jossey-Bass.
Buddy System
Prospective GAs
Interviews
Current GAs
Programs:
-Socials
-Campus tours
-Jesu-What?
-Social justice
introduction
-Divisional overview
-Classroom
Experience
-Workshops and
panels
Divisional staff
Time
Resources:
-Housing
-Food
-Preview Days guide
-Facilities
-Food
Personnel:
-Current Graduate
Assistants (GAs)
-Committees
-Divisional staff
-Hiring supervisors
-Higher Education
program faculty
Higher Education
program faculty
Hiring supervisors
Assumptions:
1. Students will attend Preview Days and attend all applicable
workshops during Preiview Days
2. Prospective GAs will have little to no prior knowledge of Loyolas
GA experience and the city of Chicago
3. The programs chosen and presented will meet students needs and
help students discern institutional fit.
4. The Preview Days programs will be able to keep students interest
throughout the experience
5. University and Divisional partners are willing to take part in the
Preview Days experiencea
Short
Understand and
articulate the Division
of Student
Developments GA
expectations and
experience.
Gain a better
understanding of the
assistantship offerings
and experiences at
Loyola University
Chicago.
Be able to select most
desirable assistantship
experience (specific
department).
34
Outcomes -- Impact
Medium
Accept an offer for a
GA position at Loyola.
Discerning whether
Loyola is a good fit for
the candidate.
Long
Their Loyola GA
experience matches
their expectations as
a result of attending
Preview Days.
Sharing their
experience with their
network of peers and
professionals.
Deeper level of
investment and a
stronger sense of
community among
GAs.
External Factors:
1.
2.
3.
4.
5.
6.
7.
8.
9.
35
36
37
38
Logistical Information
What is your current educational status?
Current graduate student at Loyola
12
Train
12
Disagree
Strongly agree
Strongly disagree
Strongly agree
Strongly agree
12
11
Disagree
Strongly agree
10
Strongly disagree
Comments
The timeliness of receiving the information about the two days could improve. The interview format was
confusing and not explained, and the communication about the interviews was very poor. Many of us did
not know if we were only interviewing with one or all of the assistantships we applied for, in addition to
39
who the main contact person was for the interviews and explaining that process.
I feel that the process of receiving responses was very confusing. We were given contradictory information
as to when and how we would hear back from GA providers. This could be more clear in the future
Lodging
Lodging information was provided in a timely manner.
Agree
Disagree
Strongly agree
Strongly disagree
22
Yes
11
Yes
Comments
I was told not to count on student housing accommodations, and I did not receive any follow up once I
made an inquiry.
I wish I would have heard further in advance because I made other arrangements.
It would have been nice to have had internet access in Baumhart Hall.
11
Yes
13
Disagree
Strongly agree
Disagree
Strongly agree
Strongly agree
Strongly agree
Staff
Faculty
Agree
Disagree
Strongly agree
Agree
Strongly agree
40
41
I felt welcomed at this event.
Agree
Disagree
Strongly agree
Disagree
Strongly agree
Comments:
Tuesday was wonderful! To improve it though, I wish there was a panel of student including some who did
not have assistantships and how they were coping economically
It was a good night, highly informative and scheduled well.
I did not realize the importance of attending the event on Tuesday evening. After attending other Preview
program, I believe meeting the professors and getting to know more a about the program could have
impacted my decision on whether to attend Loyola. I also thought the Interview day would provide me with
this information but sadly that was what the night before was all about.
I would have liked to know more about the class option before I arrived. I did not know about it and would
have planned to attend had I known.
I would have liked to talk to more current students in the program.
I thought it was an excellent way to start prior to the GA interview day following the reception. It provided
some opportunities to get to know a few of the other applicants, which helped reduce some nerves. I would
continue to do a dinner reception in the future.
I felt we should have had more time to speak with current students.
No one really moved around at the social; would it be possible to have dinner in one location and the social
in another? Prospective students just remained in their seats and it was really awkward to try to speak to a
professor or staff member who was sitting at a full table.
Great help and made me feel at ease about the whole process and people in the program
It was a great evening. It really provided me with a hands-on look at the academic program. This truly
enabled me to make a better overall judgment of whether or not the program was right for me.
20
Yes
Comments:
Loved it-loved the way the professor expected us to participate-and we did!
42
11
Ineffective
Very effective
12
12
Ineffective
Very effective
14
Ineffective
Very effective
13
Ineffective
Very effective
10
14
Ineffective
Very effective
Very ineffective
Room layout
Effective
10
Ineffective
Very effective
Very ineffective
Strongly agree
15
Icebreaker
Agree
Disagree
12
Strongly agree
Strongly disagree
43
14
Strongly disagree
44
13
Strongly agree
Strongly disagree
18
Disagree
Strongly agree
Strongly disagree
Not applicable
Strongly agree
Strongly disagree
10
Not applicable
Strongly agree
Strongly disagree
Comments:
A tour around campus, the student affairs offices, etc. A more structured way to meet/visit with everyone
More on the classes, getting to know the prospective students as well as the current students in a more
familiar basis
More structure for what do while waiting for interviews
Campus tour!
After going through this survey, I realized I signed up for interviews prior to receiving the schedule.
Because of this, I missed a session I would have preferred to attend, moving to Chicago. In the future it
may serve the applicants better to see the structure of the day prior to being offered interviews. That way,
applicants will see the structure/layout of the day and may be able to better accommodate their needs.
What to do if you do not have an assistantship
For someone who is already a Loyola graduate student, none of the day's events aside from the interview
were helpful; however, I can see their relevance to those who are not already enrolled.
45
More information about the program - I am still trying to make sure I find a good fit school and program
wise.
The dates of the reception and interview days were very inconvenient. I missed two days of class in order
to travel and attend the events. Holding these events later in the week, such as a Thursday or Friday, would
be much more convenient than a Tuesday and Wednesday. I was not able to attend the panel of current
students.
More interaction with current students and faculty.
The conference room for us to sit in between interviews was too small and it was crowded.
Great day.
The presentation on What is Loyola and the Jesuit Mission? was quite overwhelming. It was too pushy
and religion focused. I really love and appreciate Loyola's mission, it's truly inspiring, but for non-Christian
and especially atheists and agnostics the presentation was too strong. I believe it scared my non-Christian
peer.
At a glance
Logistical information should be provided earlier for students to make flight arrangements and properly
schedule interviews.
We should avoid scheduling sessions (i.e. Off-Campus living, grad panel) during interview times or hold them
multiple times throughout the day. Students may have scheduled their interview not knowing about the session
and then had to miss the session to interview.
We need to do a better job scheduling interviews for folks that have multiple interviews so they can be closer
together. Maybe we can centralize this?
Providing a program for first-year Loyola students could be beneficial but does not appear to be required.
Would be helpful to better communicate expectations to first-year students.
Offering lodging with current students would be helpful but not required. Finding graduate students to host
students is very difficult.
The Tuesday evening reception was very well received! Only improvement would be to have people mingle
more rather than staying seated. If current student, faculty, or staff were at a full table, it was difficult for
prospective students to speak with that person.
The option to attend the Social Justice class was also very well received! Art did a fantastic job integrating the
students into discussion and making them feel welcome.
Interview day: Overall well received. Improvement could be seen with the downtime students had. Offer
campus tours, have graduate students on staff to speak with perspectives.
Could we hold the interviews in CFSU either in Bremner or in individual offices? Also, use the Hague as a
gathering area for folks between interviews?
Can we offer campus tours or coffee with current grads during the downtime?
46
Allowed answers: 0
Q1 Is Loyola the only Interview/Preview Day program for which you will participate?
Yes[Code = 1]
No (What other institutions have you/will you visit?)[Code = 2] [Textbox]
Required answers: 1
Allowed answers: 1
Q2 Which Preview Days session(s) did you attend on Sunday, February 24, 2013? (Check all that apply)
Water Tower Campus Tour[Code = 1]
Higher Education Orientation Dinner[Code = 2]
Current Graduate Student Panel[Code = 3]
Optional activities (please specify)[Code = 4] [Textbox]
None of the above[Code = 0] [N/A]
Required answers: 1
Allowed answers: 5
Q3 Which Preview Days session(s) did you attend on Monday, February 25, 2013? (Check all that apply)
Breakfast[Code = 1]
Welcome, Introductions, Professional Development Opportunities[Code = 2]
Overview of the Day[Code = 3]
Jesuit Education 101: Jesu-What?[Code = 4]
Moving to Chicago[Code = 5]
Lake Shore Campus Tour(s)[Code = 6]
Informal Social[Code = 7]
Lunch[Code = 8]
Social Justice Literacy[Code = 9]
Grad Research: Scholar and Practitioner[Code = 10]
Staff Panel[Code = 11]
Lake Shore Campus Tour(s) #2[Code = 12]
Optional Class: Student Development Theory[Code = 13]
None of the above[Code = 0] [N/A]
Required answers: 1
Allowed answers: 14
Q4 Did you take advantage of the on-campus housing accommodations offered during Preview Days?
Yes[Code = 1]
No[Code = 2]
Required answers: 1
Allowed answers: 1
Next Page: Sequential
47
Page - 2
How would you rate the housing accommodations on the following components?
Q5 Information received regarding housing accommodations prior to your arrival
Excellent[Code = 5]
Good[Code = 4]
Satisfactory[Code = 3]
Poor[Code = 2]
Unacceptable[Code = 1]
Did not attend/Does not apply[Code = 0] [N/A]
Required answers: 1
Allowed answers: 1
Required answers: 1
Allowed answers: 1
Required answers: 1
Allowed answers: 1
Required answers: 1
Allowed answers: 1
Required answers: 1
Allowed answers: 1
Q6 Check-in process
Excellent[Code = 5]
Good[Code = 4]
Satisfactory[Code = 3]
Poor[Code = 2]
Unacceptable[Code = 1]
Did not attend/Does not apply[Code = 0] [N/A]
Q7 Room provided
Excellent[Code = 5]
Good[Code = 4]
Satisfactory[Code = 3]
Poor[Code = 2]
Unacceptable[Code = 1]
Did not attend/Does not apply[Code = 0] [N/A]
Q8 Check-out process
Excellent[Code = 5]
Good[Code = 4]
Satisfactory[Code = 3]
Poor[Code = 2]
Unacceptable[Code = 1]
Did not attend/Does not apply[Code = 0] [N/A]
Q9 Parking
Excellent[Code = 5]
Good[Code = 4]
Satisfactory[Code = 3]
Poor[Code = 2]
Unacceptable[Code = 1]
Did not attend/Does not apply[Code = 0] [N/A]
48
Display if Q4='Yes'
Q10 Why did you not take advantage of the on-campus housing accommodations offered during Preview Days?
[Code = 1] [Textbox]
Required answers: 0
Allowed answers: 1
Display if Q4='No'
Please rate the following aspects of your Preview Days experience:
Q11 Communication with DSDGrads
Excellent[Code = 5]
Good[Code = 4]
Satisfactory[Code = 3]
Poor[Code = 2]
Unacceptable[Code = 1]
Did not attend/Does not apply[Code = 0] [N/A]
Required answers: 1
Allowed answers: 1
Required answers: 1
Allowed answers: 1
Allowed answers: 1
Required answers: 1
Allowed answers: 1
49
Satisfactory[Code = 3]
Poor[Code = 2]
Unacceptable[Code = 1]
Did not attend/Does not apply[Code = 0] [N/A]
Required answers: 1
Allowed answers: 1
Required answers: 1
Allowed answers: 1
Required answers: 1
Allowed answers: 1
Required answers: 1
Allowed answers: 1
Allowed answers: 1
50
Allowed answers: 1
Required answers: 1
Allowed answers: 1
Required answers: 1
Allowed answers: 1
Allowed answers: 1
51
Moderately valuable[Code = 3]
Not very valuable[Code = 2]
Not valuable at all[Code = 1]
Did not attend/Does not apply[Code = 0] [N/A]
Required answers: 1
Allowed answers: 1
Required answers: 1
Allowed answers: 1
Required answers: 1
Allowed answers: 1
Required answers: 1
Allowed answers: 1
Required answers: 1
Allowed answers: 1
52
Allowed answers: 1
Required answers: 1
Allowed answers: 1
Required answers: 1
Allowed answers: 1
Required answers: 1
Allowed answers: 1
Display if Q3='Lunch'
Q30 Social Justice Literacy
Extremely valuable[Code = 5]
Very valuable[Code = 4]
Moderately valuable[Code = 3]
Not very valuable[Code = 2]
Not valuable at all[Code = 1]
Did not attend/Does not apply[Code = 0] [N/A]
Display if Q3='Social Justice Literacy'
Q31 Grad Research: Scholar and Practitioner
Extremely valuable[Code = 5]
Very valuable[Code = 4]
Moderately valuable[Code = 3]
Not very valuable[Code = 2]
Not valuable at all[Code = 1]
Did not attend/Does not apply[Code = 0] [N/A]
Display if Q3='Grad Research: Scholar and Practitioner'
Q32 Staff Panel
Extremely valuable[Code = 5]
Very valuable[Code = 4]
Moderately valuable[Code = 3]
Not very valuable[Code = 2]
Not valuable at all[Code = 1]
Did not attend/Does not apply[Code = 0] [N/A]
Display if Q3='Staff Panel'
Q33 Lake Shore Campus Tour #2
Extremely valuable[Code = 5]
Very valuable[Code = 4]
Moderately valuable[Code = 3]
Not very valuable[Code = 2]
Not valuable at all[Code = 1]
Did not attend/Does not apply[Code = 0] [N/A]
53
Required answers: 1
Allowed answers: 1
Required answers: 1
Allowed answers: 1
Sunday
Please indicate your level of agreement with the following statements:
Q35 The Water Tower Campus tour on Sunday allowed me to see the amenities of the downtown campus,
including places to dine, classrooms, and event spaces.
Strongly agree[Code = 5]
Somewhat agree[Code = 4]
Neither agree nor disagree[Code = 3]
Somewhat disagree[Code = 2]
Strongly disagree[Code = 1]
Not applicable[Code = 0] [N/A]
Required answers: 1
Allowed answers: 1
Allowed answers: 1
Allowed answers: 1
54
Monday
Please indicate your level of agreement with the following statements:
Q38 As a result of attending the Jesuit Education 101: Jesu-WHAT?, my knowledge of the Jesuit transformative
education increased.
Strongly agree[Code = 5]
Somewhat agree[Code = 4]
Neither agree nor disagree[Code = 3]
Somewhat disagree[Code = 2]
Strongly disagree[Code = 1]
Not applicable[Code = 0] [N/A]
Required answers: 1
Allowed answers: 1
Allowed answers: 1
Allowed answers: 1
Allowed answers: 1
55
Q42 The informal social allowed me an opportunity to learn more about the Loyola community through interacting
with current students and staff.
Strongly agree[Code = 5]
Somewhat agree[Code = 4]
Neither agree nor disagree[Code = 3]
Somewhat disagree[Code = 2]
Strongly disagree[Code = 1]
Not applicable[Code = 0] [N/A]
Required answers: 1
Allowed answers: 1
Allowed answers: 1
Display if Q3='Lunch'
Q44 As a result of the Social Justice Literacy session, I learned about social justice, why it is important in Higher
Education, and why it is a hallmark of Loyola education.
Strongly agree[Code = 5]
Somewhat agree[Code = 4]
Neither agree nor disagree[Code = 3]
Somewhat disagree[Code = 2]
Strongly disagree[Code = 1]
Not applicable[Code = 0] [N/A]
Required answers: 1
Allowed answers: 1
Allowed answers: 1
56
Allowed answers: 1
Allowed answers: 1
Allowed answers: 1
Q49 The Preview Days program answered most of my questions about whether Loyola University Chicago and the
Division of Student Development would be a good fit for me.
Strongly agree[Code = 5]
Somewhat agree[Code = 4]
Neither agree nor disagree[Code = 3]
Somewhat disagree[Code = 2]
Strongly disagree[Code = 1]
Not applicable[Code = 0] [N/A]
Required answers: 1
Allowed answers: 1
Allowed answers: 1
Next Page: Sequential
57
Allowed answers: 1
Required answers: 1
Allowed answers: 1
Required answers: 0
Allowed answers: 1
Required answers: 0
Allowed answers: 1
Required answers: 0
Allowed answers: 1
Q53 What did you enjoy most about your Preview Days experience?
[Code = 1] [Textbox]
Q54 What was most valuable about your Preview Days experience?
[Code = 1] [Textbox]
Q56 What is something you learned during Preview Days that you would not have learned otherwise (if applicable,
please refer to a specific session or person)?
[Code = 1] [Textbox]
Required answers: 0
Allowed answers: 1
Q57 Please use the space below to make additional comments and suggestions:
[Code = 1] [Textbox]
Required answers: 0
Allowed answers: 1
Next Page: Sequential
ThankyoufortakingtimetoevaluateyourPreviewDaysexperienceatLoyolaUniversityChicago. Thepurposeofthissurveyistoevaluatethe
variouscomponentsofthePreviewDaysprograminordertoimprovetheexperienceforfuturestudents.
Wewelcomeanyfeedbackabouttheprogramandwillensurethatallresponseswillbekeptconfidentialandwillnotimpactanyofferdecisions
madebytheDivisionorindividualdepartments. ResponseswillonlybesharedwithmembersofthePreviewDaysPlanningCommittee. Ifyou
haveanyquestions,pleasecontacttheDivisionofStudentDevelopmentviaemailatDSDgrads@luc.edu.
j InstituteofPastoralStudies
k
l
m
n
j Counseling
k
l
m
n
j Other(pleasespecify)
k
l
m
n
*
If 2. How did you hear about Loyola? (Please check all that apply)
c ProfessionalsinHigherEducation
d
e
f
g
c ProgramAlumni
d
e
f
g
c NationalAssociations(ACPA,NASPA,etc.)
d
e
f
g
c OnlineResources
d
e
f
g
c SocialMedia(Facebook,Twitter,LinkedIn,ect.)
d
e
f
g
c Family/Relatives
d
e
f
g
c Other(pleasespecify)
d
e
f
g
*
If 3. Is Loyola the only interview/Preview Days program for which you will participate?
j Yes
k
l
m
n
j No
k
l
m
n
58
59
*4. Please evaluate the following aspects of your Preview Days experience:
StronglyDisagree
Disagree
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
Theinformationrecieved
NeitherAgreeor
Agree
Disagree
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
j
k
l
m
n
Disagree
priortoPreviewDayswas
clearandeasyto
understand.
TheDivisionofStudent
DevelopmentGAwebsite
answeredmyquestions
regardingtheGA offerings.
TheGAapplicationprocess
waseasytocomplete.
Myinterviewexperince
answeredmyquestions
abouttheLoyolaGA
experience.
Pleaserateyoulevelofagreementwiththefollowingstatements:
5. Overall, I have clear expectations about what my Loyola University Chicago Graduate
*
Yes (please
Assistantshipexperience will entail.
j StronglyAgree
k
l
m
n
j Agree
k
l
m
n
j NeitherAgreeorDisagree
k
l
m
n
j Disagree
k
l
m
n
j StronglyDisagree
k
l
m
n
*6. As a result of attending Preview Days, I am able to articulate the mission and vision of
j NeitherAgreeorDisagree
k
l
m
n
j Disagree
k
l
m
n
j StronglyDisagree
k
l
m
n
j NeitherAgreeorDisagree
k
l
m
n
j Disagree
k
l
m
n
j StronglyDisagree
k
l
m
n
*8. The Graduate Assistantship positions available at Loyola will provide me with the type
j NeitherAgreeorDisagree
k
l
m
n
j Disagree
k
l
m
n
j StronglyDisagree
k
l
m
n
*9. As a result of attending Preview Days, I am more informed about making a decision
j NeitherAgreeorDisagree
k
l
m
n
j Disagree
k
l
m
n
j StronglyDisagree
k
l
m
n
j Prefernottorespond
k
l
m
n
60
*10. During Preview Days, I was able to connect with other candidates from different
backgrounds (i.e. gender, race, sexual orientation, etc.) as my own.
j StronglyAgree
k
l
m
n
j Agree
k
l
m
n
j NeitherAgreeorDisagree
k
l
m
n
j Disagree
k
l
m
n
j StronglyDisagree
k
l
m
n
*11. During Preview Days, I was able to connect with other candidates fromsimilar
j NeitherAgreeorDisagree
k
l
m
n
j Disagree
k
l
m
n
j StronglyDisagree
k
l
m
n
j NeitherAgreeorDisagree
k
l
m
n
j Disagree
k
l
m
n
j StronglyDisagree
k
l
m
n
j NeitherAgreeorDisagree
k
l
m
n
j Disagree
k
l
m
n
j StronglyDisagree
k
l
m
n
61
*14. During Preview Days, I was able to connect with Graduate Assistantsfrom different
backgrounds (i.e. gender, race, sexual orientation, etc.)as my own.
j StronglyAgree
k
l
m
n
j Agree
k
l
m
n
j NeitherAgreeorDisagree
k
l
m
n
j Disagree
k
l
m
n
j StronglyDisagree
k
l
m
n
*15. During Preview Days, I was able to connect withGraduate Assistantsfrom similar
backgrounds (i.e. gender, race, sexual orientation, etc.)as my own.
j StronglyAgree
k
l
m
n
j Agree
k
l
m
n
j NeitherAgreeorDisagree
k
l
m
n
j Disagree
k
l
m
n
j StronglyDisagree
k
l
m
n
j NeitherAgreeorDisagree
k
l
m
n
j Disagree
k
l
m
n
j StronglyDisagree
k
l
m
n
j NeitherAgreeorDisagree
k
l
m
n
j Disagree
k
l
m
n
j StronglyDisagree
k
l
m
n
62
*18. As a result of the Preview Days program, I have a better understanding of whether
Loyola is a good fit for me academically and professionally.
j StronglyAgree
k
l
m
n
j Agree
k
l
m
n
j NeitherAgreeorDisagree
k
l
m
n
j Disagree
k
l
m
n
j StronglyDisagree
k
l
m
n
*19. The Preview Days program helped me answer my questions about whether the
Loyola GA experience would be a good fit for me.
j StronglyAgree
k
l
m
n
j Agree
k
l
m
n
j NeitherAgreeorDisagree
k
l
m
n
j Disagree
k
l
m
n
j StronglyDisagree
k
l
m
n
j NeitherAgreeorDisagree
k
l
m
n
j Disagree
k
l
m
n
j StronglyDisagree
k
l
m
n
*21. What aspect(s) of the Preview Days experiencewould you consider the most
valuable?
5
6
63
64
*23. As a result of attending Preview Days, I would recommend the expereince to other
interested students?
j Yes
k
l
m
n
j No (pleasespecify)
k
l
m
n
24. Do you have any lingering questions? (please list them below)
5
6
25. Please utilize the space below for any additional comments or suggestions.
5
6
Pleaseanswerthefollowingquestionsasyoufeelcomfortable.
j Trans*
k
l
m
n
j Prefernottodisclose
k
l
m
n
j Ifnotlistedabove,pleasespecify
k
l
m
n
j Hispanic/Latino/Latina
k
l
m
n
j BlackorAfricanAmerican
k
l
m
n
j NativeAmericanorAmericanIndian
k
l
m
n
j Asian/PacificIslander
k
l
m
n
j Prefernottodisclose
k
l
m
n
j Ifnotlistedabove,pleasespecify
k
l
m
n
j Prefernottodisclose
k
l
m
n
65
66
Inadditiontothissurvey,wearelookingforagroupofstudentswhoattendedthePreviewDaysprogramtoparticipateinainterviewabouttheir
experience. Theinterviewwilltakeplaceafterassistantshipoffershavebeenacceptedandwilllastapproximately30minutesviaphone. Intentto
particpateintheinterviewdoesnoteffectyoureligibilitytorecieveanassistantshipofferfromtheDivisionorindividualdepartments.
Ifselectedforaninterview,youwillbecontactedonoraroundApril15,2015andwillrecievea$10Amazon.comgiftcardforyourtime.
*29. Would you be willing to participate in a interview about your Preview Days
experience?
j No
k
l
m
n
j Yes (pleaseinsertyouremailaddressbelow):
k
l
m
n
67
General
Information
(GI)
Outcome
Q#
GI 1.1
Q#
Question
Type of Measure
GI 1.2
GI 1.3
GI 1.4
25
GI 1.5
26
Answer Options
Radio Button:
Higher Education
Nominal (categorical) Institute of Pastoral Students
Counseling
Other (please specify):
Radio Button:
Professionals in Higher Education
Nominal (categorical) Program Alumni
Multiple responses
National Associations (ACPA,
allowed
NASPA, etc.)
Online Resources
Social Media (Facebook, Twiter,
LinkedIn, etc.)
Family/Relatives
Other (please specify)
Radio Button:
Yes
Nominal (categorical) No
Female
Male
Trans*
Prefer not to disclose
If not listed above, please specify:
ST OC 1:
Understand
and articulate
the Division of
Student
Developments
GA
expectations
and experience.
GI 1.6
27
GI 1.7
28
GI 1.6
29
STOC
1.1
STOC
1.2
As a result of attending
Preview Days, I am able to
articulate the mission and
vision of Loyola University
Chicago.
Likert Scale
STOC
1.3
21
Open-ended
68
Radio Button:
Nominal (categorical)
White
Hispanic/Latino/Latina
Black or African American
Native American or American Indian
Asian/Pacific Islander
Prefer not to disclose
If not listed above, please specify:
Radio Button:
20-24
Nominal (categorical) 25-29
30-34
35-39
40+
Prefer not to disclose
Radio Button:
No
Nominal (categorical) Yes (please insert your email address
below):
Likert Scale
ST OC 2:
Gain a better
understanding
of the
assistantship
offerings and
experiences at
Loyola
University
Chicago
ST OC 3:
Be able to
select the most
desirable
assistantship
experience
(specific
department)
MT OC 1:
Accept an offer
for a GA
position at
Loyola
69
STOC
2.1
Matrix/Likert Scale:
Strongly Agree (5)
Agree (4)
Neither Disagree
Nor Agree (3)
Disagree (2)
Strongly Disagree (1)
STOC
2.2
Radio Button:
Likert Scale
STOC
3.1
Radio Button:
Likert Scale
MTOC
1.1
As a result of attending
Preview Days, I am more
informed about making a
decision surrounding an
assistantship offer.
Radio Button:
Likert Scale
MT OC 2:
Discern
whether
Loyola is a
good fit for the
candidate
MTOC
2.1
10
MTOC
2.2
11
MTOC
2.3
12
MTOC
2.4
13
MTOC
2.5
14
70
Radio Button:
Likert Scale
Radio Button:
Likert Scale
Radio Button:
Likert Scale
Radio Button:
Likert Scale
Radio Button:
Likert Scale
LT OC 1:
The Loyola
GA experience
matches their
expectations as
a result of
attending
Preview Days
MTOC
2.6
15
MTOC
2.7
18
MTOC
2.8
19
MTOC
2.9
LTOC
1.1
24
LTOC
1.2
22
20
71
Radio Button:
Likert Scale
Radio Button:
Likert Scale
Radio Button:
Likert Scale
Open-ended
Open-ended
Radio Button:
Likert Scale
72
LTOC
2.1
23
As a result of attending
Preview Days, I would
recommend the experience to
other interested students?
Radio Button
Yes
No (please specify)
LTOC
3.1
16
Radio Button:
Likert Scale
LTOC
3.2
17
Radio Button:
Likert Scale
73
74
75
76
77
78
________
________
________
_______________________________
________
Participant Signature
Date
By typing in your name on the line above, you authorize this as your signature and accept the
terms listed.
79
YES
NO
Date/Time:
Consent Form:
YES
NO
80
81
Description
Understand and articulate the Division of Student Developments GA
expectations and experience.
Gain a better understanding of the assistantship offerings and experiences at
Loyola University Chicago.
Be able to select most desirable assistantship experience (specific department).
Accept an offer for a GA position at Loyola.
Discerning whether Loyola is a good fit for the candidate.
Their Loyola GA experience matches their expectations as a result of attending
Preview Days.
Sharing their experience with their network of peers and professionals.
Deeper level of investment and a stronger sense of community among GAs.
82
Sunday,(November(30,(2014(at(3:16:16(PM(Central(Standard(Time
Subject:
Date:
From:
To:
CC:
~"a"message"from"the"Vice"President"regarding"GA"compensa6on~
Monday,"October"27,"2014"at"2:07:18"PM"Central"Daylight"Time
VPforStudentDevelopment
VPforStudentDevelopment
Engberg,"Mark
Transparent process
Subsidiarity decisions made at lowest level feasible (department)
Respecting two-year commitment made to current GAs, where possible
Equalizing compensation for all DSD GAs (no disparity between new and returning GAs)
The task force has made their recommendations to me. After careful consideration of those
recommendations, I have decided to raise the GA compensation package to competitive rates over the
next two years. Starting next year (Summer 2015), the following packages will be offered:
Non-Residence Life Graduate Assistants will receive:
o 18 graduate credit hours of tuition remission
o $11,000 stipend (up from, $7,900)
Residence Life Graduate Assistants will be required to live on campus and will receive:
o 18 graduate credit hours of tuition remission
o $9,500 stipend (up from $6,500)
o $1,480 meal plan.
The following year (Summer 2016), the packages will be:
Non-Residence Life Graduate Assistants will receive:
o 18 graduate credit hours of tuition remission
o $15,000 stipend (up from $11,000 in FY 16)
Residence Life Graduate Assistants will be required to live on campus and will receive:
o 18 graduate credit hours of tuition remission
o $11,500 stipend (up from $9,500 in FY 16)
o $1,480 meal plan.
Page(1(of(3
83
Unfortunately, the Division will not be receiving additional funds to pay for the enhanced compensation
package. Thus, in order to balance our budget, we must eliminate 25% of our Graduate Assistant
positions over the coming two years. I am also accepting the recommendations of the Task Force
regarding where to eliminate assistantships:
"
NonMResidence"Life"posi6ons
FY16"eliminate"3"GA"posi6ons"(1"Dean"of"Students"area,"1"Campus"Ministry"area,"1"Assessment)
Increase"compensa6on"to"$11,000"for"all"remaining"GA."Increase"in"compensa6on"$3,100
Maintain"18"graduate"credit"hours"of"tui6on"remission
"
For"FY17"eliminate"3"GA"posi6ons"(1"Dean"of"Students"area,"1"Campus"Ministry"area,"1"Campus"Recrea6on)
Increase"compensa6on"to"$15,000"for"all"remaining"GA,"Increase"in"compensa6on"$4,000
Maintain"18"graduate"credit"hours"of"tui6on"remission
"
Residence"Life"Posi6ons
FY16"eliminate"1"posi6on
Increase"compensa6on"to"$9,500
Reduce"meal"plan"to"$1,480
Total"compensa6on"increase"$1,180
Maintain"18"graduate"credit"hours"of"tui6on"remission
"
FY17"eliminate"no"posi6ons
Increase"compensa6on"to"$11,500
Maintain"meal"plan"at"$1,480
Total"compensa6on"increase"$2,000
Maintain"18"graduate"credit"hours"of"tui6on"remission
"
Non Residence Life GA
compensation
FY15
Stipend
7,900
Total
compensation
$ 7,900.0
FY16
11,000
$
11,000.0
FY17
15,000
$
15,000.0
"
"
Residence Life
compensation
Stipend
Meal Plan
Total
compensation
FY15
6,500
3,300
$
FY16
9,500
1,480
9,800.0
10,980
FY15
7
7
1
13
FY16
6
6
0
12
FY17
11,500
1,480
$
12,980
"
Summary(of(GA
Director
Campus Ministry
Dean of Students
Assessment
Res. Life
FY17
5
5
0
12
Page(2(of(3
Campus Recreation
Total
2
30
2
26
84
1
23
If any of you have questions about this process, or the final decision, please feel free to contact me or
any member of the task force.
These were difficult recommendations to make, and to receive. I thank the task force for its work.
Although these were not easy decisions, I am confident it will make our graduate assistant ship
program even stronger and more competitive in years to come.
Jane Neufeld
Vice President
Division of Student Development
Page(3(of(3
85
Appendix N: Timeline
Preview Days Evaluation Timeline
DEC JAN FEB MAR APR
2014 2015 2015 2015 2015
Task
Meet with Jack McLean, JD,
and J. Curtis Main to discuss
evaluation plan and feasibility.
Pilot test quantitative and
qualitative instruments with
current GAs.
Revise plan and surveys based
on feedback.
Prepare quantitative survey for
CampusLabs distribution.
Preview Days 2015
Email quantitative survey to
Preview Days participants.
Send email reminders to
Preview Days participants to
complete quantitative survey.
Quantitative survey closes.
Compile and analyze
quantitative results (after
survey closes).
Contact possible interview
participants to set up
interviews (after survey
closes).
Schedule interviews with
participants.
Revise interview protocols
based on quantitative findings.
Conduct interviews.
Transcription and coding.
Analysis of qualitative data.
Member checking with initial
findings.
Presentation of final report and
recommendations.
MAY
2015
JUN
2015
JUL
2015
X
X
X
X
X
22-23
25
2, 4,
and 6
11
X
X
X
X
X
X
X
X
X
X
X
X
X
86
87
Appendix P: Presentation
11/30/14&
+ Logic Model
+ Evaluation Questions
! Is
+ Quantitative Approach
! Survey
! Cross-sectional
! Can
! Combination
! Does
! Census
design
questions
sampling with a 50% expected
response rate
! Data
1&
88
11/30/14&
+ Qualitative Approach
! Semi-Structured
! 5
Interviews
! 30
! Focusing
! Transcribed
+ Timeline
! Distribution
Feb 25th
! Survey
! Conduct
Interviews- Apr-May
! Presentation
coding
+ Budget
+ Next Steps
! Meet
! Discuss
feasibility of implementation
! Evaluation
2&