Академический Документы
Профессиональный Документы
Культура Документы
Summary ..................................................................................................................................................... 2
Description of the Program Evaluated .................................................................................................... 3
Context ..................................................................................................................................................... 3
Program Objectives ............................................................................................................................... 3
Program Components ........................................................................................................................... 5
Evaluation Method ..................................................................................................................................... 5
Participants.............................................................................................................................................. 5
Procedures .............................................................................................................................................. 5
Data Sources .......................................................................................................................................... 6
Results ......................................................................................................................................................... 7
Discussion ................................................................................................................................................... 8
Project Cost ................................................................................................................................................. 9
Appendices................................................................................................................................................ 10
Appendix A AIDETSM Patient and Customer Communication program .................................... 10
Session one: ..................................................................................................................................... 10
Session two: ...................................................................................................................................... 10
Appendix B Pre- Training Self-Assessment.................................................................................. 12
Appendix C Pre-Training Self-Assessment Results .................................................................... 13
Appendix D Post-Training Self-Assessment ................................................................................. 17
Appendix E Post-Training Self-Assessment Results .................................................................. 18
Appendix F AIDET Communication Competency Assessment ................................................. 22
Appendix G AIDET Communication Competency Assessment Results .................................. 23
Appendix H - AIDET Patient and Customer Communication Post-Training Survey .................. 27
Appendix I - AIDET Patient and Customer Communication Post-Training Survey Results ..... 30
Appendix J References .................................................................................................................... 34
May 8, 2014
Page 1
Summary
This document is the final report detailing the results of the formal evaluation of a
patient and customer communication program termed AIDETSM at the Archetype
Ambulatory Site of the Institution. The Midwest Regional Ambulatory Care executive
administration established both qualitative and quantitative objectives to determine the
viability of AIDETSM success to establish an institution-wide standard of service with
hospitality, respect and care.
Due to the decision-making nature of the objectives the CIPP evaluation model was
employed to assess the four integrated aspects of a developing program through the
utilization of rigorous and authentic assessments. The primary intent of the evaluation is
to provide the Midwest Regional Ambulatory Care executive administration with credible
evidence substantiating a) the program will provide the knowledge transfer required to
attain marked improvement in customer communication and service as reported by the
institution-wide Recent Patient Survey, b) a standard to effectively communicate with
customers and other representatives using language aligned with hospitality, respect
and care, c) the perception that the AIDETSM program is a meaningful, useful and
consequential component of employment.
Pre-training and post-training self-assessments were used to establish a base-line of
prior knowledge and skill levels to evaluate the context and input of the program. Posttraining surveys were implemented to determine the process and perceived success.
The product was evaluated using supervisory competency assessments and the Recent
Patient Survey rankings were monitored over a 15 month period to examine the impact
of final product.
The evaluation determined in order to establish an institution-wide standard of service
capable of achieving the desired customer service rankings a training program with the
qualities of the AIDETSM was warranted. Delivered its original format the program
achieved marked success introducing, instructing and supporting a standard of customer
communication. The program attained significant support for its merit and worthiness from the
attendees, although there were individual components of the training that were deemed less
than effective. While the Recent Patient Survey rankings in the first quarter; post-training, did
not achieve the desired level, there was marked improvement in customer perception.
Based on the results, the principle investigator recommends the implementation of the
AIDETSM Patient and Customer Communication program to achieve the objectives as
stated by the Midwest Regional Ambulatory Care executive administration. Additional
recommendations concerning the efficiency of the program indicates the need for slight
modifications as dictated by the post-training survey.
May 8, 2014
Page 2
May 8, 2014
Page 3
existing customer base, increasing the number of new customers through positive
feedback and reducing representative turnover with higher personnel satisfaction. A
secondary but equally important outcome is the fostering the goodwill, respect and
confidence toward the customers, staff and the public at large.
The overall objectives of the AIDETSM training program as indicated by the primary
stakeholder are to consistently achieve a customer service ranking above the 80th
percentile as reported by the institution-wide Recent Patient Survey. A secondary
objective as stipulated during the evaluation program design would be identifying
components of the program that could be streamlined to increase the effect and
efficiency of an institution-wide rollout. The objectives of this evaluation are stated as
follows:
90 % of the Ambulatory Site AIDETSM attendees will comprehend both the
significance and importance of the Acknowledgement, Introduction, Duration,
Explanation and Thanking elements patient communication and effectively apply
them to achieve superior customer satisfaction as indicated by both the training
and competency assessments.
90 % of the Ambulatory Site AIDETSM attendees will comprehend the benefits
associated with Managing Up to achieve superior patient communication and
customer service as indicated by both the training and competency assessments.
90 % of the Ambulatory Site AIDETSM attendees will effectively communicate with
patients and customers using language aligned with hospitality, respect and care
in accordance with the five fundamentals of patient communication as indicated
by the competency assessments.
90 % of the Ambulatory Site AIDETSM attendees will effectively communicate with
other representatives using language aligned with hospitality, respect and care in
accordance with the five fundamentals patient communication as indicated by the
competency assessment.
90 % of the Ambulatory Site AIDETSM attendees will possess a level of individual
accountability to deliver service required to attain a superior level of customer
satisfaction aligned with hospitality, respect and care as indicated by the Recent
Patient Survey.
90 % of the Ambulatory Site AIDETSM attendees should perceive the AIDETSM
program as a meaningful, useful and consequential component of the
Ambulatory Site onboarding and review process.
90 % of the Ambulatory Site AIDETSM attendees should perceive the AIDETSM
program as effectively and efficiently achieving all of the stated program
objectives.
Any of the AIDETSM program objectives that are deemed ineffective or ineffectual
should be reworked for a positive and successful outcome.
May 8, 2014
Page 4
Program Components
The AIDETSM training program is conducted in two sessions each approximately two
hours in length and one week apart. The sessions are conducted in addition to the
clinical day either in the morning or evening hours. An experienced training provider
was hired to facilitate the program. The activities associated with the program involve
brief expositional discourse, viewing content specific vignettes, self-evaluation,
testimonials, role play post-course debrief as well as obtrusive and unobtrusive
observations. A detailed description of the entire training program is included in
Appendix A.
Evaluation Method
The evaluation method selected to conduct the project is an adaptation of the CIPP
Model of evaluations. This model is a decision-focused approach and is designed to
provide both realistic and analytical substance for management decision-making. The
model utilizes all phases of a programs cycle specifically its structure, implementation,
review and revision to formulate sound effective decisions.
Participants
The participants involved in the evaluation are comprised of 22 service representatives
from the Archetype Ambulatory Site. They are between the ages of 25 and 57. They are
all female. There are 6 representatives with less than six months experience at the site
the other 16 have been at the site for anywhere between one and three years. Their
experience in the health care industry ranges from neophyte to seasoned healthcare
personnel. Some of have previously functioned in clinical or technical health related
areas.
Procedures
The CIPP Model includes four distinct components for evaluation; typically referred to
as Context evaluation, Input evaluation, Process evaluation and Product evaluation.
These components can be further clarified using four basic premises; what should be
done, how should it be done, is it being done as planned and did it work?
The initial component of the evaluation focused on existing skills and knowledge base
using a pre-training self-assessment. This instrument is designed to establish a baseline
for improvement. The self assessment consists of several mechanisms the most
critical was the introspective perception of the five fundamentals of customer
communication.
During the training program there are several opportunities for obtrusive observation. It
was deemed that these observations would be used only for instructive and edifying
purposes and would not be used for the analytical examination in the evaluation.
May 8, 2014
Page 5
The second element of the evaluation is the degree of knowledge transfer with respect
to the perception of new knowledge gained as a result of the program. This information
was gathered using a post-training self-assessment.
The final set of criteria concerning the level of skill and knowledge transferred as the
result of the program, were assessed during post-training unobtrusive supervisory
assessment using the competency assessment. This process was explained in the
training and devised solely for individual AIDET assessments to complete the AIDET
training contract and subsequent yearly performance review criteria.
In terms of the CIPP models ability to measure the formative components of a program
the post-training survey captured the essence of the program. This phase of the
evaluation assists in determining the overall efficiency and effectiveness as well as
participation satisfaction.
Data Sources
Data pertinent to the training program was collected using four specific instruments from
all 22 representatives attending the training program. The data collection strategy was
very specific. In order to guarantee 100 percent participation online form submission
after the fact was prohibited. Each participant was required to complete the data source
using paper and pencil before moving forward to the next phase of the program.
The four data sources were designed specifically for one or more aspects of the CIPP
evaluation method. The instruments have been provided in the Appendices to clarify the
nature of the information received as well as their respective outcomes. The details of
their inclusion in this document are as follows:
Data from the Recent Patient Survey; a fifth data source, is compiled and reported by
the Institution in quarterly cycles. The customer perception ranking for the Archetype
Ambulatory Site has been consistently ranging between 81 and 84 percent in Office
Staff Courtesy and Respect and customers Willingness to Recommend during the four
quarters priors to implementing the AIDET training. These rankings fall short of the
institution-wide directive to meet the 75th percentile. The details of the rankings have
been extracted from the quarterly documents (see Figure 1).
May 8, 2014
Page 6
May 8, 2014
Page 7
Data from the Recent Patient Survey compiled in the quarter since the AIDET training
was implemented showed a slightly positive effect with the Archetype Ambulatory Site
ranking 88 and 87 percent respectively in Office Staff Courtesy and Respect and
customers Willingness to Recommend. The details of the newly reported rankings have
been extracted from the quarterly documents (see Figure 2).
Figure 2. New quarterly rankings for Archetype Ambulatory Site
04/13 06/13 10/13 01/14 04/14
Office Staff Courtesy
and Respect
Willingness to
Recommend
Intuition-wide
75th percentile
94.00
92.50
Discussion
The Archetype Ambulatory Site accepted the responsibility of as the Beta site for the
AIDET training program and evaluation. The CIPP model is designed to assist in
making decisions by providing a systematic analysis of the need, how to meet the need,
whether the need was met, and could the need be met more efficient and effectively.
The pre and post training assessments in conjunction with the Recent Patient Survey
rankings indicate there is a need to define and establish a standard of effective patient
and customer communication.
The AIDET Communication Competency Assessment indicates that post training the
representatives illustrated marked improvement in effectively communicating with
patients. The assessment also indicates that participating in the training to acquire the
necessary skills and comprehend the requirements of excellent patient communication
the representatives can achieve more effective patient communication.
The AIDET Patient and Customer Communication Post-Training Survey clearly indicate
the representative perceived the program as effective. The majority of the attendees
consider the program did achieve the objectives. In terms of formative evaluation the
components of the program which reportedly were less effective and unproductive were
the vignettes and testimonials. The components of the training which reportedly were
most effective and useful were the explanations, role play and moderated discussions.
The most promising element reported as a result of the AIDET program and nearly
meeting the stated objective was the Recent Patient Survey compiled since the training
which indicated an increase of nearly 5 percentage points. If this trend continues the
percentages should soon be consistently meeting or exceeding the 90 percent
requirement.
In light of this evaluations reliable model (CIPP) substantiated through the analysis of
specific data collecting instruments there is clearly acceptance and benefit derived by
the training program. It is the recommendation of the principle investigator that the
program is creditable and worth the investment of an institution-wide rollout. Although
May 8, 2014
Page 8
some of the percentages achieved did not meet the stated 90 percent stated in the
stakeholder objective many of these soft skills require adequate time to measure the
full impact. For that reason, the principle investigator also recommends that the AIDET
Communication Competency Assessment tool should be utilized multiple times a year
with clearly communicated feedback to give recipients directives for improvement.
These actions coupled with periodic in-service components to revitalize the AIDET
philosophy would improve eventual impact realization.
In terms of streamlining the training to achieve maximum benefit and a higher degree of
satisfied attendees, it is the recommendation of the principle investigator to require
modification by the professional training facilitator to the following components: reducing
the number of vignettes and testimonials, slightly increasing the amount of role play and
moderated discussions while expanding the explanation and description components.
Project Cost
The project cost consists of four days of on-site and seven days off-site. Two on-site
days were meeting with the primary stakeholder to determine the objectives
requirements and solidify the evaluation method. One on-site day was spent with the
AIDET training facilitator to review and document the training program process and
agenda. The final on-site day was observing one of the AIDET training programs
(although the training took place on two different days only one day was billed since the
time consumed for each session was less than four hours). The additional seven days
off-site were spent preparing data source instruments, data collection, data analysis and
developing the final report. Both on and off site are billed at same daily rate however the
on-site services require an additional daily per-diem of $150.00. The full invoice follows.
Task Description
On-site formal evaluation
(primary stakeholder meeting)
On-site formal evaluation
(AIDET facilitator meeting)
On-site formal evaluation
(attending AIDET training)
Off-site formal evaluation
(data source instruments preparation, data
collection, data analysis and developing
the final report)
On-site per diem
TOTAL
May 8, 2014
Days
Daily Rate
Amount
$500.00
$1,000.00
$500.00
$500.00
$500.00
$500.00
$500.00
$3,500.00
$150.00
$600.00
$6,100.00
Page 9
Appendices
Appendix A AIDETSM Patient and Customer Communication program
Session one:
The initial session begins with a pre-training self-assessment to determine a baseline
for each participant to gain an introspective view of their own customer service skills at
present. The session then moves to a brief expositional discourse generalizing the
fundamental purpose of AIDET and defining each of its five components. Each of the
components has a specific purpose and goal. Each of these components is also
assessed separately and carries its own merit and value.
Several participants are then asked to assist one-by-one as the facilitator enacts
impromptu scenarios to illustrate both appropriate and inappropriate AIDET based
customer service examples. The viewing participants are then asked to comment on
these scenarios for a candid and open moderated discussion in order to determine what
components of the dialog seemed effective or ineffective and why.
Each of the participants is then given a worksheet to create their own unique script
using their previous skills and knowledge while incorporating their new knowledge of the
five components of AIDET. The participants are aware that once complete they will be
randomly paired up to practice and modify the scripts based on peer constructive
critique.
A question and answer period follows for the facilitator to guide any concerns, difficulty
or reservations.
The first session ends a brief introduction of the concept of managing up. Managing up
is defined as well as examples of subordinates and superordinate that could be
managed and why. Each participant has been given a worksheet to describe how they
would manage up themselves, one subordinate and one superordinate.
At the close of session each participant is given an assignment to actively practice the
AIDET principles on the job and determine a personal script. They are also asked to
contemplate the advantages and benefits of managing up.
Session two:
The second session convenes one week after session and begins by posing the
following questions:
May 8, 2014
Page 10
A brief series with testimonial from participants will be facilitated to highlight successes
using AIDET focusing its positive effect on customers and colleagues in an effort to
cultivate a buy in from all participants.
The session moves to the viewing of a series of vignettes set in various scenarios used
to illustrate adapting the AIDET components to any situation in an order to produce
positive outcomes.
After each of the vignettes the following questions will be open for discussion:
At this point the session will turn to the topic of managing up. A brief expositional
discourse will review the concept of managing up and the significance it has on
decreasing patient anxiety toward caregivers and healthcare coordination efforts as well
as assisting in gaining customer confidence. The participants will be asked to produce
their assignments on managing up.
The facilitator will ask for a series of volunteers to share their managing up worksheet.
This discourse is followed by a moderated discussion on positive effects and benefits of
managing up.
Session two ends with an informative discourse which includes a review of the
competency survey, the contract and personal AIDET responsibilities moving forward.
May 8, 2014
Page 11
May 8, 2014
Page 12
ACKNOWLEDGES the
patient/customer
9%
45%
Needs Improvement
46%
Meets
Exceeds
INTRODUCES self
32%
36%
Needs Improvement
Meets
32%
May 8, 2014
Exceeds
Page 13
Estimates DURATION
18%
27%
Needs Improvement
Meets
Exceeds
55%
27%
Needs Improvement
50%
Meets
23%
May 8, 2014
Exceeds
Page 14
MANAGES UP coworkers/departments
18%
27%
Needs Improvement
Meets
Exceeds
55%
2. Experience level
Yes
No
86%
May 8, 2014
Page 15
True?
False?
86%
May 8, 2014
Page 16
May 8, 2014
Page 17
ACKNOWLEDGES the
patient/customer
9%
27%
Needs Improvement
Meets
64%
Exceeds
INTRODUCES self
9%
36%
Needs Improvement
55%
May 8, 2014
Meets
Exceeds
Page 18
Estimates DURATION
5%
18%
Needs Improvement
Meets
77%
Exceeds
23%
27%
Needs Improvement
Meets
50%
May 8, 2014
Exceeds
Page 19
MANAGES UP coworkers/departments
18%
Needs Improvement
18%
64%
Meets
Exceeds
Yes
No
95%
May 8, 2014
Page 20
True?
False?
100%
May 8, 2014
Page 21
May 8, 2014
Page 22
Inconsistent
Consistent
64%
Role model
May 8, 2014
Role model
Page 23
Role model
INTRODUCES self:
27%
Inconsistent
27%
46%
Consistent
Role model
May 8, 2014
Page 24
41%
EXPLANATION:
27%
27%
Inconsistent
Consistent
46%
May 8, 2014
Role model
Page 25
36%
18%
Inconsistent
Consistent
Role model
46%
Consistent
Role model
May 8, 2014
Page 26
May 8, 2014
Page 27
May 8, 2014
Page 28
May 8, 2014
Page 29
5% 0%
18%
Excellent
32%
Satisfactory
Neutral
45%
Unsatisfactory
Poor
2. Was the program the right length for the material covered?
0%
14%
18%
27%
Strongly Agree
Agree
Neutral
41%
Disagree
Strongly Disagree
May 8, 2014
Page 30
4% 0%
18%
23%
Strongly Agree
Agree
Neutral
55%
Disagree
Strongly Disagree
May 8, 2014
Page 31
7. Which component(s) of the program did you find most helpful to achieve the
program objectives?
Explanations and
descriptions
32%
Testimonials
19%
12%
Role Play
9%
28%
Vingettes (video)
Moderated discussion
8. Do you feel the communication skills achieved as a result of the program will be
of benefit to patients?
0%
14%
14%
27%
Very certain
Certain
Neutral
45%
Uncertain
Very uncertain
May 8, 2014
Page 32
9. Do you feel the communication skills achieved as a result of the program will be
of benefit to colleagues?
0%
18%
23%
Very certain
Certain
41%
Neutral
18%
Uncertain
Very uncertain
10. How confident are you that this program has given you the skills to meet the
AIDET competency requirements?
0%
0%
32%
14%
Very confident
Confident
Neutral
54%
Doubtful
Very Doubtful
May 8, 2014
Page 33
Appendix J References
John, B., & Dutwin, P. (2011). The ABCs of Evaluation: Timeless Techniques for Program and
Project Managers (3rd ed.). San Francisco: Jossey-Bass.
Zhang, G., Zeller, N., Griffith, R., Metcalf, D., Williams, J., Shea, C., & Misulis, K. (2011). Using the
Context, Input, Process, and Product Evaluation Model (CIPP) as a Comprehensive Framework
to Guide the Planning, Implementation, and Assessment of Service-Learning Programs. Journal
of Higher Education Outreach and Engagement, 15(4), 5784.
May 8, 2014
Page 34