Running head: PROGRAM EVALUATION RUBRICS

Program Evaluation Rubrics

Krista Jackson
AET/570
May 9, 2016
Professor Dlabach

PROGRAM EVALUATION RUBRICS

Program Evaluation Rubrics for Starbucks

Level 1 Rubric: Reaction

Evaluation
Exce

Mod
erate

Poor

The
objectives
were stated
concisely
and clearly
for the
training on
how to make
the coffee
drinks. The
objectives
met every
need to
understand
what the
training was
covering
about how
to make the
drinks
perfectly.
The learners
felt positive
with the
knowledge
about
making
coffee that
was attained
through the
training
objectives.

The
objectives
were
communicat
ed through
the training,
but they
were not as
clear as they
could be
about the
objectives.
The
objectives
helped to
understand
how to make
the coffee,
but they did
not go in
depth
enough for
the learners.
It seemed
effective
about the
knowledge
of the
objectives of
making
coffee, but
the learners
were still
unsure

The
objectives
were not
stated
clearly or at
all during
the training
on learning
how to make
the coffee
drinks. The
missing
objectives
made the
introduction
into the
training
difficult and
almost
impossible.
The learners
felt troubled
and unsure
about the
objectives
going into
training.

llent
Design of the Training:
Traini
ng
Objectives

PROGRAM EVALUATION RUBRICS

about all of
the
objectives.

Pace
of Training

The
pace of the
training fits
because it
was learning
at the
learner's
own pace
training. The
training had
a specific
date to be
finished, but
there was
leniency
because
each barista
needed to
be ready
when they
stepped up
to the bar to
make drinks.
The learners
felt very
comfortable
with no
stress on the
pace, which
brought
incredible
morale.

The
pace of the
training was
learning at
the learners
own pace,
but there
was too little
or too much
concern on
the ending
date. The
pace helped
with learning
the drinks,
but more
time and
concentratio
n could have
helped. The
learners felt
content on
the pace of
the training,
but they
could have
learned
more before
stepping
onto the bar.

The
pace of the
training was
learning at
the learners
own pace,
but this rule
was not
followed
often. The
training had
a specific
date, but it
was not
followed, so
it brought on
more stress
on everyone
in the
environment
. The
learners did
not feel
comfortable
stepping
foot into the
bar area to
make drinks.

Diffic
ulty Level
During
Training

The
level of
difficulty
was
challenging,
but
invigorating

The
level of
difficulty
was
challenging,
and it was
hard to

The
level of
difficulty
was too
high, and
most of it
was hard to

PROGRAM EVALUATION RUBRICS

and
motivating
during the
training. The
information
was a lot to
take in, but
learning at
the learners
own pace
helped, and
the
information
stuck better
than most
training. The
learners
were
pleased with
the difficulty
level
because it
was a fun
challenge.

process
some of the
information.
Learning all
of the drinks
was a lot of
information,
and it was
harder to
remember
all of the
information.
The
information
was in the
learners
brain, but it
was difficult
to remember
because of
the difficulty
level. The
learners
were
sufficed with
the difficulty
level, but
they needed
something a
bit simpler
for
beginning
training.

understand
how to make
the drinks to
their
standards.
Learning all
of the drinks
was almost
impossible,
and
rememberin
g the drinks
was too
difficult. The
learners
were
discouraged
and
dissatisfied
with the
results in the
end.

The
instructor
was helpful,
but they
were not
always
around to
assist the
learners. The
instructor

The
instructor
was not
helpful
during the
learners
training. The
instructors
were always
missing

Evaluation of the Instructor:

Instr
uctors
Performan
ce

The
instructor
was
informative,
and they let
learners
learn on
their own.
The
instructor

PROGRAM EVALUATION RUBRICS

Instr
uctors
Knowledge

answered
any question
immediately
with a
correct and
informative
answer
about the
knowledge
of the coffee
drinks. The
instructors
would
visually
show the
learners at
the bar any
doubts or
questions
the learners
had about
any coffee
information.
The learners
felt
completely
comfortable
and
captivated
with the
instructor.

answered
the learners
questions
when the
learners
could find
them, and
the
instructor
had time for
the learners
training to
be a barista.
There was
some helpful
feedback
when the
instructor
was not
distracted,
so the
learners felt
like they
were left in
the dark
sometimes.
The learners
felt uneasy
but content
with the
instructor.

when they
had
questions,
so the
learners had
to resort to
other
employees
who were
taking care
of
customers.
There was
no feedback
on how well
the learner
was doing
with their
coffee
knowledge.
The learners
felt hopeless
and scared
of the future
working at
the store
because of
my
instructor.

The
instructor
knew
everything
there was to
know about
how to make
coffee to
how to run
the store.
The
instructor
knew how to

The
instructor
knew the
information,
but not to
the specific
level that
the learner
was
supposed to
be learning
as a new
barista. The

The
instructor
barely knew
any
information
and was not
knowledgea
ble at all.
They could
not answer a
specific
question, so
they would

PROGRAM EVALUATION RUBRICS

teach the
subject of
making each
drink in
depth. They
could
answer any
questions
without
looking up
the answer
about the
coffee
drinks, but
they could
show the
learner
where the
answer was
located. The
instructor
knew easy
ways of
rememberin
g the drinks,
and these
tips were not
located in
the training
manuals.
The
instructor
made the
learners feel
like they
were
learning
information
that they
would use
forever. The
instructors
knowledge
helped the
learners be

instructor
had to look
up questions
because
they were
forgetful or
did not often
train, which
made them
rusty with
knowledge.
The
instructor
did not want
to get
information
wrong, so
the
instructor's
research
helped
themselves,
but they
should have
the
knowledge
pouring out
of their
brains. They
pawned the
learners off
on other
employees
because the
instructor
was having
trouble with
a topic. The
instructor
made the
learner feel
selfconscious
and unsure
to begin

tell the
learner to
look it up in
the manual.
The manual
was large,
but
contained at
least fifteen
smaller
booklets,
and it was
not easy to
look up the
answer. The
instructors
knowledge
just made
learning
more
difficult, or
the learner
learned the
wrong
information.
The
instructor
made the
learner feel
like they did
not learn
anything,
especially
from the
instructor.
The
instructors
knowledge
was
extremely
lacking and
shouldnt
have been in
a training

PROGRAM EVALUATION RUBRICS

Instr
uctors
Time
Manageme
nt

the
employees
that they
are, and the
instructors
will always
be
remembered
for their
knowledge
and training
skills.

working at
the bar
making
drinks. The
instructor's
knowledge
helped, but
it was not
enough to
be
considered
to be a
trainer.

position.

The
instructor
knew how to
manage
their time
well by
multitasking;
the
instructor
would train
the learners
and do their
own work in
their spare
time. The
instructor
was always
working, but
they were
always
available for
questions.
They knew
how to make
sure that
they were
not giving
the learners
enough
attention,
and they

The
instructor
knew how to
manage
their time,
but they
were not
good at
multitasking.
It was
difficult for
the
instructor to
train and do
their work at
the same
time. They
were
consistent
with their
work, but
there were
times when
they were
not doing
anything.
They did not
check in on
the learners
during their
spare time,

The
instructor
did not know
how to
manage
their time
well, and
they did not
like training
and having
their own
work at the
same time.
They would
be negative
about the
situation, or
make the
learner feel
in the way
when they
were
supposed to
be training
the learner.
They did not
check in on
the learner,
start a
conversation
, or offer any

PROGRAM EVALUATION RUBRICS

Instr
uctors
Feedback
and
Answers to
Questions

would check
in on the
learner when
the learner
was doing
their
individual
work. The
instructor
even had
time for a sit
down to take
the learners
mind off of
the coffee
material for
a moment.
The
instructor
had their
time
managemen
t down
perfectly,
and the
learner
never felt
like they
were being
ignored.

or try to
start a
conversation
with the
learner. The
instructor
needed
better time
managemen
t skills, but
they were
still efficient
enough to
be helpful
during the
learners
training as a
barista.

help unless
the learner
asked. When
the learner
did ask it
would take
forever for
the
instructor to
get over to
help, and
someone
else had
already
helped the
learner by
then. The
instructor
had no time
managemen
t skills, and
they did not
help with the
learners
training at
all.

The
instructor
offered
feedback
every
chance they
got because
they knew it
boosted
morale and
self-esteem
in a new
employee.
The

The
instructor
offered little
feedback,
and it was
almost
forced out of
them, or
there was
not very
much
feedback. It
did not help
with

The
instructor
did not offer
any positive
or negative
feedback,
and it made
the learner
feel like they
were
invisible
while they
were
training. The

PROGRAM EVALUATION RUBRICS

instructor
gave useful
feedback,
and it was
not all
negative if
the learner
had things
to work on,
so the
instructor
used the
sandwich
method to
make the
learner feel
comfortable.
The
instructor
was always
willing and
around to
answer the
learners'
questions
and they
were always
very
informative
with their
answers. No
question
went
unanswered,
and the
instructor
would add in
extra
information
to help the
learner. The
instructor
assisted the
learner in
any way that

motivation
and
enthusiasm
for the
knowledge.
The offered
answers to
questions
but they
were not
always right,
or it took
forever to
get a
question
answered.
They were
not used to
giving
feedback, so
their
feedback
was what I
was doing
wrong.
Answering
questions
were
overlooked
or
sometimes
ignored
because of
what they
were doing
and then
they would
forget. The
instructor
did offer
some
feedback
and
informative
answers that

instructor
did not
answer
questions to
help with
learning the
coffee
material.
The learner
did not feel
confident in
the job,
training or
material
because of
the
instructors
actions.

PROGRAM EVALUATION RUBRICS

they needed
during the
training. The
instructor
was
attentive
and always
offered
answers and
feedback,
and this
made the
learner feel
lively and
passionate
about
learning
about the
coffee
material.

10

were correct
sometimes,
but they did
not make
the learner
feel
confident.

Evaluation of the Training Exercises:

Unde
rstanding
Concepts &
Their
Application

The
concepts
were in the
booklets to
the learners
manual, and
it was very
informative
on the
details of
coffee and
making
coffee. The
manuals
explained
and showed
how to apply
the concepts
when the
learner was
behind the
bar. The
instructor

The
booklets in
the manual
explained
some of the
concepts, at
least enough
information
to get by
behind the
bar. It
showed how
to apply
them in an
unclear
manner, but
sometimes it
was easy to
understand.
The
instructor
did not do
much hands-

The
booklets in
the manual
did not
explain the
concepts, so
it was hard
to
understand
anything.
Since, the
concepts
were absent,
then how to
apply the
concepts
was not in
the manual.
The
instructor
did not do
any handson training

PROGRAM EVALUATION RUBRICS

Time
to Practice

11

would show
the learner
how to apply
them in a
hands-on
environment
for further
knowledge
on how to
apply the
concepts.
The learner
had a full
understandi
ng of the
concepts,
and how and
when to
apply them
from the
manuals and
instructor.

on training
to show how
the concepts
apply in
person. The
manual gave
the learners
a vague
general
understandi
ng of the
concepts
and how to
apply them.

for learning
how to apply
the
concepts.
So, the
concepts
and how to
apply them
were not
learned.

There
was always
time to
practice
when the
learner was
at work
training. If
the learner
wanted to
practice, and
there was a
rush, then
they would
go back to
their
booklets to
the manual
about
coffee. After
a bit of
training, the

There
was time to
practice
when the
learner was
training at
work, but it
was only
during
certain
times, and it
was not
always
convenient
timing
during
training.
During a
rush
sometimes
there was
nowhere for

There
was never
any time to
practice
when the
learner was
training at
work. There
was not a
proper area
to practice
the material,
which
affected the
training and
learning as a
new learner
to coffee
and making
coffee.
During a
rush, there

PROGRAM EVALUATION RUBRICS

Exerc
ises for the
Concepts

12

instructor
would have
time for the
learner to
practice on
the bar with
real
customer's
drinks. The
slower times
were the
best times to
practice with
customer's
drinks
because
there is less
stress with
the store not
being busy.
The whole
idea of
training was
to practice,
so there was
never time
lacking to
exercise
hands-on.

the learner
to work on
their
booklets, so
time to
practice in
any way was
hindered.
The
instructor
did not let
the learner
on the bar
make
customer's
drinks until
the learner
was almost
done with
their
booklets.
The learner
did get
chances to
practice, but
it depended
on the day
and trainer
for the day.

was no time
to practice,
and the
learner could
not practice
on
customer's
drinks. There
was no
hands-on
experience
during the
training.

The
instructor for
the day
would give
the learner
their manual
with their
booklets in
it. The
booklets had
information;
guidance,
practice
questions,
quizzes, a

The
instructor for
the day
would give
the learner
their manual
with their
booklets in
it. The
booklets had
information
about
coffee, and
exercises as
quizzes to

The
instructor for
the day
would give
the learner
their manual
with their
booklets in
it. The
booklets
were just
filled with
text with no
photos,
quizzes, or

PROGRAM EVALUATION RUBRICS

hands-on
exercise to
do behind
the bar, and
photos to go
along with
the
information.
The booklets
were
interactive,
and the
learner
learned at
their own
pace with
the booklets.
The learners
got to learn
how to make
every single
drink on the
menu, the
cash
register, the
pastries,
different
coffees from
different
regions, and
cleaning
information.
The learners
had to have
the
instructor
over each
time they
started
something
new, and
then the
learner was
allowed to
keep

13

be graded,
and they
had to be
done in pen
ink. The
exercises
outside of
the booklet
were just the
instructor
showing the
learner stepby-step how
to do
something.
The learner
would go
into the
training with
no
experience,
and the
instructors
were not
always
watching, so
the learners
did not get
as much out
of the
exercises as
they could
have
because the
instructor
had their
own work
too. The
exercises
helped, but
there was
not a variety
of ways to
take in the
material. It

practice in
them. The
only exercise
was reading
the text, and
then the
learner went
straight to
the bar with
no
experience.

PROGRAM EVALUATION RUBRICS

practicing on
the bar. The
instructor
would check
the learners'
drinks to
make sure
they were
correct; the
instructor
would ask
the learners
questions or
give them
easy tips on
rememberin
g the
material.
The booklets
gave the
learners
experience
in many
ways
because it
was not all
just reading.
The quizzes
and other
answer
places were
all done in
pencil, so
the learners
could go
back on
topics they
had trouble
with at that
time. These
different
exercises for
the learner's
mind and
practice

14

was not as
in depth, so
when the
learner got
to the bar to
make drinks
they did not
feel
confident.

PROGRAM EVALUATION RUBRICS

15

making the
coffee drinks
helped
exponentiall
y.

How the Training is Relevant to Learners:

Apply
ing
Knowledge
at Work

Apply
ing
Knowledge
Personally

After
the learners
training they
were able to
apply their
knowledge
at work. The
learner was
successful
and
knowledgea
ble from
their
training, so
they had the
ability to do
anything in
their work
environment
.

After
the learners
training they
were able to
apply some
knowledge
at work. The
learner was
moderately
trained, and
their
knowledge
and ability
were just
enough to
get by at
work.

After
the learners
training they
were not
able to apply
any
knowledge
at work. The
training was
not relevant,
and this
made the
learner
unsuccessful
at work.

The
knowledge
that the
learner
acquired
during
training
helped them
at home
when they
were not
working. It
was relevant
and helpful
in their
everyday life
because

The
knowledge
the learner
learned
during the
training
helps them
sometimes if
they were
home. It was
helpful when
the learner
wanted to
drink coffee
for a
change, but
they did not

The
knowledge
the learner
learned
during the
training did
not help
them in their
personal life.
The learner
did not drink
coffee, so
there was no
need to
make coffee,
or apply any
knowledge

PROGRAM EVALUATION RUBRICS

Reco
mmendatio
ns

16

some of the
learners
made coffee
often. The
knowledge
helped the
learners
appreciate
and learn
even more
about
coffee.

learn
anything
more about
coffee
outside of
training.

unless they
were
working.

The
learner will
recommend
this type of
training for
all
businesses
and
individuals.
It had
multiple
learning
styles, very
informative,
and the
learner
attained a
large
amount of
knowledge
from the
training. The
learner
thinks
everyone
should be
able to
experience
such a
diverse way
of training.

The
learner will
probably
recommend
this training
for others to
try. The two
different
learning
styles help
learners in
some ways,
but it does
not have a
lot of
diversity.
The learner
still thinks
that people
would
benefit in
some ways
from the
training.

The
learner
would never
recommend
this training
to anyone or
any
business. It
only had one
learning
style that
gives the
learner no
experience.
The training
was not
knowledgea
ble enough,
and it is a
waste of
time even to
talk about it.

Implementation and Logistics:

PROGRAM EVALUATION RUBRICS

Layo
ut of the
Training
Environme
nt

The
training was
very laid
back, and
the
instructor
wanted
everyone
comfortable
during
training. The
learner was
able to sit
anywhere in
the lobby, at
any chair,
table, or
comfy chair
with a coffee
drink, while
they worked
on their
booklets in
their
manual. The
layout of
behind the
bar was
relatively
roomy, and
enough
room to get
practice in
while still
getting
customer's
drinks out.
For both
training
environment
s, everyone
has room,
including the

17

The
training for
the manuals
is done on a
specific
table by the
bathrooms
unless the
table is
taken. If the
table is
taken, then
there is
another
specified
table.
Behind the
bar for
practicing
drinks is not
roomy, so if
someone is
clumsy, then
they may
have
trouble. The
training
environment
s are stricter
for the
learners
training, and
not as
comfortable
as they
could be for
training.

The
training in
the booklets
for the
manual is
done in the
back room
sitting on a
fold out
chair. The
chair has to
be away
from
everything
the
employees
may need
access to,
and there is
no table to
set the
booklets on
for training.
Behind the
bar is
crowded
because of
all of the
rushes, and
there is no
room for
practice, so
then both
environment
s are
uncomfortab
le for the
learners.

PROGRAM EVALUATION RUBRICS

18

learner while
they were
training to
make coffee.
Light
ing and
Visuals

Lighti
ng and
visuals
depend on
the
individual,
but the
lighting was
a little too
bright in the
store unless
they put
down the
shades. The
instructors
always let
the learners
put the
shades
down, and
everyone
saw an
improvemen
t, and they
started
being down
during the
whole day.
The visuals
of the
drinks,
names, and
prices were
all easy to
see for
learners, so
it was easy
to refresh on
the menu.

Lighti
ng was
bright most
of the time
because
they did not
put down
the blinds
very often
during the
day. The
learner had
to find the
right spot to
work on
their
manual, so
the sun was
not shining
on them.
The visuals
of the
drinks,
names, and
prices were
relatively
easy to see,
but the font
needed to
be bigger to
read the
menu.

Brea

Break

Break

The
lighting was
always
bright with
the sun and
florescent
lights, and
this is not a
comfortable
environment
for
customers or
learners. The
lights
distracted
from the
booklets for
the manual,
and the sun
or bright
lights were
always in
the learners
eyes while
making
coffee. The
environment
was
uncomfortab
le for
customers
and the
learners.

There

PROGRAM EVALUATION RUBRICS

ks

s during
training
were the
same as if
you were a
regular
worker on
the floor.
The breaks
were on the
schedule,
organized,
and printed
out in a
notebook.
The breaks
depended on
how many
hours the
learner
worked, and
the company
gave more
breaks than
Texas gives
normally.
Instructors
always
made sure
that
everyone
got every
break, even
if there were
a lot of
rushes in a
row. During
training, the
learner
could clock
out if they
wanted to,
and they
could go on
a walk to

19

s during
training
were the
same as a
regular
employee
working on
the floor.
The breaks
were shown
on the
schedule,
but they did
not always
get to them.
During
training, the
instructor
forgot to
explain
breaks to
the learner,
so they had
to ask after
training for a
few days.
Breaks were
not as paid
attention to
because
employees
were
overlooked.

was no time
for anyone
to have
breaks; and
even in
training and
working on
the manual,
the learner
was not able
to take any
breaks. The
managemen
t did not find
breaks
important,
and the
instructors
were more
concerned
with how
high the
labor was
because
they wanted
to let people
off early.
Letting them
off early and
taking no
breaks
affected the
learners
training
because of
the
instructor
and not
getting a
break.

PROGRAM EVALUATION RUBRICS

20

stretch, and
then come
back and
clock back in
because the
instructors
were
concerned
about the
learners
being
comfortable.
Breaks were
always given
and
respected by
everyone.
Leng
th of
Training

The
training was
at the
learners own
pace, so the
length of
training was
excellent.
The learner
still got their
breaks each
day they
went into
work for
training to
be a barista.
The length
of training
was
necessary
for all of the
information,
so the
learner was
comfortable
when they
were behind

The
training is
fast pace,
and the
learner did
not get as
much time
to spend on
the booklets
and
practicing as
they
needed. The
length of
training is
set by
standards of
percentages
of how
quickly
people did
their
training. The
learner
started
having
trouble

The
training
length is
short, so the
learner did
not get the
training they
needed to
be a great
barista. The
instructor
did not pay
attention to
training
because
they wanted
everyone
working
behind the
bar waiting
on
customers
and making
drinks. The
manuals
were not
paid

PROGRAM EVALUATION RUBRICS

S
enses
(seeing
,
tasting,
smellin
g,
touch,
and
hearing
)

21

the bar
making
drinks. The
training
length
depended on
how many
days, hours,
and how
much
training the
learner got
done each
day. The
training
length made
the learner
confident
when they
walked
behind the
bar.

reading, and
they could
not miss
their
deadline for
the length of
their
training. The
learner was
unsure when
they
stepped
behind the
bar to make
drinks for
the first
time.

attention to
or used, and
there was no
formal
training. The
learner felt
lost and
confused for
months
every time
they
stepped
behind the
bar because
there was
not much
training at
all.

All of
the senses
are used in a
coffee shop.
The learner
could see
the menu,
bar, and
everything
they had to
use. The
learner
could taste
what they
smelled, and
they would
do coffee
tastings
during
training. The
smell was
wonderful

All of
the senses
are use in a
coffee shop,
but not
every
instructor
focuses on
these for the
environment
. The learner
did not have
to have
perfect
vision
because
they did not
care. The
instructors
did rare
coffee
tastings, so

All of
the senses
are
experienced
in a coffee
shop, but
during
training,
none of
them were
used. The
learner did
not have to
see the
menu or
anything,
but they
needed
touch to use
the
machines.
Tasting and

PROGRAM EVALUATION RUBRICS

every time
they walked
inside, and it
was also
useful during
the coffee
tastings.
Touch during
training is
essential
because the
learner
needed
hands-on
training for
the
machines.
During the
training with
the
instructor
the learner
could always
hear them,
or they were
both in the
back room,
which is
completely
quiet and
good for
short
discussions.

22

smell and
taste were
not used as
much. Touch
was used
during the
short
training, and
then after
because the
learner had
to touch the
cups and
machines to
do a drink.
The hearing
was not
always
focused on
in training,
so the
learner
missed out
on some
information,
and the
instructor
did not want
to repeat
themselves.
The senses
are just not
concentrate
d on
completely.

smelling did
not matter
because the
instructor
did not do
coffee
tastings
while the
learner was
training. The
hearing did
not matter
because the
instructor
was not
talking or
training. So,
none of the
senses were
necessary
for the
learners
training.

PROGRAM EVALUATION RUBRICS

23

Level 2 Rubric: Learning Evaluation

Excellent

Moderate

Poor

Five Steps for Learning Evaluation (Coscarelli & Shrock, 2008, p. 4)

Analysis What is being tested? The effectiveness of the training
program at Starbucks six to nine years ago. Two choices of analysis:
1.
Performance
Tests
Observable
performance
on a rating
scale, and
these tests
were used at
Starbucks.

The instructor sets

aside a slow time at
work to test the
learner. There are
two to three people
observing the
learner's behavior,
who are experienced
instructors, and are
honest with no bias.
The instructors
watch the learner's
performance during
the test and rate
their performance
fairly. The observers
are all professional
and nice, so the test
is not as stressful.
The learner shows
them what they
learned in training
on how to make the
coffee drinks and
other tasks.

The instructor set

aside a time without
paying attention to
busy times at the
store. The instructor
only has one other
experienced
instructor with them
sometimes. Honesty
and bias can go
either way with the
instructors
analyzing the
learner's
performance. They
are professional but
show no emotions,
and it makes the
test stressful. The
learner tests their
performance for
them, and the score
can be fair or unfair
scoring. When there
are fewer
instructors during
the testing, then
there can be a gap
between the results
scoring the learner.

The instructor pulls

the learner aside
during a rush to do
their performance
test. The instructor
is the only one
analyzing the
learners
performance, so
honesty and bias are
big issues because it
becomes more of a
test of if the
instructor likes the
learner or not. The
instructor is not
professional because
no one is watching
their performance,
so the entire test is
stressful and
uncomfortable, and
it makes it highly
likely that the
learner will mess up
from their nerves.

2. Top of the
Skills
Hierarchy
tests high-

The instructor tests

the learners on
higher skills from
the training they

The instructor tests

the learner on
higher skills from
the training the

The instructor does

not test the learner
on any higher skills
they learned from

PROGRAM EVALUATION RUBRICS

24

level skills on
the hierarchy

went through to
make coffee drinks.
The test is not about
memorizing, so it
focuses on how the
learner can apply
their knowledge to
the job, and other
skills higher on the
hierarchy. The test
shows the instructor
that the learner
understands more
than just how to
make the drinks.
The test is helpful
when the learner
wants to show their
competence in the
training and
knowledge they
attained. It shows
the instructor that
the learner does not
just know how to
perform, but they
are knowledgeable
and able to use it in
many areas. A
coffee tasting and
pairing will show the
instructor that the
learner can apply all
of the material
learned during the
test. The scoring is
accurate because
the instructor is
testing for the right
material.

trainers went
through to make
coffee. The
instructor tests on
memorization and a
couple of higher
skills. The test has
nothing to do with
memorization, but
the instructor is not
handling the testing
correctly. The
learner's
competence can be
shown with both
ways of testing, but
that is not the focus
of this type of test,
so the results and
scores are not
accurate for the
test.

the training. The

instructor goes
straight to
memorization
testing, which is a
completely different
type of test. The test
and scores do not
apply because the
test was given in the
wrong manner.

3. Validity
Does the test
measure
what it

The instructor tests

the learner on the
training; to see if
they learned the

The instructor tests

the learner on parts
of the training, but
missing some

The instructor tested

the learner on
random details or
facts about what to

PROGRAM EVALUATION RUBRICS

intends to
measure?

information, and to
make sure the
training is effective
and valid for the job.
Experts look over
the test first to
legally claim that
the test is valid. The
test is for how much
was learned,
understood, and had
knowledge of the
material for making
drinks and more.

25

important parts.
The instructor is
trying to see if they
learned the
information, and to
make sure the
training the
instructor uses is
effective and valid.
Experts are
supposed to look
over the test first to
make it legally
valid, but this test
had not been made
valid yet. The test
did not do a great
job showing how
much was learned,
understood, and
had knowledge of
the material. It was
not a completely
valid test, so it
should not have
been used,
technically.

do when they were

at work. The
instructor did not try
to find out how
much had been
learned, so they
could not tell if it
was effective or
valid. Experts are
supposed to look
over the test first to
make it legally valid,
but this test was
never looked at as a
test, let alone a valid
test. The test did not
show how much was
learned, understood,
and had knowledge
of the material that
the learner attained
during training. The
test was not valid,
and it should not
have been used, or
even considered any
type of testing. It
was more of a
conversation with
random topics,
which did not relate
to the leaners
training.

4. Construction How to write the test elements and details?

Checklists
breaks down
the process
or product
into a series
of binary
choices, for
example,
present or
absent,

The training has a

checklist of points to
cover in
performance tests.
The instructor
follows the checklist
to make sure
everything is
covered, and no
information is

The training has a

checklist of a few
points to use in the
performance test,
but the checklist
does not cover very
much information.
The instructor may
end up using a
numerical scale to

The training has a

checklist with no
information or a
couple of points to
cover for a
performance test. It
is not enough
information for a
performance test.
The instructor does

PROGRAM EVALUATION RUBRICS

26

(Coscarelli &
Shrock, 2008,
p. 6). They
are not
scored like
rubrics,
which use
words and
number
scales.

missed on the
checklist. The uses
of binary choices are
available instead of
a number scale for
the performance
tests.

rate evaluations
instead of the
binary rating for
performance tests.
The change will
make the
performance test
scores inconsistent,
and not reliable for
proof of
performance.

not use the binary

rating for the
performance test.
The checklist is not
even usable in this
situation.

Multiple
choice items
above the
memory
level
(Coscarelli &
Shrock, 2008,
p. 6)

The test is a
multiple choice
which the instructor
will make higher
cognitive level test.
The test is more in
depth than a
memorization
multiple choice test.
The test has
questions about the
material that was
presented, but the
question will be
asked differently.
The learner has to
use their higher
cognitive thinking
skills to connect the
question with an
example they have
not experienced or
covered in the
training. The test
does not have
questions that are
memory based
because the
questions are
supposed to make
the learners think in
depth about each
topic.

The test is a
multiple choice that
is supposed to tap
into the learners
higher cognitive
level. The test has
mistakes where
some of the
questions are
memory based
questions. The
questions do not
use the right
examples to
connect to the
information from
the training. The
learner does not
have to think as in
depth as they were
lead to believe.

The test is a multiple

choice that does not
go over the memory
level of thinking. So,
this makes the test
easy for the
learners, and they
do not have to use
cognitive skills to
figure out examples
for themselves. The
test does not help
the learners at work
because if a problem
occurs that they
have not
experienced, then
they are not
prepared to think
cognitively at a
higher level. The
test is dumb downed
because of the
individuals lack of
knowledge and
understanding of the
training.

PROGRAM EVALUATION RUBRICS

27

5. Standard Setting Establish a legally defensible cut-off or

mastery score. Three approaches to a score that is standard:
Informed
Judgment

The instructor did

their research to
make an informed
decision on what the
standard score
should be for
mastery after
training. The
judgment can be
backed up with
references and
individuals who
helped the
instructor make the
informed judgment
on the standard
score. The methods
they use give
learners a more
complete and
accurate standard
score.

The instructor did

some research, but
is still unsure of
what to make the
standard score for
mastery after
training. They end
up using their
research and
personal opinion for
their decision. Their
judgment can only
be backed up with
references they
found, that created
the standard score.
The method they
used does not give
a completely
accurate standard
score for learners.

The instructor uses

themselves or
nothing as an
informed judgment
on what they
personally think the
standard score
should be for
mastery after
training. The
judgment cannot be
backed up on paper
in any way. The
method of choosing
the score
themselves gives
the learners an
inaccurate standard
score for learners.

Conjecture

At least two SME's

look over the
training and
calculate the
probability of an
individual with basic
competence can
pass each part of
the training for
mastery level. By
using a numeric
scale, the SME's can
judge what the
average mastery
score will be by
debate and
agreement. How in
depth the mastery
score level is

They had one to

two SMEs looking
over the training to
calculate the
probability of an
individual with basic
competence can
pass each part of
the training for
mastery level. The
bar is set lower than
it should because
there are not
enough opinions.
The numerical scale
is used, but they
are not as
experience, so the
master score is just

There was only one

SME in training
looking over the
training to calculate
the probability of an
individual with basic
competence passing
each part of the
training for mastery
level. There is set
way too low for
competence
because there are
only one persons
opinions. They do
not pay attention to
the numeric grading
because of their lack
of experience, so the

PROGRAM EVALUATION RUBRICS

Contrasting
Groups

28

attained makes it
effective and legally
valid.

a decision or
agreement. The
mastery score level
can still make it
effective but not as
legally valid.

mastery level score

is just a decision by
the one SME. The
mastery level
scoring is not
effective, and it is
definitely not legally
valid.

The instructor and

SME's gather
information together
about similar and
different tests and
test scores. They
contrast each of the
groups to find the
standard median
score. The different
examples and
standard scores give
all of them a better
idea on how to
change the mastery
score for the
training. The
contrasting groups
approach helps
learners because it
is fairer because it
shows where their
actual level should
be after training.

The instructor
gathers a few
examples of
contrasting group
standard scores, but
does not do a lot of
research for
multiple groups to
contrast. They need
to find the standard
median score, but
with little
information to
contrast group
scores, it is harder
to come up with a
more accurate
conclusion. The few
examples they
attained will at least
help to give them a
different
perspective and
idea about changing
the mastery score
for training. The
contrasting groups
approach helps
instructors and
learners, but there
is not enough
information to make
a concrete standard
score after training.

The instructor does

not gather
contrasting group
standard scores
because they think
that they can choose
what they want.
There is no way to
find the median
score because of the
lack of contrasting
groups as examples
and comparisons.
The absence of
examples makes it
impossible to find
the standard score
of mastery level
after training. The
missing group
examples to contrast
makes the quoted
standard score by
the instructor is
inapplicable.

PROGRAM EVALUATION RUBRICS

29

PROGRAM EVALUATION RUBRICS

30

Level 3 Rubric: Evaluation/Behavior

& Level 4 Rubric: Results
Note for the beginning:
First:

- Discover to what extent training is being used or not being used.

- Assuming there is some application comes the second.

Second:

- Then, dig into the reasons that training is being used or not used.
- Sometimes, although not often, these reasons will lead one back to
deficiencies in the training
design or delivery.
- In these cases, feedback to the L&D functions is provided, enabling
leaders there to make the changes needed to resolve the problems.
- More often, though, the reasons that training is being used (or not
used) will lie in performance factors under the control of managers
and others outside of the L&D function.

Third:

- Level 3 refers to gathering information on how (and how well) the

skills, knowledge, and attitudes that were acquired in a training
program are being used in workplace performance.
- Evaluations = Behaviors = Application

Fourth:

- The design and delivery can be root causes in learners not using
their training.

Excellent

Moderate

Poor

The Method of Level 3 Evaluation: Observation (Direct and Covert)

- Observation of learners working, after training, can tell the instructor a
lot about the effectiveness because they can see if they are applying their
new knowledge in their work.
- The observation was the method that they used for the training because
it was the fastest and easiest for management in a fast paced
environment.
Direct
Observation

The trainer would

start out on the
register, so they

The instructor
would start out in
any position, and

The instructor puts

the learner
anywhere and does

PROGRAM EVALUATION RUBRICS

Covert
Observation

31

could watch the

learners make the
drinks, and see
how well the
learner was doing
after the training.
The instructor
would put the
learner in other
positions on the job
to test the whole
effectiveness of
the training. The
learner would excel
in all of the
positions, and all of
the knowledge
attained during
training would be
visible to the
instructor.

have the learner in

a random position
that was
convenient for the
instructor. The
instructor would
test the learner in
some positions to
get an idea of the
effectiveness of
the training. The
instructor is not
looking focused
completely on the
results because
they are distracted.
The instructor does
see some results
during the
observations.

not observe the

learner. The
instructor is too
busy and finds the
direct observation
as a waste of time.
The results they
recognize are
feedback from
other employees,
so the results are
not valid for the
effectiveness of
the training.

The trainer would

have the shift
leader during that
shift observe the
learners behavior
because they can
observe the learner
without them
knowing they are
still being
evaluated. The
shift leader can
report back on how
the learner is
doing. Having each
shift leader do
observations will
give the instructor
valid and unbiased
evaluations of the
training.

The instructor
would have the
shift leader during
that shift observe
the learners
behavior. The shift
leader is supposed
to report back on
how the learner is
doing, but they are
not as attentive
because they have
their own work to
do. Each shift
leader does the
observation of the
learner, but the
inconsistencies in
how dedicated the
shift leaders were
during the
observation make
the evaluations

The instructor
would have the
shift leader during
that shift observe
the learners
behavior. The shift
leader is supposed
to report back on
how the learner is
doing, but they do
not report back to
them or claim they
couldnt do it. The
shift leader does
not observe the
learner, and they
do not give
evaluations, so the
information is not
valid and usually
biased.

PROGRAM EVALUATION RUBRICS

32

harder to evaluate.

The Method of Level 3 & 4 Evaluation: SCM Success Case

Evaluation Method
- A short survey of how and how much the learners are using the
objectives during training.
- Interviews a diverse number of learners that goes more in depth on:

Level 3 Behavior and behavioral changes that resulted from the

training. All of the behavioral changes can be from design, delivery,
instructor, and/or the learner.
If the training is
being used by
the learners

The training is
being used by the
learners because
they have to use it
to be effective
workers. The
learners behavior
changes because
the learners have
to use the training
to be able to work.

The training is
usually being used
by the learners
because they need
it to be a good
enough worker.
The learners
behavior changes
slightly because
the learners have
to use the training
to be able to work.

The training is not

being used by the
learners because
they are not
interested in the
work. The learners
behavior may
change, but it is
not a positive
change to assist
them with their
ability to perform
work duties.

How often the

training is being
used by the
learners

The learner is
using all of the
training during
their whole shift.
The learner cannot
be an effective
worker if they are
not using all of the
training. A new job
makes the learner
attentive to always
using their training
to make sure they
are doing
everything
correctly. The only
objective the
learner needs is

The learner is
using the training
often, but they skip
many parts to the
training during
their shift. The
learner does not
take all of the
training seriously
and there can be
many contributors.
The new job makes
the learner
attentive to a
certain extent on
doing everything
correctly. Feedback
is needed when the

The learner is not

using the training
during their shift.
The learner is not
an effective worker
because they are
not concentrated
on the training
while working. The
new job does not
give the learner
more motivation to
make sure how
often they are
using training.
Feedback needs to
be given
immediately to fix

PROGRAM EVALUATION RUBRICS

33

feedback.

learner is not using

the training often.

this issue.

Exactly how the

training is being
used

Training is being
used exactly by the
learner in
everything that
they do while they
are working. The
learner does each
part of their work
exactly like they
were trained. Each
position in the
work environment
is done exactly as
the training
instructed the
learner to be
successful with
their job.

The training is
being used by the
learner in their
work, but they are
not following
everything exactly
to the training. The
learner is doing all
of the parts of their
work, but they
alter some parts
because of the
issues that can
present. Each
position in the
work environment
is done almost like
the training
instructed the
learner to be good
with their job.

The training is not

being used by the
learner in their own
work. The learner
is not following any
of the training
exactly because of
factors that arise.
Each position in
the work
environment is not
done as the
training instructed
the learner to do.

Difference the
training made

There is an
exponential
difference in the
learner after the
training is over.
The learner can do
each position in
the work
environment, and
they are confident
and motivated by
the training they
received. The
training covered a
lot of specific
information, and
the difference in
the employee's
work shows
because they
wanted to learn.

There is a slight
difference in the
learner after the
training is over.
The learner can do
each position in
the work
environment, but
they are not as
confident and
motivated as they
could be by
training. The
training covered
specific
information to
make a difference,
but the employees
work does not
show a big
difference because

There is no
difference in the
learner after the
training is over.
The learner cannot
do any of the
positions at work
because they did
not learn what they
needed to during
the training. The
learners are not
motivated or
confident in their
work, which leads
to low morale and
can be contagious
to other workers.

PROGRAM EVALUATION RUBRICS

34

of the lacking
parts.
Adds value to
the organization
with proof of the
changes from
training

The learner adds

value to the
organization after
the successful
training. The
learners ability to
do everything
correctly and their
knowledge help
the customers and
the organization. If
the learner knows
how to do
everything, then
there is proof that
they are helping
the organization
because it is better
to have highly
trained employees.

The learner adds

some value to the
organization after
the training. The
learners ability to
do some of the
things correctly
and their partial
knowledge help
the customers and
the organization. If
the learner knows
how to do basics,
then there is proof
that they are
helping the
organization in
some ways
because they are
trained enough for
working on the
floor.

The learner does

not add value to
the organization
after the training.
The learners lack
of ability to do
everything
correctly and lack
of knowledge does
not help the
customers or the
organization. If the
learner does not
know how to do
the basics, then
there is no proof
that they are
helping the
organization
because they need
to be retrained.

Justify the
training by
showing
behavior results

The learner's
behavior justifies
the training
because the
learner has the
knowledge, skills,
and ability to do all
of the concepts
presented in the
training for the
work environment.
The behavior does
justify the training
and shows proof by
the learner, who
has changed their
aspect on training
and learning.

The learners
behavior justifies
the training
because the
learner has some
knowledge, skill,
and ability to do
some of the
concepts
presented in the
training for the
work environment.
The behavior does
not justify the
training as well as
it could.

The learners
behavior does not
justify the training
because the
learner does not
have the
knowledge, skills,
or ability to do the
concepts
presented in the
training for the
work environment.
The behavior
presented does not
justify the training
at all.

PROGRAM EVALUATION RUBRICS

35

PROGRAM EVALUATION RUBRICS

36

Level 4 Results Evaluation

Excellent

Moderate

Poor

Level 4: Results Evaluation shows the results that were attained

for the business from the training.
Operations:
Revenue per
Employee

The revenue per

employee depends
on how long the
employee has been
with the company,
their performance,
and their title at
work. Learner start
as baristas and
their revenue
would most likely
be minimum wage.
They could start off
with a little more if
they have an
impressive work
history or higher
title status at a
previous job. The
higher pay helps
employees, but it
can have an effect
on the
organization's
revenue.

The revenue per

employee depends
on their
performance,
which is not always
paid attention to,
or their title at
their work. All
learners start as
baristas, and their
revenue is always
minimum wage
unless they ask for
more and give a
good reason. The
wages helps the
employees and the
organization
because the
revenue for
employee varies
too much.

The revenue per

employee depends
on how much the
manager wants to
pay the workers.
All learners start as
baristas, and their
revenue is always
minimum wage.
The wages helps
the organization's
revenue but not
the employee's
revenue.

Time to Delivery

The learners can

positively affect
the time to
delivery because
they are still new
to the training and
knowledge.
Successful training
will help the store
have a wonderful

The learners can

affect the time to
delivery because
they are still new
to the training and
knowledge. Subpar
training will help
the store have an
average time to
delivery to the

The learners can

affect the time to
delivery negatively
because they are
still new to the
training and
knowledge.
Unsuccessful
training will not
help the store have

PROGRAM EVALUATION RUBRICS

37

time to delivery to
the customers
because the
learners are
confident and fast.
The learners know
what they are
doing, so time is
never an issue to
deal with because
of the training.

customers because
the learners are
not as confident
and motivated as
they could be after
training. The
learners know what
they are doing, but
they are hesitant
and the time can
vary because of
the training.

a good time to
delivery to the
customers. The
learners are not
confident or
motivated in the
work or timing. The
learner does not
know what they
are doing, so
timing is always an
issue to deal with
because of the lack
of training.

Returns

The products that

were not drinks
were always
bought and never
returned. The
customer was
knowledgeable
after the learner
fully explained the
product. The
customer knew
exactly what they
were buying, and
how it was what
they were looking
for to buy.

The products that

were not drinks
were bought and
sometimes
returned. The
customer has
imparted some
knowledge that the
learner had of the
product. The
customer had an
idea of what they
were buying, and
how it might be
what they were
looking for to buy.

The products that

were not drinks
were bought and
always returned.
The customer did
not receive any
knowledge from
the learner. The
learner did not
have knowledge of
the product, so the
customer was
confused on what
they were buying
when they left the
store. The product
was not what the
customer was
looking for, and
they end up
returning the
product.

Error and Error

Rates

The learners go
through the
training, and if
they are successful
in developing the
knowledge, skills,
and abilities for
work then the error

The learner go
through the
training, and they
do an average
level with
developing
knowledge, skills,
and abilities for

The learners go
through the
training, and they
do not do well with
developing their
knowledge, skill,
and abilities for
work. The error

PROGRAM EVALUATION RUBRICS

38

rates are minimal.

They are less likely
to make errors
because of their
training.

work, then the

error rates will be
more frequent. The
learners are more
likely to make
errors when they
did not get the
entire training.

rates are
constantly high
because they did
not retain the
training.

Number of
Correct Drinks

The learners make

all of the drinks
correctly at about
98% because of
their training. The
successful training
helps the flow of
correct drinks and
does not mess up
the flow because
they have to go
back to fix drinks.

The learners make

the drinks correctly
most of the time at
about 70%
because of their
training. The
average training
helps the flow of
drinks until every
drink they do not
make correctly
leaves the
customer waiting
longer. The
incorrect drinks
mess up the flow
because they have
to go back and fix
all of the drinks.

The learners make

the drink
incorrectly most of
the time at about
20% because of
their training.

Response Time
to Customers

Learners should
have learned about
greeting each
customer as they
walk in the door, so
the learners use
this for every
customer. The
learners should
immediately be
able to assist a
customer, and
multitask if they
are in the middle of
something. When

Learners should
have learned about
greeting each
customer as they
walk in the door,
but the learners
cannot always
achieve this for
every customer.
The learners get to
the customer in a
relatively timely
manner, but they
do not get to the
customers

Learners should
have learned about
greeting each
customer as they
walk in the door,
but the learners
cannot get to the
customers in time
or do not care. The
learners get to the
customers in an
untimely manner,
and they never get
to a customer
immediately. They

Productivity:

PROGRAM EVALUATION RUBRICS

39

the response time

is immediate, then
the customers get
an amazing feeling
walking into a
positive
environment.

immediately. The
quicker the learner
gets to responding
to the customer;
then the more
satisfied the
customer will be
towards the
organization.

do not seem to
care about their
reputation or the
organizations.

Timing on How
Long the Learner
Takes to Make
the Drink

The learner is
confident,
motivated,
experienced from
practice, and fast
at making drinks
after all of the
training.

The learner is
feeling nervous,
but they have the
ability from the
practice to make
drinks. The learner
is too careful, and
the drinks do not
get out to
customers very
fast.

The learner feels

helpless, lacking
motivation, and
lacking experience
from the training.
They are not fast
on their timing to
make a drink.

Adds Value to
the Organization
with Proof of the
Changes from
Training

The learner adds

value to the
organization after
the successful
training. The
learners ability to
do everything
correctly and their
knowledge help
the customers and
the organization. If
the learner knows
how to do
everything, then
there is proof that
they are helping
the organization
because it is better
to have highly
trained employees.

The learner adds

some value to the
organization after
the training. The
learners ability to
do some of the
things correctly
and their partial
knowledge help
the customers and
the organization. If
the learner knows
how to do basics,
then there is proof
that they are
helping the
organization in
some ways
because they are
trained enough for
working on the
floor.

The learner does

not add value to
the organization
after the training.
The learners lack
of ability to do
everything
correctly and lack
of knowledge does
not help the
customers or the
organization. If the
learner does not
know how to do
the basics, then
there is no proof
that they are
helping the
organization
because they need
to be retrained.

Justify the

The learner's

The learners

The learners

PROGRAM EVALUATION RUBRICS

Training by
Showing
Behavior Results

40

behavior justifies
the training
because the
learner has the
knowledge, skills,
and ability to do all
of the concepts
presented in the
training for the
work environment.
The behavior does
justify the training
and shows proof by
the learner, who
has changed their
aspect on training
and learning.

behavior justifies
the training
because the
learner has some
knowledge, skill,
and ability to do
some of the
concepts
presented in the
training for the
work environment.
The behavior does
not justify the
training as well as
it could.

behavior does not

justify the training
because the
learner does not
have the
knowledge, skills,
or ability to do the
concepts
presented in the
training for the
work environment.
The behavior
presented does not
justify the training
at all.

Employee
Retention and
Satisfaction

The learners stay

longer because
they are happy,
and the
management is
kind, respectful,
and helpful to the
learners with the
training.
Employees
retention is high
because they are
satisfied and have
fun with the job. A
positive
environment, polite
management, and
being
knowledgeable
keeps the
employees around
longer.

The learners are

likely to stay after
the training
because they like
the job and
management.
Employees
retention is
average because
the employees
start to lack
motivation when
they are not
challenged.
Employees get
bored with the job
especially when
the management
does not create an
inviting
environment.

The learners are

not likely to stay
after the training
because they do
not like the job and
management.
Employees
retention is low
because
employees are not
happy, and only
the management
can fix this
problem.

Budget Goals

The learner should

know about the
budget goals that

The learner should

know about the
budget goals that

The learner should

know about the
budget goals that

Management:

PROGRAM EVALUATION RUBRICS

Absenteeism,
Tardiness, and
Turnover

41

management has
during their
training. The
organization will
have lower labor
expense when
people are paying
attention to the
number of people
working; compared
to how much sales
are coming in
during that period
of the day. The
learners will
understand when
they have to be let
off work early, or if
they have to help
out during rushes.
The management
keeps the labor low
for their budget.

management has
during their
training. The
organization wants
the lower labor
expense, but
people do not
always pay
attention to the
numbers. The
workers do not
care when they
have slower times,
and learners may
be let off of work
early. The
management
wants to keep the
labor low for their
budget, but this is
not always
attainable when
they are not at the
store all of the
time.

management has
during their
training, but they
left out the budget
goal and how labor
is a big factor. The
learners and
workers do not pay
attention or
address high labor.
The management
does not address
high labor either,
so it does not help
their budget goal.

The learners are

dedicated to their
work, so they do
not call in, and
they are never
late. The
management pays
attention to these
numbers and
people so that they
can give positive
feedback to the
learners. When
learners and
employees want to
go to work on time
and do not want to
quit, then it has a
lot to contribute to

The learners are

interested in their
work, but
sometimes they
are late or call into
work. The turnover
can be a problem
when learners are
not motivated. The
management does
not offer as much
feedback or
positive
environment.

The learners show

no interest in their
work, so they are
late and call in to
work consistently.
The turnover rate
is bad because the
management does
not take care of
their employees or
look at them as
assets to their
organization.
Management does
not offer feedback,
and they are
usually negative at
work.

PROGRAM EVALUATION RUBRICS

42

the management
and their positive
environment.

Financial:
Operating
Income Growth

The gross income

is growing, and the
operating
expenses are
declining. The
operating income
growth helps the
company expand,
come out with new
products, or
update something
in their stores. The
employees can
have a lot to do
within the gross
income growing,
and that is why
companies keep
loyal, motivated,
and dedicated
employees.

The gross income

is growing, or the
operating
expenses are
declining, but it is
not both situations
at once. The
operating income
growth can help
with changes, but
this can be
inconsistent with
how employees are
doing throughout
the company. The
organization has to
find ways to keep
their employees
happy to keep up
operating income.

The gross income

may be growing, or
the operating
expenses are
declining. The
operating income
growth does not
help the company
a lot because the
growth is due to
new employees.
The employees
who have been
with the company
for a while are not
motivated or
dedicated because
of many factors.
The operating
growth will
fluctuate because
of employees.

Budget Variance

The budget
variance can be
positive because
the budget has
extra money than
the actual
calculated amount.
The management
and employees can
help by going
under the budget
that their store has
that year.

The budget
variance can be
about even
because the
budgeted money
and the actual
calculated amount
are about even.
The management
and employees can
positively or
negatively affect
the budget
variance. It all
depends on how
well they do with
their budget for the

The budget
variance can be
negative because
the budget has less
money than the
actual calculated
amount. The
management and
employees can
affect the budget
variance because if
they go over their
budget, it will
affect the variance
negatively.

PROGRAM EVALUATION RUBRICS

43

year at their store.

Inventory
Turnover

The store can have

a high inventory
turnover because
their sales were
higher than usual.
The store needs to
stock up on
inventory when
they have a high
inventory turnover.
Employees have a
lot to do with
increasing sales,
and management
will keep up with
inventory to
support the sales.

The store can have

an even inventory
turnover because
their sales are not
higher or lower
than usual. The
store does not
need to stock up
on inventory
because they have
a turnover it will be
minimal.
Employees can
increase or
decrease the
inventory turnover
with their sales.

The store can have

a low inventory
turnover because
their sales were
lower than usual.
The store does not
need to stock up
on inventory
because they have
enough if sales are
down. Employees
have a lot to do
with decreasing
sales, and
management will
have to watch the
inventory carefully.

Customer
Acquisition

The employees
have proficient
customer service
skills because they
are trying to bring
in a new regular
customer. If the
customer looks
unfamiliar, then
the employee
should cater the
individual, so the
customer leaves
with a drink or
product that they
love. The
customers repeat
visits will show that
the customer
service provided by
the employee
acquired or gained
a new customer.

The employees
have limited
customer service
skills, but they are
still trying to bring
in new regular
customers. The
employee will help
the customer with
what they ask for,
but they do not go
above and beyond
to make sure to
keep the customer
coming back to
their store.

The employees
have no customer
service skills, and
they are not trying
to bring in new
regular customers.
The employee will
treat the new
customer like any
other person with
bad customer
service skills,
which ensure the
customer will not
want to repeat
their service.

Customer

The customer is

The customer is

The customer is

Customer:

PROGRAM EVALUATION RUBRICS

44

Profitability

helpful with the

revenue of the
store because each
customer spends
about the same
amount of money
each time they
come in the store.
Excellent customer
service makes a
normal customer,
who normally buys
one drink, but they
start buying bags
of coffee beans or
pastries. Many
customers will
increase their drink
order because they
are picking up
coffee for others as
well. The customer
service helps with
customer
profitability.

sometimes helpful
with the revenue of
the store because
each customer
spends around the
same amount.
Regular customer
service will be
helpful sometimes,
where a regular
customer buys
something extra
every once in a
while. The
customer service
helps a little with
the customer
profitability.

not helpful with the

revenue of the
store because each
customer spends
different amounts
and does not
always come in
regularly. Bad
customer service
will make a regular
customer into a
customer that does
not come in as
often because of
fear of that
employee and
experience. Bad
customer service
does not help with
customer
profitability.

Customer
Retention

Customers like to
have a routine,
especially if they
are regular coffee
drinkers, so
customer service is
important in
retaining
customers.
Employees
practicing good
customer service
and becoming
friends, in a way,
helps retain
customers.

Customers like to
have a routine,
especially if they
are regular coffee
drinkers, so
customer service is
important in trying
to retain some
customers.
Employees try with
their customer
service, but it does
not always retain
customers because
they get curious
about other coffee
shops. When
customer service is
exceptional, the

Customers like to
have a routine,
especially if they
are regular coffee
drinkers, so
customer service is
why the store
cannot retain
customers.
Employees practice
bad customer
service, so
customers wander
around coffee
shops until they
find a better place
to be a regular
customer. The
store does not

PROGRAM EVALUATION RUBRICS

Customer
Satisfaction

45

customers do not
wander away from
being a customer.

retain regular
customers because
of their negative
customer service
skills.

Employees make
sure that the
customer is
satisfied before
they leave the
store. If they are
not happy with
their drink, then
the employee
makes sure it is
perfect for them or
offers a free drink.
The employees
always practice
impressive
customer service,
and no customer
leaves without a
smile.

Employees try to
make sure the
customer is
content when they
leave the store. If
the customer is
unhappy with a
drink, then the
employee will
remake the drink,
but they are not as
apologetic about
the situation
because their
customer service
skills are lacking.
The customer
leaves bland
because of the
customer service
skills.

Employees do not
try to make sure
the customer is
happy before they
leave the store.
The employees are
not concerned if an
employee
complains about a
drink. They ignore
the situation,
argue, or just give
out a free drink
coupon. The bad
customer service
skills do not make
the customer
satisfied at all.

The learners are

taught how to
resolve problems in
the work
environment or
between each
other during
training. The
employees fix the
issues quickly
without negativity,
and everyone goes
home happy.

The learners are

taught how to
resolve problems in
the work
environment, but
they do not always
follow the training.
People are ruled by
their emotions, and
they do not know
how to fix issues in
a calm, positive
manner
sometimes. The
issues are
resolved, but they

The learners are

taught how to
resolve problems,
but they do not
follow the training
on resolving
problems.
Everyone in the
work environment
always thinks they
are right, so
nothing ever gets
resolved, and this
brings a negative
environment until a
resolution, or it is

Employees:
Problem
Resolution

PROGRAM EVALUATION RUBRICS

46

are not always

resolved in the
best manner.

forgotten.

Employee
Satisfaction

Employees are
satisfied with their
job and the people
that they work with
each day. The
learners are
satisfied with the
training,
coworkers, and
management.

Employees are
content with their
job and the people
that they work with
each day. The
learners are
content with the
training and
management, but
they are not
satisfied with
everything.

Employees are not

satisfied with their
job or the people
that they work with
each day. The
learners are not
satisfied with the
training,
coworkers, and
management.

Overtime

The learners and

employees will
have the overtime
if they make a
request to the
management. The
management
always tries to
accommodate all
requests. The
employees may
have to work
overtime during a
week, but the
manager lets the
employee know,
and usually throws
in an incentive
(tangible or
intangible). The
employees go into
overtime with a
positive attitude.

The learners and

employees may
have overtime if
they make a
request to the
management. The
management will
sometimes try to
help out with
overtime, but
sometimes they
are too busy, or it
is not in the
budget. The
employee may be
scheduled
overtime, but the
manager does not
always remember
to warn
employees, and
make sure that
they are
comfortable with
the overtime and
offer no incentive.

The learners and

employees may
not have overtime
if they make a
request to the
management. The
management does
not allow overtime
unless it is an
emergency
because of labor
and budget goals.
The employee may
be scheduled
overtime, but the
manager does not
discuss the change
to the employees.

Efficiency and
Productivity

The learners and

employees are

The learners and

employees are

The learners and

employees are not

PROGRAM EVALUATION RUBRICS

efficient and
productive
together because
the learners are
trying to be as
good as the older
employees. After
the training, the
employees will
help out with
motivation.
Contests on
quickness,
correctness, and
looks are practiced
to increase
efficiency and
productivity.

47

efficient and
productive
together to a
certain extent.
After training the
employees to help
learners if they
have a question,
but they do not
help as much with
motivation.

efficient and
productive
together because
they lack concern
on these topics.
Work is work to
them, so they do
not work together
to be more efficient
and productive.

Marketing and Sales:

Revenue per
Employee

All of the revenue

each employee
brings into the
organization with
marketing and
sales helps. The
employees are
excited and want
to help with sales,
and they help
learners with more
knowledge about
how to sell the
products. The
employees
dedication to each
bring in revenue
with sales helps
the budget and
store.

The revenue each

employee brings
into the
organization with
marketing and
sales is minimal,
but it helps. The
employees try a
little to gain
revenue for the
organization, but
they are not as
interested in
babysitting
learners. The
budget and the
store revenue do
not affect the
employees as
much, so they do
not always see the
point.

The revenue each

employee brings
into the
organization with
marketing and
sales is null. The
employees do not
try marketing and
sales because they
do not get
anything out of it.
The lack of
motivation and
encouragement
does not help the
employees in this
area.

Market Share

The market share

is improved

The market share

is improved a little

The market share

is not improved

PROGRAM EVALUATION RUBRICS

Improved
Delivery

48

because of
marketing and
sales in the whole
organization. The
learners and
employees
increase the
market share by
getting creative
with new
marketing ideas.
The employees
increase the sales
by offering
something as small
as paying for an
extra shot in the
customers coffee.
The employees
knowledge of
different coffee
beans and
products available
will help sales. The
market share is
increased when all
of the employees
are working
together in
marketing and
sales.

because of
marketing and
sales in the whole
organization. The
learners and
employees
sometimes come
up with marketing
ideas, but it is not
a goal unless there
is an incentive. The
employees try to
increase sales, but
some employees
are just not good at
selling more than
what the customer
came in to buy. The
sales can affect the
market share of a
company.

because of
marketing and
sales in the whole
organization. The
learners and
employees do not
involve themselves
with marketing and
extra sales. The
employees need an
incentive to try to
bring up the
companys market
share. The
negative attitude
will affect the
companys market
share in a
dramatically
unfavorable way.

The learners and

employees have
improved delivery
with marketing and
sales. The
management,
training, work
environment,
practice, and
contests help them
work quicker
together. Everyone
takes a turn with

The learners and

employees have
improved delivery
with marketing and
sales minimally.
The employees and
learners are not
always interesting
in marketing and
sales, so improving
delivery does not
always happen.

The learners and

employees have
not improved
delivery with
marketing and
sales. The
employees and
learners do not see
the point in making
marketing and
sales quicker, so
the delivery is
never improved.

PROGRAM EVALUATION RUBRICS

learning or
teaching each
other because
everyone has
something to learn.

49

PROGRAM EVALUATION RUBRICS

50

References
Coscarelli, W., & Shrock, S. (2008). Chapter 29: Level 2: Learning-Five Essential
Steps for Creating Your Tests and Two Cautionary Tales ASTD handbook for
workplace learning professionals. Alexandria, VA: ASTD Press.