Академический Документы
Профессиональный Документы
Культура Документы
Table of Contents
Introduction.......................................................................................... 3
Alpha Test............................................................................................... 3
Beta Test.................................................................................................. 5
Assessment Questions...................................................................... 7
Evaluation.............................................................................................. 9
Evaluation Results.............................................................................15
INTRODUCTION
This Formative Evaluation report details the process by which the Band Orientation Program for Almost Elton
John and the Rock Band was evaluated and tested. This review includes data on a the Alpha and Beta tests
for the Program, detailing the Objectives of the tests, the audience, the criteria, the process, and results and
feedback.
ALPHA TEST
Objective
The objective of the Alpha test was to test basic functionality of the course and the medium of the course.
This included testing for usability of the slides, continuity of content, navigation, audio and video quality,
and other technical issues.
Audience
Alpha testers were comprised of:
A Subject Matter Expert (SME)
An instructional design expert
Three instructional design graduate students
Criteria
All Alpha testers were asked to evaluate the program for clear statement of objectives and instructional
goals, content quality and accuracy, content continuity, relevance, and scope. The testers were asked to provide feedback in the form of a written report.
Process
The Alpha testers were instructed to take the program by following a weblink provided by email. The testers
were provided with a password to access the program, and informed that running time for the program was
approximately 1 hour.
3
BETA TEST
Objective
The objective of the Beta test was to further check for functionality of the course, as well as to assess aspects
of the program related to subject matter, auxiliary information, interface, navigation, and pedagogy.
Audience
The Beta testers were comprised of:
Three Instructional design graduate students
Two Current members of the Rocket Band (who are the intended audience for the program)
Procedures
Beta testers were instructed to go through the entire program. On the final slide, the testers were to follow a
hyperlink to the Google Form. The survey asked the testers to rate the program on a scale of 1 to 5 in terms
of its level of effectiveness in a number of areas (subject matter, auxiliary information, interface, navigation,
and pedagogy). For each area, the testers was given the opportunity to give feedback in the form of written
comments.
Results Feedback
Most of the feedback indicated that the testers found the program engaging, informative, functional, aesthetically attractive, and entertaining. Testers indicated that the program appeared to be an interesting and
unique way to understand the subject matter and to learn the various learning objectives.
Criticisms of the program included largely technical matters, such as slide duration, design anomolies, and
issues with the assessment questions. There were also comments about the Google Form Evaluation Survey
itself, which had two of the ratings headings switched around. Overall, the feedback again was very positive.
Comment Feedback on Subject Matter
I am not really an expert on this subject. -intro slide was a bit quick -code of conduct video bits...there was
a ghostly image and slide bar in the middle...a remnant of something that had been removed? -assessment
section, contracts...reresponsibility instead of responsibility. I learned a lot from the basic info in your presentation. Since Im not a musician I cant reliably comment on tech jargon. I think language and layout and
design fit your target audience well. -did u switch places on the good/acceptable scale? from first set of?s
to next sets of ?s? i consider good better than acceptable on the poor to excellent scale... It was absolutely
great. The interviews were informative but I felt a sense of wanting them to be expedited.
Comment Feedback on Auxiliary Information
i didnt check out the links but they looked to be related to the subject matter. the poor/needs work/good/
acceptable/excellent arrangement still bothers me...think good/acceptable should be reversed, (isnt good
better than acceptable?) but thats just me. I only put needs work because I am unsure where your help features were. Is cc considered a help feature, is the toc considered a help feature?
Comment Feedback on Interface
N/A
Comment Feedback on Navigation
i did not try to bookmark anything or restart the training and just went straight thru.... but i loved the simple
straightforward navigation bar you used. it looked good and was easy to use.
Comment Feedback on Pedagogy
If I was a musician, hired to tour with the band, this module would give me a good foundation to function
appropriately in the band. Good tips and clear goals. Excellent use of photography without being distracting. Very professional layout. I answered the quiz at the end and it said that I failed showing that I didnt even
get one answer correct - I am sure that it was an error. Not sure why it would not let me start again. I did not
hit submit after the last question because I did not see it clearly. I suppose that this is why I FAILED. GREAT
presentation - makes me very proud to be a member of the Rocket Band!!!
ASSESSMENT QUESTIONS
Please answer the following questions. (The correct answers are in red)
1. The mission statement of Almost Elton John and the Rocket Band is:
A. Be the best Elton John Tribute in the United States
B. Celebrate the songs and career of Elton John
C. Help introduce people to music of the 1970s
D. Play with symphony orchestras
2. Members of the Rocket Band:
A. Are all hired guns and are there to just play another gig
B. Have equal billing and prominence in the Almost Elton John show
C. Arent very sociable because they work so hard
D. Are like a family
3. Which of the following are acceptable behaviors for members of the band? Please select all that apply.
Get to the hotel and spend quiet time alone.
Show up at least 10 minutes early to all band engagements.
Its okay to be difficult to deal with, as long as youre a really good musician.
Show up to the performance in blue jeans and make faces at the audience.
Ask the band leader if you can have a solo number in the show.
Ask the band leader how you can help improve a certain part of the show.
Make silly jokes during a van ride.
Pass out personal business cards during the intermission at a performance.
While in flight to engagement, invite other people to a concert.
Demand that you have your own hotel room because you have 20 years of experience in the industry.
You have to miss a rehearsal, but you called two weeks ahead of time to let the band leader know.
You have to miss a concert that you are under contract to perform because a better paying offer came through, but
you called two weeks ahead of time to let the band leader know.
4. True or False:
Craig views his band as a group of highly-trained professionals who are in the group to simply do a job, nothing more.
5. Complete the following sentence:
Craig is interested in creating ____________________ in the lives of those around him.
A. Careers
B. Opportunities
C. Purpose and Meaning
D. Cheesecake
6. Contracts are:
A. Important because they obligate you to certain responsibilities with the band
B. Important because all bands have contracts
C. Negotiable and can be broken at any time with no fear of recompense or termination
D. Optional
7. During air travel, it is recommended that you:
A. Meet the band at the flight gate
7
EVALUATION
A live form of the Evaluation Survey can be found here.
10
11
12
13
14
https://docs.google.com/forms/d/1QoocXALhVbPTbhbz1me4...
EVALUATION RESULTS
Pat Strawser
5 responses
Publish analytics
Summary
Learning objectives were clear [1 SUBJECT MATTER]
Poor
0%
Needs Work
0%
Acceptable
0%
Good
0%
Excellent
100%
1 of 20
4/23/14 7:20 PM
15
0%
Needs Work
0%
Acceptable
0%
Good
40%
Excellent
60%
https://docs.google.com/forms/d/1QoocXALhVbPTbhbz1me4...
Poor
0%
Needs Work
0%
Acceptable
0%
Good
20%
Excellent
80%
Poor
0%
Needs Work
0%
2 of 20
4/23/14 7:20 PM
16
0%
Good
0%
Excellent
100%
https://docs.google.com/forms/d/1QoocXALhVbPTbhbz1me4...
Poor
0%
Needs Work
0%
Acceptable
0%
Good
0%
Excellent
100%
Poor
0%
Needs Work
0%
Acceptable
0%
Good
20%
3 of 20
4/23/14 7:20 PM
17
https://docs.google.com/forms/d/1QoocXALhVbPTbhbz1me4...
80%
Poor
0%
Needs Work
0%
Acceptable
0%
Good
20%
Excellent
80%
Poor
0%
Needs Work
0%
Acceptable
0%
Good
20%
Excellent
80%
4 of 20
4/23/14 7:20 PM
18
https://docs.google.com/forms/d/1QoocXALhVbPTbhbz1me4...
Poor
0%
Needs Work
0%
Acceptable
0%
Good
25%
Excellent
75%
video bits...there was a ghostly image and slide bar in the middle...a remnant of something
that had been removed? -assessment section, contracts..."reresponsibility" instead of
responsibility. I learned a lot from the basic info in your presentation. Since I'm not a
musician I can't reliably comment on tech jargon. I think language and layout and design fit
your target audience well. -did u switch places on the good/acceptable scale? from first set
of?s to next sets of ?s? i consider good better than acceptable on the poor to excellent
scale...
It was absolutely great. The interviews were informative but I felt a sense of
5 of 20
4/23/14 7:20 PM
19
Poor
0%
Needs Work
0%
Good
0%
Acceptable
0%
Excellent
100%
https://docs.google.com/forms/d/1QoocXALhVbPTbhbz1me4...
Poor
0%
Needs Work
0%
Good
20%
Acceptable
0%
Excellent
80%
6 of 20
4/23/14 7:20 PM
20
Poor
0%
Needs Work
20%
Good
20%
Acceptable
0%
Excellent
60%
https://docs.google.com/forms/d/1QoocXALhVbPTbhbz1me4...
Poor
0%
Needs Work
0%
Good
20%
Acceptable
0%
Excellent
80%
7 of 20
4/23/14 7:20 PM
21
https://docs.google.com/forms/d/1QoocXALhVbPTbhbz1me4...
I only put needs work because I am unsure where your help features were. Is cc
Poor
0%
Needs Work
0%
Good
0%
Acceptable
0%
Excellent
100%
Presentation modes are appropriate and engaging (i.e. video clips, audio
clips, etc.) [3 INTERFACE]
Poor
0%
Needs Work
0%
Good
20%
8 of 20
4/23/14 7:20 PM
22
20%
Excellent
60%
https://docs.google.com/forms/d/1QoocXALhVbPTbhbz1me4...
Poor
0%
Needs Work
0%
Good
0%
Acceptable
0%
Excellent
100%
Poor
0%
Needs Work
0%
Good
0%
Acceptable
0%
9 of 20
4/23/14 7:20 PM
23
https://docs.google.com/forms/d/1QoocXALhVbPTbhbz1me4...
100%
Poor
0%
Needs Work
0%
Good
0%
Acceptable
0%
Excellent
100%
Poor
0%
Needs Work
0%
Good
20%
Acceptable
0%
Excellent
80%
10 of 20
4/23/14 7:20 PM
24
https://docs.google.com/forms/d/1QoocXALhVbPTbhbz1me4...
Poor
0%
Needs Work
0%
Good
20%
Acceptable
0%
Excellent
80%
Interface Comments
It is easy to navigate the slides within the training [4 NAVIGATION]
Poor
0%
Needs Work
0%
Good
0%
Acceptable
0%
Excellent
100%
11 of 20
4/23/14 7:20 PM
25
https://docs.google.com/forms/d/1QoocXALhVbPTbhbz1me4...
Poor
0%
Needs Work
0%
Good
0%
Acceptable
0%
Excellent
100%
Poor
0%
Needs Work
0%
Good
0%
Acceptable
40%
Excellent
60%
12 of 20
4/23/14 7:20 PM
26
Poor
0%
Needs Work
0%
Good
0%
Acceptable
20%
Excellent
80%
https://docs.google.com/forms/d/1QoocXALhVbPTbhbz1me4...
Navigation Comments
i did not try to bookmark anything or restart the training and just went straight thru.... but i
loved the simple straightforward navigation bar you used. it looked good and was easy to
use.
The methods used to present the content are appropriate for the
audience [5 PEDAGOGY]
Poor
0%
Needs Work
0%
Good
0%
Acceptable
0%
13 of 20
4/23/14 7:20 PM
27
https://docs.google.com/forms/d/1QoocXALhVbPTbhbz1me4...
100%
Poor
0%
Needs Work
0%
Good
20%
Acceptable
20%
Excellent
60%
Poor
0%
Needs Work
0%
Good
20%
Acceptable
0%
Excellent
80%
14 of 20
4/23/14 7:20 PM
28
https://docs.google.com/forms/d/1QoocXALhVbPTbhbz1me4...
Poor
0%
Needs Work
0%
Good
20%
Acceptable
0%
Excellent
80%
The program gives user sufficient control (i.e. volume, pause, continue,
table of contents, etc.) [5 PEDAGOGY]
Poor
0%
Needs Work
0%
Good
0%
Acceptable
20%
Excellent
80%
4/23/14 7:20 PM
29
https://docs.google.com/forms/d/1QoocXALhVbPTbhbz1me4...
PEDAGOGY]
Poor
0%
Needs Work
0%
Good
0%
Acceptable
0%
Excellent
100%
Poor
0%
Needs Work
0%
Good
20%
Acceptable
0%
Excellent
80%
4/23/14 7:20 PM
30
Poor
0%
Needs Work
0%
Good
0%
Acceptable
0%
Excellent
100%
https://docs.google.com/forms/d/1QoocXALhVbPTbhbz1me4...
Poor
0%
Needs Work
0%
Good
20%
Acceptable
20%
Excellent
60%
17 of 20
4/23/14 7:20 PM
31
Poor
0%
Needs Work
0%
Good
20%
Acceptable
20%
Excellent
60%
https://docs.google.com/forms/d/1QoocXALhVbPTbhbz1me4...
Poor
0%
Needs Work
0%
Good
0%
Acceptable
20%
Excellent
80%
18 of 20
4/23/14 7:20 PM
32
Poor
0%
Needs Work
0%
Good
0%
Acceptable
0%
Excellent
100%
https://docs.google.com/forms/d/1QoocXALhVbPTbhbz1me4...
Poor
0%
Needs Work
0%
Good
0%
Acceptable
20%
Excellent
80%
Pedagogy Comments
If I was a musician, hired to tour with the band, this module would give me a good
19 of 20
4/23/14 7:20 PM
33
https://docs.google.com/forms/d/1QoocXALhVbPTbhbz1me4...
foundation to function appropriately in the band. Good tips and clear goals. Excellent use of
photography without being distracting. Very professional layout.
end and it said that I failed showing that I didn't even get one answer correct - I am sure that
it was an error. Not sure why it would not let me start again. I did not hit "submit" after the
last question because I did not see it clearly. I suppose that this is why I FAILED. GREAT
presentation - makes me very proud to be a member of the Rocket Band!!!
20 of 20
4/23/14 7:20 PM
34