EDIT6200 Formative Evaluation Report

Client: My Father’s Eyes Ministry of Berean Christian Church Gwinnett

Instructional Designer: LaRhonda Ware

Alpha Test
April 26 & 27

Introduction

The target audience of the alpha test is made up of ministry coordinators, SMEs and a web designer . The focus
of the alpha test was the design layout of the product, the ease of navigation, usability and content accuracy. I
requested that five reviewers test my product. Only four responded to the survey. Because there was no way
for me to identify who had not completed the survey, I left all of the participants listed even though one
participant did not offer any feedback.

Reviewer Role Age Gender Occupation

ProfilesParticipant

Participant 1 Design Expert 43 F IDD M.Ed. Student

Participant 2 Ministry Coordinator 52 M IDD M.Ed. Student

Participant 3 Ministry Coordinator 39 M IDD M.Ed. Student

Participant 4 SME 50 M IDD M.Ed Student

Participant 5 SME 51 M Writer/Researcher

Question 1*

User is able to quickly determine the basic content of the site.

 1 2 3 4 5 Responses Total

0% 0% 0% 0% 100% 4 20

Question 2*

User is able to determine the intended audience of the site.

 1 2 3 4 5 Responses Total

0% 0% 0% 50% 50% 4 18

ID View Survey

Question 3*

The site appears organized on the first glance

  1 2 3 4 5 Responses Total

0% 0% 0% 0% 100% 4 20

Question 4*

The homepage downloads efficiently

 1 2 3 4 5 Responses Total

0% 0% 25% 25% 50% 4 17

ID View Survey

1662483 too many warnings pop up View

1666483 some lessons take a long time to upload View

Question 5*

The speed of the site is appropriate for this type of program

 1 2 3 4 5 Responses Total

0% 0% 25% 25% 50% 4 17

ID View Survey

1662483 slow at times- especially after you hit submit View

Question 6*

The homepage is attractive, has strong eye appeal.

 

 1 2 3 4 5 Responses Total

0% 0% 0% 75% 25% 4 17

ID View Survey

1666434 There is dead space on most pages next to the nav bar View
Question 7*

 You can tell where you are immediately (clear title, description, image captions, etc.)
 

 1 2 3 4 5 Responses Total

0% 0% 0% 0% 100% 4 20

Question 8*

There is an index, table of contents, or some other clear indicator of the contents of the site.
 

 1 2 3 4 5 Responses Total

0% 0% 0% 0% 100% 4 20

Question 9*

Site sponsor/provider is clearly identified.

 

 1 2 3 4 5 Responses Total

0% 0% 25% 0% 75% 4 18

Question 10*

There is sufficient information to make the site worth visiting.

 1 2 3 4 5 Responses Total

0% 0% 25% 25% 50% 4 17

Question 11*

The information is clearly labeled and organized.

 
  1 2 3 4 5 Responses Total

0% 0% 0% 0% 100% 4 20

Question 12*

The same basic format is used consistently throughout site.

 1 2 3 4 5 Responses Total

0% 0% 0% 0% 100% 4 20

Question 13*

Information is easy to find (no more than three clicks, for example).
 

 1 2 3 4 5 Responses Total

0% 0% 0% 0% 100% 4 20

Question 14*

Lists of links are well organized and easy to use.

 1 2 3 4 5 Responses Total

0% 25% 0% 25% 50% 4 16

ID View Survey

1662483 There aren't any outside links View

Question 15

Ministry Coordinators - There are links to other sites that are related to my needs/purposes.
 

 1 2 3 4 5 Responses Total
 50% 0% 0% 0% 50% 2 6

ID View Survey

1662483 no outside links View

Question 16

Ministry Coordinators- The content of the site is worthwhile and appropriate to my needs/purposes.
 

 1 2 3 4 5 Responses Total

0% 0% 0% 0% 100% 2 10

Question 17

Ministry Coordinators-  the overall presentation suitable to represent our ministry

 1 2 3 4 5 Responses Total

0% 0% 0% 0% 100% 2 10

Question 18

SME- The content achieves this intended purpose effectively.

 

 1 2 3 4 5 Responses Total

0% 0% 0% 50% 50% 2 9

Question 19

SME- The content of this site is well organized.

 

 1 2 3 4 5 Responses Total

0% 0% 0% 0% 100% 2 10

Question 20

SME-  The content on the site is accurate

  1 2 3 4 5 Responses Total

0% 0% 0% 0% 100% 2 10

Question 21

SME- The information is sequenced properly.

 1 2 3 4 5 Responses Total

0% 0% 0% 0% 100% 2 10

Question 22

Design expert-  User is able to move around within the site with ease.
 

 1 2 3 4 5 Responses Total

0% 0% 50% 50% 0% 2 7

ID 100 character max View Survey

166643 some links do not work, but you can still move around using the dropdown menu View
4

Question 23

Design expert- Directions for using the site are provided if necessary.

 

 1 2 3 4 5 Responses Total

0% 0% 0% 0% 100% 1 5

ID 100 character max View Survey

1662483 It is pretty intuitive View

Question 24

Design expert- Directions are clear and easy to follow.

 
  1 2 3 4 5 Responses Total

0% 0% 0% 0% 100% 2 10

Question 25

Design expert- Each graphic, audio file, video file, etc., serves a clear purpose.
 

 1 2 3 4 5 Responses Total

0% 0% 0% 0% 100% 1 5

ID 100 character max View Survey

1662483 Would like to see more pictures of the techniques being used in the curch View

Question 26

Design expert- The graphics, animations, sounds clips, etc., make a significant contribution to the site.

 1 2 3 4 5 Responses Total

0% 0% 0% 50% 50% 2 9

Question 27

Design expert- Site is equally effective with a variety of browsers such as Netscape and Internet Explorer.
 

 1 2 3 4 5 Responses Total

0% 0% 0% 50% 50% 2 9

Question 28*

The site is engaging and held your interest

 1 2 3 4 5 Responses Total

0% 0% 25% 50% 25% 4 16

ID View Survey

1662483 Sometimes it was a bit wordy View

 1666434 content is pretty basic View

1666483 some lessons are a little wordy View

Question 29*

The quizzes helped me remember the information

 1 2 3 4 5 Responses Total

0% 0% 0% 75% 25% 4 17

ID View Survey

1666434 questions could be impoved upon- They seem to be filler questions View

Question 30*

I will use this tool if I need to review the information containd within.

 1 2 3 4 5 Responses Total

0% 0% 0% 75% 25% 4 17

Question 31*

What would you say is are the strengths of this training module?

Type your response in the textbox below. 

 

Text Answers (4) View

it is easy to use View

organized attractive easy to use View

intuitive interactive elements make it interesting to use expandable colors View

complement one another organized

It is clearly organized and attractive. The pull down menus allow you to go to View
another unit whenever you like. You are not trapped in a lesson until it is
completed.

Question 32*
Please specify the areas of this program that need to be improved because they are ineffective or irrelevant. 

Type your answer in the textbox below

Text Answers (4) View

Some links are slow - see the ones that connect you to the next lesson after View
finishing a quiz

not finished View

content needs to be better developed Only one quiz has the option to send the View
scores to the instructor.

Some links do not automatically send you to the next lesson. I had to go to the View
pull down menu to go to the next lesson.

Question 33*

Please list any topics you believe should be included  in this module

Please type your answer in the textbox below. 

500 word max

Text Answers (4) View

shots, angles, transitions, director's language a gllossary of terms View

filming roles and expectations types of shots View

There should be some indication that the program is ending. as is there is no View
closure.

There are several. camera angles, theories af photography. roles View

Results:
The overall results of the Alpha test were favorable and informative. While all of the reviewers thought the program was
easy to use and organized several were not pleased with the display time of some of the lessons. The area that seemed to
need the most revision is the wording of several lessons and the level of difficulty. This will be addressed with the lessons
that will be developed later in the module.

Revisions:
1. Glossary will be added to the page and specific words and terms will be linked to the glossary
2. Additional lessons and units will be added to cover the breadth of information required for Part II of the training program.
3. All links between lessons and activitities will be assessed and edited to reduce the wait time between events.
4. An introduction to Part II will be included to provide closure to this program and offer direction to the trainees of their
next steps.

Beta Test
April 27-30

Introduction

The target audience of this project is the next set of volunteers to the ministry. The focus of the beta test was
the look of the product, ease of navigation, and usability. I requested that five reviewers test my product. Only
three responded to the survey. Because there was no way for me to identify who had not completed the survey,
I left all of the participants listed even though two participants did not offer any feedback.

Reviewer ProfilesParticipant Age Gender

Participant 1 20 F

Participant 2 41 F

Participant 3 24 M

Participant 4 27 M

Question 1*

I am able to quickly determine the basic purpose of the site.

 1 2 3 4 5 Responses Total

0% 0% 0% 0% 100% 3 15

Question 2*

The site appears organized on the first glance

 1 2 3 4 5 Responses Total

0% 0% 0% 0% 100% 3 15

Question 3*

The time that it takes for the pages to load is adequate. 

 1 2 3 4 5 Responses Total

0% 0% 0% 33% 67% 3 14

Question 4*

The homepage is attractive, has strong eye appeal.

 

 1 2 3 4 5 Responses Total

0% 0% 0% 0% 100% 3 15

Question 5*
 You can tell where you are immediately (clear title, description, image captions, etc.)
 

 1 2 3 4 5 Responses Total

0% 0% 0% 0% 100% 3 15

Question 6*

There is an index, table of contents, or some other clear indicator of the contents of the site.
 

 1 2 3 4 5 Responses Total

0% 0% 0% 0% 100% 3 15

Question 7*

Site sponsor/provider is clearly identified.

 

 1 2 3 4 5 Responses Total

0% 0% 33% 0% 67% 3 13

Question 8*

There is sufficient information to make the site worth visiting.

 1 2 3 4 5 Responses Total

0% 0% 33% 67% 0% 3 11

ID View Survey

1666510 I al ready knew this stuff View

Question 9*

The same basic format is used consistently throughout site.

 1 2 3 4 5 Responses Total

0% 0% 0% 0% 100% 3 15

Question 10*
Information is easy to find (no more than three clicks, for example).
 

 1 2 3 4 5 Responses Total

0% 0% 0% 0% 100% 3 15

Question 11*

The audio components and video components are interesting

 1 2 3 4 5 Responses Total

0% 33% 0% 67% 0% 3 10

ID View Survey

1662593 some sections are too loud. I had to adjust my volume from lesson to lesson View

1666499 too loud at times View

Question 12*

I really feel like I am learning something

 1 2 3 4 5 Responses Total

0% 0% 33% 33% 33% 3 12

Question 13*

 It is easy for me to remember information I learned in earlier lessons.

 1 2 3 4 5 Responses Total

0% 0% 33% 67% 0% 3 11

Question 14*

 I feel prepared to begin working with the actual cameras as a result of completing these lessons.

 1 2 3 4 5 Responses Total

0% 0% 33% 33% 33% 3 12

Question 15*

The site is engaging and held your interest

  1 2 3 4 5 Responses Total

0% 0% 0% 67% 33% 3 13

Question 16*

The quizzes helped me remember the information

 1 2 3 4 5 Responses Total

0% 0% 33% 67% 0% 3 11

Question 17*

I will use this tool if I need to review the information containd within.

 1 2 3 4 5 Responses Total

0% 0% 33% 0% 67% 3 13

Question 18*

What would you say is are the strengths of this training module?

Type your response in the textbox below. 

 

Text Answers (3) View

easy to use View

easy to use I can complete this at home I can stop and start it again when I want View

Question 19*

Please specify the areas of this program that need to be improved because they are ineffective or irrelevant. 

Type your answer in the textbox below

Text Answers (3) View

The information is basic View

I thought it was good View

Some sections are too wordy and the first slide needs to be edited. View

Question 20*

Please list any topics you believe should be included  in this module
Please type your answer in the textbox below. 

500 word max

Text Answers (3) View

how to actually use the camera View

a list of terms would be nice View

what types of shots to take when View

Results:
The overall results of the Beta test were favorable and informative. Organization and usability were found to be effective.
The content was too basic for the learners used in this test. As indicated in the limitations section of the evaluation plan, the
reviewers already completed a face-to-face training and were not true representatives of the intended audience. Because
there have not been any new volunteers, we used the newest volunteers to review the project. We will have to introduce
the program to future volunteers at a later time. Thie level of difficulty will also be addressed with the lessons that will be
developed later in the module.

Revisions:
1. Glossary will be added to the page and specific words and terms will be linked to the glossary
2. Additional lessons and units will be added to cover the breadth of information required for Part II of the training program.
3. Audio levels are being reassessed to establish uniformity.