Mark McKenna 10/22/12 User-centered Design Study: AWARE 2.

0 ITLS 6290

Deliverables
Rapid Prototype For the purposes of this user-centered design study I created a rapid prototype of AWARE 2.0 using Adobe Flash, the same tool I will use to develop my final version. I incorporated a basic click-and-drag feature to allow the learner to move cards anywhere on the chart as they assign them ranks.

Though learners can complete the AWARE activity, this prototype lacks several key features that will be added later: The grey boxes will be drop targets for the cards. There will be a <click> sound when cards enter the drop target. The description window will display an in-depth description of the last card that was clicked. The most recently clicked card will have a glow so that the learner knows which card description is displayed in the description window.

There will be a printable version of the chart that will be accessed after each card is assigned a drop target. There are several other modifications that will be included based on the feedback gained from this study (see Summary section). Protocol I provided a brief statement to my participant (referred to as C.) in order to describe the intent of the user-centered design study. I did not give him any instructions regarding how to use the program since it is designed to be self-contained. This is the script I read to my participant at the beginning of the study: I am conducting a usability study to see how well this program works. I am evaluating the program, not you. The purpose of my study is to see how you use this program and, most importantly, what I can change to make it better. Because it is only a prototype, there are a few features that are not yet functional. Let me point out one just so you know what it is for. This is the description window <point to description window on screen> that will provide a more detailed description of whatever card you are working on. You can ignore it for the purposes of this study. While trying this out, I would like you to think out loud. Explain to me everything that you are doing. Describe what you are confused about and why you are doing what you are doing. This is so that I can figure out what works well with the program and what doesnt. Go ahead.

Participant
The characteristics of C. (the name by which I refer to my participant) are representative of Deseret Industries associates (the target audience that will use AWARE 2.0) in the following areas: He has been at Deseret Industries for one year (the average enrollment of current associates). He has a similar employment background as most associates (several entry-level jobs that didnt last very long). He processes ideas in a manner that is consistent with most employees I have worked with. He has similar exposure to the AWARE form as most other associates have. He is male, Caucasian, and in his mid-thirties. (Just over half of the employees are male, most are Caucasian, and most are between 30 to 40 years old.) He is less representative in the following ways: He has more formal education (a bachelor degree) than the average associate (GED or high school diploma). He has above-average work performance, though he has only recently attained it.

Given these characteristics, C. is a sufficiently representative sample of the target audience.

Data
This is an organized version of the field notes I took while observing C. as he used AWARE 2.0 for about 30 minutes. I have included headings to capture the main themes observed during the study. 1. Understanding What to Do Observation A. Almost immediately after beginning, C. asked, Do I click on one of these boxes? while pointing to the grey boxes that make up the pyramid.

Inference A. The instructions at the top of the page are not specific enough for the learner to immediately understand that he/she should drag cards from the stack and assign them to the target spots (the grey boxes) of the pyramid. B. C. was confused about why the pyramid ranking included a height element. He did not know if there was a difference between assigning a card to the lowest row in the column or the highest. C. C. didnt fully understand that this exercise is to evaluate his own personal behavior. He thought it is more to assign values concerning the importance of general work behaviors.

B. C. dragged a card to a column and then waited a long period of time before assigning it a row. He asked, Do I start at the bottom or the top? C. Throughout the course of the exercise C. assigned many cards to boxes in higher rows, saying, That is pretty important. When moving the card that says, I start work on time each day and return promptly from breaks C. commented, This one is the biggest hindrance for associates getting promotions. He then assigned it a spot on the side of to a greater extent.

2. Technical Difficulties Observation A. Many times C. squinted his eyes and put his head closer to the monitor.

Inference A. Because the text is rasterized, not vector, it is blurry and difficult to read.

At one point he said, You would think I didnt have my contacts in. I cant read this. He had to ask clarification about the wording of the text on 3 of the cards. B. C. looked at one of the cards that used a hyphen to divide a word for an extended period of time. Then he said, Are you going to keep the hyphens later on?

B. Using hyphens to let words run on to the next line further adds to the difficulty in reading the text.

3. Arrangement Observation A. After C. had looked at about 1/3 of the cards, he spread out the remaining cards and read through them all before continuing to arrange.

Inference A. It is not obvious that the space above the pyramid is intended to help the learner arrange the cards prior to assigning them a spot.

4. Ranking Observation A. As C. assigned ranks to last several cards, he hesitated for long periods of time between each card. Then he said, I strive to do all these to the greatest extent possible. He also asked, What if I want to put a bunch in the same spot? Inference A. C. struggled to know how to deal with behaviors that no longer had an available space within the rank he would have liked to assign them.

These are some screenshots showing C.s activity during the study as he arranged and ranked the behaviors:

Summary
Performing this user-centered design study was very effective in finding useful changes to make to AWARE 2.0. The solutions I propose are based on the observations and inferences described in the field notes (see Data section). 1. Instruction screen at beginning (to address 1A, 1C, & 4A) The learner did not clearly understand the purpose of AWARE 2.0 because there were insufficient instructions. He was unclear whether he was ranking his own behavior or the importance of the ideal behaviors he encountered. He also did not understand what he was supposed to do with cards and grey boxes because the instructions do not clearly connect ideal behaviors to the behaviors listed on the cards. To resolve these issues I will include a screen that the learner will be required to read prior to beginning the program. This screen will include a brief introduction to the purpose of AWARE 2.0 and simple instructions regarding how the learner will use it. It will emphasize that the learner is evaluating his/her personal behavior (1C) and that each grey box needs to be filled by dragging cards onto the pyramid diagram (1A & 4A). This will allow the learner to clearly understand what to do once he/she accesses the main screen. 2. A more clearly defined layout (to address 1A, 1B, 3A, & 4A) The learner had difficulty figuring out that there is no weighted distinction between the rows of the same column. Height often implies hierarchy, but there was no clear way of knowing this did not apply. The learner also hesitated frequently when required to make less significant value judgments between two behaviors because it was not clear to him that the most important decisions involve the columns at the to a lesser degree side. Without knowing that he was drawing cards to the pyramid, he perceived the grey boxes to be buttons. In order to show that there is no distinction needed between rows in the same column, I will put a colored border around the entire column. This will indicate to the learner that boxes within the same colored region have equal weight (1B). I will also extend this color border to include two columns (in most cases), indicating that the two columns form a grouping for the cards placed in that region (4A). The card stack will be placed on a box that appears raised, making it clearer to the learner that it is a stack to be drawn from (1A). Finally, the area around the colored pyramid will be a checker box pattern that appears to be limbo space for the learner to arrange cards before assigning them ranks (3A). I will conduct a focus group after designing this to ensure these changes provide adequate nonverbal cues for the learner to understand their use. 3. Vector-based text and increased text size (to address 2A & 2B) Due to poor resolution, the learner struggled to read the text. Much of it was grainy and the text size was inadequate. Also, the constraints of the box size caused some of

the words to be hyphenated in order to run onto the next line in the box. These technical deficiencies unnecessarily increased the cognitive load required for the learner to process the text. I will be able to increase the text size and get rid of hyphens by increasing the length of the cards and grey boxes (2B). The height I anticipated is accurate for the computer monitors available, but I can utilize more horizontal space in this fashion. I will also use vector-based text that will not become pixelated when the program is zoomed in (2A). These changes will make the text much more readable for the learner.

This user-centered design study provided me with valuable feedback to increase the effectiveness of my learning environment. The changes that I will implement will allow me to leverage the learning effect of AWARE 2.0 for Deseret Industries associates. In light of the valuable nature of this study, it will be valuable for me to conduct other user-centered design studies later on in the development of my project.