Create a Rich Internet Application

Lesson 5 Usability testing

Informatica Communicatie Academie Hogeschool Arnhem Nijmegen

Learning aims

At the end of this lesson you know

-what usability testing is -why we do a usability test -how to prepare, organize and analyze a test -about important measurable variables -all info about your next assignment

Contents

Part-01

What is Usability Testing

Part-02
Part-03

Measures and variables

1E Phase Usability Test Preparation of the test Conducting the test Structuring data Processing data

Design overview

What is usability testing

Usability testing is not: Testing functionalities (although this partly applies!) Beta-testing Doing research Focus group*
*A focus group is a group of (in general) 6 to 12 people, led by a discussion leader, that discusses a given subject. Attendants of the group can be clients, users or experts. The group is not interviewed, but a vivid interaction between attendants is the main driver. By an interactive discussion a more valuable result can be expected then in the case of separate interviews with people within the group. A focus group is a valuable instrument for discovery. It reveals hidden needs on top of the explicit needs.

What is usability testing

Usability testing concerns:

Improving the usability of the product Improving the product in general Finding the information on how to improve the product

Place of the Usability Test

User- & Task analysis

Also the basis for your action plan!

Not satisfactory
Design

Repeat Usability Testing

Prototype

Satisfactory
Endproduct

Wat is Usability Testing

The test is based on goals set in preparation of the test

Test-persons are real users and do realistic tasks (as you designed them)
Testers - observe and register what users do and say - analyze registered data -measurements, users remarks, observations - identify usability-problems - formulate advice for improvements

Characteristics Usability Test

Time for a little video http://www.2c.nl/nl/usability_onderzoek/usability_test/video.p hp http://www.usefulusability.com/10-must-see-usability-videos And software http://silverbackapp.com/ Or search for usability software testing on Google

Phases Usability Test

This lesson: 1. Plan and prepare the usability test. 2. Take the usability test. 3. Collect test data from the usability test. Next lesson: Analyze data of the test. Report test results.

Contents

Part-01

What is Usability Testing

Part-02 Measures and variables

Part-03 1E Phase Usability Test Preparation of the test Conducting the test Structuring data Processing data

Sorts of Variables

Nominal Qualitative
(kwalitief)

Ordinal

Discrete Quantitative
(kwantitatief)

Continuous

Qualitative Nominal

Latin: nomen = name Concerns the naming of the registered parameter Example: Gender: Man / Woman City: Arnhem Animal sorts: Dog, cat etc

Qualitative Ordinal
Aggregation of information that can be organized in logical categories without reference to absolute measurable meaning or interpretation Ordinal measurements only show relative discrimination between categories Examples: Education levels: VMBO, Havo Grades in education: 1-10 Scales/scores/rankings: very difficult - little difficult - neutral little easy very easy
It shows that there is a difference between little and very, but without telling you how much difference

Quantitative Discrete

Quantitative measurements result in numbers

Discrete variables comprise of real full number only: 1,2,3,4,5 etc

Example: Flipping a coin can only give you one side in a full number of cases. You cant get it 2,5 times! So thats why this quantitative number can only be a discrete one.

Quantitative Continuous

Continuous variables can have any kind of value: 1,3 5.000,92 0,001 - etc Example: If you register the weight of an item it can be any value even when limited between given thresholds like any value between 10 and 100: 30,5 40 60,58

Quantitative Continuous

Unlike discrete variables that can only hold discrete points on a scale, continuous variables can have any value. For example: To become a fire fighter ones weight must be between 75 and 100 kilo. Weight is an example of a continuous variable. The weight of a fire fighter van have any value between 75 and 100 kilo.

Qualitative or quantitative?

1. Determine what you want to measure and which values are allowed; 2. Determine how you want to respond to values that are not permitted;

Needed to present your results and draw conclusions from a usability test!
It is important to determine what you want to measure before you start!

Contents

Part-01

What is Usability Testing

Part-02
Part-03

Measures and variables

1E Phase Usability Test Preparation of the test Conducting the test Structuring data Processing data

Planning and preparation

Determine aims

Participants are real users Determine users and participants

Participants perform realistic tasks Select tasks Write scenarios Write interview questions Determine how usability will be measured

Determine aims

Often products are (very) complex.

Ask yourself: What do I want to achieve?

Write questions that reveal important issues: How fast and accurate can users find a specific task by clicking the right button? Write down aims for these issues: Users must be able to accomplish a task at once without using a method of trial and error.

Determine users and participants

The target population can be taken from the user profiles described in the user analyses. Sample size: about 4-6, minimal 3 Create some subgroups

Number of users

1 test with 8 user

2 tests with 3 users each

5 errors found in total

9 errors found in total

Which Use Cases to choose?

The aim of your usability test is your starting point. Ask yourself: What problems can be expected? Check your Use Cases and scenarios (Structure Document)

Put some Use Cases in a sequence that seems to be natural for the user. Write (test)scenarios that are understandable for the user. N.B. These Use Cases perform tasks which the user will be testing.

Write (Use Case) scenarios

A description of a task that will be performed by the tester.

Contains enough information to perform a task.

Uses terms with which the user is familiar. Is unambitious. Is NOT a list of user actions.

Scenario example (wrong) Add a new examination. Enter todays date. Enter Project4 in field Teacher and Interface project in field Description. Enter 25 in field number of questions and 1 in field parts. Enable all controls. Enter IC2B in field class.

Scenario example (right) The results of an examination for the course user interfaces have to be processed in SVS. You have been teaching this course to class IC2B. The examination consisted of 25 questions for which there is only 1 version. You want SVS to warn you if an answer contains a value that is outside the range of possible values or if a questions was not answered at all. Add a new examination to SVS to process your results.

Write interview questions Possible questions: Pre-test Before the test, to get a clear picture of the tester. Necessary when using subgroups. Post-task After each task, to ask the tester his opinion about a task. Gives a wealth of information! Use this always! Post-test After the test, to ask the tester his opinion about the product/website.

Determine how usability will be measured? Possible measurements (per task) are: Number of errors; Number of menus and screens displayed; Number of times the tester asked for Help or support; If the tester was able to accomplish a task (without asking for help); Remarks from the tester about the product/website tested; Time to accomplish a task; . Determine objective criteria (for each variable) Determine what is acceptable and what not

Inhoud

Part-01

What is Usability Testing

Part-02
Part-03

Measures and variables

1E Phase Usability Test Preparation of the test Conducting the test Structuring data Processing data

Important

Structuring and composition test team How to start Think aloud Help? List of problems identified

Organization testteam

Possible roles:

Tester
Facilitator Observer Observer

user who is performing the usability test

tells user what to do, facilitate, encourage to think aloud, listen what the user says body language (qualitative) time, mistakes, remarks (quantitative)

Start of the test

Introduction of the persons involved. Explanation of the process. Make clear that the product is tested and not the participant. Ask to think aloud.

Important during Take test

Composition of the test team Start of the test Think aloud Helping? Problem list

Contents

Part-01

What is Usability Testing

Part-02
Part-03

Measures and variables

1E Phase Usability Test Preparation of the test Conducting the test Structuring data Processing data

Structuring data

Legenda: Question 1: Was het moeilijk om uit te vinden hoe u uw risico op een hart of vaatziekte kon meten? Question 2: Waar bent u begonnen met zoeken naar de risicometer vanaf de homepage? Question 3: Wat vond u van het gebruik van de risicometer?

Processing data Conclusions: From the retrieved data we can conclude that: 1. Male testers were more able to find the Risicometer; 2. Female testers did do all the tasks in an orderly sequence and started with the Medische informatie part; 3. Male testers started by entering a search request; 4. The use of the Risicometer is rather easy; 5. Two participants complained because not all values can be entered. Someone who is younger than 40 (age) cannot use the Risicometer (value cannot be entered).

Structuring data

Legenda: Question 1: Kon u de functie om zorgverleners te vergelijken makkelijk vinden? Question 2: Waren de keuzes die u moest maken duidelijk? Question 3: Kon u de informatie vinden die u zocht en was deze informatie duidelijk? Question 4: Waren de knoppen om verschillende zorgverleners te vergelijken duidelijk? Question 5: Zou u, als u ooit zorgverleners op kwaliteit moet vergelijken, www.kiesbeter.nl nog een keer gebruiken hiervoor?

Processing data Conclusions: From the previous table we can conclude that: 1. The functionality to do the test was easy to find; 2. Some testers followed alternative routes to find the functionality; 3. The received information was clear to all testers. One tester did not recognize the outcome of the test; 4. Two testers think that the site is not user friendly. Two testers think that the site is bad. In general we can conclude that the steps to do the test are straight forward and easy to follow. The side is not really user friendly. Two participants will not visit the site and two doubt if they will visit the site again.

Be carefull

Write down positive remarks as well. Everybody is eager to hear the good news. Write them down in your conclusions!

Contents

What is Usability Testing

Measures and variables

1E Phase Usability Test Preparation of the test Conducting the test Structuring data Processing data

Data presentation

Always in a table and/or graph! Purpose: present the data in an honest way No Puzzle

The - Contents (data) - Presentation (data) - Drawn conclusions Counts

Not like this

Not like this

This is nice

Legenda: Question 1: Was het moeilijk om uit te vinden hoe u uw risico op een hart of vaatziekte kon mete Question 2: Waar bent u begonnen met zoeken naar de risicometer vanaf de homepage? Question 3: Wat vond u van het gebruik van de risicometer?

This is nice as well.

Task 1 1.1 1.2 1.3 1.4 Name Totally agree Agree Renting a Bike Bike are easy to select 3 Pimping the bike makes fun 2 Suggested Bike type conforms to my profile 0 Accesoires are easy to select 1 Do not know 2 5 1 4 3 3 6 5 Do not agree Totally disagree 2 0 1 0 0 0 2 0

Suggested Bike type conforms to my profile

Antwoord

Totally agree

Agree

Do not know

Do not agree

Totally disagree

This could be an approach

Relate conclusions to observations: - The suggested bike in relation to the profile does not fit. Advice: Improve usability Profiling a User and Selecting a Bike.
Task 1 1.1 1.2 1.3 1.4 Name Totally agree Agree Do not know Renting a Bike Bike are easy to select 3 2 Pimping the bike makes fun 2 5 Suggested Bike type conforms to my profile 0 1 Accesoires are easy to select 1 4 Do not agree 3 3 6 5 2 0 1 0 Totally disagree 0 0 2 0

Suggested Bike type conforms to my profile

Antwoord

Totall y agree

Agree

Do not know

Do not agree

Totall y disagree

Other mistakes

3D graphs (do mostly make no sense)

Overload of graphs
Wrong use of diagrams (pie charts etc.)

If you want you can use

Location measures Median = Arithmetic Average = Range Variation

midst observation sum/n

max min

See also: (strongly advised!) http://www.youtube.com/watch?v=qKPhtBIsyIY&feature=related http://www.youtube.com/watch?v=5wO8QjfYsLI&feature=related

Assignment

Assignment 5.1

Start working on the first part of your usability test (look in your study guide)
Be aware: Use the provided structure document! Dont invent it yourselves (itll cost you time) Let us have a look.

Summarized
1. Design realistic tasks for the users. 2. Set up the observation. 3. Describe to the user the purpose of the observation. 4. Demonstrate the equipment in the room. 5. Explain how to "think aloud". 6. Explain that you will not provide help. 7. Describe the tasks and introduce the product. 8. Ask if there are any questions before you start. 9. Begin the observation. 10.Conclude the observation. 11. Use the results.
Tog On Interface, Bruce Tognazzini, Addison-Wesley, 1991, ISBN 0-201-60842-1 The Art of Human Computer Interface Design, Brenda Laurel, Addison-Wesley, 1991, ISBN 0-201-51797-3

Knowing more?

About face Alan Cooper A practical guide to usability testing Tom Brinck Usability for the web - JS Dumas & JC Redish