You are on page 1of 7

REBECCASCAFEMOBILEUSABILITYTESTING

EXECUTIVESUMMARY
This studywasconductedtofindoutwhatpeoplethinkofthegeneraldesignandfunctionalityof
our Rebeccas Cafe app. The app allows users to check theopenhours,save preferencesand
favorites, and order and pay for a sandwich without having to physically go there. We tested
four Northeastern University student that are representative of our target population. Verbal
consent was obtained from each participant before each session. Each participant completed
three tasks,thefirstinvolvedfindingoutifRebeccasisopenatacertaintime,and theothertwo
were different paths users could take when ordering andpayingforasandwich.Thestudytook
place in the apartment of one of the researchersbetween4pmand8pm,asresearchershadto
accommodate to participants schedules. One of the researchers was the facilitator and note
taker, and another played computer. All participants successfully completed all tasks with very
fewerrorsandreportedahighsatisfactionandeaseofuse.

METHODOLOGY
Whowetested
Fourparticipants,havingthefollowingcharacteristics,evaluatedRebeccasCafeMobile.

Age

Gender

21
3

Female
3
24
1

Male
1
Total(participants) 4

Total(participants)
4

SchoolYear

GotoRebecca's

Middler
2

Rarely
1
Junior
1

Onceaweek
2
Grad
1

Twiceaweek
1
Total(participants) 4

Total(participants)
4

PhoneUse

OrderingAppUse

Daily
4

Never
2
Total(participants) 4

Onceamonth
2

Total(participants)
4
Major

Finance
1

HealthSciences
1

ProjectManagement 1

SpeechPathology 1

Total(participants) 4

Whatparticipantsdid
Participants met with the researchers for about 10 minutes. They completed three tasks using
thepaperprototype.Participantssatatablewiththe paperprototypein frontofthem.Theywere
given a pencil to use as a keyboard. After completing all the tasks, the facilitator asked the
participantssociodemographic,satisfaction,andopenendedquestions.

Thefollowingbriefingwasreadtoparticipantsatthebeginningofeachsession:
Hi, we'redesigningan appforRebecca'sCafe,that willallowyoutochecktheopenhours,save
preferences and favorites, and order and pay for a sandwich without having to physically go
there.
We are conducting a studytofindoutwhatpeoplethinkaboutthis.Wewillnotrecordorpublish
any information with your name. This is for a course were taking in HumanComputer
Interaction from Prof. Bickmore in the College of Computer and Information Science. Your
participation is voluntaryand youcanstopanytimeandaskthat yourdatanotbeused.Itshould
take about 20 minutes and we will compensate you with access to the beta versionoftheapp.
Canyouhelpusoutwiththis?
Great. The purpose of todays session is for you to help us figure out how to make this mobile
web interface more userfriendly before we finish developing it. But we arent going to use a
computer or phone. Weve created paper versions of the screens. The prototypestillhassome
rough edgeswere still thinking through how it should work and some parts of it are
incomplete. Before we cast it in concrete, we want to get some feedback about how well this
designworks.
Well give you some tasks that we think are representative of what people might do in reallife.
Your job is to tell us what makes sense, whats confusing, whether it works the way youd
expect it to, and any questions that come to mind. Yourquestionsareespeciallyvaluable,butI
may not answer them right away because our goal is to change the interface so it answers
them.
Audrey will be playing the computer. Since machines cant talk, shes not allowed to explain
anything. If you want to do something, youll need to interact with the prototype just as you
would on a computer. Use your finger to click on buttons. These pieces oftapeindicateplaces
whereyoucantypesomethingin,andheresyourkeyboard(givepen).ItsOKtowriteonthis.
Keep in mind that were testing the interfacewere not testing youso if you run into any
problemsitsnotyourfaultanditmeansthattheressomethingweneedtochange.
Areyoureadytostart?
OK, heres the first thing wed like you to do. Take a minute to read this and let me know if it
makessense.Ifso,thenwheneveryourereadypleaseshowuswhatyouwoulddofirst.

Whatdatawecollected
During each task we collected number of errors, time to complete task, and think aloud notes.
Afterthethreetaskshadbeencompletedwecollectedthefollowinginformation:
Sociodemographic: Age, gender, year, major, phone use, online ordering app use, how
oftentheygotoRebeccas
Interview questions: Satisfaction rating (scale from 17, 1 being very satisfied and 7
being not satisfied), ease of use rating (scale from 17, 1 being very easy and 7 being
very difficult), anything that the user found confusing, anything the user would change,

would the user prefer the app or going to Rebeccas, what else the user would like to
seeontheapp

TASK#1:
You have class until 2pm this coming Wednesday. Youre trying to figure out if you have time
forRebeccasafter.HowlateisRebeccasopenonWednesdays?

Numberofparticipants:
4
Percentsuccessful:
100%
Numberoferrors:
Mean:0errors
StandardDeviation:0
Timetocomplete:
Mean:18.25sec
StandardDeviation:2.36

Findings:
Alltheparticipantswereabletosuccessfullycompletethetaskinlessthan30seconds.

Recommendations:
None

TASK#2:
You get out of class soon and you want a sandwich. Order a BBQ pork sandwich with no
cheese.Addlettuce.Youwanttopickitupassoonasyoucan,andyoullpaythere.

Numberofparticipants:
4
Percentsuccessful:
100%
Numberoferrors:
Mean:0.5errors
StandardDeviation:0.58
Timetocomplete:
Mean:92.75sec
StandardDeviation:6.6

Findings:
50% of the users tried to checkout before adding the sandwich to their cart,assuming
thatclickingDonefromthecustomizemenuaddstheitemtothecart.

Recommendations:
Add an Add to Cart button on the customize menu, so users dont have topressAdd
to Cart after theyre done customizing. Keep a cartcountdownbythecheckoutbutton
so users have some visual feedback that theyve successfully added something to the
cart.

TASK#3:
Its 8:00 am but you know you only havealittletimeforlunchafter your10:30class. Youorder
a Caprese sandwich as is, tobepickedupatnoon. Youwantchipsandcoffee withit. Youjust

got a new credit card and you have to add it as a payment option. Card Number:
5555555555555555Expdate:10/2017SecurityCode:555

Numberofparticipants:
4
Percentsuccessful:
100%
Numberoferrors:
Mean:0.25errors
StandardDeviation:0.5
Timetocomplete:
Mean:121.75sec
StandardDeviation:12.74

Findings:
1 user had to be prompted to addthedrinkandsides,asshehadmovedontopayment
options. However, we believe this was moreofaproblemwithhowthetaskwaswritten
ratherthantheactualapp.
50% of users thought that they had to select a payment option before adding a new
creditcard.
1userwonderedifshecouldaddanewcreditcardbeforeordering.

Recommendations:
Put the Add New payment option button before choosing an already saved payment
option, some users thought they had to select a payment option before choosing Add
New.
Makesuretoaddtheoptiontoaddanewcreditcardfromthepreferencesmenu.

EXITQUESTIONSANDSAMPLERESPONSES

Wasthereanythingthatwasconfusing?
No,nothingconfusing.

Ratetheeaseofuseoftheappfrom17,1beingveryeasy,7beingverydifficult.
1,2,4)1 3)2

Rateyoursatisfactionwiththeappfrom17,1beingverysatisfied,7beingnotsatisfied.
1) 2
2,3,4)1

WouldyouchoosetogotoRebeccasorusetheapp?
Usetheapp,thatwayyoucanskiptheline!

Whatelsewouldyouliketoseeontheapp?
WaytoseehowlongthelineisatRebeccas,ornumberoftablesfilled
Profilermealoption
Abilitytochoosetoastedinsandwichcustomization
Abilitytoaddnewpaymentoptionbeforeordering

PROTOTYPEPHOTOS