iScale.

nl
An alternative to longitudinal studies

Evangelos Karapanos

Tampere, 1st June 2011

Cross-­‐sec8onal Repeated  sampling

“Longitudinal”  paradigms  in  HCI

Longitudinal Retrospec8ve
Cross-­‐sec8onal Repeated  sampling

“Longitudinal”  paradigms  in  HCI

Longitudinal Retrospec8ve
Cross-­‐sec8onal Repeated  sampling
(within-­‐subject)

“Longitudinal”  paradigms  in  HCI

Longitudinal Retrospec8ve
How  do  people  evaluate  products  
over  0me?

Hassenzahl,  2004
 User  experience  over  0me
An  exploratory  study

10  subjects  received  an  innova0ve  poin0ng  device  for  4  weeks

and  rated  it  using  Hassenzahl’s  AErakDiff2  ques0onnaire

during  Week  1,  and  Week  4

User  experience  over  0me
An  exploratory  study

Items  close  together  are  highly  correlated.  Lines  represent  clusters.

Cross-­‐sec8onal Repeated  sampling

“Longitudinal”  paradigms  in  HCI

Longitudinal Retrospec8ve
Cross-­‐sec8onal Repeated  sampling

“Longitudinal”  methods  in  HCI

Longitudinal Retrospec8ve
Cross-­‐sec8onal Repeated  sampling

“Longitudinal”  methods  in  HCI

Longitudinal Retrospec8ve
Cross-­‐sec8onal Repeated  sampling

Now  think  of  the  three  experiences  that  were  for  you  personally  
most  sa.sfying  or  unsa.sfying  experiences  of  today.  
Please,  use  your  own  feeling  or  a  defini.on  of  what  “sa.sfying”  and  
“unsa.sfying  e“Longitudinal”  
xperience”  means.  Take   methods  
a  couple  oif  n  
mH CI to  be  sure  to  
inutes  
come  up  with  three  most  crucial  experiences;  you  may  also  want  to  write  
them   down  for  yourself.  We  want  you  to  be  open  as  to  Retrospec8ve
Longitudinal which  experiences  
to  report.
Cross-­‐sec8onal Repeated  sampling

“Longitudinal”  methods  in  HCI

Longitudinal Retrospec8ve
Cross-­‐sec8onal Repeated  sampling

“Longitudinal”  methods  in  HCI

Longitudinal Retrospec8ve
Cross-­‐sec8onal Repeated  sampling

“Longitudinal”  paradigms  in  HCI

Longitudinal Retrospec8ve
How  do  we  recall  experiences?

Remembering  is  an  act  of  reconstruc.on  

rather  than  reproduc.on
BarleE  (1932)
 How  do  we  recall  experiences?

• Validity?  i.e.  do  memories  reflect  what  we  really  

experienced?

• Reliability?  i.e.  in  a  second  trial,  will  we  recall  the  

same  experiences?
       
        in recalling
Can we assist people
their experiences with a product?
 How  do  we  recall  emo$onal  
experiences?

Two  schools  of  thought

-­‐  The  Construc.ve  approach
-­‐  The  Value-­‐Account  approach
 The  Construc0ve  approach
• Robison  &  Clore  (2002)
– “The  emo$onal  experience  can  neither  be  stored  nor  
retrieved”,  it  is  reconstructed  from  recalled    contextual  
details

4  different  sources  of  informa0on

• Experien0al  knowledge
• Episodic  informa0on Episodic  memory

• Situa0on-­‐specific  beliefs
• Iden0ty-­‐related  beliefs Seman0c  memory
 The  Value-­‐Account  approach
• Reconstruc0on  happens  in  a  top-­‐down  fashion,  people  may  recall  the  gist  of  
an  event  even  when  they  fail  to  recall  contextual  details

• Value-­‐Account  (Betsch  et  al.,  2001)

– Memory  structure  that  stores  frequency  &  intensity  of  responses  to  s0muli
– More  easily  accessible  than  concrete  details  from  episodic  memory
– May  be  used  to  cue  the  reconstruc0on  from  episodic  memory

• Gist  is  beEer  retained  over  0me

• Autobiographical  memories  (Conway  and  Pleydell-­‐Pearce  2000)

– Three  levels  of  specificity,  i.e.  life0me  periods,  general  events,  and  event-­‐specific  
knowledge  
– knowledge  stored  at  the  level  of  a  life0me  period  may  cue  informa0on  at  the  two  
lower  levels
 Results  from  experiment
• When  using  iScale  
–par0cipants  recalled  more  experiences
–with  more  details
–were  more  reliable  in  recalling  when  their  experiences  
took  place

21
Two  Kinds  of  insights
Temporal  dynamics  graphs
 Two  Kinds  of  insights
A  huge  amount  of  experience  narra8ves!
Analyzing  Narra0ves
 Analyzing  Narra0ves
• Rich  situated  insights
–How  a  product  is  experience  by  a  par0cular  user,  in  a  
par0cular  context
 Analyzing  Narra0ves
• Rich  situated  insights
–How  a  product  is  experience  by  a  par0cular  user,  in  a  
par0cular  context
 Analyzing  Narra0ves
• Rich  situated  insights
–How  a  product  is  experience  by  a  par0cular  user,  in  a  
par0cular  context

• Abstracted  knowledge
 Analyzing  Narra0ves
• Rich  situated  insights
–How  a  product  is  experience  by  a  par0cular  user,  in  a  
par0cular  context

• Abstracted  knowledge
–How  frequent  is  a  certain  kind  of  experience?
 Analyzing  Narra0ves
• Rich  situated  insights
–How  a  product  is  experience  by  a  par0cular  user,  in  a  
par0cular  context

• Abstracted  knowledge
–How  frequent  is  a  certain  kind  of  experience?
–How  does  the  dominance  of  different  experiences  
fluctuate  over  .me?
LCA  demo
 Automated  Content  Analysis
• Latent  Seman0c  Analysis  /  Latent  Dirichlet  
Alloca0on
–Compu$ng  document  similarity  based  on  term  co-­‐
occurence
 Latent  Concept  Analysis

Can  we  combine  computa$onal  techniques  with  

tradi0onal  qualita0ve  coding  procedures  into  a  
semi-­‐automated  method?
 Thank  you
Evangelos  Karapanos
ekarapanos.com

Relevant  publica8ons
• Karapanos,  E.,  Martens,  J.,  &  Hassenzahl,  M.  (2009).  Reconstruc8ng  Experiences  through  
Sketching.  (submiEed)  ACM  TOCHI.  Available  as  a  pre-­‐print  from  hEp://arxiv.org/abs/
0912.5343.
• Karapanos,  E.,  Zimmerman,  J.,  Forlizzi,  J.,  &  Martens,  J.  (2009).  User  experience  over  
8me:  an  ini8al  framework.  In  CHI  '09:  Proceedings  of  the  27th  interna$onal  conference  
on  Human  factors  in  compu$ng  systems  (pp.  729-­‐738).  New  York,  NY,  USA:  ACM.