Input Techniques and Models

Properties of an input device

Evaluation and analysis of input devices
Input device states
Interaction modalities

Hinckley, Ken., Input Technologies and Techniques, Chapter 7. Handbook of Human-Computer Interaction, ed. by Andrew
Sears and Julie A. Jacko. Lawrence Erlbaum & Associates.
Whats an input device
Input devices sense physical properties of people, places or
things

However, they do not operate in isolation, i.e. need visual

feedback, haptic feedback, etc...
otherwise similar to a pen without paper

Must include:
the physical sensor (positioning wheels)
the feedback presented to the user (cursor)
the ergonomic of the device (fits in hand)
interplay between all the interaction techniques supported by a
system (clicking, moving, selecting, etc.)

Need an understanding of input technologies to design

interaction techniques that match a users natural workflow

1/29/2013 Comp 4020 - HCI 2 (PPI) 2

Pointing Devices
Mouse:
Invented in 67
Used for pointing
Picks up changes in x, y
As good as pointing with finger
Integrated with buttons/wheels etc

Trackball:
Senses relative motion of partially exposed ball in
2DOF
Engage different muscle groups than the mouse,
but an alternative for those who experience
discomfort
1/29/2013 Comp 4020 - HCI 2 (PPI) 3
Pointing Devices
Isometric joystick:
Force sensing
Rate of cursor is proportional to the force exerted
Returns to center when released
Good when space is at a premium

Isotonic joystick:
Sense angle of deflection
Different than isometric joystick

J. Lipscomb and M. Pique (1993). Analog Input Device

Physical Characteristics. SIGCHI Bulletin 25 (3): 40-45.
1/29/2013 Comp 4020 - HCI 2 (PPI) 4
Pointing Devices
Touchpads:
Small touch-sensitive devices found on laptops
Use relative mode for cursor control
Can operate in absolute mode by dragging
finger and leaving it on edge of the pad
Necessitates multiple clutchings

Touchscreens/pen-operated devices:
Fingers, or electromagnetic digitizers
Parallax error, mismatch between sensed input
position and apparent position

1/29/2013 Comp 4020 - HCI 2 (PPI) 5

Input Device Properties
Several properties characterize most
devices:
Property sensed
Number of dimensions
Input mode
Positioning
Device acquisition time

1/29/2013 Comp 4020 - HCI 2 (PPI) 6

Property Sensed
Most devices sense:
Linear position (tablets sense position of pen)
1
Motion (mice sense change in position)
Force (Isometric joysticks, IBM Trackpoint)
Angle or change in angle (rotary input)

Absolute input device

position sensing

Relative input device

motion sensing

Relative device requires visual feedback but also

can be inefficient due to clutching
1/29/2013 Comp 4020 - HCI 2 (PPI) 7
Number of Dimensions
Devices sense one or more input dimensions
Two linear dimensions (mouse, x/y)
2
Angular dimension (knob)
6 degree-of-freedom (magnetic tracker, senses 3-position
dimensions and 3 orientation dimensions)

A pair of knobs is a 1D+1D device, mouse with scroll

wheel is a 2D+1D multi-channel device

Multiple degree-of-freedom devices sense three or

more simultaneous dimensions of spatial position or
orientation

1/29/2013 Comp 4020 - HCI 2 (PPI) 8

Degrees of Freedom vs. Dimensions
Usually confuse the idea of dimensions and degrees-of-freedom 2

the bat

Ware, C. and Jessome, D. (1988), Using the Bat: A Six Dimensional Mouse for Object
Placement. IEEE Computer Graphics and Applications. November 8-6, 65-70.

1/29/2013 Comp 4020 - HCI 2 (PPI) 9

Mapping DoF to Dimensions
3DOF Mouse 2

A Two-Ball Mouse Affords Three Degrees of Freedom

I. Scott MacKenzie, R. William Soukoreff, & Chris Pal
CHI 97 Electronic Publications: Late-Breaking/Short Talks
1/29/2013 Comp 4020 - HCI 2 (PPI) 10
 Mapping DoF to Dimensions
4DOF Mouse 2

Ravin Balakrishnan, Thomas Baudel, Gordon Kurtenbach, George W.

Fitzmaurice. (1997). The Rockin'Mouse: Integral 3D manipulation on a plane.
Proceedings of ACM CHI 1997 Conference on Human Factors in Computing
Systems, p. 311-318.

1/29/2013 Comp 4020 - HCI 2 (PPI) 11

 Mapping DoF to Dimensions
5DOF Mouse 2

Hinckley, K., Sinclair, M., Hanson, E., Szeliski, R., Conway, M., The
VideoMouse: A Camera-Based Multi-Degree-of-Freedom Input Device, ACM
UIST'99 Symposium on User Interface Software & Technology, pp. 103-112.
1/29/2013 Comp 4020 - HCI 2 (PPI) 12
Input mode: Indirect vs. Direct
Primary method of operation of a device
3
Mouse is indirect, i.e. must move mouse to
move pointer on screen

Touch-screens or tablets are direct input

devices, i.e. unified input and display
surface
Occlusion is typically a problem

1/29/2013 Comp 4020 - HCI 2 (PPI) 13

Indirect input
3

visual space

motor space
Direct input
3

motor & visual space

Positioning: Absolute vs. Relative
Primary method of operation of a device
4
Mouse is indirect, i.e. must move mouse to
move pointer on screen

Touch-screens or tablets are direct input

devices, i.e. unified input and display
surface
Occlusion is typically a problem

1/29/2013 Comp 4020 - HCI 2 (PPI) 16

Absolute positioning
4
Absolute positioning
4
Absolute positioning
4
Absolute positioning
4
Absolute positioning
4
Relative positioning

clutching
4
Gain
Control-to-display gain or C:D ratio
4
distance moved by an input device/distance
moved on the display

Composite measurement taking into

account device size and display size

1/29/2013 Comp 4020 - HCI 2 (PPI) 23

 Direct Indirect

Relative

Absolute

1/29/2013 Comp 4020 - HCI 2 (PPI) 24

Quiz
Can you think of ways to get the benefits of
both indirect and direct input?

1/29/2013 Comp 4020 - HCI 2 (PPI) 25

ARC-Pad:
Absolute + Relative Cursor Positioning for
Large Displays with a Mobile Touchscreen

David McCallum, Pourang Irani

ACM UIST 2009

Large Displays
Reach is problematic

Custom hardware

Not always available or expensive

1/29/2013 Comp 4020 - HCI 2 (PPI) 28

Coupling a mobile device
Relative
large distance to travel
lots of clutching!

Absolute
imprecise

1/29/2013 Comp 4020 - HCI 2 (PPI) 30

 Hybrid Pointing

a b c

d e

Forlines et al, ACM UIST 2006

Goals
Reduce cognitive overhead
no explicit toggle between input modes
both modes are always available

Can be used with available hardware

with devices you may already own

1/29/2013 Comp 4020 - HCI 2 (PPI) 32

ARC-Pad
ARC-Pad
ARC-Pad
1/29/2013 Comp 4020 - HCI 2 (PPI) 36
Video
1/29/2013 Comp 4020 - HCI 2 (PPI) 38
Summary
Seamless transition between absolute &
4
relative positioning is crucial
no explicit mode switch
via interpretation of input gesture

Software based solution requires some

training

1/29/2013 Comp 4020 - HCI 2 (PPI) 39

Device Acquisition Time
Acquisition time
5
average time to move hand to device

Homing time
average time to return to a home position, i.e.
mouse to keyboard

For desktop workflows, pointing and

selecting dominate acquisition/homing time
integration of pointing with keyboard may not
improve overall performance
1/29/2013 Comp 4020 - HCI 2 (PPI) 40
Input Device States
Disaccord between states of a GUI and
states and events sensed by devices

Difficult to support interface primitives such

as click, drag, double-click, and right-click

Useful to diagram device states

Identifies relationship between events sensed by
input device and interaction technique
demands

1/29/2013 Comp 4020 - HCI 2 (PPI) 41

Three-State Model of Graphical Input
Buxton's 3-state model for graphical input devices

Expression of the operation of computer pointing

devices in terms of state transitions

Expressive vocabulary for exploring the relationship

between pointing devices and the interaction
techniques they afford

1/29/2013 Comp 4020 - HCI 2 (PPI) 42

Input Device States
Three-states:
The states are Out of range (State 0, for clutching or
repositioning a mouse on a mouse pad;
Tracking (State 1) for moving a tracking symbol such as a
cursor about a display
Dragging (State 2) for moving an icon on the desktop or for
grouping a set of objects or a range of text

1/29/2013 Comp 4020 - HCI 2 (PPI) 43

Three-State Model of Graphical Input
Seems simple and obvious but can add insight to
the existing body of pointing device research
can be extended to multi-button interaction, stylus input,
and direct vs. indirect input

1/29/2013 Comp 4020 - HCI 2 (PPI) 44

Input Device States
Based on Buxtons model, the mouse &
touch sensitive devices are a two-state
device

1/29/2013 Comp 4020 - HCI 2 (PPI) 45

Activity: Input Device States
Based on the previous state diagram, can you
describe a limitation of touch-sensing input (PDAs)

Do you know of a device that supports all 3 states

1/29/2013 Comp 4020 - HCI 2 (PPI) 46

Evaluation and Analysis of Input Devices
Fitts Law
Accots Steering Law

1/29/2013 Comp 4020 - HCI 2 (PPI) 47

Models and Modeling
A model is a simplification of reality, but useful only if
it helps in understanding some phenomenon or
behavior:
Design, evaluate or help understand complex behavior

Models sit on a continuum:

Analogy & Mathematical

Metaphor Equations

Descriptive Predictive
Models Models

Guiards Model Fitts Law

1/29/2013 Comp 4020 - HCI 2 (PPI) 48

A Quiz Designed to Give You Fitts
http://www.asktog.com/columns/022Designed
ToGiveFitts.html

Microsoft Toolbars offer the user

the option of displaying a label
below each tool. Name at least one
reason why labeled tools can be
accessed faster. (Assume, for this,
that the user knows the tool and
does not need the label just simply
to identify the tool.)

1/29/2013 Comp 4020 - HCI 2 (PPI) 54

A Quiz Designed to Give You Fitts
1. The label becomes part of the target. The
target is therefore bigger. Bigger targets, all
else being equal, can always be
acccessed faster. Fitt's Law.

2. When labels are not used, the tool icons

crowd together.

1/29/2013 Comp 4020 - HCI 2 (PPI) 55

A Quiz Designed to Give You Fitts
You have a palette of tools in a
graphics application that consists of a
matrix of 16x16-pixel icons laid out as a
2x8 array that lies along the left-hand
edge of the screen. Without moving the
array from the left-hand side of the screen
or changing the size of the icons, what
steps can you take to decrease the time
necessary to access the average tool?

1/29/2013 Comp 4020 - HCI 2 (PPI) 56

A Quiz Designed to Give You Fitts
1. Change the array to 1X16, so
all the tools lie along the edge
of the screen.

2. Ensure that the user can click

on the very first row of pixels
along the edge of the screen
to select a tool. There should
be no buffer zone.

1/29/2013 Comp 4020 - HCI 2 (PPI) 57

A Quiz Designed to Give You Fitts
Microsoft offers a Taskbar which can be oriented
along the top, side or bottom of the screen,
enabling users to get to hidden windows and
applications. This Taskbar may either be hidden or
constantly displayed. Describe at least two reasons
why the method of triggering an auto-hidden
Microsoft Taskbar is grossly inefficient.

1/29/2013 Comp 4020 - HCI 2 (PPI) 58

A Quiz Designed to Give You Fitts
Screen edges are prime real estate. You don't
waste an entire edge that could be housing a
couple of dozen different fast-access icons just for
one object, the Taskbar

The auto-hidden Taskbar is entirely too easy to

display by accident. Users are constantly triggering
it when trying to access something that is close to,
but not at, the edge

The Taskbar would not have any of these

problems, yet be even quicker to get to if it were
located at any one of four corners of the display.
Throw the mouse up and to the left, for example,
and you'll have a taskbar displayed. Fast access
without the false triggering
1/29/2013 Comp 4020 - HCI 2 (PPI) 59
Fitts Law
Robust and highly adopted model of human movement

Originated as interest of applying information theory to the

analysis and understanding of difficulty of movement tasks &
human rate of information processing

Used Shannons law for information capacity

C = B log2(S / N + 1)
S is the signal power and N is the noise power

Based on the following analogies:

Amplitude of aimed movement == electronic signal
Spatial accuracy of movement == electronic noise

1/29/2013 Comp 4020 - HCI 2 (PPI) 60

 Fitts Law
Described the analogy in two
papers:
a serial, or reciprocal, target
acquisition task wherein subjects
alternately tapped on targets of
width W separated by amplitude A

experiment using a discrete task,

wherein subjects selected one of
two targets in response to a stimulus
light
Fitts, P. M. (1954). The information capacity of the human motor
system in controlling the amplitude of movement. Journal of
Experimental Psychology, 47, 381-391.

Fitts, P. M., & Peterson, J. R. (1964). Information capacity of

discrete motor responses. Journal of Experimental Psychology, 67,
103-112.
1/29/2013 Comp 4020 - HCI 2 (PPI) 61
Fitts Law
Quantify a movement task's difficulty ID, the
index of difficulty
ID = log2(A / W + 1) (bits)
A = amplitude, W = width of object

Movement time to complete a task is predicted

using a linear equation
MT = a + b * ID (secs)
a & b are empirically determined using linear regression

Throughput (TP) or Index of Performance (IP) is

TP = ID / MT (bits/sec)
1/29/2013 Comp 4020 - HCI 2 (PPI) 62
Paul M. Fitts (19121965)
A psychologist at Ohio State University (later at the
University of Michigan). He developed a model of
human movement, Fitts's law, based on rapid,
aimed movement, which went on to become one
of the most highly successful and well studied
mathematical models of human motion. By
focusing his attention on human factors during his
time as Lieutenant Colonel in the US Air Force, Fitts
became known as one of the pioneers in improving
aviation safety. In 1965 he died unexpectedly at the
age of 53.

1/29/2013 Comp 4020 - HCI 2 (PPI) 63

Fitts Law
To determine a and b design a set of tasks with
varying values for A and W (conditions)

For each task condition

multiple trials conducted and the time to execute each is
recorded and stored electronically for statistical analysis

Accuracy is also recorded

either through the x-y coordinates of selection or
through the error rate the percentage of trials selected
with the cursor outside the target

1/29/2013 Comp 4020 - HCI 2 (PPI) 64

Fitts Law

Target 1 Target 2

Same ID Same Difficulty

1/29/2013 Comp 4020 - HCI 2 (PPI) 65
Fitts Law

Target 1 Target 2

Smaller ID Easier
1/29/2013 Comp 4020 - HCI 2 (PPI) 66
Fitts Law

Target 1 Target 2

Larger ID Harder
1/29/2013 Comp 4020 - HCI 2 (PPI) 67
Fitts Law
Device 'A' Device 'B'
A (pixels) W (pixels) ID (bits)
ER (%) MT (ms) ER (%) MT (ms)
40 10 2.32 2.08 665 1.25 1587
40 20 1.58 3.33 501 2.08 1293
40 40 1.00 1.25 361 0.42 1001
80 10 3.17 2.92 762 2.08 1874
80 20 2.32 1.67 604 2.08 1442
80 40 1.58 1.67 481 0.83 1175
160 10 4.09 3.75 979 2.08 2353
160 20 3.17 5.42 823 1.67 1788
160 40 2.32 4.17 615 0.83 1480
Mean: 2.40 2.92 644 1.48 1555

Example data sets for two devices from a Fitts' law experiment
1/29/2013 Comp 4020 - HCI 2 (PPI) 68
Fitts Law

1/29/2013 Comp 4020 - HCI 2 (PPI) 69

Fitts Law
If primary goal in Fitts law experiment is to determine
performance between devices/interaction techniques, then
throughput (TP) is best criterion
TP = ID/MT

If for a given device ID = 4.09 bits and task is executed in MT =

979 ms
human rate of information processing for that task is 4.09 / 0.979 =
4.18 bits/s or TP = 4.18 bits/s

Mean throughput across all the A-W conditions for Device 'A' is
TP = 2.40 / 0.644 = 3.73 bits/s

For Device 'B', TP = 2.40 / 1.555 = 1.57 bits/s

Using throughput we conclude users' performance with

Device 'A' was about 3.73 / 1.57 = 2.4 times better than
performance with Device 'B'

1/29/2013 Comp 4020 - HCI 2 (PPI) 70

Setting it up
MacKenzie, I. S. (1995). Movement time prediction in human-computer interfaces. In
R. M. Baecker, W. A. S. Buxton, J. Grudin, & S. Greenberg (Eds.), Readings in human-
computer interaction (2nd ed.) (pp. 483-493). Los Altos, CA: Kaufmann. [reprint of
MacKenzie, 1992]
http://www.yorku.ca/mack/GI92.html
Vary A,W values for at least 4 ID conditions, 9 is
better
Small A, small W
Small A, large W
Large A, small W
Large A, large W
2-4 variations in between

Clicking start position presents object to click on

Record whether user missed
Record time to click on stimulus

Design with several repetitions and several blocks

1/29/2013 Comp 4020 - HCI 2 (PPI) 71
Case Study: Text Entry Rates on Mobile Phones

Can we predict text entry rate on mobiles using Fitts

Law?

1/29/2013 Comp 4020 - HCI 2 (PPI) 72

Case Study: Text Entry Rates on Mobile Phones

Two main approaches:

Multi-tap:
presses each key one or more times to specify the input
character
large overhead: 33 key presses 15 characters of text
"on average" multi-tap method requires 2.034 keystrokes
per character

77 88 444 222 55 0 22 777 666 9 66 0 333 666 99

q u i c k _b r o w n _ f o x
One-key disambiguation:
Add linguistic knowledge to make best guess
Can be ambiguous in some cases, have to correct

784250276960369
qu ick_ brown_f ox
1/29/2013 Comp 4020 - HCI 2 (PPI) 73
Case Study: Text Entry Rates on Mobile Phones

Text entry on a mobile phone, for example, consists

of aiming for and acquiring (pressing) a series of
keys "as quickly and as accurately as possible

Time to press any key, given any previous key, can

be readily predicted using Fitts' law

For index finger input =

MT = 165 + 52 ID

and for thumb input =

MT = 176 + 64 ID

1/29/2013 Comp 4020 - HCI 2 (PPI) 74

Case Study: Text Entry Rates on Mobile Phones

Elements to build a
text-entry prediction
A B C D Z Space

model are:
Information on position A 0.00002 0.00130 0.00290 0.00360 0.00011 0.00047

and size of keys (ruler)

Letter assignment to
keys (any phone) B 0.00130 0.00013 0.00000 0.00000 0.00000 0.00034

Relative probabilities of
digrams (probabilities C 0.00340 0.00000 0.00012 0.00000 0.00000 0.00044
of letter pairs) in target
language (sources) D 0.00099 0.00001 0.00000 0.00035 0.00000 0.02500

t-h or e-space have high P

g-k or f-v have low P Z 0.00003 0.00000 0.00000 0.00000 0.00008 0.00002

Space 0.01800 0.00960 0.00810 0.00480 0.00002 0.00000

1/29/2013 Comp 4020 - HCI 2 (PPI) 75

 Case Study: Text Entry Rates on Mobile Phones

Time to enter each i-j sequence is predicted using

Fitts law giving MTij, weighted by the probability
of the digram in the target language Pij
MTL = (Pij MTij )
WPM = MTL (60 / 5) (avg 5 chars/word)

Predicted Expert Entry Rate (wpm)

Method
Index Finger Thumb
Multi-tap
- wait for timeout 22.5 20.8
- timeout kill 27.2 24.5
One-key with disambiguation 45.7 40.6
2 assumptions:
- all words are in dictionary
- when ambiguity arises the intended word is the most probable
1/29/2013 Comp 4020 - HCI 2 (PPI) 76
 Case Study: Using Fitts to redesign text entry
Nesbat, S. A System for Fast, Full-Text Entry for Small Electronic
Devices, Proceedings of the Fifth International Conference on Multimodal
Interfaces, ICMI 2003 (ACM-sponsored), Vancouver, November 5-7, 2003.

MessagEase Onscreen Keyboard

Example of an interface design

which can be adapted to multiple
devices

http://www.exideas.com/ME/
http://www.exideas.com/ME/Pressfolder/PressReleaseDec2-2003.html

1/29/2013 Comp 4020 - HCI 2 (PPI) 77

Diverging Text Entry Technologies

Device Text Entry Technology

Cell phones Multi-tap

PDAs and Tablet computers Graffiti, Jot, QWERTY

Email devices QWERTY

TV remote controllers, watches Scroll and Pick

Car navigation systems Several proprietary

Point of Sale devices ABCD, multi-tap

1/29/2013 Comp 4020 - HCI 2 (PPI) 78

Using Letter Frequency

1/29/2013 Comp 4020 - HCI 2 (PPI) 79

Nine Most Frequent Letters: Double Click

E
T
N
R
O
I
A
S
H

1/29/2013 Comp 4020 - HCI 2 (PPI) 80

Eight Less Frequent Letters: Two Taps

D
C
U
P
G
B
Q
J

1/29/2013 Comp 4020 - HCI 2 (PPI) 81

Remaining Nine Letters: Two Taps

F
M
Y
W
V
X
K
Z

1/29/2013 Comp 4020 - HCI 2 (PPI) 82

Adding Space, Shift, and Mode

1/29/2013 Comp 4020 - HCI 2 (PPI) 83

Special Characters

38 special characters entered by two taps;

6000+ characters can be entered with combine.
1/29/2013 Comp 4020 - HCI 2 (PPI) 84
Soft Keyboard Design

The same
mapping
used for
letters

Hard Key Soft Key

Most Frequent Letters Double Click Single Tap

Less Frequent Letters Two Clicks Single Drag

1/29/2013 Comp 4020 - HCI 2 (PPI) 85
Special Characters

Entered with a single drag

1/29/2013 Comp 4020 - HCI 2 (PPI) 86

Optimization and Evaluation
Exhaustively simulated
all permutations of
letters within each group

The configuration with

the max speed was
selected

1/29/2013 Comp 4020 - HCI 2 (PPI) 87

Fitts Law

Movement Time from one

key to another:

MT = a + b*log2(A/W+1)

A W

1/29/2013 Comp 4020 - HCI 2 (PPI) 88

Digraph Probability
A B C D Z Space

A 0.00002 0.00130 0.00290 0.00360 0.00011 0.00047

B 0.00130 0.00013 0.00000 0.00000 0.00000 0.00034

C 0.00340 0.00000 0.00012 0.00000 0.00000 0.00044

D 0.00099 0.00001 0.00000 0.00035 0.00000 0.02500

Z 0.00003 0.00000 0.00000 0.00000 0.00008 0.00002

Space 0.01800 0.00960 0.00810 0.00480 0.00002 0.00000

The probability Pij that letter j will

follow letter i in a body of text:
Pij = 1
1/29/2013 Comp 4020 - HCI 2 (PPI) 89
Performance Measure Hard Key

Calculation of max theoretical entry speed:

Movement Time
MT = a + b log2(A/W+1)
Total time (2 Clicks)
CT = MT1 + MT2
Total time (Dble Click no
movement)
CTDC = 2a + b log2(A/W+1)
Average Time
CTav = (Pij CTij)
Speed
WPM = (1/ CTav) (60/5)
1/29/2013 Comp 4020 - HCI 2 (PPI) 90
Performance Measurement Soft Key

Most frequent characters Single tap:

TLi = (1/4.9) log2 [(D0-i/W) + 1]; if D0-i> 0,
TLi = a; if D0-i = 0

Less frequent characters Drag:

TLjk = t0-j + tdown + tj-k + tup
TLjk = (t0-j + tdown+ tup) +
(tj-k + tup+ tdown)
(tdown+ tup)
TLjk = TLj + TLk a
t0-j: time to move to key j
tj-k: time to move from key j to key k
tdown: time to move stylus down
tup: time to move stylus up

1/29/2013 Comp 4020 - HCI 2 (PPI) 91

Hard Key (Cell Phone) Comparison

Theoretical
130% User Study
30
12
209%
10

20
8
WPM

WPM
6

10 4

0 0

Multi-tap MessagEase Multi-tap MessagEase

1/29/2013 Comp 4020 - HCI 2 (PPI) 92

Comparison with Other Soft Keyboards
60

(all based on Fitts law except for Graffiti)

50
50
43

40 38
36

30
30

20

10
10

0
Graffiti QWERTY Fitaly Opti Metropolis MessagEase

Graffiti

Includes all special

1/29/2013 Comp 4020 - HCI 2 (PPI) characters 93
Comparison with Other Soft Keyboards

Given the same area MessagEase has:

Only 11 keys, but enters more characters
Its Keys are 3-5 times bigger; therefore faster!

1/29/2013 Comp 4020 - HCI 2 (PPI) 94

Advantages and Disadvantages

Disadvantages Advantages
New letter assignment, requires Applicable to both hard-key and Soft-
learning a new pattern key
Full text entry:
Some keys may become cluttered
letters, numbers, special Chars.

Size and language agnostic

Deterministic and unambiguous

One handed (hard-key) operation

1/29/2013 Comp 4020 - HCI 2 (PPI) 95

 Applications:

Any small mobile device that cannot sport a full QWERTY keyboard

Scroll-and-Pick is too slow

One-handed touch typing

1/29/2013 Comp 4020 - HCI 2 (PPI) 96
Application of Fitts law
Fitts law can give a theoretical upper
bound on performance (i.e. without testing
directly with users)

Of course, user evaluation should also

complement Fitts law calculations

1/29/2013 Comp 4020 - HCI 2 (PPI) 97

Example: Expanding targets
Acquisition of Expanding Targets. Proceedings of
ACM Conference on Human Factors in Computing
Systems (CHI) 2002, pages 57-64

1/29/2013 Comp 4020 - HCI 2 (PPI) 98

Example: Expanding targets

Furnas Mackinlay, Robertson, Card

Generalized fisheye views The Perspective Wall
CHI 1986 CHI 1991

Bederson
Fisheye Menus
UIST 2000
1/29/2013 Comp 4020 - HCI 2 (PPI) 99
Mac OS X dock

Size of the interface widget (viewing region)

changes dynamically

Provide the user with a magnified target area at their focus

of attention (area around the cursor)

Expanding toolbar implemented in latest Apple OS X

operating system

Does this make acquisition easier ?

1/29/2013 Comp 4020 - HCI 2 (PPI) 100

Advantages and Disadvantages
Advantages
Icons are displayed in reduced size to solve the
increasing number of commands and icons
Larger amount of screen real estate devoted to
the display of the underlying data

Disadvantages
Can reduce the users ability to select the
desired icon efficiently

1/29/2013 Comp 4020 - HCI 2 (PPI) 101

What does Fitts Law really model?
W
Open-loop

Closed-loop
Speed

Overshoot
Undershoot

Distance

1/29/2013 Comp 4020 - HCI 2 (PPI) 102

 Expanding Targets
Basic Idea:
Big targets can be acquired faster, but take up
more screen space
So: keep targets small until user heads toward
them
Can this be used for devices with small viewing
space?

Click Me !

Okay

Cancel

1/29/2013 Comp 4020 - HCI 2 (PPI) 103

Experiment Goals
The experiment was designed to answer the
following questions for a typical expanding target
selection task:
Can such a task be modeled by Fitts law?
If it can be modeled by Fitts law, is it possible to predict
performance in such tasks from a base set of data where
no expansion takes place?
Is movement time dependent on the final target width
and not the initial one at onset of movement?
At what point should the target begin expanding?
Do different target expansion strategies affect
performance?

1/29/2013 Comp 4020 - HCI 2 (PPI) 104

Experimental Setup
W

Target

Start Position A

1/29/2013 Comp 4020 - HCI 2 (PPI) 105

Experimental Setup
Expansion:
How ?

Animated
Expansion

1/29/2013 Comp 4020 - HCI 2 (PPI) 106

Experimental Setup
Expansion:
How ?

Fade-in
Expansion

1/29/2013 Comp 4020 - HCI 2 (PPI) 107

Experimental Setup
Expansion:
How ?
When ? P = 0.25

1/29/2013 Comp 4020 - HCI 2 (PPI) 108

Experimental Setup
Expansion:
How ?
When ? P = 0.5

1/29/2013 Comp 4020 - HCI 2 (PPI) 109

Experimental Setup
Expansion:
How ?
When ? P = 0.75

1/29/2013 Comp 4020 - HCI 2 (PPI) 110

Pilot Study
7 conditions:
No expansion (to establish a, b values)
Expanding targets
Either animated growth or fade-in
P is one of 0.25, 0.5, 0.75

(Expansion was always by a factor of 2)

1/29/2013 Comp 4020 - HCI 2 (PPI) 111

Pilot Study

7 conditions
x 16 (A,W) values
x 5 repetitions
x 2 blocks
x 3 participants
= 3360 trials

1/29/2013 Comp 4020 - HCI 2 (PPI) 112

 Pilot Study: Results

Time
(seconds)

1/29/2013 ID (index of difficulty)

Comp 4020 - HCI 2 (PPI) 113
 Pilot Study: Results

A
a b log 2 ( 1)
W

Time
(seconds)

1/29/2013 ID (index of difficulty)

Comp 4020 - HCI 2 (PPI) 114
 Pilot Study: Results

A
a b log 2 ( 1)
W

Time
(seconds)
1 A
a b log 2 ( 1)
2W

1/29/2013 ID (index of difficulty)

Comp 4020 - HCI 2 (PPI) 115
 Pilot Study: Results

Time
(seconds) P = 0.25

1/29/2013 ID (index of difficulty)

Comp 4020 - HCI 2 (PPI) 116
 Pilot Study: Results

Time
(seconds) P = 0.5

1/29/2013 Comp 4020 - HCI 2 (PPI) ID (index of difficulty) 117

 Pilot Study: Results

Time
(seconds) P = 0.75

1/29/2013 Comp 4020 - HCI 2 (PPI) ID (index of difficulty) 118

Implications
Pilot Study suggests the advantage of
expansion doesnt depend on P
So, set P = 0.9 and perform a more rigorous
study

1/29/2013 Comp 4020 - HCI 2 (PPI) 119

Full Study
2 conditions:
No expansion (to establish a, b values)
Expanding targets, with
Animated growth
P = 0.9
Expansion factor of 2

1/29/2013 Comp 4020 - HCI 2 (PPI) 120

Full Study

2 conditions
x 13 (A,W) values
x 5 repetitions
x 5 blocks
x 12 participants
= 7800 trials

1/29/2013 Comp 4020 - HCI 2 (PPI) 121

 Results

Time
(seconds)

A, W values

1/29/2013 Comp 4020 - HCI 2 (PPI) 122

 Results

Time
(seconds)

1/29/2013 ID (index of difficulty)

Comp 4020 - HCI 2 (PPI) 123
 Results

Time
(seconds)

1/29/2013 ID (index of difficulty)

Comp 4020 - HCI 2 (PPI) 124
 Results

Time
(seconds)

1/29/2013 ID (index of difficulty)

Comp 4020 - HCI 2 (PPI) 125
 Results

Time
(seconds) P = 0.9

1/29/2013 ID (index of difficulty)

Comp 4020 - HCI 2 (PPI) 126
Implications
For single-target selection task,
Expansion yields a significant advantage, even
when P=0.9

What about multiple targets ?

1/29/2013 Comp 4020 - HCI 2 (PPI) 127

Implications for Design (1)
Experimental results can influence the design of
buttons, menus, or other selectable widgets

Interface with multiple expanding targets does not

need to predict cursor's trajectory to anticipate
which widgets to expand
Instead, just expand widgets as the cursor approaches
them

1/29/2013 Comp 4020 - HCI 2 (PPI) 128

Implications for Design (2)
Expansion Strategies for adjacent
widgets (e.g. toolbars)

Expanding a widget around its center

will cause overlap & occlusion with
nearby widgets
Expanding a group of widgets around a
groups center

Expand nearest widgets and move

adjacent widgets away

Expand nearest widgets, but allow some

overlap as well as expand adjacent widgets
so they are easier to see

1/29/2013 Comp 4020 - HCI 2 (PPI) 129

Summary
Expanding targets acquisition can be accurately
modeled by Fitts Law

User performance is aided by target expansion

Targets that are always expanded can be acquired

just as fast as targets that expand just as the user
reaches them

Implications of results can be applied towards the

design of UI widgets for devices with limited viewing
space
1/29/2013 Comp 4020 - HCI 2 (PPI) 130
Models for Trajectory-Based HCI Tasks
Trajectory tasks are becoming more common
Navigating through nested menus
Drawing curves
Moving in 3D worlds

Cannot be successfully modeled using Fitts law

Steering through tunnel as paradigm to represent

trajectory-based tasks

Beyond Fitts Law: Models for trajectory based HCI tasks.

Proceedings of ACM CHI 1997
1/29/2013 Comp 4020 - HCI 2 (PPI) 131
Beyond pointing: Trajectory based tasks

1/29/2013 Comp 4020 - HCI 2 (PPI) 132

Beyond pointing: Trajectory based tasks
Experimental paradigm focused on is steering between
boundaries (constrained motion)

It appears that the time to produce trajectories sets the

relative speed-accuracy ratio: the larger the amplitude, the
less precise the result is

Want to derive and validate quantitative relationships

between completion time and movement constraints in
trajectory-based task

1/29/2013 Comp 4020 - HCI 2 (PPI) 133

Beyond pointing: Trajectory based tasks
EXPERIMENT 1: GOAL PASSING

A steering task with constraints only at the ends of the

movement
Goal 1 Goal 2

Result: goal passing task follows same law as in Fitts tapping

task

MT = -1347 + 391 log2 (A/W + 1) r2 = 0.987

1/29/2013 Comp 4020 - HCI 2 (PPI) 134

Beyond pointing: Trajectory based tasks
EXPERIMENT 2: INCREASING CONSTRAINTS

What happens if you place more goals along the trajectory?

Allows to formulate a hypothetical relationship of the
steering task

Result: model successful in describing the difficulty of the task

MT = -188 + 78 x ID r2 = 0.968

1/29/2013 Comp 4020 - HCI 2 (PPI) 135

Beyond pointing: Trajectory based tasks

1/29/2013 Comp 4020 - HCI 2 (PPI) 136

Steering Law

1/29/2013 Comp 4020 - HCI 2 (PPI) 137

Example of an application of Steering
Lets consider an application of Steering for
improving an interface

1/29/2013 Comp 4020 - HCI 2 (PPI) 138

Adaptive Activation Area Menus
Erum Tanvir, Jonathan Cullen, Pourang Irani, Andy
Cockburn: AAMU: adaptive activation area menus
for improving selection in cascading pull-down
menus. CHI 2008: 1381-1384.

1/29/2013 Comp 4020 - HCI 2 (PPI) 139

Introduction

Parent
Cascaded/Submenu

140
Steering problems

Corner steering

Off-path errors result in gross inefficiencies

141
Common solution

Impose time delay

Increases navigation time

142
Steering
Steering law

W
D

143
Design criteria
1. Avoid time delay in posting/unposting
2. Reduce steering time
3. Reduce accidental triggers

144
Decrease distance
Gesture posting
Force menus

W W

D D

145
Gesture posting

146
[Kobayashi & Igarashi, 2003]
Gesture Posting (video)

147
Force Menus

148
[Ahlstorm, 2005]
Force Menus (video)

149
Increase width
EMU (Enlarged Activation Area Menus)

W W

150
EMU Enlarged Activation Area Menu

[Cockburn & Gin, 2006] 151

EMU (video)

152
AAMU - Adaptive Activation Area Menu

Increase width adaptively

153
AAMU

154
AAMU

155
AAMU (video)

156
Force-AAMU

Increase width and decrease distance

157
Study
Technique Type
Default, EMU, Gesture Posting, Force Menu, AAMU,
Force-AAMU

Menu Depth
2, 3, 4

Completion time/errors

158
Task
Follow path select red target

159
Completion times

160
by depth

161
Preferences

162
Descriptive Models
Do not yield empirical or quantitative measure of user
performance
Not predictive

Provide a framework/context for thinking methodologically

about a problem or situation

Typically a verbal or graphical description of classes or

identifiable features in an interface
Facilitates categorization

Purpose is to give the designer a tool for studying and thinking

about user-interaction experience

Two examples:
KAM
Guiards Theory of Bimanual Interaction

1/29/2013 Comp 4020 - HCI 2 (PPI) 163

Key-Action Model
Keyboards are essential in interacting with a
computer

101-keys categorized by function keys, numeric

keys, characters, control keys, etc.

Define a descriptive model referred to as Key-

Action Model (KAM) as follows:
Symbol keys deliver graphical symbols (a,z,1,?)
Executive keys invoke actions in the application or at the
system level (ENTER, F1, ESC)
Modifier keys set up a condition to modify effect of
pressing a key (ALT, SHIFT, CTRL)
1/29/2013 Comp 4020 - HCI 2 (PPI) 164
Key-Action Model
Some questions to consider regarding this model:
Is the model correct?
Is it flawed?
Do all keyboard keys fit the model?
Are there additional categories or sub-categories?
Can it be improved to become more comprehensive?
Is the model really useful?

1/29/2013 Comp 4020 - HCI 2 (PPI) 165

Guiards Model
Originated from the area of motor behavior
referred to as bimanual control or laterality

Both hands perform a different set of tasks

Given this knowledge and handedness of people,

interesting to evaluate how interaction
accommodates best the division

Results in descriptive model of bimanual skill, given

by Guiard in 1987 paper
Guiard, Y. (1987). Asymmetric division of labor in human skilled bimanual action:
The kinematic chain as a model. Journal of Motor Behavior, 19, 486-517.
1/29/2013 Comp 4020 - HCI 2 (PPI) 166
Guiards Model

Hand Role and Action

leads the preferred hand

Non-preferred
sets the spatial frame of reference for the preferred hand
performs coarse movements

follows the non-preferred hand

Preferred works within established frame of reference set by the non-
preferred hand
performs fine movements

1/29/2013 Comp 4020 - HCI 2 (PPI) 167

Guiards Model
Example:
a right-handed artist sketches a design of car
acquires a template with left hand (non-preferred hand
leads)
template is manipulated over the workspace (coarse
movement, sets the frame of reference)
Right hand picks stylus (preferred hand follows) and placed
close to the template (works within frame of reference set
by the non-preferred hand)
Artist sketches (preferred hand makes precise movements)

1/29/2013 Comp 4020 - HCI 2 (PPI) 168

Guiards Model
People naturally gravitate to using two hands

Performance times are reduced

Can be used for interfaces that employ:

Drawing designs
Fabricating virtual objects
Positioning
Reshaping

1/29/2013 Comp 4020 - HCI 2 (PPI) 169

Bimanual Control and Desktop Computer Affordances

How does the distribution of keys on a keyboard

facilitate task division between right/left hands?
where does interaction with the mouse fit into the model?

Right side bias toward power keys (executive keys +

modifier keys, marked in red dots)
1/29/2013 Comp 4020 - HCI 2 (PPI) 170
Bimanual Control and Desktop Computer Affordances

Dominance on right hand side,

good for the 80s but how does this
work with GUIs and pointing
devices that are now
commonplace?

Right handed have to reach over

with left or leave the mouse

Is there an advantage for left-

handed users?

1/29/2013 Comp 4020 - HCI 2 (PPI) 171

Bimanual Control and Desktop Computer Affordances

Trailing/Overlapping
Task Leading Movement
Movement
Left hand manipulate pointer
Right hand press DELETE
Delete with mouse and select text/object
(probably with little finger)
by double clicking or dragging

Select an option Left hand manipulate pointer Right hand press ENTER
in a window with mouse and click on an option (Note: OK button is the default)

Right hand navigate to link via Left hand manipulate

Click on a link in a
PAGE UP and/or PAGE DOWN pointer with mouse and select
browser
keys link by clicking on it
Open a file, open Right hand press ENTER
Left hand manipulate pointer
a folder, or launch (Note: avoids error prone
with mouse and single click on icon
a program double-click operation)

Common tasks performed by a left-handed user manipulating mouse in the left hand

1/29/2013 Comp 4020 - HCI 2 (PPI) 172

Bimanual Control and Desktop Computer Affordances

Tasks described previously are faster for left-handed

users than right-handed users

When pointing is juxtaposed with power key

activation (excluding SHIFT, ALT, & CONTROL), the
desktop interface presents a left-hand bias

1/29/2013 Comp 4020 - HCI 2 (PPI) 173

Bimanual Control and Desktop Computer Affordances

Scrolling typically accomplished by dragging elevator

of scrollbar along the right-hand side of an applications
window

Takes up to 2 secs per trial and is obtrusive and non-

transparent

In perspective of Guiards model of bimanual control, we

can delegate scrolling to non dominant hand

Task Characteristics

precedes/overlaps other tasks

Scrolling
sets the frame of reference
minimal precision needed (coarse)

Selecting, editing, reading, follows/overlaps scrolling

drawing, etc. works within frame of reference set by scrolling
demands precision (fine)
1/29/2013 Comp 4020 - HCI 2 (PPI) 174
Redesigning the Scrolling Interface

1/29/2013 Comp 4020 - HCI 2 (PPI) 175

What did we cover
Text entry rate prediction on mobile phones

Application of Fitts, expanding targets

MessagEase: a commercial application making use

of Fitts law for improving text entry rates

Bimanual control and analysis of task division

between right and left hands
Application to guide design choices

1/29/2013 Comp 4020 - HCI 2 (PPI) 176