Lecture 2:

Pointing Devices and Fitts’ Law

Brad Myers

05-640: Interaction Techniques

Spring, 2022

© 2022 - Brad Myers 1

Logistics
 Homework 1 starts today
 Full instructions are now on the HW#1 page
 There may be quizzes on the required readings before any lecture,
even if remote
 Recordings of videos and transcript and chat on Canvas and Google
Drive (not public)
 Links emailed – ask me if you need them
 Will set up Piazza for discussions and questions
 Really do want feedback on IxT book
 V34 of pdf of book uploaded
 Comments on pdf, emails with typos, etc.
 Waitlist – getting bigger! – 20 people now
 Can stay for 2 weeks, but not when back in person - sorry
2

© 2022 - Brad Myers

Pointing Device
 Pointing Device - allows a user to indicate a
particular location or object of interest,
typically as a prelude to doing some
operation
 Only 2D pointing for now (lecture 15 is 3D)

© 2022 - Brad Myers

Some Pointing Devices
 (List from HW#1):
 Mouse
 Laptop touchpad
 IBM Pointing Stick on Thinkpad laptops
 Touchscreen with fingers (phones, tablets)
 Touchscreen with Stylus
 Wii controller pointer in the air, pointing at a web page on a "smart TV"
 Microsoft Kinect using your hand to point at a web page on a "smart TV"
 Large "Smart Board" direct touch wall-size display
 Game controller connected to a PC to control the cursor
 Contour's "RollerMouse Red plus"
 Trackball
 Others
 Step keys
 Light pen
 Laser pointer at a screen
 Eye tracking
 Joysticks 4
 Game consoles © 2022 - Brad Myers
Cursor

 Pointer Cursor: Shows where the user is pointing

 Sometimes called the “mouse cursor”
 Text cursor: shows where the next character will go
 Originally were in the same place
 Now, are usually in different places
 Is there a cursor on a smartphone?

© 2022 - Brad Myers

The “Selection”
 The item that is selected is different from the pointer
or text cursor
 The item or items that will be affected by the next command
 Typically, only 1 active selection at a time
 Selection is usually connected to text cursor
 Text cursor usually between characters
 Older systems had text cursor as a single selected
character

© 2022 - Brad Myers

Pointer “Buttons”
 Need to signal when the pointer position is of interest
 Mouse buttons, or touch on screen, or button on the side of a stylus
 Press = “down”, release = “up”
 JavaScript calls it: MouseDown/MouseUp
 Can differentiate left-xxx vs. right-xxx on a mouse
 Press & release = “click”
 Press & hold
 Press & move = “drag”
 Press & up & press & up = double-click
 May or may not need to be quickly or in the same place
 Triple-click, etc.
 Can modify with keyboard keys, e.g., shift-click, control-drag
 Touch screen – use “Touch”, “Tap”
 JavaScript uses TouchStart/TouchEnd
 Count of number of fingers – first touch, second touch 7

© 2022 - Brad Myers

 Lightpen
 Starting ~ 1950s
 Sketchpad, 1963
 Camera in pen looks for
pixel on screen
 Calculates
position based
on timing
 Disadvantages:
 Low resolution
 Need to hold
hand in the air
 “Gorilla arm” 8
Photo credit: https://design.osu.edu/carlson/history/lesson2.html © 2022 - Brad Myers
Step Keys
 Usually to move text cursor
 Sometimes can move
pointer cursor as well
 Microsoft Windows, can
move window with ALT-
Space, “m”, arrow-keys
 Moves pointer cursor as well

© 2022 - Brad Myers

 Mouse
 Bill English and Doug Engelbart
credited with the invention of the
mouse (1967) at SRI
 Comfortable, fast and accurate
 Engineered to not move when
push a button
 Cursor moves straight as hand
moves in an arc

10

© 2022 - Brad Myers

 Mouse Gain Functions
 As the mouse moves quicker, the speed of
the cursor increases more
 Non-linear curves

Gery Casiez, Daniel Vogel, Ravin Balakrishnan

and Andy Cockburn. “'The Impact of Control-
Display Gain on User Performance in
Pointing Tasks,” Human-Computer Interaction.
2008. vol. 23, no. 3. pp. 215-250. 11
http://dx.doi.org/10.1080/07370020802278163
© 2022 - Brad Myers
“Clutching”
 When move a mouse to the edge of the area,
raise it off the table and move = “clutching”
 Like using the clutch on a car to disconnect the
engine from the wheels
 Also for touchpads
 Not on touchscreens

12

© 2022 - Brad Myers

Joysticks
 Often used in aviation
 Computer input for games
 Also on consumer-electronics, e.g., cars
 Most are “rate controlled”
 Self-centering springs
 Movement controls rate of cursor movement
 “isometric” – stick doesn’t seem to move
 Position controlled
 Absolute position of stick controls position
 No spring – stick stays where leave it
13

© 2022 - Brad Myers

Trackballs
 Like an “upside down” mouse
 Typically use any finger to move
 Always “relative”
 Gain-function like a mouse
for speed
 No need for clutching
 Good for accessibility
 I always use a trackball due
to tendonitis in my right index finger
14

© 2022 - Brad Myers

IBM Pointing Stick
 See lecture from 2014 from Ted Selker
 “Trackpoint”
 Rate-controlled isometric joystick
 Significant iteration and experimentation
 “10 years of human factors work”
 Material of the stick matters
 Placement of buttons
 Transfer (gain) function
 Training helps performance
15

© 2022 - Brad Myers

 Graphics Tablets
 Rand Tablet: 1964: http://www.rand.org/content/dam/rand/pubs/research_memoranda/2005/RM4122.pdf

 “Digitizer”
 Handwriting and graphics input
 CAD/CAM
 Move stylus or puck or finger on a surface – not a
display screen
 Many can measure force of press
 Angle of pen

16

© 2022 - Brad Myers

 Touchscreens
 Stylus versus finger
 Tradeoffs:
 Naturalness
 Accuracy
 Amount of content that
can fit on the screen
 Blocking the view of the
content
 Use to have a “loupe” interaction
 How hold a tablet?
 Palm & hand removal?

17

© 2022 - Brad Myers

Large Touchscreens
 Projector or large TV
 Vertical or tabletop
 Smartboard
 Original Microsoft
“surface”
 Single or multiple people
 Classrooms, museums,
small meetings, etc.
 Microsoft Surface
Hub - 2015
18

© 2022 - Brad Myers

 Remote interaction with
large screens
 Examples
 Laser pointing at screen
 Brad A. Myers, et. al. "Interacting At a Distance: Measuring the
Performance of Laser Pointers and Other Devices." CHI'2002.
pp. 33-40. pdf.

 Wii controller
 Microsoft Kinect – just fingers
 Very inaccurate
 Tiring

19

© 2022 - Brad Myers

 Issues
 Direct vs. Indirect
 Direct – point at a screen
 Light-pen, stylus, finger
 Often no visible cursor
 + More natural & faster
 - Low res; obscured; jitter Image credit:
http://www.drdigitizer.com/learn.html
 Indirect – movement moves a cursor
 +Must have a cursor – Shape can be used as feedback
 -Must be learned
 +Transfer (gain) function – can be more accurate and faster
 +Software can move position of the cursor
 Issue – direction of movement (up or down is arbitrary)
 Absolute vs. Relative
 Absolute – required for Direct device = touchscreen
 Touchpads can be either
 Relative – only movement used = mouse
20

© 2022 - Brad Myers

 Issues, 2
 How many states supported?
 Can have “hover” state?
 Pointing but not “pressed” or “selected”
 Common with mouse, not with touchscreen
 Is possible with magnetic stylus on touchscreen
 Location but not pressed
 How many buttons?
 Stylus buttons on side of pen
 Other dimensions, like force of press – “3D touch”
 Single or multiple touch (multiple fingers)
 Resistive vs. capacitive vs. other sensing
 For big displays – which person’s hand?
21

© 2022 - Brad Myers

Testing
 How decide whether an input device is “better” than
another?
 What are the important measures?

22

© 2022 - Brad Myers

Testing
 How decide whether an input device is “better” than
another?
 Important measures: speed, accuracy, comfort,
learnability
 As usual in HCI, need to decide tasks and relative
importance of measures
 Often (but not always) will have tradeoffs
 Focus on speed and accuracy
 Since are numeric, can use standard statistical measures
 JMP, R, SPSS, Excel, and others to help with calculations
 Fitts’ law
23

© 2022 - Brad Myers

Fitts’ Law
 1954, Paul Fitts

 Figure out how quickly people could move a

pointing device to a target and select it
 Predictive model
 e.g., keystroke-level analysis

 To compare pointing devices

 Throughput – combines both speed and accuracy

24

© 2022 - Brad Myers

 +

Cognitive Processes + Physical Processes

=
Pointing Performance

25

© 2022 - Brad Myers

 Laser Pointer
Example

Source:
26
hcibook.com
© 2022 - Brad Myers
Card, Moran, Newell
studies
 Card, English, Burr 1978
paper – distance
 Time is linear for distance for
step keys
 Time increases with the log of
the distance for continuous
devices like the mouse

27

© 2022 - Brad Myers

Old pictures

Tom Moran

Bill Curtis, Stu Card, and Allen Newell

Source: SIGCHI Archives

28

© 2022 - Brad Myers

 Card, Moran, Newell studies, 2
 Card, English, Burr 1978
paper – width
 Positioning time for both
the mouse and the
joystick decreases
with the log of the
target size

29

© 2022 - Brad Myers

 T = a + b log2(D/W + 1)
T = time to complete a movement
a = fixed cost to start/stop moving &

click
b = inherent speed of device
log2(D/W + 1) = “index of difficulty” or ID
bits
 Higher with distance (D) and lower with width 30

(W) © 2022 - Brad Myers

 ID = log2(D/W + 1)

“double the distance, double the width”

equals

31

© 2022 - Brad Myers

Card, English, Burr 1978

32

© 2022 - Brad Myers

Homework 1 Analysis
 You can actually do this with your data!
 Compute IDs from Distance and Width
 Chart them and see if they are linear
 Use Excel “Fit Trendline” to get coefficients
for a and b and compare to other papers

33

© 2022 - Brad Myers

 Newer Fitts’ Law tests
 Use circles instead of two rectangles
 ISO 9241-9 standard
 Doesn’t fit as well on non-square screens
 Horizontal and vertical movements may not be equal
difficulty
 Muscles used
 Card, English, Burr 1978 paper showed differences for joystick,
etc. but not mouse
 Laser pointer study: up to 10x more wiggle vertically
 Device properties
 Contour's "RollerMouse Red plus“
 Even for a trackball

34

© 2022 - Brad Myers

Homework 1 for 2022
 TAs Chaoran Chen (2022) & Asit Parida (2019) created
HW #1 test program using circles
 Stores summary of results onto class Google Sheet
 So please log in with Andrew ID
 Even though gives scary warnings
 Your data as CSV, JSON, on-screen

 Extra credit for doing “interesting” device

 Besides a tablet, phone, mouse, touchpad
 Note: devices must run a browser to do the test (so not a watch?)
 Who has a stylus? Who has access to a large display?
 Who wants to borrow one of mine?
 Maybe 2 people on each – get to other person in time? 35

© 2022 - Brad Myers

Homework 1 Analysis
 Keep in mind that the papers you read collected
MUCH more data
 E.g., 1200 to 1800 trials (four to six hours) before
learning curve flattened out – [Card, 1978]
 It’s okay to not have any measurable difference
as long as you explain why that’s reasonable
 Check out the error rates too
 Is horizontal different from vertical?
 The laser pointer paper is good example of how
to structure a report
36

© 2022 - Brad Myers

Pointing Devices Available for
Borrowing
 If you are in Pittsburgh, and can stop by my
office – NSH 3513
 1 Wacom tablet
 1 trackball
 1 Contour RollerMouse Red
 2 LeapMotion devices
 1 tiny mouse
 About 7 regular mice
 What do you have?

37

© 2022 - Brad Myers

 Research on Pointing:
Bubble Cursor
 Tovi Grossman and Ravin Balakrishnan. “The bubble cursor: enhancing target acquisition by
dynamic resizing of the cursor's activation area,” Proceedings of CHI'05: the SIGCHI
Conference on Human Factors in Computing Systems. Portland, OR, ACM. 2005. 281–290.

 Make cursor into a circle big enough to capture

exactly 1 target
 Called “area cursor” (instead of point)
 Dynamically changes
size based on density
of targets
 Reduces Fitts’ law size
and distance to target

38

Video from: https://hci-museum.lri.fr/bubble-cursor

© 2022 - Brad Myers
 Research on Pointing:
Drag-and-Pop
 P. Baudisch, E. Cutrell, D. Robbins, M. Czerwinski, P. Tandler, B. Bederson and A.
Zierlinger. “Drag-and-Pop and Drag-and-Pick: Techniques for Accessing Remote
Screen Content on Touch- and Pen-operated Systems,” Proceedings of Interact 2003.
Zurich Switzerland, August, 2003. 57-64.

 Bring the objects in the direction of

movement closer to the cursor
 Reduces the
Fitts’ law distance
 Same size

39
Video from: https://hci-museum.lri.fr/https://hci-museum.lri.fr/drag-and-pop

© 2022 - Brad Myers

Research on Pointing
 Most current research is focused on pointing
in 3D and VR/AR
 See lecture 15

40

© 2022 - Brad Myers