Lecture 4:

History of Mobile Devices and

their Interaction Techniques
Brad Myers

05-440, 05-640: Interaction Techniques

Spring, 2022

© 2022 - Brad Myers 1

Logistics
 Grades on quiz are on Canvas
 Will drop your lowest quiz grade
 I added point counts to HW1 description
 Reminder: back to in-person on Tuesday
 Like fall semester – wear masks, no eating in class
 Will be recorded in case miss a class
 But recordings might not be wonderful, e.g., when write on the
board
 Note that the required “reading” will be starting to include
videos
 Future “pop” quizzes won’t be announced
 So you should read the book chapters and watch the videos
before lectures 2

© 2022 - Brad Myers

“Computers”

(cite,
slide 24, 25)

© 2022 - Brad Myers

 Definitions
 Mobile device - a computing device small enough to hold
and operate in the hand
 Not a laptop
 Handheld device – same
 But “Handheld PC” is specifically
a clamshell mobile device:
 Palm-size device
 Tablets – mobile but bigger
 Wearable devices – watch, glasses, ring, …
 Portable device – includes laptops
 Touchscreens – can be on non-mobile devices
 Touchpads – not a screen
 Pen computing – requires stylus 4

© 2022 - Brad Myers

Early Handwriting Input
 Handwriting recognition has
been an active research
topic since 1960’s:
 Rand Tablet: 1964:
http://www.rand.org/content/dam/rand/pubs/research_memoranda/2005/RM4122.pdf

 Early: hand printing

 Lots of work on handwriting and
gestures
 E.g., W. Buxton, E. Fiume, R. Hill, A. Lee, C. Woo,
“Continuous hand-gesture driven input,” Graphics
Interface '83 (1983), pp. 191–195

© 2022 - Brad Myers

Programmable Calculators
 The first programmable pocket calculator was the HP-
65, in 1974 – Wikipedia
 First graphing calculator was the Casio FX-
7000G released in 1985
 Continued to improve and get cheaper through 80’s
and 90’s
 HP and TI
 HP used reverse polish
notation (RPN) = postfix
 No need for parentheses:
4 5 + 6 * instead of (4+5)*6 6

© 2022 - Brad Myers

 “Ubiquitous Computing”
 Term coined by Mark Weiser
at Xerox PARC, 1988
 Mark Weiser. “The Computer for the 21st
http://en.wikipedia.org/wiki/File:Mark_weiser.jpg
Century”, Scientific American, 94-104, Sep 1991.
 Mark Weiser. “Some Computer Science Issues in Ubiquitous Computing,” CACM. July,
1993. 36(7). pp. 74-83.
 (Died at 46 in 1999 of cancer)
 “I called these three sizes of computers boards, pads, and tabs, and adopted the
slogan that, for each person in an office, there should be hundreds of tabs, tens of
pads, and one or two boards.” [p. 76]

© 2022 - Brad Myers

PARC Tab
 ~1989
 Low speed wireless network using IR
 Touch-sensitive screen
 Quick writing – unistrokes, write on top of each other
 David Goldberg and Cate Richardson. “Touch Typing with a Stylus,” Human Factors in
Computing Systems, Proceedings INTERCHI'93. Amsterdam, Netherlands, Apr, 1993.
pp. 80-87.

© 2022 - Brad Myers

Go Corp’s “PenPoint” OS
 Founded 1987, released in 1991
 One of the founders was Robert Carr
from Xerox PARC; Alto designer
 Hardware by NCR, IBM and EO
 Styled to look like a tabbed notebook
 Conventional tapping on menus
 Lots of gestures for editing,
page turning, etc.
 Flick to scroll and turn pages, circle, insert
space, cross-out, insert word, get help, …
 Press and hold to start moving or selecting
 Hand printing for text entry
 Hyperlinks
 Instant on-off 9

© 2022 - Brad Myers

PenPoint
 User’s guide
 http://research.microsoft.com/en-us/um/people/bibuxton/buxtonco
llection/a/pdf/
Go%20PenPoint%20Getting%20Started.pdf

 Pictures:
http://www.digibarn.com/collections/systems/go/index.html

 Video: 10

 http://www.youtube.com/watch?v=x0XE08BjQDQ
© 2022 - Brad Myers
PenPoint
 Handprinting into boxes
 Proofreading marks for
editing – like insert space
 Needed to print neatly!

Source: pp. 13-14 of

https://www.microsoft.com/buxtoncollection/a/p
df/go%20penpoint%20getting%20started.pdf?fr
om=https%3A%2F%2Ffanyv88.com%3A443%2Fhttp%2Fresearch.microsoft.com
%2Fen-us%2Fum%2Fpeople%2Fbibuxton%2F
buxtoncollection%2Fa%2Fpdf%2Fgo%2520pe
npoint%2520getting%2520started.pdf

11

© 2022 - Brad Myers

GRiDPad
 Jeff Hawkins
 1989
 http://www.computerhistory.org/revolution/mobile-computing/18/319/1727
 under 5 lbs
 386SL 20MHz processor with an 80387SX coprocessor
with 20MB RAM and 40, 60, 80 or 120MB hard drive. It
had a 10" diagonal backlit VGA display with 32 gray
scales. There was a built in PCMCIA card slot, an internal
fax/modem card, a floppy drive
port and a standard keyboard
port. Operating time was about
3 hours on NiCad battery pack.
http://www.digibarn.com/collections/systems/gridpad/index.html

12

© 2022 - Brad Myers

 Microsoft Pen Windows
 From: 1991
 Version of Windows 3.1 for pen computing
 Added handwriting or handprinting recognition
 Versions for Windows NT, Windows 95, etc.
Images: http://retrocosm.net/2012/01/, http://www.betaarchive.com/imageupload/1298947809.or.94950.png

13

© 2022 - Brad Myers

 Apple Newton
 Started 1987, released 1993
 Newton “MessagePad”
 Coined term “Personal Digital
Assistant (PDA)
 Note: PDAs are not phones
 Was on sale for 6 years
 Fairly large & heavy
John Sculley III
 Interesting OS using an interpreted
programming language: NewtonScript
 “Prototype-Instance” OO model like JavaScript
 English-like similar to HyperTalk 14

© 2022 - Brad Myers

Apple Newton
 Key issue: handwriting recognition was main input
technique
http://www.computerhistory.org/revolution/mobile-computing/18/319/1714
 Often not successful
 Famously panned for an entire week by Doonesbury
(August 1993)

15

© 2022 - Brad Myers

 General Magic’s “Magic Cap” OS
 1994
 Ran on Sony MagicLink
hardware
 Object-oriented OS for
PDAs
 3D Room metaphor
 Special AT&T wireless
network (very slow)
Pictures: http://en.wikipedia.org/wiki/File:Magic_Cap_OS.gif
http://en.wikipedia.org/wiki/File:SonyMagicLink.jpg
16

© 2022 - Brad Myers

 Early mobile phone UIs
 Feature Phones – some functions, like address book
 Keypad (12 buttons)
 1993 – first Nokia soft keys & scrolling
 Standardized on 2 or 4 directions,
2 action keys
 Motorola Razr – 2004
 Thinner is better
 Text entry by multi-tap or T9
 Note: not touch screens
 Covered in lecture 8
 WAP – starting 1997
 Wireless Application Protocol
 Bring web-like access to these
devices
 Terrible usability 17
 Nielsen study © 2022 - Brad Myers
Early mobile phone + PDAs
 Mobile phone + PDA = smartphone
 Usually a full keyboard (not just keypad)
 IBM Simon
 Shipped in 1994 by BellSouth
 Nokia 9110 Communicator
 1996
 Added full physical keyboard
 Typical PDA features:
 Address book, calendar
 Slow processor
and network (2G)
18

© 2022 - Brad Myers

Palm
 Founded by Jeff Hawkins who did
GridPad
 US Robotics (1995), 3Com (1997),
Handspring (1998), Palm (2000), HP
(2010)
 First released version: 1996 = “Pilot”
 Name changed due to lawsuit
 They did lots of user testing with
prototypes created using HyperCard
 Graffiti for data entry
19

© 2022 - Brad Myers

Palm Graffiti
 Jeff Hawkins had seen Xerox
QuickWriting
 Lawsuit
 Designed to be easier to learn
 Still required practice
 Unistroke except for “X”
 Two sides – numbers look the
same as some letters

20

© 2022 - Brad Myers

 Palm’s design Principles
“Designing the Palm Pilot: A conversation with Rob Haitani”, by Eric Bergman and Rob Haitani, chapter 4 in Information
Appliances and Beyond, Eric Bergman, ed. (2000)

 Fast access to key features on small screens ->

 Only a few commands used a lot
 Leave commands off main screen, even if not symmetric
 new vs. delete
 (think stapler and stapler remover)
 Note that violates consistency
 Tap and then type in schedule and to-do
 Only four buttons – which ones?
 Vs. Windows CE -> if know PC, this is familiar
 But usage models are different
 PC: infrequent long usage
 Palm: frequent short bursts of usage
21

© 2022 - Brad Myers

Palm Watch
 Fossil,
 Announced 2002, shipped 2003-5
 160 x 160 illuminated screen
with a stylus integrated into the band,
8MB internal memory, rechargeable
battery and
standard Palm platform features
 $250
 Heavy, short battery life, tiny stylus
22

© 2022 - Brad Myers

Palm Phones
 Kyocera QCP-6035 about 2001
 Physical phone buttons, or regular
Palm
 Low-speed internet
 Handspring (then Palm) Treo
 Blackberry-like keyboard
replaces Graffiti
 Starting 2002

23

© 2022 - Brad Myers

Windows CE
 CE 1.0 released in 1996 (same year as 1st PalmPilot)
 Many names: Windows Compact Edition (WinCE), Windows Palm PC, Windows
Pocket PC (PPC), Windows Handheld PC (HPC), Windows Mobile,
WindowsPhone
 May still be available for purchase?
 HPC for landscape devices with a keyboard, PPC for portrait
 Similarities to Windows, but different OS
 Instant on
 Different UI interactions
 Compaq iPaq became very popular (2000)

24

© 2022 - Brad Myers

 Studies for Original Windows CE
“The Interaction Design of Microsoft Windows CE”, by Sarah Zuberec, chapter 5 in Information Appliances and
Beyond, Eric Bergman, ed. (2000)
 Studies: minimum target: stylus = 5.04mm2, finger = 9.04mm2
 Drag between down and up for “tap” = 2mm
 Many usage scenarios
 User tests identified Tahoma 10 bold as best system font, but
couldn’t be used because not enough content fit in the dialogs
 So used Tahoma 9
 Novice users did better with keyboard, but experts preferred
character recognizer
 Problem with initial designs: too many taps
 Achieved “walk up and use” but too slow for experts
 Double tap with stylus difficult and unnatural
 “Consistency worked against learning and use.”
25

© 2022 - Brad Myers

 RIM Blackberry
 Starting 1999
 Research in Motion (RIM)
 Blackberry 850
 Email & pager
 Originally, proprietary network
 Key features:
 Two-thumb keyboard
 Roller dial (“scroll wheel”) for navigation
 Moved to side of device
 Eventually, became 2D navigation
 Later, regular phone networks
 Awkward attempts at full-screen
touchscreen
 Attempted to be backwards
compatible with old applications
 Insufficient 3rd party applications
26
 Late to have good APIs
© 2022 - Brad Myers
 Windows TabletPC
 2001 spec (Windows XP), first
devices in 2002
 Bill Gates said it would be big (2002)
 Handwriting recognition was much
better, but still not sufficiently
accurate
 Windows UI not
changed for pen
 Lower accuracy than mouse
 Quite poor UIs for
correction 27

© 2022 - Brad Myers

 2G, 3G, 4G, GSM, CDMA, etc.
 1G = analog, 1980s
 2G = GSM digital data, 1992, CDMA version followed
 About 40 kbit/s
 3G = about 2001 – hybrid analog & digital
 200 kbit/s
 4G = about 2008
 100 Mbit/s, up to 1 Gbit/s
 5G = 2019 or 2020 start, ~20 Gbps
  Dates Data Standards Features
speeds
1G 1980s None AMPS, TACS Analog transmission
2G 1992 40 kps up to GSM, CDMA, Digital for data,
236.8 kps EDGE, GPRS analog for voice;
SMS
3G 2001 200 kps up UTMS, CDMA2000, Packet switching for
to 384 kps HSPDA, EVDO data
4G 2008 100 mbs, up LTE Advanced, IEEE All digital 28
to 1 gbs ©802.16
2022 - Brad (WiMax)
Myers
Wifi and BlueTooth
 Wifi – from 1988
 Officially IEEE 802.11
 Whole family: 802.11a, b, g, n …
 Originally called “WaveLan”
 CMU was first fully wireless campus starting in 1997 = “wireless Andrew”
 “WiFi” trademark in 1999

 BlueTooth started by Ericsson in 1994

 Standardized as IEEE 802.15 in 2002 and 2005
 Name from 1997
 Named for Danish tenth-century king
 Short range, exactly 2 devices
 Original use: phone to earpiece
 Now, mice, keyboards, etc.
29

© 2022 - Brad Myers

iPod
 2001
 Apple iPod lauded for design
and user interface
 Unique dial interaction technique
 Enabled easy access to thousands
of songs
 Highly tuned speed ratio
 iTunes (2003)  entire service design
 5 GB hard drive that put “1,000 songs in your
pocket.”
30

© 2022 - Brad Myers

iPhone
 Starting 2007
 Went against the conventional wisdom in
many aspects
 --- what? ---

31

© 2022 - Brad Myers

iPhone
 Starting 2007
 Went against the conventional wisdom in many
aspects
 No blackberry-style keyboard
 Capacitive screen (multi-touch)
 No stylus
 Only one button – focus on
easy to use
 Some unique interaction
techniques
 --- what? --- 32

© 2022 - Brad Myers

iPhone
 Starting 2007
 Went against the conventional wisdom in many
aspects
 No blackberry-style keyboard
 Capacitive screen (multi-touch)
 No stylus
 Only one button – focus on
easy to use
 Some unique interaction
techniques
 Scroll bounce, swipe login, … 33

© 2022 - Brad Myers

 Android
 Unveiled 2007, first phone in 2008
 Google offers it free to phone
manufacturers
 Open source
 Based on Linux and Java
 About 700,000 different device types
 Hundreds of screen sizes
 3 main (virtual) buttons
34

© 2022 - Brad Myers

Phone MarketShare

http://en.wikipedia.org/wiki/File:World_Wide_Smartphone_Sales_Share.png
35

© 2022 - Brad Myers

iPad
 2010
 Very different from TabletPC
 Media machine
 Little text entry facilities
 Interactions same as a Phone,
instead of mimicking a PC
 Focuses on ease of use

36

© 2022 - Brad Myers

Many other devices not
covered
 Personal organizers
 Casio, Sharp, etc.
 Book readers (Amazon Kindle, etc.)
 Custom devices for vertical markets
 Warehouses, doctors, etc.

37

© 2022 - Brad Myers

Timeline http://whenintime.com/tl/bradamyers/Handhelds/

38

© 2022 - Brad Myers