Unit- 2 Computers in HCI

Input Technologies and Techniques, Sensor- and Recognition-Based Input for

Interaction, Visual Displays, Haptic Interface, Non-speech Auditory and
Cross modal Output, Network-Based Interaction, Wearable Computers,
Design of Fixed, Portable, and Mobile Information Devices
 Unit- 2 Computers in HCI

Input Technologies and Techniques, Sensor- and Recognition-Based Input for

Interaction, Visual Displays, Haptic Interface, Non-speech Auditory and
Cross modal Output, Network-Based Interaction, Wearable Computers,
Design of Fixed, Portable, and Mobile Information Devices
 Input Technologies and Techniques- Terminology

1. Input device- An input device is a transducer that senses physical properties of people,
places, or things.

2. Conceptual model- A conceptual model is a coherent model that users visualize about the
function of a system—what the system is, how it works, and how it will respond to users’
input.

3. Interaction technique- An interaction technique is the fusion of input and output, consisting
of all hardware and software elements, that provides a way for a user to accomplish a task
for a particular conceptual model.

4. User interface- A user interface is the representation of a system—the summation of all its
input devices, conceptual models, and interaction techniques— with which a user interacts

5. User experience
 Input Technologies and Techniques- Input device properties
-These properties help a designer in understanding a device and in
anticipating potential problems.
1. Property sensed: Most devices sense linear position, motion, or force; rotary devices
sense angle, change in angle, and torque. For example, touch screens sense the position
of one or more fingers, mice sense motion (change in position), and isometric joysticks
sense force.

2. States sensed: Direct-input devices, such as touch screens, touchpads, and pen input
devices, are unique in that they sense not only position but also contact events (i.e.,
finger-down and finger-up events), an ability that traditional pointing devices lack
 Input Technologies and Techniques- Input device properties
-These properties help a designer in understanding a device and in
anticipating potential problems.
3. Number of dimensions: Devices sense one or more input dimensions. For
example, a mouse senses two linear dimensions of motion, a knob senses one
angular dimension

4. Device acquisition time: The average time to move one’s hand to a device is
known as acquisition time. Homing time is the time to return from a device to a
“home” position (e.g., return from mouse to keyboard).

5. Other metrics: pointing speed and accuracy, error rates, learning time, footprint
and gain, user preference, comfort, and cost. Other important engineering
parameters include sampling rate, resolution, accuracy, and linearity
 Input Technologies and Techniques- DIRECT versus INDIRECT input
devices

1. Direct devices such as touchscreens, or display tablets operated with a pen,

are not necessarily easier to use than indirect devices.

2. Direct devices lack buttons for state transitions.

3. Occlusion is also a major design challenge

4. The finger or pen occludes the area at which a user is pointing, so the user
may not realize that he or she has activated a control
 Input Technologies and Techniques- Input Device States

1. Input devices taken in general support three possible states : (a) out-of-range, (b)
tracking, and (c) dragging; practitioners refer to these as State 0, State 1, and State
2, respectively, of the three-state model

2. mouse as a two-state device, supporting State 1, the cursor tracking state, as well
as State 2, the dragging state. State 1 provides cursor feedback of the screen
position that the device will act upon, while State 2 allows the user to drag an
object by holding down the primary mouse button while moving the mouse
Input Technologies and Techniques- Input Device States
 Input Technologies and Techniques- Input Device States

3. Many touch-activated devices such as touchscreens, touchpads, and PDA

screens are also two-state devices, but do not sense the same two states as the mouse

4. The Tablet PC is an example of a pen-operated device that senses all three

states of the three-state model
Input Output Channels

• Information i/o …
• visual, auditory, haptic, movement
• Information stored in memory
• sensory, short-term, long-term
• Information processed and applied
• reasoning, problem solving, skill, error
• Emotion influences human capabilities
• Each person is different
Vision

Two stages in vision

• physical reception of stimulus

• processing and interpretation of stimulus

Interpreting the signal

• Size and depth

• visual angle indicates how much of view object occupies

(relates to size and distance from eye)

• visual acuity is ability to perceive detail (limited)

• familiar objects perceived as constant size

(in spite of changes in visual angle when far away)
Interpreting the signal
• Brightness
• subjective reaction to levels of light
• affected by luminance of object
• measured by just noticeable difference
• visual acuity increases with luminance as does flicker

• Colour
• made up of hue, intensity, saturation
• cones sensitive to colour wavelengths
• blue acuity is lowest
• 8% males and 1% females colour blind
Interpreting the signal

• The visual system compensates for:

• movement

• changes in luminance.

• Context is used to resolve ambiguity

• Optical illusions sometimes occur due to over compensation

Optical Illusions

• The Ponzo illusion The Muller–Lyer illusion

Hearing
• Provides information about environment:
distances, directions, objects etc.

• Physical apparatus:
• outer ear – protects inner and amplifies sound
• middle ear – transmits sound waves as
vibrations to inner ear

• inner ear – chemical transmitters are released

and cause impulses in auditory nerve

• Sound
• pitch – sound frequency
• loudness – amplitude
• timbre – type or quality
Touch
• Provides important feedback about environment.

• May be key sense for someone who is visually impaired.

• Stimulus received via receptors in the skin:

• thermoreceptors – heat and cold
• nociceptors – intense pressure, heat, pain
• mechanoreceptors – pressure
(some instant, some continuous)

• Some areas more sensitive than others e.g. fingers.

• Kinethesis - awareness of body position

• affects comfort and performance.
Movement
• Time taken to respond to stimulus:
reaction time + movement time

• Movement time dependent on age, fitness etc.

• Reaction time - dependent on stimulus type:

• visual ~ 200ms
• auditory ~ 150 ms
• pain ~ 700ms

• Increasing reaction time decreases accuracy in the unskilled operator but not in the skilled operator.
• Fitts' Law describes the time taken to hit a screen target:

Mt = a + b log2(D/S + 1)

where: a and b are empirically determined constants

Mt is movement time

D is Distance

S is Size of target

targets as large as possible

distances as small as possible
Human Memory

• The capacity of the human’s short-term memory (STM) is about 5–9 chunks of
information (or items meaningful with respect to the task), famously known as the
“magic number”.
Long Term Memory :Items are associated to each other in classes, and may inherit attributes from parent classes. This
model is known as a semantic network. As an example, our knowledge about dogs may be stored in a network
 THINKING: REASONING AND PROBLEM SOLVING

• Reasoning

• inferring new information from what is already known

• Problem Solving

• finding a solution to an unfamiliar task, using the knowledge we have

• Skill Acquisition

• Error

• incorrect understanding, or model, of a situation or system

• People build their own theories to understand the causal behavior of systems.
These have been termed mental models.
Emotion

• build interfaces that promote positive responses

• aesthetics or reward
Produce a semantic network of the main information in
this chapter
Text entry devices, Positioning, pointing and drawing,
Display devices, Devices for virtual reality and 3D interaction,
Physical controls, sensors and special devices,
Paper: printing and scanning
Text entry devices

• The alphanumeric keyboard

• Chord keyboards

• Phone pad and T9 entry

• Handwriting recognition

• Speech recognition
 POSITIONING, POINTING AND DRAWING

• The mouse

• Touchpad

• Trackball and thumbwheel

• Joystick and keyboard nipple

• Touch-sensitive screens

• Stylus and light pen

• Digitizing tablet

• Eyegaze
POSITIONING, POINTING AND DRAWING

• In computer-aided design (CAD), where positioning and drawing

are the major activities
• Pointing devices allow the user to point, position and select
items, either directly or by manipulating a pointer on the screen.
• Many pointing devices can also be used for free-hand drawing
although the skill of drawing with a mouse is very different from
using a pencil.
• The mouse is still most common for desktop computers
 The mouse
The mouse has become a major component of the majority of desktop computer
systems sold today.

• It is a small, palm-sized box housing a weighted ball – as the box is moved

over the tabletop

• The ball is rolled by the table and so rotates inside the housing.

• This rotation is detected by small rollers that are in contact with the ball, and
these adjust the values of potentiometers

• The potentiometers are aligned in different directions so that they can detect
both horizontal and vertical motion.
 The mouse ( cont…)

• The relative motion information is passed to the computer via a wire attached to
the box, or in some cases using wireless or infrared, and moves a pointer on the
screen, called the cursor.
• In addition to detecting motion, the mouse has typically one, two or three buttons
on top.
• These are used to initiate action.
• Single-button mice tend to have similar functionality
• multi-button mice,-achieve different operations for a single and a double button
click.
The first mouse. Photograph courtesy of Douglas Engelbart and Bootstrap Institute
 Optical mice
• Optical mice work differently from mechanical mice.
• A light-emitting diode emits a weak red light from the base of the
mouse.
• This is reflected off a special pad with a metallic grid-like pattern
upon which the mouse has to sit, and the fluctuations in reflected
intensity
• As the mouse is moved over the gridlines are recorded by a sensor in
the base of the mouse and translated into relative x, y motion.
 Touchpad

• Touchpads are touch-sensitive tablets usually around 2–3 inches (50–75

mm) square.
• They were first used extensively in Apple Power book portable computers
but are now used in many other notebook computers and can be obtained
separately to replace the mouse on the desktop.
• They are operated by stroking a finger over their surface, rather like using
a simulated trackball
 Trackball and thumbwheel

Trackball

• The trackball is really just an upside-down mouse!

• A weighted ball faces upwards and is rotated inside a static housing, the motion
being detected in the same way as for a mechanical mouse, and the relative motion
of the ball moves the cursor.
• Because of this, the trackball requires no additional space in which to operate, and is
therefore a very compact device.
Thumbwheel

• Thumbwheels are different in that they have two orthogonal dials to control the
cursor position.
• Such a device is very cheap, but slow, and it is difficult to manipulate the cursor in
any way other than horizontally or vertically.
• This limitation can sometimes be a useful constraint in the right application
 Joystick and keyboard nipple

Joystick :
• The joystick is an indirect input device, taking up very little space.
• Consisting of a small palm-sized box with a stick or shaped grip sticking up from it, the
joystick is a simple device with which movements of the stick cause a corresponding
movement of the screen cursor.
• There are two types of joystick: the absolute and the isometric.
Absolute joystick :
• movement is the important characteristic, since the position of the joystick in the
base corresponds to the position of the cursor on the screen.
Isometric joystick:
• The pressure on the stick corresponds to the velocity of the cursor, and when
released, the stick returns to its usual upright centered position. This type of joystick
is also called the velocity-controlled joystick, for obvious reasons.
• The buttons are usually placed on the top of the stick, or on the front like a trigger.
• Joysticks are inexpensive and fairly robust, and for this reason they are often found
in computer games.
Keyboard nipple

• A smaller device but with the same basic characteristics is used on many laptop
computers to control the cursor.
• Some older systems had a variant of this called the keymouse, which was a single key.
• More commonly a small rubber nipple projects in the center of the keyboard and acts as
a tiny isometric joystick.
• It is usually difficult for novices to use, but this seems to be related to fine adjustment
of the speed settings.
• Like the joystick the nipple controls the rate of movement across the screen and is thus
less direct than a mouse or stylus
Eyegaze

• Eyegaze systems allow you to control the computer by simply looking at it!
• Some systems require you to wear special glasses or a small head-mounted box, others are
built into the screen or sit as a small box below the screen.
• A low-power laser is shone into the eye and is reflected off the retina.
• The reflection changes as the angle of the eye alters, and by tracking the reflected beam
the eyegaze system can determine the direction in which the eye is looking
• Used in military applications, notably for guiding air-to air missiles to their targets
Cursor keys and discrete positioning
 Touch-sensitive screens (touchscreens)

• Another method of allowing the user to point and select objects on

the screen.
• They are much more direct than the mouse, as they detect the
presence of the user’s finger, or a stylus, on the screen itself.
• The work in one of a number of different ways: by the finger (or
stylus) interrupting a matrix of light beams, or by capacitance
changes on a grid overlaying the screen, or by ultrasonic reflections.
• The touchscreen is very fast, and requires no specialized pointing
device.
Disadvantages :

• Using the finger to point is not always suitable, it can leave greasy marks on the screen,
and, being a fairly blunt instrument, it is quite
• inaccurate. This means small regions is very difficult, as is accurate drawing.
 Stylus and light pen

• More accurate positioning (and to avoid greasy screens), systems with

touch sensitive surfaces often emply a stylus.
• Instead of pointing at the screen directly a small pen-like plastic stick is
used to point and draw on the screen.
• This is particularly popular in personal digital assistant (PDAs), but they are
also being used in some laptop computers.
• An older technology that is used in the same way is the light pen.
Stylus and light pen (cont…)

• The pen is connected to the screen by a cable and, in operation, is held to

the screen and detects a burst of light from the screen phosphor during
the display scan.
• The light pen can therefore address individual pixels and so is much
more accurate than the touchscreen.
• Stylus, light pen and touchscreen are all very direct in that the
relationship between the device and the thing selected is immediate.
 Digitizing tablet
• More specialized device typically used for freehand drawing, but may also be used as a
mouse substitute.
• Digitizing tablets are capable of high resolution, and are available in a range of sizes.
• Sampling rates vary, affecting the resolution of cursor movement, which gets progressively
finer as the sampling rate increases.
• Is an indirect device since there is a mapping from the plane of operation of the tablet to the
screen. It can also be used for text input; if supported by character recognition software,
handwriting can be interpreted.
• Problems with digitizing tablets are that they require a large amount of desk space, and may
be awkward to use if displaced to one side by the keyboard.
DISPLAY DEVICES

• Bitmap displays

• Technologies

o Cathode ray tube

o Liquid crystal display

o Special displays

• Large displays and situated displays

• Digital paper
DEVICES FOR VIRTUAL REALITY AND 3D INTERACTION

• Positioning in 3D space
• Cockpit and virtual controls
• The 3D mouse
• Dataglove
• Virtual reality helmets
• Whole-body tracking

• 3D displays
• Seeing in 3D
• VR motion sickness
• Simulators and VR caves
PHYSICAL CONTROLS, SENSORS AND SPECIAL DEVICES
• Special displays
• Sound output
• Touch, feel and smell
• Physical controls
• Environment and bio-sensing
PHYSICAL CONTROLS, SENSORS AND SPECIAL DEVICES
PAPER: PRINTING AND SCANNING

• Printing

• Fonts and page description languages

• Screen and page

• Scanners and optical character recognition

Question
What input and output devices would you use for the following systems? For each, compare
and contrast alternatives, and if appropriate indicate why the conventional keyboard, mouse
and CRT screen may be less suitable.

(a) portable word processor

(b) tourist information system

(c) tractor-mounted crop-spraying controller

(d) air traffic control system

(e) worldwide personal communications system

(f) digital cartographic system.

 Answer discussion
(a) Portable word processor The determining factors are size, weight and battery power. However,
remember the purpose: this is a word processor not an address book or even a data entry
device.

(i) LCD screen – low-power requirement

(ii) trackball or stylus for pointing

(iii) real keyboard – you can’t word process without a reasonable keyboard and stylus handwriting
recognition is not good enough

(iv) small, low-power bubble-jet printer – although not always necessary, this makes the package
stand alone. It is probably not so necessary that the printer has a large battery capacity as printing
can probably wait until a power point is found.
 Answer discussion
(b) Tourist information system This is likely to be in a public place. Most users will only visit the
system once, so the information and mode of interaction must be immediately obvious.

(i) touchscreen only – easy and direct interaction for first-time users (see also Chapter 3)

(ii) NO mice or styluses – in a public place they wouldn’t stay long!

(c) Tractor-mounted crop-spraying controller A hostile environment with plenty of mud and
chemicals. Requires numerical input for flow rates, etc., but probably no text

(i) touch-sensitive keypad – ordinary keypads would get blocked up

(ii) small dedicated LED display (LCDs often can’t be read in sunlight and large screens are
fragile)

(iii) again no mice or styluses – they would get lost.

 Answer discussion
(d) Air traffic control system The emphasis is on immediately available information and rapid
interaction. The controller cannot afford to spend time searching for information; all frequently
used information must be readily available.

(i) several specialized displays – including overlays of electronic information on radar

(ii) light pen or stylus – high-precision direct interaction

(iii) keyboard – for occasional text input, but consider making it fold out of the way.
 Answer discussion
(e) Worldwide personal communications system Basically a super mobile phone! If it is to be
kept on hand all the time it must be very light and pocket sized. However, to be a
‘communications’ system one would imagine that it should also act as a personal
address/telephone book, etc.

(i) standard telephone keypad – the most frequent use

(ii) small dedicated LCD display – low power, specialized functions

(iii) possibly stylus for interaction – it allows relatively rich interaction with the address book
software, but little space

(iv) a ‘docking’ facility – the system itself will be too small for a full-sized keyboard(!), but you
won’t want to enter in all your addresses and telephone numbers by stylus!
 Answer discussion
(f) Digital cartographic system This calls for very high-precision input and output facilities. It is
similar to CAD in terms of the screen facilities and printing, but in addition will require
specialized data capture.

(i) large high-resolution color VDU (20 inch or bigger) – these tend to be enormously big
(from back to front). LCD screens, although promising far thinner displays in the long term,
cannot at present be made large enough

(ii) digitizing tablet – for tracing data on existing paper maps. It could also double up as a
pointing device for some interaction

(iii) possibly thumbwheels – for detailed pointing and positioning tasks

(iv) large-format printer – indeed very large: an A2 or A1 plotter at minimum.