Module 2

Chapter 5

Direct Manipulation and

Virtual Environments
• Introduction
• Examples of Direct Manipulation
• Discussion of Direct Manipulation
• 3D Interfaces
• Teleoperation
• Virtual and Augmented Reality
 Introduction

Definition
•Visual representation (metaphor) of the "world
of action"
•Objects and actions are shown
•Analogical reasoning is tapped
•Rapid, incremental, and reversible actions
•Replacement of typing with pointing and selecting
•Immediate visibility of results of actions

2
 Introduction
Benefits over commands
•Control/display compatibility
•Less syntax reduces error rates
•Errors are more preventable
•Faster learning and higher retention
•Encourages exploration

Concerns
•Increased system resources, possibly
•Some actions may be cumbersome
•Macro techniques are often weak
•History and other tracing may be difficult
•Visually impaired users may have more difficulty3
 Introduction
Positive feelings with good user interfaces

• Mastery of the interface

• Competence in performing tasks
• Ease in learning the system originally and in
assimilating advanced features
• Confidence in the capacity to retain mastery
over time
• Enjoyment in using the system
• Eagerness to show the system off to novices
• Desire to explore more powerful aspects of
4
the system
 Examples of Direct-Manipulation…
Command Line v/s Display Editors and Word Processors

• Display editors - much easier to learn than Line editors

• Advantages of WYSIWYG Word Processors
• Users see a full page of text
• The document is seen as it will when printed
• Cursor action is visible
• Cursor motion natural
• Labeled icons make frequent actions rapid
• Immediate display of the results of an action
• Rapid response and display
• Easily reversible actions 5
 …Examples of Direct-Manipulation…
Technologies derived from the word processor:
• Graphics, Spreadsheets, and Animations etc. in the
body of a document - Object Embedding, OLE
• DeskTop Publication (DTP) software - Adobe Page
Maker, Ventura, QuarkXpress
• Presentation Software: MS-Power Point, Keynote
• Hypermedia Environments
• Improved Macro Facilities: Construct, Edit and Save
frequently used actions, Templates (for various needs),
Style sheets …
• Spell checker and Thesauri
• Grammar Checkers
• Document Assemblers: Composing complex documents 6
from standard paragraphs to fit a situation – Eg. Sale deed
 …Examples of Direct-Manipulation…
VISICALC Spreadsheet and its descendants

• Dan Bricklin & Bob Frankston - Harvard

Business School
• Electronic Spreadsheet: 253 rows X 63 Columns
• Can be programmed
• Graphics, Database, Multiple windows etc.
• Lotus 1-2-3, MS-Excel…

7
 …Examples of Direct-Manipulation…
Spatial Data Representation
• Geographic applications - Maps
• Provide a better model of reality
• Examples
• Mapping Software – Mapping Info, ArcView
• GIS software – ArcGIS, Google Earth, Google Maps…
• 3D-Home architect
• Successful spatial data-management systems
depend on choosing appropriate:
• Icons
• Graphical representations
• Natural and comprehensible data layouts 8
 …Examples of Direct-Manipulation…
Video Games
• 2D, 3D-Graphics and Animation
• Gaming platforms (Play stations), 2 or Multi-player
facilities, Joysticks, Wireless controllers of various
shapes and sizes
• Field of action is visual and compelling
• No syntax to remember – No error messages as
actions are obvious and reversible
• Players can generate their own characters
• Score cards – Top 10 scores -
• Bricks, Pacman, Space Invaders, Tennis, Karate,
Missile Command, Nintendo, Guitar Hero, Rock Band,
9
EverQuest, World of Warcraft …
 …Examples of Direct-Manipulation…
Computer-Aided Design (CAD)
• Systems for Automobiles, Electronic Circuitry,
Aircraft,, Mechanical Engineering, Building and
Home architects
• Objects can be manipulated directly
• Quick designs and quick modifications
• More details – Multiple views, Estimates, Materials list…
• AutoCAD

CAM and Process Control (Computer Aided Manufacturing)

• Schematic views of the Plants

• Detailed view in simple mouse clicks - Controlling
10
of devices by direct manipulation …
 …Examples of Direct-Manipulation…
Continuing Evolution of Direct Manipulations
Applications for:
• Personal finance and Travel arrangements
• Cheque book maintenance, Reservations
• Home Automation Systems
• Dashboard Views of data – Graphical representations

• Virtual Reality, Ubiquitous Computing,

Augmented Reality
• …..
Refer Section 5.2.7
11
 Discussion of Direct Manipulation…
Problems with Direct Manipulation…
1. Spatial/ Visual representation are not useful for
vision-impaired users
2. Direct-manipulation designs consume screen space
and may force scrolling/ multiple actions
3. Users must learn the meanings of visual
representations (may not be obvious)
4. Visual representations may mislead (Users may
over/underestimate functions)
5. Experienced typists feel selection to be slower than typing
6. Selection may be difficult on small screens – mobile devices
7. Choosing right objects and actions for a direct-
manipulation interface may be difficult
8. Designs require fast perception-action loop 12 and
reversibility – may be difficult to implement
 Discussion of Direct Manipulation…

Three Principles of Direct Manipulation

1. Continuous representation of the objects and

actions of interest with meaningful visual
metaphors

2. Physical actions or presses of labeled buttons

instead of complex syntax

3. Rapid, incremental, reversible operations

whose effect on the object of interest is
immediately visible
13
 Discussion of Direct Manipulation…
Beneficial Attributes of Direct Manipulation
• Novices learn quickly
• Experts work rapidly
• Intermittent users can retain concepts
• Error messages are rarely needed
• Users can immediately see if their actions are
furthering their goals/ counterproductive and
act
• Users experience less anxiety - reversibility
• Users gain confidence and mastery as 14they
15
 Discussion of Direct Manipulation…
Visual Thinking and Icons…
• The visual nature of computers can attracts users
with artistic, intuitive personalities
• Linear , text-oriented and rational users may
not like it
• Traditionalists scorn refer to visual UIs as WIMP
(Windows, Icons, Mouse and Pull-down Menu)
• Visual system proponents see the command-line
as stubborn and inflexible
Icon is a small (less than 1”X1” or 64X64
pixels) graphical representation of an
object or action
16
Semiotics: Study of signs and symbols
 Discussion of Direct Manipulation…
Icon-Specific Guidelines
1.Represent the object or action in a familiar and
recognizable manner
2.Limit the number of icons and make them
distinctive
3.Ensure harmoniousness of icons
4.Make icons stand out from the background
5.Consider three-dimensional icons
6.Ensure that a selected icon is visible from
unselected icons
7.Design the movement animation – for dragging
8.Add detailed information using size, colour,
animation … 17
9.Explore combinations of icons to create new ones
 Discussion of Direct Manipulation…

Direct Manipulation Programming

It should be possible to do programming so that

a set of actions can be repeatedly performed

• There may be a programming language

• A sequence of actions may be recorded and

repeatedly used - Macro - Demonstrational
programming

18
 3-D Interfaces…

• Useful for Medical Imagery, Architectural

drawing, CAD, and Chemical structure
visualization

• May cause clutter and be disturbing/

misleading occlusion

• Successful implementations – in games

• Examples – Myst, Second Life, The Sims Online…

• Modest use – Buttons that appear depressed/

raised, icons resembling real world objects …
19
 …3-D Interfaces…
Features for effective 3-D Interfaces
• Use occlusion, shadows and other 3D techniques
carefully
• Minimize the number of navigation steps
• Keep text readable
• Avoid unnecessary visual clutter, distraction/
contrast shifts, and reflections
• Simplify user movement (avoid surprises like
going through walls)
• Prevent errors
• Simplify object movement
• Organize groups of items
20
• Enable users to construct visual groups
 …3-D Interfaces…
Enhanced 3-D features…
• Provide overviews so users can see
aggregated views Allow teleportation
• Offer x-ray vision - users can see into or
beyond objects
• Provide history keeping
• Permit rich user actions on objects (save,
copy, annotate, share, send)
• Enable remote collaboration
• Give users control over explanatory text 21
 …3-D Interfaces
…Enhanced 3-D features
• Offer tools to select, mark, and measure.
• Implement dynamic queries to rapidly filter
out unneeded items.
• Support semantic zooming and movement
• Enable landmarks to show themselves even at
a distance
• Allow multiple coordinated views
• Develop novel 3D icons
22
 Teleoperation…
Physical processes takes place in a remote
location

•E.g. Process Control, Telepathology,

Telemedicine, Flying Airplanes, …

•Users must be helped to cope with slower

responses, incomplete feedback, breakdowns,
and complex error-recovery procedures

More Details– See Text

23
 …Teleoperation…
Complicating factors for Teleoperations
•Time delays
• Transmission delays + Operation delays

•Incomplete feedback

•Feedback from multiple sources

•Unanticipated interferences

24
 Virtual and Augmented Reality…
• Virtual reality breaks the physical limitations
of space and allow users to act as though they
were somewhere else
• Interface Objects and actions must be simple
• E.g. Flight Simulators, Treatment for Phobias…

• Augmented reality shows the real world with

an overlay of additional information
• E.g. While users see walls of a building, their
semitransparent eyeglasses may show the
location of electrical wires

Examples – See Text

25
 …Virtual and Augmented Reality…
Successful virtual environments depend on the
smooth integration of different technologies like:
•Visual Display
•Head-position Sensing
•Hand-position Sensing
•Hand-held Manipulatives
•Force Feedback and Haptics
•Sound input and Output
•Other Sensations
•Collaborative and Competitive environmnets
26
 …Virtual and Augmented Reality…
• Visual Display
• Normal size, Head mounted, Boom mounted…

• Head-position Sensing
• Head mounted displays can give Different views
based on head position, High precision sensors
desired…

• Hand-position Sensing
• Gloves, A six-degrees of freedom sensor on it
provides hand orientation.

• Hand-held Manipulatives
• Electronically active surgical tools to operate devices
27
 …Virtual and Augmented Reality…
• Force Feedback and Haptics
• Hand operated remote-control devices, Tactile
feedback to get a sense of objects

• Sound input and Output

• Adds realism, Speech recognition for initiating
actions and making menu selections when other
devices are restricted

• Other Sensations
• Tilting, Vibrating, Raindrops (virtual weather), Smell

• Collaborative and Competitive environmnets

• Allow 2 people at remote sites to design together,
Game playing… 28
 Question Bank
1. What is meant by Direct Manipulation? What are its benefits and concerns?
2. What are the positive feelings invoked by good User Interfaces?
(What are the feelings that convey an image of a truly pleased user?)
3. List some examples of direct manipulation systems.
4. What are the advantages of WYSIWYG processors?
5. What are the offshoots of (Technologies derived from) word processing
software?
6. Write a note on direct manipulation in Spatial Data representation, Video
Games, CAD-CAM applications.
7. What are the problems with direct manipulation?
8. What are the three principles of direct manipulation? What are its beneficial
attributes?
9. What is an icon? What are the guidelines for designing icons?
10. What is meant by direct manipulation programming?
11. Comment on use of 3-D Interfaces. List the features of effective 3-D
interfaces. (Give the guidelines for 3-D interfaces)
12. What is meant by Teleoperation? Give an example. What are the
complicated issues in it?
13. Explain the terms: Virtual Reality, Augmented Reality.
29
14. What are the technologies used in virtual environments?