UNIT-1

Augmented Reality
 Outline
▪ Taxonomy
▪ technology and features of augmented reality
▪ difference between AR and VR
▪ Challenges with AR
▪ AR systems and functionality
▪ Augmented reality methods
▪ visualization techniques for augmented reality
▪ wireless displays in educational augmented
reality applications
▪ mobile projection interfaces
▪ marker-less tracking for augmented reality
▪ enhancing interactivity in AR environments
▪ evaluating AR systems.
What is
Augmented
Reality?
A combination of a real
scene viewed by a user
and a virtual scene
generated by a computer
that augments the scene
with additional
information.

3
An AR system adds
virtual computer-
generated objects,
audio and other
sense enhancements
to a real-world
enviornment in real
time.

4
What is the Goal of AR?

To enhance a person’s performance

and perception of the world

But, what is the ultimate goal????

5
The Ultimate Goal of AR

Create a system such that a user

CANNOT tell the difference between
the real world and the virtual
augmentation of it.

6
Augmented Reality
vs. Virtual Reality

Augmented Reality Virtual Reality:

System augments the Totally immersive
real world scene environment
User maintains a Visual senses are
sense of presence in under control of
real world system (sometimes
aural and
Needs a mechanism
to combine virtual proprioceptive senses
and real worlds too)
7
 Miligram’s Reality-
Virtuality
Continuum
Mixed Reality (MR)

Real Augmented Augmented Virtual

Environmen Reality (AR) Virtuality (AV) Environment
t

Miligram coined the term “Augmented Virtuality” to

identify systems which are mostly synthetic with some
real world imagery added such as texture mapping video
onto virtual objects.
8
 Miligram’s Taxonomy for
Mixed Reality Displays

Extent of World γ Reproduction Fidelity –

Knowledge quality of computer
generated imagery
Extent of γ Extent of Presence Metaphor
Presence – level of immersion of the
Metaphor user within the displayed
γ scene
Extent of World Knowledge –
knowledge of relationship
Reproduction between frames of reference
Fidelity for the real world, the camera
viewing it, and the user
Features of AR

▪AR can be defined as a system that

incorporates three basic features:
▪a combination of real and virtual worlds,
▪real-time interaction
▪accurate 3D registration of virtual and
real objects.
 Challenges with AR
1. Implementation cost
▪By far the most common concern was the perceived
cost of implementing the new technology, with 33% of
respondents referencing financial constraints.
▪Take, for example, the furniture retailer IKEA. Their AR
app allows customers to shop their site as well as scan
their homes with a smart phone to see how products
will look before purchasing them.
▪While the price IKEA paid is a commercial
secret, analysts have estimated a cost of
$30,000-$60,000 for the build and implementation,
based on market rates for AR app developers.
 Challenges with AR
2. Technology and skill gaps
▪Twelve percent of respondents expressed concerns
over “talent and technological gaps.” These gaps - and
the fear of them – are common with any new
technology
▪But as the second-most cited concern, it appears that
executives need more guidance on training,
use-cases, implementation, and maintenance.
 Challenges with AR
3. Resource scarcity and competing priorities
▪Executives struggled to find ways to devote already
scarce financial and human resources to the
implementation of AR.
▪It takes time to train and/or recruit workers to operate
complex systems, making executives weary of
diverting time and talent away from what already works
or other plans in the pipeline.
 Challenges with AR
4. Management buy-in
▪Justifying the adoption of AR to management felt like
the biggest hurdle to 11% of respondents.
▪Luckily, there are now success cases in nearly every
industry, showing improved customer experiences and
dramatic impacts on staff productivity using AR.
 Challenges with AR
5. Cyber security challenges and regulatory
concerns
▪Despite excitement about the possibilities of
augmented reality, some executives feel that
“Regulations aren’t enough” at the moment.
▪Five percent of those polled were concerned that
policymakers and AR developers have not done
enough to address data privacy, copyright, and liability
concerns.
 Challenges with AR
6. Perceptions about the technology
▪A few respondents mentioned that AR simply doesn’t
apply to their business model or is still “a gimmick.”
But as AR Insider has recently discovered, a growing
number of people are using mobile AR for the first time
and reporting high levels of satisfaction with the
technology.
▪Combined with data on AR’s contributions to
productivity, collaboration, the utilization of scarce
resources, controlling labor unit rates, and even
minimizing accidents, it’s becoming much easier to see
the benefits of joining the early majority of adopters.
AR components and workflow
AR components and workflow
Components of an
Augmented Reality System
Combining the Real and
Virtual Worlds

We need:
γ Precise models
γ Locations and optical properties of the

viewer (or camera) and the display

γ
Calibration of all devices
γ
To combine all local coordinate systems
centered on the devices and the objects in
the scene in a global coordinate system
Combining the Real and
Virtual Worlds (cont)
γRegister models of all 3D objects of interest with
their counterparts in the scene
γTrack the objects over time when the user
moves and interacts with the scene
Realistic Merging

Requires:
γ Objects to behave in physically plausible

manners when manipulated

γ
Occlusion
γ
Collision detection
γ
Shadows
**All of this requires a very detailed description of
the physical scene
Research Activities

γDevelop methods to register the two

distinct sets of images and keep them
registered in real-time
– New work in this area has started to use
computer vision techniques
γDevelopnew display technologies for
merging the two images
Performance Issues
Augmented Reality systems are expected:
γ To run in real-time so that the user can move

around freely in the environment

γ
Show a properly rendered augmented image
Therefore, two performance criteria are placed on
the system:
γ Update rate for generating the augmenting

image
γ
Accuracy of the registration of the real and
virtual image
Limitations for
Updating the
Generated Images
γ Must be at 10 times/second
γ More photorealistic graphics rendering
γ Current technology does not support fully
lit, shaded and ray-traced images of
complex scenes
Failures in Registration

Failures in registration due to:

– Noise
• Position and pose of camera with respect to the
real scene
• Fluctuations of values while the system is running
– Time delays
• In calculating the camera position
• In calculating the correct alignment of the
graphics camera
This is how AR
works
Pick A Real World Scene
Add your Virtual Objects in it.
Delete Real World Objects
Not Virtual Reality since
Environment Real.

27
DISPLAY
Head-mounted
Display(HMD)
– device paired toa
headset such as a
harness or helmet
Eye Glasses
– eye wear that
employs
cameras to intercept
the real world view and
re-display it's
augmented view
through the eye pieces

28
DISPLAY(cont..)

Contact Lenses
– Contain the elements for display
embedded into the lens including
integrated circuitry, LEDs and an
antenna for wireless
communication.
– Under development
Virtual Retina Display
– a personal display device
under development .
– a display is scanned directly
11
onto the retina of a viewer's eye.
DISPLAY(cont..)
□
a small display that fits in a user's hand.
Handheld
Portable
Ubiquitous
Physical constraints of the user having to hold the device
Distorting effect

□ Spatial
makes use of digital projectors to display
graphical information.
user is not required to carry equipment or wear the
display over their eyes.
can be used by multiple people at the same time without
each having to wear a head-mounted display.

30
Augmented reality methods /
Display Technologies

Monitor Based
Head Mounted Displays:
– Video see-through
– Optical see-through

31
Monitor Based Augmented
Reality

Simplest available
Little feeling of being immersed in
environment

32
Optical see-through HMD
▪ Optical see through is not fully realized yet. It is supposed to
consist of ordinary looking pair of glasses that will have light
source on the side to project images onto the retina.

33
Video see-through HMD
▪ They block out the wearer's surrounding environment, using small
video cameras attached to the outside of the goggles to capture
images.
▪ On the inside of the display, the video image is played in real-time
and the graphics are superimposed on the video.

34
Video Composition
for Video see-through
HMD
Chroma-keying
– Used for special effects
– Background of computer graphics images is
set to a specific color
– Combining step replaces all colored areas
with corresponding parts from video
Depth Information
– Combine real and virtual images by a pixel-
by-pixel depth comparison
35
Advantages of
Video see-through HMD

Flexibility in composition strategies

Wide field of view
Real and virtual view delays can be
matched

36
Advantages of
Optical see-through HMD

Simplicity
Resolution
No eye offset

37
visualization techniques for
augmented reality
▪Please go through the reference pdf named

Visualization Techniques for Augmented Reality.pdf

wireless displays in educational
augmented reality applications
▪Please go through the reference pdf named

Wireless Displays in Educational Augmented

Reality Applications.pdf
mobile projection interfaces

▪Please go through the reference pdf named

Mobile Projection Interfaces.pdf

marker-less tracking for
augmented reality
▪Please go through the reference pdf named

Marker-based vs markerless augmented reality_

pros & cons_Overly app.pdf
enhancing interactivity in AR
environments
▪Please go through the reference pdf named

Marker-based vs markerless augmented reality_

pros & cons_Overly app.pdf
evaluating AR systems

▪Please go through the reference pdf named

Evaluating AR Systems.pdf
▪ Also refer the reference pdf named

Augmented Reality - All in one.pdf

For better understanding of contents and for better

answers of the exam questions.
Applications

Medical Manufacturing,
Entertainment Maintenance, and
Military Training Repair
Consumer Design
Engineering
Design Hazard Detection
Robotics and Audio
Telerobotics

45
Medical

46
Entertainment

47
Defence

48
Education

49