Chapter 5

Augmented Reality (AR)

1 Compiled By: Tsegaye B.

 Outline
 Introduction to Augmented Reality
 Virtual Reality (VR) Vs Augmented Reality (AR) Vs
Mixed Reality (MR)
 Architecture of AR systems
 Application of AR systems:
 Education
 Medical
 Assistance
 Entertainment
 Workshop-oriented hands demo
2 Augmented Reality
 Learning Outcome
After accomplishing this chapter, Students will be able to:
 Explain augmented reality
 Explain the features of augmented reality
 Explain the difference between AR, VR, and
MR
 Explain the architecture of augmented reality
systems
 Describe the application areas of augmented
reality

3 Augmented Reality
 Overview of Augmented Reality
 The fundamental idea of Augmented reality
(AR) is to combine, or mix, the view of the real
environment with additional, virtual content
that is presented through computer graphics.
 The virtual content is aligned and registered
with the real objects.
 As a person moves in an environment and their
perspective view of real objects changes, the
virtual content should also be presented from
the same perspective
4 Augmented Reality
 Overview of Augmented Reality
 Augmented reality (AR) is a form of emerging
technology that allows users to overlay
computer generated content in the real world.
 AR refers to a live view of a physical real-world
environment whose elements are merged with
augmented computer-generated images creating
a mixed reality.

5 Augmented Reality
 Overview of Augmented Reality
 The augmentation is typically done in real-time
and in semantic context with environmental
elements.
 By using the latest AR techniques and
technologies, the information about the
surrounding real world becomes interactive and
digitally usable.
 Through this augmented vision, a user can
digitally interact with and adjust information
about their surrounding environment
6 Augmented Reality
 Overview of Augmented Reality
 Augmented Reality (AR) as a real-time direct or
indirect view of a physical real-world
environment that has been enhanced/augmented
by adding virtual computer-generated
information to it.
 Augmented reality is the integration of digital
information with the user's environment in real
time.

7 Augmented Reality
 Overview of Augmented Reality
 Unlike virtual reality (VR), which creates a
totally artificial environment, augmented reality
uses the existing environment and overlays new
information on top of it.
 A live direct or indirect view of a physical, real-
world environment whose elements are
augmented by computer-generated sensory
input such as sound, video, graphics or GPS
data.

8 Augmented Reality
 Virtual reality (VR), Augmented
Reality (AR) vs Mixed reality (MR)
 With constant development in computer vision
and the exponential advancement of computer
processing power, virtual reality (VR),
augmented reality (AR), and mixed reality
(MR) technology is becoming more and more
prominent.
 With some overlap in the applications and
functions of these emerging technologies,
sometimes these terms get confused or are used
incorrectly.
9 Augmented Reality
 Virtual reality (VR), Augmented
Reality (AR) vs Mixed reality (MR)
a

10 Augmented Reality
 Virtual Reality (VR)
 VR is fully immersive, which tricks your senses
into thinking you’re in a different environment
or world apart from the real world.
 Using a head-mounted display (HMD) or
headset, you’ll experience a computer-
generated world of imagery and sounds in
which you can manipulate Objects and move
around using haptic controllers while tethered
to a console or PC
 It is also called a computer-simulated reality.

11 Augmented Reality
 Virtual Reality (VR)
 It refers to computer technologies using reality
headsets to generate realistic sounds, images
and other sensations that replicate a real
environment or create an imaginary world.

 Advanced VR environment will engage all five

senses (taste, sight, smell, touch, sound), but it
is important to say that this is not always
possible.

12 Augmented Reality
 Virtual Reality (VR)

13 Augmented Reality
 Virtual Reality (VR)
 Using VR devices such as HTC Vive, Oculus
Rift or Google Cardboard, users can be
transported into a number of real-world and
imagined environments.
 The most advanced VR experiences even
provide freedom of movement – users can move
in a digital environment and hear sounds.
 Moreover, special hand controllers can be used
to enhance VR experiences.
14 Augmented Reality
 Virtual reality (VR)
 Most VR headsets are connected to a computer
(Oculus Rift) or a gaming console (PlayStation
VR) but there are standalone devices (Google
Cardboard is among the most popular) as well.
 Most standalone VR headsets work in
combination with smartphones – you insert a
smartphone, wear a headset, and immerse in the
virtual reality.

15 Augmented Reality
 Virtual reality (VR)
 Most VR headsets are connected to a computer
(Oculus Rift) or a gaming console (PlayStation
VR) but there are standalone devices (Google
Cardboard is among the most popular) as well.
 Most standalone VR headsets work in
combination with smartphones – you insert a
smartphone, wear a headset, and immerse in the
virtual reality.

16 Augmented Reality
 Virtual reality (VR)

17 Augmented Reality
 Augmented Reality (AR)
 In augmented reality, users see and interact with the
real world while digital content is added to it.
 If you own a modern smartphone, you can easily
download an AR app and try this technology.
 There’s a different way to experience augmented
reality, though – with special AR headsets, such as
Google Glass, where digital content is displayed on
a tiny screen in front of a user’s eye.

18 Augmented Reality
 Augmented Reality (AR)
 AR adds digital elements to a live view often by
using the camera on a smartphone.
 Examples of augmented reality experiences
include Snapchat lenses and the game Pokemon
Go.
 AR is a live, direct or indirect view of a
physical, real-world environment whose
elements are augmented (or supplemented) by
computer-generated sensory input such as
sound, video, graphics or GPS data
19 Augmented Reality
 Mixed Reality (MR)
 Mixed Reality (MR), sometimes referred to as
hybrid reality, is the merging of real and virtual
worlds to produce new environments and
visualizations where physical and digital objects
co-exist and interact in real-time.
 It means placing new imagery within a real
space in such a way that the new imagery is
able to interact, to an extent, with what is real in
the physical world we know.
20 Augmented Reality
 Mixed Reality (MR)
 For example, with MR, you can play a virtual
video game, grab your real-world water bottle,
and smack an imaginary character from the
game with the bottle.
 Imagination and reality have never been so
intermingled.
 The key characteristic of MR is that the
synthetic content and the real-world content are
able to react to each other in real-time.
21 Augmented Reality
 Mixed Reality (MR)
Mixed Reality in Engineering and Medicine

22 Augmented Reality
 Mixed Reality (MR)
 In mixed reality, you interact with and
manipulate both physical and virtual items and
environments, using next-generation sensing
and imaging technologies.
 MR allows you to see and immerse yourself in
the world around you even as you interact with
a virtual environment using your own hands—
all without ever removing your headset.

23 Augmented Reality
 Mixed Reality (MR)
Mixed Reality in Entertainment

24 Augmented Reality
 Mixed Reality (MR)
 It provides the ability to have one foot (or hand)
in the real world, and the other in an imaginary
place, breaking down basic concepts between
real and imaginary, offering an experience that
can change the way you game and work today.

25 Augmented Reality
 Mixed Reality (MR)
 One of the most obvious differences among
augmented reality, virtual reality, and mixed reality
is:
 The hardware requirements
 VR is content which is 100% digital and can be
enjoyed in a fully immersive environment,
 AR overlays digital content on top of the real-world.
 MR is a digital overlay that allows interactive virtual
elements to integrate and interact with the real-world
environment.
 Numerous augmented reality apps and games can
run on almost every smartphone on the market.

26 Augmented Reality
 Mixed Reality (MR)
 Mixed reality hardware is still emerging and
hasn’t quite broken into the mainstream
consumer market, most likely due to the price.
 The consumer releases of the Microsoft
HoloLens and Magic Leap One retail for over
$2000 USD, which is 3 to 4 times the cost of
the PlayStation VR and HTC Vive VR headsets.
 However, mixed reality applications sometimes
require exponentially more processing power
and thus require more powerful hardware.
27 Augmented Reality
 Mixed Reality (MR)
 For example, the Microsoft HoloLens includes
a built-in microphone array, binaural sound
capabilities, a built-in camera for recording, a
depth sensor, head-tracking cameras, and an
inertial measurement unit which helps track
your head movement.
 On top of the traditional CPU and GPU,
Microsoft also created a Holographic
Processing Unit to help track where the user is
looking and understand command gestures.
28 Augmented Reality
 The Architecture of AR Systems
 The first Augmented Reality Systems (ARS) were usually
designed with a basis on three main blocks:
I. Infrastructure Tracker Unit
II. Processing Unit
III. Visual Unit.
 The Infrastructure Tracker Unit was responsible for
collecting data from the real world, sending them to the
Processing Unit, which mixed the virtual content with the
real content and sent the result to the Video Out module of
the Visual Unit.
 Some designs used a Video In, to acquire required data for
the Infrastructure Tracker Unit.
29 Augmented Reality
 The architecture of AR Systems
Augmented Reality Systems (ARS) standard architecture

30 Augmented Reality
 The architecture of AR Systems
 The Visual Unit can be classified into two types
of system, depending on the followed
visualization technology:
1. Video see-through: It uses a Head-Mounted
Display (HMD) that employs a video-mixing and
displays the merged images on a closed-
view HMD.
2. Optical see-through: It uses an HMD that employs
optical combiners to merge the images within an
open-view HMD.

31 Augmented Reality
 The architecture of AR Systems
 HMDs are currently the dominant display
technology in the AR field.
 However, they lack in several aspects, such as:
 ergonomics
 high prices
 low mobility due to their sizes and connectivity
features.
 interaction with the real environment, which places
virtual interactive zones to the user, making the
collision with these zones hard due to the difficulty
to interact with multiple points in different depths.
32 Augmented Reality
 The architecture of AR Systems
 Alternative approaches to developing ARS
involve the use of monitors and tablets.
 Monitors are used as an option for indirect view
since the user does not look directly into the
mixed world.
 Tablets are used in direct view since the user
points the camera to the scene and looks
directly into the mixed world.
 Both approaches still have difficulties in getting
a collision.
33 Augmented Reality
 Applications of AR Systems
 Technology is ever-changing and ever-growing.
 One of the newest developing technologies is
augmented reality (AR), which can be applied
to many different disciplines such as:
Education,
Medicine,
Entertainment,
Military

34 Augmented Reality
 AR in Education
 Augmented reality allows flexibility in use that is attractive to
education.
 AR technology can be utilized through a variety of mediums
including desktops, mobile devices, and smartphones.
 AR technology is portable and adaptable to a variety of
scenarios.
 AR can be used to enhance content and instruction within the
traditional classroom,
 AR can be used to supplement instruction in the special
education classroom,
 AR can be used to extend content into the world outside the
classroom,
 AR can be used to combined with other technologies to enrich
their individual applications.

35 Augmented Reality
 AR in Education
 Affordable learning materials - posters, digital
illustrations, physical models, prototypes are very
expensive and it’s impossible for schools to find enough
money to purchase all the supplementary materials they
would like to. Using AR technology allows for avoiding
investments in physical materials. Besides, students can get
access to learning materials and interact with them
anytime.
 Interactive lessons - when AR technology is used in
classrooms, students can view models on their own
smartphones and get a better idea of the concepts they are
studying. That increases engagements and reinforces the
learning.

36 Augmented Reality
 AR in Education
Higher engagement –
 when teachers integrate augmented reality into their lectures,
they attract the attention of their students and make lessons
more effective.
 When students are interested, it is much easier to make them
work more productively.
Higher retention -
 using the AR app, students can get access to augmented
models that represent any real objects from a famous
monument or work of art to a molecule.
 Besides, students can get access to a website with specific
information.
 When learning with AR technology, students use different
senses and retain more knowledge for a long time.

37 Augmented Reality
 AR in Education
Boost intellectual curiosity
 augmented reality makes students more excited
about learning certain subjects.
 Modern students were born in a digital era so they
will always be excited with innovative technologies
that can help them learn new ideas and develop their
critical thinking skills.
 When using AR technology in the classroom,
teachers can create an authentic learning
environment for students with different learning
styles.
38 Augmented Reality
 AR in Medicine
 Augmented reality is one of the current technologies
changing all industries, including healthcare and medical
education.
 The purpose of any invention and technology is to simplify
our life.
 Augmented reality has the potential to play a big role in
improving the healthcare industry.
 Only a few years since the first implementations of
augmented reality in medicine, it has already filled an
important place in doctors’ and nurses’ routine, as well as
patients’ lives.
 This new technology is enhancing medicine and healthcare
towards more safety and efficiency.
39 Augmented Reality
 AR In Medicine
 For now, augmented reality has already made
significant changes in the following medical
areas:
 surgery (minimally invasive surgery)
 education of future doctors
 diagnostics
 AR tools may also aid to detect the signs of
depression and other mental illnesses by reading
from facial expressions, voice tones, and physical
gestures.
40 Augmented Reality
 AR In Medicine
In medicine, AR has the following applications:
1. Describing symptoms – Have you ever been in a
situation when it was hard to describe to the doctor what was
bothering you? It is a common problem for all us, the roots
of which extend to overreacting and lack of knowledge. And
what is most important, it impacts on finding out the
accurate diagnosis. The first steps to find the solutions are
already made. To increase patients’ education, medical app
AyeDecide is using augmented reality to show the simulation
of the vision, harmed by the different diseases. It helps
patients to understand their conditions and describe correctly
their symptoms.
41 Augmented Reality
 AR In Medicine
2. Nursing care - About 40% of the first intravenous
injections fail, and this ratio is even higher in the case of
children and elderly patients.
 The AccuVein uses augmented reality to cope with
this negative statistic.
 A handheld scanner projects on the skin and shows
the patients’ veins.
 It increases the successful finding of the vein from
the first try in 3,5 times. That is why this invention
got the greatest recognition among the general public
and medical staff.
42 Augmented Reality
 AR in Medicine
3. Surgery - In no sphere augmented reality does not have
such practical application as in the medicine, especially in surgery,
where it literally helps to save lives.
 Three dimensional reconstructions of organs or tumors will help
surgeons become more efficient at surgery operations.
 For example, spinal surgery, as usually, is a long and difficult
process. But with the use of AR, it can reduce the time, cut the
risks and improve the results. The Israeli startup Augmedics had
created an augmented reality headset for spine surgeons. This
technology overlays a 3D model of the CT-scan on the spine, so,
the surgeon gets some kind of “X-ray” vision.

43 Augmented Reality
 AR in Medicine
4) Ultrasounds - Some time ago ultrasound made a
small revolution in medicine. Today, it has another
one chance to make the same with using augmented
reality. Already a few AR software companies
developed handy ultrasound scanner, which with the
help of smart glasses works as a traditional one. It is
hard to overestimate the usefulness of this
technology. Especially when we talk about using it in
the developing countries, in military medicine (on
the battlefields) and even in the ambulance.
44 Augmented Reality
 AR in Medicine
5. Diabetes management - In 2017, the number of
people struggle with diabetes reached up to 425
million adults worldwide. And the amount of
diagnosed people is increasing every year. In 2014,
Google revealed the plans for creating a smart
contact lens (Google Contact Lens), in which the
main function will be to measure the glucose levels
in the tears. It will help people with this disease to
live the life they used to, without permanent worries
about sugar level in the blood.
45 Augmented Reality
 AR in Medicine
6. Navigation - The using AR in navigation apps has
already become a “traditional” way. By pointing your
phone to the city landscape, you get the information
about nearby
 objects of your interest (museums, hotels, shops, metro
stations, etc.). The same way AR
 can be useful to provide information about the nearest
hospitals. For example, the EHBO
 app helps to find the nearest to you AEDs (automated
external defibrillators).

46 Augmented Reality
 AR In Medicine
Generally, AR provides the following benefits to
patients and healthcare workers:
 Reduce the risks associated with minimally invasive
surgery.
 Better informed decisions about the right treatment
and illness prevention.
 Make procedures more tolerable.
 Better aftercare
 Medical training and education.
 Assistance in medical procedures and routine tasks.
47 Augmented Reality
 AR In Entertainment
Augmented reality can be used in various
“entertainment” industries as entertainment covers quite
a number of different industries – music, movies, live
shows, games – and all of them can benefit from using
augmented reality.
 AR in games - the AR games were praised for
increasing physical activity in people – you actually
have to move around to find your target. For
example, Pokémon GO has rightfully earned its
popularity and opened the world of AR games to us.

48 Augmented Reality
 AR In Entertainment
AR in music
 music is not only about listening to favorite
tracks and putting together playlists. When we
like a piece, we often want to find out more about
its background:
 The performers’ bios, the lyrics of the song, the making of the
recording or the music video.
 Augmented reality can do all that and much more providing
complete information on the track or its performer.
 Augmented reality can enhance live performances
by illustrating the story told by a track or displaying
the way it was created by the band.
49 Augmented Reality
 AR In Entertainment
AR on TV
 this may seem a bit far-fetched, as television
already shows a virtual world, thus adding
augmented reality will raise it to the second
power. However, some experiments of fusing
augmented reality on TV are already being made
with the promise of future enhancements.
 One way of integrating augmented reality in
television is adding supplementary information to
what is going on the TV screen – such as match
scores, betting options, and the like.

50 Augmented Reality
 AR In Entertainment
AR in eSports
 recently, the industry of eSports has been gaining
popularity in all parts of the globe.
 Competitive online gaming has become as
fascinating as real sports, and the technology is
following it closely with new solutions and unusual
implementations.
 Augmented reality turns eSports shows into
interactive experiences allowing the watchers to
become participants.

51 Augmented Reality
 AR In Entertainment
AR in the theater
 augmented reality can serve not only for entertainment
purposes but also for the purposes of accessibility.
 The possibility to overlay virtual objects over the real
environment can be used, for example, for subtitling in
various theater shows.
 Now, many theaters use LED displays either to provide
subtitles for translation or to assist hearing-impaired
visitors.
 However, LED equipment is not available in each
theater and even when it is, it can distract the viewers
from the show.
52 Augmented Reality
 End of Chapter 5

53 Augmented Reality