Lecture 1
1. What is Virtual Reality (VR)?
Virtual reality is a computer generated virtual world in which the user in
control and is provided a natural interaction(using headsets,controller etc.)
2. What is Augmented Reality (AR)?
Augmented Reality is an enhanced physical world, virtual and real world are
merged to create composite view.
3. What is Augmented Virtuality (AV)?
Augmented Virtuality describes that class of displays that enrich the virtual
experience by adding elements of the real environment.
4. What is Mixed Reality (MR)?
MR is the spectrum of augmented reality and augmented virtuality combined.
5. What is Extended Reality (XR)?
XR is an umbrella term encapsulating Augmented Reality (AR), Virtual Reality
(VR), Mixed Reality (MR), and everything in between.
6. Virtual Reality vs Augmented Reality.
AR uses a real-world setting while VR is completely virtual. AR users can
control their presence in the real world; VR users are controlled by the
system. VR requires a headset device, but AR can be accessed with a
smartphone. AR enhances both the virtual and real world while VR only
enhances a fictional reality
7. Main Characteristics of Virtual Reality.
-VR highlight autonomy/agency -user at all time stays in control(head tracking
and body as input.
-VR provides natural interaction using gestures (controller,hand &finger) or
speech as input.
-Gives a sense of presence immersive,multi-sensory(visual,audio..)
8. Main Characteristics of Augmented Reality.
AR incorporates three features: a combination of digital and physical worlds,
interactions made in real time, and accurate 3D identification of virtual and
real objects.
9. Reality-Virtuality Continuum
10. Four Classes of XR Technology
We identify four classes of XR technology: devices, platforms,
applications, and tools. We distinguish between devices and platforms
(devices like the Rift S or Quest are compatible with platforms like Oculus or
SteamVR), and between applications and tools (applications are software that
give a user an XR experience while tools are software for designers or
developers to create new applications).
11. Virtual Reality Devices examples
Ocolus Rift, Google Cardboard,Ocolus Quest
12. Augmented Reality Devices examples
Holokit, AppleARKit, Microsoft Hololens
13. Virtual Reality Platforms
Ocolus Store(for Rift ,Go), Vive Port(Vive, Vive Pro), Microsoft Store (WMR)}
SteamVR, WebXR
14. Augmented Reality Platforms
Apple Store(iPhone,iPad) , Windows Store(Hololens) ,Play Store(android) }
WebXR
15. XR Applications – give an example (Trying an AR App on Your
Smartphone)
Some of the popular mobile apps with AR features include Amazon Shopping,
Google Maps, and increasingly also Google Search.
Lecture 2
1. Key Characteristics of VR
Autonomy/Agency(head tracking,body input),
Natural interaction(gestures & speech, hand/controller input),
Presence(immersive,multi sensory)
Virtual Environment,
Immersive task,
Believable experience
2. VR Displays
Head-mounted (HMD) , Room-sized (CAVE)
3. VR key Concepts
Autonomy/Agency,
Presence & Immersion ,
Embodiment & Avatars ,
Cognition & Empathy,
Proprioception & Motion sickness,
Haptics & Pseudo-haptics,
Boundary & Redirected walking.
4. What is Autonomy/Agency?
The user can choose their own perspective on the scene;
can choose to navigate the scene in different ways ; can choose to interact
with any objects.
5. What is Presence?
Place illusion: ATQ Am I there?
Plausibility illusion: ATQ Is this happening?
6. What is Immersion?
Immersion, referring to the level of physical or psychological submergence of
a user within a virtual space relative to that user's consciousness of the real-
world environment. The goal of Immersive VR is to completely immerse the
user inside the computer generated world, giving the impression to the user
that he/she has “stepped inside” the synthetic world.
7. What is embodiment?
Embodiment is the possibility in VR to visually switch a person's real body by
a life-sized virtual one, seen from the person's own first-person perspective.
8. What is avatar?
Avatars are on-screen or virtual manifestations of the user, and technically,
they can take on any shape or form as long as they have humanoid features
such as moveable limbs, upper and lower torsos, and a face capable of
expression. Avatar represents who we are in a virtual space or gaming world
and the avatar’s actions/decisions are identical to our own.
9. What is cognition and empathy? Give an example!
Cognition is the mental action or process of acquiring knowledge and
understanding through thought, experience, and the senses.(ex. Visual
attention,memory)
Virtual reality (VR) has been touted as a way to promote empathy, by helping
people virtually experience what it is like to be in someone else's situation.
10. What is proprioception? Give an example!
Proprioception is the awareness of the position and movement of our body in
space and results from the processing of information from muscle, joint,
tendon, and skin receptors. Ex. physical rehabilitation.
11. What is Motion sickness? Give an example!
Virtual reality sickness (VR motion sickness) is the physical discomfort that
occurs when an end user's brain receives conflicting signals about self-
movement in a digital environment. A pronounced feeling of illness typically
occurs when the viewer is watching a digital representation of themselves
appear to move quickly in a digital environment while the person's physical
body remains motionless.
Ex: Motion sickness occurs when your brain can't make sense of information
sent from your eyes, ears and body. Lots of motion — in a car, airplane, boat,
or even an amusement park ride.
12. What is Haptics and Pseudo-haptics? Give an example!
Are elated to the sense of touch .
-haptics can replicate the sensation of physical button presses
-When used with virtual reality devices, haptic feedback allows the user to
touch and feel something in the virtual environment that is not in the real
world.
The most well-known examples of haptics are probably the vibration in a
mobile phone or the rumble in a game controller,
-Pseudo-haptics, a form of haptic illusion exploiting the brain's capabilities
and limitations.
13. What is Boundary and Redirected Walking? Give an example!
- VR boundary – to mark the area that is safe to use
- The headset finds all the obstacles in the room
- Redirected walking allows users of virtual reality applications to explore
virtual environments larger than the available physical space.
14. VR Technologies
360 Photos&Videos , Stereoscopic Displays , CAVEs . 3DOF , 6DOF, Outside
in ,Inside out , Hand Tracking , Spatial audio.
15. Explain 360 Photos and Videos
360 photos and videos by themselves are not VR. They are often associated
with VR, but they only become a VR when we see them through a
stereoscopic viewer with a rendering for each eye.
16. Explain Stereoscopic Displays
The concept of rendering a slightly different view for each eye. There is a little
shift between the pupils, and that little shift then creates a depth view for our
eyes. (illusion of depth)
17. Explain CAVEs
CAVEs are different display technology. It also creates a VR display, but in
this case, you are actually stepping inside what is called a Cave Automatic
Virtual Environment (CAVE).
A VR CAVE is a virtual reality space; essentially an empty room in the shape
of a cube in which each of the surfaces – the walls, floor and ceiling – may be
used as projection screens to create a highly immersive virtual environment.
18. What is Three Degrees of Freedom (3DOF) Tracking? Explain!
3DoF head-tracking means you can only track orientation.
3 DoF means orientation tracking. This means the 3 axes which an object can
be rotated about are tracked. if you turn your head while wearing a headset, it
is able to tracked the angle change of this axes and allows you to look around
in the environment.
19. What is Six Degrees of Freedom (6DOF) Tracking? Explain!
6DoF head-tracking means you can track both orientation and position. If
you'd like to walk around the reality you are experiencing then a user would
require a 6 DoF headset.
6 DoF VR headsets allows for the position of the headset to be tracked, as
well as the orientation of the headset. Movement can be tracked in 3 axes x,
y and z and any combination of these three axes can express movement, this
is called a vector.
20. Outside In vs Inside Out Tracking – explain
Outside-in tracking – you have some sensors in the environment that are
pointing towards you and they track you as you move around in a certain area
that is covered by these sensors.
Inside-out tracking means you have those sensors inbuilt into the device, the
cameras are inside the device
21. What is Hand Tracking?
Track your hand's position, depth, pace, and orientation. This tracking data is
then analyzed and translated into a virtual, real-time representation of your
hands and their movements inside the virtual world.
22. What is Spatial Audio?
- sound effect that manipulate the sound produced by stereo speakers,
surround-sound speakers, speaker-arrays, or headphones. This frequently
involves the virtual placement of sound sources anywhere in three-
dimensional space, including behind, above or below the listener.
Spatial audio is a powerful way to fully immerse a user and direct attention
within a 360 video or VR experience via sound.
23.Key Characteristics of AR
- AR incorporates three features: a combination of digital
and physical worlds, interactions made in real time, and
accurate 3D identification of virtual and real objects.
24.AR Displays (and give an example)
- Head-worn (example HoloLens 2)
- Spatial (projector and camera setup with projection
mapping)
- Hand-held (smartphone; for example using Google
Tango)
- Monitor-based (don’t need to wear anything, just look at
the screen)
25.AR Concepts
26.What is Augmentation and Mediation?
Augmentation – adding something (e.g.objects)
Mediation emphasizes the fact that we can mediate how
you perceive your surroundings. Mediation includes
overlaying virtual objects on "real life," and taking away or
otherwise visually altering objects.
Alters our perception of reality by adding and removing
information through a device such as a headset or
smartphone in real-time. In other words, augmented reality
plus diminished reality equals mediated reality. Diminished
reality – removing objects from the environment
27.Strong AR vs Weak AR. Give examples!
28.What is Marker-based AR?
Marker based AR, also known as image recognition AR,
requires a trigger photo or QR code to activate the
experience. The user is able to scan the marker using their
phone camera and the digital experience will appear. This
allows the user to move around the marker and see the
digital experience in 3D
29.What is Marker-less AR?
Unlike marker-based AR, markerless AR does not need a
marker such as a QR code to be scanned. You simply
launch the mobile app or web AR experience. Then it scans
the real environment around you, placing the digital
element on a flat surface such as the ground or a wall.
30.What is SLAM (Simultaneous Localization and
Mapping)? 31.What is VIO (Visual Inertial Odometry)?
-
32.What is Tracking?
33.What is Registration?
Tracking and Registration – process of positioning virtual
objects with respect to the real world.
The registration in augmented reality is a process which
merges virtual objects generated by computer with real
world image caught by camera.
34. What is eld of view?
-Field of View is the terminology which explains how "big" an augmented reality
image is when viewed through a headset or smartglasses.
35. Explain the Plane Detection & Object Recognition
-Augmented reality (AR) plane detection lets learners place a 3D model of any
object onto a tabletop or oor surface in front of them. Once placed, learners can
complete any of the following actions:
Rotate and resize the object
Walk around the object
View internal components of the object
fi
fl
-AR Object Recognition attaches a digital 3D model to a real-world object that
learners can pick up and manipulate.
36. AR Technologies
37. Explain the Marker Tracking
-This technique gives AR applications a simple way to guess the position and
orientation, in 3d space, of an object in a video stream.
38. Explain the Motion Tracking
39. Explain the Body Tracking
-3D reconstruction of the body
-Body tracking is a playful and expressive use of AR because it unlocks the ability
to wear otherworldly digital garments or avatars that mimic your movements. It
can be rephrased to human detection.
40. What is Spatial Mapping?
-Spatial mapping provides a detailed representation of real-world surfaces in the
environment around the HoloLens, allowing developers to create a convincing
mixed reality experience. Spatial mapping makes it possible to place objects on
real surfaces.
41. Explain Scene Understanding
-It estimates approximately where the object is placed so it can orient
accordingly. For example if we have a camera placed from the right side of scene
and there is an object showing on some plane, if we start to move our hand
through that object, this means that the camera will adjust showing the object in
terms of where our hand is placed. If it is from the right side of the object, the
object won’t be visible, if it is from the left side of the object, the hand won’t be
visible.
Scene understanding, provides structured information about your environment, it
uses spatial mapping capabilities of the device. It can classify where we have
oor, wall, ceilings
fl
42. Explain the Light Estimation
-Lighting estimation, brie y, refers to obtaining lighting information from
environment scenes. Adjusting the light in assembly with the overall light in the
room. For example if we turn off the lights in the room, the object is adjusting
and it’s getting brighter, i.e it ts better with the light in the room.
43. VR vs AR – key differences
fl
fi
Lecture 3
1. Explain the Technical HCI (Human Computer Interaction) Perspective,
which involves People, Tasks and Technology
as people advance technology, new things will open up for people, who
communicate circularly with each other, i.e. technology takes care of how far
people can interact, tasks take care of how to help people during their interaction
with technology, and technology also cares how to make people's tasks easier.
2. What are Tethered Technologies?
Digital Technologies that typically require a constant electrical source, such as
desktop computers. Tethering is the act of connec ng two devices together so that
one device can use the data of the other.
3. What are Standalone Technologies?
Standalone is the ability of a technology to work independently without being
connected. This usually means a technology designed to work without an internet
connection but can also refer to other types of connection such as a self-powered
building that doesn't need to connect to a grid.
4. What are Build-In Technologies?
A build is a version of a program that, is a pre-release version and is iden ed by a
build number rather than by a release number.
ti
ti
fi
5. What are Cloud-Based apps?
A cloud-based app is an Internet-run program with components stored online
with some (or all) processes executed in the cloud.
6. What is Cross Reality?
Extended or Cross Reality (XR) systems are immersive technologies that bring
physical objects into digital environments and take digital objects into physical
reality.
7. What are Key Issues in XR?
Social and Ethical Concerns,
Accessibility and Equity Concerns,
Privacy and Security Concerns.
8. Explain the Ethical & Social Concerns in XR. What is a particular issue
related to social and ethical implications of XR use that is important to you?
One of the biggest concerns about VR and AR is the social and ethical concerns
because rstly AR products can be used for facial recognition so that people
never have any real privacy, also with VR there is a concern over increasing
social isolation with the rise of such technologies.
9. Explain the Accessibility & Equity in XR. What is a particular issue
related to social and ethical implications of XR use that is important to you?
Accessibility and equity is how can we make XR available for everyone
especially how can it be available for impaired users, broaden its access and how
to increase the user abilities using XR.
10. Explain the Privacy & Security in XR. What is a particular issue related
to social and ethical implications of XR use that is important to you?
There is a big privacy concern when using technologies such as this. because
they can steal personal
fi
information and controlling access to our data. What I consider problematic and
scary at the same time is managing personal data,because personal data will no
longer just include people's credit card numbes, activity on social media and so
on. Personal data will mean everything taht makes someone a peroson, for
example: feeling,physical apperance,behaviours...
11. Which of the concerns are most important to you—social and ethical
concerns, accessibility and equity, or privacy and security? Give an example.
individual answer but here are two possible answers
What I consider problematic and scary at the same time is managing personal
data,because personal data will no longer just include people's credit card
numbes, activity on social media and so on. Personal data will mean everything
taht makes someone a peroson, for example: feeling,physical
apperance,behaviours... On the other hand the interesting thing for AR/VR is that
they are changing our lives. It can create a virtual environment for us, we can
experience things that maybe we are scared to try in real life.
For example if someone reaaly want to drive a rollercoaster but it’s to scared
to do that in real life, it will be easy to experience it with VR with knowing
that there is is no danger of anything dangerous happening to him.
Lecture 4
1. What is Digital Prototyping?
Digital prototyping is a process by which a new product
is designed and a prototype for it is created digitally,
rather than physically. Prototype is a sample or a model
to test the concept.
2. Explain the XR Design process
Sketching: best way to visualize our ideas is to sketch
them by using a piece of paper, a whiteboard or a digital
tool.
Wireframes:which are simple line drawings, like blueprints
and they lay out the size and structure of page elements,
navigation and site features.They help the designer
understand the ows, contents and the elements of
design. Two types: low delity and high delity.
Prototype: a sample, or a model to test the concept a
concept or process.
3. Explain the Signi cance of Prototyping
A prototype is an early sample or mock-up of the product
we wish to build. It’s a quick model explaining the actual
plans for the nal product. Prototypes are built using
various types of tools, including Sketch-type tools for
designing and screen design tools, to complete,
integrated design-to- code tools.
Adobe XD supports dynamic and interactive prototypes
for the designs to interact between di erent layouts. We
can add animations transitions to simulate the ow of our
application. It provides a real-time preview of the
application. In design mode, we create the design for our
application using XD tools and design features. Then we
switch to prototype mode to specify the interactive links
between artboards and components.
Lecture 5
1. Explain the XR Process.
Need nding and Brainstorming: faming the problem via
scenarios and use cases, personas and competitors.
Storyboarding and Prototyping: creating mockups using
paper and digital tools, involving XR devices.
fi
fi
fl
fi
fi
ff
fi
fl
Development and Testing: creating application using
WebXR, Unity/Unreal or native SDK.
Deployment and Analytics: deploying application on XR
device, collecting data during usage.
2. What are the three main approaches to XR
development? Give a short explanation about each of
them.
Three main approaches to XR development: web-based
using WebXR, cross-platform using Unity and Unreal, and
native using speci c platforms’ SDKs such as Oculus,
VIVE, ARKit, or ARCore.
Web-based: Web-based means you will build on web
technologies around the WebXR standard to create AR/
VR apps.
Cross-platform: Cross-platform means you would use a
development toolkit that allows you to develop against
multiple AR/VR devices using a common set of methods
and tools. Two of the most popular examples of this
approach are Unity and Unreal.
Native: Finally, native means you would be using a
particular software development kit (SDK) for a speci c
AR or VR device. Examples of this are the Oculus, VIVE,
or Windows Mixed Reality (WMR) SDKs for VR and ARKit
or ARCore for AR.
3. Give a short comparison between 2D and 3D
objects.
fi
fi
4. Explain 2D vs. 3D Transforms.
2D is " at", using the horizontal and vertical (X and Y)
dimensions, the image has only two dimensions and if
turned to the side becomes a line. 3D adds the depth (Z)
dimension. This third dimension allows for rotation and
visualization from multiple perspectives.
5. Explain 2D vs. 3D Interactions.
For mouse or touch input, we only have to deal with 2D
coordinates and usually don’t have a lot of overlapping
elements in an interface. If you do, we can control the
order of elements via z-index, which determines which
elements are rendered on top and rst respond to input.
Again, in 3D, the user’s distance and angle towards a 3D
object have a huge in uence on its rendering size and
shape, so selecting objects can be a lot harder.
So, clicking in 2D return all elements taht contain the
mouse position. Clicking in 3D cast ray from mouse into
scene, return all intersected elements.
6. What are 3D primitives?
fl
fl
fi
3D Primitive materials are basic shapes that are used in
3D modeling software. Clip Studio Paint comes with ve
types of primitive: Cube, Sphere, Prism, Pyramid, and
Plane. You can add textures and combine them freely, as
well as use them in backgrounds.
7. What is WebXR?
WebXR is an API on web browsers, that allows the XR
hardware (such as headsets and mobile devices) to work
on websites through the use of WebGL. This means that
when you visit a webpage you will be able to interact with
the 3D content regardless of what device you are on.
8. What are WebGL and Three.js?
WebGL (Web Graphics Library) is a JavaScript API for
rendering high- performance interactive 3D and 2D
graphics within any compatible web browser without the
use of plug-ins.
Three.js is a 3D library that tries to make it as easy as
possible to get 3D content on a webpage. Three.js uses
WebGL to draw 3D. WebGL is a very low-level system
that only draws points, lines, and triangles. To do
anything useful with WebGL generally requires quite a bit
of code and that is where three.js comes in. It handles
stu like scenes, lights, shadows, materials, textures, 3d
math, all things that you'd have to write yourself if you
were to use WebGL directly.
ff
fi
9. What is AFrame?
A-Frame is a popular open-source web framework for
building virtual reality experiences in the browser. It is
based on top of HTML, making it super easy to get
started. But A-Frame doesn’t just produce a static visual
3D scene. The language’s core is a entity-component
framework that can be programmed extensively by
three.js, a JavaScript library used to render 3D graphics.
