12303: Virtual Reality

Lecture 2
Visual immersion

Elizabeth Martinez - Byron Pérez Gutiérrez

Mechatronics Engineering
Universidad Militar Nueva Granada

2023
Visual system

For the development of

software and hardware that
seek to take advantage of the
properties of the human visual
system it is important to first
know its characteristics and
features.
https://pixabay.com/es/photos/ojo-color-iris-punto-de-vista-229957/

2
What is vision and its purpose?

• Process that produces from the

images of the external world a
useful description for the observer
• The effectiveness of the
representation depends on what
will be used
• Spiders use to hunt or search for
their peer
• Rabbits to detect aerial predators
"lrI32A9994-2" by Shahin Abasov is licensed under CC BY-ND 2.0

3
What is the eye?

Is a visual organ that detects light and converts it into

electrochemical impulses that travel through neurons to the
brain.

"Ian's Eye" by dbang is licensed under CC BY-NC-SA 2.0

4
Head anatomy

https://commons.wikimedia.org/wiki/File:1106_Front_and_Side_Views_of_the_Muscles_of_Facial_Expressions.jpg https://cnx.org/contents/[email protected]:1w-m01MB@4/The-Skull

5
CT scan

Computed tomography of the

head.
– Nose
– Eyeball
– Optic nerve
– Skull
– Brain

6
Eye anatomy

7
Eye anatomy

8
 Retina

Ophthalmoscopy: test to see inside the fundus of the eye using an ophtalmoscope.

9
Eye anatomy

10
 Retina

• 126 million photo-receptors

• The fovea is the area of high
resolution and color
perception.
• The portion of the image
projected on the fovea
represents the area of focus.

11
 Retina

• Rods: 120 million. Dark-

adapted vision
• Cones: 6 million. Color vision
• Cones are more concentrated
in the central yellow spot known
as macula.
• In the center of this region is the
"central fovea", an area of 0.3
mm in diameter.

12
Color blind test

13
Refractive errors
• Myopia. Close objects are clear, and
distant objects are blurry.
• Hyperopia. Close objects are more blurry
than distant objects.
• Presbyopia. Aging of the lens in the eye.
• Astigmatism. Astigmatism usually occurs
when the front surface of the eye, the
cornea, has an asymmetric curvature.

14
 Stereo vision

• 120 vertical
• 150 horizontal for each eye
• 180 horizontal for both eyes
[Kalawasky, 1993]
• Stereopsis area: both eyes record
the same image.

15
Stereo vision

16
Stereo vision

The brain uses the

horizontal shift in the images
to measure the depth and
distance of objects from our
position.

17
 Normal vision
• The curved cornea bends the light into the eye and
the lens changes shape to bring things into focus

• When you look at things that are far away, muscles

in your eye relax and your lens looks like a slim disc:
Contracted muscle

Thicker lens Near object

• When you look at things that are close, muscles in

your eye contract and make your lens thicker:
Expanded muscle

Thinner lens Far object

18
 Reference plane
Crossed vision
• The right eye focuses on left side and the left eye focuses on right
side on a reference plane.
• In the reference plane two images are presented to the user:

Left side:
perspective for right eye

Right side:
perspective for left eye

Stereo image created on the brain

Left side: perspective for right eye Right side: perspective for left eye

19
 Stereo image created
on the brain
Reference plane
Parallel vision
• The right eye focuses on right side and the left eye focuses on left Left side:
perspective for left eye
side on a reference plane.
• In the reference plane two images are presented to the user:

Right side:
perspective for right eye

Left side: perspective for left eye Right side: perspective for right eye

20
Is a crossed or parallel vision image?

21
Is a crossed or parallel vision image?

22
Is a crossed or parallel vision image?

A Cone or a tunnel?

23
Is a crossed or parallel vision image?

24
Is a crossed or parallel vision image?

25
Is a crossed or parallel vision image?

26
Parallel and crossed vision examples
• 3D Roller Coasters S VR Videos 3D SBS [Google Cardboard VR Experience]
https://www.youtube.com/watch?v=ix9Ioymijfw
• VR VIDEOS 3D SBS Underwater for VR BOX 3D not 360 VR
https://www.youtube.com/watch?v=LpQYs54IY5Q&pbjreload=101
• F1 Formula Racing in Virtual reality - Grand Prix Race Highlights 3D VR SBS
HTC Vive https://www.youtube.com/watch?v=2sCmWmOGqMY
• 3D without glasses, Cross-Eye HD https://www.youtube.com/watch?v=zBa-
bCxsZDk&t=21s
• 3D Without Glasses ...or VR... is back! (Cross-eye)
https://www.youtube.com/watch?v=Tt0d9DhnmGc

27
Stereo cameras

• Two pictures taken at the

same time
• The lenses center are
separated according to
the interpupillary distance
(IPD) of the human eyes.

Kodak camera from 1905

28
Stereo cameras

Kodak camera from 1955

29
Stereo cameras

Canon camera array from 2005

30
Stereo cameras

Fuji FinePix Real 3D camera from 2009

31
Stereo cameras

Panasonic 3D camcorder

32
Stereo cameras

Nintendo 3DS with stereo camera

33
Stereo filtering techniques

• Wavelength filtering: Anaglyph (red-cyan,red-blue, red-

green, etc)
• Polarized filtering: Linear, circular
• Time multiplexing: Active stereo (shutter glasses)
• Autostereoscopic screens: (Parallax barrier)

34
Anaglyph

Anaglyph 3D is the name

given to the stereoscopic 3D
effect achieved by means of
encoding each eye's image
using filters of different
(usually chromatically
opposite) colors, typically
red and cyan
Gizmodo – How 3D works

35
Anaglyph

36
Anaglyph glasses

37
Anaglyph example

Left image Right image

RED CYAN

38
Anaglyph example

39
Anaglyph example

40
Anaglyph example

Anaglyph render with a stereo camera in Autodesk Maya

41
Stereogram

A stereogram is a picture within a picture. Hidden inside each image is an object which appears in 3D when viewed correctly (doing parallel vision).

42
Stereogram

A stereogram is a picture within a picture. Hidden inside each image is an object which appears in 3D when viewed correctly (doing parallel vision).

43
Stereogram

44
Stereogram

45
Polarized filtering

The light of each eye is polarized in

quadrature (vertical/horizontal,
clockwise/counter clockwise)

46
Polarized filtering

47
Time multiplexing filtering
Infrared emitter

48
Time multiplexing filtering: 3D with no glasses

http://www.jonathanpost.com/
http://cargocollective.com/jpost/filter/Projects/3D-No-Glasses
https://www.youtube.com/watch?v=Uef17zOCDb8

49
Time multiplexing vs polarization glasses

LG Cinema 3D https://www.youtube.com/watch?v=LOFCRuWN5kg

50
 3D Format: top-down L R

Original stereo pair

http://www.bke.org/Bayreuth2005/CosimaHeadStereo.htm

Top-down image Stereo image

Stretching

51
3D Format: side-by-side
L R
Original stereo pair
https://www.mediacollege.com/photography/types/stereoscopy/

L R

Side-by-side image Stereo image

Stretching

52
3D Format: interlaced images

Left image Column interlaced image Checkered interlaced image

Right image Row interlaced image

3D interlaced image

53
 3D Format: Frame-packed
The frame
1920 pixels The encoded signal: 24 fps delivered by the player

1080 pixels
L 1920 px

L L L L L L L L

2160 px
2160 pixels R R R R R R R R
1080 pixels
R 1 2 3 4 5 6 7 8

The projected/displayed signal:

1920 px 48 fps displayed by the television or projector with shutter glasses
1080 px

L R L R L R L R L R L R L R L R
1 2 3 4 5 6 7 8

54
 3D Format: Top-bottom progressive
The frame
1280/1920 pixels The encoded signal: 60 fps delivered by the player

1280/1920px

360/540 pixels

720/1080 px
Half vertical
resolution
720/1080 pixels
1 2 3 4 5 6 7 8
360/540 pixels

The projected/displayed signal:

1280/1920px
120 fps displayed by the television or projector with shutter glasses
720/1080 px

1 2 3 4 5 6 7 8

55
 3D Format: Side-by-side progressive
The frame
960 960
pixels pixels
The encoded signal: 60 fps delivered by the player

1920px
1080 pixels

1080 px
1920 1 2 3 4 5 6
pixels

Half horizontal
resolution

The projected/displayed signal:

1920px 120 fps displayed by the television or projector with shutter glasses
1080 px

1 2 3 4 5 6 7 8
Each image is stretched to 1920px on the screen

56
 3D Format: Side-by-side interlaced
Single filed (1/2 frame)
960 960
pixels pixels
The encoded signal: 30 fps delivered by the player

1920px
540 pixels

1720/1080 px
1920 odd even odd even odd even
pixels
1 frame at 30fps
Half vertical and horizontal resolution (60 fields per sec)

The projected/displayed signal:

1920px 120 fps displayed by the television or projector with shutter glasses
1080 px

1 (odd field) 1 (even field) 2 (odd field) 3 (even field) 3 (odd field) 4 (odd field) 4 (even field) ….
1 frame at 120fps

57
Stereo vision equipment

Head Mounted Displays 3D TV / Monitor

Large screens with projection

CAVE Autostereoscopic 3d displays

58
Head Mounted Displays - HMD

Virtual image generation Mobile phone HMD CRT HMD

59
Stereoscopes

Wheatstone mirror stereoscope Carl Zeiss Holmes

Brewster http://loc.gov/pictures/resource/ppmsca.08781/ https://en.wikipedia.org/wiki/Stereoscope

60
Modern stereoscope

Carl Zeiss stereoscope - http://bibliotecadigital.ciren.cl/handle/123456789/26233

61
 http://www.oculus.com

Oculus

Development Kit 1 Development Kit 2 Rift (CV 1)

Go Quest S

62
Meta Quest (Oculus) https://www.meta.com/quest/

Quest Pro Quest 2 Quest

63
HTC Vive http://www.vive.com

Vive (2160×1200 (1080×1200 per eye) , FOV 110°) Vive Pro 2 (5K, FOV 120°)

Vive XR Elite (1920*1920 pixels per eye (3840*1920 pixels combined), FOV 110°)

64
Pico interactive

Neo G2

https://www.pico-interactive.com/

65
VALVE INDEX

https://store.steampowered.com/valveindex

66
PSVR

https://www.playstation.com/en-us/ps-vr/

67
OSVR

Hardware development kit 2 https://www2.razer.com/osvr https://osvr.github.io/

68
Professional HMDs: VRGineers

XTAL 3 VR: 3840×2160 per eye (8k), FOV 180° XTAL 3 MR: 3864 x 2192 pixel per eye, FOV 63°/84°/170°
https://vrgineers.com/xtal-3-virtual-reality/ https://vrgineers.com/xtal-3-mixed-reality/

69
Professional HMDs: Varjo

Varjo VR-3: 2880x2720 per eye, FOV 115° Varjo XR-3: 2880 x 2720 px per eye, FOV 115°
https://varjo.com/products/vr-3/ https://varjo.com/products/xr-3/

70
Mobile phone HMD

Durovis dive
Google Daydream

Google Cardboard VR Box Samsung Gear VR

71
3D TV and videowall

3D displays Videowall

72
 CAVE Automatic Virtual Environment - CAVE

1991 – Electronic Visualization Lab. University of Illinois at Chicago, Dr. Carolina Cruz-Neira https://www.youtube.com/watch?v=-Sf6bJjwSCE

73
CAVE Automatic Virtual Environment - CAVE

74
CAVE Automatic Virtual Environment - CAVE

75
CAVE examples
• Caterpillar https://www.youtube.com/watch?v=r9N1w8PmD1E
• Université de Caen Normandie https://www.youtube.com/watch?v=iik95LoBGxI
• StarCAVE UCSD Calit2 https://www.youtube.com/watch?v=bAO5zhgdRUE
• SimuTrain Canada https://www.youtube.com/watch?v=MDx3H8jobO4
• RWTH Aachen aixCAVE https://www.youtube.com/watch?v=x02-Um-xIIY
• VisionaiR 3D front projection CAVE https://www.youtube.com/watch?v=Gb9ayYGM-4c
• Transportable VR Cave https://www.youtube.com/watch?v=_3hYbzZHdU0

www.umng.edu.co 76
 https://www.youtube.com/watch?v=z-JpD91NPvs
CAVE with TVs https://www.youtube.com/watch?v=bI7Qj2RrnVE
https://www.youtube.com/watch?v=jZt0mP-ZkUU

77
CAVE 2 – University of Illinois https://www.youtube.com/watch?v=yf0sllpZx3w

78
Autostereoscopic displays

http://www.sharp-world.com/products/device/about/lcd/3d/index.html https://www.youtube.com/watch?v=-tq2LObfdr4

79