Human Perception System

Virtual Reality 2. Human perception system NCU IPVR Lab.
2. Human Perception System

2.1 Introduction ………………… 2
2.2 Human visual system …….. 11
2.3 Color spaces ……………..… 62
2.4 Human haptic system …….. 86
2.5 Human auditory system …... 97
Virtual Reality 2. Human perception system NCU IPVR Lab. 2
2.1 Introduction
 視覺是人類最重要的知覺，而視覺中最重要的是色彩與立體感覺。
 顏色是完完全全要給眼睛看的。
 顏色是不存在的。
 顏色完全是不同波長的電磁波刺激眼睛視網膜上的三種錐狀感光細
胞 (cone) 所造成的。三種錐狀細胞受光線刺激後會產生脈衝
(impulse) 傳輸到大腦的視覺皮質 (visual cortex) 。三種錐狀細胞各
有其不同的感光曲線，因而可以在視覺皮質中組合出多種不同的感
受，不同的感受造成不同顏色的認知。
 顏色的認知是後天學習而來的，不是與身俱來。
 所以顏色是大腦所產生的感知，而不是存在我們的環境中。
 Three components of studying color science:

(1) physics and chemistry of color,
(2) physiology and psychophysics of color vision, and
(3) colorimetry: the science of measuring color quantitatively.
 Light is the electromagnetic radiation that stimulates our visual

response. It is expressed as a spectral energy distribution L(λ),
where λ is the wavelength that lies in the visible region, 380 nm
to 780 nm, of electromagnetic spectrum, where 1 nm =10-9 m.
Cosmic Gamma x Ultra
Infrared Microwaves TV Radio Electric power
rays rays rays violet
10 -5 10 -3 1 nm 10 nm 10 -4 m 10 -2 m 1 m 10 2 m 10 3 m 10 6 m
400 nm 500 nm 600 nm 700 nm
 White light (sun light) consists of seven visible color lights

which are red, orange, yellow, green, blue, indigo, and purple.
 Any combination of these colors will produce another color.

 Due to the structure of the human eye, all colors are seen as
variable combinations of the three so-called primary color RGB.
For the purpose of standardization, the CIE designated in 1931
the following specific wavelength values to the three primary
colors: blue = 435.8nm, green = 546.1nm, red = 700nm.
綠洋紅
黃靛藍紅
白黑
紅洋紅藍靛綠黃
 The principle of subtractive light mixture

 The amounts of red, green and blue needed to form any given
color are called the tristimulus values and are denoted,
respectively, by X, Y and Z. A color is then specified by its
trichromatic coefficients (x,y,z),
X Y Z
x , y , z
X Y  Z X Y  Z X Y  Z
and
x + y + z = 1.
 Three tristimulus values construct a 3-D geometric structure of

color space.
 Two trichromatic coefficients construct a 2-D chromaticity

diagram. That is, a chromaticity diagram is used for displaying
all visible color on a 2-D map.
 The characteristic generally used to distinguish one color from

another are brightness, hue and saturation.
brightness = intensity (明暗度)
hue (色調，與波長有關)
saturation (飽和度，指加入多少比率的白光).
We can take
hue + saturation = chromaticity, then we have
color = [brightness, chromaticity].
 Pure color on the boundary of the diagram. It is said to be
completely saturated. The point of equal energy corresponds to
equal fractions of the three primary colors; it represents the CIE
standard for white light; and the saturation at the point is zero.
 The chromaticity diagram is useful for color mixing.
 The color space is a 3-D solid model and a 2-D chromaticity
diagram is just a cut plane of the 3-D model; the plane is
perpendicular to the intensity axis of the color model.
2.2 Human visual system

A. Structure of visual system
 The structure of the human
visual system in the horizontal
cross section of a head. The
visual neurotic fibers of two
eyes are semidecussation
(top view).
 The structure of the human eye in the horizontal cross section of

a right eye (top view).
太陽穴側鞏膜 (第一層)
脈絡膜 (第二層)
虹膜玻璃體
視網膜 (第三層)
角膜水晶體
視軸
中心凹
房水視神經
盲點
鼻側
 The relative sizes and positions of the macula and optic disc in
the left eye (back view).
5o
太陽穴側鼻側
5o
7o
2o
盲點
中心凹
15o
 Courses of the capillaries (heavy lines) and nerve fibers (light

lines) (back view).
 Blind axis
Side view Top view
invisible
15o
Optic axis
2o Optic axis
invisible
Optic axis
15o
invisible
 Demonstrating blind regions

Look at the below figure. Hood the book about 45 cm away,
close your right eye, and fixate on the cross with your left eye.
Move the book around slightly, and eventually you will find a
position where the black circle disappears. At this position, the
spot is images on the optic disk of the left eye. This region of
visual space is called the blind spot for the left eye.
Look at another figure below. Fixate on the cross with your left
eye. You will find that where the break in the line falls is imaged
on the blind spot, your perception will be of a continuous,
uninterrupted line.
 The retina of the human eye contains two types of photoreceptors

called rods and cones.
桿狀細胞
錐狀細胞
視紫質
粒線體
細胞核
神經鍵
樹突
 The schematic diagram of the primate retina structure.

桿狀細胞
傳輸至大腦
脈衝視
視神光
網經
膜
神經節細胞
錐狀細胞二極細胞
 The rods, about 130 million in number, are relatively long and
thin. They provide scotopic vision, which is visual response at the
lower several orders of magnitude of illumination.
 The cones, about 7 million in number, are shorter and thicker and
less sensitive than the rods. They provide photopic vision, the
visual response at the higher 5 to 6 orders of magnitude of
illumination. The cones are also responsible for color vision.
 Rod and cones 細胞中帶有對光線十分敏感的視紫質 (Rhodopsin)。
視紫質碰到光線時會立刻產生化學反應，視紫質吸收光能，分解成
維生素 A 醛和視蛋白而變成白色，同時產生電衝動 (impulse) 傳給
兩極細胞。維生素 A 醛和視蛋白在黑暗中又會結合成視紫質，以便
再次循環使用。在眼睛完全適應黑暗時，每個 rod 細胞中有 107 個
視紫質分子。
Light
Rhodopsin Vitamins A + Protein + Impulse
Enzyme
Vitamins A + Protein + Nutrition Rhodopsin
(酵素)
 Adaptation of light intensity (眼睛對光線亮度的調整能力)
 眼睛對於光線亮度 (light intensity) 的感受相當敏感。眼睛感受亮度

的範圍從最亮到最暗的可感受亮度差異可達 1013 倍。沒有人造的
亮度感應器具有如此大的感光範圍。 [Kalawsky, 1993]。
 在黑暗中逗留 1 分鐘後，眼睛對光的敏感度會提高 10 倍，逗留 20
分鐘後，敏感度會增加為 6,000 倍，逗留 40 分鐘後，敏感度會增
加為 25,000 倍，這是人眼對光線敏感度的極限。
 The distributions of rods and cones
視軸每
mm2
的 180,000 盲點
桿 160,000
狀 140,000 桿狀細胞桿狀細胞

120,000
及
100,000
80 o 錐
80 o 80,000
狀
60,000
60 o 60 o 細
盲點胞
40,000
20,000
40 o 40 o 數錐狀細胞錐狀細胞
20 o 20 o 80∘70∘60∘50∘40∘30∘20∘10∘0∘10∘20∘30∘40∘50∘60∘70∘80∘
0o
以中心凹為中點的視角
中心凹視神經
 The density of photoreceptors in human eye is about 200,000

per mm2, but that in the hawk (獵鷹) is over 1,000,000 per mm2;
thus hawk eye is sharper than human eye.
 Foveal cone inner-segment mosaic with 10μm scale bar

(1μm = 10 -6 m).
 The luminance efficiency function of the visual system
For the human eye, the function is a bell-shaped curve whose

characteristic depend on whether it is scotopic or photopic vision.
Why ?
Most raincoats are yellow.
 There are three different types of cones in the (normal) human

retina with absorption spectra S1( ), S 2 ( ), and S 3 ( ) based on
different kinds of Rhodopsin, where  min     max with
 min  380 nm and  max  780 nm.
10 5
10 4
敏 3
感 10
度 10 2
10
1 波長
400 500 600 700
Why ?
Someone has color blindness.
 Based on the three-color sensitivity, the spectral energy

distribution of a "colored" light, C(λ), will produce a color
sensation that can be described by spectral responses as
 i (C )  min
max
S i (  )C (  )d, i  1, 2, 3.
 If C1() and C2() are two spectral distributions that produce

responses and such that  i (C1)   i (C 2), i  1, 2, 3,
then the colors C1 and C2 are perceived to be identical. Hence
two colors that look identical could have different spectral
distributions.
 The luminance, intensity, or lightness of a spatially distributed

object with light distribution I(x,y,λ) is defined as
f ( x, y )   max I ( x, y ,  )v (  )d ,
min
where v(λ) is the relative luminance efficiency function of the

visual system.
 The influence of sensitivity for visual system
Optic sensitivity of photoreceptors

sensing intensity
Number of photoreceptors
sensing speed
Density of photoreceptors
B. Properties of human vision
 Brightness range of the eye.

Typical to all psychometric
functions, the apparent
brightness is not a linear
function of the luminance,
rather than two logarithmic
functions.
Lateral inhibition
 Lateral inhibition refers to an architecture for neural networks in

which neurons inhibit spatially neighboring neurons.
 Lateral inhibition is a physiological mechanism of human vision;

more accurately speaking, lateral inhibition is one of the most
pervasive architectures in the human visual nervous system. The
phenomena:
simultaneous lightness contrast,
simultaneous color contrast,
Mach bands, and
Hermann grid illusion
are all belonging to the effect of lateral inhibition.
 The luminance of an object is independent of the luminances of

the surrounding objects. The brightness (or apparent brightness)
of an object is the perceived luminance and depends on the
luminance of the surround.
All small squares have exactly the same intensity.

 The perceived color depends on the color of the surround.
The upper and lower circles have exactly the same color.
How many colors in the picture ?

 Mach band effect
 The Hermann grid illusion

 The illusion is one of lateral inhibition effects. In which illusory

dark spots seem to occupy the light intersections. One can
convince himself that the darks spots indeed illusory because
when he look directly at an intersection, the spot disappears. That
is, dark spot appear at every intersection except the one on which
he is fixated.
 The perception of the dark spots can be simulated by a lateral

inhibitory network because the amount of inhibition at the light
intersections is appreciably greater than that in other light regions.
Each intersection is surrounded by light regions on four sides, in
contrast to just two in other places.
 The fact that dark spots do not appear when you look at them
directly can be explained if there is less lateral inhibition in the
fovea than in the periphery.
A simplified model of lateral

inhibition in a 1-D neural network.
Lateral inhibition occurs in a two-layer
network when each neuron in the first
layer excites a corresponding neuron
(or set of neurons) in the second layer
but inhibits the second-layer neurons
lateral to the excitatory connection. In
this diagram, excitatory connections
(arrowheads) between layers are
indicated by positive numbers, and
inhibitory connections (black dots) are
indicated by negative numbers. As
with most neural network models, the
output of a target neuron is calculated
by multiplying the activation of each of
its input neurons by the weight of its
connection to the target neuron and
summing these products over all
inputs.
Lateral inhibition implemented in a 2-D network. (A) A single receptor

projects to many ganglion cells in a center/surround organization. (B)
Each ganglion cell receiving such connections therefore has a
center/surround receptive field. The illustrated network exhibits an
excitatory center and inhibitory surround, but the opposite organization
(inhibitory center and excitatory surround) also exists in the retina.
Lateral inhibition and

lightness contrast.
(A) A lightness contrast stimulus
is shown above its luminance
profile through the center
(dashed line). Notice that the
two central squares have the
same physical intensity even
though they appear different
in lightness.
(B) A lateral inhibitory network

whose output predicts that
the left square will look
darker than the right square.
 當你眼睛凝視 (fixation) 圖中的色點，刻意固定眼球不動；你

會發現圖中的淺藍或淺紅色圈會慢慢的消失；但你只要稍微
動了一下眼球，該色圈即刻出現。這個特性是瑞士物理學家
裘斯勒 (Ignaz Paul Vital Troxler) 在1804年所發現的，稱為
裘斯勒褪色或效應 (Troxler's fading or Troxler's effect)。
 刻意或不經意的讓你的視線在圖中左右移動，你會或多或少
看到似乎有三個圓筒左右滾動。這是眼球運動所造成的錯覺 -
微顫動 (microsaccade) 。
 Human visual perception (just noticeable difference, JND)
The increment threshold BT as a function of reference intensity B.
 Factors for color discrimination (影響色彩判定的因素)

(1) 色彩的調適能力 (chromatic adaptation)
目前眼睛對色彩的分辨能力會受到剛剛所看到色彩的影響。色
彩飽和度的增減也會影響色彩判定的能力。
(2) 亮度 (luminance) 亮度不足將會影響色彩的分辨能力。
(3) 視網膜上的位置 (retinal position)，視網膜中心凹處對色彩能夠
感受較大的飽和度。
(4) 持續曝光時間 (duration)
在持續曝光時間大於 0.02 秒的情況下，增加對眼睛刺激的時間
會提高對色彩的分辨能力。
(5) 物體大小 (object size) 面積太小的色彩區域不易分辨色彩。
 Accommodation of depth of field (眼睛的景深適應能力)

 瞳孔可以調整眼睛的進光量亦可改變視野景深 (depth of field)。
 瞳孔收縮到最小時直徑只有 1.5 mm，而擴散到最大時直徑則有 8.5
mm，兩者相差 5.6 倍。進入眼睛的光線相差 30 多倍。
 在瞳孔直徑為 4mm 時，注視無窮遠處，景深範圍約從眼前 3.5m
至無窮遠處；注視 1m 處，景深範圍約在 0.8m 到 1.4m 之間。
 當光線亮度增大，瞳孔直徑縮小為 2mm 時，注視無窮遠處的景深
範圍約為 2.3m 至無窮遠處；注視 1m 處，景深範圍約在 0.7m 至
1.8m 間。
radius of pupil = 4 mm
depth of field
radius of pupil = 2 mm
depth of field
景深短景深長
 View of field (視界、視角)

單眼瞬間視角 (instantaneous field of view) 約上下 120o，左右
150o。而雙眼視角約上下120o，左右 200o，中央重疊約100o。
Binocular vision
 Not all animals have binocular vision.
 Categories
1. Semidecussation (半交錯): The visual fields of two eyes of
some animals are heavily overlapped; their visual nerves are
semi-chiastic at the optic chiasma; for example, human, dog,
cat, monkey, etc. Their vision systems have stereo vision
based on the disparity.
2. Decussation (交錯): The visual
fields of two eyes of some animals
are heavily overlapped, but their
visual nerves are fully chiastic at
the optic chiasma; for example,
fogs, owl (貓頭鷹), etc. These
animals have no stereo vision.
3. Decussation (交錯): Even if mammals, the visual fields of two

eyes of some animals are less overlapped; their visual nerves
are fully chiastic at the optic chiasma, for example, rabbit,
goat, etc. These animals have still no stereo vision.
4. Decussation (交錯): The view fields of two eyes are non-
overlapped; their visual nerves are fully chiastic at the optic
chiasma, for example, fishes, birds, hawk (獵鷹), etc. These
animals have no stereo vision.
 The animals who have binocular vision move their eyes

synchronously.
 Binocular vision or stereopsis (雙眼視覺)

The depth information (range data) can be obtained by using
stereoscopic (stereo for short) imaging techniques.
影像1
yc
鏡片中心點
xc q
影像2
(x1, y1) 光軸
yc
(x2, y2)
B - zc
xc 基底線
w (X, Y, Z)
空間點
- zc
 The principal of binocular vision is using the disparity (雙眼像差)

to infer the depth or range information (深度資訊).
(0, 0, 0)
q 影像1
xc 
(x1, y1)
(0, 0, 0)
(x2, y2)
B


影像2 A - zc

w (X, Y, Z)
 Assume (X,Y,Z) is a world point, (x1, y1) and (x2, y2) are two
corresponding image points on image1 and image2, respectively.
 Suppose that we bring the first camera into coincidence with the
world coordinate system. Then by the similar triangle principle, we
have
 q Z
 
x1 A Z
  A  x1
 q  Z q
  x 2 B  A
q Z Z Z qZ
   
x2 A  B x1 Z  B x1 Z  qB x1 Z  qB
q q
 q x1 Z  q 2 B  q x 2 Z
 x1 Z  qB  x 2 Z
 x1 Z  x 2 Z  qB
qB
Z = .
x1  x 2 The term (x1 - x2) is called the disparity.
 雙眼視覺的對映問題 (correspondence problem)

The problem is to match one specified point on the left image to
the point on the right image.
Left image. Right image.
 錯誤的對映
Wrong range data come from wrong correspondence.
 Stereogram
 Stereo image pair

 Partial binocular overlap (雙眼視角部份重疊)

 單眼左右視角約為 155o，雙眼左右視角為 190o，中央約為 120o 重疊。
 雙眼視角重疊可分為收斂 (convergent) 與發散 (divergent) 兩類。雙眼視角
內側夾角重疊區域為發散重疊，雙眼視角外側夾角重疊區域為收斂重疊。
 發散重疊的光學系統通常能產生較好的立體效果 (because it allows the
combining optics to be tilted outwards.)
Divergent area. Convergent area.
 Binocular rivalry (雙眼競賽)

 幾乎每個人都有主控眼睛 (dominant eye)。主控眼睛是指能夠對景
觀或物體注視較久的一眼。當兩個不同畫面同時各別呈現在兩眼
前，視覺系統通常會抑制其中一個畫面而忽視其實質的資訊。偶而
主控畫面 (dominant image) 會在雙眼間交替著，這種現象稱為雙
眼競賽。兩眼中大小、亮度和色調不同的畫面都可能造成雙眼競賽
的現象。例如，當其一眼看到較亮的景觀，則人腦即會抑制看到較
暗的另一眼。
 在雙眼顯示器中，顯示畫面的表現方式 (representation) 及複雜度

(complexity) 會誘導雙眼競賽的效果。例如，雙眼注視彩繪影像
(raster or rendered images) 會比注視線條畫面 (vector or drawing
images) 更易引起雙眼競賽的效果 [Kalawsky, 1993]。
 一個消除雙眼競賽的方法是減弱畫面的清晰度，也就是將畫面變模
糊些。
 Depth cue (深度知覺的線索)

深度知覺的線索可分為生理 (physiological) 及心理 (psychological)
兩方面。
 生理方面的 (physiological) 線索有：

(a) Accommodation (眼睛的適應力)
眼睛看遠看近會自動調整水晶體對焦以求得最清晰的畫面。因水
晶體的改變是人類獲得深度知覺的一個線索。
q p
q : 像距
p : 物距
Retina
f : 焦距
q p
1 1 1
= p+q
f
(b) Convergence (雙眼視線的收斂性)

雙眼看遠看近眼球會左右轉動改變視角對焦以注視目標物。
(c) Binocular disparity (雙眼像差)

雙眼視網膜上之映像在水平方向的距離。
(d) Motion parallax (移動視差)

移動中的眼睛看到近距離物體的相對移動速度比遠距離物體的
相對移動速度快。
 心理方面的 (psychological) 線索：

(a) Linear perspective (線性透視效果)
物體成像在視網膜上的大小改變和物體到眼睛的距離成反比。
(size of the image of an object on the retinal changes is inverse
proportion to its change in distance.)
(b) Shading and shadows (漸層和陰影)

物體曲面的漸層顏色變化及物體造成的陰影都可以用來決定物體
與觀測點的相對距離。
(c) Overlap (重疊)

物體前後部份重疊與遮蔽可以顯示物體間的前後相對位置。
(d) Size constancy (物體大小的恆常性)
物體的大小是深度知覺的一個線索。但當眼睛看到熟悉的物體
時，心理上常會傾向認定為熟悉的大小而忽略視網膜上成像的大
小 (tendency to perceive objects of familiar size as being the
same real size, regardless of their retinal size)。如此會誤導深
度知覺。
(e) Texture gradients (紋理層次)
距離愈遠，紋理將會愈小愈密集。
 Temporal resolution (視覺時間的解晰度)

眼睛成像的時間性與照像底片的曝光時間性不同。眼睛成像的暫留
時間必需要很短促，如此才能捕獲快速的動態畫面。此短促時間可
能是數十分之一秒，因此眼睛必需具有高感度的感光細胞及快速整
合視覺資訊的功能。
2.3 Color spaces

 There are several color models (color spaces, or color
coordinate systems) which are
CIE RGB
CIE XYZ
CIE Yuv (UVW)
CIE U*V*W*
NTSC RGB
NTSC YIQ
CIE L*a*b*
CIE L*u*v*
IHS
Any three coordinates (attributes) is a basis of a color space.
CIE : Commission Internationale de L'Eclairage, the

International committee on color standards.
NTSC : National Television Standards (Systems) Committee.
(1) CIE RGB (1931)

 R, G, B = three spectral primary sources.
 Reference white : R = G = B = 1.
 There is a negative tristimulus value.
Geometric structure Chromaticity diagram
 Theoretically, the color is fully dependent on the wavelength; the

three fixed RGB components acting alone cannot generate all
spectrum colors (pure colors). This is an unresolved defect for
color presentation.
(2) CIE XYZ ( )

 Y = luminance, X = ?, Z = ?
It is truly said that these three coordinates are physically
nonrealizable.
 Reference white : X = Y = Z = 1.
 This model is modified from RGB model such that all spectral
tristimulus values are positive.
Spectral tristimulus Chromaticity diagram
dist (a,b) < dist (a,c)
cd (a,b) > cd (a,c)
cd: color distance.
a
b
c
(3) CIE Yuv (UVW) ( )

 Y = luminance, u, v = chromaticities.
 This model is modified from XYZ model to achieve a uniform
chromaticity scale (UCS).
 This is the first UCS model, but it is just uniform on uv-plane.
 The MacAdam ellipses in the non-UCS XYZ chromaticity diagram
Chromaticity diagram
MacAdam ellipses Chromaticity diagram
(4) CIE U*V*W* (1964)

 This is a modified UCS system whose origin is shifted to the
reference white in the u, v chromaticity plane.
 This model is useful for measuring color differences
quantitatively. In this model, for unsaturated colors, (i.e., for
colors lying near the grays in the color solid), the difference
between two color is, to a good approximation, proportional to
the length of the straight line joining them. That is, the unit
shifts in luminance and chrominance are uniformly perceptible.
(5) CIE L*u*v* (1976)

 This model is modified from uvY model.
 (uo, vo, Yo) is the reference white, where uo=0.201, vo=0.461,
Yo=1.
 This model is similar to the U*V*W* model.
 The color model was mainly developed for displays, while
the next L*a*b* color model was initially designed for
reflective prints.
Geometric structure Monitor gamut

(6) CIE L*a*b* (1976)

 L* = brightness
a* = red-green content (changes red/green balance)
b* = yellow-blue content (changes green/blue balance)
Neither a* nor b* corresponds to known psychophysical properties of
visual perception.
 This model gives a quantitative expression for Munsell system.
 L*a*b* is more appropriate over the small color differences
encountered near the end of a match than L*u*v* model.
 Xo, Yo, Zo = tristimulus values of the reference white.
Geometric structure Monitor gamut
(7) NTSC RGB ( )

 R,G,B = three receiver primaries for reproducing colors.
 Reference white: R = G = B = 1.
The reference white for NTSC is different from that for the
CIE system.
Geometric structure Chromaticity diagram

(8) NTSC YIQ (1953)

 This model was developed to facilitate transmission of color
images using the existing monochrome television channels
without increasing the bandwidth requirement.
 Y = luminance
I, Q = chrominances (I: inphase, Q: quadrature)
I axis encoding chrominance information along a blue-green
to orange vector. Q axis encoding chrominance information
along a yellow-green to magenta vector. Neither corresponds
to a known psychophysical quantity.
G
R
I
Q
B
 To achieve high-fidelity audio and video quality in consumer

electronic devices, the color models have been progressively
improved in the devices:
YIQ ← audio and video are composed together
A / V ← audio and video are separated
S / A ← video is further separated into two channels
Color difference / A ← video is further separated into three
channels Y Cb Cr
 The relationship diagram of these color models
1931 CIE RGB

1964
1931
CIE XYZ, Yxy NTSC RGB 1953
1964
1976 1953
CIE L* a* b* CIE Yuv, uvw NTSC YIQ NTSC YCbCr
1975 CIE Yu’v’ CIE U* V* W* 1964
1976 CIE L* u* v*
 Transformation formulas for color models.

CIE XYZ  CIE RGB NTSC RGB  CIE XYZ
 X  0.490 0.310 0.200  R  R   1.910  0.533  0.288   X 
1 Y   0.177 0.813 0.011 G . 2 G    0.985 2.000  0.028  Y .
         
 Z  0.000 0.010 0.990   B   B   0.058  0.118 0.896   Z 
CIE XYZ  NTSC RGB NTSC YIQ  NTSC RGB

 X  0.607 0.174 0.201 R  Y  0.299 0.587 0.114  R 
3 Y    0.29 0.587 0.114  G . 4  I   0.596  0.275  0.321 G .
         
 Z  0.000 0.066 1.117   B  Q  0.212  0.523 0.311   B 
NTSC RGB  NTSC YIQ CIE RGB  NTSC RGB

R  1.000 0.956 0.621  Y  R   1.167  0.146  0.151 R 
5 G   1.000  0.273  0.647   I . 6 G    0.114 0.753 0.159  G .
         
 B  1.000  1.104 1.701  Q   B   0.001 0.059 1.128   B 
 Transformation formulas for color models (continued).

UVW  NTSC RGB UVW  CIE XYZ
 U  0.405 0.116 0.133  R   U  (2/3) X

7  V   0.299 0.587 0.144  G . 8  V Y .
    
W  0.145 0.827 0.627   B   W  (- X  3Y  Z ) / 2

Yuv  CIE XYZ L * a * b *  CIE XYZ

Y  Y  L *  25 (100Y/ Y o )1/ 3 - 16
 
9  u  4 X / ( X  15Y  3Z ).
10  a *  500[( X/ X o )1/ 3 - (Y/ Y o )1/ 3] .
 v  6Y / ( X  15Y  3Z )
  b *  200[(Y/ Y )1/ 3 - ( Z/ Z )1/ 3]
 o o
U *V * W *  Yuv L * u * v *  Yuv
 W *  25 (100Y / Y )1/ 3 - 17  L *  25 (100Y / Y )1/ 3 - 16
 * o
 * o
11  U  13 W (u  u o )
* . 12  u  13 L (u  u o )
* .
 V *  13 W * (v  v )  v *  13 L * (1.5v  v )
 o  o
 Transformation between NTSC RGB and YCbCr

RGB to YCbCr transform
 Y   0.299 0.587 0.114  R 
Cb   0.168  0.331 0.499  G 
     9 , 6 +
Cr   0.500  0.419  0.081  B 
YCbCr to RGB transform

R  1.000  0.0009 1.4017   Y 
G   1.000  0.3443  0.7137  Cb 
    
 B  1.000 1.7753  0.0015  Cr 
 Another format with less computation

Y = 0.299 (R - G) + G + 0.114 (B - G)
Cb = 0.564 (B - Y) 4 , 6 +
Cr = 0.713 (R - Y)
IHS model (perceptual color model)

 I : intensity (the more black in it, the darker (less light) it is),
H: hue,
S: saturation (the more white mixed in the color, the less saturated it is).
Hue and saturation taken together are called chromaticity.
Intensity Hue Saturation
 intensity = gray level: [0, 255]

hue = angle : [0o, 360o]  [0, 255]
saturation = percentage : [0,1]  [0, 255]
Geometric structure
White
G R
Hue ( )
Saturation
B
Intensity
A horizontal cross
section of the HIS
Black
color model.
 IHS color model is a generalized cylindrical coordinate form of a

color model. Many IHS forms have been derived for known color
models.
 Conversion from CIE RGB to IHS
1
I R  G  B 
3
S  1
3
min R, G, B 
R  G  B 


1
R  G   R  B  
H  cos 1 2
1 2


 R  G   R  B G  B  
2 
where H = 360o - H , if (B/I) > (G/I).
 Conversion from IHS to RGB

If 0o  H  120o , then
1 1 S cosH 
b  1  S , r  1  , g  1  r  b  .
3 3  cos(60 o  H ) 
If 120o  H  240o , then

H  H  120 o
r 
1
1  S , g  1 1  S cosH 
, b  1  r  g  .
3 3  cos(60  H ) 
o
If 240o  H  360o , then

H  H  240 o
g
1
1  S , b  1 1  S cosH 
, r  1  g  b  .
3 3  cos(60  H ) 
o
Where R, G, B are in the range [0, 1] and r + g + b = 1; we take

R = 3I r, G = 3I g, and B = 3I b.
 IHS of PAL YIQ

Y   0.299 0.587 0.11  R 
 I    0.147  0.289 0.437 G 
    
Q PAL  0.615  0.515  0.1  B NTSC
I  Y , S  I 2  Q2, H  tan 1(Q I ).
 IHS of L*a*b*
2 2
I  L* , S  (a* )  (b* ) , H  tan 1(b* a* ).
 If a color model is a uniform chromaticity scale (UCS) system,

then its IHS form is also a uniform chromaticity scale system.
Color reproduction
 The classical theory states that any color can be reproduced by
mixing an appropriate set of three primary colors.
 Three primary colors: Red, Green, and Blue.
 Three complementary colors: Cyan, Magenta, and Yellow.
 In application, the primary colors are mainly used for describing color
light, and the complementary colors are mainly used for describing
color printing.
 The relationship between RGB and CMY

R G B
complement
C M Y
R=W-C C=W–R  C  255  R 

i .e., M   255   G 
G=W-M M=W–G      
Y  255   B 
B=W-Y Y=W–B
where "R = W - C" means that cyan is absorbed from white light and red
is reflected to the viewer's eye.
W (White) = R + G + B R G B
K (black) = C + M + Y
C = G+B R = M+Y
M = R+B G = C+Y
C M Y
Y = R+G B = C+M
 Laws of color matching

(1) Any color can be matched by mixing at most three colored lights.
(2) The luminance of a color mixture is equal to the sum of the
luminances of its components.
(3) A color match at one luminance level holds over a wide range of
luminance, where color = luminance + chrominance.
(4) Color addition:
C1  D1 and C 2  D 2  1C1   2 C 2  1D1   2 D 2
(5) Color subtraction:
C1  C 2  D1  D 2 and C 2  D 2  C1  D1
(6) Transitive law:
C1  C 2 and C 2  C 3  C1  C 3
(7) Color matches:
C  1C1   2 C 2  3 C 3
C  1C1   2 C 2  3 C 3
C  1C1   2 C 2  3 C 3
These are also called Grassman's laws (1853).
 Color matching by human inspection
2.4 Human haptic system

 人們要為虛擬環境構建觸覺設備，則必需充分了解觸覺感受器感受
程度及範圍。
 接觸 (touch) 是用來描述皮膚上的力學接觸 (mechanical contact)。
接觸，更準確的字眼是“皮膚的感覺” (cutaneous sensitivity)，是一
個複雜的議題，因為這其中還牽涉到非力學的刺激；例如，冷熱。
A. Structure of human skin

 皮膚 (skin) 覆蓋在身體的表面，以一個成年男人而言，皮膚的表面
積約為 1.8 m2，重量約為 4.8 kg，約佔總體重的 8 %。
 皮膚是由表皮 (epidermis) 及緊貼表皮的真皮 (dermis) 所構成的。
皮膚內含有汗腺、皮脂腺、毛根、毛、血管、淋巴管、末梢神經
等、以及有助於皮膚功能的皮下組織 (subcutaneous tissue)。
 手掌、腳掌、手指、腳趾的表皮厚度約在 0.1 cm 以上，其他部份
的厚度約為 0.01 ~ 0.02 cm。
 皮膚構造
 Layer structure of skin

 皮膚的觸覺
 皮膚的觸覺感受器 (tactile receptor) 包含:
髮根神經叢 (hair root plexuses)，
自由神經末梢 (free nerve endings)，
觸覺碟 (tactile discs)，
觸覺小體 (corpuscle of touch) 和
type II 皮膚力學感受器 (type II cutaneous mechanoreceptors)：
 髮根神經叢可以偵測皮膚外表的移動；
 自由神經末梢可以偵測痛覺及連續觸覺；
 觸覺碟是用以協助判定皮膚上的觸覺位置；
 觸覺小體是用以確認皮膚上的接觸位置，觸覺小體在手指頭及手掌
上最多；
 type II 皮膚力學感受器是在真皮的深層裡，主要是感受較重和連續
的刺激。
 不同區域的皮膚觸覺是由不同的脊髓神經所控制，這也是造成不同
部位的皮膚有著不同觸感的原因之一。
 觸覺神經系統
 神經系統分為中樞神經與末梢神經。中樞神經是腦與脊髓，而末梢
神經是連接中樞神經與身體各部位末梢的情報連絡路線，分為腦神
經及脊髓神經。
 主管皮膚觸覺的中樞神經是脊髓，而其末梢神經即為脊髓神經。
 脊髓全長成人約為 44cm，重約 25g，生長在脊椎骨體的背側孔穴

中。脊髓外側是白質，內部為 H 形的灰白質。白質是由神經細胞的軸
索構成，灰白質則由神經元所構成。H 形灰白質的前側叫前角
(anterior horn)，後側叫後角 (posterior horn)。前角的神經元為運動性
的，連接運動神經，而後角為感覺性的神經元，連接感覺神經。
 脊髓可以當做腦部與身體末梢的連絡媒體 (media)。它可以接收身
體末梢傳來的訊號，傳送至腦部；然後再將腦部所下達的指令傳達
到身體末梢。其次是當身體面臨緊急狀況時，脊髓會發揮中樞的功
能，直接引起反射運動，而省略對腦部的連絡。(脊髓也是內臟與
血管的自律反射中樞。)
B. Properties of human skin
 作用在皮膚上的力學刺激 (mechanical stimuli) 有三種形態：

(1) 持續刺激 (step function)。皮膚在一段時間內的垂直移位
(displacement)；例如，有一小點壓迫在皮膚上一段時間。
(2) 瞬間刺激 (impulse function)。指皮膚在極短時間內的垂直位移。
(3) 週期性刺激 (periodic function)。是指皮膚受到規律性的短暫重複
刺激。
 皮膚具有觸覺門檻值 (touch threshold or vibrotactile threshold)；

也就是說皮膚上的刺激要大於此門檻值才會有觸感。
 觸覺門檻值與刺激的形態、位置及、時間有關。
 Hill (1967) 曾導出觸覺門檻值與刺激脈衝持續時間的關係

2
At 
( t )
1  e 1.5
 其中 At 是觸覺力量門檻值 (threshold amplitude)，單位是μm (10 -6
m)，t 是刺激脈衝持續時間，單位是 ms (10 -3 s)。
 觸覺門檻值與刺激脈衝持續時間的範例，
t 0 0.5 1.0 1.5 2.0 2.5
At ∞ 7.06 4.11 3.76 2.72 2.47
 也就是說刺激的持續時間愈小，則刺激力量要愈大才會有感覺。
 上述關係僅適用於手指的刺激，刺激面積為直徑 0.06 cm 的圓形範

圍。
 觸覺門檻值是皮膚刺激的一大特性。
 第二個皮膚刺激的特性是持續刺激的時間愈久，觸覺的敏感性愈
低，甚至於低到完全沒有感覺。
 另一個特性是當皮膚上的刺激移去後，刺激的感覺仍然殘留在皮膚
上。
Sensorial adaptation for a “slow adapting” receptor, for three stimulus intensities.
2.5 Human auditory system

A. Ear structure
 耳在結構分為：外耳 (external ear)，中耳 (middle ear)及內耳
(internal ear) 三部份。
 外耳包括耳廓 (pinna) 及外耳道 (ear canal)。耳廓是用來收集聲

波，經由約 2.5cm 長的外耳道傳輸至鼓膜。
 中耳包括鼓膜 (tympanic membrane)、鼓室、耳小骨 (auditory
ossicle) 及耳咽管 (eustachian tube)。鼓膜是凹面朝外呈圓椎狀張
開的半透明薄膜。鼓膜會受聲波的共鳴而振動。鼓膜中心附著鎚
骨柄。耳小骨包含鎚骨 (malleus)、砧骨 (incus) 及鐙骨 (stapes)
三部份。鎚骨及砧骨會擴大鼓膜的振動並傳到鐙骨。
 內耳包含三個主要結構：耳蝸 (cochlea)、耳石器及半規管
(semicircular canal)。鐙骨的振動經前庭窗 (oval window) 傳入呈
螺旋狀，充滿淋巴液的耳蝸前庭階 (scala vestibuli)、鼓階 (scala
tympani)，最後被第二鼓膜吸收；而振動的訊號則會經耳蝸中的柯
氏器轉成電訊，經耳蝸神經 (聽覺神經) 傳送到大腦。
 鼓膜的表面積大約是前庭窗的 22 倍大，因此中耳的功能就像是一
個小小的擴大機將鼓膜的振動訊號擴大、傳入耳蝸中。
 Transmission of sound
 聲波的傳送及轉換
 人體平衡器 - 半規管 (semicircular canal)
 聽覺通往大腦的途徑  平衡感通往大腦的途徑
B. Properties of auditory system

 人類聽覺系統是一個相當敏感的感覺器官。它對於頻率的感受範圍
非常廣，同時能分辨 0.2% 的差異。而對於聲音振幅的容忍範圍大
約在 110 分貝 (dB, decibel) 以內。
0 分貝：可聽聞的極限
30 分貝：非常安靜
140 分貝：耳朵受傷
 合成一個似真的聽覺環境，最重要的是回音 (echo) 或稱聲音反射
(reverberation) 的問題。凝聽的位置 (position) 和方向 (orientation)
也扮演著重要的角色。
 在一個吵雜的環境中，個人的聽覺系統能夠凝聽某一特定地方或較
遠處的聲音而忽略其它處或較近處的聲音。在分析這種狀況，我們
要找出雙耳同時接收到多種音源的表示方式 (representation)。當
一個凝聽者移動頭部，則音源僅能做與頭部位移和轉動相關的改
變。這也就是音源的改變是一個頭部相關的轉移函數 (head
related transfer function (HRTF))。
 與音源相關的兩個機制 (mechanism) 是聲音大小的差異 (intensity

difference) 和到達時間的差異 (temporal or phase difference)。
 聲音大小在雙耳間的差異最大可達 20 分貝。而聲波到達雙耳的時
差在 10 ms 以上者可以被偵測出來；到達雙耳的時差在 650 ms 以
上則可以準確的判斷音源的方向。
 人耳可以非常準確地判斷音源的方位，即使是有強烈的擾亂回音
(conflicting echoes) 也是一樣。
 聽覺系統具有優先效果 (precedence effect)，也就是在一個會產生
回聲的室內 (reverberant room)，人耳會拒絕由室內所產生的回
聲。當人耳聽到兩個時差在 1~5 ms 之內的聲音，則第二個聲音將
不被接受。
 人類聽覺系統非常的複雜。一個最令人印象深刻的能力是它可以從
來自不同方位的諸多音源中獨立出某一特色音源。
 而某一聲音遮蔽另一聲音的程度與相對頻率 (relative
frequency) 、大小聲及、音源位置有關。頻率愈高，遮蔽的效果愈
明顯。
 聲音到達雙耳的時差及大小聲是人耳判斷音源方位的主要線索。假
設人耳與音源的相對方位如下圖所示：
則音源到達雙耳的距離相差 d, d = r sin + r ,
d r
雙耳時差為 t , t   (sin   ) ,
c c
其中 c 是聲波在空氣中傳播的速度。
 假設在頭顱半徑為 8.75cm，
聲波速度為 344 m/sec 的情
形下，方位角與雙耳時差的
關係如右圖所示：
 雙耳時差與聲波頻率無關。
 在諸多聲音中要能聽到某特定的聲音則其頻率要能與別的聲音有所區
別。當同時發生的兩個聲音頻率太相近，則會聽成另一個聲音。這種
現象稱為歐姆音響定則 (Ohm's acoustic law)。
 人耳對於在 1KHZ 至 3KHZ 頻率範圍內的音調變化 (sound pitch
variation) 極為敏感。平均人耳可以感覺到的最小頻率變化是 0.3 %；
也就是說 1000 至 1003 的變化可以被偵測出來。受過專業訓練的人
可以分辨更細微的音調變化。在較低頻的聲音中（從 32HZ 到
64HZ)，人耳可以感受到的最小頻率變化約為 1 %；而在較高頻的範
圍中 (16 KHZ 至 20 KHZ)，人耳對於音調變化的感覺就變了非常的
差。正常的聽覺系統約能區別 2000 個音調的變化。

Human Perception System

Uploaded by

Copyright:

Available Formats

Human Perception System

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Human Perception System

Uploaded by

Copyright:

Available Formats

Virtual Reality 2. Human perception system NCU IPVR Lab.

2. Human Perception System

Virtual Reality 2. Human perception system NCU IPVR Lab. 2

 Three components of studying color science:

 Light is the electromagnetic radiation that stimulates our visual

400 nm 500 nm 600 nm 700 nm

Virtual Reality 2. Human perception system NCU IPVR Lab. 4

 White light (sun light) consists of seven visible color lights

 Any combination of these colors will produce another color.

Virtual Reality 2. Human perception system NCU IPVR Lab. 6

 The principle of subtractive light mixture

Virtual Reality 2. Human perception system NCU IPVR Lab. 8

 Three tristimulus values construct a 3-D geometric structure of

 Two trichromatic coefficients construct a 2-D chromaticity

Virtual Reality 2. Human perception system NCU IPVR Lab. 10

 The characteristic generally used to distinguish one color from

2.2 Human visual system

Virtual Reality 2. Human perception system NCU IPVR Lab. 12

 The structure of the human eye in the horizontal cross section of

Virtual Reality 2. Human perception system NCU IPVR Lab. 14

 Courses of the capillaries (heavy lines) and nerve fibers (light

Virtual Reality 2. Human perception system NCU IPVR Lab. 16

 Demonstrating blind regions

 The retina of the human eye contains two types of photoreceptors

Virtual Reality 2. Human perception system NCU IPVR Lab. 18

 The schematic diagram of the primate retina structure.

Virtual Reality 2. Human perception system NCU IPVR Lab. 20

 Adaptation of light intensity (眼睛對光線亮度的調整能力)

 眼睛對於光線亮度 (light intensity) 的感受相當敏感。眼睛感受亮度

Virtual Reality 2. Human perception system NCU IPVR Lab. 22

 The distributions of rods and cones

狀 140,000 桿狀細胞 桿狀細胞

 The density of photoreceptors in human eye is about 200,000

 Foveal cone inner-segment mosaic with 10μm scale bar

Virtual Reality 2. Human perception system NCU IPVR Lab. 24

 The luminance efficiency function of the visual system

For the human eye, the function is a bell-shaped curve whose

 There are three different types of cones in the (normal) human

Virtual Reality 2. Human perception system NCU IPVR Lab. 26

 Based on the three-color sensitivity, the spectral energy

 If C1() and C2() are two spectral distributions that produce

 The luminance, intensity, or lightness of a spatially distributed

where v(λ) is the relative luminance efficiency function of the

 The influence of sensitivity for visual system

Optic sensitivity of photoreceptors

Virtual Reality 2. Human perception system NCU IPVR Lab. 28

B. Properties of human vision

 Brightness range of the eye.

 Lateral inhibition refers to an architecture for neural networks in

 Lateral inhibition is a physiological mechanism of human vision;

Virtual Reality 2. Human perception system NCU IPVR Lab. 30

 The luminance of an object is independent of the luminances of

All small squares have exactly the same intensity.

 The perceived color depends on the color of the surround.

Virtual Reality 2. Human perception system NCU IPVR Lab. 32

How many colors in the picture ?

 Mach band effect

Virtual Reality 2. Human perception system NCU IPVR Lab. 34

 The Hermann grid illusion

狀 140,000 桿狀細胞桿狀細胞

(4) CIE UVW* (1964)

(5) CIE Luv* (1976)

(6) CIE Lab* (1976)