Chapter 3 Foundation of Mathematical Analysis

R. J. Chang
Department of Mechanical Engineering
NCKU

§ 3.1 Statistics and Probability

§ 3.2 Random Variables and Magnitude Distribution

§ 3.3 Probability Density Functions and Usages

§ 3.4 Stochastic Processes and Representation

§ 3.1 Statistics and Probability(1)
1. Introduction
(1) Terminologies
(a) Statistics
Randomization, Outcome, Events
(b) Probability
Axioms, Probability space, Random processes
(c) Experiments
Sample space, Probability measure

Ex: Die experiment, Coin experiment

§ 3.1 Statistics and Probability(2)
(2)Mathematical Model

Randomization Probability
design and test axioms

Uncertain Statistical Probability Math

Physical Experiment Space Representation
World
Statistical Probability
modeling modeling

Modeling and mapping

§ 3.1 Statistics and Probability(3)
2. Probability Space
(1) Definition
In probability theory, the probability space is
defined as a triplet {Ω, £, P}. Ω is called the
sure event which is a space of points ωi. £ is
called the sets of events and are subsets of Ω.
P is called a probability measure.
§ 3.1 Statistics and Probability(4)

Ex: Coin experiment-Two consecutive tosses of fair coin

Outcomes (―Points‖):
ω1 (H, H);
ω2 (H, T);
ω3 (T, H);
ω4 (T, T).

Tail(T)
Head(H)
§ 3.1 Statistics and Probability(5)
Specific Events:
Ai : Subsets of points.
A1 : At least a tail was thrown ω2, ω3, ω4.
A2 : Exactly a tail was thrown ω2, ω3.
A3 : Exactly two tail were thrown ω4.

A1 W
w1
w4
A2 A3
w2
w3
§ 3.1 Statistics and Probability(6)
(2) Probability measure
Define P as a function mapping P: £ R, and P
satisfy the following axioms.
(a) P(A)≧0, where A is an event and P(A) is called
the probability of the event.
(b) P(Ω)=1.
(c) P(A∪B)=P(A)+P(B) provided that A,B £ and
A∩B= Φ,Φ is called the impossible event.
 § 3.1 Statistics and Probability(7)
(3) Structure of Probability Space

s - Algebra ¢
G Probality
* Sample space (£
[ )
¢ ÌW
* Outcome  [£
G Axiom P
Events P [0,1]
§ 3.2 Random Variables and Magnitude Distribution (1)
1. Random Variables and Distribution Function
(1) Random variables:
In a probability space (Ω，£，P), X :W  Rn is a
random variable if and only if X is measurable w.r.t
the field £.
(2) Distribution function:
The function F ( x)  P{w | X (ω )  x} is defined as
the distribution function of X(ω).
§ 3.2 Random Variables and Magnitude Distribution (2)
(a) Fundamental properties of F(x)
A non-decreasing function with the properties:

F ( x)
lim F ( x) lim F ( x)
x -¥ x ¥
1
1. lim F ( x )  0 + ¥
¡ -¥
xú

2. lim F ( x )  1
¡ ¥
xú

- ¥
0 x
§ 3.2 Random Variables and Magnitude Distribution (3)
(b) Function Mapping

Sample X(£s) Realization of F(x)

space random Probability
w W variables in in [0,1]
Rn
Ai Ì W
P(A)
§ 3.2 Random Variables and Magnitude Distribution (4)
1. Probability Density Function
(1) Definition
p(x) is called the probability density function of
x(ω) if
x
F ( x)   p( y )dy
-∞

If F(x) is differentiable w.r.t. x then

p( x)  dF ( x) / dx
§ 3.2 Random Variables and Magnitude Distribution (5)
Ex: Die Experiment
Outcome: Six faces of the Dice
Ω = { f 1 , f2 , f3 , f 4 , f5 , f6 }
Subsets(total number): 26
Events: ―Even‖, Outcomes are f2 , f4 , f6
Probability=3/6=1/2
Random Variable: Define X (fi)=10 i
Face1, X (f1)=10
Face2, X (f2)=20
……

Face6, X (f6)=60
 § 3.2 Random Variables and Magnitude Distribution (6)
F(x)

1
Distribution Function 5/6

F(100) = P{ fi | X ( fi )  100} = 1 4/6

F(35) = P{fi | X ( fi )  35 } = 1/2

3/6
2/6
1/6

10 20 30 40 50 60 100 x
Nondifferentiable in classical
sense

p(x)
Probability Density Function
—Uniform discontinuous
1/6
function
10 20 30 40 50 60 x
Discrete uniform
distribution
§ 3.3 Probability Density Functions and Usages (1)
1. Densities and Distributions
Probability Distribution
Type
Density p(x) Function F(x)

p(x) F(x)
Uniform
(bounded) 1 1
b-a
a b x x
a b

F(x)
p(x)

Gaussian +¥
1
Normal
(Error function) sx 0.5
-¥ +¥ -¥
x x
x
x
§ 3.3 Probability Density Functions and Usages (2)
2. Fundamental Usages
Total probability : 
domain
p(x)dx  1

Mean value (1st - Moment) : 

domain
p(x)xdx  E[x]   x

Mean square (2nd - Moment) : 

domain
p(x)x 2dx  E[x 2 ]   x2

Variance : 
domain
p(x)(x -  x ) 2 dx  E[(x -  x ) 2 ]  s x2

Root Mean Square (R.M.S)  Mean square

§ 3.3 Probability Density Functions and Usages (3)
Ex: Gaussian density and probability distribution
1 -( x -  x ) 2
p ( x)  exp[ ]
s x 2 2s x
2

 x - Mean value
s x - Standard deviation
p(x) p(x)

a b x  x - 3s x  x + 3s x
2
P[| x -μ x |  3s ]  99
b
Probability P[a  x  b]   p( x) dx %
a
3
§ 3.4 Stochastic Processes and Representation (1)
1. Time Domain Representation
(1) Stochastic processes
A stochastic process X(t, ω) is a family of random
variables defined on the probability space {Ω，£，P}
and indexed by time t .

For fixed time, we obtain a random variable

which is measurable w.r.t. £.

For each ω, we obtain a function mapping

X: T R called a sample function.
§ 3.4 Stochastic Processes and Representation (2)

X(ω i ,t) Random variable

w1

t
w2
w3

t i (fixed) t i+ j
§ 3.4 Stochastic Processes and Representation (3)
(2) Gaussian process
A random process is a Gaussian process if for
any finite collection of n parametric values at t1,
t2…tn, the corresponding n random variables X(t1),
X(t2)…X(tn) are jointly Gaussian. The probability
density function of the random variables X1, X2,
X3…Xn can be given by p(x1, t1; x2, t2; …; xn, tn):
1  1 
p( x1 ,t1 ; x2 ,t2 ;...; xn ,tn )  exp - ( x -μ x ) -1( x -μ x )
 2 
n 1
(2) 
2 2

where x   x1, x2 ,..., xn  n - states as a vector space

T

μ x  E [ x] mean vector

  E [ xx ] - Tx , covariance matrix

T
§ 3.4 Stochastic Processes and Representation (4)
(3) Properties of Gaussian process
(a) Invariant property

x y
Linear Time-invariant System
Input Guassian Output Guassian

(b)Ergodicity property
For weakly stationary (up to 2nd moment) process
Rxx ( )

R
lim S z
1 T U
R ( ) d V  0
TT W
Envelop decay !
xx
T ¥ -T

τ
-T T
§ 3.4 Stochastic Processes and Representation (5)
2. Frequency Domain Representation
(1) Magnitude representations
(a) Amplitude spectrum xi
x  xi  sin wit + i 
(b) Energy spectrum Ei
x 2 t dt
T
E 0
(c) Power spectrum Poi
1 T 2
P0  0 x t dt
T
i
(d) Power spectral density Ps
dP0
Ps 
dw
§ 3.4 Stochastic Processes and Representation (6)
(2) Mathematical and physical spectrum
S xx w 

-w w
S xx w 
t
Phy. Gxx(w) = 2Sxx(w)
for w¥

Math
w

Magnitude in dB:
logGxx w   log2 + logSxx w 
10logGxx w   10log2 + 10logSxx w 
§ 3.4 Stochastic Processes and Representation (7)
3. Gaussian white process
(1) Definition
A Gaussian process v(t) define on {Ω，£，P} is a
white process if its mean and covariance functions
are given by
(a) E[v(t)]=0
(b) E[v(t)‧v(s)]=Qδ(t-s)=Qδ(τ)
(2) Interpretation of whiteness
Frequency–domain interpretation
constant Rxx ( ) Sxx (w )
 (t )

t w  w

Power spectrum is Fourier transform of autocorrelation function.

§ 3.4 Stochastic Processes and Representation (8)
(3) Role of Gaussian white process
(a) Mathematics: a model of ideal random signal
source
(b) Physics: a model of physical noise
(c) Engineering: signal for dynamic testing
Physics Math Eng.

Phenomenon Ideal Analytical Model Testing signal

Local (Band limited) white

Sxx (w )

0
Bandwidth £s
§ 3.4 Stochastic Processes and Representation (9)
Gaussian white and colored process

Desired color
White
output
System

Desired LTI Model

(Even include nonlinearity)