Scribd
Upload
20 views26 pages

Lecture 4 Sep 16

Lecture, multivariate statistics

Uploaded by

Miguel D.
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
20 views26 pages

Lecture 4 Sep 16

Lecture, multivariate statistics

Uploaded by

Miguel D.
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 26

DASC6510: Applied Multivariate Statistics

Fall 2024

Jabed Tomal, PhD

Associate Professor
Department of Mathematics and Statistics
Thompson Rivers University
Kamloops, BC
Canada

September 17, 2024

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 1 / 26


Random Sample and Expected
Values

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 2 / 26


Random Sample and Expected Values:

Measurements of Data/Variables

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 3 / 26


Random Sample and Expected Values:

Random Sample and Joint PDF

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 4 / 26


Random Sample and Expected Values:

Correlated Variables and Independent


Measurements

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 5 / 26


Random Sample and Expected Values:

Sample Mean and its Covariance Matrix

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 6 / 26


Random Sample and Expected Values:

Sample Covariance Matrix

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 7 / 26


Random Sample and Expected Values:

Unbiased Sample Covariance Matrix

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 8 / 26


Random Sample and Expected Values:

The density function of univariate Normal

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 9 / 26


Random Sample and Expected Values:

Density Curve - Univariate Normal

Figure: Univariate Normal Density Curve with Mean µ ∈ ℜ and Standard


Deviation σ ∈ ℜ+ , Denoted by N(µ, σ).

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 10 / 26


Multivariate Normal Distribution:
The term ((x − µ)/σ)2 , the exponent of the univariate normal
density function which measures the square of distance from x to
µ in standard deviation units, can be written as
 2
x −µ
= (x − µ)(σ 2 )−1 (x − µ).
σ
The above term can be generalized for a p × 1 random vector X
measured on several variables as

(x − µ)T Σ−1 (x − µ),


where
µ is a p × 1 vector of expected values of X, and
Σ is the p × p covariance matrix of X.
We assume that the symmetric matrix Σ is positive definite, so the
expression measures the square of the generalized distance from
x to µ.
Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 11 / 26
Multivariate Normal Distribution:

Multivariate Normal Density

The constant in the univariate normal density function is replaced


by (2π)−p/2 |Σ|−1/2 .
Therefore, a p-dimensional normal density for the random vector
X has the form

1 1 T Σ−1 (x−µ)
f (x) = e− 2 (x−µ) ,
(2π)p/2 |Σ|1/2
where x ∈ ℜp , µ ∈ ℜp , and Σ is a p × p symmetric positive definite
matrix.
We denote this normal density by Np (µ, Σ), which is analogous to
the normal density in the univariate case.

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 12 / 26


Multivariate Normal Distribution:

Bivariate Normal Density

Let us consider that p = 2, E(X1 ) = µ1 , E(X2 ) = µ2 ,


Var (X1 ) = σ11 , Var (X2 ) = σ22 , Cov (X1 , X2 ) = σ12 ,
√ √ √ √
Cor (X1 , X2 ) = ρ12 = σ12 / σ11 σ22 , and σ12 = ρ12 σ11 σ22 .
The inverse of the covariance matrix
 
σ11 σ12
Σ=
σ12 σ22
can be written as
 
1 σ22 −σ12
Σ−1 = 2
σ11 σ2 − σ12 −σ12 σ11

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 13 / 26


Multivariate Normal Distribution:
Bivariate Normal Density
The squared distance becomes

(x − µ)T Σ−1 (x − µ) =

Therefore, the density of the Bivariate Normal Distribution


becomes

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 14 / 26


Multivariate Normal Distribution:

Figure: Bivariate normal density with σ11 = σ22 and ρ12 = 0.

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 15 / 26


Multivariate Normal Distribution:

Figure: Bivariate normal density with σ11 = σ22 and ρ12 = 0.75.
Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 16 / 26
Multivariate Normal Distribution:

Distribution of Squared Distance

Figure: The 100(1 − α)% contour.

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 17 / 26


Multivariate Normal Distribution:

Figure: The 50% and 90% contours of the bivariate normal distribution.

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 18 / 26


Multivariate Normal Distribution:

Additional Properties of MVN

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 19 / 26


Multivariate Normal Distribution:

Linear Combination of MVN

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 20 / 26


Multivariate Normal Distribution:

Any Subset of MVN is also MVN

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 21 / 26


Multivariate Normal Distribution:

Independence and Covariance

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 22 / 26


Multivariate Normal Distribution:

Independence and Covariance

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 23 / 26


Multivariate Normal Distribution:

Conditional Distribution

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 24 / 26


Multivariate Normal Distribution:

Distribution of Quadratic Form

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 25 / 26


Multivariate Normal Distribution:

Some Remarks on Distance

Jabed Tomal (TRU) Applied Multivariate Statistics September 17, 2024 26 / 26

You might also like