Scribd
Upload
73 views6 pages

Covariance Matrix

The covariance matrix is a square matrix that displays the variance of datasets and the covariance between pairs of datasets, with diagonal elements representing variance and off-diagonal elements representing covariance. It is essential for understanding relationships between variables, indicating positive, negative, or zero covariance. The matrix is always square, symmetric, and positive semi-definite, making it a crucial tool in data analysis.

Uploaded by

oasisolga
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
73 views6 pages

Covariance Matrix

The covariance matrix is a square matrix that displays the variance of datasets and the covariance between pairs of datasets, with diagonal elements representing variance and off-diagonal elements representing covariance. It is essential for understanding relationships between variables, indicating positive, negative, or zero covariance. The matrix is always square, symmetric, and positive semi-definite, making it a crucial tool in data analysis.

Uploaded by

oasisolga
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 6

What is Covariance Matrix?

Covariance matrix is a square matrix that displays the variance exhibited by


elements of datasets and the covariance between a pair of datasets.
Variance is a measure of dispersion and can be defined as the spread of
data from the mean of the given dataset. Covariance is calculated
between two variables and is used to measure how the two variables vary
together.

Covariance Matrix Definition

Variance covariance matrix is defined as a square matrix where the


diagonal elements represent the variance and the off-diagonal elements
represent the covariance. The covariance between two variables can be
positive, negative, and zero. A positive covariance indicates that the two
variables have a positive relationship whereas negative covariance shows
that they have a negative relationship. If two elements do not vary
together then they will display a zero covariance.

Covariance Matrix Example

Suppose there are two data sets X = {3, 2} and Y = {7, 4}. The sample
variance of dataset X = 0.5, and Y = 4.5. The covariance between X and Y
is 1.5. The covariance matrix is expressed as follows:

[0.51.51.54.5][0.51.51.54.5]
A detailed description of how to find the variance covariance matrix will
be covered in the upcoming sections.
Covariance Matrix Formula
To determine the covariance matrix, the formulas for variance and
covariance are required. Depending upon the type of data available, the
variance and covariance can be found for both sample data and
population data. These formulas are given below.

Population Variance:

Population Covariance:

Sample Variance:

Sample Covariance:
Using these formulas, the general form of a variance covariance matrix is
given as follows:

Covariance Matrix 2 × 2

A 2 × 2 matrix is one which has 2 rows and 2 columns. The formula for a 2
× 2 covariance matrix is given as follows:

Covariance Matrix 3 × 3

If there are 3 datasets, x, y, and z, then the formula to find the 3 × 3


covariance matrix is given below:

How To
Calculate Covariance Matrix?

The number of variables determines the dimension of a variance-


covariance matrix. For example, if there are two variables (or datasets) it
indicates that the covariance matrix will be 2 dimensional. Suppose the
math and science scores of 3 students are given as follows:

Student Math (X) Science (Y)

1 92 80
Student Math (X) Science (Y)

2 60 30

3 100 70

The steps to calculate the covariance matrix for the sample are given
below:

 Step 1: Find the mean of one variable (X). This can be done by
dividing the sum of all observations by the number of
observations. Thus, (92 + 60 + 100) / 3 = 84
 Step 2: Subtract the mean from all observations; (92 - 84), (60 -
84), (100 - 84)
 Step 3: Take the sum of the squares of the differences obtained
in the previous step. (92 - 84)2 + (60 - 84)2 + (100 - 84)2.
 Step 4: Divide this value by 1 less than the total to get the
sample variance of the first variable (X). var(X) = [(92 - 84) 2 +
(60 - 84)2 + (100 - 84)2] / (3 - 1) = 448
 Step 5: Repeat steps 1 to 4 to find the variances of all variables.
Using these steps, var(Y) = 700.
 Step 6: Choose a pair of variables (X and Y).
 Step 7: Subtract the mean of the first variable (X) from all
observations; (92 - 84), (60 - 84), (100 - 84).
 Step 8: Repeat step 7 for the second variable (Y); (80 - 60), (30 -
60), (70 - 60).
 Step 9: Multiply the corresponding observations. (92 - 84)(80 -
60), (60 - 84)(30 - 60), (100 - 84)(70 - 60).
 Step 10: Add these values and divide them by (n - 1) to get
the covariance. cov(x, y) = cov(y, x) = [(92 - 84)(80 - 60) + (60 -
84)(30 - 60) + (100 - 84)(70 - 60)] / (3 - 1) = 520.
 Step 11: Repeat steps 6 to 10 for different pairs of variables.
 Step 12: Now using the general formula for covariance matrix
arrange these values in matrix form. Thus, the variance
covariance matrix for the example is given as
The same steps can be followed while calculating the covariance matrix
for a population. The only difference is that the population variance and
covariance formulas will be applied.
Properties of Covariance Matrix

Covariance matrix is a very important tool used by data scientists to


understand and analyze multivariate data. Listed below are the various
properties of this matrix that make it extremely useful.
 A covariance matrix is always a square matrix. This means that
the number of rows of the matrix will be equal to the number of
columns.
 The matrix is symmetric. Suppose M is the covariance matrix then
MT = M.
 It is positive semi-definite. Let u be a column vector, u T is the
transpose of that vector and M be the covariance matrix then
uTMu ≥ 0.
 All eigenvalues of the variance covariance matrix are real and
non-negative.

Related Articles:

 Types of Matrices
 Covariance Calculator
 Correlation Coefficient

Important Notes on Covariance Matrix

 The covariance matrix depicts the variance of datasets and


covariance of a pair of datasets in matrix format.
 The diagonal elements represent the variance of a dataset and
the off-diagonal terms give the covariance between a pair of
datasets.
 The variance covariance matrix is always square, symmetric, and
positive semi-definite.
 The general formula to represent a covariance matrix is

You might also like