Scribd
Upload
18 views6 pages

ML Mod32019

Uploaded by

mrbuttler001
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
18 views6 pages

ML Mod32019

Uploaded by

mrbuttler001
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 6

Expectation-maximisation algorithm

The maximum likelihood estimation method (MLE) is a method for estimating the
parameters of a statistical model, given observations (see Section 6.5 for details). The
method attempts to find the parameter values that maximize the likelihood function, or
equivalently the log-likelihood function, given the observations.

The expectation-maximisation algorithm (sometimes abbreviated as the EM algorithm) is


used to find maximum likelihood estimates of the parameters of a statistical model in
cases where the equations cannot be solved directly. These models generally involve
latent or unobserved variables in addition to unknown parameters and known data
observations. For example, a Gaussian mixture model can be described by assuming that
each observed data point has a corresponding unobserved data point, or latent variable,
specifying the mixture component to which each data point belongs.

In the case of Gaussian mixture problems, because of the nature of the function, finding a
maximum likelihood estimate by taking the derivatives of the log-likelihood function
with respect to all the parameters and simultaneously solving the resulting equations is
nearly impossible. So we apply the EM algorithm to solve the problem.

As already indicated, the EM algorithm is a general procedure for estimating the


parameters in a statistical model.
Dimensionality Reduction
Dimensionality reduction technique can be defined as, "It is a way of converting the higher
dimensions dataset into lesser dimensions dataset ensuring that it provides similar
information." These techniques are widely used in machine learning for obtaining a better fit
predictive model while solving the classification and regression problems.

It is commonly used in the fields that deal with high-dimensional data, such as speech
recognition, signal processing, bioinformatics, etc. It can also be used for data visualization,
noise reduction, cluster analysis, etc.
Benefits of applying Dimensionality Reduction
 By reducing the dimensions of the features, the space required to store the dataset also
gets reduced.
 Less Computation training time is required for reduced dimensions of features.
 Reduced dimensions of features of the dataset help in visualizing the data quickly.
 It removes the redundant features (if present) by taking care of multicollinearity.

Disadvantages of dimensionality Reduction


 Some data may be lost due to dimensionality reduction.
 In the PCA dimensionality reduction technique, sometimes the principal components
required to consider are unknown.
Principal Component Analysis

Principal Component Analysis is an unsupervised learning algorithm that is used for the
dimensionality reduction in machine learning. It is a statistical process that converts the
observations of correlated features into a set of linearly uncorrelated features with the help of
orthogonal transformation. These new transformed features are called the Principal
Components. It is one of the popular tools that is used for exploratory data analysis and
predictive modeling. It is a technique to draw strong patterns from the given dataset by
reducing the variances.

The PCA algorithm is based on some mathematical concepts such as:

 Variance and Covariance


 Eigenvalues and Eigen factors

Some common terms used in PCA algorithm:

 Dimensionality: It is the number of features or variables present in the given dataset.


More easily, it is the number of columns present in the dataset.
 Correlation: It signifies that how strongly two variables are related to each other. Such
as if one changes, the other variable also gets changed. The correlation value ranges from
-1 to +1. Here, -1 occurs if variables are inversely proportional to each other, and +1
indicates that variables are directly proportional to each other.
 Orthogonal: It defines that variables are not correlated to each other, and hence the
correlation between the pair of variables is zero.
 Eigenvectors: If there is a square matrix M, and a non-zero vector v is given. Then v will
be eigenvector if Av is the scalar multiple of v.
 Covariance Matrix: A matrix containing the covariance between the pair of variables is
called the Covariance Matrix.

Principal Components in PCA

As described above, the transformed new features or the output of PCA are the Principal
Components. The number of these PCs are either equal to or less than the original features
present in the dataset. Some properties of these principal components are given below:

 The principal component must be the linear combination of the original features.
 These components are orthogonal, i.e., the correlation between a pair of variables is
zero.
 The importance of each component decreases when going to 1 to n, it means the 1 PC
has the most importance, and n PC will have the least importance.

Steps for PCA algorithm

1. Getting the dataset


Firstly, we need to take the input dataset and divide it into two subparts X and Y, where
X is the training set, and Y is the validation set.
2. Representing data into a structure
Now we will represent our dataset into a structure. Such as we will represent the two-
dimensional matrix of independent variable X. Here each row corresponds to the data
items, and the column corresponds to the Features. The number of columns is the
dimensions of the dataset.
3. Standardizing the data
In this step, we will standardize our dataset. Such as in a particular column, the features
with high variance are more important compared to the features with lower variance.
If the importance of features is independent of the variance of the feature, then we will
divide each data item in a column with the standard deviation of the column. Here we
will name the matrix as Z.
4. Calculating the Covariance of Z
To calculate the covariance of Z, we will take the matrix Z, and will transpose it. After
transpose, we will multiply it by Z. The output matrix will be the Covariance matrix of Z.
5. Calculating the Eigen Values and Eigen Vectors
Now we need to calculate the eigenvalues and eigenvectors for the resultant covariance
matrix Z. Eigenvectors or the covariance matrix are the directions of the axes with high
information. And the coefficients of these eigenvectors are defined as the eigenvalues.
6. Sorting the Eigen Vectors
In this step, we will take all the eigenvalues and will sort them in decreasing order,
which means from largest to smallest. And simultaneously sort the eigenvectors
accordingly in matrix P of eigenvalues. The resultant matrix will be named as P*.
7. Calculating the new features Or Principal Components
Here we will calculate the new features. To do this, we will multiply the P* matrix to the
Z. In the resultant matrix Z*, each observation is the linear combination of original
features. Each column of the Z* matrix is independent of each other.
8. Remove less or unimportant features from the new dataset.
The new feature set has occurred, so we will decide here what to keep and what to
remove. It means, we will only keep the relevant or important features in the new
dataset, and unimportant features will be removed out.

Applications of Principal Component Analysis


 PCA is mainly used as the dimensionality reduction technique in various AI applications
such as computer vision, image compression, etc.
 It can also be used for finding hidden patterns if data has high dimensions. Some fields
where PCA is used are Finance, data mining, Psychology, etc.

You might also like