Lecture-1

M.S. 1st Semester

Session: 2021-22

Factor Analysis
Professor Dr. Luthful Alahi Kawsar
Department of Statistics
Shahjalal University of Science and Technology, Sylhet-3114,
Bangladesh
Factor Analysis: Introduction

• There are many forms of data analysis used to report on and

study survey data. Factor analysis is best when used to
simplify complex data sets with many variables.

• Factor analysis is the practice of condensing many variables

into just a few, so that your research data is easier to work
with.
Factor Analysis: Introduction

• As with any kind of process that simplifies complexity, there

is a trade-off between the accuracy of the data and how easy
it is to work with. With factor analysis, the best solution is the
one that yields a simplification that represents the true nature
of your data, with minimum loss of precision.
• Factor analysis is commonly used in market research, as well
as other disciplines like technology, medicine, sociology,
field biology, education, psychology and many more.
Factor Analysis: Introduction
• Factor analysis is a statistical technique to study the inter-relationships
among the variables in an effort to find a new set of factors, fewer in
number than the original variables so that the factors are common
among the original variables.
• Therefore, there is a difference between factor analysis and principal
component analysis:
• In principal component analysis, the components are so selected that they can
explain maximum variation in the original data set.
• On the other hand, in factor analysis a small number of common factors are
extracted so that these common factors are sufficient to study the relationships
of original variables.
Factor Analysis: Introduction

• In simple words, Factor analysis is a correlational method used to find

and describe the underlying factors driving data values for a large set
of variables.
• It identifies correlations between and among variables to bind them
into one underlying factor driving their values.
Factor Analysis: Introduction
• Consider the set of variables
Factor Analysis: Introduction
• In this set of variables (V1, V2, V3, V4, V5 and V6) a correlational
relationship may be found between V1, V3 and V4. This means that these
three variables may be in fact be only one value or factor.
• Accordingly, large number of variables may be reduced to only several
factors.
• We have discovered that
• V1, V3 and V4 are a factor
• V2 and V6 are a factor
• V5 is a factor
• This data set is explained by 3 factors rather than by 6 variables. Factor
analysis is often referred to as data reduction process.
Factor Analysis: Introduction

• Now we only need to

• Determine the assumptions for factor analysis,
• Develop a means of identifying factors,
• Determine if a factor is important or not, and
• Examine the interaction of the variables on the factor.
Factor Analysis: Introduction
• Factor analysis is based on a model in which the observed vector is
partitioned into an unobserved ‘systematic’ part and an unobserved
‘error’ part.
• The components of the ‘error’ vector are considered as uncorrelated or
independent, while the ‘systematic’ part is taken as a linear
combination of a relatively small number of unobserved random
variables, called factors.
• The analysis separates the effects of factors from the errors and gives a
description or explanation of the interdependence of a set of variables
in terms of the factors, without regard to the observed variability.
Factor analysis and Principal component analysis
• Factor analysis has similar aims to principal component analysis. The basic
idea of the analysis is same as the principal component analysis, i.e. it may
be possible to describe a set of p-variables in terms of a smaller number of
factors, and hence explain the relationship between these variables.
• There is, however, one important difference: Principal component analysis
is not based on any particular statistical model, but factor analysis is based
on a rather special model.
• Factor analysis (using ML) is invariant under changes of scale of
measurement of the variables, so the same results are obtained whether the
correlation or the covariance matrix are analyzed. In principal component
analysis, different results are obtained from the correlation matrix than that
are obtained from the covariance matrix.
• The two methods are similar in one respect, namely, that they are both
pointless if all the observed variables are approximately uncorrelated.
Factor analysis: Example
• Say you have a list of questions and you don’t know exactly which responses will
move together and which will move differently; for example, purchase barriers of
potential customers.
The following are possible barriers to purchase:
• Price is prohibitive • The product benefits don’t outweigh the
• Overall implementation costs cost
• We can’t reach a consensus in our • We have no reason to switch
organization • Our IT department cannot support your
• Product is not consistent with our product
business strategy • We do not have sufficient technical
• I need to develop an ROI, but cannot or resources
have not • Your product does not have a feature we
• We are locked into a contract with require
another product • Other (please specify)
Factor analysis: Example
• Factor analysis can uncover the trends of how these questions will move together.
The following are loadings for 3 factors for each of the variables.

• Notice how each of the principal

components have high weights
for a subset of the variables.
• The first component heavily
weights variables related to
cost,
• The second weights variables
related to IT, and
• The third weights variables
related to organizational
factors.
Factor analysis: Example
Factor analysis: Example
• If we were to cluster the customers based on these three components, we can see some
trends. Customers tend to be high in Cost barriers or Org barriers, but not both.
The Orthogonal Factor Model
The Orthogonal Factor Model
The Orthogonal Factor Model
The Orthogonal Factor Model
The Orthogonal Factor Model
The Orthogonal Factor Model
The Orthogonal Factor Model
The Orthogonal Factor Model
Non-Uniqueness of Factor Loadings
Non-Uniqueness of Factor Loadings
Non-Uniqueness of Factor Loadings
• This lecture is prepared with the help of
https://www.qualtrics.com/experience-management/research/factor-analysis/
Thank You