FACTOR ANALYSIS

Conducting Factor Analysis

i. Problem Formulation
ii. Construction of the Correlation Matrix
iii. Method of Factor Analysis
iv. Number of Factors
v. Rotation of Factors
vi. Interpretation of Factors
vii. Factor Scores
viii. Selection of Surrogate Variables
ix. Model Fit
Factor Analysis
• Factor analysis is a general name denoting a class
of procedures primarily used for data reduction and
summarization.
• Factor analysis is an interdependence technique in
that an entire set of interdependent relationships is
examined without making the distinction between
dependent and independent variables
• Factor analysis is used in the following circumstances:
• To identify underlying dimensions, or factors, that
explain the correlations among a set of variables.
• To identify a new, smaller, set of uncorrelated
variables to replace the original set of correlated
variables in subsequent multivariate analysis
(regression or discriminant analysis).
• To identify a smaller set of salient variables from a
larger set for use in subsequent multivariate
analysis
Statistics Associated with Factor
Analysis
• Bartlett's test of sphericity. Bartlett's test of
sphericity is a test statistic used to examine the
hypothesis that the variables are uncorrelated in the
population.

• .
Statistics Associated with Factor
Analysis
• Factor scores. Factor scores are composite scores estimated for
each respondent on the derived factors.
• Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy. The
Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy is an index
used to examine the appropriateness of factor analysis. High values
(between 0.5 and 1.0) indicate factor analysis is appropriate. Values
below 0.5 imply that factor analysis may not be appropriate.
• Percentage of variance. The percentage of the total variance
attributed to each factor.
• Residuals are the differences between the observed correlations, as
given in the input correlation matrix, and the reproduced correlations,
as estimated from the factor matrix.
• Scree plot. A scree plot is a plot of the Eigenvalues against the
number of factors in order of extraction
Conducting Factor Analysis
Conducting Factor Analysis
Problem Formulation

Construction of the Correlation Matrix

Method of Factor Analysis

Determination of Number of Factors

Rotation of Factors

Interpretation of Factors

Calculation of Selection of
Factor Scores Surrogate Variables

Determination of Model Fit

Conducting Factor Analysis: Formulate the
Problem
• The objectives of factor analysis should be identified.

• The variables to be included in the factor analysis

should be specified based on past research, theory,
and judgment of the researcher. It is important that
the variables be appropriately measured on an
interval or ratio scale.

• An appropriate sample size should be used. As a

rough guideline, there should be at least four or five
times as many observations (sample size) as there
are variables
Correlation Matrix

Variables V1 V2 V3 V4 V5 V6
V1 1.000
V2 -0.530 1.000
V3 0.873 -0.155 1.000
V4 -0.086 0.572 -0.248 1.000
V5 -0.858 0.020 -0.778 -0.007 1.000
V6 0.004 0.640 -0.018 0.640 -0.136 1.000
Conducting Factor Analysis:
Construct the Correlation Matrix
• The analytical process is based on a matrix of correlations
between the variables.
• Bartlett's test of sphericity can be used to test the null
hypothesis that the variables are uncorrelated in the
population: in other words, the population correlation matrix is
an identity matrix. If this hypothesis cannot be rejected, then
the appropriateness of factor analysis should be questioned.
• Another useful statistic is the Kaiser-Meyer-Olkin (KMO)
measure of sampling adequacy. Small values of the KMO
statistic indicate that the correlations between pairs of
variables cannot be explained by other variables and that factor
analysis may not be appropriate
Conducting Factor Analysis:
Determine the Method of Factor Analysis
• In principal components analysis, the total variance
in the data is considered. Principal components analysis
is recommended when the primary concern is to
determine the minimum number of factors that will
account for maximum variance in the data for use in
subsequent multivariate analysis. The factors are called
principal components.

• In common factor analysis, the factors are estimated

based only on the common variance. This method is
appropriate when the primary concern is to identify the
underlying dimensions and the common variance
Results of Principal Components
Analysis

Communalities
Variables Initial Extraction
V1 1.000 0.926
V2 1.000 0.723
V3 1.000 0.894
V4 1.000 0.739
V5 1.000 0.878
V6 1.000 0.790

Initial Eigen values

Factor Eigen value % of variance Cumulat. %
1 2.731 45.520 45.520
2 2.218 36.969 82.488
3 0.442 7.360 89.848
4 0.341 5.688 95.536
5 0.183 3.044 98.580
6 0.085 1.420 100.000
Extraction Sums of Squared Loadings
Factor Eigen value % of variance Cumulat. %
1 2.731 45.520 45.520
2 2.218 36.969 82.488

Factor Matrix
Variables Factor 1 Factor 2
V1 0.928 0.253
V2 -0.301 0.795
V3 0.936 0.131
V4 -0.342 0.789
V5 -0.869 -0.351
V6 -0.177 0.871

Rotation Sums of Squared Loadings

Factor Eigenvalue % of variance Cumulat. %
1 2.688 44.802 44.802
2 2.261 37.687 82.488
Rotated Factor Matrix
Variables Factor 1 Factor 2
V1 0.962 -0.027
V2 -0.057 0.848
V3 0.934 -0.146
V4 -0.098 0.845
V5 -0.933 -0.084
V6 0.083 0.885
Results of Principal Components
Analysis

Rotated Factor Matrix

Variables Factor 1 Factor 2
V1 0.962 -0.027
V2 -0.057 0.848
V3 0.934 -0.146
V4 -0.098 0.845
V5 -0.933 -0.084
V6 0.083 0.885

Factor Score Coefficient Matrix

Variables Factor 1 Factor 2
V1 0.358 0.011
V2 -0.001 0.375
V3 0.345 -0.043
V4 -0.017 0.377
V5 -0.350 -0.059
V6 0.052 0.395
Conducting Factor Analysis:
Determine the Number of Factors
• A Priori Determination. Sometimes, because of prior
knowledge, the researcher knows how many factors to expect and
thus can specify the number of factors to be extracted
beforehand.

• Determination Based on Eigenvalues. In this approach, only

factors with Eigenvalues greater than 1.0 are retained. An
Eigenvalue represents the amount of variance associated with the
factor. Hence, only factors with a variance greater than 1.0 are
included. Factors with variance less than 1.0 are no better than a
single variable, since, due to standardization, each variable has a
variance of 1.0. If the number of variables is less than 20, this
approach will result in a conservative number of factors.
Conducting Factor Analysis:
Determine the Number of Factors
• Determination Based on Scree Plot. A scree plot is a
plot of the Eigenvalues against the number of factors in order
of extraction. Experimental evidence indicates that the point
at which the scree begins denotes the true number of
factors. Generally, the number of factors determined by a
scree plot will be one or a few more than that determined by
the Eigenvalue criterion.

• Determination Based on Percentage of Variance. In

this approach the number of factors extracted is determined
so that the cumulative percentage of variance extracted by
the factors reaches a satisfactory level. It is recommended
that the factors extracted should account for at least 60% of
the variance.
Scree Plot
3.0

2.5

Eigenvalue 2.0

1.5

1.0

0.5
0.0

1 2 3 4 5 6
Component Number
Conducting Factor Analysis: Rotate
Factors
• Although the initial or unrotated factor matrix indicates the
relationship between the factors and individual variables, it
seldom results in factors that can be interpreted, because
the factors are correlated with many variables. Therefore,
through rotation, the factor matrix is transformed into a
simpler one that is easier to interpret.
• In rotating the factors, we would like each factor to have
nonzero, or significant, loadings or coefficients for only
some of the variables. Likewise, we would like each
variable to have nonzero or significant loadings with only a
few factors, if possible with only one.
• The rotation is called orthogonal rotation if the axes are
maintained at right angles
Factor Matrix Before and After
Rotati
Conducting Factor Analysis: Rotate
Factors
• The most commonly used method for rotation is the
varimax procedure. This is an orthogonal method
of rotation that minimizes the number of variables
with high loadings on a factor, thereby enhancing the
interpretability of the factors. Orthogonal rotation
results in factors that are uncorrelated.
• The rotation is called oblique rotation when the axes
are not maintained at right angles, and the factors are
correlated. Sometimes, allowing for correlations
among factors can simplify the factor pattern matrix.
Oblique rotation should be used when factors in the
population are likely to be strongly correlated.
 Factor Matrix Before and After
Rotation

Factors Factors
Variables 1 2 Variables 1 2
1 X 1 X
2 X X 2 X
3 X 3 X
4 X X 4 X
5 X X 5 X
6 X 6 X
(a) (b)
High Loadings High Loadings
Before Rotation After Rotation
Conducting Factor Analysis: Interpret
Factors
• A factor can then be interpreted in terms of
the variables that load high on it.
• Another useful aid in interpretation is to plot
the variables, using the factor loadings as
coordinates. Variables at the end of an axis
are those that have high loadings on only
that factor, and hence describe the fac
 Component Plot in
Rotated Space
Component 1 Rotated Component Matrix
Component 2
1.0 ** * V6
V4
Component
V2
Variable 1 2
0.5
V1 0.962 -2.66E-02
0.0 V1
* V2 -5.72E-02 0.848
* V5 V3 *
-0.5 V3 0.934 -0.146
V4 -9.83E-02 0.854
-1.0
V5 -0.933 -8.40E-02
V6 8.337E-02 0.885
1.0 0.5 0.0 -0.5 -1.0
Conducting Factor Analysis:
Select Surrogate Variables
• By examining the factor matrix, one could select
for each factor the variable with the highest
loading on that factor. That variable could then
be used as a surrogate variable for the
associated factor.
• However, the choice is not as easy if two or
more variables have similarly high loadings. In
such a case, the choice between these variables
should be based on theoretical and
measurement considerations
 Results of Common Factor Analysis

Communalities
Variables Initial Extraction Bartlett test of sphericity
V1
V2
0.859
0.480
0.928
0.562
• Approx. Chi-Square = 111.314
V3
V4
0.814
0.543
0.836
0.600 • df = 15
V5 0.763 0.789
V6 0.587 0.723 • Significance = 0.00000
• Kaiser-Meyer-Olkin measure of
sampling adequacy = 0.660

Initial Eigenvalues
Factor Eigenvalue % of variance Cumulat. %
1 2.731 45.520 45.520
2 2.218 36.969 82.488
3 0.442 7.360 89.848
4 0.341 5.688 95.536
5 0.183 3.044 98.580
6 0.085 1.420 100.000
Results of Common Factor Analysis

Extraction Sums of Squared Loadings

Factor Eigenvalue % of variance Cumulat. %
1 2.570 42.837 42.837
2 1.868 31.126 73.964

Factor Matrix
Variables Factor 1 Factor 2
V1 0.949 0.168
V2 -0.206 0.720
V3 0.914 0.038
V4 -0.246 0.734
V5 -0.850 -0.259
V6 -0.101 0.844

Rotation Sums of Squared Loadings

Factor Eigenvalue % of variance Cumulat. %
1 2.541 42.343 42.343
2 1.897 31.621 73.964
Results of Common Factor Analysis

Rotated Factor Matrix

Variables Factor 1 Factor 2
V1 0.963 -0.030
V2 -0.054 0.747
V3 0.902 -0.150
V4 -0.090 0.769
V5 -0.885 -0.079
V6 0.075 0.847

Factor Score Coefficient Matrix

Variables Factor 1 Factor 2
V1 0.628 0.101
V2 -0.024 0.253
V3 0.217 -0.169
V4 -0.023 0.271
V5 -0.166 -0.059
V6 0.083 0.500
When I hear, I forget
When I see, I remember
When I do, I learn

3
0
QUESTIONS