Factor analysis

Factor analysis is used to uncover the latent structure of a set of variables. It reduces attribute
space from a large no. of variables to a smaller no. of factors and as such is a non dependent
procedure.

Factor analysis could be used for any of the following purpose-

1. To reduce a large no. of variable to a smaller no. of factors for modeling purposes, where
the large number of variables precludes modeling all the measures, individually. As such factor
analysis is integrated in structural equation modeling, helping create the latent variables
modeled by SEM (structure equation model).

2. To select a subset of variables from a large set based on which original variable have the
highest correlations with the principal component factors.

3. To create a set of factors to be treated as uncorrelated variable as one approach to handling

multicollinearity regression.

Assumptions of factor analysis model

1) Measurement error has constant variance and is on average zero, i.e.,
Var (ei) = σi2
E[ei] = 0

2) No association between the factor and measurement error, cov (F, ej ) = 0

3) No association between errors,cov(ej ,ek ) = 0

4) Local (i.e., conditional independence): Given factor, observed variables are independent of one
another, cov( Xj, Xk l F) = 0

Advantages of Factor Analysis:

1. Both objective and subjective attributes can be used.
2. It can be used to identify the hidden dimensions or constraints which may or may not be
apparent from direct analysis.
3. It is not extremely difficult to do and at the same time its inexpensive and gives accurate results.
4. There is flexibility in naming and using dimensions.
Disadvantages of Factor Analysis:
1. The usefulness depends on the researcher’s ability to develop a complete and accurate set of
product attributes. If important attributes are missed the value of procedure is reduced
accordingly.
2. Naming of the factors can be difficult multiple attributes can be highly correlated with no
apparent reasons.
3. If the observed variables are completely unrelated the factor analysis is unable to produce
meaningful pattern.
4. It is not possible to know factors actually represents, only theory can help inform the
researcher’s on this.

STEPS OF FACTOR ANALYSIS:

 Step 1: Prepare the Data

Factor analysis assumes certain conditions that must be met for the technique to be appropriate:

 Interval or Ratio Scale: The variables involved should be continuous (interval or ratio
scale). This is because factor analysis aims to identify correlations between variables,
which work best with continuous data.
 Linearity: Factor analysis assumes that there is a linear relationship between the
variables. This means that the variables should correlate in a straight-line fashion.
 Correlation: Factor analysis is most useful when variables are correlated with each
other. If the variables are uncorrelated, there is little or no common variance to be
explained by factors.
 Sample Size: A larger sample size is preferred to ensure the stability and reliability of the
factor analysis results. In general, a minimum sample size of 100-200 is recommended.

 Step 2: Correlation Matrix

The first task in factor analysis is to examine the relationships between the observed variables
through a correlation matrix. A correlation matrix displays the correlations between each pair
of variables. Factor analysis assumes that the underlying structure of the data is revealed through
these correlations.

The correlation matrix provides insight into how much of the variance in one variable can be
explained by the other variables. If the variables show high intercorrelations, it indicates that
they may share a common underlying factor. The correlation matrix is the starting point for
identifying the number of latent factors that can explain the correlations.
  Step 3: Factor Extraction

Factor extraction is the process of identifying the underlying factors (latent variables) that
explain the relationships between the observed variables. The goal is to reduce the
dimensionality of the data by extracting a few factors that can summarize the data.

There are various methods for factor extraction, with the most common being:

1. Principal Component Analysis (PCA): PCA is a method that creates new variables
(principal components) by linearly combining the original variables. These components
are ordered by the amount of variance they explain in the data. The first principal
component accounts for the largest variance, and so on. PCA is often used in exploratory
factor analysis to determine how many factors should be retained.
2. Principal Axis Factoring (PAF): This method focuses on identifying the shared
variance among the observed variables. It aims to extract factors that explain the common
variance (the variance shared by the variables), as opposed to the total variance (which
includes both shared and unique variance).
3. Maximum Likelihood Estimation (MLE): MLE is another factor extraction method
that assumes the data follows a multivariate normal distribution. It estimates factors by
maximizing the likelihood that the observed data comes from a specific factor model.

 Step 4: Factor Rotation

Once the factors are extracted, the next step is rotation. Rotation helps to make the factors more
interpretable and meaningful. Without rotation, the extracted factors may be hard to interpret
because they could be a mixture of different dimensions.

There are two main types of rotation:

1. Orthogonal Rotation (e.g., Varimax): In orthogonal rotation, the factors remain

uncorrelated with each other. This means that each factor is independent, and no overlap
exists between the factors. Varimax is the most common orthogonal rotation method. It
maximizes the variance of squared factor loadings within each factor to achieve a
simpler, more interpretable structure.
2. Oblique Rotation (e.g., Promax): In oblique rotation, the factors are allowed to correlate
with each other. This method assumes that the factors may be related in some way, which
can lead to a more accurate representation of the underlying structure. Promax is a
common oblique rotation method that begins with an orthogonal rotation and then allows
for correlations between the factors.

Rotation makes the factor loadings (the correlations between variables and factors) easier to
interpret. High factor loadings indicate a strong association between the variable and the factor,
while low loadings indicate a weak association.
  Step 5: Interpretation of Factors

After rotation, the factors are examined to understand what they represent. Each factor is
interpreted based on the variables that have high loadings on it. For example:

 A factor that loads highly on variables like "Product Quality," "Customer Service," and
"Reliability" might be interpreted as a "Quality Factor".
 A factor that loads highly on "Price Satisfaction," "Discounts," and "Value for Money"
might represent a "Pricing Factor".

The goal is to label and describe the latent factors in a way that is meaningful and useful for
further analysis or decision-making.

 Step 6: Factor Scores

Once the factors are identified and interpreted, factor scores are calculated for each observation
(or case) in the dataset. Factor scores represent an individual's position on each factor and can be
treated as new variables for further analysis. They provide a summary of the individual's
responses or characteristics as defined by the extracted factors.

Factor scores can be used in subsequent analyses (e.g., regression, clustering, or classification)
instead of the original observed variables, allowing for a reduced and simplified representation of
the data.