An introduction to the psych package: Part II

Scale construction and psychometrics

William Revelle
Department of Psychology
Northwestern University

September 23, 2023

Contents
0.1 Jump starting the psych package–a guide for the impatient . . . . . . . . . . . . . 4
0.2 Psychometric functions are summarized in this vignette . . . . . . . . . . . . . . . 5

1 Overview of this and related documents 7

2 Getting started 8

3 Basic data analysis 9

4 Item and scale analysis 9

4.1 Dimension reduction through factor analysis and cluster analysis . . . . . . . . . . 10
4.1.1 Minimum Residual Factor Analysis . . . . . . . . . . . . . . . . . . . . . 12
4.1.2 Principal Axis Factor Analysis . . . . . . . . . . . . . . . . . . . . . . . . 12
4.1.3 Alpha Factor Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.1.4 Weighted Least Squares Factor Analysis . . . . . . . . . . . . . . . . . . . 15
4.1.5 Principal Components analysis (PCA) . . . . . . . . . . . . . . . . . . . . 15
4.1.6 Hierarchical and bi-factor solutions . . . . . . . . . . . . . . . . . . . . . 15
4.1.7 Item Cluster Analysis: iclust . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.2 Estimates of fit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.3 Confidence intervals using bootstrapping techniques . . . . . . . . . . . . . . . . . 29
4.4 Comparing factor/component/cluster solutions . . . . . . . . . . . . . . . . . . . . 29
4.5 Determining the number of dimensions to extract. . . . . . . . . . . . . . . . . . . 31
4.5.1 Very Simple Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.5.2 Parallel Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.6 Factor extension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

1
 4.7 Exploratory Structural Equation Modeling (ESEM) . . . . . . . . . . . . . . . . . 37

5 Classical Test Theory and Reliability 38

5.1 Reliability of a single scale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
5.2 Using omega to find the reliability of a single scale . . . . . . . . . . . . . . . . . 44
5.3 Estimating ωh using Confirmatory Factor Analysis . . . . . . . . . . . . . . . . . 49
5.3.1 Other estimates of reliability . . . . . . . . . . . . . . . . . . . . . . . . . 51
5.4 Reliability and correlations of multiple scales within an inventory . . . . . . . . . 51
5.4.1 Scoring from raw data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
5.4.2 Forming scales from a correlation matrix . . . . . . . . . . . . . . . . . . 53
5.5 Scoring Multiple Choice Items . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
5.6 Item analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
5.6.1 Exploring the item structure of scales . . . . . . . . . . . . . . . . . . . . 58
5.6.2 Empirical scale construction . . . . . . . . . . . . . . . . . . . . . . . . . 58

6 Item Response Theory analysis 59

6.1 Factor analysis and Item Response Theory . . . . . . . . . . . . . . . . . . . . . . 59
6.2 Speeding up analyses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
6.3 IRT based scoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
6.3.1 1 versus 2 parameter IRT scoring . . . . . . . . . . . . . . . . . . . . . . 69

7 Multilevel modeling 70
7.1 Decomposing data into within and between level correlations using statsBy . . . 71
7.2 Generating and displaying multilevel data . . . . . . . . . . . . . . . . . . . . . . 71
7.3 Factor analysis by groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
7.4 Multilevel reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

8 Set Correlation and Multiple Regression from the correlation matrix 72

9 Simulation functions 74

10 Graphical Displays 76

11 Converting output to APA style tables using LATEX 77

12 Miscellaneous functions 79

13 Data sets 80

14 Development version and a users guide 81

15 Psychometric Theory 81

2
16 SessionInfo 82

3
0.1 Jump starting the psych package–a guide for the impatient
You have installed psych (section 2) and you want to use it without reading much more. What
should you do?
1. Activate the psych package:
library(psych)
library(psychTools)

2. Input your data (see the Introduction to Psych vignette section 3.1). There are two ways to
do this:
• Find and read standard files using read.file. This will open a search window for
your operating system which you can use to find the file. If the file has a suffix of .text,
.txt, .csv, .data, .sav, .r, .R, .rds, .Rds, .rda, .Rda, .rdata, or .RData, then the file will be
opened and the data will be read in.
myData <- read.file() # find the appropriate file using your normal operating system

• Alternatively, go to your friendly text editor or data manipulation program (e.g., Excel)
and copy the data to the clipboard. Include a first line that has the variable labels. Paste
it into psych using the read.clipboard.tab command:
myData <- read.clipboard.tab() # if on the clipboard

Note that there are number of options for read.clipboard for reading in Excel
based files, lower triangular files, etc.
3. Make sure that what you just read is right. Describe it (see the Introduction to Psych vignette
section 3.3) on how to describe data) and perhaps look at the first and last few lines. If
you have multiple groups, try describeBy.
dim(myData) #What are the dimensions of the data?
describe(myData) # or
descrbeBy(myData,groups="mygroups") #for descriptive statistics by groups
headTail(myData) #show the first and last n lines of a file

4. Look at the patterns in the data. If you have fewer than about 12 variables, look at the
SPLOM (Scatter Plot Matrix) of the data using pairs.panels ( (see the Introduction to
Psych vignette section 3.4 for a discussion of graphics)) . Then, use the outlier function
to detect outliers.
pairs.panels(myData)
outlier(myData)

5. Note that you might have some weird subjects, probably due to data entry errors. Either edit
the data by hand (use the edit command) or just scrub the data).
cleaned <- scrub(myData, max=9) #e.g., change anything great than 9 to NA

6. Graph the data with error bars for each variable ( (see the Introduction to Psych vignette
section 3.1)).

4
 error.bars(myData)

7. Find the correlations of all of your data. lowerCor will by default find the pairwise corre-
lations, round them to 2 decimals, and display the lower off diagonal matrix.
• Descriptively (just the values) (section ??)
r <- lowerCor(myData) #The correlation matrix, rounded to 2 decimals

• Graphically (section ??). Another way is to show a heat map of the correlations with
the correlation values included.
corPlot(r) #examine the many options for this function.

• Inferentially (the values, the ns, and the p values) (section ??)
corr.test(myData)

8. Apply various regression models.

Several functions are meant to do multiple regressions, either from the raw data or from a
variance/covariance matrix, or a correlation matrix.
• lmCor will take raw data or a correlation matrix and find (and graph the path diagram)
for multiple y variables depending upon multiple x variables.
myData <- sat.act
colnames(myData) <- c("mod1","med1","x1","x2","y1","y2")
lmCor(y1 + y2 ~ x1 + x2 , data = myData)

• mediate will take raw data or a correlation matrix and find (and graph the path dia-
gram) for multiple y variables depending upon multiple x variables mediated through a
mediation variable. It then tests the mediation effect using a boot strap.
mediate(y1 + y2 ~ x1 + x2 + (med1) , data = myData)

• mediate will take raw data and find (and graph the path diagram) a moderated mul-
tiple regression model for multiple y variables depending upon multiple x variables
mediated through a mediation variable. It then tests the mediation effect using a boot
strap.
mediate(y1 + y2 ~ x1 + x2* mod1 +(med1), data = myData)

0.2 Psychometric functions are summarized in this vignette

Many additional functions, particularly designed for basic and advanced psychometrics are
discussed more fully in this Vignette. A brief review of the functions available is included
here. For basic data entry and descriptive statistics, see the Vignette Intro to Psych https:
//personality-project.org/r. In addition, there are helpful tutorials for Finding
omega, How to score scales and find reliability, and for Using psych for factor analysis at
https://personality-project.org/r.

5
 • Test for the number of factors in your data using parallel analysis (fa.parallel,
section 4.5.2) or Very Simple Structure (vss, 4.5.1) .
fa.parallel(myData)
vss(myData)

• Factor analyze (see section 4.1) the data with a specified number of factors (the de-
fault is 1), the default method is minimum residual, the default rotation for more than
one factor is oblimin. There are many more possibilities (see sections 4.1.1-4.1.4).
Compare the solution to a hierarchical cluster analysis using the ICLUST algorithm
(Revelle, 1979) (see section 4.1.7). Also consider a hierarchical factor solution to find
coefficient ω (see 4.1.6).
fa(myData)
iclust(myData)
omega(myData)

If you prefer to do a principal components analysis you may use the principal
function. The default is one component.
principal(myData)

• Some people like to find coefficient α as an estimate of reliability. This may be done for
a single scale using the alpha function (see 5.1). Perhaps more useful is the ability to
create several scales as unweighted averages of specified items using the scoreItems
function (see 5.4) and to find various estimates of internal consistency for these scales,
find their intercorrelations, and find scores for all the subjects.
alpha(myData) #score all of the items as part of one scale.
myKeys <- make.keys(nvar=20,list(first = c(1,-3,5,-7,8:10),second=c(2,4,-6,11:15,-16)))
my.scores <- scoreItems(myKeys,myData) #form several scales
my.scores #show the highlights of the results

At this point you have had a chance to see the highlights of the psych package and to do some basic
(and advanced) data analysis. You might find reading this entire vignette as well as the Overview
Vignette to be helpful to get a broader understanding of what can be done in R using the psych.
Remember that the help command (?) is available for every function. Try running the examples for
each help page.

6
1 Overview of this and related documents
The psych package (Revelle, 2023) has been developed at Northwestern University since 2005
to include functions most useful for personality, psychometric, and psychological research. The
package is also meant to supplement a text on psychometric theory (Revelle, prep), a draft of
which is available at https://personality-project.org/r/book/.
Some of the functions (e.g., read.file, read.clipboard, describe, pairs.panels,
scatter.hist, error.bars, multi.hist, bi.bars) are useful for basic data entry and
descriptive analyses.
Psychometric applications emphasize techniques for dimension reduction including factor analysis,
cluster analysis, and principal components analysis. The fa function includes five methods of fac-
tor analysis (minimum residual, principal axis, weighted least squares, generalized least squares
and maximum likelihood factor analysis). Principal Components Analysis (PCA) is also avail-
able through the use of the principal or pca functions. Determining the number of factors
or components to extract may be done by using the Very Simple Structure (Revelle and Rock-
lin, 1979) (vss), Minimum Average Partial correlation (Velicer, 1976) (MAP) or parallel analysis
(fa.parallel) criteria. These and several other criteria are included in the nfactors func-
tion. Two parameter Item Response Theory (IRT) models for dichotomous or polytomous items
may be found by factoring tetrachoric or polychoric correlation matrices and expressing
the resulting parameters in terms of location and discrimination using irt.fa.
Bifactor and hierarchical factor structures may be estimated by using Schmid Leiman transfor-
mations (Schmid and Leiman, 1957) (schmid) to transform a hierarchical factor structure into a
bifactor solution (Holzinger and Swineford, 1937). Higher order models can also be found using
fa.multi.
Scale construction can be done using the Item Cluster Analysis (Revelle, 1979) (iclust) func-
tion to determine the structure and to calculate reliability coefficients α (Cronbach, 1951)(alpha,
scoreItems, score.multiple.choice), β (Revelle, 1979; Revelle and Zinbarg, 2009)
(iclust) and McDonald’s ωh and ωt (McDonald, 1999) (omega). Guttman’s six estimates of
internal consistency reliability (Guttman (1945), as well as additional estimates (Revelle and Zin-
barg, 2009) are in the guttman function. The six measures of Intraclass correlation coefficients
(ICC) discussed by Shrout and Fleiss (1979) are also available.
For data with a a multilevel structure (e.g., items within subjects across time, or items within
subjects across groups), the describeBy, statsBy functions will give basic descriptives by
group. StatsBy also will find within group (or subject) correlations as well as the between group
correlation.
multilevel.reliability mlr will find various generalizability statistics for subjects over
time and items. mlPlot will graph items over for each subject, mlArrange converts wide data
frames to long data frames suitable for multilevel modeling.

7
Graphical displays include Scatter Plot Matrix (SPLOM) plots using pairs.panels, correlation
“heat maps” (corPlot) factor, cluster, and structural diagrams using fa.diagram, iclust.diagram,
structure.diagram and het.diagram, as well as item response characteristics and item
and test information characteristic curves plot.irt and plot.poly.
This vignette is meant to give an overview of the psych package. That is, it is meant to give a
summary of the main functions in the psych package with examples of how they are used for data
description, dimension reduction, and scale construction. The extended user manual at psych_
manual.pdf includes examples of graphic output and more extensive demonstrations than are
found in the help menus. (Also available at https://personality-project.org/r/
psych_manual.pdf). The vignette, psych for sem, at psych_for_sem.pdf, discusses how
to use psych as a front end to the sem package of John Fox (Fox et al., 2013). (The vignette
is also available at https://personality-project.org/r/book/psych_for_sem.
pdf).
In addition, there are a growing number of “HowTo"s at the personality project. Currently these
include:
1. An introduction (vignette) of the psych package
2. An overview (vignette) of the psych package
3. Installing R and some useful packages
4. Using R and the psych package to find omegah and ωt .
5. Using R and the psych for factor analysis and principal components analysis.
6. Using the scoreItems function to find scale scores and scale statistics.
7. Using mediate and lmCor to do mediation, moderation and regression analysis.
For a step by step tutorial in the use of the psych package and the base functions in R for basic per-
sonality research, see the guide for using R for personality research at https://personalitytheory.
org/r/r.short.html. For an introduction to psychometric theory with applications in R, see
the draft chapters at https://personality-project.org/r/book).

2 Getting started
Some of the functions described in this overview require other packages. Particularly useful for ro-
tating the results of factor analyses (from e.g., fa, factor.minres, factor.pa, factor.wls,
or principal) or hierarchical factor models using omega or schmid, is the GPArotation pack-
age. These and other useful packages may be installed by first installing and then using the task
views (ctv) package to install the “Psychometrics" task view, but doing it this way is not neces-
sary.

8
install.packages("ctv")
library(ctv)
task.views("Psychometrics")
The “Psychometrics” task view will install a large number of useful packages. To install the bare
minimum for the examples in this vignette, it is necessary to install just 3 packages:
install.packages(list(c("GPArotation","mnormt","psychTools")
Because of the difficulty of installing the package Rgraphviz, alternative graphics have been de-
veloped and are available as diagram functions. If Rgraphviz is available, some functions will
take advantage of it. An alternative is to use “dot” output of commands for any external graphics
package that uses the dot language.

3 Basic data analysis

A number of psych functions facilitate the entry of data and finding basic descriptive statistics.
These are described in more detail in the companion vignette: An introduction to the psych pack-
age: Part I which is also available from the personality-project site. https://personality-project.
org/r/psych/vignettes/intro.pdf. Please consult that vignette first for information on
how to read data (particularly using the read.file and read.clipboard commands), Also,
the describe and describeBy functions are described in more detail in the introductory vi-
gnette. For even though you probably want to jump immediately to factor analyze your data, this is
a mistake. It is very important to first describe them and look for weird responses. It is also useful
to scrub your data when removing outliers, to graphically display them using pairs.panesl
and corPlot. Basic multiple regression and moderated or mediated regressions may be done
from either the raw data or from correlation matrices using lmCor, or mediation.
Remember, to run any of the psych functions, it is necessary to make the package active by using
the library command:
library(psych)
The other packages, once installed, will be called automatically by psych.
It is possible to automatically load psych and other functions by creating and then saving a “.First"
function: e.g.,
.First <- function(x) {library(psych)}

4 Item and scale analysis

The main functions in the psych package are for analyzing the structure of items and of scales
and for finding various estimates of scale reliability. These may be considered as problems of

9
dimension reduction (e.g., factor analysis, cluster analysis, principal components analysis) and of
forming and estimating the reliability of the resulting composite scales.

4.1 Dimension reduction through factor analysis and cluster analysis

Parsimony of description has been a goal of science since at least the famous dictum commonly
attributed to William of Ockham to not multiply entities beyond necessity1 . The goal for parsimony
is seen in psychometrics as an attempt either to describe (components) or to explain (factors) the
relationships between many observed variables in terms of a more limited set of components or
latent factors.
The typical data matrix represents multiple items or scales usually thought to reflect fewer un-
derlying constructs2 . At the most simple, a set of items can be be thought to represent a random
sample from one underlying domain or perhaps a small set of domains. The question for the psy-
chometrician is how many domains are represented and how well does each item represent the
domains. Solutions to this problem are examples of factor analysis (FA), principal components
analysis (PCA), and cluster analysis (CA). All of these procedures aim to reduce the complexity of
the observed data. In the case of FA, the goal is to identify fewer underlying constructs to explain
the observed data. In the case of PCA, the goal can be mere data reduction, but the interpretation of
components is frequently done in terms similar to those used when describing the latent variables
estimated by FA. Cluster analytic techniques, although usually used to partition the subject space
rather than the variable space, can also be used to group variables to reduce the complexity of the
data by forming fewer and more homogeneous sets of tests or items.
At the data level the data reduction problem may be solved as a Singular Value Decomposition of
the original matrix, although the more typical solution is to find either the principal components
or factors of the covariance or correlation matrices. Given the pattern of regression weights from
the variables to the components or from the factors to the variables, it is then possible to find (for
components) individual component or cluster scores or estimate (for factors) factor scores.
Several of the functions in psych address the problem of data reduction.
fa incorporates six alternative algorithms: minres factor analysis, principal axis factor analysis,
alpha factor analysis, weighted least squares factor analysis, generalized least squares fac-
tor analysis and maximum likelihood factor analysis. That is, it includes the functionality of
three other functions that are deprecated and will be eventually phased out.
fa.poly (deprecated) is useful when finding the factor structure of categorical items. fa.poly first finds the tetrachoric or polychoric correlations between the categorical
variables and then proceeds to do a normal factor analysis. By setting the n.iter option to be greater than 1, it will also find confidence intervals for the factor solution.
Warning. Finding polychoric correlations is very slow, so think carefully before doing so. These options are now part of the fa function and can be controlled by setting
the cor parameter to ‘tet’ or ‘poly’.

1 Although probably neither original with Ockham nor directly stated by him (Thorburn, 1918), Ockham’s razor

remains a fundamental principal of science.

2 Cattell (1978) as well as MacCallum et al. (2007) argue that the data are the result of many more factors than

observed variables, but are willing to estimate the major underlying factors.

10
factor.minres (deprecated) Minimum residual factor analysis is a least squares, iterative solution to the factor problem. minres attempts to minimize the residual (off-
diagonal) correlation matrix. It produces solutions similar to maximum likelihood solutions, but will work even if the matrix is singular.

factor.pa (deprecated) Principal Axis factor analysis is a least squares, iterative solution to the factor problem. PA will work for cases where maximum likelihood techniques
(factanal) will not work. The original communality estimates are either the squared multiple correlations (smc) for each item or 1.

factor.wls (deprecated) Weighted least squares factor analysis is a least squares, iterative solution to the factor problem. It minimizes the (weighted) squared residual matrix.
The weights are based upon the independent contribution of each variable.

principal Principal Components Analysis reports the largest n eigen vectors rescaled by the
square root of their eigen values. Note that PCA is not the same as factor analysis and the
two should not be confused.
factor.congruence The congruence between two factors is the cosine of the angle between
them. This is just the cross products of the loadings divided by the sum of the squared load-
ings. This differs from the correlation coefficient in that the mean loading is not subtracted
before taking the products. factor.congruence will find the cosines between two (or
more) sets of factor loadings.
vss Very Simple Structure Revelle and Rocklin (1979) applies a goodness of fit test to determine
the optimal number of factors to extract. It can be thought of as a quasi-confirmatory model,
in that it fits the very simple structure (all except the biggest c loadings per item are set to
zero where c is the level of complexity of the item) of a factor pattern matrix to the original
correlation matrix. For items where the model is usually of complexity one, this is equivalent
to making all except the largest loading for each item 0. This is typically the solution that
the user wants to interpret. The analysis includes the MAP criterion of Velicer (1976) and a
χ 2 estimate.
nfactors combines VSS, MAP, and a number of other fit statistics. The depressing reality is
that frequently these conventional fit estimates of the number of factors do not agree.
fa.parallel The parallel factors technique compares the observed eigen values of a correla-
tion matrix with those from random data.
fa.plot will plot the loadings from a factor, principal components, or cluster analysis (just a
call to plot will suffice). If there are more than two factors, then a SPLOM of the loadings is
generated.
fa.diagram replaces fa.graph and will draw a path diagram representing the factor struc-
ture. It does not require Rgraphviz and thus is probably preferred.
fa.graph requires Rgraphviz and will draw a graphic representation of the factor structure.
If factors are correlated, this will be represented as well.
iclust is meant to do item cluster analysis using a hierarchical clustering algorithm specifically
asking questions about the reliability of the clusters (Revelle, 1979). Clusters are formed
until either coefficient α Cronbach (1951) or β Revelle (1979) fail to increase.

11
4.1.1 Minimum Residual Factor Analysis
The factor model is an approximation of a correlation matrix by a matrix of lower rank. That is, can
the correlation matrix, n~Rn be approximated by the product of a factor matrix, n~Fk and its transpose
plus a diagonal matrix of uniqueness.

R = FF 0 +U 2 (1)

The maximum likelihood solution to this equation is found by factanal in the stats package as
well as the fa function in psych. Seven alternatives are provided in psych, all of them are included
in the fa function and are called by specifying the factor method (e.g., fm=“minres", fm=“pa",
fm=“alpha" fm=‘wls", fm=“gls", fm = “min.rank", and fm=“ml"). In the discussion of the other
algorithms, the calls shown are to the fa function specifying the appropriate method.
factor.minres attempts to minimize the off diagonal residual correlation matrix by adjusting
the eigen values of the original correlation matrix. This is similar to what is done in factanal,
but uses an ordinary least squares instead of a maximum likelihood fit function. The solutions tend
to be more similar to the MLE solutions than are the factor.pa solutions. min.res is the default
for the fa function.
A classic data set, collected by Thurstone and Thurstone (1941) and then reanalyzed by Bech-
toldt (1961) and discussed by McDonald (1999), is a set of 9 cognitive variables with a clear
bi-factor structure (Holzinger and Swineford, 1937). The minimum residual solution was trans-
formed into an oblique solution using the default option on rotate which uses an oblimin trans-
formation (Table 1). Alternative rotations and transformations include “none", “varimax", “quar-
timax", “bentlerT", “varimin” and “geominT" (all of which are orthogonal rotations). as well as
“promax", “oblimin", “simplimax", “bentlerQ, and “geominQ" and “cluster" which are possible
oblique transformations of the solution. The default is to do a oblimin transformation. The mea-
sures of factor adequacy reflect the multiple correlations of the factors with the best fitting linear
regression estimates of the factor scores (Grice, 2001).
Note that if extracting more than one factor, and doing any oblique rotation, it is necessary to have
the GPArotation installed. This is checked for in the appropriate functions.

4.1.2 Principal Axis Factor Analysis

An alternative, least squares algorithm (included in fa with the fm=pa option or as a standalone
function (factor.pa), does a Principal Axis factor analysis by iteratively doing an eigen value
decomposition of the correlation matrix with the diagonal replaced by the values estimated by the
factors of the previous iteration. This OLS solution is not as sensitive to improper matrices as is
the maximum likelihood method, and will sometimes produce more interpretable results. It seems
as if the SAS example for PA uses only one iteration. Setting the max.iter parameter to 1 produces
the SAS solution.

12
Table 1: Three correlated factors from the Thurstone 9 variable problem. By default, the solution
is transformed obliquely using oblimin. The extraction method is (by default) minimum residual.
> if(!require('GPArotation')) {stop('GPArotation must be installed to do rotations')} else {
+ library(psych)
+ library(psychTools)
+ f3t <- fa(Thurstone,3,n.obs=213)
+ f3t }
Factor Analysis using method = minres
Call: fa(r = Thurstone, nfactors = 3, n.obs = 213)
Standardized loadings (pattern matrix) based upon correlation matrix
MR1 MR2 MR3 h2 u2 com
Sentences 0.90 -0.03 0.04 0.82 0.18 1.0
Vocabulary 0.89 0.06 -0.03 0.84 0.16 1.0
Sent.Completion 0.84 0.03 0.00 0.74 0.26 1.0
First.Letters 0.00 0.85 0.00 0.73 0.27 1.0
Four.Letter.Words -0.02 0.75 0.10 0.63 0.37 1.0
Suffixes 0.18 0.63 -0.08 0.50 0.50 1.2
Letter.Series 0.03 -0.01 0.84 0.73 0.27 1.0
Pedigrees 0.38 -0.05 0.46 0.51 0.49 2.0
Letter.Group -0.06 0.21 0.63 0.52 0.48 1.2

MR1 MR2 MR3

SS loadings 2.65 1.87 1.49
Proportion Var 0.29 0.21 0.17
Cumulative Var 0.29 0.50 0.67
Proportion Explained 0.44 0.31 0.25
Cumulative Proportion 0.44 0.75 1.00

With factor correlations of

MR1 MR2 MR3
MR1 1.00 0.59 0.53
MR2 0.59 1.00 0.52
MR3 0.53 0.52 1.00

Mean item complexity = 1.2

Test of the hypothesis that 3 factors are sufficient.

df null model = 36 with the objective function = 5.2 with Chi Square = 1081.97
df of the model are 12 and the objective function was 0.01

The root mean square of the residuals (RMSR) is 0.01

The df corrected root mean square of the residuals is 0.01

The harmonic n.obs is 213 with the empirical chi square 0.52 with prob < 1
The total n.obs was 213 with Likelihood Chi Square = 2.98 with prob < 1

Tucker Lewis Index of factoring reliability = 1.026

RMSEA index = 0 and the 90 % confidence intervals are 0 0
BIC = -61.36
Fit based upon off diagonal values = 1
Measures of factor score adequacy
MR1 MR2 MR3
Correlation of (regression) scores with factors 0.96 0.92 0.90
Multiple R square of scores with factors 0.93 0.85 0.82
Minimum correlation of possible factor scores 0.86 0.71 0.63

13
The solutions from the fa, the factor.minres and factor.pa as well as the principal
functions can be rotated or transformed with a number of options. Some of these call the GPAro-
tation package. Orthogonal rotations include varimax, quartimax, varimin, bifactor .
Oblique transformations include oblimin, quartimin, biquartimin and then two targeted
rotation functions Promax and target.rot. The latter of these will transform a loadings ma-
trix towards an arbitrary target matrix. The default is to transform towards an independent cluster
solution.
Using the Thurstone data set, three factors were requested and then transformed into an independent
clusters solution using target.rot (Table 2).

Table 2: The 9 variable problem from Thurstone is a classic example of factoring where there is a
higher order factor, g, that accounts for the correlation between the factors. The extraction method
was principal axis. The transformation was a targeted transformation to a simple cluster solution.
> if(!require('GPArotation')) {stop('GPArotation must be installed to do rotations')} else {
+ f3 <- fa(Thurstone,3,n.obs = 213,fm="pa")
+ f3o <- target.rot(f3)
+ f3o}
Call: NULL
Standardized loadings (pattern matrix) based upon correlation matrix
PA1 PA2 PA3 h2 u2
Sentences 0.89 -0.03 0.07 0.81 0.19
Vocabulary 0.89 0.07 0.00 0.80 0.20
Sent.Completion 0.83 0.04 0.03 0.70 0.30
First.Letters -0.02 0.85 -0.01 0.73 0.27
Four.Letter.Words -0.05 0.74 0.09 0.57 0.43
Suffixes 0.17 0.63 -0.09 0.43 0.57
Letter.Series -0.06 -0.08 0.84 0.69 0.31
Pedigrees 0.33 -0.09 0.48 0.37 0.63
Letter.Group -0.14 0.16 0.64 0.45 0.55

PA1 PA2 PA3

SS loadings 2.45 1.72 1.37
Proportion Var 0.27 0.19 0.15
Cumulative Var 0.27 0.46 0.62
Proportion Explained 0.44 0.31 0.25
Cumulative Proportion 0.44 0.75 1.00
PA1 PA2 PA3
PA1 1.00 0.02 0.08
PA2 0.02 1.00 0.09
PA3 0.08 0.09 1.00

4.1.3 Alpha Factor Analysis

Introduced by Kaiser and Caffrey (1965) and discussed by Loehlin and Beaujean (2017), alpha
factor analysis factors the matrix of correlations or covariances corrected for their communalities.
This has the effect of making all correlations corrected for relability to reflect their true, latent
correlations. alpha factor analysis was added in August, 2017 to increase the range of EFA options
available. This is added more for completeness rather than an endorsement of the procedure. It is

14
worth comparing solutions from minres, alpha, and MLE, for they are not the same.

4.1.4 Weighted Least Squares Factor Analysis

Similar to the minres approach of minimizing the squared residuals, factor method “wls" weights
the squared residuals by their uniquenesses. This tends to produce slightly smaller overall residuals.
In the example of weighted least squares, the output is shown by using the print function with
the cut option set to 0. That is, all loadings are shown (Table 3).
subsectionDisplaying factor solutions The unweighted least squares solution may be shown graphi-
cally using the fa.plot function which is called by the generic plot function (Figure 1). Factors
were transformed obliquely using a oblimin. These solutions may be shown as item by factor plots
(Figure 1) or by a structure diagram (Figure 2).
A comparison of these three approaches suggests that the minres solution is more similar to a max-
imum likelihood solution and fits slightly better than the pa or wls solutions. Comparisons with
SPSS suggest that the pa solution matches the SPSS OLS solution, but that the minres solution
is slightly better. At least in one test data set, the weighted least squares solutions, although fit-
ting equally well, had slightly different structure loadings. Note that the rotations used by SPSS
will sometimes use the “Kaiser Normalization”. By default, the rotations used in psych do not
normalize, but this can be specified as an option in fa.

4.1.5 Principal Components analysis (PCA)

An alternative to factor analysis, which is unfortunately frequently confused with factor analysis,
is principal components analysis. Although the goals of PCA and FA are similar, PCA is a descrip-
tive model of the data, while FA is a structural model. Some psychologists use PCA in a manner
similar to factor analysis and thus the principal function produces output that is perhaps more
understandable than that produced by princomp in the stats package. Table 4 shows a PCA of
the Thurstone 9 variable problem rotated using the Promax function. Note how the loadings from
the factor model are similar but smaller than the principal component loadings. This is because the
PCA model attempts to account for the entire variance of the correlation matrix, while FA accounts
for just the common variance. This distinction becomes most important for small correlation ma-
trices. Also note how the goodness of fit statistics, based upon the residual off diagonal elements,
is much worse than the fa solution.

4.1.6 Hierarchical and bi-factor solutions

For a long time structural analysis of the ability domain have considered the problem of factors
that are themselves correlated. These correlations may themselves be factored to produce a higher
order, general factor. An alternative (Holzinger and Swineford, 1937; Jensen and Weng, 1994) is
to consider the general factor affecting each item, and then to have group factors account for the
residual variance. Exploratory factor solutions to produce a hierarchical or a bifactor solution are

15
Table 3: The 9 variable problem from Thurstone is a classic example of factoring where there is
a higher order factor, g, that accounts for the correlation between the factors. The factors were
extracted using a weighted least squares algorithm. All loadings are shown by using the cut=0
option in the print.psych function.
> f3w <- fa(Thurstone,3,n.obs = 213,fm="wls")
> print(f3w,cut=0,digits=3)
Factor Analysis using method = wls
Call: fa(r = Thurstone, nfactors = 3, n.obs = 213, fm = "wls")
Standardized loadings (pattern matrix) based upon correlation matrix
WLS1 WLS2 WLS3 h2 u2 com
Sentences 0.905 -0.034 0.040 0.822 0.178 1.01
Vocabulary 0.890 0.066 -0.031 0.835 0.165 1.01
Sent.Completion 0.833 0.034 0.007 0.735 0.265 1.00
First.Letters -0.002 0.855 0.003 0.731 0.269 1.00
Four.Letter.Words -0.016 0.743 0.106 0.629 0.371 1.04
Suffixes 0.180 0.626 -0.082 0.496 0.504 1.20
Letter.Series 0.033 -0.015 0.838 0.719 0.281 1.00
Pedigrees 0.381 -0.051 0.464 0.505 0.495 1.95
Letter.Group -0.062 0.209 0.632 0.527 0.473 1.24

WLS1 WLS2 WLS3

SS loadings 2.647 1.864 1.489
Proportion Var 0.294 0.207 0.165
Cumulative Var 0.294 0.501 0.667
Proportion Explained 0.441 0.311 0.248
Cumulative Proportion 0.441 0.752 1.000

With factor correlations of

WLS1 WLS2 WLS3
WLS1 1.000 0.591 0.535
WLS2 0.591 1.000 0.516
WLS3 0.535 0.516 1.000

Mean item complexity = 1.2

Test of the hypothesis that 3 factors are sufficient.

df null model = 36 with the objective function = 5.198 with Chi Square = 1081.968
df of the model are 12 and the objective function was 0.014

The root mean square of the residuals (RMSR) is 0.006

The df corrected root mean square of the residuals is 0.01

The harmonic n.obs is 213 with the empirical chi square 0.531 with prob < 1
The total n.obs was 213 with Likelihood Chi Square = 2.885 with prob < 0.996

Tucker Lewis Index of factoring reliability = 1.0264

RMSEA index = 0 and the 90 % confidence intervals are 0 0
BIC = -61.45
Fit based upon off diagonal values = 1
Measures of factor score adequacy
WLS1 WLS2 WLS3
Correlation of (regression) scores with factors 0.964 0.923 0.902
Multiple R square of scores with factors 0.929 0.853 0.814
Minimum correlation of possible factor scores 0.858 0.706 0.627

16
> plot(f3t)

Factor Analysis
0.0 0.2 0.4 0.6 0.8

0.8
0.6
MR1

0.4
0.2
0.0
0.8
0.6

MR2
0.4
0.2
0.0

0.8
0.6
0.4
MR3 0.2
0.0

0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8

Figure 1: A graphic representation of the 3 oblique factors from the Thurstone data using plot.
Factors were transformed to an oblique solution using the oblimin function from the GPArotation
package.

17
> fa.diagram(f3t)
Factor Analysis

Sentences
0.9
Vocabulary 0.9
MR1
Sent.Completion 0.8

0.6
First.Letters
0.9

Four.Letter.Words 0.7 MR2 0.5

0.6
Suffixes
0.5

Letter.Series 0.8
MR3
Letter.Group 0.6

0.5
Pedigrees

Figure 2: A graphic representation of the 3 oblique factors from the Thurstone data using
fa.diagram. Factors were transformed to an oblique solution using oblimin.

18
Table 4: The Thurstone problem can also be analyzed using Principal Components Analysis. Com-
pare this to Table 2. The loadings are higher for the PCA because the model accounts for the unique
as well as the common variance.The fit of the off diagonal elements, however, is much worse than
the fa results.
> p3p <-principal(Thurstone,3,n.obs = 213,rotate="Promax")
> p3p
Principal Components Analysis
Call: principal(r = Thurstone, nfactors = 3, rotate = "Promax", n.obs = 213)
Standardized loadings (pattern matrix) based upon correlation matrix
RC1 RC2 RC3 h2 u2 com
Sentences 0.92 0.01 0.01 0.86 0.14 1.0
Vocabulary 0.90 0.10 -0.05 0.86 0.14 1.0
Sent.Completion 0.91 0.04 -0.04 0.83 0.17 1.0
First.Letters 0.01 0.84 0.07 0.78 0.22 1.0
Four.Letter.Words -0.05 0.81 0.17 0.75 0.25 1.1
Suffixes 0.18 0.79 -0.15 0.70 0.30 1.2
Letter.Series 0.03 -0.03 0.88 0.78 0.22 1.0
Pedigrees 0.45 -0.16 0.57 0.67 0.33 2.1
Letter.Group -0.19 0.19 0.86 0.75 0.25 1.2

RC1 RC2 RC3

SS loadings 2.83 2.19 1.96
Proportion Var 0.31 0.24 0.22
Cumulative Var 0.31 0.56 0.78
Proportion Explained 0.41 0.31 0.28
Cumulative Proportion 0.41 0.72 1.00

With component correlations of

RC1 RC2 RC3
RC1 1.00 0.51 0.53
RC2 0.51 1.00 0.44
RC3 0.53 0.44 1.00

Mean item complexity = 1.2

Test of the hypothesis that 3 components are sufficient.

The root mean square of the residuals (RMSR) is 0.06

with the empirical chi square 56.17 with prob < 1.1e-07

Fit based upon off diagonal values = 0.98

19
found using the omega function. This technique has more recently been applied to the personality
domain to consider such things as the structure of neuroticism (treated as a general factor, with
lower order factors of anxiety, depression, and aggression).
Consider the 9 Thurstone variables analyzed in the prior factor analyses. The correlations between
the factors (as shown in Figure 2 can themselves be factored. This results in a higher order factor
model (Figure 3). An an alternative solution is to take this higher order model and then solve for
the general factor loadings as well as the loadings on the residualized lower order factors using the
Schmid-Leiman procedure. (Figure 4). Yet another solution is to use structural equation modeling
to directly solve for the general and group factors.
> om.h <- omega(Thurstone,n.obs=213,sl=FALSE)
> op <- par(mfrow=c(1,1))
Omega

Sentences

0.9
Vocabulary
0.9
F1
0.8
Sent.Completion

0.8
First.Letters
0.9

Four.Letter.Words 0.7 F2 0.8 g

0.6 0.4
Suffixes
0.7

Letter.Series
0.2 0.8
F3
0.6
Letter.Group
0.5

Pedigrees

Figure 3: A higher order factor solution to the Thurstone 9 variable problem

Yet another approach to the bifactor structure is do use the bifactor rotation function in either

20
> om <- omega(Thurstone,n.obs=213)
Omega

Sentences

0.6
0.7 Vocabulary
0.6
F1*
0.7 0.5
Sent.Completion
0.7
First.Letters
0.6 0.6

g 0.6 Four.Letter.Words 0.5 F2*

0.6 0.4 0.2

Suffixes
0.6
Letter.Series
0.5 0.6
F3*
0.5
0.6 Letter.Group
0.3

Pedigrees

Figure 4: A bifactor factor solution to the Thurstone 9 variable problem

21
psych or in GPArotation. This does the rotation discussed in Jennrich and Bentler (2011).

4.1.7 Item Cluster Analysis: iclust

An alternative to factor or components analysis is cluster analysis. The goal of cluster analysis
is the same as factor or components analysis (reduce the complexity of the data and attempt to
identify homogeneous subgroupings). Mainly used for clustering people or objects (e.g., projectile
points if an anthropologist, DNA if a biologist, galaxies if an astronomer), clustering may be used
for clustering items or tests as well. Introduced to psychologists by Tryon (1939) in the 1930’s, the
cluster analytic literature exploded in the 1970s and 1980s (Blashfield, 1980; Blashfield and Alden-
derfer, 1988; Everitt, 1974; Hartigan, 1975). Much of the research is in taxonmetric applications
in biology (Sneath and Sokal, 1973; Sokal and Sneath, 1963) and marketing (Cooksey and Soutar,
2006) where clustering remains very popular. It is also used for taxonomic work in forming clus-
ters of people in family (Henry et al., 2005) and clinical psychology (Martinent and Ferrand, 2007;
Mun et al., 2008). Interestingly enough it has has had limited applications to psychometrics. This
is unfortunate, for as has been pointed out by e.g. (Tryon, 1935; Loevinger et al., 1953), the theory
of factors, while mathematically compelling, offers little that the geneticist or behaviorist or per-
haps even non-specialist finds compelling. Cooksey and Soutar (2006) reviews why the iclust
algorithm is particularly appropriate for scale construction in marketing.
Hierarchical cluster analysis forms clusters that are nested within clusters. The resulting tree
diagram (also known somewhat pretentiously as a rooted dendritic structure) shows the nesting
structure. Although there are many hierarchical clustering algorithms in R (e.g., agnes, hclust,
and iclust), the one most applicable to the problems of scale construction is iclust (Revelle,
1979).
1. Find the proximity (e.g. correlation) matrix,
2. Identify the most similar pair of items
3. Combine this most similar pair of items to form a new variable (cluster),
4. Find the similarity of this cluster to all other items and clusters,
5. Repeat steps 2 and 3 until some criterion is reached (e.g., typicallly, if only one cluster
remains or in iclust if there is a failure to increase reliability coefficients α or β ).
6. Purify the solution by reassigning items to the most similar cluster center.
iclust forms clusters of items using a hierarchical clustering algorithm until one of two measures
of internal consistency fails to increase (Revelle, 1979). The number of clusters may be specified
a priori, or found empirically. The resulting statistics include the average split half reliability, α
(Cronbach, 1951), as well as the worst split half reliability, β (Revelle, 1979), which is an estimate
of the general factor saturation of the resulting scale (Figure 5). Cluster loadings (corresponding to

22
the structure matrix of factor analysis) are reported when printing (Table 7). The pattern matrix is
available as an object in the results.
> data(bfi)
> ic <- iclust(bfi[1:25])
ICLUST

O5 0.57
0.57 C7
O2 0.32 C21
α = 0.41
O4 −0.42 β = 0.27
C5 0.69
0.69 C1
C4 0.68 C15
α = 0.73
C3 C14β = 0.67
−0.91
0.7 α = 0.63
C2 0.65 0.84 β = 0.58
0.65 C4
C1
N4 0.72
0.72 C8
N3 0.72 C13
α = 0.71
N5 0.75 β = 0.650.86 C16
α = 0.81
N2 0.84 0.76 β = 0.76
0.84 C5
N1
O3 0.63
0.63 C2
O1 0.73 C12
α = 0.68
E5 0.63 0.9 β = 0.64
0.63 C9 0.69
E3 C19
α = 0.76
E4 0.72 β = 0.64
−0.72 C6 0.69
E2 0.77 C11 0.63
α = 0.72
E1 −0.77 β = 0.69 C20
α = 0.81
A5 0.71 β = 0.63
0.71 C3
A3 0.77 C10
α = 0.72 0.69
A2 0.77 β = 0.68 0.67 C17
α = 0.72
A4 0.7 C18
0.61
β = 0.54
α = 0.71
A1 −0.61 β = 0.5

Figure 5: Using the iclust function to find the cluster structure of 25 personality items (the three
demographic variables were excluded from this analysis). When analyzing many variables, the tree
structure may be seen more clearly if the graphic output is saved as a pdf and then enlarged using
a pdf viewer.

The previous analysis (Figure 5) was done using the Pearson correlation. A somewhat cleaner
structure is obtained when using the polychoric function to find polychoric correlations (Fig-
ure 6). Note that the first time finding the polychoric correlations some time, but the next three
analyses were done using that correlation matrix (r.poly$rho). When using the console for input,
polychoric will report on its progress while working using progressBar.

23
Table 5: The summary statistics from an iclust analysis shows three large clusters and smaller
cluster.
> summary(ic) #show the results
ICLUST (Item Cluster Analysis)Call: iclust(r.mat = bfi[1:25])
ICLUST

Purified Alpha:
C20 C16 C15 C21
0.80 0.81 0.73 0.61

Guttman Lambda6*
C20 C16 C15 C21
0.82 0.81 0.72 0.61

Original Beta:
C20 C16 C15 C21
0.63 0.76 0.67 0.27

Cluster size:
C20 C16 C15 C21
10 5 5 5

Purified scale intercorrelations

reliabilities on diagonal
correlations corrected for attenuation above diagonal:
C20 C16 C15 C21
C20 0.80 -0.291 -0.40 -0.33
C16 -0.24 0.815 0.29 0.11
C15 -0.30 0.221 0.73 0.30
C21 -0.23 0.074 0.20 0.61

Table 6: The polychoric and the tetrachoric functions can take a long time to finish and
report their progress by a series of dots as they work. The dots are suppressed when creating a
Sweave document.
> data(bfi)
> r.poly <- polychoric(bfi[1:25],correct=0) #the ... indicate the progress of the function

1.7.1

24
> ic.poly <- iclust(r.poly$rho,title="ICLUST using polychoric correlations")
> iclust.diagram(ic.poly)
ICLUST using polychoric correlations

N2 0.87
0.87 C4
N1 0.7 C16
α = 0.84
N5 C13 0.96 β = 0.79
0.78 α = 0.74
N4 0.75 0.86 β = 0.69
0.75 C8
N3
O4 C21
0.52 α = 0.48
O5 0.63 −0.57 β = 0.35
0.63 C7 0.31
O2 C22
α = 0.81
A5 0.76 β = 0.29
0.76 C1
A3 0.8 C10
α = 0.77
A2 0.81 β = 0.73 0.69 C17 0.31
α = 0.77
A4 0.73 C18
0.65
β = 0.59
α = 0.76
A1 −0.65 0.63C20
β = 0.56
α = 0.83
E5 0.66 β = 0.66
0.66 C9 0.6
E3 0.77 C12
α = 0.72 0.72
O3 0.67 0.89 β = 0.68
0.67 C5 0.7
O1 C19
α = 0.8
E1 C11 −0.68 β = 0.66
0.8 α = 0.77
E4 0.76 −0.91 β = 0.73
−0.76 C6
E2 C23
α = 0.83
C3 C14 β = 0.58
0.72 α = 0.67
C2 0.7 0.84 β = 0.61 0.49
0.7 C2 0.8
C1 C15
α = 0.77
C5 0.73 −0.81 β = 0.71
0.73 C3
C4

Figure 6: ICLUST of the BFI data set using polychoric correlations. Compare this solution to the
previous one (Figure 5) which was done using Pearson correlations.

25
> ic.poly <- iclust(r.poly$rho,5,title="ICLUST using polychoric correlations for nclusters=5")
> iclust.diagram(ic.poly)
ICLUST using polychoric correlations for nclusters=5

O5 0.63
0.63 C7
O2
O4
C3 C14
0.72 α = 0.67
C2 0.7 0.84 β = 0.61
0.7 C2 0.8
C1 C15
α = 0.77
C5 0.73 −0.81 β = 0.71
0.73 C3
C4
N2 0.87
0.87 C4
N1 0.7 C16
α = 0.84
N5 C13 0.96 β = 0.79
0.78 α = 0.74
N4 0.75 0.86 β = 0.69
0.75 C8
N3
A5 0.76
0.76 C1
A3 0.8 C10
α = 0.77
A2 0.81 β = 0.73 0.69 C17
α = 0.77
A4 0.73 C18
0.65
β = 0.59
α = 0.76
A1 −0.65 0.63C20
β = 0.56
α = 0.83
E5 0.66 β = 0.66
0.66 C9
E3 0.77 C12
α = 0.72 0.72
O3 0.67 0.89 β = 0.68
0.67 C5 0.7
O1 C19
α = 0.8
E1 C11 −0.68 β = 0.66
0.8 α = 0.77
E4 0.76 −0.91 β = 0.73
−0.76 C6
E2

Figure 7: ICLUST of the BFI data set using polychoric correlations with the solution set to 5
clusters. Compare this solution to the previous one (Figure 6) which was done without specifying
the number of clusters and to the next one (Figure 8) which was done by changing the beta criterion.

26
> ic.poly <- iclust(r.poly$rho,beta.size=3,title="ICLUST beta.size=3")
ICLUST beta.size=3

N2 0.87
0.87 C4
N1 0.7 C16
α = 0.84
N5 C13 0.96 β = 0.79
0.78 α = 0.74
N4 0.75 0.86 β = 0.69
0.75 C8
N3
O4 C21
0.52 α = 0.48
O5 0.63 −0.57 β = 0.35
0.63 C7 0.31
O2 C22
α = 0.81
A5 0.76 β = 0.29
0.76 C1
A3 0.8 C10
α = 0.77
A2 0.81 β = 0.73 0.69 C17 0.31
α = 0.77
A4 0.73 C18
0.65
β = 0.59
α = 0.76
A1 −0.65 0.63C20
β = 0.56
α = 0.83
E5 0.66 β = 0.66
0.66 C9 0.6
E3 0.77 C12
α = 0.72 0.72
O3 0.67 0.89 β = 0.68
0.67 C5 0.7
O1 C19
α = 0.8
E1 C11 −0.68 β = 0.66
0.8 α = 0.77
E4 0.76 −0.91 β = 0.73
−0.76 C6
E2 C23
α = 0.83
C3 C14 β = 0.58
0.72 α = 0.67
C2 0.7 0.84 β = 0.61 0.49
0.7 C2 0.8
C1 C15
α = 0.77
C5 0.73 −0.81 β = 0.71
0.73 C3
C4

Figure 8: ICLUST of the BFI data set using polychoric correlations with the beta criterion set to 3.
Compare this solution to the previous three (Figure 5, 6, 7).

27
Table 7: The output from iclust includes the loadings of each item on each cluster. These
are equivalent to factor structure loadings. By specifying the value of cut, small loadings are
suppressed. The default is for cut=0.su
> print(ic,cut=.3)
ICLUST (Item Cluster Analysis)
Call: iclust(r.mat = bfi[1:25])

Purified Alpha:
C20 C16 C15 C21
0.80 0.81 0.73 0.61

G6* reliability:
C20 C16 C15 C21
0.83 1.00 0.50 0.38

Original Beta:
C20 C16 C15 C21
0.63 0.76 0.67 0.27

Cluster size:
C20 C16 C15 C21
10 5 5 5

Item by Cluster Structure matrix:

O P C20 C16 C15 C21
A1 C20 C20
A2 C20 C20 0.59
A3 C20 C20 0.65
A4 C20 C20 0.43
A5 C20 C20 0.65
C1 C15 C15 -0.54
C2 C15 C15 -0.62
C3 C15 C15 -0.54
C4 C15 C15 0.31 0.66
C5 C15 C15 -0.30 0.36 0.59
E1 C20 C20 -0.50
E2 C20 C20 -0.61 0.34
E3 C20 C20 0.59 -0.39
E4 C20 C20 0.66
E5 C20 C20 0.50 -0.40 -0.32
N1 C16 C16 0.76
N2 C16 C16 0.75
N3 C16 C16 0.74
N4 C16 C16 -0.34 0.62
N5 C16 C16 0.55
O1 C20 C21 -0.53
O2 C21 C21 0.44
O3 C20 C21 0.39 -0.62
O4 C21 C21 -0.33
O5 C21 C21 0.53

With Sums of squares of:

C20 C16 C15 C21
3.2 2.6 1.9 1.5

Purified scale intercorrelations

reliabilities on diagonal
correlations corrected for attenuation above diagonal:
C20 C16 C15 C21
C20 0.80 -0.29 -0.40 -0.33 28
C16 -0.24 0.81 0.29 0.11
C15 -0.30 0.22 0.73 0.30
C21 -0.23 0.07 0.20 0.61

Cluster fit = 0.68 Pattern fit = 0.96 RMSR = 0.05

NULL
A comparison of these four cluster solutions suggests both a problem and an advantage of clustering
techniques. The problem is that the solutions differ. The advantage is that the structure of the items
may be seen more clearly when examining the clusters rather than a simple factor solution.

4.2 Estimates of fit

Exploratory factoring techniques are sometimes criticized because of the lack of statistical infor-
mation on the solutions. There are perhaps as many fit statistics as there are psychometricians.
When using Maximum Likelihood extraction, many of these various fit statistics are based upon
the χ 2 which is minimized using ML. If not using ML, these same statistics can be found, but they
are no longer maximum likelihood estimates. They are, however, still useful. Overall estimates of
goodness of fit including χ 2 and RMSEA are found in the fa and omega functions.

4.3 Confidence intervals using bootstrapping techniques

Confidence intervals for the factor loadings may be found by doing multiple bootstrapped itera-
tions of the original analysis. This is done by setting the n.iter parameter to the desired number
of iterations. This can be done for factoring of Pearson correlation matrices as well as poly-
choric/tetrachoric matrices (See Table 8). Although the example value for the number of itera-
tions is set to 20, more conventional analyses might use 1000 bootstraps. This will take much
longer.
Bootstrapped confidence intervals can also be found for the loadings of a factoring of a polychoric
matrix. fa.poly will find the polychoric correlation matrix and if the n.iter option is greater than
1, will then randomly resample the data (case wise) to give bootstrapped samples. This will take a
long time for large number of items or interations.

4.4 Comparing factor/component/cluster solutions

Cluster analysis, factor analysis, and principal components analysis all produce structure matrices
(matrices of correlations between the dimensions and the variables) that can in turn be compared in
terms of Burt’s congruence coefficient (also known as Tucker’s coefficient) which is just the cosine
of the angle between the dimensions

∑nk=1 fik f jk
c fi f j = .
∑ fik2 ∑ f jk
2

Consider the case of a four factor solution and four cluster solution to the Big Five problem.
> f4 <- fa(bfi[1:25],4,fm="pa")
> factor.congruence(f4,ic)

C20 C16 C15 C21

PA1 0.92 -0.32 -0.44 -0.40
PA2 -0.26 0.95 0.33 0.12

29
Table 8: An example of bootstrapped confidence intervals on 10 items from the Big 5 inventory.
The number of bootstrapped samples was set to 20. More conventional bootstrapping would use
100 or 1000 replications.
> fa(bfi[1:10],2,n.iter=20)
Factor Analysis with confidence intervals using method = fa(r = bfi[1:10], nfactors = 2, n.iter = 20)
Factor Analysis using method = minres
Call: fa(r = bfi[1:10], nfactors = 2, n.iter = 20)
Standardized loadings (pattern matrix) based upon correlation matrix
MR1 MR2 h2 u2 com
A1 0.08 -0.41 0.15 0.85 1.1
A2 0.01 0.68 0.46 0.54 1.0
A3 -0.03 0.76 0.56 0.44 1.0
A4 0.14 0.44 0.25 0.75 1.2
A5 0.03 0.60 0.37 0.63 1.0
C1 0.57 -0.06 0.31 0.69 1.0
C2 0.64 -0.01 0.40 0.60 1.0
C3 0.54 0.03 0.31 0.69 1.0
C4 -0.65 0.00 0.42 0.58 1.0
C5 -0.56 -0.06 0.34 0.66 1.0

MR1 MR2
SS loadings 1.80 1.78
Proportion Var 0.18 0.18
Cumulative Var 0.18 0.36
Proportion Explained 0.50 0.50
Cumulative Proportion 0.50 1.00

With factor correlations of

MR1 MR2
MR1 1.00 0.32
MR2 0.32 1.00

Mean item complexity = 1

Test of the hypothesis that 2 factors are sufficient.

df null model = 45 with the objective function = 2.03 with Chi Square = 5664.89
df of the model are 26 and the objective function was 0.17

The root mean square of the residuals (RMSR) is 0.04

The df corrected root mean square of the residuals is 0.05

The harmonic n.obs is 2762 with the empirical chi square 397.07 with prob < 5e-68
The total n.obs was 2800 with Likelihood Chi Square = 468.37 with prob < 1.2e-82

Tucker Lewis Index of factoring reliability = 0.864

RMSEA index = 0.078 and the 90 % confidence intervals are 0.072 0.084
BIC = 261.99
Fit based upon off diagonal values = 0.98
Measures of factor score adequacy
MR1 MR2
Correlation of (regression) scores with factors 0.86 0.88
Multiple R square of scores with factors 0.74 0.77
Minimum correlation of possible factor scores 0.49 0.54

Coefficients and bootstrapped confidence intervals

low MR1 upper low MR2 upper
A1 0.04 0.08 0.12 -0.46 -0.41 -0.37
A2 -0.02 0.01 0.05 0.64 0.68 0.71
A3 -0.05 -0.03 -0.01 0.72 0.76 0.79
A4 0.10 0.14 0.18 0.41 0.44 0.47
A5 -0.01 0.03 0.07 0.56 0.60 0.64
C1 0.51 0.57 0.62 -0.09 -0.06 -0.03
C2 0.59 0.64 0.68 -0.06 -0.01 0.02
C3 0.51 0.54 0.60 0.00 0.03 0.06
C4 -0.69 -0.65 -0.61 -0.04 0.00 0.02
C5 -0.60 -0.56 -0.53 -0.10 -0.06 -0.03

Interfactor correlations and bootstrapped confidence intervals

lower estimate upper
MR1-MR2 0.25 0.32 0.38
>

30
PA3 0.35 -0.24 -0.88 -0.37
PA4 0.29 -0.12 -0.27 -0.90

A more complete comparison of oblique factor solutions (both minres and principal axis), bifactor
and component solutions to the Thurstone data set is done using the factor.congruence
function. (See table 9).

Table 9: Congruence coefficients for oblique factor, bifactor and component solutions for the Thur-
stone problem.
> factor.congruence(list(f3t,f3o,om,p3p))
MR1 MR2 MR3 PA1 PA2 PA3 g F1* F2* F3* h2 RC1 RC2 RC3
MR1 1.00 0.06 0.09 1.00 0.06 0.13 0.72 1.00 0.06 0.09 0.73 1.00 0.08 0.04
MR2 0.06 1.00 0.08 0.03 1.00 0.07 0.60 0.06 1.00 0.08 0.57 0.04 0.99 0.12
MR3 0.09 0.08 1.00 0.01 0.01 1.00 0.52 0.09 0.08 1.00 0.51 0.06 0.02 0.99
PA1 1.00 0.03 0.01 1.00 0.04 0.05 0.67 1.00 0.03 0.01 0.68 1.00 0.06 -0.04
PA2 0.06 1.00 0.01 0.04 1.00 0.00 0.57 0.06 1.00 0.01 0.54 0.04 0.99 0.05
PA3 0.13 0.07 1.00 0.05 0.00 1.00 0.54 0.13 0.07 1.00 0.54 0.10 0.01 0.99
g 0.72 0.60 0.52 0.67 0.57 0.54 1.00 0.72 0.60 0.52 0.99 0.69 0.58 0.50
F1* 1.00 0.06 0.09 1.00 0.06 0.13 0.72 1.00 0.06 0.09 0.73 1.00 0.08 0.04
F2* 0.06 1.00 0.08 0.03 1.00 0.07 0.60 0.06 1.00 0.08 0.57 0.04 0.99 0.12
F3* 0.09 0.08 1.00 0.01 0.01 1.00 0.52 0.09 0.08 1.00 0.51 0.06 0.02 0.99
h2 0.73 0.57 0.51 0.68 0.54 0.54 0.99 0.73 0.57 0.51 1.00 0.71 0.56 0.50
RC1 1.00 0.04 0.06 1.00 0.04 0.10 0.69 1.00 0.04 0.06 0.71 1.00 0.06 0.00
RC2 0.08 0.99 0.02 0.06 0.99 0.01 0.58 0.08 0.99 0.02 0.56 0.06 1.00 0.05
RC3 0.04 0.12 0.99 -0.04 0.05 0.99 0.50 0.04 0.12 0.99 0.50 0.00 0.05 1.00

4.5 Determining the number of dimensions to extract.

How many dimensions to use to represent a correlation matrix is an unsolved problem in psycho-
metrics. There are many solutions to this problem, none of which is uniformly the best. Henry
Kaiser once said that “a solution to the number-of factors problem in factor analysis is easy, that he
used to make up one every morning before breakfast. But the problem, of course is to find the so-
lution, or at least a solution that others will regard quite highly not as the best" Horn and Engstrom
(1979).
Techniques most commonly used include
1) Extracting factors until the chi square of the residual matrix is not significant.
2) Extracting factors until the change in chi square from factor n to factor n+1 is not signifi-
cant.
3) Extracting factors until the eigen values of the real data are less than the corresponding eigen val-
ues of a random data set of the same size (parallel analysis) fa.parallel (Horn, 1965).
4) Plotting the magnitude of the successive eigen values and applying the scree test (a sudden drop
in eigen values analogous to the change in slope seen when scrambling up the talus slope of a
mountain and approaching the rock face (Cattell, 1966).

31
5) Extracting factors as long as they are interpretable.
6) Using the Very Structure Criterion (vss) (Revelle and Rocklin, 1979).
7) Using Wayne Velicer’s Minimum Average Partial (MAP) criterion (Velicer, 1976).
8) Extracting principal components until the eigen value < 1.
Each of the procedures has its advantages and disadvantages. Using either the chi square test or the
change in square test is, of course, sensitive to the number of subjects and leads to the nonsensical
condition that if one wants to find many factors, one simply runs more subjects. Parallel analysis is
partially sensitive to sample size in that for large samples the eigen values of random factors will
all tend towards 1. The scree test is quite appealing but can lead to differences of interpretation
as to when the scree “breaks". Extracting interpretable factors means that the number of factors
reflects the investigators creativity more than the data. vss, while very simple to understand, will
not work very well if the data are very factorially complex. (Simulations suggests it will work fine
if the complexities of some of the items are no more than 2). The eigen value of 1 rule, although
the default for many programs, seems to be a rough way of dividing the number of variables by 3
and is probably the worst of all criteria.
An additional problem in determining the number of factors is what is considered a factor. Many
treatments of factor analysis assume that the residual correlation matrix after the factors of interest
are extracted is composed of just random error. An alternative concept is that the matrix is formed
from major factors of interest but that there are also numerous minor factors of no substantive
interest but that account for some of the shared covariance between variables. The presence of
such minor factors can lead one to extract too many factors and to reject solutions on statistical
grounds of misfit that are actually very good fits to the data. This problem is partially addressed
later in the discussion of simulating complex structures using sim.structure and of small
extraneous factors using the sim.minor function.

4.5.1 Very Simple Structure

The vss function compares the fit of a number of factor analyses with the loading matrix “simpli-
fied" by deleting all except the c greatest loadings per item, where c is a measure of factor com-
plexity Revelle and Rocklin (1979). Included in vss is the MAP criterion (Minimum Absolute
Partial correlation) of Velicer (1976).
Using the Very Simple Structure criterion for the bfi data suggests that 4 factors are optimal (Fig-
ure 9). However, the MAP criterion suggests that 5 is optimal.
> vss

Very Simple Structure of Very Simple Structure of a Big 5 inventory

Call: vss(x = bfi[1:25], title = "Very Simple Structure of a Big 5 inventory")
VSS complexity 1 achieves a maximimum of 0.58 with 4 factors
VSS complexity 2 achieves a maximimum of 0.74 with 5 factors

32
> vss <- vss(bfi[1:25],title="Very Simple Structure of a Big 5 inventory")

Very Simple Structure of a Big 5 inventory

1.0

4 4
0.8

4
3 4
4
3 3 3 3
2 2
Very Simple Structure Fit

3
2 2 2
2
2
0.6

1
1 1
1 1
1 1
1
0.4
0.2
0.0

1 2 3 4 5 6 7 8

Number of Factors

Figure 9: The Very Simple Structure criterion for the number of factors compares solutions for
various levels of item complexity and various numbers of factors. For the Big 5 Inventory, the
complexity 1 and 2 solutions both achieve their maxima at four factors. This is in contrast to
parallel analysis which suggests 6 and the MAP criterion which suggests 5.

33
The Velicer MAP achieves a minimum of 0.01 with 5 factors
BIC achieves a minimum of -513.09 with 8 factors
Sample Size adjusted BIC achieves a minimum of -106.39 with 8 factors

Statistics by number of factors

vss1 vss2 map dof chisq prob sqresid fit RMSEA BIC SABIC complex eChisq SRMR eCRMS eBIC
1 0.49 0.00 0.024 275 11863 0.0e+00 25.9 0.49 0.123 9680 10554 1.0 23725 0.119 0.124 21542
2 0.54 0.63 0.018 251 7362 0.0e+00 18.6 0.63 0.101 5370 6168 1.2 12173 0.085 0.093 10180
3 0.56 0.70 0.017 228 5096 0.0e+00 14.6 0.71 0.087 3286 4010 1.3 7019 0.065 0.074 5209
4 0.58 0.74 0.015 206 3422 0.0e+00 11.5 0.77 0.075 1787 2441 1.4 3606 0.046 0.056 1971
5 0.54 0.74 0.015 185 1809 4.3e-264 9.4 0.81 0.056 341 928 1.5 1412 0.029 0.037 -57
6 0.54 0.71 0.016 165 1032 1.8e-125 8.3 0.84 0.043 -277 247 1.8 649 0.020 0.027 -661
7 0.51 0.70 0.019 146 708 1.2e-74 7.9 0.85 0.037 -451 13 1.9 435 0.016 0.023 -724
8 0.51 0.66 0.022 128 503 7.1e-46 7.4 0.85 0.032 -513 -106 2.0 278 0.013 0.020 -738

4.5.2 Parallel Analysis

An alternative way to determine the number of factors is to compare the solution to random data
with the same properties as the real data set. If the input is a data matrix, the comparison includes
random samples from the real data, as well as normally distributed random data with the same
number of subjects and variables. For the BFI data, parallel analysis suggests that 6 factors might
be most appropriate (Figure 10). It is interesting to compare fa.parallel with the paran
from the paran package. This latter uses smcs to estimate communalities. Simulations of known
structures with a particular number of major factors but with the presence of trivial, minor (but not
zero) factors, show that using smcs will tend to lead to too many factors.
Experience with problems of various sizes suggests that parallel analysis is useful for less than
about 1,000 subjects, and that using the number of components greater than a random solution is
more robust than using the number of factors greater than random factors.
A more tedious problem in terms of computation is to do parallel analysis of polychoric correlation
matrices. This is done by fa.parallel.poly. By default the number of replications is 20.
This is appropriate when choosing the number of factors from dicthotomous or polytomous data
matrices.

4.6 Factor extension

Sometimes we are interested in the relationship of the factors in one space with the variables in a
different space. One solution is to find factors in both spaces separately and then find the structural
relationships between them. This is the technique of structural equation modeling in packages such
as sem or lavaan. An alternative is to use the concept of factor extension developed by (Dwyer,
1937). Consider the case of 16 variables created to represent one two dimensional space. If factors
are found from eight of these variables, they may then be extended to the additional eight variables
(See Figure 11).
Another way to examine the overlap between two sets is the use of set correlation found by lmCor
(discussed later).

34
> fa.parallel(bfi[1:25],main="Parallel Analysis of a Big 5 inventory")
Parallel analysis suggests that the number of factors = 6 and the number of components = 6

Parallel Analysis of a Big 5 inventory

eigenvalues of principal components and factor analysis

PC Actual Data
PC Simulated Data
PC Resampled Data
FA Actual Data
4

FA Simulated Data
FA Resampled Data
3
2
1
0

5 10 15 20 25

Factor/Component Number

Figure 10: Parallel analysis compares factor and principal components solutions to the real data as
well as resampled data. Although vss suggests 4 factors, MAP 5, parallel analysis suggests 6. One
more demonstration of Kaiser’s dictum.

35
> v16 <- sim.item(16)
> s <- c(1,3,5,7,9,11,13,15)
> f2 <- fa(v16[,s],2)
> fe <- fa.extension(cor(v16)[s,-s],f2)
> fa.diagram(f2,fe=fe)
Factor analysis and extension

V15 V14
−0.6 −0.6
V5 V16
0.6 −0.6

V13 −0.6 MR1 0.6 V6

0.6 0.6
V7 V8

V1 V2
0.6 0.6

V11 −0.6 MR2 −0.6 V10

0.6 0.6
V3 V4
−0.5 −0.6
V9 V12

Figure 11: Factor extension applies factors from one set (those on the left) to another set of variables
(those on the right). fa.extension is particularly useful when one wants to define the factors
with one set of variables and then apply those factors to another set. fa.diagram is used to show
the structure.

36
4.7 Exploratory Structural Equation Modeling (ESEM)
Generaizing the procedures of factor extension, we can do Exploratory Structural Equation Mod-
eling (ESEM). Traditional Exploratory Factor Analysis (EFA) examines how latent variables can
account for the correlations within a data set. All loadings and cross loadings are found and rota-
tion is done to some approximation of simple structure. Traditional Confirmatory Factor Analysis
(CFA) tests such models by fitting just a limited number of loadings and typically does not allow
any (or many) cross loadings. Structural Equation Modeling then applies two such measurement
models, one to a set of X variables, another to a set of Y variables, and then tries to estimate the
correlation between these two sets of latent variables. (Some SEM procedures estimate all the pa-
rameters from the same model, thus making the loadings in set Y affect those in set X.) It is possible
to do a similar, exploratory modeling (ESEM) by conducting two Exploratory Factor Analyses, one
in set X, one in set Y, and then finding the correlations of the X factors with the Y factors, as well as
the correlations of the Y variables with the X factors and the X variables with the Y factors.
Consider the simulated data set of three ability variables, two motivational variables, and three
outcome variables:
Call: sim.structural(fx = fx, Phi = Phi, fy = fy)

$model (Population correlation matrix)

V Q A nach Anx gpa Pre MA
V 1.00 0.72 0.54 0.00 0.00 0.38 0.32 0.25
Q 0.72 1.00 0.48 0.00 0.00 0.34 0.28 0.22
A 0.54 0.48 1.00 0.48 -0.42 0.50 0.42 0.34
nach 0.00 0.00 0.48 1.00 -0.56 0.34 0.28 0.22
Anx 0.00 0.00 -0.42 -0.56 1.00 -0.29 -0.24 -0.20
gpa 0.38 0.34 0.50 0.34 -0.29 1.00 0.30 0.24
Pre 0.32 0.28 0.42 0.28 -0.24 0.30 1.00 0.20
MA 0.25 0.22 0.34 0.22 -0.20 0.24 0.20 1.00

$reliability (population reliability)

V Q A nach Anx gpa Pre MA
0.81 0.64 0.72 0.64 0.49 0.36 0.25 0.16
We can fit this by using the esem function and then draw the solution (see Figure 12) using the
esem.diagram function (which is normally called automatically by esem.
Exploratory Structural Equation Modeling Analysis using method = minres
Call: esem(r = gre.gpa$model, varsX = 1:5, varsY = 6:8, nfX = 2, nfY = 1,
n.obs = 1000, plot = FALSE)

For the 'X' set:

37
 X1 X2
V 0.91 -0.06
Q 0.81 -0.05
A 0.53 0.57
nach -0.10 0.81
Anx 0.08 -0.71

For the 'Y' set:

Y1
gpa 0.6
Pre 0.5
MA 0.4

Correlations between the X and Y sets.

X1 X2 Y1
X1 1.00 0.19 0.68
X2 0.19 1.00 0.67
Y1 0.68 0.67 1.00

The degrees of freedom for the null model are 56 and the empirical chi square
The degrees of freedom for the model are 7 and the empirical chi square functio
with prob < 0.0027

The root mean square of the residuals (RMSR) is 0.02

The corrected root mean square of the residuals is 0.04
with the empirical chi square 21.83 with prob < 0.0027
The total number of observations was 1000 with fitted Chi Square = 2175.06 w

RMSEA index = 0.046 and the 90 % confidence intervals are 0.025 0.069
Empirical BIC = -26.53
ESABIC = -4.29
Fit based upon off diagonal values = 1
To see the item loadings for the X and Y sets combined, and the associated fa ou

5 Classical Test Theory and Reliability

Surprisingly, 113 years after Spearman (1904) introduced the concept of reliability to psycholo-
gists, there are still multiple approaches for measuring it. Although very popular, Cronbach’s α
(Cronbach, 1951) underestimates the reliability of a test and over estimates the first factor saturation
(Revelle and Zinbarg, 2009).

38
 Exploratory Structural Model

V
0.9 gpa
Q 0.8 X1
0.6
0.7
A Y1
0.6 0.7 0.5
Pre

nach 0.8 X2 0.4

−0.7
Anx MA

Figure 12: An example of a Exploratory Structure Equation Model.

39
α (Cronbach, 1951) is the same as Guttman’s λ 3 (Guttman, 1945) and may be found by

n  tr(~V )x  n Vx − tr(~Vx )
λ3 = 1− = =α
n−1 Vx n−1 Vx

Perhaps because it is so easy to calculate and is available in most commercial programs, alpha is
without doubt the most frequently reported measure of internal consistency reliability. Alpha is
the mean of all possible spit half reliabilities (corrected for test length). For a unifactorial test, it
is a reasonable estimate of the first factor saturation, although if the test has any microstructure
(i.e., if it is “lumpy") coefficients β (Revelle, 1979) (see iclust) and ωh (see omega) are more
appropriate estimates of the general factor saturation. ωt is a better estimate of the reliability of the
total test.
Guttman’s λ6 (G6) considers the amount of variance in each item that can be accounted for the lin-
ear regression of all of the other items (the squared multiple correlation or smc), or more precisely,
the variance of the errors, e2j , and is

∑ e2j 2 )
∑(1 − rsmc
λ6 = 1 − = 1− .
Vx Vx

The squared multiple correlation is a lower bound for the item communality and as the number of
items increases, becomes a better estimate.
G6 is also sensitive to lumpiness in the test and should not be taken as a measure of unifactorial
structure. For lumpy tests, it will be greater than alpha. For tests with equal item loadings, alpha
> G6, but if the loadings are unequal or if there is a general factor, G6 > alpha. G6 estimates item
reliability by the squared multiple correlation of the other items in a scale. A modification of G6,
G6*, takes as an estimate of an item reliability the smc with all the items in an inventory, including
those not keyed for a particular scale. This will lead to a better estimate of the reliable variance of
a particular item.
Alpha, G6 and G6* are positive functions of the number of items in a test as well as the average
intercorrelation of the items in the test. When calculated from the item variances and total test
variance, as is done here, raw alpha is sensitive to differences in the item variances. Standardized
alpha is based upon the correlations rather than the covariances.
More complete reliability analyses of a single scale can be done using the omega function which
finds ωh and ωt based upon a hierarchical factor analysis.
Alternative functions scoreItems and cluster.cor will also score multiple scales and report
more useful statistics. “Standardized" alpha is calculated from the inter-item correlations and will
differ from raw alpha.
Functions for examining the reliability of a single scale or a set of scales include:

40
alpha Internal consistency measures of reliability range from ωh to α to ωt . The alpha function
reports two estimates: Cronbach’s coefficient α and Guttman’s λ6 . Also reported are item -
whole correlations, α if an item is omitted, and item means and standard deviations.
guttman Eight alternative estimates of test reliability include the six discussed by Guttman (1945),
four discussed by ten Berge and Zergers (1978) (µ0 . . . µ3 ) as well as β (the worst split half,
Revelle, 1979), the glb (greatest lowest bound) discussed by Bentler and Woodward (1980),
and ωh andωt ((McDonald, 1999; Zinbarg et al., 2005).
omega Calculate McDonald’s omega estimates of general and total factor saturation. (Revelle and
Zinbarg (2009) compare these coefficients with real and artificial data sets.)
cluster.cor Given a n x c cluster definition matrix of -1s, 0s, and 1s (the keys) , and a n x n
correlation matrix, find the correlations of the composite clusters.
scoreItems Given a matrix or data.frame of k keys for m items (-1, 0, 1), and a matrix or data.frame
of items scores for m items and n people, find the sum scores or average scores for each per-
son and each scale. If the input is a square matrix, then it is assumed that correlations or
covariances were used, and the raw scores are not available. In addition, report Cronbach’s
alpha, coefficient G6*, the average r, the scale intercorrelations, and the item by scale cor-
relations (both raw and corrected for item overlap and scale reliability). Replace missing
values with the item median or mean if desired. Will adjust scores for reverse scored items.
score.multiple.choice Ability tests are typically multiple choice with one right answer. score.multiple.choice
takes a scoring key and a data matrix (or data.frame) and finds total or average number right
for each participant. Basic test statistics (alpha, average r, item means, item-whole correla-
tions) are also reported.
splitHalf Given a set of items, consider all (if n.items < 17) or 10,000 random splits of the item
into two sets. The correlation between these two split halfs is then adjusted by the Spearman-
Brown prophecy formula to show the range of split half reliablities.

5.1 Reliability of a single scale

A conventional (but non-optimal) estimate of the internal consistency reliability of a test is coef-
ficient α (Cronbach, 1951). Alternative estimates are Guttman’s λ6 , Revelle’s β , McDonald’s ωh
and ωt . Consider a simulated data set, representing 9 items with a hierarchical structure and the
following correlation matrix. Then using the alpha function, the α and λ6 estimates of reliability
may be found for all 9 items, as well as the if one item is dropped at a time.
> set.seed(17)
> r9 <- sim.hierarchical(n=500,raw=TRUE)$observed
> round(cor(r9),2)

V1 V2 V3 V4 V5 V6 V7 V8 V9
V1 1.00 0.58 0.47 0.44 0.36 0.31 0.30 0.17 0.13
V2 0.58 1.00 0.36 0.39 0.26 0.26 0.30 0.24 0.15

41
V3 0.47 0.36 1.00 0.32 0.34 0.26 0.28 0.17 0.15
V4 0.44 0.39 0.32 1.00 0.45 0.41 0.31 0.17 0.16
V5 0.36 0.26 0.34 0.45 1.00 0.34 0.22 0.13 0.21
V6 0.31 0.26 0.26 0.41 0.34 1.00 0.17 0.10 0.12
V7 0.30 0.30 0.28 0.31 0.22 0.17 1.00 0.39 0.30
V8 0.17 0.24 0.17 0.17 0.13 0.10 0.39 1.00 0.22
V9 0.13 0.15 0.15 0.16 0.21 0.12 0.30 0.22 1.00

> alpha(r9)

Reliability analysis
Call: alpha(x = r9)

raw_alpha std.alpha G6(smc) average_r S/N ase mean sd median_r

0.77 0.77 0.78 0.28 3.4 0.015 -0.013 0.59 0.27

95% confidence boundaries

lower alpha upper
Feldt 0.74 0.77 0.8
Duhachek 0.75 0.77 0.8

Reliability if an item is dropped:

raw_alpha std.alpha G6(smc) average_r S/N alpha se var.r med.r
V1 0.73 0.73 0.73 0.26 2.8 0.018 0.0094 0.26
V2 0.74 0.74 0.74 0.26 2.9 0.017 0.0121 0.27
V3 0.75 0.75 0.75 0.27 3.0 0.017 0.0142 0.26
V4 0.74 0.74 0.74 0.26 2.8 0.018 0.0124 0.26
V5 0.75 0.75 0.75 0.27 3.0 0.017 0.0143 0.27
V6 0.76 0.76 0.76 0.28 3.2 0.016 0.0137 0.29
V7 0.75 0.75 0.75 0.27 3.0 0.017 0.0160 0.26
V8 0.77 0.77 0.77 0.30 3.4 0.015 0.0128 0.30
V9 0.78 0.78 0.78 0.30 3.5 0.015 0.0127 0.30

Item statistics
n raw.r std.r r.cor r.drop mean sd
V1 500 0.70 0.70 0.68 0.58 0.0247 0.98
V2 500 0.66 0.66 0.62 0.53 -0.0022 0.98
V3 500 0.62 0.62 0.55 0.49 -0.1027 0.93
V4 500 0.69 0.68 0.64 0.56 -0.0661 1.05
V5 500 0.61 0.62 0.55 0.48 -0.0622 0.98
V6 500 0.55 0.55 0.46 0.40 -0.0163 0.99
V7 500 0.61 0.61 0.54 0.47 0.0706 1.01
V8 500 0.49 0.48 0.37 0.32 0.0134 1.02
V9 500 0.45 0.45 0.33 0.29 0.0247 0.96

Some scales have items that need to be reversed before being scored. Rather than reversing the
items in the raw data, it is more convenient to just specify which items need to be reversed scored.
This may be done in alpha by specifying a keys vector of 1s and -1s. (This concept of keys
vector is more useful when scoring multiple scale inventories, see below.) As an example, consider
scoring the 7 attitude items in the attitude data set. Assume a conceptual mistake in that items 2
and 6 (complaints and critical) are to be scored (incorrectly) negatively.
> alpha(attitude,keys=c("complaints","critical"))

Reliability analysis
Call: alpha(x = attitude, keys = c("complaints", "critical"))

42
 raw_alpha std.alpha G6(smc) average_r S/N ase mean sd median_r
0.18 0.27 0.66 0.05 0.37 0.19 53 4.7 0.16

95% confidence boundaries

lower alpha upper
Feldt -0.36 0.18 0.56
Duhachek -0.18 0.18 0.55

Reliability if an item is dropped:

raw_alpha std.alpha G6(smc) average_r S/N alpha se var.r med.r
rating -0.023 0.090 0.52 0.0162 0.099 0.265 0.22 -0.12
complaints- 0.689 0.666 0.76 0.2496 1.995 0.078 0.13 0.43
privileges -0.133 0.021 0.58 0.0036 0.022 0.282 0.26 -0.12
learning -0.459 -0.251 0.41 -0.0346 -0.201 0.363 0.21 -0.15
raises -0.178 -0.062 0.47 -0.0098 -0.058 0.295 0.20 -0.12
critical- 0.364 0.473 0.76 0.1299 0.896 0.139 0.29 0.43
advance -0.137 -0.016 0.52 -0.0026 -0.016 0.258 0.27 -0.12

Item statistics
n raw.r std.r r.cor r.drop mean sd
rating 30 0.600 0.601 0.63 0.28 65 12.2
complaints- 30 -0.542 -0.559 -0.78 -0.75 50 13.3
privileges 30 0.679 0.663 0.57 0.39 53 12.2
learning 30 0.857 0.853 0.90 0.70 56 11.7
raises 30 0.719 0.730 0.77 0.50 65 10.4
critical- 30 0.015 0.036 -0.29 -0.27 42 9.9
advance 30 0.688 0.694 0.66 0.46 43 10.3

>

Note how the reliability of the 7 item scales with an incorrectly reversed item is very poor, but if
items 2 and 6 is dropped then the reliability is improved substantially. This suggests that items 2
and 6 were incorrectly scored. Doing the analysis again with the items positively scored produces
much more favorable results.
> alpha(attitude)

Reliability analysis
Call: alpha(x = attitude)

raw_alpha std.alpha G6(smc) average_r S/N ase mean sd median_r

0.84 0.84 0.88 0.43 5.2 0.042 60 8.2 0.45

95% confidence boundaries

lower alpha upper
Feldt 0.74 0.84 0.92
Duhachek 0.76 0.84 0.93

Reliability if an item is dropped:

raw_alpha std.alpha G6(smc) average_r S/N alpha se var.r med.r
rating 0.81 0.81 0.83 0.41 4.2 0.052 0.035 0.45
complaints 0.80 0.80 0.82 0.39 3.9 0.057 0.035 0.43
privileges 0.83 0.82 0.87 0.44 4.7 0.048 0.054 0.53
learning 0.80 0.80 0.84 0.40 4.0 0.054 0.045 0.38
raises 0.80 0.78 0.83 0.38 3.6 0.056 0.048 0.34
critical 0.86 0.86 0.89 0.51 6.3 0.038 0.030 0.56
advance 0.84 0.83 0.86 0.46 5.0 0.043 0.048 0.49

43
 Item statistics
n raw.r std.r r.cor r.drop mean sd
rating 30 0.78 0.76 0.75 0.67 65 12.2
complaints 30 0.84 0.81 0.82 0.74 67 13.3
privileges 30 0.70 0.68 0.60 0.56 53 12.2
learning 30 0.81 0.80 0.78 0.71 56 11.7
raises 30 0.85 0.86 0.85 0.79 65 10.4
critical 30 0.42 0.45 0.31 0.27 75 9.9
advance 30 0.60 0.62 0.56 0.46 43 10.3

It is useful when considering items for a potential scale to examine the item distribution. This is
done in scoreItems as well as in alpha.
> items <- sim.congeneric(N=500,short=FALSE,low=-2,high=2,categorical=TRUE) #500 responses to 4 discrete item
> alpha(items$observed) #item response analysis of congeneric measures

Reliability analysis
Call: alpha(x = items$observed)

raw_alpha std.alpha G6(smc) average_r S/N ase mean sd median_r

0.7 0.7 0.66 0.37 2.4 0.022 0.079 0.74 0.38

95% confidence boundaries

lower alpha upper
Feldt 0.66 0.7 0.74
Duhachek 0.66 0.7 0.75

Reliability if an item is dropped:

raw_alpha std.alpha G6(smc) average_r S/N alpha se var.r med.r
V1 0.57 0.57 0.48 0.31 1.3 0.033 0.0067 0.30
V2 0.60 0.60 0.51 0.33 1.5 0.031 0.0075 0.36
V3 0.67 0.67 0.60 0.40 2.0 0.026 0.0167 0.36
V4 0.71 0.71 0.63 0.45 2.4 0.023 0.0077 0.40

Item statistics
n raw.r std.r r.cor r.drop mean sd
V1 500 0.79 0.79 0.71 0.59 0.066 1.02
V2 500 0.77 0.77 0.68 0.56 0.098 1.01
V3 500 0.69 0.70 0.52 0.44 0.022 0.98
V4 500 0.66 0.65 0.44 0.38 0.130 1.06

Non missing response frequency for each item

-2 -1 0 1 2 miss
V1 0.07 0.22 0.36 0.28 0.07 0
V2 0.06 0.21 0.39 0.27 0.08 0
V3 0.06 0.23 0.40 0.25 0.06 0
V4 0.07 0.21 0.33 0.29 0.09 0

5.2 Using omega to find the reliability of a single scale

Two alternative estimates of reliability that take into account the hierarchical structure of the inven-
tory are McDonald’s ωh and ωt . These may be found using the omega function for an exploratory
analysis (See Figure 13) or omegaSem for a confirmatory analysis using the lavaan package based
upon the exploratory solution from omega.

44
McDonald has proposed coefficient omega (hierarchical) (ωh ) as an estimate of the general factor
saturation of a test. Zinbarg et al. (2005) https://personality-project.org/revelle/
publications/zinbarg.revelle.pmet.05.pdf compare McDonald’s ωh to Cronbach’s
α and Revelle’s β . They conclude that ωh is the best estimate. (See also Zinbarg et al. (2006) and
Revelle and Zinbarg (2009) https://personality-project.org/revelle/publications/
revelle.zinbarg.08.pdf ).
One way to find ωh is to do a factor analysis of the original data set, rotate the factors obliquely,
factor that correlation matrix, do a Schmid-Leiman (schmid) transformation to find general factor
loadings, and then find ωh .
ωh differs slightly as a function of how the factors are estimated. Four options are available, the
default will do a minimum residual factor analysis, fm=“pa" does a principal axes factor analysis
(factor.pa), fm=“mle" uses the factanal function, and fm=“pc" does a principal components
analysis (principal).
For ability items, it is typically the case that all items will have positive loadings on the general
factor. However, for non-cognitive items it is frequently the case that some items are to be scored
positively, and some negatively. Although probably better to specify which directions the items
are to be scored by specifying a key vector, if flip =TRUE (the default), items will be reversed so
that they have positive loadings on the general factor. The keys are reported so that scores can be
found using the scoreItems function. Arbitrarily reversing items this way can overestimate the
general factor. (See the example with a simulated circumplex).
β , an alternative to ω , is defined as the worst split half reliability. It can be estimated by us-
ing iclust (Item Cluster analysis: a hierarchical clustering algorithm). For a very complimen-
tary review of why the iclust algorithm is useful in scale construction, see Cooksey and Soutar
(2006).
The omega function uses exploratory factor analysis to estimate the ωh coefficient. It is important
to remember that “A recommendation that should be heeded, regardless of the method chosen to
estimate ωh , is to always examine the pattern of the estimated general factor loadings prior to esti-
mating ωh . Such an examination constitutes an informal test of the assumption that there is a latent
variable common to all of the scale’s indicators that can be conducted even in the context of EFA. If
the loadings were salient for only a relatively small subset of the indicators, this would suggest that
there is no true general factor underlying the covariance matrix. Just such an informal assumption
test would have afforded a great deal of protection against the possibility of misinterpreting the
misleading ωh estimates occasionally produced in the simulations reported here." (Zinbarg et al.,
2006, p 137).
Although ωh is uniquely defined only for cases where 3 or more subfactors are extracted, it is
sometimes desired to have a two factor solution. By default this is done by forcing the schmid
extraction to treat the two subfactors as having equal loadings.

45
There are three possible options for this condition: setting the general factor loadings between the
√
two lower order factors to be “equal" which will be the rab where rab is the oblique correlation
between the factors) or to “first" or “second" in which case the general factor is equated with either
the first or second group factor. A message is issued suggesting that the model is not really well
defined. This solution discussed in Zinbarg et al., 2007. To do this in omega, add the option=“first"
or option=“second" to the call.
Although obviously not meaningful for a 1 factor solution, it is of course possible to find the sum
of the loadings on the first (and only) factor, square them, and compare them to the overall matrix
variance. This is done, with appropriate complaints.
In addition to ωh , another of McDonald’s coefficients is ωt . This is an estimate of the total reliability
of a test.
McDonald’s ωt , which is similar to Guttman’s λ6 , (see guttman) uses the estimates of uniqueness
u2 from factor analysis to find e2j . This is based on a decomposition of the variance of a test score,
Vx into four parts: that due to a general factor, ~g, that due to a set of group factors, ~f , (factors
common to some but not all of the items), specific factors, ~s unique to each item, and ~e, random
error. (Because specific variance can not be distinguished from random error unless the test is given
at least twice, some combine these both into error).
Letting ~x = c~g + A~ f + Ds~ +~e then the communality of item j , based upon general as well as group
factors, h j = c j + ∑ fi j and the unique variance for the item u2j = σ 2j (1−h2j ) may be used to estimate
2 2 2

the test reliability. That is, if h2j is the communality of item j , based upon general as well as group
factors, then for standardized items, e2j = 1 − h2j and

~ 0~1 +~1AA
~1cc ~ 0~10 ∑(1 − h2j ) ∑ u2
ωt = = 1− = 1−
Vx Vx Vx

Because h2j ≥ rsmc

2 ,ω ≥λ .
t 6

It is important to distinguish here between the two ω coefficients of McDonald, 1978 and Equa-
tion 6.20a of McDonald, 1999, ωt and ωh . While the former is based upon the sum of squared
loadings on all the factors, the latter is based upon the sum of the squared loadings on the general
factor.
~ 0~1
~1cc
ωh =
Vx

Another estimate reported is the omega for an infinite length test with a structure similar to the
observed test. This is found by
~ 0~1
~1cc
ωinf =
~ 0~1 +~1AA
~1cc ~ 0~10

46
> om.9 <- omega(r9,title="9 simulated variables")
9 simulated variables

V1

0.5
0.7 V2
0.4
F1*
0.6 0.2
V3
0.5
V5
0.5 0.4

g 0.6 V4 0.4 F2*

0.4 0.3
V6
0.4
V7
0.3 0.6
F3*
0.5
0.3 V8
0.3

V9

Figure 13: A bifactor solution for 9 simulated variables with a hierarchical structure.

47
In the case of these simulated 9 variables, the amount of variance attributable to a general factor
(ωh ) is quite large, and the reliability of the set of 9 items is somewhat greater than that estimated
by α or λ6 .
> om.9

9 simulated variables
Call: omegah(m = m, nfactors = nfactors, fm = fm, key = key, flip = flip,
digits = digits, title = title, sl = sl, labels = labels,
plot = plot, n.obs = n.obs, rotate = rotate, Phi = Phi, option = option,
covar = covar)
Alpha: 0.77
G.6: 0.78
Omega Hierarchical: 0.64
Omega H asymptotic: 0.78
Omega Total 0.82

Schmid Leiman Factor loadings greater than 0.2

g F1* F2* F3* h2 u2 p2
V1 0.69 0.54 0.78 0.22 0.62
V2 0.56 0.37 0.46 0.54 0.68
V3 0.50 0.21 0.32 0.68 0.79
V4 0.60 0.36 0.50 0.50 0.73
V5 0.52 0.39 0.43 0.57 0.64
V6 0.45 0.32 0.31 0.69 0.65
V7 0.44 0.56 0.51 0.49 0.37
V8 0.28 0.48 0.31 0.69 0.26
V9 0.25 0.33 0.19 0.81 0.35

With Sums of squares of:

g F1* F2* F3*
2.21 0.49 0.42 0.68

general/max 3.26 max/min = 1.63

mean percent general = 0.57 with sd = 0.19 and cv of 0.33
Explained Common Variance of the general factor = 0.58

The degrees of freedom are 12 and the fit is 0.03

The number of observations was 500 with Chi Square = 16.31 with prob < 0.18
The root mean square of the residuals is 0.02
The df corrected root mean square of the residuals is 0.03
RMSEA index = 0.027 and the 10 % confidence intervals are 0 0.056
BIC = -58.26

Compare this with the adequacy of just a general factor and no group factors
The degrees of freedom for just the general factor are 27 and the fit is 0.37
The number of observations was 500 with Chi Square = 184.27 with prob < 2.3e-25
The root mean square of the residuals is 0.09
The df corrected root mean square of the residuals is 0.1

RMSEA index = 0.108 and the 10 % confidence intervals are 0.094 0.123
BIC = 16.48

Measures of factor score adequacy

g F1* F2* F3*
Correlation of scores with factors 0.83 0.64 0.55 0.70
Multiple R square of scores with factors 0.69 0.41 0.30 0.49
Minimum correlation of factor score estimates 0.38 -0.18 -0.40 -0.02

48
 Total, General and Subset omega for each subset
g F1* F2* F3*
Omega total for total scores and subscales 0.82 0.74 0.67 0.59
Omega general for total scores and subscales 0.64 0.53 0.46 0.20
Omega group for total scores and subscales 0.15 0.22 0.22 0.39

5.3 Estimating ωh using Confirmatory Factor Analysis

The omegaSem function will do an exploratory analysis and then take the highest loading items
on each factor and do a confirmatory factor analysis using the sem package. These results can
produce slightly different estimates of ωh , primarily because cross loadings are modeled as part of
the general factor.
> omegaSem(r9,n.obs=500,lavaan=TRUE)

Call: omegaSem(m = r9, n.obs = 500, lavaan = TRUE)

Omega
Call: omegah(m = m, nfactors = nfactors, fm = fm, key = key, flip = flip,
digits = digits, title = title, sl = sl, labels = labels,
plot = plot, n.obs = n.obs, rotate = rotate, Phi = Phi, option = option,
covar = covar)
Alpha: 0.77
G.6: 0.78
Omega Hierarchical: 0.64
Omega H asymptotic: 0.78
Omega Total 0.82

Schmid Leiman Factor loadings greater than 0.2

g F1* F2* F3* h2 u2 p2
V1 0.69 0.54 0.78 0.22 0.62
V2 0.56 0.37 0.46 0.54 0.68
V3 0.50 0.21 0.32 0.68 0.79
V4 0.60 0.36 0.50 0.50 0.73
V5 0.52 0.39 0.43 0.57 0.64
V6 0.45 0.32 0.31 0.69 0.65
V7 0.44 0.56 0.51 0.49 0.37
V8 0.28 0.48 0.31 0.69 0.26
V9 0.25 0.33 0.19 0.81 0.35

With Sums of squares of:

g F1* F2* F3*
2.21 0.49 0.42 0.68

general/max 3.26 max/min = 1.63

mean percent general = 0.57 with sd = 0.19 and cv of 0.33
Explained Common Variance of the general factor = 0.58

The degrees of freedom are 12 and the fit is 0.03

The number of observations was 500 with Chi Square = 16.31 with prob < 0.18
The root mean square of the residuals is 0.02
The df corrected root mean square of the residuals is 0.03
RMSEA index = 0.027 and the 10 % confidence intervals are 0 0.056
BIC = -58.26

Compare this with the adequacy of just a general factor and no group factors

49
The degrees of freedom for just the general factor are 27 and the fit is 0.37
The number of observations was 500 with Chi Square = 184.27 with prob < 2.3e-25
The root mean square of the residuals is 0.09
The df corrected root mean square of the residuals is 0.1

RMSEA index = 0.108 and the 10 % confidence intervals are 0.094 0.123
BIC = 16.48

Measures of factor score adequacy

g F1* F2* F3*
Correlation of scores with factors 0.83 0.64 0.55 0.70
Multiple R square of scores with factors 0.69 0.41 0.30 0.49
Minimum correlation of factor score estimates 0.38 -0.18 -0.40 -0.02

Total, General and Subset omega for each subset

g F1* F2* F3*
Omega total for total scores and subscales 0.82 0.74 0.67 0.59
Omega general for total scores and subscales 0.64 0.53 0.46 0.20
Omega group for total scores and subscales 0.15 0.22 0.22 0.39

The following analyses were done using the lavaan package

Omega Hierarchical from a confirmatory model using sem = 0.68

Omega Total from a confirmatory model using sem = 0.86
With loadings of
g F1* F2* F3* h2 u2 p2
V1 0.68 1.23 1.98 -0.98 0.23
V2 0.61 0.39 0.61 0.95
V3 0.57 0.33 0.67 0.98
V4 0.62 0.40 0.55 0.45 0.70
V5 0.51 0.33 0.37 0.63 0.70
V6 0.43 0.35 0.31 0.69 0.60
V7 0.46 0.56 0.53 0.47 0.40
V8 0.29 0.45 0.29 0.71 0.29
V9 0.25 0.32 0.17 0.83 0.37

With sum of squared loadings of:

g F1* F2* F3*
2.37 1.53 0.40 0.63

The degrees of freedom of the confirmatory model are 18 and the fit is 25.42543 with p = 0.1136393
general/max 1.55 max/min = 3.86
mean percent general = 0.58 with sd = 0.28 and cv of 0.48
Explained Common Variance of the general factor = 0.48

Measures of factor score adequacy

g F1* F2* F3*
Correlation of scores with factors 0.86 1.59 0.57 0.70
Multiple R square of scores with factors 0.73 2.52 0.32 0.48
Minimum correlation of factor score estimates 0.46 4.04 -0.36 -0.03

Total, General and Subset omega for each subset

g F1* F2* F3*
Omega total for total scores and subscales 0.86 0.95 0.67 0.58
Omega general for total scores and subscales 0.68 0.60 0.45 0.21
Omega group for total scores and subscales 0.17 0.35 0.22 0.37

To get the standard sem fit statistics, ask for summary on the fitted object

50
5.3.1 Other estimates of reliability
Other estimates of reliability are found by the splitHalf and guttman functions. These are
described in more detail in Revelle and Zinbarg (2009) and in Revelle and Condon (2018). They
include the 6 estimates from Guttman, four from TenBerge, and an estimate of the greatest lower
bound.
> splitHalf(r9)

Split half reliabilities

Call: splitHalf(r = r9)

Maximum split half reliability (lambda 4) = 0.82

Guttman lambda 6 = 0.78
Average split half reliability = 0.77
Guttman lambda 3 (alpha) = 0.77
Guttman lambda 2 = 0.78
Minimum split half reliability (beta) = 0.7
Average interitem r = 0.28 with median = 0.27

5.4 Reliability and correlations of multiple scales within an inventory

A typical research question in personality involves an inventory of multiple items purporting to
measure multiple constructs. For example, the data set bfi includes 25 items thought to measure
five dimensions of personality (Extraversion, Emotional Stability, Conscientiousness, Agreeable-
ness, and Openness). The data may either be the raw data or a correlation matrix (scoreItems)
or just a correlation matrix of the items ( cluster.cor and cluster.loadings). When
finding reliabilities for multiple scales, item reliabilities can be estimated using the squared multi-
ple correlation of an item with all other items, not just those that are keyed for a particular scale.
This leads to an estimate of G6*.

5.4.1 Scoring from raw data

To score these five scales from the 25 items, use the scoreItems function and a list of items to
be scored on each scale (a keys.list). Items may be listed by location (convenient but dangerous),
or name (probably safer).
Make a keys.list by by specifying the items for each scale, preceding items to be negatively keyed
with a - sign:
> #the newer way is probably preferred
>
> keys.list <- list(agree=c("-A1","A2","A3","A4","A5"),
+ conscientious=c("C1","C2","C2","-C4","-C5"),
+ extraversion=c("-E1","-E2","E3","E4","E5"),
+ neuroticism=c("N1","N2","N3","N4","N5"),
+ openness = c("O1","-O2","O3","O4","-O5"))
> #this can also be done by location--
> keys.list <- list(Agree=c(-1,2:5),Conscientious=c(6:8,-9,-10),
+ Extraversion=c(-11,-12,13:15),Neuroticism=c(16:20),
+ Openness = c(21,-22,23,24,-25))

51
> #These two approaches can be mixed if desired
> keys.list <- list(agree=c("-A1","A2","A3","A4","A5"),conscientious=c("C1","C2","C3","-C4","-C5"),
+ extraversion=c("-E1","-E2","E3","E4","E5"),
+ neuroticism=c(16:20),openness = c(21,-22,23,24,-25))
> keys.list

$agree
[1] "-A1" "A2" "A3" "A4" "A5"

$conscientious
[1] "C1" "C2" "C3" "-C4" "-C5"

$extraversion
[1] "-E1" "-E2" "E3" "E4" "E5"

$neuroticism
[1] 16 17 18 19 20

$openness
[1] 21 -22 23 24 -25
In the past (prior to version 1.6.9, the keys.list was then converted a keys matrix using the helper function make.keys. This is no longer necessary. Logically, scales are merely
the weighted composites of a set of items. The weights used are -1, 0, and 1. 0 implies do not use that item in the scale, 1 implies a positive weight (add the item to the total score),
-1 a negative weight (subtract the item from the total score, i.e., reverse score the item). Reverse scoring an item is equivalent to subtracting the item from the maximum + minimum
possible value for that item. The minima and maxima can be estimated from all the items, or can be specified by the user.

There are two different ways that scale scores tend to be reported. Social psychologists and educational psychologists tend to report the scale score as the average item score while
many personality psychologists tend to report the total item score. The default option for scoreItems is to report item averages (which thus allows interpretation in the same
metric as the items) but totals can be found as well. Personality researchers should be encouraged to report scores based upon item means and avoid using the total score although
some reviewers are adamant about the following the tradition of total scores.

The printed output includes coefficients α and G6*, the average correlation of the items within the scale (corrected for item ovelap and scale relliability), as well as the correlations
between the scales (below the diagonal, the correlations above the diagonal are corrected for attenuation. As is the case for most of the psych functions, additional information is
returned as part of the object.

First, create keys matrix using the make.keys function. (The keys matrix could also be prepared externally using a spreadsheet and then copying it into R). Although not normally
necessary, show the keys to understand what is happening. There are two ways to make up the keys. You can specify the items by location (the old way) or by name (the newer and
probably preferred way). To use the newer way you must specify the file on which you will use the keys. The example below shows how to construct keys either way.

Note that the number of items to specify in the make.keys function is the total number of items in the inventory. This is done automatically in the new way of forming keys, but
if using the older way, the number must be specified. That is, if scoring just 5 items from a 25 item inventory, make.keys should be told that there are 25 items. make.keys
just changes a list of items on each scale to make up a scoring matrix. Because the bfi data set has 25 items as well as 3 demographic items, the number of variables is specified
as 28.

Then, use this keys list to score the items.

> scores <- scoreItems(keys.list,bfi)
> scores

Call: scoreItems(keys = keys.list, items = bfi)

(Unstandardized) Alpha:
agree conscientious extraversion neuroticism openness
alpha 0.7 0.72 0.76 0.81 0.6

Standard errors of unstandardized Alpha:

agree conscientious extraversion neuroticism openness
ASE 0.014 0.014 0.013 0.011 0.017

Average item correlation:

agree conscientious extraversion neuroticism openness
average.r 0.32 0.34 0.39 0.46 0.23

Median item correlation:

agree conscientious extraversion neuroticism openness

52
 0.34 0.34 0.38 0.41 0.22

Guttman 6* reliability:
agree conscientious extraversion neuroticism openness
Lambda.6 0.7 0.72 0.76 0.81 0.6

Signal/Noise based upon av.r :

agree conscientious extraversion neuroticism openness
Signal/Noise 2.3 2.6 3.2 4.3 1.5

Scale intercorrelations corrected for attenuation

raw correlations below the diagonal, alpha on the diagonal
corrected correlations above the diagonal:
agree conscientious extraversion neuroticism openness
agree 0.70 0.36 0.63 -0.245 0.23
conscientious 0.26 0.72 0.35 -0.305 0.30
extraversion 0.46 0.26 0.76 -0.284 0.32
neuroticism -0.18 -0.23 -0.22 0.812 -0.12
openness 0.15 0.19 0.22 -0.086 0.60

Average adjusted correlations within and between scales (MIMS)

agree cnscn extrv nrtcs opnns
agree 0.32
conscientious 0.22 0.34
extraversion 0.44 0.26 0.39
neuroticism -0.20 -0.26 -0.28 0.46
openness 0.11 0.15 0.18 -0.08 0.23

Average adjusted item x scale correlations within and between scales (MIMT)
agree cnscn extrv nrtcs opnns
agree 0.68
conscientious 0.18 0.69
extraversion 0.33 0.19 0.71
neuroticism -0.14 -0.18 -0.17 0.76
openness 0.10 0.12 0.14 -0.05 0.62

In order to see the item by scale loadings and frequency counts of the data
print with the short option = FALSE

To see the additional information (the raw correlations, the individual scores, etc.), they may be
specified by name. Then, to visualize the correlations between the raw scores, use the pairs.panels
function on the scores values of scores. (See figure 14

5.4.2 Forming scales from a correlation matrix

There are some situations when the raw data are not available, but the correlation matrix between
the items is available. In this case, it is not possible to find individual scores, but it is possible to
find the reliability and intercorrelations of the scales. This may be done using the cluster.cor
function or the scoreItems function. The use of a keys matrix is the same as in the raw data
case.
Consider the same bfi data set, but first find the correlations, and then use scoreItems.
> r.bfi <- cor(bfi,use="pairwise")

53
> png('scores.png')
> pairs.panels(scores$scores,pch='.',jiggle=TRUE)
> dev.off()
pdf
2

Figure 14: A graphic analysis of the Big Five scales found by using the scoreItems function. The
pair.wise plot allows us to see that some participants have reached the ceiling of the scale for these
5 items scales. Using the pch=’.’ option in pairs.panels is recommended when plotting many cases.
The data points were “jittered” by setting jiggle=TRUE. Jiggling this way shows the density more
clearly. To save space, the figure was done as a png. For a clearer figure, save as a pdf.

54
> scales <- scoreItems(keys.list,r.bfi)
> summary(scales)

Call: NULL

Scale intercorrelations corrected for attenuation

raw correlations below the diagonal, (standardized) alpha on the diagonal
corrected correlations above the diagonal:
agree conscientious extraversion neuroticism openness
agree 0.71 0.35 0.64 -0.242 0.25
conscientious 0.25 0.73 0.36 -0.287 0.30
extraversion 0.47 0.27 0.76 -0.278 0.35
neuroticism -0.18 -0.22 -0.22 0.815 -0.11
openness 0.16 0.20 0.24 -0.074 0.61

To find the correlations of the items with each of the scales (the “structure" matrix) or the correla-
tions of the items controlling for the other scales (the “pattern" matrix), use the cluster.loadings
function. To do both at once (e.g., the correlations of the scales as well as the item by scale corre-
lations), it is also possible to just use scoreItems.

5.5 Scoring Multiple Choice Items

Some items (typically associated with ability tests) are not themselves mini-scales ranging from low
to high levels of expression of the item of interest, but are rather multiple choice where one response
is the correct response. Two analyses are useful for this kind of item: examining the response pat-
terns to all the alternatives (looking for good or bad distractors) and scoring the items as correct
or incorrect. Both of these operations may be done using the score.multiple.choice func-
tion. Consider the 16 example items taken from an online ability test at the Personality Project:
https://www.sapa-project.org/. This is part of the Synthetic Aperture Personality As-
sessment (SAPA) study discussed in Revelle et al. (2011, 2010).
> data(iqitems)
> iq.keys <- c(4,4,4, 6,6,3,4,4, 5,2,2,4, 3,2,6,7)
> score.multiple.choice(iq.keys,iqitems)

Call: score.multiple.choice(key = iq.keys, data = iqitems)

(Unstandardized) Alpha:
[1] 0.84

Average item correlation:

[1] 0.25

item statistics
key 0 1 2 3 4 5 6 7 8 miss r n mean sd
reason.4 4 0.05 0.05 0.11 0.10 0.64 0.03 0.02 0.00 0.00 0 0.59 1523 0.64 0.48
reason.16 4 0.04 0.06 0.08 0.10 0.70 0.01 0.00 0.00 0.00 0 0.53 1524 0.70 0.46
reason.17 4 0.05 0.03 0.05 0.03 0.70 0.03 0.11 0.00 0.00 0 0.59 1523 0.70 0.46
reason.19 6 0.04 0.02 0.13 0.03 0.06 0.10 0.62 0.00 0.00 0 0.56 1523 0.62 0.49
letter.7 6 0.05 0.01 0.05 0.03 0.11 0.14 0.60 0.00 0.00 0 0.58 1524 0.60 0.49
letter.33 3 0.06 0.10 0.13 0.57 0.04 0.09 0.02 0.00 0.00 0 0.56 1523 0.57 0.50
letter.34 4 0.04 0.09 0.07 0.11 0.61 0.05 0.02 0.00 0.00 0 0.59 1523 0.61 0.49
letter.58 4 0.06 0.14 0.09 0.09 0.44 0.16 0.01 0.00 0.00 0 0.58 1525 0.44 0.50

55
matrix.45 5 0.04 0.01 0.06 0.14 0.18 0.53 0.04 0.00 0.00 0 0.51 1523 0.53 0.50
matrix.46 2 0.04 0.12 0.55 0.07 0.11 0.06 0.05 0.00 0.00 0 0.52 1524 0.55 0.50
matrix.47 2 0.04 0.05 0.61 0.07 0.11 0.06 0.06 0.00 0.00 0 0.55 1523 0.61 0.49
matrix.55 4 0.04 0.02 0.18 0.14 0.37 0.07 0.18 0.00 0.00 0 0.45 1524 0.37 0.48
rotate.3 3 0.04 0.03 0.04 0.19 0.22 0.15 0.05 0.12 0.15 0 0.51 1523 0.19 0.40
rotate.4 2 0.04 0.03 0.21 0.05 0.18 0.04 0.04 0.25 0.15 0 0.56 1523 0.21 0.41
rotate.6 6 0.04 0.22 0.02 0.05 0.14 0.05 0.30 0.04 0.14 0 0.55 1523 0.30 0.46
rotate.8 7 0.04 0.03 0.21 0.07 0.16 0.05 0.13 0.19 0.13 0 0.48 1524 0.19 0.39

> #just convert the items to true or false

> iq.tf <- score.multiple.choice(iq.keys,iqitems,score=FALSE)
> describe(iq.tf) #compare to previous results

vars n mean sd median trimmed mad min max range skew kurtosis se
reason.4 1 1523 0.64 0.48 1 0.68 0 0 1 1 -0.58 -1.66 0.01
reason.16 2 1524 0.70 0.46 1 0.75 0 0 1 1 -0.86 -1.26 0.01
reason.17 3 1523 0.70 0.46 1 0.75 0 0 1 1 -0.86 -1.26 0.01
reason.19 4 1523 0.62 0.49 1 0.64 0 0 1 1 -0.47 -1.78 0.01
letter.7 5 1524 0.60 0.49 1 0.62 0 0 1 1 -0.41 -1.84 0.01
letter.33 6 1523 0.57 0.50 1 0.59 0 0 1 1 -0.29 -1.92 0.01
letter.34 7 1523 0.61 0.49 1 0.64 0 0 1 1 -0.46 -1.79 0.01
letter.58 8 1525 0.44 0.50 0 0.43 0 0 1 1 0.23 -1.95 0.01
matrix.45 9 1523 0.53 0.50 1 0.53 0 0 1 1 -0.10 -1.99 0.01
matrix.46 10 1524 0.55 0.50 1 0.56 0 0 1 1 -0.20 -1.96 0.01
matrix.47 11 1523 0.61 0.49 1 0.64 0 0 1 1 -0.47 -1.78 0.01
matrix.55 12 1524 0.37 0.48 0 0.34 0 0 1 1 0.52 -1.73 0.01
rotate.3 13 1523 0.19 0.40 0 0.12 0 0 1 1 1.55 0.40 0.01
rotate.4 14 1523 0.21 0.41 0 0.14 0 0 1 1 1.40 -0.03 0.01
rotate.6 15 1523 0.30 0.46 0 0.25 0 0 1 1 0.88 -1.24 0.01
rotate.8 16 1524 0.19 0.39 0 0.11 0 0 1 1 1.62 0.63 0.01

Once the items have been scored as true or false (assigned scores of 1 or 0), they made then be
scored into multiple scales using the normal scoreItems function.

5.6 Item analysis

Basic item analysis starts with describing the data (describe, finding the number of dimensions
using factor analysis (fa) and cluster analysis iclust perhaps using the Very Simple Structure
criterion (vss), or perhaps parallel analysis fa.parallel. Item whole correlations may then be
found for scales scored on one dimension (alpha or many scales simultaneously (scoreItems).
Scales can be modified by changing the keys matrix (i.e., dropping particular items, changing
the scale on which an item is to be scored). This analysis can be done on the normal Pearson
correlation matrix or by using polychoric correlations. Validities of the scales can be found using
multiple correlation of the raw data or based upon correlation matrices using the lmCor function.
However, more powerful item analysis tools are now available by using Item Response Theory
approaches.
Although the response.frequencies output from score.multiple.choice is useful
to examine in terms of the probability of various alternatives being endorsed, it is even better to
examine the pattern of these responses as a function of the underlying latent trait or just the total
score. This may be done by using irt.responses (Figure 15).

56
> data(iqitems)
> iq.keys <- c(4,4,4, 6,6,3,4,4, 5,2,2,4, 3,2,6,7)
> scores <- score.multiple.choice(iq.keys,iqitems,score=TRUE,short=FALSE)
> #note that for speed we can just do this on simple item counts rather than IRT based scores.
> op <- par(mfrow=c(2,2)) #set this to see the output for multiple items
> irt.responses(scores$scores,iqitems[1:4],breaks=11)

reason.4 reason.16

0 2 4 0 2 4
0.8

0.8
1 3 5 1 3 5
P(response)

P(response)
0.4

0.4
0.0

0.0
0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0

theta theta

reason.17 reason.19

0 2 4 0 2 4
0.8

0.8

1 3 5 1 3 5
P(response)

P(response)
0.4

0.4
0.0

0.0

0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0

theta theta

Figure 15: The pattern of responses to multiple choice ability items can show that some items
have poor distractors. This may be done by using the the irt.responses function. A good
distractor is one that is negatively related to ability.

57
5.6.1 Exploring the item structure of scales
The Big Five scales found above can be understood in terms of the item - whole correlations, but
it is also useful to think of the endorsement frequency of the items. The item.lookup function
will sort items by their factor loading/item-whole correlation, and then resort those above a certain
threshold in terms of the item means. Item content is shown by using the dictionary developed
for those items. This allows one to see the structure of each scale in terms of its endorsement
range. This is a simple way of thinking of items that is also possible to do using the various IRT
approaches discussed later.
> m <- colMeans(bfi,na.rm=TRUE)
> item.lookup(scales$item.corrected[,1:3],m,dictionary=bfi.dictionary[1:2])

agree conscientious extraversion means ItemLabel Item

A3 0.71 0.22 0.49 4.60 q_1206 Know how to comfort others.
A2 0.67 0.23 0.40 4.80 q_1162 Inquire about others' well-being.
A5 0.62 0.23 0.55 4.56 q_1419 Make people feel at ease.
A4 0.49 0.29 0.30 4.70 q_1364 Love children.
A1 -0.39 -0.05 -0.10 2.41 q_146 Am indifferent to the feelings of others.
N1 -0.22 -0.20 -0.11 2.93 q_952 Get angry easily.
N2 -0.21 -0.18 -0.12 3.51 q_974 Get irritated easily.
C4 -0.23 -0.66 -0.23 2.55 q_626 Do things in a half-way manner.
C2 0.21 0.62 0.18 4.37 q_530 Continue until everything is perfect.
C5 -0.26 -0.59 -0.29 3.30 q_1949 Waste my time.
C1 0.13 0.54 0.20 4.50 q_124 Am exacting in my work.
C3 0.21 0.54 0.15 4.30 q_619 Do things according to a plan.
N3 -0.13 -0.19 -0.14 3.22 q_1099 Have frequent mood swings.
O2 -0.01 -0.18 -0.07 2.71 q_316 Avoid difficult reading material.
O5 -0.10 -0.15 -0.11 2.49 q_1964 Will not probe deeply into a subject.
E2 -0.39 -0.24 -0.70 3.14 q_901 Find it difficult to approach others.
E4 0.52 0.22 0.68 4.42 q_1410 Make friends easily.
E3 0.45 0.20 0.61 4.00 q_1205 Know how to captivate people.
E1 -0.31 -0.06 -0.59 2.97 q_712 Don't talk a lot.
E5 0.35 0.40 0.55 4.42 q_1768 Take charge.
O3 0.26 0.20 0.43 4.44 q_492 Carry the conversation to a higher level.
N4 -0.22 -0.29 -0.39 3.19 q_1479 Often feel blue.
O1 0.17 0.20 0.33 4.82 q_128 Am full of ideas.
N5 -0.04 -0.13 -0.19 2.97 q_1505 Panic easily.
O4 0.06 0.00 -0.10 4.89 q_1738 Spend time reflecting on things.

5.6.2 Empirical scale construction

There are some situations where one wants to identify those items that most relate to a particular
criterion. Although this will capitalize on chance and the results should interpreted cautiously, it
does give a feel for what is being measured. Consider the following example from the bfi data set.
The items that best predicted gender, education, and age may be found using the bestScales
function. This also shows the use of a dictionary that has the item content.
> data(bfi)
> bestScales(bfi,criteria=c("gender","education","age"),cut=.1,dictionary=bfi.dictionary[,1:3])

Call = bestScales(x = bfi, criteria = c("gender", "education", "age"),

cut = 0.1, dictionary = bfi.dictionary[, 1:3])
The items most correlated with the criteria yield r's of
correlation n.items
gender 0.32 9
education 0.14 1
age 0.24 10

The best items, their correlations and content are

$gender

58
 gender ItemLabel Item Giant3
N5 0.21 q_1505 Panic easily. Stability
A2 0.18 q_1162 Inquire about others' well-being. Cohesion
A1 -0.16 q_146 Am indifferent to the feelings of others. Cohesion
A3 0.14 q_1206 Know how to comfort others. Cohesion
A4 0.13 q_1364 Love children. Cohesion
E1 -0.13 q_712 Don't talk a lot. Plasticity
N3 0.12 q_1099 Have frequent mood swings. Stability
O1 -0.10 q_128 Am full of ideas. Plasticity
A5 0.10 q_1419 Make people feel at ease. Cohesion

$education
education ItemLabel Item Giant3
A1 -0.14 q_146 Am indifferent to the feelings of others. Cohesion

$age
age ItemLabel Item Giant3
A1 -0.16 q_146 Am indifferent to the feelings of others. Cohesion
C4 -0.15 q_626 Do things in a half-way manner. Stability
A4 0.14 q_1364 Love children. Cohesion
A5 0.13 q_1419 Make people feel at ease. Cohesion
E5 0.11 q_1768 Take charge. Plasticity
A2 0.11 q_1162 Inquire about others' well-being. Cohesion
N3 -0.11 q_1099 Have frequent mood swings. Stability
E2 -0.11 q_901 Find it difficult to approach others. Plasticity
N5 -0.10 q_1505 Panic easily. Stability
N2 -0.10 q_974 Get irritated easily. Stability

6 Item Response Theory analysis

The use of Item Response Theory has become is said to be the “new psychometrics". The empha-
sis is upon item properties, particularly those of item difficulty or location and item discrimination.
These two parameters are easily found from classic techniques when using factor analyses of corre-
lation matrices formed by polychoric or tetrachoric correlations. The irt.fa function
does this and then graphically displays item discrimination and item location as well as item and
test information (see Figure 16).

6.1 Factor analysis and Item Response Theory

If the correlations of all of the items reflect one underlying latent variable, then factor analysis of
the matrix of tetrachoric correlations should allow for the identification of the regression slopes
(α ) of the items on the latent variable. These regressions are, of course just the factor loadings.
Item difficulty, δ j and item discrimination, α j may be found from factor analysis of the tetrachoric
correlations where λ j is just the factor loading on the first factor and τ j is the normal threshold
reported by the tetrachoric function.

Dτ λj
δj = q , αj = q (2)
1 − λ j2 1 − λ j2

59
where D is a scaling factor used when converting to the parameterization of logistic model and is
1.702 in that case and 1 in the case of the normal ogive model. Thus, in the case of the normal
model, factor loadings (λ j ) and item thresholds (τ ) are just

αj δj
λj = q , τj = q .
1 + α 2j 1 + α 2j

Consider 9 dichotomous items representing one factor but differing in their levels of difficulty

> set.seed(17)
> d9 <- sim.irt(9,1000,-2.0,2.0,mod="normal") #dichotomous items
> test <- irt.fa(d9$items,correct=0)

> test

Item Response Analysis using Factor Analysis

Call: irt.fa(x = d9$items, correct = 0)

Item Response Analysis using Factor Analysis

Summary information by factor and item

Factor = 1
-3 -2 -1 0 1 2 3
V1 1.02 1.17 0.16 0.01 0.00 0.00 0.00
V2 0.17 0.41 0.47 0.22 0.06 0.02 0.00
V3 0.09 0.36 0.65 0.37 0.10 0.02 0.00
V4 0.04 0.21 0.64 0.60 0.18 0.04 0.01
V5 0.01 0.08 0.43 0.94 0.40 0.07 0.01
V6 0.01 0.04 0.19 0.61 0.62 0.20 0.04
V7 0.00 0.01 0.08 0.37 0.78 0.41 0.09
V8 0.00 0.01 0.04 0.19 0.58 0.59 0.20
V9 0.00 0.00 0.00 0.02 0.21 1.15 0.82
Test Info 1.34 2.28 2.66 3.33 2.93 2.49 1.18
SEM 0.86 0.66 0.61 0.55 0.58 0.63 0.92
Reliability 0.26 0.56 0.62 0.70 0.66 0.60 0.15

Factor analysis with Call: fa(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate,
fm = fm)

Test of the hypothesis that 1 factor is sufficient.

The degrees of freedom for the model is 27 and the objective function was 2.01
The number of observations was 1000 with Chi Square = 2001.9 with prob < 0

The root mean square of the residuals (RMSA) is 0.07

The df corrected root mean square of the residuals is 0.08

Tucker Lewis Index of factoring reliability = 0.596

RMSEA index = 0.27 and the 10 % confidence intervals are 0.261 0.281
BIC = 1815.39

Similar analyses can be done for polytomous items such as those of the bfi extraversion scale:

60
> op <- par(mfrow=c(3,1))
> plot(test,type="ICC")
> plot(test,type="IIC")
> plot(test,type="test")
> op <- par(mfrow=c(1,1))

Item parameters from factor analysis

Probability of Response

0.8

V1 V2 V3 V4 V5 V6 V7 V8 V9
0.4
0.0

−3 −2 −1 0 1 2 3

Latent Trait (normal scale)

Item information from factor analysis

V1
Item Information

V9
1.0

V5
V4 V6 V7 V8
V3
V2
0.0

−3 −2 −1 0 1 2 3

Latent Trait (normal scale)

Test information −− item parameters from factor analysis

Test Information

3.0

Reliability
0.6
1.5

−0.2
0.0

−3 −2 −1 0 1 2 3

Latent Trait (normal scale)

Figure 16: A graphic analysis of 9 dichotomous (simulated) items. The top panel shows the prob-
ability of item endorsement as the value of the latent trait increases. Items differ in their location
(difficulty) and discrimination (slope). The middle panel shows the information in each item as
a function of latent trait level. An item is most informative when the probability of endorsement
is 50%. The lower panel shows the total test information. These items form a test that is most
informative (most accurate) at the middle range of the latent trait.

61
> data(bfi)
> e.irt <- irt.fa(bfi[11:15])
> e.irt

Item Response Analysis using Factor Analysis

Call: irt.fa(x = bfi[11:15])

Item Response Analysis using Factor Analysis

Summary information by factor and item

Factor = 1
-3 -2 -1 0 1 2 3
E1 0.36 0.56 0.72 0.73 0.59 0.39 0.21
E2 0.47 0.87 1.16 1.22 0.98 0.58 0.25
E3 0.36 0.51 0.61 0.61 0.51 0.37 0.23
E4 0.61 0.95 1.09 0.94 0.66 0.37 0.16
E5 0.34 0.44 0.49 0.47 0.38 0.28 0.18
Test Info 2.14 3.34 4.07 3.96 3.12 1.98 1.03
SEM 0.68 0.55 0.50 0.50 0.57 0.71 0.99
Reliability 0.53 0.70 0.75 0.75 0.68 0.49 0.02

Factor analysis with Call: fa(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate,
fm = fm)

Test of the hypothesis that 1 factor is sufficient.

The degrees of freedom for the model is 5 and the objective function was 0.05
The number of observations was 2800 with Chi Square = 138.2 with prob < 4.3e-28

The root mean square of the residuals (RMSA) is 0.04

The df corrected root mean square of the residuals is 0.06

Tucker Lewis Index of factoring reliability = 0.931

RMSEA index = 0.098 and the 10 % confidence intervals are 0.084 0.112
BIC = 98.51

The item information functions show that not all of items are equally good (Figure 17):
These procedures can be generalized to more than one factor by specifying the number of factors
in irt.fa. The plots can be limited to those items with discriminations greater than some value
of cut. An invisible object is returned when plotting the output from irt.fa that includes the
average information for each item that has loadings greater than cut.
> print(e.info,sort=TRUE)

Item Response Analysis using Factor Analysis

Summary information by factor and item

Factor = 1
-3 -2 -1 0 1 2 3
E2 0.47 0.87 1.16 1.22 0.98 0.58 0.25
E4 0.61 0.95 1.09 0.94 0.66 0.37 0.16
E1 0.36 0.56 0.72 0.73 0.59 0.39 0.21
E3 0.36 0.51 0.61 0.61 0.51 0.37 0.23
E5 0.34 0.44 0.49 0.47 0.38 0.28 0.18
Test Info 2.14 3.34 4.07 3.96 3.12 1.98 1.03
SEM 0.68 0.55 0.50 0.50 0.57 0.71 0.99
Reliability 0.53 0.70 0.75 0.75 0.68 0.49 0.02

62
> e.info <- plot(e.irt,type="IIC")

Item information from factor analysis

−E2
1.2

E4
1.0
0.8
Item Information

−E1

E3
0.6

E5
0.4
0.2
0.0

−3 −2 −1 0 1 2 3

Latent Trait (normal scale)

Figure 17: A graphic analysis of 5 extraversion items from the bfi. The curves represent the amount
of information in the item as a function of the latent score for an individual. That is, each item is
maximally discriminating at a different part of the latent continuum. Print e.info to see the average
information for each item.

63
More extensive IRT packages include the ltm and eRm and should be used for serious Item Re-
sponse Theory analysis.

6.2 Speeding up analyses

Finding tetrachoric or polychoric correlations is very time consuming. Thus, to speed up the pro-
cess of analysis, the original correlation matrix is saved as part of the output of both irt.fa and
omega. Subsequent analyses may be done by using this correlation matrix. This is done by doing
the analysis not on the original data, but rather on the output of the previous analysis.
In addition, recent releases of the psych take advantage of the parallels package and use multi-
cores. The default for Macs and Unix machines is to use two cores, but this can be increased
using the options command. The biggest step up in improvement is from 1 to 2 cores, but for large
problems using polychoric correlations, the more cores available, the better.
For example of taking the output from the 16 ability items from the SAPA project when scored
for True/False using score.multiple.choice we can first do a simple IRT analysis of one
factor (Figure 20) and then use that correlation matrix to do an omega analysis to show the sub-
structure of the ability items . We can also show the total test information (merely the sum of the
item information. This shows that even with just 16 items, the test is very reliable for most of the
range of ability. The fa.irt function saves the correlation matrix and item statistics so that they
can be redrawn with other options.
detectCores() #how many are available
options("mc.cores") #how many have been set to be used
options("mc.cores"=4) #set to use 4 cores

> iq.irt

Item Response Analysis using Factor Analysis

Call: irt.fa(x = ability)

Item Response Analysis using Factor Analysis

Summary information by factor and item

Factor = 1
-3 -2 -1 0 1 2 3
reason.4 0.05 0.24 0.66 0.54 0.16 0.03 0.01
reason.16 0.08 0.23 0.39 0.32 0.14 0.05 0.01
reason.17 0.08 0.34 0.73 0.43 0.10 0.02 0.00
reason.19 0.06 0.18 0.36 0.37 0.19 0.07 0.02
letter.7 0.05 0.18 0.45 0.47 0.20 0.06 0.01
letter.33 0.05 0.15 0.34 0.38 0.21 0.08 0.02
letter.34 0.05 0.19 0.49 0.50 0.20 0.05 0.01
letter.58 0.02 0.09 0.30 0.55 0.36 0.12 0.03
matrix.45 0.05 0.12 0.21 0.25 0.18 0.09 0.04
matrix.46 0.05 0.13 0.24 0.28 0.19 0.09 0.03
matrix.47 0.06 0.17 0.35 0.37 0.19 0.07 0.02
matrix.55 0.04 0.08 0.13 0.17 0.16 0.11 0.06
rotate.3 0.00 0.02 0.08 0.31 0.65 0.42 0.12
rotate.4 0.00 0.01 0.06 0.33 0.89 0.50 0.10
rotate.6 0.01 0.04 0.16 0.48 0.59 0.24 0.06

64
> iq.irt <- irt.fa(ability)

Item information from factor analysis

rotate.4
0.8

reason.17
reason.4
rotate.3
rotate.6
0.6
Item Information

letter.34
letter.58
letter.7 rotate.8

reason.19
letter.33
reason.16
matrix.47
0.4

matrix.46
matrix.45
0.2

matrix.55
0.0

−3 −2 −1 0 1 2 3

Latent Trait (normal scale)

Figure 18: A graphic analysis of 16 ability items sampled from the SAPA project. The curves
represent the amount of information in the item as a function of the latent score for an individual.
That is, each item is maximally discriminating at a different part of the latent continuum. Print
iq.irt to see the average information for each item. Partly because this is a power test (it is given
on the web) and partly because the items have not been carefully chosen, the items are not very
discriminating at the high end of the ability dimension.

65
> plot(iq.irt,type='test')

Test information −− item parameters from factor analysis

0.84
6
5

0.78
4
Test Information

Reliability
0.67
3
2

0.34
1
0

−3 −2 −1 0 1 2 3

Latent Trait (normal scale)

Figure 19: A graphic analysis of 16 ability items sampled from the SAPA project. The total test
information at all levels of difficulty may be shown by specifying the type=’test’ option in the plot
function.

66
rotate.8 0.01 0.02 0.09 0.28 0.51 0.35 0.12
Test Info 0.66 2.17 5.03 6.03 4.93 2.35 0.68
SEM 1.23 0.68 0.45 0.41 0.45 0.65 1.22
Reliability -0.52 0.54 0.80 0.83 0.80 0.57 -0.48

Factor analysis with Call: fa(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate,
fm = fm)

Test of the hypothesis that 1 factor is sufficient.

The degrees of freedom for the model is 104 and the objective function was 1.92
The number of observations was 1525 with Chi Square = 2906.16 with prob < 0

The root mean square of the residuals (RMSA) is 0.08

The df corrected root mean square of the residuals is 0.09

Tucker Lewis Index of factoring reliability = 0.722

RMSEA index = 0.133 and the 10 % confidence intervals are 0.129 0.137
BIC = 2143.86

6.3 IRT based scoring

The primary advantage of IRT analyses is examining the item properties (both difficulty and dis-
crimination). With complete data, the scores based upon simple total scores and based upon IRT
are practically identical (this may be seen in the examples for scoreIrt). However, when work-
ing with data such as those found in the Synthetic Aperture Personality Assessment (SAPA) project,
it is advantageous to use IRT based scoring. SAPA data might have 2-3 items/person sampled from
scales with 10-20 items. Simply finding the average of the three (classical test theory) fails to con-
sider that the items might differ in either discrimination or in difficulty. The scoreIrt function
applies basic IRT to this problem.
Consider 1000 randomly generated subjects with scores on 9 true/false items differing in diffi-
culty. Selectively drop the hardest items for the 1/3 lowest subjects, and the 4 easiest items for
the 1/3 top subjects (this is a crude example of what tailored testing would do). Then score these
subjects:
> v9 <- sim.irt(9,1000,-2.,2.,mod="normal") #dichotomous items
> items <- v9$items
> test <- irt.fa(items)

> total <- rowSums(items)

> ord <- order(total)
> items <- items[ord,]
> #now delete some of the data - note that they are ordered by score
> items[1:333,5:9] <- NA
> items[334:666,3:7] <- NA
> items[667:1000,1:4] <- NA
> scores <- scoreIrt(test,items)
> unitweighted <- scoreIrt(items=items,keys=rep(1,9))
> scores.df <- data.frame(true=v9$theta[ord],scores,unitweighted)
> colnames(scores.df) <- c("True theta","irt theta","total","fit","rasch","total","fit")

67
> om <- omega(iq.irt$rho,4)
Omega

rotate.3
rotate.4 0.7
0.6 0.6
rotate.8 0.6
0.6 F1*
0.5
rotate.6
0.5
0.6 letter.34
0.5
0.6 letter.7 0.5
0.6 F2*
letter.33 0.4
0.6
0.3
0.6 letter.58
g
0.5 matrix.47
0.7
reason.17 0.6 F3*
0.6 0.3
0.6 reason.4 0.2
0.2
0.5 reason.19
0.5
reason.16
0.5 F4*
0.4 matrix.45 0.8
0.3
matrix.46
matrix.55

Figure 20: An Omega analysis of 16 ability items sampled from the SAPA project. The items rep-
resent a general factor as well as four lower level factors. The analysis is done using the tetrachoric
correlations found in the previous irt.fa analysis. The four matrix items have some serious
problems, which may be seen later when examine the item response functions.

68
These results are seen in Figure 21.

Figure 21: IRT based scoring and total test scores for 1000 simulated subjects. True theta values
are reported and then the IRT and total scoring systems.
> pairs.panels(scores.df,pch='.',gap=0)
> title('Comparing true theta for IRT, Rasch and classically based scoring',li
Comparing true theta for IRT, Rasch and classically based scoring
−2 0 1 2 0.4 0.8 1.2 0.0 0.4 0.8

True theta

2
0.83 0.40 0.05 0.55 0.40 −0.19

0
−3
0 1 2

irt theta
0.50 0.07 0.68 0.50 −0.20
−2

total

0.0 0.4 0.8

0.10 0.96 1.00 −0.01
1.2

fit
−0.02 0.10 0.82
0.8
0.4

rasch

0.5
0.96 −0.14

−1.0
total
0.0 0.4 0.8

−0.01

fit 1.4
1.0
0.6

−3 0 2 0.0 0.4 0.8 −1.0 0.5 0.6 1.0 1.4

6.3.1 1 versus 2 parameter IRT scoring

In Item Response Theory, items can be assumed to be equally discriminating but to differ in their
difficulty (the Rasch model) or to vary in their discriminability. Two functions (scoreIrt.1pl
and scoreIrt.2pl) are meant to find multiple IRT based scales using the Rasch model or the
2 parameter model. Both allow for negatively keyed as well as positively keyed items. Consider
the bfi data set with scoring keys key.list and items listed as an item.list. (This is the same as the
key.list, but with the negative signs removed.)

69
> keys.list <- list(agree=c("-A1","A2","A3","A4","A5"),
+ conscientious=c("C1","C2","C3","-C4","-C5"),
+ extraversion=c("-E1","-E2","E3","E4","E5"),
+ neuroticism=c("N1","N2","N3","N4","N5"),
+ openness = c("O1","-O2","O3","O4","-O5"))
> item.list <- list(agree=c("A1","A2","A3","A4","A5"),
+ conscientious=c("C1","C2","C3","C4","C5"),
+ extraversion=c("E1","E2","E3","E4","E5"),
+ neuroticism=c("N1","N2","N3","N4","N5"),
+ openness = c("O1","O2","O3","O4","O5"))
> bfi.1pl <- scoreIrt.1pl(keys.list,bfi) #the one parameter solution
> bfi.2pl <- scoreIrt.2pl(item.list,bfi) #the two parameter solution
> bfi.ctt <- scoreFast(keys.list,bfi) # fast scoring function
>

We can compare these three ways of doing the analysis using the cor2 function which correlates
two separate data frames. All three models produce vey simillar results for the case of almost
complete data. It is when we have massively missing completely at random data (MMCAR) that
the results show the superiority of the irt scoring.
> #compare the solutions using the cor2 function
> cor2(bfi.1pl,bfi.ctt)

agree-A conscientious-A extraversion-A neuroticism-A openness-A

agree 0.93 0.27 0.44 -0.20 0.19
conscientious 0.25 0.97 0.25 -0.22 0.17
extraversion 0.43 0.25 0.94 -0.23 0.23
neuroticism -0.19 -0.23 -0.22 1.00 -0.09
openness 0.12 0.19 0.19 -0.07 0.95

> cor2(bfi.2pl,bfi.ctt)

agree-A conscientious-A extraversion-A neuroticism-A openness-A

agree 0.98 0.25 0.49 -0.16 0.15
conscientious 0.25 0.97 0.25 -0.23 0.18
extraversion 0.43 0.25 0.94 -0.24 0.21
neuroticism -0.19 -0.22 -0.19 0.99 -0.08
openness 0.17 0.21 0.27 -0.11 0.98

> cor2(bfi.2pl,bfi.1pl)

agree conscientious extraversion neuroticism openness

agree 0.89 0.25 0.44 -0.17 0.12
conscientious 0.26 1.00 0.23 -0.23 0.16
extraversion 0.43 0.23 0.99 -0.24 0.20
neuroticism -0.21 -0.22 -0.20 0.99 -0.06
openness 0.20 0.19 0.28 -0.11 0.92

7 Multilevel modeling
Correlations between individuals who belong to different natural groups (based upon e.g., ethnic-
ity, age, gender, college major, or country) reflect an unknown mixture of the pooled correlation
within each group as well as the correlation of the means of these groups. These two correlations
are independent and do not allow inferences from one level (the group) to the other level (the indi-
vidual). When examining data at two levels (e.g., the individual and by some grouping variable), it

70
is useful to find basic descriptive statistics (means, sds, ns per group, within group correlations) as
well as between group statistics (over all descriptive statistics, and overall between group correla-
tions). Of particular use is the ability to decompose a matrix of correlations at the individual level
into correlations within group and correlations between groups.

7.1 Decomposing data into within and between level correlations using statsBy
There are at least two very powerful packages (nlme and multilevel) which allow for complex
analysis of hierarchical (multilevel) data structures. statsBy is a much simpler function to give
some of the basic descriptive statistics for two level models.
This follows the decomposition of an observed correlation into the pooled correlation within groups
(rwg) and the weighted correlation of the means between groups which is discussed by Pedhazur
(1997) and by Bliese (2009) in the multilevel package.

rxy = ηxwg ∗ ηywg ∗ rxywg + ηxbg ∗ ηybg ∗ rxybg (3)

where rxy is the normal correlation which may be decomposed into a within group and between
group correlations rxywg and rxybg and η (eta) is the correlation of the data with the within group
values, or the group means.

7.2 Generating and displaying multilevel data

withinBetween is an example data set of the mixture of within and between group correlations.
The within group correlations between 9 variables are set to be 1, 0, and -1 while those between
groups are also set to be 1, 0, -1. These two sets of correlations are crossed such that V1, V4, and
V7 have within group correlations of 1, as do V2, V5 and V8, and V3, V6 and V9. V1 has a within
group correlation of 0 with V2, V5, and V8, and a -1 within group correlation with V3, V6 and V9.
V1, V2, and V3 share a between group correlation of 1, as do V4, V5 and V6, and V7, V8 and V9.
The first group has a 0 between group correlation with the second and a -1 with the third group.
See the help file for withinBetween to display these data.
sim.multilevel will generate simulated data with a multilevel structure.
The statsBy.boot function will randomize the grouping variable ntrials times and find the
statsBy output. This can take a long time and will produce a great deal of output. This output
can then be summarized for relevant variables using the statsBy.boot.summary function
specifying the variable of interest.
Consider the case of the relationship between various tests of ability when the data are grouped by
level of education (statsBy(sat.act)) or when affect data are analyzed within and between an affect
manipulation (statsBy(affect) ).

71
7.3 Factor analysis by groups
Confirmatory factor analysis comparing the structures in multiple groups can be done in the lavaan
package. However, for exploratory analyses of the structure within each of multiple groups, the
faBy function may be used in combination with the statsBy function. First run pfunstatsBy
with the correlation option set to TRUE, and then run faBy on the resulting output.
sb <- statsBy(bfi[c(1:25,27)], group="education",cors=TRUE)
faBy(sb,nfactors=5) #find the 5 factor solution for each education level

7.4 Multilevel reliability

The mlr and multilevelReliablity functions follow the advice of Shrout and Lane (2012)
for estimating multievel reliablilty. A detailed discussion of this procedure is given in Revelle
and Wilt (2017) which is available at https://personality-project.org/revelle/
publications/rw.paid.17.final.pdf.

8 Set Correlation and Multiple Regression from the correlation ma-

trix
An important generalization of multiple regression and multiple correlation is set correlation de-
veloped by Cohen (1982) and discussed by Cohen et al. (2003). Set correlation is a multivariate
generalization of multiple regression and estimates the amount of variance shared between two
sets of variables. Set correlation also allows for examining the relationship between two sets when
controlling for a third set. This is implemented in the lmCor function. Set correlation is
n
R2 = 1 − ∏(1 − λi )
i=1

where λi is the ith eigen value of the eigen value decomposition of the matrix

R = R−1 −1 −1
xx Rxy Rxx Rxy .

Unfortunately, there are several cases where set correlation will give results that are much too high.
This will happen if some variables from the first set are highly related to those in the second set,
even though most are not. In this case, although the set correlation can be very high, the degree
of relationship between the sets is not as high. In this case, an alternative statistic, based upon the
average canonical correlation might be more appropriate.
lmCor has the additional feature that it will calculate multiple and partial correlations from the
correlation or covariance matrix rather than the original data.

72
Consider the correlations of the 6 variables in the sat.act data set. First do the normal multiple
regression, and then compare it with the results using lmCor. Two things to notice. lmCor works
on the correlation or covariance or raw data matrix, and thus if using the correlation matrix, will
report standardized or raw β̂ weights. Secondly, it is possible to do several multiple regressions
simultaneously. If the number of observations is specified, or if the analysis is done on raw data,
statistical tests of significance are applied.
For this example, the analysis is done on the correlation matrix rather than the raw data.
> C <- cov(sat.act,use="pairwise")
> model1 <- lm(ACT~ gender + education + age, data=sat.act)
> summary(model1)

Call:
lm(formula = ACT ~ gender + education + age, data = sat.act)

Residuals:
Min 1Q Median 3Q Max
-25.2458 -3.2133 0.7769 3.5921 9.2630

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 27.41706 0.82140 33.378 < 2e-16 ***
gender -0.48606 0.37984 -1.280 0.20110
education 0.47890 0.15235 3.143 0.00174 **
age 0.01623 0.02278 0.712 0.47650
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 4.768 on 696 degrees of freedom

Multiple R-squared: 0.0272, Adjusted R-squared: 0.02301
F-statistic: 6.487 on 3 and 696 DF, p-value: 0.0002476

Compare this with the output from lmCor.

> #compare with lmCor

> lmCor(gender + education + age ~ ACT + SATV + SATQ, data = C, n.obs=700)

Call: lmCor(y = gender + education + age ~ ACT + SATV + SATQ, data = C,

n.obs = 700)

Multiple Regression from raw data

DV = gender
slope se t p lower.ci upper.ci VIF Vy.x
(Intercept) 0.00 0.02 0.00 1.000 -0.08 0.08 1.00 0.00
ACT -0.29 0.60 -0.48 0.680 -2.88 2.30 1029.17 0.21
SATV 1.10 0.28 3.98 0.058 -0.09 2.29 217.77 -0.58
SATQ -1.55 0.35 -4.45 0.047 -3.06 -0.05 347.59 1.36

Residual Standard Error = 0.04 with 2 degrees of freedom

Multiple Regression
R R2 Ruw R2uw Shrunken R2 SE of R2 overall F df1 df2 p
gender 1 1 0.73 0.54 1 0 947.92 3 2 0.00105

DV = education

73
 slope se t p lower.ci upper.ci VIF Vy.x
(Intercept) 0.00 0.55 0.00 1.00 -2.35 2.35 1.00 0.00
ACT 2.30 17.52 0.13 0.91 -73.07 77.68 1029.17 1.39
SATV -0.39 8.06 -0.05 0.97 -35.06 34.28 217.77 -0.25
SATQ -1.36 10.18 -0.13 0.91 -45.17 42.44 347.59 -0.74

Residual Standard Error = 1.22 with 2 degrees of freedom

Multiple Regression
R R2 Ruw R2uw Shrunken R2 SE of R2 overall F df1 df2 p
education 0.64 0.4 0.6 0.37 -0.49 0.09 0.45 3 2 0.744

DV = age
slope se t p lower.ci upper.ci VIF Vy.x
(Intercept) 0.00 0.48 0.00 1.00 -2.07 2.07 1.00 0.00
ACT 3.01 15.43 0.20 0.86 -63.39 69.42 1029.17 -2.16
SATV -1.87 7.10 -0.26 0.82 -32.42 28.67 217.77 1.34
SATQ -1.99 8.97 -0.22 0.85 -40.58 36.60 347.59 1.36

Residual Standard Error = 1.08 with 2 degrees of freedom

Multiple Regression
R R2 Ruw R2uw Shrunken R2 SE of R2 overall F df1 df2 p
age 0.73 0.54 0.72 0.52 -0.16 0.08 0.77 3 2 0.606

Various estimates of between set correlations

Squared Canonical Correlations
[1] 1.000 0.988 0.013
Chisq of canonical correlations
[1] 31.020 4.433 0.013

Average squared canonical correlation = 0.67

Cohen's Set Correlation R2 = 1
Shrunken Set Correlation R2 = 1
F and df of Cohen's Set Correlation -Inf 12 -12.94
Unweighted correlation between the two sets = 0.46

Note that the lmCor analysis also reports the amount of shared variance between the predictor set
and the criterion (dependent) set. This set correlation is symmetric. That is, the R2 is the same
independent of the direction of the relationship.

9 Simulation functions
It is particularly helpful, when trying to understand psychometric concepts, to be able to generate
sample data sets that meet certain specifications. By knowing “truth" it is possible to see how
well various algorithms can capture it. Several of the sim functions create artificial data sets with
known structures.
A number of functions in the psych package will generate simulated data. These functions include
sim for a factor simplex, and sim.simplex for a data simplex, sim.circ for a circumplex
structure, sim.congeneric for a one factor factor congeneric model, sim.dichot to simu-
late dichotomous items, sim.hierarchical to create a hierarchical factor model, sim.item

74
is a more general item simulation, sim.minor to simulate major and minor factors, sim.omega
to test various examples of omega, sim.parallel to compare the efficiency of various ways of
determining the number of factors, sim.rasch to create simulated rasch data, sim.irt to create
general 1 to 4 parameter IRT data by calling sim.npl 1 to 4 parameter logistic IRT or sim.npn
1 to 4 paramater normal IRT, sim.structural a general simulation of structural models, and
sim.anova for ANOVA and lm simulations, and sim.vss. Some of these functions are sep-
arately documented and are listed here for ease of the help function. See each function for more
detailed help.
sim The default version is to generate a four factor simplex structure over three occasions, al-
though more general models are possible.
sim.simple Create major and minor factors. The default is for 12 variables with 3 major factors
and 6 minor factors.
sim.structure To combine a measurement and structural model into one data matrix. Useful
for understanding structural equation models.
sim.hierarchical To create data with a hierarchical (bifactor) structure.
sim.congeneric To create congeneric items/tests for demonstrating classical test theory. This
is just a special case of sim.structure.
sim.circ To create data with a circumplex structure.
sim.item To create items that either have a simple structure or a circumplex structure.
sim.dichot Create dichotomous item data with a simple or circumplex structure.
sim.rasch Simulate a 1 parameter logistic (Rasch) model.
sim.irt Simulate a 2 parameter logistic (2PL) or 2 parameter Normal model. Will also do 3 and
4 PL and PN models.
sim.multilevel Simulate data with different within group and between group correlational
structures.
Some of these functions are described in more detail in the companion vignette: psych for sem.
The default values for sim.structure is to generate a 4 factor, 12 variable data set with a
simplex structure between the factors.
Two data structures that are particular challenges to exploratory factor analysis are the simplex
structure and the presence of minor factors. Simplex structures sim.simplex will typically
occur in developmental or learning contexts and have a correlation structure of r between adjacent
variables and rn for variables n apart. Although just one latent variable (r) needs to be estimated,
the structure will have nvar-1 factors.

75
Many simulations of factor structures assume that except for the major factors, all residuals are
normally distributed around 0. An alternative, and perhaps more realistic situation, is that the there
are a few major (big) factors and many minor (small) factors. The challenge is thus to identify the
major factors. sim.minor generates such structures. The structures generated can be thought of
as having a a major factor structure with some small correlated residuals.
Although coefficient ωh is a very useful indicator of the general factor saturation of a unifactorial
test (one with perhaps several sub factors), it has problems with the case of multiple, independent
factors. In this situation, one of the factors is labelled as “general” and the omega estimate is too
large. This situation may be explored using the sim.omega function.
The four irt simulations, sim.rasch, sim.irt, sim.npl and sim.npn, simulate dichoto-
mous items following the Item Response model. sim.irt just calls either sim.npl (for logistic
models) or sim.npn (for normal models) depending upon the specification of the model.
The logistic model is
ζj −γj
P(x|θi , δ j , γ j , ζ j ) = γ j + . (4)
1 + eα j (δ j −θi

where γ is the lower asymptote or guessing parameter, ζ is the upper asymptote (normally 1),
α j is item discrimination and δ j is item difficulty. For the 1 Paramater Logistic (Rasch) model,
gamma=0, zeta=1, alpha=1 and item difficulty is the only free parameter to specify.
(Graphics of these may be seen in the demonstrations for the logistic function.)
The normal model (irt.npn calculates the probability using pnorm instead of the logistic func-
tion used in irt.npl, but the meaning of the parameters are otherwise the same. With the a = α
parameter = 1.702 in the logiistic model the two models are practically identical.

10 Graphical Displays
Many of the functions in the psych package include graphic output and examples have been shown
in the previous figures. After running fa, iclust, omega, irt.fa, plotting the resulting ob-
ject is done by the plot.psych function as well as specific diagram functions. e.g., (but not
shown)
f3 <- fa(Thurstone,3)
plot(f3)
fa.diagram(f3)
c <- iclust(Thurstone)
plot(c) #a pretty boring plot
iclust.diagram(c) #a better diagram
c3 <- iclust(Thurstone,3)
plot(c3) #a more interesting plot
data(bfi)
e.irt <- irt.fa(bfi[11:15])
plot(e.irt)

76
ot <- omega(Thurstone)
plot(ot)
omega.diagram(ot)

The ability to show path diagrams to represent factor analytic and structural models is discussed
in somewhat more detail in the accompanying vignette, psych for sem. Basic routines to draw
path diagrams are included in the dia.rect and accompanying functions. These are used by the
fa.diagram, structure.diagram and iclust.diagram functions.

11 Converting output to APA style tables using LATEX

Although for most purposes, using the Sweave or KnitR packages produces clean output, some pre-
fer output pre formatted for APA style tables. This can be done using the xtable package for almost
anything, but there are a few simple functions in psych for the most common tables. fa2latex
will convert a factor analysis or components analysis output to a LATEXtable, cor2latex will take
a correlation matrix and show the lower (or upper diagonal), irt2latex converts the item statis-
tics from the irt.fa function to more convenient LATEXoutput, and finally, df2latex converts
a generic data frame to LATEX.
An example of converting the output from fa to LATEXappears in Table 10.

Table 10: fa2latex

A factor analysis table from the psych package in R
Variable MR1 MR2 MR3 h2 u2 com
Sentences 0.91 -0.04 0.04 0.82 0.18 1.01
Vocabulary 0.89 0.06 -0.03 0.84 0.16 1.01
Sent.Completion 0.83 0.04 0.00 0.73 0.27 1.00
First.Letters 0.00 0.86 0.00 0.73 0.27 1.00
4.Letter.Words -0.01 0.74 0.10 0.63 0.37 1.04
Suffixes 0.18 0.63 -0.08 0.50 0.50 1.20
Letter.Series 0.03 -0.01 0.84 0.72 0.28 1.00
Pedigrees 0.37 -0.05 0.47 0.50 0.50 1.93
Letter.Group -0.06 0.21 0.64 0.53 0.47 1.23

SS loadings 2.64 1.86 1.5

MR1 1.00 0.59 0.54

MR2 0.59 1.00 0.52
MR3 0.54 0.52 1.00

77
> xlim=c(0,10)
> ylim=c(0,10)
> plot(NA,xlim=xlim,ylim=ylim,main="Demonstration of dia functions",axes=FALSE,xlab="",ylab="")
> ul <- dia.rect(1,9,labels="upper left",xlim=xlim,ylim=ylim)
> ll <- dia.rect(1,3,labels="lower left",xlim=xlim,ylim=ylim)
> lr <- dia.ellipse(9,3,"lower right",xlim=xlim,ylim=ylim,e.size=.09)
> ur <- dia.ellipse(7,9,"upper right",xlim=xlim,ylim=ylim,e.size=.1)
> ml <- dia.ellipse(3,6,"middle left",xlim=xlim,ylim=ylim,e.size=.1)
> mr <- dia.ellipse(7,6,"middle right",xlim=xlim,ylim=ylim,e.size=.08)
> bl <- dia.ellipse(1,1,"bottom left",xlim=xlim,ylim=ylim,e.size=.08)
> br <- dia.rect(9,1,"bottom right",xlim=xlim,ylim=ylim)
> dia.arrow(from=lr,to=ul,labels="right to left")
> dia.arrow(from=ul,to=ur,labels="left to right")
> dia.curved.arrow(from=lr,to=ll$right,labels ="right to left")
NULL
> dia.curved.arrow(to=ur,from=ul$right,labels ="left to right")
NULL
> dia.curve(ll$top,ul$bottom,"double",-1) #for rectangles, specify where to point
> dia.curved.arrow(mr,ur,"up") #but for ellipses, just point to it.
NULL
> dia.curve(ml,mr,"across")
> dia.curved.arrow(ur,lr,"top down")
NULL
> dia.curved.arrow(br$top,lr$bottom,"up")
NULL
> dia.curved.arrow(bl,br,"left to right")
NULL
> dia.arrow(bl$top,ll$bottom)
> dia.curved.arrow(ml,ll$top,scale=-1)
NULL
> dia.curved.arrow(mr,lr$top)
NULL

Demonstration of dia functions

left to right
upper left left to right upper right

up
across

double middle left right to left middle right top down

right to left
lower left lower right

up
left to right
bottom left bottom right

78

Figure 22: The basic graphic capabilities of the dia functions are shown in this figure.
12 Miscellaneous functions
A number of functions have been developed for some very specific problems that don’t fit into any
other category. The following is an incomplete list. Look at the Index for psych for a list of all of
the functions.
block.random Creates a block randomized structure for n independent variables. Useful for
teaching block randomization for experimental design.
df2latex is useful for taking tabular output (such as a correlation matrix or that of describe
and converting it to a LATEX table. May be used when Sweave is not convenient.
cor2latex Will format a correlation matrix in APA style in a LATEX table. See also fa2latex
and irt2latex.
cosinor One of several functions for doing circular statistics. This is important when studying
mood effects over the day which show a diurnal pattern. See also circadian.mean,
circadian.cor and circadian.linear.cor for finding circular means, circular
correlations, and correlations of circular with linear data.
fisherz Convert a correlation to the corresponding Fisher z score.
geometric.mean also harmonic.mean find the appropriate mean for working with different
kinds of data.
ICC and cohen.kappa are typically used to find the reliability for raters.
headtail combines the head and tail functions to show the first and last lines of a data set
or output.
topBottom Same as headtail. Combines the head and tail functions to show the first and last
lines of a data set or output, but does not add ellipsis between.
mardia calculates univariate or multivariate (Mardia’s test) skew and kurtosis for a vector, ma-
trix, or data.frame
p.rep finds the probability of replication for an F, t, or r and estimate effect size.
partial.r partials a y set of variables out of an x set and finds the resulting partial correlations.
(See also lmCor.)
rangeCorrection will correct correlations for restriction of range.
reverse.code will reverse code specified items. Done more conveniently in most psych func-
tions, but supplied here as a helper function when using other packages.
superMatrix Takes two or more matrices, e.g., A and B, and combines them into a “Super
matrix” with A on the top left, B on the lower right, and 0s for the other two quadrants. A

79
 useful trick when forming complex keys, or when forming example problems.

13 Data sets
A number of data sets for demonstrating psychometric techniques are included in the psych pack-
age. These include six data sets showing a hierarchical factor structure (five cognitive examples,
Thurstone, Thurstone.33, Holzinger, Bechtoldt.1, Bechtoldt.2, and one from
health psychology Reise). One of these (Thurstone) is used as an example in the sem package
as well as McDonald (1999). The original data are from Thurstone and Thurstone (1941) and rean-
alyzed by Bechtoldt (1961). Personality item data representing five personality factors on 25 items
(bfi) or 13 personality inventory scores (epi.bfi), and 14 multiple choice iq items (iqitems).
The vegetables example has paired comparison preferences for 9 vegetables. This is an exam-
ple of Thurstonian scaling used by Guilford (1954) and Nunnally (1967). Other data sets include
cubits, peas, and heights from Galton.
Thurstone Holzinger-Swineford (1937) introduced the bifactor model of a general factor and un-
correlated group factors. The Holzinger correlation matrix is a 14 * 14 matrix from their
paper. The Thurstone correlation matrix is a 9 * 9 matrix of correlations of ability items.
The Reise data set is 16 * 16 correlation matrix of mental health items. The Bechtholdt data
sets are both 17 x 17 correlation matrices of ability tests.
bfi 25 personality self report items taken from the International Personality Item Pool (ipip.ori.org)
were included as part of the Synthetic Aperture Personality Assessment (SAPA) web based
personality assessment project. The data from 2800 subjects are included here as a demon-
stration set for scale construction, factor analysis and Item Response Theory analyses.
sat.act Self reported scores on the SAT Verbal, SAT Quantitative and ACT were collected as part
of the Synthetic Aperture Personality Assessment (SAPA) web based personality assessment
project. Age, gender, and education are also reported. The data from 700 subjects are in-
cluded here as a demonstration set for correlation and analysis.
epi.bfi A small data set of 5 scales from the Eysenck Personality Inventory, 5 from a Big 5 inven-
tory, a Beck Depression Inventory, and State and Trait Anxiety measures. Used for demon-
strations of correlations, regressions, graphic displays.
epiR The EPI was given twice to 474 participants. This is a useful data set for exploring test-retest
reliability,
sai, msqR 20 anxiety items and 75 mood items were given at least twice to 3032 participants.
These are useful for understanding reliability structures.
iq 14 multiple choice ability items were included as part of the Synthetic Aperture Personality
Assessment (SAPA) web based personality assessment project. The data from 1000 subjects

80
 are included here as a demonstration set for scoring multiple choice inventories and doing
basic item statistics.
galton Two of the earliest examples of the correlation coefficient were Francis Galton’s data sets
on the relationship between mid parent and child height and the similarity of parent genera-
tion peas with child peas. galton is the data set for the Galton height. peas is the data set
Francis Galton used to ntroduce the correlation coefficient with an analysis of the similarities
of the parent and child generation of 700 sweet peas.
Dwyer Dwyer (1937) introduced a method for factor extension (see fa.extension that finds
loadings on factors from an original data set for additional (extended) variables. This data
set includes his example.
miscellaneous cities is a matrix of airline distances between 11 US cities and may be used
for demonstrating multiple dimensional scaling. vegetables is a classic data set for
demonstrating Thurstonian scaling and is the preference matrix of 9 vegetables from Guil-
ford (1954). Used by Guilford (1954); Nunnally (1967); Nunnally and Bernstein (1994), this
data set allows for examples of basic scaling techniques.

14 Development version and a users guide

The most recent development version is available as a source file at the repository maintained at
https://personality-project.org/r. That version will have removed the most re-
cently discovered bugs (but perhaps introduced other, yet to be discovered ones). To download that
version, go to the repository https://personality-project.org/r/src/contrib/
and wander around. For a Mac and PC this version can be installed directly using the “other repos-
itory" option in the package installer.
> install.packages("psych", repos="https://personality-project.org/r", type="sou
Although the individual help pages for the psych package are available as part of R and may be
accessed directly (e.g. ?psych) , the full manual for the psych package is also available as a pdf
at https://personality-project.org/r/psych_manual.pdf
News and a history of changes are available in the NEWS and CHANGES files in the source files.
To view the most recent news,
> news(Version > "2.3.3", package="psych")

15 Psychometric Theory
The psych package has been developed to help psychologists do basic research. Many of the
functions were developed to supplement a book (https://personality-project.org/

81
r/book An introduction to Psychometric Theory with Applications in R (Revelle, prep) More
information about the use of some of the functions may be found in the book .
For more extensive discussion of the use of psych in particular and R in general, consult https:
//personality-project.org/r/r.guide.html A short guide to R.

16 SessionInfo
This document was prepared using the following settings.
> sessionInfo()

R version 4.3.1 (2023-06-16)

Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS Ventura 13.5.2

Matrix products: default

BLAS: /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRlapack.dylib; LAPACK version 3.11.0

locale:
[1] C

time zone: America/Chicago

tzcode source: internal

attached base packages:

[1] stats graphics grDevices utils datasets methods base

other attached packages:

[1] GPArotation_2023.3-1 psychTools_2.3.9 psych_2.3.9

loaded via a namespace (and not attached):

[1] compiler_4.3.1 parallel_4.3.1 tools_4.3.1 foreign_0.8-84 lavaan_0.6-16 nlme_3.1-162 mnormt_2.1.1 grid_4.3.1 pbivnorm_0.6.0 knitr_
[11] xfun_0.38 stats4_4.3.1 lattice_0.21-8 quadprog_1.5-8

82
References
Bechtoldt, H. (1961). An empirical study of the factor analysis stability hypothesis. Psychometrika,
26(4):405–432.
Blashfield, R. K. (1980). The growth of cluster analysis: Tryon, Ward, and Johnson. Multivariate
Behavioral Research, 15(4):439 – 458.
Blashfield, R. K. and Aldenderfer, M. S. (1988). The methods and problems of cluster analysis. In
Nesselroade, J. R. and Cattell, R. B., editors, Handbook of multivariate experimental psychology
(2nd ed.), pages 447–473. Plenum Press, New York, NY.
Bliese, P. D. (2009). Multilevel Modeling in R (2.3) A Brief Introduction to R, the multilevel
package and the nlme package.
Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research,
1(2):245–276.
Cattell, R. B. (1978). The scientific use of factor analysis. Plenum Press, New York.
Cohen, J. (1982). Set correlation as a general multivariate data-analytic method. Multivariate
Behavioral Research, 17(3).
Cohen, J., Cohen, P., West, S. G., and Aiken, L. S. (2003). Applied multiple regression/correlation
analysis for the behavioral sciences. L. Erlbaum Associates, Mahwah, N.J., 3rd ed edition.
Cooksey, R. and Soutar, G. (2006). Coefficient beta and hierarchical item clustering - an analytical
procedure for establishing and displaying the dimensionality and homogeneity of summated
scales. Organizational Research Methods, 9:78–98.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16:297–
334.
Dwyer, P. S. (1937). The determination of the factor loadings of a given test from the known factor
loadings of other tests. Psychometrika, 2(3):173–178.
Everitt, B. (1974). Cluster analysis. John Wiley & Sons, Cluster analysis. 122 pp. Oxford, England.
Fox, J., Nie, Z., and Byrnes, J. (2013). sem: Structural Equation Models. R package version 3.1-3.
Grice, J. W. (2001). Computing and evaluating factor scores. Psychological Methods, 6(4):430–
450.
Guilford, J. P. (1954). Psychometric Methods. McGraw-Hill, New York, 2nd edition.
Guttman, L. (1945). A basis for analyzing test-retest reliability. Psychometrika, 10(4):255–282.
Hartigan, J. A. (1975). Clustering Algorithms. John Wiley & Sons, Inc., New York, NY, USA.

83
Henry, D. B., Tolan, P. H., and Gorman-Smith, D. (2005). Cluster analysis in family psychology
research. Journal of Family Psychology, 19(1):121–132.
Holzinger, K. and Swineford, F. (1937). The bi-factor method. Psychometrika, 2(1):41–54.
Horn, J. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika,
30(2):179–185.
Horn, J. L. and Engstrom, R. (1979). Cattell’s scree test in relation to Bartlett’s chi-square test
and other observations on the number of factors problem. Multivariate Behavioral Research,
14(3):283–300.
Jennrich, R. and Bentler, P. (2011). Exploratory bi-factor analysis. Psychometrika, 76(4):537–549.
Jensen, A. R. and Weng, L.-J. (1994). What is a good g? Intelligence, 18(3):231–258.
Kaiser, H. F. and Caffrey, J. (1965). Alpha factor analysis. Psychometrika, 30(1):1–14.
Loehlin, J. C. and Beaujean, A. (2017). Latent variable models: an introduction to factor, path,
and structural equation analysis. Routledge, Mahwah, N.J., 5th edition.
Loevinger, J., Gleser, G., and DuBois, P. (1953). Maximizing the discriminating power of a
multiple-score test. Psychometrika, 18(4):309–317.
MacCallum, R. C., Browne, M. W., and Cai, L. (2007). Factor analysis models as approximations.
In Cudeck, R. and MacCallum, R. C., editors, Factor analysis at 100: Historical developments
and future directions, pages 153–175. Lawrence Erlbaum Associates Publishers, Mahwah, NJ.
Martinent, G. and Ferrand, C. (2007). A cluster analysis of precompetitive anxiety: Relationship
with perfectionism and trait anxiety. Personality and Individual Differences, 43(7):1676–1686.
McDonald, R. P. (1999). Test theory: A unified treatment. L. Erlbaum Associates, Mahwah, N.J.
Mun, E. Y., von Eye, A., Bates, M. E., and Vaschillo, E. G. (2008). Finding groups using model-
based cluster analysis: Heterogeneous emotional self-regulatory processes and heavy alcohol
use risk. Developmental Psychology, 44(2):481–495.
Nunnally, J. C. (1967). Psychometric theory. McGraw-Hill, New York,.
Nunnally, J. C. and Bernstein, I. H. (1994). Psychometric theory. McGraw-Hill, New York„ 3rd
edition.
Pedhazur, E. (1997). Multiple regression in behavioral research: explanation and prediction. Har-
court Brace College Publishers.
Revelle, W. (1979). Hierarchical cluster-analysis and the internal structure of tests. Multivariate
Behavioral Research, 14(1):57–74.

84
Revelle, W. (2023). psych: Procedures for Personality and Psychological Research. Northwestern
University, Evanston, https://cran.r-project.org/web/packages=psych. R package version 2.3.5.
Revelle, W. (in prep). An introduction to psychometric theory with applications in R. Springer.
Revelle, W., Condon, D., and Wilt, J. (2011). Methodological advances in differential psychology.
In Chamorro-Premuzic, T., Furnham, A., and von Stumm, S., editors, Handbook of Individual
Differences, chapter 2, pages 39–73. Wiley-Blackwell.
Revelle, W. and Condon, D. M. (2018). Reliability. In Irwing, P., Booth, T., and Hughes, D.,
editors, Wiley-Blackwell Handbook of Psychometric Testing. Wiley-Blackwell.
Revelle, W. and Rocklin, T. (1979). Very Simple Structure - alternative procedure for estimating
the optimal number of interpretable factors. Multivariate Behavioral Research, 14(4):403–414.
Revelle, W., Wilt, J., and Rosenthal, A. (2010). Individual differences in cognition: New methods
for examining the personality-cognition link. In Gruszka, A., Matthews, G., and Szymura, B.,
editors, Handbook of Individual Differences in Cognition: Attention, Memory and Executive
Control, chapter 2, pages 27–49. Springer, New York, N.Y.
Revelle, W. and Wilt, J. A. (2017). Analyzing dynamic data: a tutorial. Personality and Individual
Differences, (in press).
Revelle, W. and Zinbarg, R. E. (2009). Coefficients alpha, beta, omega and the glb: comments on
Sijtsma. Psychometrika, 74(1):145–154.
Schmid, J. J. and Leiman, J. M. (1957). The development of hierarchical factor solutions. Psy-
chometrika, 22(1):83–90.
Shrout, P. and Lane, S. P. (2012). Psychometrics. In Handbook of research methods for studying
daily life. Guilford Press.
Shrout, P. E. and Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability.
Psychological Bulletin, 86(2):420–428.
Sneath, P. H. A. and Sokal, R. R. (1973). Numerical taxonomy: the principles and practice of
numerical classification. A Series of books in biology. W. H. Freeman, San Francisco.
Sokal, R. R. and Sneath, P. H. A. (1963). Principles of numerical taxonomy. A Series of books in
biology. W. H. Freeman, San Francisco.
Spearman, C. (1904). The proof and measurement of association between two things. The Ameri-
can Journal of Psychology, 15(1):72–101.
Thorburn, W. M. (1918). The myth of Occam’s razor. Mind, 27:345–353.
Thurstone, L. L. and Thurstone, T. G. (1941). Factorial studies of intelligence. The University of
Chicago press, Chicago, Ill.

85
Tryon, R. C. (1935). A theory of psychological components–an alternative to "mathematical fac-
tors.". Psychological Review, 42(5):425–454.
Tryon, R. C. (1939). Cluster analysis. Edwards Brothers, Ann Arbor, Michigan.
Velicer, W. (1976). Determining the number of components from the matrix of partial correlations.
Psychometrika, 41(3):321–327.
Zinbarg, R. E., Revelle, W., Yovel, I., and Li, W. (2005). Cronbach’s α , Revelle’s β , and McDon-
ald’s ωH : Their relations with each other and two alternative conceptualizations of reliability.
Psychometrika, 70(1):123–133.
Zinbarg, R. E., Yovel, I., Revelle, W., and McDonald, R. P. (2006). Estimating generalizability to a
latent variable common to all of a scale’s indicators: A comparison of estimators for ωh . Applied
Psychological Measurement, 30(2):121–144.

86
Index
agnes, 22 edit, 4
alpha, 6, 7, 41, 42, 44, 56 epi.bfi, 80
alpha factor analysis, 10 eRm, 64
error.bars, 7
Bechtoldt.1, 80 esem, 37
Bechtoldt.2, 80 esem.diagram, 37
bestScales, 58
bfi, 51–53, 58, 69, 80 FA, 10, 15
bi.bars, 7 fa, 7, 8, 10, 12, 14, 15, 19, 29, 56, 76, 77
bifactor, 7, 14, 20 fa.diagram, 8, 11, 18, 36, 77
biquartimin, 14 fa.extension, 36, 81
block.random, 79 fa.graph, 11
fa.irt, 64
circadian.cor, 79 fa.multi, 7
circadian.linear.cor, 79 fa.parallel, 6, 7, 11, 31, 34, 56
circadian.mean, 79 fa.parallel.poly, 34
circular statistics, 79 fa.plot, 11, 15
cities, 81 fa.poly, 10, 29
cluster analysis, 10, 22 fa2latex, 77, 79
cluster scores, 10 faBy, 72
cluster.cor, 40, 51, 53 factanal, 11, 12
cluster.loadings, 51, 55 factor analysis, 7, 10, 15
cohen.kappa, 79 factor extension, 34
common variance, 15 factor scores, 10
component scores, 10 factor.congruence, 11, 31
congruence coefficient, 29 factor.minres, 8, 11, 12, 14
cor2, 70 factor.pa, 8, 11, 12, 14, 45
cor2latex, 77, 79 factor.wls, 8, 11
corPlot, 8, 9 factors, 10
cosinor, 79 fisherz, 79
ctv, 8
cubits, 80 galton, 81
generalized least squares, 7
describe, 4, 7, 9, 56, 79 generalized least squares factor analysis, 10
describeBy, 4, 7, 9 geometric.mean, 79
df2latex, 77, 79 GPArotation, 8, 12, 14, 22
dia.rect, 77 guttman, 7, 46, 51
diagram, 9
harmonic.mean, 79

87
hclust, 22 mlArrange, 7
head, 79 mlPlot, 7
headtail, 79 mlr, 7, 72
heights, 80 multi.hist, 7
het.diagram, 8 multilevel, 71
hierarchical cluster analysis, 22 multilevel.reliability, 7
Holzinger, 80 multilevelReliablity, 72

ICC, 7, 79 nfactors, 7, 11
iclust, 7, 11, 22, 23, 28, 40, 45, 56, 76 nlme, 71
iclust.diagram, 8, 77
Index, 79 oblimin, 14
introduction to psychometric theory with appli- omega, 7, 8, 20, 29, 40, 44, 45, 64, 76
cations in R, 8 omegaSem, 44, 49
iqitems, 80 outlier, 4
irt.fa, 7, 59, 62, 64, 68, 76, 77 p.rep, 79
irt.npl, 76 pairs.panels, 4, 7, 8, 53
irt.npn, 76 pairs.panesl, 9
irt.responses, 56, 57 parallels, 64
irt2latex, 77, 79 paran, 34
item.lookup, 58 partial.r, 79
keys, 42 PCA, 10, 15
KnitR, 77 pca, 7
peas, 80, 81
lavaan, 34, 44, 72 plot, 15, 17
library, 9 plot.irt, 8
lmCor, 5, 8, 9, 34, 56, 72–74, 79 plot.poly, 8
logistic, 60 plot.psych, 76
lowerCor, 5 pnorm, 76
ltm, 64 polychoric, 7, 23, 24, 34, 59
principal, 6–8, 11, 14, 15, 45
make.keys, 52 principal axis, 7
MAP, 7, 11, 32 principal axis factor analysis, 10
mardia, 79 principal components, 10
maximum likelihood, 7 principal components analysis, 10, 15
maximum likelihood factor analysis, 10 princomp, 15
mediate, 5, 8 print, 15
mediation, 9 print.psych, 16
min.res, 12 progressBar, 23
minimum residual, 7 Promax, 14, 15
minres factor analysis, 10 psych, 4, 6–10, 12, 22, 52, 64, 76, 77, 79–82

88
quartimax, 14 fa.multi, 7
quartimin, 14 fa.parallel, 6, 7, 11, 31, 34, 56
fa.parallel.poly, 34
R function fa.plot, 11, 15
agnes, 22 fa.poly, 10, 29
alpha, 6, 7, 41, 42, 44, 56 fa2latex, 77, 79
Bechtoldt.1, 80 faBy, 72
Bechtoldt.2, 80 factanal, 11, 12
bestScales, 58 factor.congruence, 11, 31
bfi, 51–53, 58, 69, 80 factor.minres, 8, 11, 12, 14
bi.bars, 7 factor.pa, 8, 11, 12, 14, 45
bifactor, 14, 20 factor.wls, 8, 11
biquartimin, 14 fisherz, 79
block.random, 79 galton, 81
circadian.cor, 79 geometric.mean, 79
circadian.linear.cor, 79 guttman, 7, 46, 51
circadian.mean, 79 harmonic.mean, 79
cities, 81 hclust, 22
cluster.cor, 40, 51, 53 head, 79
cluster.loadings, 51, 55 headtail, 79
cohen.kappa, 79 heights, 80
cor2, 70 het.diagram, 8
cor2latex, 77, 79 Holzinger, 80
corPlot, 8, 9 ICC, 7, 79
cosinor, 79 iclust, 7, 11, 22, 23, 28, 40, 45, 56, 76
cubits, 80 iclust.diagram, 8, 77
describe, 4, 7, 9, 56, 79 iqitems, 80
describeBy, 4, 7, 9 irt.fa, 7, 59, 62, 64, 68, 76, 77
df2latex, 77, 79 irt.npl, 76
dia.rect, 77 irt.npn, 76
edit, 4 irt.responses, 56, 57
epi.bfi, 80 irt2latex, 77, 79
error.bars, 7 item.lookup, 58
esem, 37 library, 9
esem.diagram, 37 lmCor, 5, 8, 9, 34, 56, 72–74, 79
fa, 7, 8, 10, 12, 14, 15, 19, 29, 56, 76, lowerCor, 5
77 make.keys, 52
fa.diagram, 8, 11, 18, 36, 77 MAP, 7, 11, 32
fa.extension, 36, 81 mardia, 79
fa.graph, 11 mediate, 5, 8
fa.irt, 64 mediation, 9

89
mlArrange, 7 circadian.cor, 79
mlPlot, 7 circadian.linear.cor, 79
mlr, 7, 72 circadian.mean, 79
multi.hist, 7 cities, 81
multilevel.reliability, 7 cluster.cor, 40, 51, 53
multilevelReliablity, 72 cluster.loadings, 51, 55
nfactors, 7, 11 cohen.kappa, 79
oblimin, 14 cor2, 70
omega, 7, 8, 20, 29, 40, 44, 45, 64, 76 cor2latex, 77, 79
omegaSem, 44, 49 corPlot, 8, 9
outlier, 4 cosinor, 79
p.rep, 79 cubits, 80
pairs.panels, 4, 7, 8, 53 describe, 4, 7, 9, 56, 79
pairs.panesl, 9 describeBy, 4, 7, 9
paran, 34 df2latex, 77, 79
partial.r, 79 dia.rect, 77
pca, 7 epi.bfi, 80
peas, 80, 81 error.bars, 7
plot, 15, 17 esem, 37
plot.irt, 8 esem.diagram, 37
plot.poly, 8 fa, 7, 8, 10, 12, 14, 15, 19, 29, 56, 76,
plot.psych, 76 77
pnorm, 76 fa.diagram, 8, 11, 18, 36, 77
polychoric, 7, 23, 24, 59 fa.extension, 36, 81
principal, 6–8, 11, 14, 15, 45 fa.graph, 11
princomp, 15 fa.irt, 64
print, 15 fa.multi, 7
print.psych, 16 fa.parallel, 6, 7, 11, 31, 34, 56
progressBar, 23 fa.parallel.poly, 34
Promax, 14, 15 fa.plot, 11, 15
psych, 81 fa.poly, 10, 29
psych package fa2latex, 77, 79
alpha, 6, 7, 41, 42, 44, 56 faBy, 72
Bechtoldt.1, 80 factor.congruence, 11, 31
Bechtoldt.2, 80 factor.minres, 8, 11, 12, 14
bestScales, 58 factor.pa, 8, 11, 12, 14, 45
bfi, 51–53, 58, 69, 80 factor.wls, 8, 11
bi.bars, 7 fisherz, 79
bifactor, 14, 20 galton, 81
biquartimin, 14 geometric.mean, 79
block.random, 79 guttman, 7, 46, 51

90
harmonic.mean, 79 plot.psych, 76
headtail, 79 polychoric, 7, 23, 24, 59
heights, 80 principal, 6–8, 11, 14, 15, 45
het.diagram, 8 print, 15
Holzinger, 80 print.psych, 16
ICC, 7, 79 progressBar, 23
iclust, 7, 11, 22, 23, 28, 40, 45, 56, 76 Promax, 14, 15
iclust.diagram, 8, 77 psych, 81
iqitems, 80 rangeCorrection, 79
irt.fa, 7, 59, 62, 64, 68, 76, 77 read.clipboard, 4, 7, 9
irt.npl, 76 read.clipboard.tab, 4
irt.npn, 76 read.file, 4, 7, 9
irt.responses, 56, 57 Reise, 80
irt2latex, 77, 79 response.frequencies, 56
item.lookup, 58 reverse.code, 79
lmCor, 5, 8, 9, 34, 56, 72–74, 79 sat.act, 73
lowerCor, 5 scatter.hist, 7
make.keys, 52 schmid, 7, 8, 45
MAP, 7, 11, 32 score.multiple.choice, 7, 55, 56, 64
mardia, 79 scoreIrt, 67
mediate, 5, 8 scoreIrt.1pl, 69
mediation, 9 scoreIrt.2pl, 69
mlArrange, 7 scoreItems, 6–8, 40, 44, 45, 51–53, 55,
mlPlot, 7 56
mlr, 7, 72 scrub, 4, 9
multi.hist, 7 sim, 74, 75
multilevel.reliability, 7 sim.anova, 75
multilevelReliablity, 72 sim.circ, 74, 75
nfactors, 7, 11 sim.congeneric, 74, 75
omega, 7, 8, 20, 29, 40, 44, 45, 64, 76 sim.dichot, 74, 75
omegaSem, 44, 49 sim.hierarchical, 74, 75
outlier, 4 sim.irt, 75, 76
p.rep, 79 sim.item, 74, 75
pairs.panels, 4, 7, 8, 53 sim.minor, 32, 75, 76
pairs.panesl, 9 sim.multilevel, 71, 75
partial.r, 79 sim.npl, 75, 76
pca, 7 sim.npn, 75, 76
peas, 80, 81 sim.omega, 75, 76
plot, 17 sim.parallel, 75
plot.irt, 8 sim.rasch, 75, 76
plot.poly, 8 sim.simple, 75

91
 sim.simplex, 74, 75 sim.anova, 75
sim.structural, 75 sim.circ, 74, 75
sim.structure, 32, 75 sim.congeneric, 74, 75
sim.vss, 75 sim.dichot, 74, 75
smc, 11 sim.hierarchical, 74, 75
splitHalf, 51 sim.irt, 75, 76
StatsBy, 7 sim.item, 74, 75
statsBy, 7, 71, 72 sim.minor, 32, 75, 76
statsBy.boot, 71 sim.multilevel, 71, 75
statsBy.boot.summary, 71 sim.npl, 75, 76
structure.diagram, 8, 77 sim.npn, 75, 76
superMatrix, 79 sim.omega, 75, 76
target.rot, 14 sim.parallel, 75
tetrachoric, 7, 24, 59 sim.rasch, 75, 76
Thurstone, 80 sim.simple, 75
Thurstone.33, 80 sim.simplex, 74, 75
topBottom, 79 sim.structural, 75
varimin, 14 sim.structure, 32, 75
vegetables, 80, 81 sim.vss, 75
vss, 6, 7, 11, 32, 56 smc, 11
withinBetween, 71 splitHalf, 51
quartimax, 14 StatsBy, 7
quartimin, 14 statsBy, 7, 71, 72
rangeCorrection, 79 statsBy.boot, 71
read.clipboard, 4, 7, 9 statsBy.boot.summary, 71
read.clipboard.tab, 4 structure.diagram, 8, 77
read.file, 4, 7, 9 superMatrix, 79
Reise, 80 tail, 79
response.frequencies, 56 target.rot, 14
reverse.code, 79 tetrachoric, 7, 24, 59
Rgraphviz, 11 Thurstone, 80
sat.act, 73 Thurstone.33, 80
scatter.hist, 7 topBottom, 79
schmid, 7, 8, 45 varimax, 14
score.multiple.choice, 7, 55, 56, 64 varimin, 14
scoreIrt, 67 vegetables, 80, 81
scoreIrt.1pl, 69 vss, 6, 7, 11, 32, 56
scoreIrt.2pl, 69 withinBetween, 71
scoreItems, 6–8, 40, 44, 45, 51–53, 55, 56 R package
scrub, 4, 9 ctv, 8
sim, 74, 75 eRm, 64

92
 GPArotation, 8, 12, 14, 22 sim.congeneric, 74, 75
KnitR, 77 sim.dichot, 74, 75
lavaan, 34, 44, 72 sim.hierarchical, 74, 75
ltm, 64 sim.irt, 75, 76
multilevel, 71 sim.item, 74, 75
nlme, 71 sim.minor, 32, 75, 76
parallels, 64 sim.multilevel, 71, 75
paran, 34 sim.npl, 75, 76
psych, 4, 6–10, 12, 22, 52, 64, 76, 77, 79– sim.npn, 75, 76
82 sim.omega, 75, 76
Rgraphviz, 9 sim.parallel, 75
sem, 8, 34, 49, 80 sim.rasch, 75, 76
stats, 12, 15 sim.simple, 75
Sweave, 77 sim.simplex, 74, 75
xtable, 77 sim.structural, 75
rangeCorrection, 79 sim.structure, 32, 75
read.clipboard, 4, 7, 9 sim.vss, 75
read.clipboard.tab, 4 Singular Value Decomposition, 10
read.file, 4, 7, 9 smc, 11
Reise, 80 splitHalf, 51
response.frequencies, 56 stats, 12, 15
reverse.code, 79 StatsBy, 7
Rgraphviz, 9, 11 statsBy, 7, 71, 72
rooted dendritic structure, 22 statsBy.boot, 71
statsBy.boot.summary, 71
SAPA, 55, 64–67, 80 structure.diagram, 8, 77
sat.act, 73 superMatrix, 79
scatter.hist, 7 Sweave, 77
schmid, 7, 8, 45
Schmid-Leiman, 20 tail, 79
score.multiple.choice, 7, 55, 56, 64 target.rot, 14
scoreIrt, 67 tetrachoric, 7, 24, 59
scoreIrt.1pl, 69 Thurstone, 80
scoreIrt.2pl, 69 Thurstone.33, 80
scoreItems, 6–8, 40, 44, 45, 51–53, 55, 56 topBottom, 79
scrub, 4, 9 tree diagram, 22
sem, 8, 34, 49, 80
set correlation, 34, 72 varimax, 14
sim, 74, 75 varimin, 14
sim.anova, 75 vegetables, 80, 81
sim.circ, 74, 75 vss, 6, 7, 11, 32, 56

93
weighted least squares, 7
weighted least squares factor analysis, 10
withinBetween, 71

xtable, 77

94