See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/235944648

Confirmatory Factor Analysis of the Performance Management Audit

Questionnaire

Article  in  South African Journal of Psychology · December 2000

DOI: 10.1177/008124630003000405

CITATIONS READS

3 72

2 authors, including:

Callie Theron
Stellenbosch University
38 PUBLICATIONS   274 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

The development and empirical testing of an employee engagement structural model containing latent polynomial effects View project

Adverse impact project View project

All content following this page was uploaded by Callie Theron on 21 June 2018.

The user has requested enhancement of the downloaded file.

 Confirmatory factor analysis of the performance management audit questionnaire

Callie C. Theron*
Department of Industrial Psychology, University of Stellenbosch

Hermann H. Spangenberg
Centre For Leadership Studies, Graduate School of Business,University of Stellenbosch

This article reports on the findings of a more sophisticated and demanding re-analysis of a data set collected and
analyzed by Spangenberg and Theron (1997) to examine the validity of a Performance Management audit
questionnaire. L1SREL 8 was used for a confirmatory factor analysis to determine the fit of the measurement
model that guided the construction of the Performance Management Audit Questionnaire (PMAQ). The findings
reported here corroborate the position explicated in Spangenberg and Theron (1997) that t!'le PMAQ provides a
content valid measure of the Performance Management domain as conceptualised by the systems model.

• Towhom correspondence should be addressed

In view of ever increasing global and national competitiveness A second major role of Performance Management is the
which is putting strong pressure on performance excellence and improvement of organisational processes and of team and
cost-effectiveness at all organisational levels, an integrative HR individual performance (Rummier & Brache, 1990). Both Lane
system such as Performance Management has become almost (1994) and Egan (1995) emphasize continuous improvement of
indispensable. Bevan and Thompson (1991) defined the elements employee performance as a major goal of Performance
of Performance Management as being: Management. Lawler (1994) contends that a team approach makes
particular sense in organisations that rely heavily on self-
• A shared vision of organisational objectives, or a mission management teams.
statement, communicated to all employees. A third role is the development of a desired organisational culture.
• Individual Performance Management targets related both McLagan (1993) considers Performance Management as a driving
to operating unit and wider organisational objectives. force in creating a participative culture. The author is probably
• Regular formal reviews of progress towards these targets. correct when she states: "In a real sense, how an organisation
manages performance is its culture" (p.9). Wellins and Schulz
• The review process used to identify employee training, Murphy (1995) describe how Performance Management facilitates
development and reward outcomes. culture change during an organisational reengineering process.
• Evaluation of effectiveness of the whole process and its For example, new value systems, structures and roles translate into
contribution to overall organisational performance to allow new accountabilities and new skills.
changes and improvements to be made.
The fourth and traditional role of Performance Management is to
provide inputs into human resource systems and decisions.
The present turbulent organisational environment mentioned above
However, its increasingly strategic role in culture and performance
is making major demands on all organisational processes and
improvement requires that its human resource ties be weakened,
systems, including Performance Management. This has led to
relative to the more strategic roles.
considerable widening of the scope of Performance Management
with four roles particularly important, namely implementation of When implementing a complex system such as Performance
strategy, developing culture, improving performance, and Management and integrating it with other organisational systems,
providing information for human resources decisions. Facilitating all prerequisites for successful implementation will not be met
the implementation of strategy has become a challenging new role during the initial phases of implementation. Therefore, feedback
for Performance Management (Schneier, Shaw & Beatty, 1991). is required to identity shortcomings. For this purpose
According to the authors, Performance Management facilitates the Spangenberg and Theron (1997) developed the Performance
implementation of business strategy by indicating what to Management Audit Questionnaire (PMAQ), based on a systems
measure, determining appropriate ways to measure, and fixing model. The systems model is described elsewhere (Spangenberg,
accountabilityfor performance according to these measures. 1994a; I994b). See Figure I.

INPUTS

S"eIlogIcDrl_.
t. Corponrte olrategy;
q t.
PROCESSES

Developing organiZational m_on,

go.la and lItrateglc C8p11b1lti..
I :> OUTPUTS
purpooe of PM. 2. Formulating goat. and alignmant at Short Term
2. L_...hlp team and IndMdu.llevela t. Producitan
3. Culture 3. Designing & redle6gnlng ItructUr88 LINKAGES

q
4. Managing performance.t three
q ,.
2. Efllclency
3. Satioloctlon

c>
Internol _holder. '0Y01o ReWlird l)'8tem
4. Manllgement 5. Reviewing performance 2. General Longer Term
5. Employ... 4.
• Stlbllizatlon cI PM
8. Trade union
Perfotm..c. M..~nt
Sy,_
¢:=J •
Career mlnagement
Training .nd deVelopment
5. Organlza1lonll
adaptability and
development
~

I Feeclback I I F.ecIb.ck I I F.edback I

Figure 1. PMAQ systems model

32 S. Afr. J. Psychol. 2000,30 (4)

The actual usage of the PMAQ for this purpose depends, abbreviations of the subscales are indicated in brackets. These are
however, on the assurance that the PMAQ provides an strategic drivers (strat), internal stakeholders (inter), Performance
uncontaminated and comprehensive assessment of the Management system (psystem), clarifying organisational mission
Performance Management domain. Spangenberg and Theron and goals (clarify), formulating team and individual goals, that is,
(1997) reported limited confirmation for the hypothesis that the plan performance (plan), devising structures (design), managing
PMAQ provides a reliable and content valid measure of the performance (rnanagp), reviewing performance (review), reward
Performance Management domain as conceptualised by the systems (reward), linkages (link) and outputs (outputs). A 5-point
systems model (Spangenberg, I994a; 1994b). However, Likert scale, ranging from strongly agree to strongly disagree, was
Spangenberg and Theron (1997) raised points of used in each ofthe subscales.
methodological criticism against their own approach and
consequently undertook to conduct and report on a more Statistical analysis and results
sophisticated form of confirmatory factor analysis to A total of 626 completed Performance Management audit
corroborate (or modify) the position explicated in the initial questionnaires were received, representing a response rate of
report. Specific intentions in this regard were outlined. The 34%. Listwise deletion of missing cases was performed, thus
purpose of this study is thus to honour those commitments, to reducing the effective sample size to 522 (29%). This
reanalyse the original data and to corroborate (or modify) the represents a ratio of cases to variables of 5.22 to I which could,
position explicated in the initial report. in terms of the factor analytic guidelines presented by Comrey
(1973), be considered acceptable.
Methodology
Sample Each of the PMAQ subscales, representing a facet of the multi-
The sample consisted of 626 managers and employees from eight dimensional Performance Management domain, was item
South African commercial, industrial and utility organisations. analysed through the SPSS Reliability procedure (SPSS, 1990)
Staff members' eligibility for participation in the survey was based to identify and eliminate items not contributing to an internally
on their competence to assess the current state of Performance consistent description of the facet/dimension in question. In
Management in their respective organisations. total, three items were deleted from the original questionnaire;
one item from the Internal subscale and two items from the
Reward subscale. The item analysis was repeated on the
Developing the measuring instrument
reduced item set. The results of the fmal item analysis are
The development of the Performance Management Audit
reported in Spangenberg and Theron (1997). Only the
Questionnaire (PMAQ) started in 1992, coinciding with the
remaining 100 items were used in the reanalysis of the data.
development of the Problems Questionnaire used to survey South
African middle and senior managers regarding problems Structural equation modelling (SEM) was used to perform a
experienced with the application of Performance Management. confirmatory factor analysis on the data set. The argument
presented by Spangenberg and Theron (1997) in defence of the
Step I: Interviews with Performance Management facilitators. decision to use the Procrustes method of rotation in their
Interviews were conducted with facilitators from 12 organisations. original analysis thus also forms the basis of the justification for
These interviews were relatively unstructured. Nonetheless, two the present analysis. The purpose of both analyses is to explain
questions were asked in all of these interviews: "What problems do the observed inter-item correlation/covariance matrix in terms
you experience with Performance Management?" and "What could of a set of underlying processes that could have 'created' these
we do to make the system work?" A large number of responses to correlations. In contrast to an exploratory approach where no
both questions were elicited. prior assumptions are made about the underlying factor
Step 2: Development of a systems model of Performance structure, a very specific network of processes is in this case
Management. Because of the large number of problems with the hypothesised as an explanation of the observed correlation
system revealed by the interviews, it was decided to develop a matrix. Each of the items comprising the various sub-scales of
systems model of Performance Management. The systems model the PMAQ was written to serve as relatively uncontaminated
would provide a holistic picture of organisational issues and other behavioural indicators of one of the facets of the multi-
forces that negatively impact on the system. dimensional Performance Management domain depicted in
Figure I. The design and construction of the PMAQ thus
Step 3: Developing the Prerequisites Questionnaire. In addition to implies a specific factor structure or measurement model. The
prerequisites elicited during interviews with Performance measurement model underlying the PMAQ is shown in matrix
Management facilitators, items were obtained from two further format as equation I.
sources; namely the Problems Questionnaire described above and
relevant scientific literature. From the Problems Questionnaire, 43
items rated as either important or very serious problems were
considered for inclusion. Altogether 297 items were formulated.
Where:
X is a 100xl-column vector of item scores
A questionnaire entitled Performance Management Prerequisites
Questionnaire was prepared and distributed to 15 Performance
Ax is a IOOx I I matrix of factor loadings
~ is a l l x l column vector of latent variables (i.e. the I I facets
Management facilitators from II organisations that were asked to
rate items in terms of their importance for the successful of the Performance Management domain)
implementation of Performance Management, on a three-point o is a 100xl vector of unique/measurement error components
scale. Items were retained on the basis of a pre-determined comprising the sum effect of both systematic, non-relevant
arithmetical decision rule. The final instrument consists of 103 influences and random error on the observable indicators
items. (Joreskog, 1993).

Step 4: Converting the Prerequisites Questionnaire to a The path diagram implied by the PMAQ design is depicted in
Performance Management Audit Questionnaire. This was effected Figure 2. The path diagram constitutes a visual representation of
by rewording items to statements of fact, e.g. "In this organisation the aforementioned measurement model. The path diagram
complex objectives are supported by action plans". The PMAQ implies two additional matrices. A symmetric
consists of I I subscales, each representing a primary facet of the covariance/correlation matrix <1> indicates the correlations
multi-dimensional Performance Management domain. The between the latent variables (all correlations between latent

s. Afr. J. Psychol. 2000,30 (4) 33

 <P14

050 _ _~~

866 _ _--.1

80(98,100)

Figure 2. PMAQ measurement model

variables are not shown in Figure 2), and a diagonal matrix 0 0 model portrayed in Figure 2 can explain the observed
depicts the variance in the error components (all the covariance correlationlcovariance matrix. This is achieved by firstly
terms of the 0 B matrix are not shown in Figure 2). The diagonal estimating the free model parameters in the Ax, <1> and 0 B
nature of the 0 0 matrix implies that the error components are matrices through an iterative numerical estimation procedure in
assumed to be uncorrelated across items. The critical question which a specific fitting criterion (describing the difference
that the current analysis needs to answer is to what extent the between the observed and implied covariance matrices) is

34 S. Afr. J. Psycho\. 2000,30 (4)

minimised. The degree/extent to which the covariance matrix Maximum likelihood estimation was used to solve for the
implied by the final parameter solution deviates from the parameters set free in the model. This has the advantage,
observed covariance matrix is then subsequently assessed assuming multivariate normality of the observed variables and a
through numerous indices of model fit. If unsatisfactory sufficiently large sample, that the minimum of the maximum
absolute fit would be found (Kelloway, 1998), the conclusion likelihood fitting function (not shown) multiplied by one minus
would necessarily follow that the PMAQ does not measure the the number of observations, is distributed as a X2 distribution
Performance Management domain as intended. The converse, (Kelloway, 1998). In specifying the model the factor loading A
however, does not hold. A high degree of fit (i.e. the fitted of the first item in each sub-scale has been set equal to I in the
covariance matrix closely corresponds to the observed corresponding columns of Ax to fix the scales of measurement
covariance matrix) would not imply that the measurement of the latent variables (Joreskog & Sorbom, 1996a; 1996c).
model is correct but only that it constitutes a plausible The remaining items in each sub-scale were set free in the
explanation for the observed covariance/correlation matrix. The corresponding columns of Ax. All remaining elements in Ax
measurement model could be considered corroborated in the were left at the default zero loading to reflect the assumed pure
sense that it survived an opportunity to be refuted (Popper, nature of the items (Tabachnick & Fidell, 1989). The elements
1972). In principle, therefore, both the procedure followed of <I> and 0 0 were treated by default as free.
here, and that originally followed by Spangenberg and Theron
(1997), is no different from that characterising any other true An assessment of model fit through multiple fit indices.
explanatory scientific endeavour in which hypotheses are The relevant indices reflecting the absolute and comparative fit
developed and confronted with empirical data (Popper, 1972). of the model are show in Table I. Tests of absolute fit are
The important point, however, is that confirmatory factor concerned with the ability of the fitted model to reproduce the
analysis through structural equation modelling provides a much observed covariance matrix (Kelloway, 1998). Tests of
stronger challenge to the PMAQ than the classical factor comparative fit, by contrast, examined the question whether the
analysis procedure, in that the risk of being refuted involved in model under consideration is better than a baseline model
the current analysis procedure is significantly higher than that (Kelloway, 1998). The null or independence model and the
involved in the procedure originally used by Spangenberg and saturated model served as baseline for the comparison. The null
Theron (1997). The original procedure presented the PMAQ model refers to a model in which all the parameters have been
with a relatively mild challenge due the propensity of Procrustes set to zero (i.e., a model in which no variables are connected
rotation to take advantage of chance (Nunnally, 1978; and the degrees of freedom equal the number of equations in the
Tabachnick & Fidell, 1989).

Raykov, Tomer and Nesselroade (1991) suggest that reports of

structural equation modelling analysis should at least include:
Degrees of Freedom 4795 =
=
Minimum Fit Function Chi-Square 10700.36 (P 0.0) =
Chi-Square/degrees of Freedom = 2.2315662
• A graphic presentation of the structural equation model in Estimated Non-centrality Parameter (NCP) = 7283.28
terms of a path diagram utilising conventional symbols;
• Information on the parameters for the structural equation Minimum Fit Function Value 20.62 =
modelling analysis, including the type of matrix analysed, Population Discrepancy Function Value (FO) 14.03 =
Root Mean Square Error of Approximation (RMSEA) = 0.054
the treatment of missing values and the method of parameter
estimation; Expected Cross-Validation Index (ECVI) = 24.25
• An assessment of model fit through multiple fit indices =
ECVI for saturated Model 19.46
• An examination of the obtained solution; ECVI for Independence Model 67.54 =
• Nested model comparisons (not applicable in this study);
and Chi-Square for Independence Model with 4950 Degrees of
• Reference to model modification indices.
=
Freedom 34853.85

=
Independence AIC 35053.85
This report will attempt to comply with these recommendations. =
Model AIC 12588.28
Equation I and Figure 2 represent efforts to comply with the =
Saturated AIC 10100.00
first requirement. =
Independence CAlC 35579.24
=
Model CAlC 13928.01
Information on the parameters for the structural equation Saturated CAlC = 36631.84
modelling analysis.
LISREL 8 (Joreskog & Sorborn, 1996a) was used for the
Root Mean Square Residual (RMR) 0.047 =
confirmatory factor analysis to determine the fit of the model
=
Standardized RMR 0.047
Goodness of Fit Index (GFI) = 0.68
depicted in Figure 2. The raw data was read directly into Adjusted Goodness of Fit Index (AGFI) 0.67 =
LISREL in fixed format. Listwise deletion of missing cases Parsimony Goodness of Fit Index (PGFI) = 0.65
was used. The model test was based on an analysis of a matrix
of polychoric correlations" rather than the more usual Normed Fit Index (NFl) = 0.69
covariance matrix, due to the ordinal nature of the item scales Non-Normed Fit Index (NNFI) 0.80 =
(Joreskog & Sorborn, I996b). Joreskog and Sorborn (l996b) Parsimony Normed Fit Index (PNFI) 0.67 =
Comparative Fit Index (CFI) = 0.80
strongly advise against the analysis of the covariance matrix Incremental Fit Index (IFI) 0.80 =
when one or more of the variables to be analysed in LISREL are Relative Fit Index (RFI) 0.68=
ordinal. They also point out the importance of choosing the
correct type of correlation matrix to analyse. In a Monte Carlo Critical N (CN) = 244.76
study of six correlation measures for ordinal variables Joreskog
and Sorborn (1996b) concluded that the polychoric correlation
Confidence limits could not be computed due to too small p-
appears to be (a) the most consistent estimator of p, (b) almost
value for chi-square
always the best estimator in the sense of being closest to p, (c)
not very sensitive to the shape of the marginal distributions and
(d) least negatively biased for moderate sample sizes. Table 1. Goodness of Fit Statistics (Condition 10 tests)

S. Afr. J. Psycho\. 2000,30 (4) 35

model; 4950 in the case of the PMAQ). The saturated model less affected by sample size, seem to suggest reasonable model
(i. e., a just identified model) refers to a model in which the fit.
number of equations equal the number of estimated coefficients
with no degrees of freedom; 4950 coefficients in the case of the The indices of comparative fit in Table 1 seem to suggest that
PMAQ). the fitted model provides a borderline to questionable
improvement over the baseline independence model, depending
The X2 test statistic is the measure that was traditionally used to on the fit index. The normed fit index (NFl), the non-normed fit
test the null hypothesis that the population covariance matrix is index (NNFI), the incremental fit index (lFI), the comparative
the covariance matrix implied by the model. The p-value fit index (CFI) and the relative fit index (RFI) all range between
reported by LISREL is the probability of obtaining a X2 value o and 1, with 0.90 generally considered an indicator of a well
larger than the calculated value, given that the fitted model is fitting model (Bentler, 1990; Bentler & Bonett, 1980; Hair,
correct. The aim, contrary to conventional inferential statistics, Anderson, Tatham and Black, 1995; Kelloway, 1998).
is thus to obtain an insignificant X2 since it would imply a small Especialy the NFl and RFI values in Table 1 seem somewhat
enough discrepancy between the fitted and observed sample worrying.
covariance matrices that could have arisen by chance under H o
(Hair, Anderson, Tatham and Black, 1995; Joreskog, 1993). The parsimonious normed fit index (PNFI) and the
The p-values associated with the X2 value in Table I clearly parsimonious goodness-of-fit index (POFI) in Table I relate the
indicate a highly significant (p<0.05) X2 • Joreskog and Sorborn goodness-of-fit of the model to the number of parameters that
need to be estimated to achieve this degree of fit. Its
(I996a), however, recommend that X2 should not be treated as a
meaningful use, however, necessitates a second, explicitly
test statistic, but rather as a goodness-of-fit index where the
degrees of freedom serve as a frame of reference to judge formulated and fitted, model. Such an alternative model could
stem from the freeing of additional parameters in the model
whether X2 is large or small. Small x2-values indicate good fit.
suggested by the modification indices for Ax. The comparison
However, as was indicated earlier, the x2-measure is distributed
of parsimonious fit indices across models would then indicate
asymptotically (i.e. when calculated for large samples) as a chi-
whether the increase in fit achieved by estimating additional
square distribution (Joreskog & Sorborn, 1996a) thus resulting
parameters is worth the sacrifice of degrees offreedom.
in the frustrating dilemma that just at the point where the
distributional assumption becomes tenable, the statistical power
The values for the expected cross-validation index (ECVI), the
of the test becomes extremely high. The chances of obtaining a
Aiken information criterion (AIC) and the consistent Aiken
non-significant X2 statistics in a large sample consequently
information criterion (CAlC) shown in Table 1 suggest that the
becomes extremely small even when the model fits the
measurement model provides a more parsimonious fit than the
empirical data quite well. Given the current sample size, it
independent/null model. The ECVI and the AIC, however,
would be thus be imprudent to conclude poor fit based on the
contradict the CAlC when comparing the measurement model
large and significant X2 alone. to the saturated model. The saturated model provides a single,
unique solution and the model always provides a perfect fit to
The ratio X2/df has been proposed to alleviate the the data (Hair, Anderson, Tatham and Black, 1995; Kelloway,
aforementioned problems associated with the X2 measure. This 1998).
measure, however, seems to suffer from somewhat arbitrary
standards of interpretation (Kelloway, 1998; Medsker, Williams An examination of the obtained solution
& Holahan, 1994). Ratios of less than 5 have been interpreted All estimated factor loadings Aij in Ax (not shown) differ
as a good fit to the data. Ratios between 2 and 5 have, however, significantly (p<0.05) from zero. Modifying the existing model
also been interpreted as indicating good fit (Kelloway, 1998). by eliminating any item from its current subscale (i.e, fixing any
of the free parameters in Ax to zero or pruning away any of the
The root mean squared residual (RMR) and standardised RMR
existing paths in Figure 2) would therefore significantly
reflect the square root of the mean of the squared difference deteriorate the fit of the model. The question, however, remains
between the observed and fitted covariance matrices. Values
whether the addition of more paths to the model would
less than 0.05 are considered indicative of a good fit to the data
significantly improve the parsimonious fit of the model. The
(Kelloway, 1998). The root mean squared error of proportion of item variance that the model explains is shown in
approximation (RMSEA) essentially expresses the discrepancy
Table 2 for each of the PMAQ items. The values shown in
between the observed and fitted covariance matrices in terms of
Table 2 could, simultaneously be interpreted as the reliabilities
the degrees of freedom of the model (Steiger, 1990). RMSEA-
of the respective items Pii where the reliability of item i is
values below 0.10 indicate good fit, while values below 0.05
defined by equation 2.
indicate a very good fit to the data (Kelloway, 1998). Hair,
Anderson, Tatham and Black (1995) consider RMSEA- values
between 0.05 and 0.08 indicative of acceptable fit. Brown and Pii = AV[A2i + OOi]
Cudeck (1993) regard a RMSEA-value of 0.5 indicative of a = I - (00/[A2 j + 0oi])
close fit and RMSEA-values up to 0.08 indicative of reasonable = I - 001 2
errors of approximation. The goodness of fit index (OFI) Since:
measure the extent to which the model fits the data compared to a 2 i = A2 j + °Oi
a model in which all parameters have been set to zero (Joreskog =1
& Sorbom, 1996c). The adjusted OF! (AOFl) adjusts the OFI where:
for degrees of freedom in the model (Joreskog & Sorborn, Ooi represent the error variance elements of the completely
1996c). Values above 0.9 on the OFI and the AOFI indicate a standardised diagonal matrix 0 0 and Aj the factor loadings in the
good fit to the data (Kelloway, 1998). Kelloway (1998), completely standardised Ax matrix.
however, warns that the guidelines on the OFI and AOFI are
based on experience, are somewhat arbitrary, and should However, since the error term 8i is considered to be the sum ofa
consequently be used with caution. systematic error component unique to Xi and a true (random)
measurement error component, Pi; should be interpreted as a
Although the traditional X2 measure suggested poor model fit, lower bound for the true reliability (Joreskog & Sorbom,
the aforementioned alternative measures of fit, designed to be 1996a). Pii could for the same reason also be interpreted as an

36 S. Afr. 1. Psychol, 2000,30 (4)

 STRAT1 STRAT2 STRAT3 STRAT4 STRAT5 STRAT6 STRAT7 STRAT8 STRAT9 STRAT10 INTER1 INTER2
0.35 0.39 0.54 0.30 0.42 0.51 0.53 0.49 0.29 0.39 0.54 0.50

INTER3 PSY1 PSY2 PSY3 PSY4 PSY5 PSY6 PSY7 PSY8 CLARIFY1 CLARIFY2 CLARIFY3 PLAN1 PLAN2
0.50 0.36 0.42 0.46 0.39 0.44 0.58 0.55 0.50 0.59 0.45 0.46 0.39 0.51

PLAN3 PLAN4 PLAN5 PLAN6 PLAN7 PLAN8 PLAN9 PLAN10 PLAN11 PLAN12 PLAN13 PLAN14 PLAN15
0.40 0.39 0.39 0.46 0.46 0.43 0.56 0.40 0.37 0.44 0.38 0.51 0.50

PLAN16 PLAN17 PLAN18 PLAN19 PLAN20 DESIGN1 DESIGN2 DESIGN3 DESIGN4 DESIGN5 MANAGP1
0.39 0.37 0.29 0.40 0.23 0.52 0.57 0.55 0.52 0.56 0.49

MANAGP2 MANAGP3 MANAGP4 MANAGP5 MANAGP6 MANAGP7 MANAGP8 MANAGP9 MANAGP10

0.39 0.37 0.46 0.46 0.40 0.27 0.42 0.43 0.45

MANAGP11 MANAGP12 MANAGP13 MANAGP14 MANAGP15 MANAGP16 MANAGP17 REVIEW1 REVIEW2

0.35 0.34 0.36 0.48 0.28 0.13 0.32 0.33 0.46

REVIEW3 REVIEW4 REVIEW5 REVIEW6 REVIEW7 REVIEW8 REVIEW9 REVIEW10 REVIEW11 REVIEW12
0.50 0.61 0.61 0.44 0.23 0.35 0.36 0.43 0.48 0.46

REVIEW13 REVIEW14 REVIEW15 REWARD1 REWARD2 REWARD3 REWARD4 REWARD5 REWARD6

0.49 0.30 0.30 0.52 0.56 0.54 0.49 0.40 0.40

REWARD7 REWARD8 REWARD9 REWARD10 REWARD11 L1NK1 L1NK2 L1NK3 L1NK4 L1NK5 OUTPUT1
0.40 0.54 0.62 0.55 0.41 0.52 0.54 0.42 0.55 0.43 0.46

OUTPUT2 OUTPUT3
0.54 0.57

Table 2. Squared multiple correlations for the observed variables

item validity coefficient expressing the extent to which the variables. Conversely, large negative residuals indicate that the
latent variable ~j is reflected in the indicator variable Xij' Even model overestimates the covariances that actually exist between
so, quite a few of the values shown in Table 2 are alarmingly pairs of variables (Joreskog, 1993). To rectify the
low. This finding corresponds to (but can not be equated to) the underestimation problem Joreskog and Sorbom (I996c) suggest
finding in Spangenberg and Theron (1997) that the squared that additional paths should be added to the model which could
multiple correlation of item scores regressed on the eleven account for the discrepancies between the observed and fitted
factors tend to be relatively low. Spangenberg and Theron covariance matrices. Pruning away paths associated with the
(1997), however, reported satisfactory reliability coefficients for problematic fitted covariance terms, on the other hand, could be
the various sub-scales. used to deal with overestimation problems.

The <1> matrix of correlations between the eleven Performance Model modification indices
Management facets ~j are shown as Table 3. The off-diagonal Examining the modification indices and the completely
elements of the <1> matrix contain the disattenuated correlations standardised expected parameter change associated with the
between the eleven Performance Management facets. fixed parameters in the Ax matrix indicate a number of items
that, if set free, would result in minor decreases in the X2
Inspection of a stem-and-leaf plot of the standardised residuals measure. In the case of only two items does the modification
(not shown) indicates the presence of a limited number of large index exceed 40 but in no instance is the expected decrease in
positive and negative residuals. The stem-and-leaf plot seems X2 more than 58. Considering the X2 value reported in Table I
to be slightly skewed to the right thus indicating a and the magnitude of the reduction required to achieve
preponderance of large positive residuals. Since a residual is insignificance very little would be gained by setting these
the difference between an element in the observed covariance parameters free.
matrix and the corresponding element in the fitted covariance
matrix, large positive residuals mean that the model seriously Two further considerations argue against modification of the
underestimates the covariances between specific pairs of model based on these empirical indicators. The low expected

KSI_1 • KSI_2 KSI_3 KSI_4 KSI_5 KSI_6 KSU KSI_8 KSI_9 KSI_10 KSU1
KSI_1 1.00
KSI_2 0.81 1.00
KSI_3 0.87 0.78 1.00
KSI_4 0.74 0.71 0.84 1.00
KSI_5 0.79 0.74 0.88 0.88 1.00
KSI_6 0.76 0.64 0.72 0.68 0.76 1.00
KSU 0.80 0.75 0.83 0.79 0.88 0.76 1.00
KSI_8 0.79 0.71 0.86 0.76 0.85 0.71 0.90 1.00
KSI_9 0.67 0.66 0.74 0.65 0.68 0.66 0.75 0.75 1.00
KSUO 0.76 0.73 0.86 0.75 0.81 0.76 0.83 0.81 0.76 1.00
KSU1 0.67 0.62 0.77 0.68 0.73 0.64 0.77 0.75 0.70 0.80 1.00

Table 3. Disattenuated correlations between the eleven facets of Performance Management

S. Afr. J. Psycho\. 2000,30 (4) 37

values of the completely standardised factor loadings of the providing the diagnostic feedback required to refine the system
affected items suggest that the items are not influential in order to ensure optimum functioning and results. The actual
indicators of the latent variables they are linked with. usage of the PMAQ for this purpose depends, however, on the
Furthermore, no a priori substantive theoretical justification assurance that the PMAQ provides an uncontaminated and
exists to link the affected items to the designated latent comprehensive assessment of the Performance Management
variables. None of the item pairs showing large positive domain.
residuals appear amongst the items with sizeable modification
Only one study on the psychometric integrity of the PMAQ
indices thus indicating that the underestimation problem does
exists in the literature (Spangenberg & Theron, 1997). The
not originate from a lack of paths between the affected terms
findings reported here corroborate the position explicated in
and the existing eleven latent variables.
Spangenberg and Theron (1997) that the PMAQ provides a
reliable and content valid measure of the Performance
Examining the modification indices and the completely
Management domain as conceptualised by the systems model
standardised expected parameter change associated with the
(Spangenberg, I994a; I994b). The PMAQ measurement model
fixed parameters in the 0 0 matrix indicate only two pairs of
(equation I and Figure 2) should, however, also be confronted
items (XI & X 2 and X S2 & X S3 ) that, if set free, would result in
with a newly sampled data set to confirm the findings and
minor decreases in the X2 measure. This in turn suggests that
decisions reported here. In addition, the fit of the higher-order
the assumption of uncorrelated error terms is largely tenable. It
factor structure of the PMAQ described in Spangenberg and
is of particular significance to note that the two problematic
Theron (1997) should also be assessed via LI SREL.
pairs of items in the 0 0 matrix are also the item pairs with the
largest positive residuals. This seems to suggest that additional References
latent variables need to be grafted on to the existing model to Bentler, P.M. (1990). Comparative fit indexes in structural models.
account for the common systematic error in o-error terms. Psychological Bulletin, 107(2),238-246.
Inspection of the wording of the affected items, however, Bentler, P.M. and Bonett, D.G. (1980). Significance tests and
suggests that the correlated error terms in the case of the first goodness offit in the analysis of covariance structures.
two items of the reward sub-scale (X S2 & X S3 ) might be due to Psychological Bulletin, 88(3), 588-606.
an unfortunate oversight in the construction of the items which Bevan, S. & Thompson, M. (1991). Performance Management at
resulted in the two items sharing the same stem. In the case of the crossroads. Personnel Management, 23, November, 36-39.
the first two items of the strategy sub-scale (XI & X 2) , however, Brown, M.W. and Cudeck, R. (1993). Alternative ways of
the hypothesis of an additional latent variable does not seem assessing model fit. In KA Bollen & 1.S. Long (Editors):
altogether implausible but certainly in no way beyond dispute. Testing Structural Equation Models. Newbury Park: Sage
The most prudent option seem to retain the current model but to Publications.
rewrite the first two items of the reward sub-scale and to Comrey, AL. (1973). Ajirst course infactor analysis. New
postpone the decision on additional strategy related latent York: Academic Press.
variables till independent follow-up studies corroborate the Egan, G. (1995). A clear path to peak performance. People
finding reported here. Management, 18 May, 34-37.
Hair, J.F., Anderson, R.E., Tatham, R.L. & Black, W.e. (1995).
Conclusion Multivariate data analysis with readings. Upper Sadie River,
Human resources constitute a pivotal production factor due to the New Jersey: Prentice-Hall, Inc.
fact that organisations are managed and operated by people. Joreskog, KG. (1993). Testing structural equation models. In
Human resources are the life giving production factor through KA Bollen & J.S. Long. Testing Structural Equation
which the other factors of production are mobilised and thus Models. Newbury Park: Sage Publications.
determine the effectiveness and efficiency of the organisation. Joreskog, K.G. & Sorborn D. (l996a). L1SREL 8: User's
The management of human resources is, however, complicated by reference guide. Chicago: SSI Scientific Software
the intricate, and to a certain extent enigmatic, nature of people as International.
employees. This leads to the basic premise that a valid theoretical Joreskog, K.G. & Sorborn D. (l996b). PRELIS 2: User's
understanding of the different facets of employee work behaviour reference guide. Chicago: SSI Scientific Software
constitute a fundamental prerequisite for efficient and equitable International.
management of performance. A real danger exists, however, that
Joreskog, K.G. & Sorborn D. (l996c). L1SREL 8: Structural
the factors underlying the employee's behaviour will be treated
equation modeling with the SIMPLIS command language.
in a piece meal fashion when trying to theoretically explain or
Chicago: SSI Scientific Software International.
practically affect work performance. The Performance
Kelloway, E.K. (1998). Using L1SRELfor structural equation
Management concept shows substantial promise as a way of
modeling; a researcher's guide. Thousand Oaks: Sage
conceptually integrating a wide diversity of topics traditionally
Publications, Inc.
treated in a compartmentalised fashion by Industrial/
Lane, L.M. (1994). Old failures and new opportunities: public
Organisational Psychology. The Performance Management
sector Performance Management. Review of Public
concept could, furthermore, serve as an integrated managerial
Personnel Administration, Summer, 26-44.
approach, which synthesises the human resource functions into a
coherent whole and links it to the strategic objectives of the Lawler, E.E. (1994). Performance Management: the next
organisation. generation. Compensation and Benefits Review. 26, May-
June, 16-19.
However, the potential benefits that could be derived from McLagan, P.A (1993). Performance Management: can it
Performance Management depend on whether a number of work? McLagan International Inc.: St Paul, Minnesota.
critical prerequisites are adhered to. These critical prerequisites Medskar, G.J. , Williams, L.J. & Holahan, P.J. (1994). A review
would probably not all be satisfied during the initial phases of ofcurrent practices for evaluating causal models in
implementing a Performance Management system. The eventual organizational behavior and human resourses management
successful implementation of Performance Management would research Journal ofManagement. 20, 439-464.
therefore require the regular auditing of the Performance Nunnally, lC. (1978). Psychometric theory. New York:
Management system to provide the necessary information to McGraw-Hili Book Company.
design "therapeutic" interventions for improving the functioning Popper, KR. (1972). Conjectures and refutations; the growth of
of the system. The PMAQ was developed with the purpose of SCientific knowledge. London: Routledge and Paul.

38 S. Afr. J. Psycho\. 2000,30 (4)

 Raykov, T., Tomer, A. & Nesselroade, 1.R. (1991). Reporting Spangenberg, H.H. & Theron, c.c. (1997). Developing a
structural equation modeling results in Psychology and Performance Management audit questionnaire. South African
Aging: some proposed guidelines. Psychology and Aging, Journal ofPsychology, 1997,27(3),143-149.
6(4),499-503. SPSS (1990). SPSS reference guide. Chicago, Illinois: SPSS
RummIer, G.A. & Brache, A.P. (1990). Improving peformance. International.
San Fransisco: Jossey-Bass. Steiger, 1.H. (1990). Structural model evaluation and
Schneier, C.E., Shaw, D.G. & Beatty, RW. (1991). Performance modification: an interval estimation approach.
Measurement and Management: a tool for strategy execution. Multivariate Behavioral Research, 25(2), 173-180.
Human Resource Management 30(3),279-301. Tabachnick, B.G. & Fidell, L.S. (1989). Using
Spangenberg, H.H. (1994a). Performance Management- multivariate statistics (Second edition). New York:
problems and possible solutions. Journal ofIndustrial Harper Collins Publishers.
Psychology, 20(4), 1-6. Wellins, R.S. & Schulz Murphy, 1. (1995). Reengineering:
Spangenberg, H.H. (1994b). Understanding and implementing Plug into the human factor. Training and development,
Performance Management. Cape Town: Juta January, 33-37.

S. Afr. 1. Psychol. 2000,30 (4) 39

View publication stats