See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/242332755

Measuring individual work performance: Identifying and selecting indicators

Article  in  Work · June 2013

DOI: 10.3233/WOR-131659 · Source: PubMed

CITATIONS READS

123 40,906

5 authors, including:

Linda Koopmans Claire Bernaards

TNO Active Cues / tover
27 PUBLICATIONS   1,047 CITATIONS    42 PUBLICATIONS   2,118 CITATIONS   

SEE PROFILE SEE PROFILE

Vincent Hildebrandt Henrica De Vet

TNO Amsterdam University Medical Center
93 PUBLICATIONS   6,264 CITATIONS    701 PUBLICATIONS   75,098 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Onderwijs in Wetenschap/Research Notes View project

Psychomotor Therapy in Chronic Pain Rehabilitation View project

All content following this page was uploaded by Linda Koopmans on 16 May 2014.

The user has requested enhancement of the downloaded file.

 Chapter 3

Measuring Individual Work Performance -

Identifying and Selecting Indicators

Linda Koopmans, Claire M. Bernaards, Vincent H. Hildebrandt,

Henrica C.W. de Vet, Allard J. van der Beek

WORK: A Journal of Prevention, Assessment & Rehabilitation. 2013; 45(3)

DOI 10.3233/WOR-131659

3
Chapter 3

Abstract

Background: Theoretically, individual work performance (IWP) can be divided into

four dimensions: task performance, contextual performance, adaptive performance,
and counterproductive work behavior. However, there is no consensus on the
indicators used to measure these dimensions.

Objective: This study was designed to 1) identify indicators for each dimension, 2)
select the most relevant indicators, and 3) determine the relative weight of each
dimension in ratings of work performance.

Methods: IWP indicators were identified from multiple research disciplines, via
literature, existing questionnaires, and expert interviews. Subsequently, experts
selected the most relevant indicators per dimension and scored the relative weight
of each dimension in ratings of IWP.

Results: In total, 128 unique indicators were identified. Twenty-three of these

indicators were selected by experts as most relevant for measuring IWP. Task
performance determined 36% of the work performance rating, while the other three
dimensions respectively determined 22%, 20% and 21% of the rating.

Conclusions: Notable consensus was found on relevant indicators of IWP, reducing

the number from 128 to 23 relevant indicators. This provides an important step
towards the development of a standardized, generic and short measurement
instrument for assessing IWP.

62
 Identifying and Selecting Indicators

Introduction

Although an individual’s performance at work is one of the most important

outcomes of studies in the occupational setting, recent research has shown that
there is no consensus on the definition and measurement of individual work
performance [1-3]. Various terms (often used interchangeably) are used to describe
individual work performance (IWP), such as presenteeism, performance, or
productivity. The definitions of these terms are often unclear. This is undesirable,
because a clear definition and theoretical framework of IWP is a prerequisite for its
valid measurement. Valid measurement, in turn, is necessary to accurately establish
the causes and consequences of IWP.

Defining individual work performance

Considering the importance of IWP, it is not surprising that disciplines other than
occupational medicine have concerned themselves with defining and measuring the
concept. Within work and organizational psychology, defining the construct of IWP
and attempting to understand its underlying structure has received much attention
[4]. In the latter discipline, IWP is generally defined as “behaviors or actions that are
relevant to the goals of the organization” [5]. Thus, IWP is defined in terms of
behaviors or actions of employees, rather than the results of these actions. In
addition, IWP consists of behaviors that are under the control of the individual, thus
excluding behaviors that are constrained by the environment [6].
Recently, a heuristic framework of IWP was proposed in a multi-disciplinary,
systematic literature review [7], in which IWP consisted of four broad and generic
dimensions. The first dimension, task performance, refers to the proficiency with
which an employee performs central job tasks [5]. The second dimension, contextual
performance, refers to employee behaviors that support the organizational, social,
and psychological environment in which the central job tasks are performed [8]. The
third dimension, adaptive performance, refers to an employee’s proficiency in
adapting to changes in work roles or environment [9]. The fourth dimension,
counterproductive work behavior, refers to behavior that is harmful to the well-being
of the organization [6].

Measuring individual work performance

While four generic dimensions of IWP can be distinguished, there is still little
consensus on how to measure the concept. Within occupational medicine, various

63
Chapter 3

questionnaires exist to measure IWP or similar constructs, such as the Stanford

Presenteeism Scale [10], Work Productivity and Activity Impairment [11], and Health
and Performance Questionnaire [12]. Within work and organizational psychology,
numerous scales have also been developed to measure task performance [e.g., 13],
contextual performance [e.g., 14], or counterproductive work behavior [e.g., 15].
The multitude of scales in this discipline is perhaps best illustrated by LePine, Erez
and Johnson [16], who identified more than 40 different measures of contextual
performance.
The heterogeneous content of IWP measures is likely related to the use of
different definitions of IWP, or a lack thereof, and by the use of different
developmental or target populations. What is noticeable in the measures developed
in occupational medicine is that a clear definition and theoretical model of IWP is
often lacking [17-19]. Although the measures developed in work and organizational
psychology do use definitions of IWP dimensions, none of them captured the
complete range of individual behaviors at work. Moreover, measures from
occupational medicine are often designed for individuals with physical or mental
health problems [20]. This makes these measures less suitable for assessing IWP in
healthy workers. In addition, the measures developed in work and organizational
psychology, although intended to be generic, were often developed and refined
based on a specific occupation.

Goal of the current study

The current lack of consensus on how to measure IWP impedes valid measurement
of the construct Research on IWP would benefit greatly from a standardized, generic,
short instrument. The four-dimensional framework of IWP [7] provides a theoretical
starting point for developing such an instrument. Next, it is essential to gain
consensus on the indicators (important employee behaviors or actions) for each
dimension of IWP. Therefore, we designed a study to: 1) identify indicators for each
IWP dimension, 2) select the most relevant indicators, and 3) determine the relative
weight of each dimension in ratings of IWP. Secondary aims of this study were to
determine: 4) whether there were differences between the views of experts from
different professional backgrounds (researchers, managers, human resource
managers, and occupational health professionals) on the aforementioned research
questions, and 5) whether the experts preferred a generic or job-specific
questionnaire.

64
 Identifying and Selecting Indicators

Methods

Identification of indicators
First, indicators were identified from a literature review on conceptual frameworks
of IWP [7]. Second, indicators were identified from existing IWP related
questionnaires. For this purpose, a systematic search was conducted to identify
questionnaires measuring individual work performance or work productivity. As the
terms work performance and work productivity are often used synonymously, we
incorporated both terms in our search strategies. Search strategies were developed
with the aid of experienced search specialists. Search terms included work
performance, work productivity, job performance, employee performance, or
employee productivity and questionnaire, scale, or index. Searches were conducted
in two medical databases (PubMed and Embase.com) and one psychological
(PsycINFO) database in September 2010. Additional questionnaires were identified
by scanning the authors’ personal collection of IWP literature. Third, interviews with
16 key-experts were held in November 2010. Key-experts were national and
international researchers with over 20 years of experience in the areas of
occupational health, psychology, or management.
A review of the literature, questionnaires and data from the experts
interviewed resulted in an initial list of IWP indicators. From this list, the first and
second author independently removed causal variables (e.g., motivation). In
addition, overlapping indicators between dimensions (e.g., ‘concentrating’ in
contextual performance, ‘not concentrating’ in counterproductive work behavior)
were removed from the dimension where they were least well suited. Differences in
judgment were resolved through a consensus procedure. Based on conceptual
overlap, the first and second author independently reduced the remaining list of
indicators, and using the definitions categorized each indicator into the dimension
where it best fitted. Differences in judgment were again resolved through a
consensus procedure.

Selection of indicators

Participants
A sample of 695 experts from different professional backgrounds (researchers,
managers, human resource managers (HRM), and occupational health professionals
(OHP)), including the key-experts, were invited to select the most relevant indicators

65
Chapter 3

of IWP. This sample consisted of participants from one national and one
international occupational health conference that took place in The Netherlands.
Each participant was invited by e-mail to participate in the study. A link to the online
questionnaire was included. Depending on their country of residence, participants
completed the questionnaire in Dutch or English. Participants had three weeks to
complete the questionnaire. After two weeks, non-responders received a reminder
via e-mail.

Questionnaire
After a brief introduction to the study, participants were shown a list of all indicators
per IWP dimension (task performance, contextual performance, adaptive
performance, and counterproductive work behavior). Within dimensions, indicators
were presented in random order. Per dimension, participants were asked to select
the 6 most important indicators, keeping in mind work performance on a generic
level. This means that they had to select indicators that were important in all types
of professions, not just in their own profession. For practical reasons, contextual
performance was split into two sub-dimensions (interpersonally directed and
organizationally directed) because of the large number of indicators related to this
dimension. In total, each participant selected 30 (5x6) indicators from the full list of
indicators, which he or she believed to be most relevant. After every category of
indicators, space was provided for comments and/or suggestions for additional
indicators. In the second part of the questionnaire, participants were asked to divide
100 points between the four dimensions, according to the relative weight they would
assign to each dimensions when rating IWP. They were also asked whether they
believed it possible to develop a generic questionnaire, or whether a combination of
a generic and job-specific questionnaire, or a job-specific questionnaire, was more
likely. Finally, participants were asked to fill out their gender, age, highest
educational level completed, profession, number of years experience in this
profession, number of people they manage, and branch of industry. Participants
were asked to leave their e-mail address if they wanted to receive the results of the
study and be eligible for one of the five gift vouchers to be allotted.

Data analysis
To rank the indicators in order of importance, the percentage of participants who
selected an indicator was calculated. Indicators were regarded relevant when they
were selected by 40% or more of the participants. Chi-square tests were performed

66
 Identifying and Selecting Indicators

to examine whether there were significant differences in scores between subgroups

(gender, age, educational level, profession, number of years experience in this
profession, number of people managed, and branch of industry). Although it was not
possible to compute a statistical agreement score (e.g., Cohen’s kappa), subgroups
showing few statistically significant differences were considered to be in agreement.
To determine the relative weight of each dimension in IWP ratings, the
mean number of points assigned to each dimension was calculated. Independent
samples t-tests were performed to examine differences in weights between
subgroups with two levels (gender). One-way analyses of variance were performed
to examine differences in weights between subgroups with more than two levels
(age, educational level, profession, number of years experience in this profession,
number of people managed, and branch of industry). Post-hoc tests with Bonferroni
correction were performed to determine which subgroups differed from one
another. If assumptions of homogeneity of variance were not met, then Tamhane’s
T2 tests were performed to determine which subgroups differed [21].
Finally, the percentage of participants that believed in the development of
a generic questionnaire, a combination of a generic and job-specific questionnaire,
or a job-specific questionnaire, was calculated. SPSS version 17 was used for the
analyses.

Results

Identification of indicators
In the literature review on conceptual frameworks, 54 IWP indicators were
identified. In the systematic questionnaire search, 77 questionnaires were identified
that aimed to measure the construct of individual work performance or productivity.
Of these, full texts of 14 questionnaires could not be retrieved. Another 11
questionnaires were excluded based on full text, because they either did not
measure work performance at the individual level, or did not measure work
performance at all. Additional scanning of personal collections added 29
questionnaires. The 81 questionnaires yielded 231 IWP indicators which were not
already identified in the literature review. None of the questionnaires were found to
measure all dimensions of IWP. Only one fifth of the questionnaires explicitly
measured one or two of the dimensions of IWP (task performance, contextual
performance, adaptive performance, or counterproductive work behavior [e.g.,
13,22,15]). More than one third of all questionnaires contained causal variables in

67
Chapter 3

combination with indicator variables [e.g., 10,23]. In addition, a quarter of the

questionnaires were developed for individuals with a mental or physical health
problem [e.g., 24,25]. More than two third of the questionnaires were developed for
generic purposes [e.g., 26,27], whereas around one third were developed for a
specific job [e.g., 28,29]. The interviews with 16 key-experts yielded 32 additional
IWP indicators, which were not identified in the literature or questionnaires.
In total, the literature, questionnaires, and expert interviews resulted in a
list of 317 IWP indicators. The first and second author removed causal variables and
indicators overlapping between dimensions. Based on conceptual overlap, the first
and second author reduced the remaining list to 128 unique IWP indicators and
categorized each indicator into one of the IWP dimensions. Task performance
consisted of 26 indicators. Both sub-dimensions of contextual performance
consisted of 30 indicators. Adaptive performance consisted of 18 indicators.
Counterproductive work behavior consisted of 24 indicators.

Selection of indicators
In total, 253 participants (response rate of 36.4%) participated in the study, including
14 out of 16 key-experts. See Table 1 for participant characteristics. Table 2 presents
a list of the indicators that were selected as most relevant for each dimension. The
indicators are ranked based on the percentage of votes they received from the total
group. Data on the indicators that did not make the final selection is available from
the authors upon request. No additional indicators were suggested by the
participants during the selection process.
There was high agreement between all subgroups on the importance of the
indicators. For 20 items we found statistically significant differences between
experts from different professional backgrounds. Hence, subgroups based on
profession attributed similar importance to 84% (108 of 128) of the indicators.
Subgroups based on gender agreed on 95% of the indicators. Subgroups based on
age agreed on 93% of the indicators. Subgroups based on educational level agreed
on 87% of the indicators. Subgroups based on number of years work experience
agreed on 97% of the indicators. Subgroups based on the number of people one
manages agreed on 93% of the indicators. Subgroups based on branch of industry
agreed on 92% of the indicators.

68
 Identifying and Selecting Indicators

Table 1. Participant characteristics

Profession
Researchers Managers HRM OHP
Total
N 253 113 48 54 38
Gender (% female) 47% 59% 33% 44% 29%
Age
30 years or younger 11% 18% 0% 9% 5%
31 – 50 years 51% 51% 58% 52% 42%
51 years or older 38% 31% 42% 39% 53%
Completed educational level
Middle-level applied –
apprenticeship or
certificate 1% 0% 2% 0% 5%
Higher vocational, -
Bachelor’s degree 26% 0% 35% 67% 34%
Higher academic, -
Master’s degree 38% 37% 40% 31% 50%
Postgraduate
academic, PhD 35% 63% 23% 2% 11%
Work experience
0 – 5 years 23% 26% 25% 17% 21%
6 – 10 years 21% 21% 19% 28% 16%
10 or more years 56% 53% 56% 56% 63%
Employees managed
None 48% 47% 19% 59% 71%
1 – 5 employees 23% 28% 15% 22% 18%
6 – 20 employees 18% 17% 33% 15% 10%
21 – 50 employees 6% 4% 21% 2% 0%
50 or more employees 5% 4% 13% 2% 0%
Branch of industry
Scientific 42% 88% 5% 2% 9%
Policy 5% 1% 2% 16% 3%
Commercial 5% 2% 12% 10% 0%
Service 26% 5% 50% 37% 43%
Trade & industry 4% 0% 10% 4% 6%
Other 18% 4% 21% 31% 40%

69
Chapter 3

Task performance
The task performance dimension originally included 26 indicators. From these, 5
relevant task performance indicators were identified: work quality (69%), planning
and organizing work (56%), being result-oriented (46%), prioritizing (45%), and
working efficiently (44%).
There was a difference between experts from different professional
backgrounds on one of these indicators. On average, researchers judged being
result-oriented to be significantly less important (29%) than managers (64%) and
human resources managers (66%).

Contextual performance
Initially, there were 60 indicators included in contextual performance. For practical
reasons, these were split into two sub-dimensions (30 indicators at the
interpersonal level and 30 indicators at the organizational level). Four relevant
indicators at the interpersonal level were identified: taking initiative (51%),
accepting and learning from feedback (48%), cooperating with others (45%), and
communicating effectively (45%). Four relevant indicators at the organizational
level were also identified: showing responsibility (67%), being customer-oriented
(42%), being creative (41%), and taking on challenging work tasks (40%).
There were differences between experts from different professional
backgrounds on two of these indicators. On average, managers found taking
initiative significantly more important (75%) than researchers (41%), human
resources managers (53%), and occupational health professionals (50%).
Researchers found being customer-oriented significantly less important (21%) than
managers (57%), human resources managers (69%), and occupational health
professionals (47%).

Adaptive performance
The adaptive performance dimension originally included 18 indicators. Six relevant
adaptive performance indicators were identified: showing resiliency (coping with
stress, difficult situations and adversities; 70%), coming up with creative solutions
to novel, difficult problems (66%), keeping job knowledge up-to-date (57%),
keeping job skills up-to-date (52%), dealing with uncertain and unpredictable work
situations (48%), and adjusting work goals when necessary (43%).
Experts from different professional backgrounds differed on two of these
indicators. Managers found coming up with creative solutions to novel, difficult

70
 Identifying and Selecting Indicators

problems significantly more important (82%) than human resources managers

(57%) and occupational health professionals (58%). Researchers found keeping job
skills up-to-date significantly more important (64%) than managers (41%), human
resources managers (43%), and occupational health professionals (42%).

Counterproductive work behavior

Initially, there were 24 indicators included in counterproductive work behavior.
Four relevant indicators were identified: displaying excessive negativity (62%),
doing things that harm your organization (54%), doing things that harm your co-
workers or supervisor (52%), and purposely making mistakes (48%).
Experts from different professional backgrounds differed on three of these
indicators. Occupational health professionals found displaying excessive negativity
significantly more important (86%) than researchers (57%), managers (55%), and
human resources managers (61%). Managers (73%) and occupational health
professionals (66%) found doing things that harm your organization significantly
more important than researchers (44%). Last, human resources managers found
purposely making mistakes significantly less important (31%) than researchers and
occupational health professionals (both 54%).

Weight of dimensions in IWP ratings

Table 3 presents the relative weight that experts assigned to each of the IWP
dimensions. On average, task performance received the heaviest weight when rating
an employee’s work performance (36 points). Contextual performance (22 points),
adaptive performance (20 points), and counterproductive work behavior (21 points)
received almost equal weightings. Experts from different professional backgrounds
differed significantly on the mean weight they assigned to task performance
(F(3,225) = 3.318; p < 0.05). Researchers (39 points) assigned a marginally significant
(p = 0.058) greater weight to task performance than managers (33 points).

Generic versus job-specific questionnaire

Forty-four percent of the experts believed it possible to develop a generic
questionnaire of IWP, while 23% of the experts believed that a combination of a
generic and job-specific questionnaire was more likely to be effective, whereas 33%
of the experts believed that a job-specific questionnaire was more likely appropriate.

71
Chapter 3

Table 2. Individual work performance indicators that were selected as most

relevant for each dimension and the percentage (%) of votes they received from
the total group and per profession
Profession
Resear- Mana-
Total chers gers HRM OHP
(n=253) (n=113) (n=48) (n=54) (n=38)
Indicator % % % % %
Task performance
1 Work quality 69 70 67 70 66
2 Planning and organising
work 56 55 56 58 58
3 Being result-oriented * 46 29 ab 64 a 66 b 45
4 Prioritising 45 39 42 57 47
5 Working efficiently 44 47 38 43 42
Contextual performance - interpersonal
1 Taking initiative * 51 41 a 75 abc 53 b 50 c
2 Accepting and learning
from feedback 48 50 34 55 50
3 Cooperating with others 45 41 48 47 50
4 Communicating
effectively (e.g.,
adequately expressing
ideas and intentions) 45 43 48 45 50
Contextual performance - organizational
1 Showing responsibility 67 67 70 35 67
2 Being customer-
oriented * 42 21 abc 57 a 69 bd 47 cd
3 Being creative 41 44 41 39 33
4 Taking on challenging
work tasks 40 45 41 25 42
Adaptive performance
1 Showing resiliency
(coping with stress,
difficult situations and
adversities) 70 71 70 73 67
2 Coming up with creative
solutions to novel,
difficult problems * 66 67 82 ab 57 a 58 b
3 Keeping job knowledge
up-to-date 57 59 50 57 56

72
 Identifying and Selecting Indicators

Table 2. Continued
Adaptive performance
4 Keeping job skills up-to-
date * 52 64 abc 41 a 43 b 42 c
5 Dealing with uncertain
and unpredictable work
situations 48 41 64 53 44
6 Adjusting work goals
when necessary 43 42 48 37 47
Counterproductive work behavior
1 Displaying excessive
negativity
(e.g., complaining,
making problems bigger
than they are) * 62 57 a 55 b 61 c 86 abc
2 Doing things that harm
your organization (e.g.,
not following rules,
discussing confidential
information) * 54 44 ab 73 ac 51 c 66 b
3 Doing things that harm
your co-workers or
supervisor (e.g., arguing,
leaving work for others
to finish) 52 58 43 47 54
4 Purposely making
mistakes * 48 54 a 50 31 ab 54 b
Notes: * = significant difference between profession subgroups.
abcd
= Denote which subgroups significantly differed from each other, for example,
in item 3 of task performance the score of the researchers (29%) differed statistically
significant from the score of the managers (64%) and HRM (66%).

73
Chapter 3

Table 3. The relative weight (scale 0 – 100) of each dimension in IWP ratings, in total
and per profession
Profession
Resear- Mana-
chers gers HRM OHP
Total (n=113) (n=48) (n=54) (n=38)
(n=253) Mean Mean Mean Mean
Dimension Mean (SD) (SD) (SD) (SD) (SD)
Task performance * 36 (13) 39 (15) a 33 (10) a 35 (13) 33 (10)
Contextual performance 22 (8) 22 (8) 23 (8) 23 (8) 23 (7)
Adaptive performance 20 (8) 19 (8) 20 (6) 22 (9) 22 (10)
Counterproductive work
behavior 21 (13) 20 (12) 24 (11) 20 (15) 22 (13)
Total 100 100 100 100 100
Notes: * = significant difference between profession subgroup.
a
Marginally significant difference (p = 0.058).

Discussion

The main goal of the current study was to gain consensus on how to measure IWP,
which would enable the development of a standardized, generic, short instrument.
Four broad, generic dimensions of IWP were used as a theoretical basis: task
performance, contextual performance, adaptive performance, and counter-
productive work behavior. Using a multi-disciplinary approach, possible employee
behaviors or actions (indicators) were identified for each dimension, via a review of
the literature, existing questionnaires, and data from interviews with experts. In
total, 128 unique IWP indicators were identified, of which 23 were considered most
relevant for measuring IWP, based on notable consensus among experts. On
average, task performance received greatest weight when rating an employee’s
work performance. Contextual performance, adaptive performance, and
counterproductive work behavior received almost equal weightings. There was
agreement on 84% of the indicators between experts from different professional
backgrounds. Furthermore, experts agreed on the relative weight of each IWP
dimension in rating work performance. However, researchers weighed task
performance slightly higher than managers. Almost half of the experts believed in
the possibility of developing a completely generic questionnaire of IWP.
A multitude of measurement instruments aiming to measure IWP (or a
similar construct such as presenteeism or productivity) were identified in a
systematic search. Considering the large number of questionnaires (81), it is not

74
 Identifying and Selecting Indicators

surprising that most IWP indicators were identified from questionnaires. Far more
indicators were identified for contextual performance than for the other dimensions
of IWP, although contextual performance was not rated higher than other
dimensions. As task performance rated the highest; one could expect more
indicators to be found for that dimension. This finding may indicate that task
performance is a less complex and more uniform dimension to measure than
contextual performance. However, it may also indicate that it is harder to think of
generic behaviors for task performance than for contextual performance. This may
mean that many task performance items are job-specific.
In the literature and questionnaire reviews, an indicator often (if not most
often) used for assessing task performance was quantity of work [7]. Surprisingly,
quantity of work was not selected as one of the most important indicators of task
performance in the current study. In fact, it was selected by only 13% of the
participants as an important indicator of IWP. This finding could be due to our sample
containing relatively few participants from trade and industrial work. Alternatively,
it could be due to the fact that quantity of work is captured in being result-oriented.
While being result-oriented was not mentioned in the literature or questionnaires,
it was selected as an important indicator for task performance in the current study,
mainly by managers and human resources managers. These findings indicate that it
may be more important to look at other indicators than work quantity to assess task
performance, such as work quality or being result-oriented.
Based on the current findings, some of the most often used IWP scales do
not incorporate all relevant indicators, or incorporate irrelevant indicators. Scales
often used to assess contextual performance include for example Podsakoff and
MacKenzie [13] or Van Scotter and Motowidlo [14]. The former focused on
measuring altruism, conscientiousness, sportsmanship, courtesy, and civic virtue.
The latter focused on measuring interpersonal facilitation and job dedication. The
first dimension of both scales is mainly operationalized by helpful behaviors, such as
helping others who have heavy work loads. In the current study, helping others was
not one of the most relevant behaviors for contextual performance (selected by 16%
of participants). Two of the top three contextual performance behaviors identified
in the current study (showing responsibility and accepting and learning from
feedback) are not directly included in either of these questionnaires.
Adaptive performance is a new and upcoming dimension in the work
performance literature [7]. Except for the Job Adaptability Index developed by
Pulakos, Arad, Donovan, and Plamondon [30], few researchers have identified

75
Chapter 3

indicators or developed measures of adaptive performance. Adaptive performance

behaviors, such as resiliency, coming up with creative solutions to difficult, novel
problems, and keeping job skills and job knowledge up-to-date, were found to be
very relevant for work performance in the current study. These types of behaviors
can scarcely be found in existing literature or questionnaires. The aforementioned
findings may represent new and contemporary developments in the concept of work
performance or in today’s job requirements.
Scales often used to assess counterproductive work behavior include for
example Bennett and Robinson [15] or Spector et al. [31]. The former authors focus
on measuring deviance directed at the organization (organizational deviance) and
deviance directed at members of the organization (interpersonal deviance). The
latter authors focus on measuring sabotage (e.g., damaging company equipment),
withdrawal (e.g., taking longer breaks), production deviance (e.g., doing work
incorrectly), theft (e.g., stealing company property), and abuse (e.g., making fun of
someone at work). In the literature, these behaviors have also often been used to
describe counterproductive work behavior [7]. This is largely in line with the findings
of our current study, where displaying excessive negativity, doing things that harm
your organization, doing things that harm your co-workers or supervisor, and
purposely making mistakes, were found to be the most important indicators of
counterproductive work behavior.

Strengths and limitations

The present study has several strengths. To our knowledge, this is the first study that
has systematically identified the numerous indicators used for measuring IWP.
Indicators of IWP were derived from reviews of the literature, questionnaires, and
interviews with experts from multiple disciplines. In addition, during the selection of
indicators, participants were given the opportunity to suggest additional items. This
minimized the chance of missing indicators. Also, this is one of the first studies that
aimed to gain consensus on which indicators were most important for measuring
IWP. In selecting the most important indicators, not only researchers, but also
stakeholders from practice were involved, which improved the practical
representativeness and applicability of the findings.
The present study has some limitations. The first and second author
categorized each indicator into one of the generic IWP dimensions from the list of
128 unique IWP indicators. This categorization may not be valid for every job, as the
place of an indicator may differ depending on the context. For example, in some jobs,

76
 Identifying and Selecting Indicators

communicating effectively may be an aspect of contextual performance, while in

others jobs it may be an aspect of task performance. Future research will need to
determine whether the indicators belong to the expected dimensions, and whether
this categorization is similar over jobs. Far more indicators were identified for
contextual performance than for the other dimensions. For practical reasons, it was
decided to split the contextual performance indicators into two sub-dimensions (30
indicators at the interpersonal level and 30 indicators at the organizational level).
From each sub-dimension, experts were asked to select the six most important
indicators. This may have resulted in an overrepresentation of indicators from one
of these sub-dimensions.
In addition, the extent to which participants were able to think on a generic
level when selecting indicators is debatable. Participants were asked to select the
most important indicators keeping in mind all types of professions, not just in their
own profession. However, the extent to which participants were able to transcend
their own company or field of work remains questionable. Despite the high
similarities in ratings between professions, there were also some differences. For
example, researchers found being customer-oriented significantly less important
than other experts and managers found taking initiative significantly more
important than other experts. This finding may indicate that, to a certain extent, the
relevance of an indicator may depend on the job being evaluated or the evaluator.
In general, however, the high agreement (84% or higher) between all subgroups
indicates good generalizability of the findings over jobs. Nevertheless, future
research should further establish whether the relevance of indicators is similar
across jobs, as well as across companies and countries. Furthermore, we only looked
at the number of times an indicator was selected as relevant and we did not ask the
participants to rate each indicator on importance. The latter would have been too
time-consuming for participants. Therefore, we cannot be sure that the most
frequently selected indicator in our study was also the indicator judged to be most
important by participants. However, it seemed reasonable to assume that an
indicator selected by more than 40% of the participants is an important indicator for
the measurement of IWP.
The decision to deem indicators relevant when selected by more than 40%
of the participants may seem somewhat arbitrary. This decision was made for two
reasons. First, this was based on a graph of the data, where 40% seemed to be a
natural cut-off point. Second, ideally there should be a minimum of three items
contributing to one dimension [32]. What would happen if the cut-off point was set

77
Chapter 3

at 50% or 30% was also examined. For example, when using a 50% cut-off point, only
two relevant indicators for task performance remained. When using a 30% cut-off
point, 9 relevant indicators for task performance appeared. Therefore, to construct
a short, but comprehensive questionnaire, a cut-off point of 40% was deemed to be
sufficient.

Conclusion
Research on IWP would benefit greatly from a standardized, generic, short
measurement instrument. In the current study, 128 unique IWP indicators were
identified, of which 23 indicators were considered most relevant for measuring IWP,
based on notable consensus among experts. This provides an important step
towards the development of a standardized, generic, short instrument. Hopefully,
results of the current study remove some of the uncertainty regarding the definition
and measurement of IWP, and brings us one step closer to unraveling IWP and its
causes and consequences.

78
 Identifying and Selecting Indicators

References

1. Escorpizo R. Understanding work productivity and its application to work-

related musculoskeletal disorders. International Journal of Industrial
Ergonomics. 2008; 38: 291-297.
2. Schultz AB, Chen CY, Edington DW. The cost and impact of health conditions
on presenteeism to employers. Pharmacoeconomics. 2009; 27: 365-378.
3. Cancelliere C, Cassidy JD, Ammendolia C, Cote P. Are workplace health
promotion programs effective at improving presenteeism in workers? a
systematic review and best evidence synthesis of the literature. BMC Public
Health. 2011; 11: 395.
4. Dalal RS. A meta-analysis of the relationship between organizational
citizenship behavior and counterproductive work behavior. J Appl Psychol.
2005; 90: 1241-1255.
5. Campbell J.P. Modeling the performance prediction problem in industrial and
organizational psychology. In: Dunnette MD, Hough LM, editors. Handbook of
industrial and organizational psychology. Palo Alto, CA, US: Consulting
Psychologists Press; 1990, Vol.1 (2nd ed.). p. 687-732.
6. Rotundo M, Sackett PR. The relative importance of task, citizenship, and
counterproductive performance to global ratings of performance: a policy-
capturing approach. J Appl Psychol. 2002; 87(1): 66-80.
7. Koopmans L, Bernaards CM, Hildebrandt VH, Schaufeli WB, De Vet HCW, Van
der Beek AJ. Conceptual frameworks of individual work performance - A
systematic review. Journal of Occupational and Environmental Medicine.
2011; 53(8): 856-866.
8. Borman WC, Motowidlo SJ. Expanding the criterion domain to include
elements of contextual performance. In: Schmitt N, Borman WC, editors.
Personnel Selection in Organizations San Francisco, CA: Jossey Bass; 1993. p.
71-98.
9. Griffin MA, Neal A, Parker SK. A new model of work role performance: positive
behavior in uncertain and interdependent contexts. Academy of
Management Journal. 2007; 50: 327-347.
10. Koopman C, Pelletier KR, Murray JF, et al. Stanford Presenteeism Scale: health
status and employee productivity. Journal of Occupational and Environmental
Medicine. 2002; 44: 14-20.

79
Chapter 3

11. Reilly MC, Zbrozek AS, Dukes EM. The validity and reproductibility of a work
productivity and activity impairment instrument. Pharmacoeconomics. 1993;
4: 353-365.
12. Kessler RC, Barber C, Beck A, Berglund P, Cleary PD, McKenas D, et al. The
World Health Organization Health and Work Performance Questionnaire
(HPQ). Journal of Occupational and Environmental Medicine. 2003; 45: 156-
174.
13. Podsakoff PM, MacKenzie SB. A second generation measure of organizational
citizenship behavior. Indiana University, Bloomington; 1989.
14. Van Scotter JR, Motowidlo SJ. Interpersonal facilitation and job dedication as
separate facets of contextual performance. Journal of Applied Psychology.
1996; 81(5): 525-531.
15. Bennett RJ, Robinson SL. Development of a Measure of Workplace Deviance.
Journal of Applied Psychology. 2000; 85(3): 349-360.
16. LePine J.A., Erez A., Johnson D.E. The nature and dimensionality of
organizational citizenship behavior: a critical review and meta-analysis. J Appl
Psychol. 2002; 87(1): 52-65.
17. Beaton D., Bombardier C., Escorpizo R., Zhang W., Lacaille D., Boonen A., et
al. Measuring worker productivity: frameworks and measures. J Rheumatol.
2009; 36(9): 2100-2109.
18. Evans C.J. Health and work productivity assessment: state of the art or state
of flux? J Occup Environ Med. 2004; 46(6): S3-S11.
19. Hunt S.T. Generic work behavior: an investigation into the dimensions of
entry-level, hourly job performance. Personnel Psychology. 1996; 49(1): 51-
83.
20. Mattke S, Balakrishnan A, Bergamo G, Newberry SJ. A review of methods to
measure health-related productivity loss. The American Journal of Managed
Care. 2007; 13(4): 211-217.
21. Baarda, De Goede, Van Dijkum. Basisboek Statistiek met SPSS.
Groningen/Houten, The Netherlands: Wolters-Noordhoff bv; 2003.
22. Williams LJ, Anderson SE. Job satisfaction and organizational commitment as
predictors of organizational citizenship and in-role behaviors. Journal of
Management. 1991; 17(3): 601-617.
23. Karidi MV, Papakonstantinou K, Stefanis N, Zografou M, Karamouzi G, Skaltsi
P, et al. Occupational abilities and performance scale: Reliability-validity

80
 Identifying and Selecting Indicators

assessment factor analysis. Social Psychiatry and Psychiatric Epidemiology.

2005; 40(5): 417-424.
24. Griffiths RD. A standardized assessment of the work behavior of psychiatric
patients. British Journal of Psychiatry. 1973; 123: 403-408.
25. Lerner D, Amick BC, Rogers WH, Malspeis S, Bungay K, Cynn D. The Work
Limitations Questionnaire. Medical Care. 2001; 39(1): 72-85.
26. Endicott J, Nee J. Endicott Work Productivity Scale (EWPS): A new measure to
assess treatment effects. Psychopharmacol Bull. 1997; 33(1): 13-16.
27. Stewart WF, Ricci JA, Leotta C, Chee E. Validation of the work and health
interview. Pharmacoeconomics. 2004; 22: 1127-1140.
28. Wang G, Netemeyer RG. Salesperson creative performance:
Conceptualization, measurement, and nomological validity. Journal of
Business Research. 2004; 57(8): 805-812.
29. Greenslade J.H., Jimmieson N.L. Distinguishing between task and contextual
performance for nurses: Development of a job performance scale. J Adv Nurs.
2007; 58(6): 602-611.
30. Pulakos ED, Arad S, Donovan MA, Plamondon KE. Adaptability in the
Workplace: Development of a Taxonomy of Adaptive Performance. Journal of
Applied Psychology. 2000; 85(4): 612-624.
31. Spector P.E., Fox S., Penney L.M., Bruursema K., Goh A., Kessler S. The
dimensionality of counterproductivity: Are all counterproductive behaviors
created equal? J Vocat Behav. 2006; 68: 446-460.
32. De Vet H, Terwee C, Mokkink L, Knol D. Measurement in Medicine. Cambridge
University Press; 2011.

81

View publication stats