Technical Report 1249

Team Composition Optimization:

The Team Optimal Profile System (TOPS)

Jamie S. Donsbach, Scott I. Tannenbaum,

and George M. Alliger
The Group for Organizational Effectiveness

John E. Mathieu
University of Connecticut

Eduardo Salas
University of Central Florida

Gerald F. Goodwin and Kimberly A. Metcalf

U.S. Army Research Institute

June 2009

United States Army Research Institute

for the Behavioral and Social Sciences

Approved for public release: distribution is unlimited

U.S. Army Research Institute
for the Behavioral and Social Sciences

A Directorate of the Department of the Army

Deputy Chief of Staff, G1

Authorized and approved for distribution:

BARBARA A. BLACK, Ph.D. MICHELLE SAMS, Ph.D.

Research Program Manager Director
Training and Leader Development
Division

Research accomplished under contract

for the Department of the Army

The Group for Organizational Effectiveness

Technical review by

Gregory A. Ruark, U.S. Army Research Institute

Brooke Schaab, U.S. Army Research Institute

NOTICES

DISTRIBUTION: Primary distribution of this Technical Report has been made by ARI.
Please address correspondence concerning distribution of reports to: U.S. Army
Research Institute for the Behavioral and Social Sciences, Attn: DAPE-ARI-ZXM,
2511 Jefferson Davis Highway, Arlington, Virginia 22202-3926.

FINAL DISPOSITION: This Technical Report may be destroyed when it is no longer

needed. Please do not return it to the U.S. Army Research Institute for the Behavioral
and Social Sciences.

NOTE: The findings in this Technical Report are not to be construed as an official
Department of the Army position, unless so designated by other authorized documents.
 REPORT DOCUMENTATION PAGE

1. REPORT DATE (dd-mm-yy) 2. REPORT TYPE 3. DATES COVERED (from. . . to)

June 2009 Final December 2006 – December 2007

4. TITLE AND SUBTITLE 5a. CONTRACT OR GRANT NUMBER

Team Composition Optimization: The Team Optimal Profile W91WAW-07-P-0022
System (TOPS)
5b. PROGRAM ELEMENT NUMBER
622785970
6. AUTHOR(S) 5c. PROJECT NUMBER
Jamie S. Donsbach, Scott I. Tannenbaum and George M. Alliger A790
(The Group for Organizational Effectiveness); John Mathieu
(University of Connecticut); Eduardo Salas (University of 5d. TASK NUMBER
Central Florida); Gerald F. Goodwin and Kimberly A. Metcalf 333
(U.S. Army Research Institute) 5e. WORK UNIT NUMBER

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION REPORT NUMBER

The Group for Organizational Effectiveness (gOE)
727 Walden Ponds Road
Albany, NY 12203

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. MONITOR ACRONYM

U.S. Army Research Institute for the Behavioral and Social ARI-FLRU
Sciences
851 McClellan Avenue
Fort Leavenworth, KS 66027-1360 11. MONITOR REPORT NUMBER
Technical Report 1249
12. DISTRIBUTION/AVAILABILITY STATEMENT

Approved for public release; distribution is unlimited.

13. SUPPLEMENTARY NOTES
Contracting Officer’s Representative and Subject Matter POC: Dr. Kimberly A. Metcalf

14. ABSTRACT (Maximum 200 words):

Teams have become strategic features in organizations. Research and practice suggest team effectiveness is driven
considerably by the mix of team member attributes. Given the impact a team’s composition has on its objectives, private industry
and military leaders place a premium on making optimal team staffing decisions. Nonetheless, the challenges associated with
achieving optimal team composition are significant and indicate a need for a tool/system to help commanders optimize personnel
allocation. Accordingly, this report lays the foundation for a system that incorporates the elements required to help leaders
optimize team composition. For our first task, leaders with extensive team staffing experience were interviewed to uncover the
implicit decision models used by team staffing experts. Supplementing extant research, the interviews contributed to our second
task: the development of a team composition decision taxonomy. The taxonomy defines and organizes elements of the team
staffing decision domain. The interviews and taxonomy culminated in the development of a generic, customizable team
composition optimization algorithm that models team composition-effectiveness relationships. Finally, we designed a
framework/methodology for a Team Optimal Profiling System (TOPS) and demonstrated its use for making an optimal team
composition decision.
15. SUBJECT TERMS
Teams, team composition, team composition optimization, personnel allocation, team staffing

SECURITY CLASSIFICATION OF 19. LIMITATION OF 20. NUMBER 21. RESPONSIBLE PERSON

ABSTRACT OF PAGES
16. REPORT 17. ABSTRACT 18. THIS PAGE Ellen Kinzer
Unclassified Unclassified Unclassified Unlimited 34 Technical Publications Specialist
703-602-8049

i
ii
Technical Report 1249

Team Composition Optimization:

The Team Optimal Profile System (TOPS)

Jamie S. Donsbach, Scott Tannenbaum, and George M. Alliger

The Group for Organizational Effectiveness

John Mathieu
University of Connecticut

Eduardo Salas
University of Central Florida

Gerald. F. Goodwin and Kimberly Metcalf

U.S. Army Research Institute

Fort Leavenworth Research Unit (Leader Development)

Stanley M. Halpin, Chief

U.S. Army Research Institute for the Behavioral and Social Sciences
2511 Jefferson Davis Highway, Arlington, Virginia 22202-3926

June 2009

Army Project Number Personnel Performance and

622785A790 Training Technology

Approved for public release: distribution is unlimited

iii
iv
TEAM COMPOSITION OPTIMIZATION: THE TEAM OPTIMAL PROFILE SYSTEM
(TOPS)

EXECUTIVE SUMMARY

Research Requirement:

As reflected in the Quadrennial Defense Review Report (2006), our military is facing continually
changing conditions which requires greater agility, flexibility, and partnering. These conditions
only magnify the importance of being able to rapidly form and re-form teams to optimize our
force capability. Whether the question is who, from an available pool of Soldiers, is the best
replacement member for a given combat team, or how a pool of personnel are best allocated to
several teams, or whether units should be rotated intact or in pieces, team staffing is a critical
issue facing today’s military forces. Nonetheless, the challenges associated with achieving
optimal team composition are significant and indicate a need for a tool/system to help
commanders optimize personnel allocation. Accordingly, this work lays the foundation for a
system that incorporates the elements required to help leaders optimize team composition.

Procedure:

Military and private sector leaders with extensive team staffing experience were interviewed to
uncover the implicit decision models used by team staffing experts. Supplementing extant
research, the interviews contributed to the development of a team composition decision
taxonomy that defines and organizes elements of the team staffing decision domain. The
interviews and taxonomy culminated in the development of a generic, customizable team
composition optimization algorithm that models relationships of team composition with team
effectiveness. Finally, a framework/methodology for a Team Optimal Profiling System (TOPS)
was developed and its use for making an optimal team composition decision was demonstrated.

Findings:

Interviews with team staffing experts from a variety of industries and jobs suggest most leaders
find team composition decisions to be complex. The interviews identified decision factors and
constraints that appear to represent the broad spectrum of factors/constraints relevant to team
staffing decision scenarios across most team types. Although the specific factors a leader
considers in his team composition decision depends in part on the team’s function, structure, and
environment, most team staffing decision makers try to consider multiple factors and face a
variety of situational constraints when staffing teams. The interviews also indicted that most
leaders are not equipped with the tools, support, time, or processes needed to effectively manage
the information relevant to making effective team staffing decisions.

Along with the team staffing decision types, the major types of factors and constraints identified
during the interviews were organized into a team staffing decision taxonomy. The taxonomy,
which is grounded in team composition research and theory, highlights elements that should be
considered when making team staffing decisions and captured within a team composition
optimization tool. These elements include: 1) types of team staffing decisions, 2) factors decision

v
makers consider when staffing teams, 3) factors that define the team staffing process, 4) factors
that define the candidate pool, and 5) constraints placed on team staffing decisions.

The interview results and the taxonomy influenced the foundation of a TOPS algorithm and
framework. The TOPS algorithm simultaneously models individuals’ knowledge, skills, abilities,
and other characteristics (KSAOs) as related to both their job or role performances and the
accomplishment of joint team tasks. Moreover, the algorithm drives a methodology for
determining “ideal mixes” or “profiles” of team composition and offers an index of team
composition which overcomes the weaknesses of traditional team composition approaches.

A conceptual framework that specifies the functional characteristics of the TOPS system was
also developed. The generic TOPs framework consists of several interlinking modules, which
together with the algorithm will provide a user-friendly software application that accomplishes
several objectives. The TOPS algorithm and framework are generic in that they will be
applicable for aiding a wide variety of staffing decisions, from identifying an individual team
member’s replacement, to optimizing large scale force deployment. Although the algorithm and
system framework are generic, the specific elements are customizable to fit specific applications.

An illustration of how TOPS can be used for making a specific team composition decision within
a specific domain was also developed. Specifically, a storyboard prototype illustrates how a
decision maker could use TOPS for making an optimal single team member replacement
decision. The illustration demonstrates how the candidate with the best fit for the individual
position, who would be selected for the team under a traditional HR selection approach, might
not result in the most effective team. This illustrates the algorithm’s ability to help a decision
maker balance the competing demands of the individual position and the team, thereby offering
the greatest combined value of the replacement.

Utilization and Dissemination of Findings:

The interview results suggest there is clearly a need for tools to help leaders make effective team
staffing decisions. An important contribution of the interviews and taxonomic work is it began to
bridge a gap between team composition optimization theory and the current state of team staffing
practices and needs. An immediate benefit of the taxonomy is it can guide leaders as they make
decisions that impact the composition of their teams. As a more long-term benefit, researchers
can draw from the taxonomy to ensure the future team composition decision making algorithms
and tools are grounded in actual team staffing experiences.

The TOPS algorithm forwards the application of team composition research since it represents a
more flexible method to index and study the influences of members’ KSAOs to the collective
competencies of teams. The algorithm has potential to provide valuable insights if it is
customized for particular domains and situations. The TOPS framework and methodology
articulates how a team optimization system might work in practice to help commanders work
through real team staffing scenarios. With an enhanced understanding of how a team
composition algorithm can be applied, researchers can now move onto: 1) refining and validating
the elements of algorithms that model relationships of team composition with team effectiveness,
and 2) building prototype team optimization decision support tools.

vi
TEAM COMPOSITION OPTIMIZATION: THE TEAM OPTIMAL PROFILE SYSTEM
(TOPS)

CONTENTS

Page

INTRODUCTION ................................................................................................................ 1
Team Composition: Background .............................................................................. 2

OBJECTIVES ....................................................................................................................... 4
Define the Team Staffing/Optimization Decision Making Domain ......................... 4
Develop a First Generation, Generic Customizable Team Composition
Optimization Algorithm that Models Relationships of Team Composition with
Team Effectiveness .................................................................................................. 4
Demonstrate the Use of the Generic Optimization Algorithm with a Specific
Occupational Domain ............................................................................................... 4

PROJECT TASK PROCEDURES AND RESULTS ........................................................... 5

Task 1: Conduct Structured Interviews with Team Staffing Subject Matter
Experts ...................................................................................................................... 5
Interview Participant Sample........................................................................ 5
Knowledge Elicitation Process and Procedures............................................ 6
Interview Results .......................................................................................... 6
Task 2: Develop a Taxonomy of the Team Staffing Decision Domain.................... 12
Team Staffing Decision Types...................................................................... 12
Factors that Influence Team Staffing Decisions........................................... 13
Factors that Define the Team Staffing Decision Process.............................. 14
Factors that Define the Candidate Pool ........................................................ 14
Constraints .................................................................................................... 15
Task 3: Build a Generic Team Composition Optimization Algorithm..................... 15
TOPS Framework ......................................................................................... 16
Foundational Modules .................................................................................. 16
Decision and Team Composition Results Modules ...................................... 17
TOPS Customization Process ....................................................................... 17
TOPS Population Process ............................................................................. 17
TOPS Algorithm ........................................................................................... 17
TOPS Demonstration ................................................................................... 19

THE FUTURE OF TOPS: TOPS LITE................................................................................ 20

ANTICIPATED APPLICATIONS AND BENEFITS FOR GOVERNMENT AND/OR

PRIVATE SECTOR ............................................................................................................. 21

vii
 Page
CONTENTS

REFERENCES ..................................................................................................................... 23

APPENDIX A. TEAM STAFFING EXPERT INTERVIEW PROTOCOL ........................ A-1

LIST OF TABLES

TABLE 1. TYPES OF TEAM STAFFING DECISIONS DISCUSSED IN SME 7

INTERVIEWS .....................................................................................................................

TABLE 2. FACTORS WHICH INFLUENCED SME TEAM STAFFING DECISIONS .. 7

TABLE 3. TYPES OF TEAM STAFFING DECISIONS .................................................... 13

LIST OF FIGURES

FIGURE 1. OVERVIEW OF PHASE 1 WORK PLAN ...................................................... 5

FIGURE 2. REASONS WHY A TEAM OF STRONG INDIVIDUAL PERFORMERS

CAN STRUGGLE ................................................................................................................ 8

FIGURE 3. DECIDING FACTOR FOR CHOOSING BETWEEN EQUALLY

QUALIFIED CANDIDATES............................................................................................... 9

FIGURE 4. REASONS FOR NOT CHOOSING THE MOST QUALIFIED

CANDIDATE ...................................................................................................................... 9

FIGURE 5. TYPE OF TEAM CRITERIA USED; FORMAL VS. INFORMAL ................ 10

FIGURE 6. FACTORS CONSIDERED WHEN EVALUATING TEAM STAFFING

DECISIONS.......................................................................................................................... 11

FIGURE 7. INTERACTION AMONG MODULES IN TOPS............................................ 16

viii
 Introduction

“If you don’t have the right mix of people, the team is dead from the start.” – CEO and
Chairman of CNA

To enable more rapid and adaptive reactions to change and to maximize performance,
organizations are reconfiguring their workforce and restructuring their work around teams (Ilgen,
Hollenbeck, Johnson, & Jundt, 2005; Kozlowski & Ilgen, 2006; Salas, Bowers, & Cannon-
Bowers, 1995). Although team effectiveness is dependent upon a variety of factors, research and
practice suggests the best teams are well designed up-front. Specifically, the mix of knowledge,
skills, abilities, and other characteristics (KSAOs) of team members contributes significantly to
the amount of effort team members apply to a task, their coordination, and ultimately to team
task performance (Bell, 2007; Ilgen, 1999). Given the powerful impact a team’s composition has
on its processes and objectives, team leaders and organizational decision makers place a
premium on making optimal team staffing decisions.

The team literature, as well as our own interviews with multiple team staffing subject matter
experts (see quotes embedded throughout this report), indicate the need to optimize team
composition is pervasive among a variety of industries, including the armed forces. As reflected
in the Quadrennial Defense Review Report (2006), our military is facing continually changing
conditions which requires greater agility, flexibility, and partnering. These conditions only
magnify the importance of being able to rapidly form and re-form teams to optimize our force
capability. Whether the question is who, from an available pool of Soldiers, is the best
replacement member for a given combat team, or how a pool of personnel are best allocated to
several teams, or whether units should be rotated intact or in pieces, team staffing is a critical
issue facing today’s military forces.

Researchers and practitioners have shown great interest in team composition as a means for
increasing team performance because team composition can be manipulated through placement
and selection. However, the challenges associated with achieving optimal team composition are
non-trivial. Given the numerous constraints commanders face, and the multitude of factors they
must consider when making team staffing decisions, optimizing the composition of teams poses
a particularly arduous challenge to today’s military commanders.

“I first consider the mission complexity and visibility. I start by filling the flight lead and
the critical positions. I first ask: Who is available? What kind of experience do they
have? Time dictates the extent to which we can consider other factors in our (team
staffing) decisions. If we have time, we definitely think about things like whether guys can
work together, or if they should be doing something else.” – Lieutenant Colonel, United
States Air Force.

“I often didn’t get my top picks because the person’s commander wouldn’t let them go, or
the personnel system wouldn’t let me have them.” – Retired Commander of the F-16
Division, United States Air Force Weapons School

This challenge, spread throughout the military system, indicates a call for a tool/system to help
commanders make more rapid and optimal team staffing decisions, and ultimately maximize the
1
effectiveness of force deployment overall. Such a system could also allow team leaders to fully
explore multiple team staffing options, adjusting the fit if necessary.

The key to such a system is to employ a sophisticated algorithm that considers individuals’
position specific KSAOs, as well as the members’ interdependencies and team-level composition
profiles. Specifically, this system must simultaneously incorporate a variety of considerations
including, how one: 1) best aligns members’ KSAOs with job/role demands; 2) optimizes the
mix of members’ KSAOs that are critical for team functioning; 3) incorporates the fact that
candidate pools, members, teams, pre-requisites, and situations change over time; and 4)
accounts for membership movement in and out of teams. The engine of this system must be an
algorithm that accommodates all of these drivers. Such an algorithm must be generic enough to
be applicable to a wide variety of staffing decisions such as choosing a given member’s
replacement, to optimizing large scale force allocations. At the same time, however, the
algorithm must be readily customizable so it is specific enough to provide valuable insights in
particular circumstances.

The vision for this line of research is the development of a fully functional, user friendly system,
which is driven by an algorithm that incorporates all of the elements required for making optimal
team composition decisions. Accordingly, we constructed a framework for a team composition
system, the Team Optimal Profiling System (TOPS). The elements of the TOPS framework are
based on the extant team composition research and grounded by interviews with team staffing
subject matter experts. In the section that follows, a brief overview of the research is provided
that supports a need for a team optimization system. We then describe the results of our work,
including the subject matter expert interviews and the development of a team staffing decision
taxonomy, which culminated in the full scale TOPS framework and the vision for a “TOPS
Lite.” Finally, the anticipated benefits of TOPS for optimizing the composition of teams are
described for the private and government sectors.

Team Composition: Background

Team composition research indicates that team processes and effectiveness are impacted by
aspects of group composition such as members’ skills, job and organizational experiences, values
and group heterogeneity (e.g., Bell, 2007; Gladstein, 1984). Yet, challenges concerning how to
best operationalize and index team composition, and how to model its influences, continue to
plague the field (Arrow & McGrath, 1995; Ilgen, 1999; Ilgen et al., 2005; Kozlowski & Bell,
2003; Morgan & Lassiter, 1992).

Historically, work on team composition has essentially progressed along two lines: 1) an
individual-based approach; and 2) a team-based approach. The first approach derives from the
application of traditional individual-focused personnel psychology or Human Resource (HR)
frameworks. This approach seeks to optimize the fit between individuals’ KSAO profiles and the
positions or roles they occupy. Based on a thorough job analysis, the relative importance of
various dimensions or tasks that must be accomplished in any given position are identified, and
the individual KSAOs that are important for performing those tasks are specified and used for
staffing decisions (e.g., McCormick, 1979; Stevens & Campion, 1994). With the advent of team
based organizations, KSAOs have been extended to address team-relevant competencies (e.g.,

2
Hirschfeld, Jordan, Field, Giles, & Armenakis, 2006; Stevens & Campion, 1994). Nevertheless,
this approach remains an individually-oriented design and assumes team effectiveness will be
optimized to the extent members are well suited for the positions or roles they occupy.

Several disciplines have operationalized team composition in terms of summary indices of

members’ characteristics. This team-based approach indexes composition in terms of descriptive
statistics of members’ KSAOs. The key feature of these approaches is that team members’
KSAOs are considered collectively rather than in terms of individuals’ position/role fit. For
example, researchers have employed indices of central tendency, such as average member
tenure, task related knowledge, and agreeableness (Barrick, Stewart, Neubert, & Mount, 1998;
Devine & Philips, 2001) as predictors of team effectiveness. Other researchers have focused on
the diversity of member attributes, such as functional backgrounds, demographics, or
personalities (Jackson, 1992; Jackson, Brett, Sessa, Cooper, Julin, & Peyronnin, 1991).
Implicitly, both the central tendency and diversity approaches weigh the relevant attributes of all
members equally. In other words, no members or positions individuals occupy are viewed as any
more (or less) important than others.

Another variation of the team-based approach employs a relative or selected score approach
which focuses upon the attributes of one, or a subset, of members’ KSAOs. Based mostly on the
work of Steiner (1972), these studies have considered attributes such as the competencies of the
weakest or strongest member (Halfhill, Nielsen, Sundstrom, & Weilbaecher, 2005; LePine,
Hollenbeck, Ilgen, & Hedlund, 1997). More recently, Ellis, Bell, Ployhart, Hollenbeck and Ilgen
(2005) argued that the knowledge levels of members who occupy more critical team roles would
exhibit higher correlations with team outcomes than would the knowledge levels of members
who occupy less critical roles. In this approach, the relative standing of individuals implies that
some members’ KSAOs are more important to success than are others. Depending on the
situation, such focal attributes could be anything from the physical fitness of the weakest
member, to the capabilities of the member with the greatest leadership qualities. In effect, this
most recent approach is advocating a network style approach to the study of team composition.
In concert with such an approach, rather than focusing on individuals as though they are purely
independent entities, or treating the entire team only as a unified whole, we are advocating that
team composition be viewed as a set of interlocking dyadic relationships along the lines of
network theory and analyses. In this fashion, one can detail the relative interdependencies among
roles within a team (and perhaps across teams) and the extent to which they are symmetrical, and
thereby better represent the nature of the relationships within the team. Moreover, employing a
network style approach should allow for more appropriate team-level operationalization of the
target dimensions, which has been shown to provide greater validity for predicting team
performance (Arthur, Bell, & Edwards, 2007; Bell, 2007).

In summary, the extant literature provides us with many insights. From the individual-level
approach, team effectiveness is likely to be enhanced to the extent that members’ possess
profiles of KSAOs that are well aligned with the demands of the roles they occupy. From the
team-based approach, team effectiveness is enhanced to the extent some overall combination of
member attributes is optimized, whether that is in terms of some index of central tendency,
variance, or a relative score. However, the literature is silent in terms of the relative priority of
maximizing individual member position/role fits vs. establishing an ideal team mix. Moreover,

3
the literature to date tends to treat individual members as though they were unique contributors –
or as equal contributors to a unified whole team composite. In contrast, our approach adopts
more of a network approach and details the nature of the interdependencies among different
members. For example, consider an instance where a communications officer for a given team
needs to be replaced. Consider further that the incumbent’s background contributed greatly to the
demographic and functional diversity of the team, and the person has been the primary
“peacekeeper” when other members were in conflict. What factor(s) should be given priority
when choosing a replacement? Is it better to get the “best communications officer” at the
expense of team diversity and the peacekeeper role? Is it better to select a qualified
communications officer who might contribute more to the team attributes but is not the most
skilled communications officer available? What is needed, therefore, is an algorithm that will
help a decision maker balance these competing demands and make a selection that will optimize
the combined profile of individual and team characteristics that offer the greatest combined value
of the replacement.

Objectives

The ultimate objective for this project was to develop and illustrate the use of a prototype
algorithm and TOPS decision-aid tool for the placement of personnel onto teams. As proximal
goals, we set out to clarify the domain of team optimization decisions and develop a prototype
algorithm to assist in the best allocation of personnel to teams. The underlying purpose was to
bridge the gap between team composition optimization theory and the current state of personnel
allocation procedures. Accordingly, the technical objectives were as follows:

Define the team staffing/optimization decision making domain

This incorporates the types of decisions commanders face when staffing teams, the types of
factors they consider (i.e., in terms of both individual role and team attributes) and constraints
they deal with during such decisions, and how they balance filling individual positions with
existing team characteristics and dynamics.

Develop a first-generation, generic, customizable team composition optimization algorithm that

models relationships of team composition with team effectiveness

The plan for the algorithm was that it would be designed to fill a vacant position on an existing
team. The objective was to develop a generic algorithm composed of parameters that could be
customized and weighted to fit specific target domains. The algorithm was intended to be the
engine of the first-generation TOPS decision-aid tool. As part of this effort, a conceptual
framework for modifying team composition algorithms has been articulated.

Demonstrate the use of the generic optimization algorithm with a specific occupational domain

The purpose of this objective was to showcase how a customized, SME-driven, prototype
decision aid could be used to enable a commander to make quick, informed team staffing
decisions for filling vacant positions within a particular domain (e.g., Special Forces). The
objective was to illustrate how a TOPS decision-aid tool could help optimize a team’s
composition for a realistic team composition decision.

4
To meet these objectives, three primary tasks as well as a demonstration were completed. These
tasks are illustrated in Figure 1 and the procedures and results of each task are discussed in detail
below.

TASK 1
Interview team
staffing SMEs

TASK 3
Build generic Demonstrate
TOPS algorithm algorithm

TASK 2
Develop team
staffing decision
taxonomy

Figure 1. Overview of Phase 1 Work Plan

Project Task Procedures and Results

Task 1: Conduct Structured Interviews with Team Staffing Subject Matter Experts

During Task 1, team leaders with extensive experience making team staffing decisions were
interviewed. The purpose of the interviews was to uncover and understand the implicit decision
models used by individuals who are experts at making decisions that impact the composition of
their team(s). For example, what kinds of team staffing decisions do team leaders or commanders
need to make; what individual, team, and situational factors do they consider; what processes do
they use to make team staffing decisions (e.g., how do they balance or weigh individual vs. team
factors); and what kinds of constraints place limitations on their decisions. The primary purpose
of this task was to ensure the decision making algorithms and tools developed in later tasks and
phases are grounded in actual team staffing needs and experiences.

Interview participant sample. Participants in the team staffing interviews were individuals who
were considered team staffing experts in their respective organizations. Participants were
identified based on their extensive experience and expertise in making decisions that influence
the composition of teams. Our interview sample consisted of 21 team staffing subject matter
experts (SMEs) from 17 well-respected organizations, representing a cross-section of industries
and the military. Represented organizations included: United States Air Force, United States
Marine Corps, United States Geological Survey, Boeing, BP, The Walt Disney Company,
Johnson & Johnson, Crayola, and Merck. While all interview participants were in leadership
roles, they held a variety of ranks and job titles, such as Lieutenant Colonel, Major, Captain,
Chair and CEO, Vice President of Human Resources, Fire Department Chief, Senior Vice
President of Distribution and Operations, and Director of Organization and Talent. On average,

5
the team staffing SMEs had over 15 years experience staffing teams. The teams staffed by the
SMEs were representative of all the categories in Sundstrom’s team typology (i.e., production,
service, management, project, action, advisory/parallel) (Sundstrom et al., 1990; 2000).
Representation of various team types is beneficial for this investigation given recent team
research, which suggests the influence of some team composition variables on team effectiveness
may be dependent upon the type of work the team performs (Bell, 2007).

Knowledge elicitation process and procedures. Appendix A provides a structured interview

protocol that was developed to elicit information from the SMEs about their team staffing
decisions. The first section of the interview asked the interview participants about the types of
staffing decisions faced. For example, do they need to: a) choose replacements for individual
positions on teams, b) staff entire teams, and/or c) optimize the mix of individuals on one or
more teams? The interviews also prompted the SMEs to describe specific team staffing situations
they have encountered, and to “think aloud” and explain the processes they used for making the
decision (e.g., amount and source of information considered; temporal dynamics of the process).
When describing specific team staffing experiences, the SMEs were also prompted to discuss the
individual and team factors they considered, as well as the relative importance of these factors.
The SMEs also described the kinds of constraints placed on staffing decisions in a
natural/operational environment. Additionally, SMEs were asked to describe experiences in
which they needed to balance filling individual positions and roles with team needs or dynamics.

Interview results. As illustrated in Figure 1, the interview and taxonomy development processes
were conducted in concert and informed one another. Therefore, in addition to the interview
results described here, much of the insight gained from the interviews is reflected in the
discussion of the taxonomy in the following section.

The open-ended interview responses were content coded and analyzed. During an early portion
of the interview, interview respondents were asked to describe a specific situation in which they
needed to make a decision that influenced the composition of a team or teams. Interview
respondents were instructed to describe the situation of their choice, regardless of the function of
the team, whether it involved existing teams or new teams, filling one or multiple team positions,
or composing one or multiple teams. As shown in Table 1, the majority of specific team staffing
decisions discussed by the interview participants represented five major types of team staffing
decisions. The two most frequently cited team staffing decisions required filling multiple
positions on a single team, but differed in terms of whether the team was new or existing. The
most frequently cited (48%) type of team staffing experience was a single team formation
decision and involved assigning multiple people to a new team. In contrast, the second most cited
(22%) decision, a multiple member replacement decision, involved assigning more than one
person to an existing team. A decision that involved assigning a single individual to an existing
team, a team member replacement decision, was discussed in 13% of the scenarios discussed.
The next most frequently discussed decision type was multiple team formation. Discussed within
8% of the responses, this decision involved assigning people to multiple new teams. Just 4% of
the respondents described scenarios where new talent was distributed across multiple existing
teams. These results suggest team leaders are faced with making a variety of team staffing
decisions, more frequently with a single team, but at times involving multiple teams.

6
Table 1. Types of Team Staffing Decisions Discussed in SME Interviews

Team Staffing Decision Type Percentage

Single team formation 48%
Multiple member replacement 22%
Team member replacement 13%
Multiple team formation 9%
New talent distribution 4%
Other 4%

While the interviews with the team staffing experts revealed a certain level of savvy and
sophistication in the strategies used to make team staffing decisions, it also suggested that even
the most experienced team leaders find these decisions to be cognitively complex and
challenging. One reason for this is the number of factors the leaders want or need to consider.
Some of these are individual factors (e.g., individual KSAOs), others are at the team level (e.g.,
fit with existing team members), and a third portion are mission or task-based factors (e.g.,
criticality of the mission). Table 2 displays the categories of factors SMEs considered during
their team staffing scenarios, and the number of times each factor was mentioned across the 20
interviews.

Table 2. Factors which Influenced SME Team Staffing Decisions

Factor Frequency
Individual
Availability 2
Individual KSAOs 15
Amount/Type of experience 8
Previous performance record 2
Team
Fit with the existing team 9
Enhanced team diversity 7
Fill team gaps 5
Plan for future team needs 2
Provide back-up for others on team 3
Features of the task/mission 2
Other 6

These results suggest leaders need or try to weigh a number of factors when making team
staffing decisions. It is also clear team staffing experts often consider team needs and dynamics
when making staffing decisions.

7
The important role of team needs, and the need to balance these with individual factors, is also
evident in the interview participants’ responses to three other experience-based questions. The
first of these three questions asked, “Have you ever had an experience where each team member
was good at their individual job, but the team as a whole struggled? Why do you think the team
struggled?” Several of the 14 team staffing experts who responded to this question indicated that,
in their experience, multiple factors contributed to this situation. Each factor mentioned was
counted separately and categorized. As shown in Figure 2, it seems there are a variety of reasons
why a team of strong individuals can struggle as a team. The reason most frequently cited (30%)
by the team staffing interview participants was that strong individual performers are often not
team oriented. For example, they are more concerned with their personal performance and
outcomes than they are with working well as a team. Also, multiple team staffing experts
mentioned they have seen teams of strong individuals struggle because of poor leadership (20%),
individual or interpersonal differences between team members (15%), or role clarity issues
(10%).

“Yes, I see this happen more often than not. People are trained to be individual
contributors. They receive no training in or see no value of working in teams, so they are
not playing for the team.” - Manager, Organizational Capability, BP

“In one situation, team members, who were experts in their specialty, were not on the
same page because they could not identify with the team.” - Senior Director, Medtronic

35
30
25
Percent

20
15
10
5
0
Have not Role issues Poor Members Individual Other
experienced leadership were not differences
this team
oriented
Reason

Figure 2. Reasons why a Team of Strong Individual Performers can Struggle

The importance of balancing team needs with individual role needs was also demonstrated
through the responses to a question which asked, “Have you ever had an experience where more
than one person was highly qualified for a single position? If so, what was the deciding factor?”
As illustrated in Figure 3, four themes, or deciding factors, emerged from the results. Of the 18
respondents to this question, 45% indicated that when more than two candidates were equally
qualified for a single position, they picked the candidate who was a better fit with the team.

8
Another 17% of the respondents said they picked the candidate who brought more functional or
demographic diversity to the team, and 11% indicated they considered which candidate would
get a better developmental experience from the position. Finally, 6% stated they picked the
candidate that would fill a team competency gap.
40
35
30
25
Percent

20
15
10
5
0
Best team fit Would add team Greatest Would fill team Other
diversity developmental competency gap
need/Opportunity

Deciding Factor

Figure 3. Deciding Factor for Choosing Between Equally Qualified Candidates

The responses to another experience-based question further suggest leaders weigh team needs
along with individual and logistical needs when staffing individual positions. Specifically, this
question asked, “Have you ever had an experience where you decided not to pick the most
qualified person to fill a position. Why?” Figure 4 shows that 45% of the 20 people who
responded to this question indicated they have passed over someone who was most qualified for
a position because they were not a good fit for the team. Highlighting a more logistical factor,
15% indicated they did not pick the most qualified because that person was unavailable, or of
greater value filling some other role. Finally, 10% of respondents indicated they passed over the
most qualified to give someone else a developmental opportunity, and another 10% said rather
than picking the most qualified candidate, they decided to pick someone who brought greater
diversity to the team.
50
40
Percent

30
20
10
0
Most qualified was Picked someone To give someone Most qualified is Other
a poor team fit who added greater else developmental busy/unavailable
diverstiy to team opportunity
Reason

Figure 4. Reasons for Not Choosing the Most Qualified Candidate

9
Team staffing experts were also asked about their experiences with evaluating their team staffing
decisions. First, team staffing SMEs were asked to indicate whether they used formal evaluation
criteria (e.g., performance metrics), informal criteria (e.g., general feeling about the team’s
cohesion or performance), or some combination of formal and informal criteria to evaluate their
team staffing decisions. The participants’ responses to this question are categorized in Figure 5.
All of the 13 team staffing subject matter experts who responded to this question indicated they
rely on some informal criteria when evaluating their team staffing decisions. The majority (62%)
reported evaluating their team staffing decisions on informal criteria alone.

I don’t use formal metrics to evaluate my team staffing decisions. I’m more of a
process/results person. I look at how effective the team is.” – Lieutenant Colonel, United
States Air Force

The other 39% of the experts who answered this question reported employing some combination
of formal and informal evaluation criteria to evaluate their team staffing decisions.

“We have a lot of formal metrics that measures team productivity (e.g., the number of
jobs filled), customer satisfaction. We also, more informally, gauge how much the team
members like each other and talk to each other.” – Client Manager for IT Staffing, The
Hartford

70
60
50
Percent

40
30
20
10
0
Mainly uses informal evaluation criteria Uses both formal and informal criteria

Figure 5. Type of Criteria Used; Formal vs. Informal

In addition to asking about the formal or informal nature of the criteria used, the team staffing
experts were asked to describe the factors they consider and the signs they look for to help them
determine whether they have put together a good team. Each factor mentioned by a given
participant was counted separately. Therefore, the total number of factors mentioned (N = 32) is
greater than the number of responses to this question (N = 16). As shown in Figure 6, the factors
mentioned generally fell into two major categories. Specifically, within their responses, half of
the team staffing experts stated they considered the team’s performance or whether the team met

10
their objectives. Ten of the experts indicated they examine whether the team members get along
well with one another. Other factors considered by team staffing experts included team member
turnover, whether the teams needed to be reconfigured, whether the teams learned from their
errors, and whether the rest of the organization reacts positively to the team composition
decision.
60
50
40
Percent

30
20
10
0
Team meets objectives/performs Members get along Other
well
Factors Considered

Figure 6. Factors Considered when Evaluating Team Staffing Decisions

In sum, the results of the interviews with team staffing experts suggest leaders across industries
and jobs find team composition decisions to be complex. Although the specific factors a leader
considers in his team composition decision depends in part on the team’s function, structure, and
environment, most team staffing decision makers try to consider multiple factors and face a
variety of situational constraints when staffing teams.

Based on previous research in the area of team composition, it is not entirely surprising to learn
leaders find team composition decisions to be challenging and the factors and constraints
considered depend greatly on the team and situation (Bell, 2007; Ilgen, 1999). However, it was
somewhat surprising to discover that although many of the experts recognized a need to consider
and balance multiple factors in their team staffing decision process, they often could not consider
all of the information they would like to include because they did not have the tools, support,
time, or a process needed to effectively manage and analyze all of the relevant information.

The interview results identified decision factors and constraints that appear to be representative
of the broad spectrum of factors/constraints relevant to team staffing decision scenarios across
most team types. Although the domain of decision factors and constraints is likely to be infinite,
we believe a system which can incorporate and adjust for the ones uncovered during the
interviews will be flexible enough to handle most factors that drive team effectiveness. Along
with the team staffing decision types, the major types of factors and constraints identified during
the expert interviews were organized into a team staffing decision taxonomy (Task 2), which is
described in the following section.

11
Task 2: Develop a Taxonomy of the Team Staffing Decision Domain

The second task was to develop a taxonomy of the team staffing/optimization decision domain.
The taxonomy draws upon findings from team composition research and theory (e.g., Cannon-
Bowers, Tannenbaum, Salas, & Volpe, 1995; Mathieu, Gilson & Ruddy, 2006) and was
grounded by the results of the team staffing expert interviews. The taxonomy highlights elements
that should be considered when making team staffing decisions. These elements include: 1) types
of team staffing decisions, 2) factors decision makers consider when staffing teams, 3) factors
that define the team staffing process, 4) factors that define the candidate pool, and 5) constraints
placed on team staffing decisions. The results of this task have influenced the foundation of the
TOPS algorithm and framework, and should continue to inform future TOPS generations as we
test and refine the algorithm.

Team staffing decision types. As shown in Table 3, the team staffing decision domain can be
organized into six major types of decisions. The first three types are decisions relevant to
existing teams. The first type, Team Member Replacement, represents a fairly common scenario,
wherein a decision maker must place a single person onto a single existing team (e.g., hiring
someone to fill a vacant position on a work team). Multiple Member Replacement scenarios also
target a single existing team, but involve assigning multiple people to this team, as a leader might
decide to do in a situation where she is growing a team or refilling multiple vacant spots
concurrently. When making New Talent Distribution decisions, decision makers must place
multiple people on multiple existing teams (e.g., when members of a recent class of recruits are
allocated to multiple existing teams). In contrast to the first three types, the fourth and fifth types
of decisions represent situations where the decision maker forms new teams. Specifically, when
decision makers build a single new team from scratch they are making a Single Team Formation
decision. A Multiple Team Formation decision is similar, but requires the decision maker to
allocate people to multiple new teams, as often takes place when teams within a new
organization are formed. The final decision type, Reconfiguration, is one which represents a type
of highly complex team staffing decision and is relevant to an entire organization or unit (e.g., a
reorganization/restructure that involves the elimination of entire divisions and the redistribution
of the former staff within these divisions to other existing divisions. The ultimate objective is to
develop a TOPS system that can be utilized for making each of the existing and new team
decisions (numbers 1-5 in Table 3).

12
Table 3. Types of Team Staffing Decisions

Decision Type Decision Description Example

Existing Team Decisions
1. Team member Assigning a person to an existing Replacing a person who left a team
replacement team “I had to replace a leader on a single off-shore oil
rig. The previous leader was promoted.”
2. Multiple member Assigning people to multiple Replacing several people when their tour of duty ends
replacement positions on an existing team “I had to reorganize a team after I had promoted a
number of people. I had to back-fill their positions
and reorganize.”
3. New talent Assigning new people to several Assigning a new group of graduates to existing teams
distribution existing teams “We partnered with a local university, and filled the
additional plant positions with the group of new
graduates.”
New Team Decisions
4. Single team Assigning multiple people to a Forming a project or mission team
formation new team “I had to build a new helo crew.”
5. Multiple team Assigning people to several new Forming several teams at once
formation teams “I was tasked with filling hundreds of positions
within multiple departments of a new store.”
Organizational Decisions
6. Reconfiguration Assigning or re-assigning Restructuring a business unit post-merger
multiple people within the
organization to multiple existing
or new teams

Factors that influence team staffing decisions. The taxonomy also classifies factors that
influence team staffing decisions. As shown in Table 2, factors associated with the individual
candidates (e.g., individual KSAOs, interests, availability), the team (e.g., functional diversity
among the team, balance of experienced and new members, personality or work style fit between
the leader and members), or the task/mission (e.g., criticality of the mission, potential risks
involved) can all influence team composition decisions. While situational constraints may
prevent decision makers from being able to consider the full scope of factors they wish to
evaluate, it seems experienced decision makers try to consider multiple factors in their decisions.

“I typically look for how the person will fit with the team, someone who is flexible (can
fill other roles when needed), someone with certain skills (e.g., interpersonal) and
experience, and whether they can fill existing skill gaps on the team… I also look for
people to be able to back-up one another.” – Director of Workforce Development,
SEFCU

Ultimately, the TOPS user will be able to specify the factors that are relevant for their particular
domain and decision. The intent for future TOPS research and development is to use

13
computational modeling to assess the fidelity of the TOPS algorithm when considering different
factors and combinations of factors.

Factors that define the team staffing decision process. The taxonomy also includes factors that
define the processes people use to make team staffing decisions. As indicated by the team
staffing experts interviewed, the processes used for making team staffing decisions can vary on
several dimensions. First, the centrality of decision making can differ. For example, some team
leaders may staff an entire team on their own without input or help from others (central decision
making), while others divide the team staffing responsibility among several parties (distributed
decision making), as is sometimes done when staffing cross functional teams. Many use a
negotiated or coordinated process, which falls between central and distributed approaches (e.g.,
regional managers nominate team members and then negotiate with central decision makers to
ensure the team is staffed appropriately).

“I first identify the core team lead. I then work with the management team - those that
represent Medtronic’s seven functional disciplines (e.g., design, clinical, marketing) - to
select the other team members. Generally, the managers of each functional discipline
decide who to assign from their function, but I give some input.” - Senior Director for the
Core Development Program Management Office, Medtronic

Second, the amount, source, and formality of information considered is another factor that
defines the decision making process. For example, some decision makers have access to data on
all of the variables they would like to consider (full data). In most operational settings, team
staffing decisions are based on partial data (e.g., data on several key factors), or in some cases,
no quantitative data at all. Also, decisions could be based on existing sources of information
about candidates, newly gathered information, or some combination of the two. Finally, the
source of the information (e.g., self vs. other) or the type tool used to collect the data (e.g.,
objective tests vs. performance ratings) can differ.

“Candidates were interviewed by team members and completed an inventory that

assessed their fit with the team.” - Head of Organizational Effectiveness, Boeing

Third, the temporal dynamics associated with the process can also influence the team staffing
decision. For example, when faced with multiple openings, some team leaders focus on filling
targeted positions before filling other positions, while others work on filling all positions at one
time. A second temporal issue is whether a multiple hurdle evaluation approach is used (e.g.,
candidates must demonstrate a minimum level of technical skill before they can move onto an
interview process), or all factors are considered in a single hurdle.

“Three strong candidates met my initial hurdle (leadership style) and were then
interviewed.” –Manager, Organizational Capability, BP

Factors that define the candidate pool. When making team staffing decisions, the candidate pool
can act as a limitation or a facilitating feature, depending on the status of the factors that define
the candidate pool. Accordingly, the taxonomy also captures factors that define the candidate
pool. In some instances, the decision maker can control aspects of the candidate pool. For

14
example, someone might choose to select from the external market rather than the internal pool.
Or, they might choose to recruit world-wide when looking to build a global team. However,
other candidate pool characteristics are not as easily controlled. For example, in a tight labor
market, the list of candidates could be smaller than desired.

“I often look for people internally, because they can come in and back people up in other
positions sooner than someone from outside.” - Director of Workforce Development,
SEFCU

Constraints. Team commanders and leaders frequently are faced with obstacles or constraints
when making team composition decisions. Despite these constraints, they must still be able to
make effective team staffing decisions. Given this, an effective team composition optimization
algorithm must account for the constraints placed on the decision maker. Therefore, the
taxonomy captures many of the common constraints placed on team staffing decisions. Although
it is expected TOPS users will ultimately be able to customize the system to incorporate unique
situational constraints, identifying common types of constraints enables us to develop a “usable”
TOPS system. Some common constraints placed on team staffing decisions include:



Candidate availability (e.g., candidates unavailable/busy/not local)


Missing information (e.g., about the candidate/team; about the task/mission)


Costs (e.g., recruiting, candidate compensation requirements)


Lack of time (e.g., deadlines for putting a team in place)
Timing of the decision (e.g., right person, wrong time)

“The biggest constraint for me is availability of personnel.” – Major, US Marine Corps

“Cost considerations were a major factor. They wanted a team with global experience.
However, it is expensive to move candidates to New Jersey from other parts of the
world.” – Senior Director, Merck

In sum, the team staffing taxonomy, based on the team composition literature and our interviews
with team staffing subject matter experts, highlights features that should be captured when
developing a team composition optimization tool. As such, these factors influenced the
components of the initial TOPS algorithm and framework (described below), and should
continue to contribute to future evolutions of the TOPS system.

Task 3: Build a Generic TOPS Composition Optimization Algorithm

The first and second tasks identified: 1) types of situations in which managers/officers confront
decisions about team composition; 2) a better understanding of how managers currently deal
with team composition situations; 3) types of logistical constraints they face that limit their
options; and 4) the need to make such decisions in a better way. This resulted in the development
of a team composition taxonomy and highlighted features that should be incorporated in TOPS.

The third task leveraged information from Tasks 1 and 2 to build a generic team composition
optimization algorithm. Specifically, this algorithm simultaneously models individuals’

15
knowledge, skills, abilities, and other characteristics (KSAOs) as related to both their job or role
performances and the accomplishment of joint team tasks. This resulted in a methodology for
determining “ideal mixes” or “profiles” of team composition. Moreover, the TOPS algorithm
offers an index of team composition which overcomes the weaknesses of the traditional HR,
diversity, central tendency, and selected score approaches previously reviewed. Specifically, the
algorithm’s profiling approach models team competencies as a joint function of: 1) members’
competencies related to performing their individual jobs; 2) the interdependence or relationships
between members’ jobs; 3) members’ competencies related to performing team tasks; and 4) the
interrelationships among members relevant to performing those team tasks. In other words, team
performance is a joint function of individual members’ job performances and their combined
efforts on team tasks. This approach represents a more flexible method to index and study the
influences of members’ KSAOs to the collective competencies of teams.

TOPS Framework. A conceptual framework for a full scale TOPS system was also developed. In
this framework, the functional characteristics of this team decision making system, including the
algorithm, are specified. The generic TOPs framework consists of several interlinking modules,
which together with the algorithm, will provide a user-friendly software application that
accomplishes several objectives, as depicted in Figure 7.

Candidate
Data

Position Module Linkage Module Team Module

Decision Module Team

Algorithm
Composition

Figure 7: Interaction among Modules in TOPS

Foundational Modules. The four TOPS modules are referred to as “foundational”

modules (Position, Team, and Candidate Data, and Linkage Modules) because they must be pre-
populated in order to produce team composition recommendations. For example, information
about the position and team requirements as well as the team members is stored in these
modules.

16
 Decision and Team Composition Results Modules. In addition to the foundational
modules, the decision maker (e.g., team leader, commander) will work with the TOPS Decision
Module to confirm or modify underlying assumptions and establish certain boundary conditions
for their current staffing decision. The algorithm, which drives the TOPS system, incorporates
the assumptions and constraints introduced in the foundational and decision modules to index
and examine team composition. Finally, the team composition output (e.g., proposed team
composition solutions) is presented for the user in the Team Composition Results Module.

To arrive at a fully functional TOPS that can be employed within a specific domain, the
foundation modules need to be customized and populated by the decision maker in order to
ground the generic TOPS algorithm and system in a given community.

TOPS customization process. The customization process is what makes TOPS scalable across
situations and applications. This step, carried out by the decision maker, provides a number of
valuable pieces of information. First, it specifies the information that is captured in the Position,
Team, and Linkage Modules. For both the Position and Team Modules, the customization process
identifies the “attributes” that will be considered. Moreover, it specifies which of these attributes
are prerequisites associated with minimum requirements, and which of these are attributes to be
considered in the algorithm. Each attribute that will be considered in the algorithm is assigned a
weight that represents its relative importance for performance in the position or team. These
weights come from a weighting procedure that can be conducted through various means (e.g.,
team tasks analyses, observations, using judgments from SMEs). For the Linkage Module, the
customization process derives interdependency weights, which reflect the relative contribution or
influence that a particular position has on the performance of other positions on the team.

TOPS population process. A population process is also completed by the decision maker. This
process could be completed in concert with, or subsequent to, the customization process. During
the population process, the decision maker enters candidate and team member data (e.g., attribute
scores) into the Candidate Data Module.

In sum, the customization and population process is an “upfront” decision maker activity that
grounds TOPS in a target population. It specifies and populates the foundational modules in
TOPS (i.e., Position, Team, Candidate Data, and Linkage Modules). While existing position and
team task analyses can be leveraged for this effort, a general customization process will be
developed that could be followed even if no existing background information is available for a
target population.

TOPS algorithm. The information stored in the foundational modules is extracted and
manipulated by the TOPS algorithm to ultimately generate team composition solutions. It is
helpful to envision the TOPS algorithm’s operation in a “fully unconstrained environment.”
Theoretically, the algorithm will generate every conceivable combination of members in
positions within teams. Whereas this would never be the case in real life, this theoretical baseline
is important for understanding the system. The theoretical potential number of solutions is a
finite number determined by: 1) the number of candidates in the pool; 2) the number of team
positions; 3) and the number of teams under consideration. This assumes that anyone can occupy
any position in any team, at any time, and members can be arranged in any combination. Again,

17
this is merely a fully unconstrained theoretical baseline. It is general in the sense that it can
accommodate any number of candidates, positions, teams, and elements to be considered at the
individual and team levels, as related to any sort of arrangement. What TOPS then does is
successively overlay “constraints” that preclude certain combinations (for various reasons noted
below). These constraints are introduced by the decision maker in the Position, Team, and
Decision Modules. The position prerequisites stored in the Position Module work as constraints
that eliminate some of the solutions from the fully unconstrained possibilities, because a
potential team member may fail to possess a prerequisite for the position to which he or she is
assigned.

Further constraints are introduced in the Team Module. The Team Module features team
summary indices and relative scores at the team-level of analysis as derived from individual team
members’ attributes. The module includes aggregate prerequisites that are not associated with
any specific position (e.g., the team must have a representative from each of “X” countries or
functional units, or two or more members must possess fluency in a particular language). In other
words, of the viable solutions that surpass the position prerequisites, some will be eliminated
because they fail to surpass the team level prerequisites. Therefore, from the theoretical baseline,
a limited number of potential solutions will surpass the constraints that are imposed at the
position and team levels.

After the candidate data are compared against the position and team prerequisites to determine
all feasible member combinations (i.e., solutions), the algorithm calculates relative fit indices, or
a Predicted Effectiveness Index (PEI), based on position- and team-level attributes. A separate
Team PEI is calculated based on the team attributes. Each of these indices represents a weighted
composite of the relevant attributes. The TOPS algorithm combines weighted Position PEI and
Team PEI scores to yield an overall TOPS PEI for each potential team composition solution.
This PEI, then, represents the predicted effectiveness of each combination of members and is
presented to decision makers in a Team Composition Solution Module.

Notably, after the TOPS system is customized and populated for a particular domain and
decision type, the decision maker (e.g., team leader, commander) will work with the TOPS
Decision Module first to confirm or modify underlying assumptions and establish certain
boundary conditions for their current staffing decision. For example, the decision maker might
specify that certain candidates should not be placed on a team together. The boundary conditions
and assumptions introduced by the decision maker act as constraints over the fully unconstrained
potential solutions. The least constrained Decision Module would be a full-scale organization
reconfiguration solution (see the Team Staffing Taxonomy) where all candidates could be
potential members of any team. Therefore, few combinations would be precluded from the onset.
On the other extreme, the most highly constrained situation is the case of looking to replace a
single position in a single team. In this instance, the only viable solutions from the fully
unconstrained environment that are maintained for consideration, are those that depict the
remaining members of the specific given team in their respective positions. In short, this reduces
to a traditional selection decision, but now with the additional weightings and considerations that
are depicted in the team and linkage modules.

18
In practice, the generic fully unconstrained TOPS algorithm will be built in and function as the
underlying engine to the TOPS system. For a given team composition decision application, a set
of constraints is introduced to the system by the decision maker via the position and team
modules. The algorithm compares the candidate data against the position and team prerequisites
(specified in their respective modules) to determine all potential member combinations (i.e.,
solutions) for the decision that the user specified. Each of these potential solutions is then
weighted by the parameters of the position, team, and linking modules to yield the TOPS PEI for
each combination. These TOPS PEI values, which represent the primary team composition
solution results, are presented for the user in the Team Composition Solution Module. This
module will include additional features and summary information, such as the ability to rank
order composition solutions, bracket comparable solutions, and drill down to highlight the details
of each solution.

The Team Composition Solution Module will work in conjunction with the Decision Module
such that users could return to the Decision Module and change their specified conditions and
constraints to explore alternative scenarios (e.g., Jim should not be on the same team as Jeff,
locking Sue into a specific position). For example, it is possible the initially specified constraints
eliminated all potential solutions. It is also possible the decision maker could receive new
information about one or more candidates that was not initially considered by the system (e.g.,
candidates suggested as part of the optimal team within the Team Composition Solution Module
are not available). Or, the decision maker may want to examine the PEI of a team that did not
include candidates or combinations of candidates listed in the initially generated team solutions.
Accordingly, the decision maker will be able to examine and modify any initial assumptions or
conditions in the Decision Module and produce new TOPS PEI results. By relaxing or adding
such constraints and re-executing TOPS, the decision maker can explore alternative solutions to
the team composition decision.

TOPS Demonstration. Following the development of the TOPS conceptual framework and
algorithm, we developed a simple TOPS demonstration for making a single member replacement
decision within a specific domain. Specifically, the demonstration, based on fictitious data,
depicted how a decision maker within the Special Forces community could use TOPS to guide
him in filling a vacant Commander position on an existing Special Forces team. This illustrated
how a decision maker could use TOPS within a specific team staffing scenario as well as the
technical feasibility of the TOPS concept for a realistic staffing decision. The TOPS
demonstration also showed how the candidate with the best fit for the individual position (i.e.,
the candidate with the highest Position PEI), who would be selected for the team under a
traditional HR selection approach, might not result in the most effective team (as depicted by the
TOPS PEI). This illustrates the algorithm’s ability to help a decision maker balance the
competing demands of the individual position and the team, thereby offering the greatest
combined value of the replacement.

In sum, the TOPS algorithm and framework are generic in that they will be applicable for aiding
a wide variety of staffing decisions from identifying a single team member’s replacement, to
forming multiple new teams. Although the algorithm and system framework are generic, the
specific attributes, weights, and linking function are customizable to fit a specific application.
The scalability of these elements also allows users within complex and dynamic environments,

19
such as those within the military, to easily adjust the system to reflect fluctuations in the team
composition situation (e.g., changes in candidate pools, team members, pre-requisites, situational
changes over time). In operation, each TOPS application will consist of a number of
interdependent modules, which together provide a user-friendly software application that
accomplishes several objectives as depicted in Figure 5. Four of these modules must be pre-
populated with foundational information before the system is ready for use by the decision
maker. This customized information will then guide the application of the underlying elements of
the algorithm as the decision maker interacts with TOPS to examine potential team staffing
(composition) options.

The Future of TOPS: TOPS Lite

The aforementioned TOPS framework represents a “full scale” version of TOPS that features
customization to the specific team composition user domain. While this full scale version of
TOPS accounts for a great deal of the decision space in optimizing team composition, it requires
the user to access and input a substantial amount of data specific to the composition decision and
candidates. In the future, it is possible that a scaled down version of TOPS (“TOPS Lite”) will be
developed. TOPS Lite would require less user data and customization to set up and run. It would
meet usability requirements of many organizations that do not have access to or the resources to
generate the domain or candidate specific data that would be required to customize the full
version of TOPS. TOPS Lite would still include a small set of required data fields (e.g., number
of roles on the team, number of people needed in each role) to run. However, many other data
fields (e.g., attribute weights) would be optional for the user. While TOPS Light might include
the same modules as a full TOPS system, it would have more boundaries than a full TOPS. For
example, there might be limits on the number of roles or candidates and it might be a system that
is programmed to solve only one type of team composition decision (e.g., single team
formation). These boundaries would enhance the efficiency of the TOPS algorithm and system.

In practice, a leader (or some other decision maker) faced with a specific team composition
scenario (e.g., “I need to form a new team composed of 9 people from a pool of 45 candidates)
would open TOPS Lite and arrive at a main page. The main page would hold a short list of menu
options (Describe Team Roles, Confirm/Modify Requirements, List Potential Team
Members/Candidates, Enter Candidate Information, Run Analysis/Review Results) which
correspond with many of the modules described previously. The leader would use this menu to
navigate through the various steps of customizing and populating the system. At each step, the
leader would enter the required information and decide whether to enter any “optional”
information. Once the leader has entered all of the required information and any optional
information relevant to the decision requirements, team, and candidates, the leader will run the
analysis and review the results. It is expected after all data are entered and assumptions are set
(or changed), it should take about 60 seconds or less to run the analysis.

The results would present one or more team composition solutions, which would represent the
most optimal teams in terms of team effectiveness. If the leader is not satisfied with the
suggested solution, the leader would have the option to adjust the requirements or assumptions
he entered into the system initially. For example, the leader could lock certain individuals into
specific roles and/or reduce the number of people who need a specific qualification. After

20
making these adjustments the leader could then re-run the program to generate a new set of team
composition solutions. The system would also incorporate embedded tips and help that would
provide the user with instructions, tips, and examples.

In sum, relative to the full version of TOPS, TOPS Lite would be a more manageable tool with
greater likelihood of use. It would work much like a full TOPS system, but would be
programmed for working a specific type of team composition decision and would require less
data from the user. Whether the full TOPS system or TOPS Lite would be best for a given
organization would depend on its team composition needs (e.g., Does it need to make multiple
types of team composition decisions or does it primarily make one type of decision?) and its
willingness and ability to gather and input team and candidate data.

Anticipated Applications and Benefits for Government and/or Private Sector

The interviews with leaders in the private and government sectors indicated that organizations
increasingly rely both on permanent teams with regularly changing personnel, as well as
temporary teams that must be staffed quickly to meet mission objectives. Virtually all of the 17
organizations interviewed face team staffing challenges similar to those that would be addressed
by the TOPS system. Leaders in these organizations need to juggle individual competencies and
team needs, and make quick, effective team staffing decisions. They often do this based on
instinct or by selecting the person they think has the best individual competencies to fill a
position, with little or no consideration of team needs, interdependencies, or dynamics. While
some leaders appear to staff their teams effectively, many struggle with this task. Hence, there is
clearly a need for tools or decision aids to help leaders make informed and effective team
staffing decisions.

Although the team composition literature has recently begun to explore the characteristics team
staffing decision makers consider, and has discussed potential methods for indexing team
composition, a gap between the team composition theory and an understanding of real team
composition scenarios has limited the application of team composition optimization strategies in
organizations. An important contribution of the team staffing SME interviews and taxonomic
work is that it began to bridge the gap between team composition optimization theory and the
current state of team staffing practices and needs. For example, the team staffing interviews
revealed the team composition decision models and strategies used by those who are regarded as
experts in making team staffing decisions. The majority of the team staffing experts interviewed
identified factors they consider when making team staffing decisions and processes or techniques
they employ to help them balance individual position requirements with the needs of the team.
The information gathered during these interviews grounded the team composition decision
taxonomy, which highlights elements leaders should consider when making team staffing
decisions. Therefore, an immediate benefit of this taxonomy is that it can guide leaders as they
make decisions that impact the composition of their teams. As a more long-term benefit,
researchers can draw from the taxonomy to ensure the future team composition decision making
algorithms and tools (those developed for TOPS or other team composition applications) are
grounded in actual team staffing needs and experiences.

21
This work also resulted in a generic team composition algorithm designed to assist commanders
in filling a vacant position on an intact team. The design of this algorithm advances the
application of team composition research in that it represents a more flexible method to index
and study the influences of members’ KSAOs to the collective competencies of teams. The
particular strengths of this approach are: 1) a recognition that team performance is a joint
function of members’ individual job performances and their contributions to combined team
activities; 2) incorporation of the relative interdependencies of members’ individual job
performances in a network fashion; and 3) a differentiation of members’ job versus team related
KSAOs. Although the structure of this algorithm is generic, it is applicable for aiding a wide
variety of staffing decisions from identifying an individual team member’s replacement (as
shown in the prototype illustration), to optimizing large scale force deployment. Specifically, the
specific attributes, weights, and linking function would be customizable to fit a specific
application. Therefore, the algorithm has potential to provide valuable insights if it is customized
for particular domains and situations.

A final contribution of this work is the TOPS system framework and user methodology.
Outlining how a team optimization system, including the information captured within the
interlinking TOPS modules, might work in practice is beneficial in that it articulates how a team
optimization algorithm could be applied to help commanders work through real team staffing
scenarios. With an enhanced understanding of how a team composition algorithm can be applied,
the stage has been set for researchers to move onto: 1) refining and validating the elements of
algorithms that model relationships of team composition with team effectiveness, and 2) building
prototype team optimization decision support tools.

22
 References

Arrow, H. & McGrath, J. E. (1995). Membership dynamics in groups at work - A theoretical

framework. Research in Organizational Behavior: An Annual Series of Analytical Essays
and Critical Reviews, 17, 373-411.

Barrick, M. R., Stewart, G. L., Neubert, M. J., & Mount, M. K. (1998). Relating member ability
and personality to work-team processes and team effectiveness. Journal of Applied
Psychology, 83, 377-391.

Bell, S. T. (2007). Deep level composition variables as predictors of team performance: A meta-
analysis. Journal of Applied Psychology, 92, 395-615.

Cannon-Bowers, J.A., Tannenbaum, S.I., Salas, E., & Volpe, C.E. (1995). Defining team
competencies: Implications for training requirements and strategies. In R. Guzzo & E. Salas
(Eds.), Team Effectiveness and Decision Making in Organizations (pp. 333-380). San
Francisco: Jossey-Bass .

Devine, D. J. & Philips, J. L. (2001). Do smarter teams do better - A meta-analysis of cognitive

ability and team performance. Small Group Research, 32(5), 507-532.

Ellis, A. P. J., Bell, B. S., Ployhart, R. E., Hollenbeck, J. R. & Ilgen, D. R. (2005). An evaluation
of generic teamwork skills training with action teams: Effects on cognitive and skill-based
outcomes. Personnel Psychology, 58, 641-672.

Gladstein, D. (1984). Groups in context: A model of task group effectiveness. Administrative

Science Quarterly, 29, 499-517.

Gully, S. M. (2000). Work teams research: Recent findings and future trends. In M. M. Beyerlein
(Ed.), Work Teams: Past, Present and Future (pp. 25-44). Netherlands: Kluwer Academic
Publishers.

Halfhill, T., Nielsen, T. M., Sundstrom, E., & Weilbaecher, A. (2005). Group personality
composition and performance in military service teams. Military Psychology, 17(1), 41-54.

Heslin, R. (1964). Predicting group task effectiveness from member characteristics.

Psychological Bulletin, 62, 248-256.

Hirschfeld, R. R., Jordan, M. H., Feild, H. S., Giles, W. F. & Armenakis, A. A. (2006).
Becoming team players: Team members’ mastery of teamwork knowledge as a predictor of
team task proficiency and observed teamwork effectiveness. Journal of Applied Psychology,
91, 467-474.

Ilgen, D. R. (1999). Teams embedded in organizations - Some implications. American

Psychologist, 54(2), 129-139.

Ilgen, D. R., Hollenbeck, J. R., Johnson, M., & Jundt, D. (2005). Teams in organizations: From
I-P-O models to IMOI models. Annual Review of Psychology, 56, 517-543.

23
Jackson, S. E. (1992). Team composition in organizational settings: Issues in managing an
increasingly diverse workforce. In S. Worchel & W. Wood & J. Simpson (Eds.), Group
Process and Productivity. Newbury Park, CA: Sage.

Jackson, S. E., Brett, J. F., Sessa, V. I., Cooper, D. M., Julin, J. A., & Peyronnin, K. (1991).
Some differences make a difference: Interpersonal dissimilarity and group heterogeneity as
correlates of recruitment, promotion, and turnover. Journal of Applied Psychology, 76, 675-
689.

Kozlowski, S. W. J. & Bell, B. S. (2003). Work groups and teams in organizations. In W. C.

Borman, D. R. Ilgen, & R. J. Klimoski (Eds.), Handbook of Psychology: Industrial and
Organizational Psychology (Vol. 12, pp. 333-375). London: Wiley.

LePine, J. A., Hollenbeck, J. R., Ilgen, K. R., & Hedlund, J. (1997). Effects of individual
differences on the performance of hierarchical decision-making teams: Much more than g.
Journal of Applied Psychology, 82, 803-811.

Mann, R. D. (1959). A review of the relationship between personality and performance in small
groups. Psychological Bulletin, 56(4), 241-270.

Marks, M. A., Mathieu, J. E., & Zaccaro, S. J. (2001). A temporally based framework and
taxonomy of team processes. Academy of Management Review, 26(3), 356-376.

Mathieu, JE, Gilson, LL & Ruddy, TM (2006). Empowerment and team effectiveness: An
empirical test of an integrated model. Journal of Applied Psychology, 91, 97-108.

McCormick, E. J. (1979). Job Analysis: Methods and Applications. New York: ANACOM.

Mitchell, T. R. & James, L. R. (2001). Building better theory: Time and the specification of
when things happen. Academy of Management Review, 26(4), 530-547.

Morgan, B. B. & Lassiter, D. L. (1992). Team composition and staffing. In R. Sweezy and E.
Salas (Ed.), Teams: Their Training and Performance (pp. 75-100). Norwood, NJ: Ablex.

Salas, E., Bowers, C. A., & Cannon-Bowers. (1995). Military team research: 10 Years of
progress. Military Psychology, 7, 55-76.

Steiner, I. D. (1972). Group Processes and Productivity. New York: Academic Press.

Stevens, M. J. & Campion, M. A. (1994). The knowledge, skill, and ability requirements for
teamwork: Implications for human resource management. Journal of Management, 20, 503-
530.

Tannenbaum & Alliger (2000). Knowledge Management: Clarifying the Key Issues. Rector-
Duncan & Associates: Austin, TX.

24
 Appendix A
Team Staffing Expert Interview Protocol

Interview and Demographic Information

Interviewer Name: ______________________________ Date:
_______________________________
SME Name: __________________________________ Organization:
_________________________________
Position/Title: _____________________________ Years in Position: ________ Years
staffing teams: ________
Type of team(s) staffed (pull from answer to question #1):
______________________________________________

______________________________________________________________________________
Opening Comments

 Introduce yourself

 I am talking with you today because you were recommended as a team staffing expert.
As part of a research and development effort supported by the US Army Research
Institute, The Group for Organizational Effectiveness, Inc. is conducting interviews with
team staffing experts. The purpose of the interviews is to gather insights from experts in
team staffing. In other words, we would like to uncover what you, as a team staffing
expert, think about and the decision processes you go through when you staff a team.

 During the next 30 minutes or so, I will ask you questions about your own experiences
staffing teams. For example, we’ll focus on:

o The types of decisions you make

o Previous experiences you’ve had in staffing teams
o Some of your reasoning in staffing teams the way you do


.
I will be taking notes throughout the interview. It would be helpful if we were able to use
some of your comments or quotes in our report. If at any time, you mention something
you’d rather not be attributed to you, just let me know and I’ll make note that it is “off the
record.” Our summarized results will be reported to the Army Research Institute, and will

 Before we begin, do you have any questions?

be used to influence future team composition research and development.

______________________________________________________________________________

Demographic Verification

I’d like to begin by collecting some basic background information from you. [Go through items
in the Interview and Demographic Information section, verify any information you already have,
and fill in any blanks].

A-1
Definition of Team Staffing

I’d like to begin by clarifying what we mean by “team staffing.” Team staffing involves placing
one or more people into a team of individuals who perform interdependent tasks. As you know, it
is a decision that influences the composition of the team and potential team performance.

Interview Questions

1. Teams can be categorized into different types, based mainly on the kind of work they
perform. I’d like to get a general idea about the types of teams you typically staff. It seems
like you normally staff teams that ______________ [describe a team type using the
Sundstrom descriptions]. Is this correct? If not, what kinds of teams do you usually staff?
[May need to describe other types]

2. What type(s) of staffing decisions have you had to make in your career? [Use the probes

 Filling one position on a team?

below if needed]

 Staffing a team from scratch?

 Picking individuals for a team from a pool of individuals? Making subgroups from a

 No flexibility with who is placed on a team, but some control over role/task
larger pool?

assignment?

3. Think about a particular situation where you needed to make a decision that influenced a
team’s composition (e.g., you needed to fill in a single position, multiple positions). [cycle

 Please describe the situation.

through two situations if possible...try to get two different kinds of decisions, if possible]

 What process did you use in making your decision(s)? Did you make the decision
yourself or were others involved (e.g., current team members)? Is this how you

 What factors influenced, or were important to, your decision? What were the three
typically make staffing decisions?

most influential factors you considered? Could you rank order these factors according

 What information about the individual(s) (e.g., performance info, career preferences,
to importance?

location), if any, did you consider? Does your organization track or otherwise make

 Did you consider existing characteristics of the team (e.g., composition/capabilities of

such information available to you?

current team, ability to cover/back-up, function/demographic diversity) when making

 Looking back, what information do you wish you had that was unavailable at the
such decisions?

 Were there any obstacles or challenges (within or outside of your control) in making
time?

your decision(s)? If so, what were they? How did you overcome them?

A-2
  Do you follow-up on your staffing decisions? Formally? Informally?
4. How do you know when you’ve put together a good team? What are the signs?

5. Have you ever had an experience where:

A. Each team member was good at their individual job but the team as a whole struggled?
Why do you think the team struggled?

B. More than one candidate was highly qualified for a single position on a team? What was
the deciding factor in choosing a particular candidate?

C. You decided not to pursue or choose a person who might have had the best individual
qualifications for an opening on the team? If so, why? (e.g., wanted to provide the person
with a developmental opportunity; highly qualified for position, but not a team player;
neurotic; diversity needs; best qualified person can’t work with someone else on the
team)

6. If you had a magic wand (e.g., limitless resources/information, no constraints) and could
form a team from scratch, what three specific factors would you focus on to guide you in
your decision? [get them to be specific – so if they say “individual information” – ask what
individual information]

Conclusion

Thank you for your participation in this interview. Would you like us to send you a copy of the
summarized interview results that will be developed at the conclusion of all interviews? If so,
where should we send this report?

Contact information: _________________________________________________________

Thank you for participating in this interview!

A-3