STRATA: Unified Framework For Task Assignments in Large Teams of Heterogeneous Agents

Autonomous Agents and Multi-Agent Systems (2020) 34:38
https://doi.org/10.1007/s10458-020-09461-y
STRATA: unified framework for task assignments in large

teams of heterogeneous agents
Harish Ravichandar1 · Kenneth Shaw1 · Sonia Chernova1
Published online: 13 May 2020

© Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract
Large teams of heterogeneous agents have the potential to solve complex multi-task prob-
lems that are intractable for a single agent working independently. However, solving com-
plex multi-task problems requires leveraging the relative strengths of the different kinds
of agents in the team. We present Stochastic TRAit-based Task Assignment (STRATA), a
unified framework that models large teams of heterogeneous agents and performs effective
task assignments. Specifically, given information on which traits (capabilities) are required
for various tasks, STRATA computes the assignments of agents to tasks such that the trait
requirements are achieved. Inspired by prior work in robot swarms and biodiversity, we
categorize agents into different species (groups) based on their traits. We model each trait
as a continuous variable and differentiate between traits that can and cannot be aggregated
from different agents. STRATA is capable of reasoning about both species-level and agent-
level variability in traits. Further, we define measures of diversity for any given team based
on the team’s continuous-space trait model. We illustrate the necessity and effectiveness of
STRATA using detailed experiments based in simulation and in a capture-the-flag game
environment.
Keywords Multi-agent systems · Task assignment · Heterogeneous agents
1 Introduction
The study of multi-agent systems has produced significant insights into the process of engi-
neering collaborative behavior in groups of agents [6, 25]. These insights have resulted
in large teams of agents capable of accomplishing complex tasks that are intractable for a
single agent, with applications including environmental monitoring [31], agriculture [32],
warehouse automation [36], construction [35], defense [4], and targeted drug delivery [19].
* Harish Ravichandar
[email protected]
Kenneth Shaw
[email protected]
Sonia Chernova
[email protected]
1
Georgia Institute of Technology, Atlanta, GA, USA
13
Vol.:(0123456789)
38 Page 2 of 25 Autonomous Agents and Multi-Agent Systems (2020) 34:38
Efficient solutions to the above problems typically rely on a wide range of capabilities.
Teams of heterogeneous agents are particularly well suited for performing complex tasks
that require a variety of skills, since they can leverage the relative advantages of the differ-
ent agents and their capabilities. In this work, we are motivated by robotics applications,
and the multi-robot task assignment (MRTA) problem in particular [10, 17, 18] which for-
mally defines the challenges involved in optimally assigning agents to tasks.
We present Stochastic TRAit-based Task Assignment (STRATA), a unified mod-
eling and task assignment framework, to solve an instance of the MRTA problem with
an emphasis on large heterogeneous teams. We model the topology of tasks as a strongly
connected graph, with each node representing a task and or a physical location and the
edges indicating the possibility of switching between any two tasks. We assume that the
optimal agent-to-task associations are unknown and that the task requirements are speci-
fied in terms of the various traits (capabilities) required for each task. Thus, in order to
effectively perform the tasks, the agents must reason about their combined capabilities and
the limited resources of the team. To enable this reasoning, we take inspiration from prior
work in robot swarms [28] and biodiversity [27], and propose a group modeling approach
[1] to model the capabilities of the team. Specifically, we assume that each agent in the
team belongs to a particular species.1 Further, each species is defined based on the traits
possessed by its members. Assuming that the agents are initially sub-optimally assigned to
tasks on the task graph, STRATA computes assignments such that the agents can reorgan-
ize themselves to collectively aggregate the traits necessary to meet the task requirements
as quickly as possible.
In Fig. 1, we illustrate the basic building blocks of STRATA and the task assignment
problem. As seen in the top row, STRATA models the effects of task assignments (task-
species distribution X) and the species’ traits (species-trait model Q) on how the traits are
aggregated for each task (task-trait distribution Y). The example species-trait model in
Fig. 1 illustrates the distribution of two traits (Trait 1 and Trait 2) in four different species
(S1, S2, S3, and S4). Colored bar graphs near each node of the graph are used to illus-
trate either the task-trait (Y) or the task-species (X) distributions. As can be seen, for any
distribution of agents across the tasks, we can compute the corresponding trait distribu-
tion across the tasks. We also derive a closed-form expression to quantify the effect of the
variability in the agents’ traits on the achieved task-trait distribution. The task assignment
problem, as seen in the bottom row, involves computing the optimized task assignments,
given a desired distribution of traits across the tasks and a team of heterogeneous agents
described by a species-trait distribution.
Using the above model, STRATA allows for the optimization of two separate task
assignment goals: (1) exact matching and (2) minimum matching. In exact matching, the
algorithm aims to distribute the agents such that the achieved trait distribution is as close to
the desired as possible. In minimum matching, the algorithm aims to distribute the agents
such that the achieved trait distribution is higher than or equal to the desired as possible,
i.e., over-provisioning is not penalized.
The STRATA representation of both task and species traits is inspired by [28], which
considered binary instantiations of traits. However, binary models fail to capture the
nuances in the scales and natural variations of the agents’ traits. For instance, consider
an unmanned aerial vehicle and a bipedal robot. While both agents share the mobility
1
Similar to prior work in multi-robot systems [28], we use the term “species” to describe a group of agents
with similar traits. This does not imply any similarity to biological species.
13
Autonomous Agents and Multi-Agent Systems (2020) 34:38 Page 3 of 25 38
Fig. 1 Top row: STRATA defines the effects of task-species distribution and the species-trait model on the
task-trait distribution. Bottom row: Given a team defined by the species-trait model, we aim to perform task
assignments such that the desired task-trait distribution is achieved
trait (the ability to move), their speeds are likely to be considerably different. To address
these challenges, in STRATA we have extended the representation to model traits in the
continuous space. Additionally, STRATA also captures agent-level differences within
each species by using a stochastic trait model.
When reasoning about the collective strengths of the team, attention must be paid
to the fact that not all capabilities are improved in quantity by aggregation of individ-
ual agents’ abilities. For instance, a coalition of any number of slow robots does not
compensate for a faster robot. Taking this observation into account, we consider two
types of traits: cumulative and non-cumulative. We consider a trait to be (non) cumula-
tive if it can (not) be aggregated from different agents in order to achieve certain task
requirements.
In this work, we also extend the diversity measures introduced in [28] to the continu-
ous space. We derive two separate diversity measures, one for each goal function. The
diversity measures provide insights about the trait-based heterogeneity of the team. Spe-
cifically, the diversity measures help define a a minimum subset of the species that can
collectively compensate for the rest of the team.
In summary, the key contributions of our work include a unified framework for effec-
tive task assignment of large heterogeneous teams that:
1. incorporates a stochastic trait model that captures both between-species and within-
species variations,
2. assigns tasks to agents with respect to two separate goals: exact matching and minimum
matching, and
3. computes measures of diversity in teams with continuous trait models.
13
We evaluate STRATA using detailed simulations and a capture-the-flag game environment.

Our results demonstrate the necessity and effectiveness of STRATA in terms of effective
task assignment and improved team performance, when compared to a baseline that only
considers binary traits.
Finally, we note that heterogeneous teams can be composed of both robotic and human
agents. Human traits can be vastly different from and complementary to those of robots [8].
For instance, when compared to humans, robots can carry heavier payloads, move faster,
and be immune to fatigue. On the other hand, humans’ abilities to assimilate and maintain
situational awareness, process noisy information, and adapt to highly unstructured envi-
ronments are unmatched by the abilities of their robotic counterparts. Further, individual
differences are considerable in teams involving humans (see [8] and references therein).
Although not yet evaluated with human-robot teams, STRATA’s ability to characterize
humans as one or more separate species (e.g., soldiers, pilots, medics) possessing stochas-
tic traits makes it a promising representation for modeling human-robot teaming.
2 Related work
Significant efforts have been focused on problems in multi-robot systems task assignment
(MRTA) [10, 17, 18]. Broadly, the problems are categorized based on three binary charac-
teristics: (1) Task type (single-robot [SR] vs multi-robot [MR]), (2) robot type (single-task
[ST] vs multi-task [MT], and (3) assignment type (instantaneous [IA] vs time-extended
[TA]) [10]. While task type indicates the number of robots required to complete each task,
robot type indicates whether the robots are capable of simultaneously performing a sin-
gle task or multiple tasks. The assignment type is used to differentiate between tasks that
involve scheduling constraints and those that do not. Indeed, numerous approaches related
to the different variants of MRTA, including assignments involving single-robot tasks,
are available in the literature. However, we limit our coverage of related work to variants
of MRTA that involve multi-robot tasks - tasks that involve the coordination of several
robots. We refer readers to [10, 18] for comprehensive categorizations and examples of all
approaches pertaining to task assignment.
Several methods for task assignment with homogeneous agents have been proposed. A
graph-theoretic framework, named SCRAM, is proposed in [22]. SCRAM maps agents to
target locations while simultaneously avoiding collisions and minimizing the time required
to reach target locations. The work in [21] presents a hierarchical algorithm that is cor-
rect, complete, and optimal for simultaneously task assignment and path finding. A fast
bounded-suboptimal algorithm, that automatically generates highways for a team of homo-
geneous agents to reach their target locations, is introduced in [7]. Notably, the methods in
[7, 21, 22] emphasize optimal path finding for each robot and collision avoidance in order
to assign each robot to a single task (reaching a target location). However, these methods
assume that all the agents in the team are interchangeable, and thus are not suitable for
multi-task scenarios that involve several heterogeneous agents.
Approaches involving single-task robots solve the assignment problem by assuming that
each robot is specialized and can only perform one task. The method proposed in [29]
addresses a transportation task involving multiple single-task robots. Some of the items to
be transported can be transported by a single robot, while others need coordinated efforts
from several robots. [29] use a greedy set-partitioning algorithm to form coalitions of
robots required to perform the tasks. Potential coalitions are iteratively computed for each
13
task involved. The coalition formation algorithm introduced in [29] was later extended in
[34]. The extended algorithm in [34] reduces the communication effort, balances the coali-
tions, and constrains the requirements to specify if and when all the required traits must be
possessed by a single robot. These approaches, however, require the listing of all potential
coalitions and thus are not suitable for problems involving large number of possible coali-
tions. Indeed, the number of possible coalitions is a factor of both the number of robots in
the team and the inherent diversity of the team. Specifically, as the number of robots in the
team and their similarities increase, so does the number of possible coalitions. STRATA,
on the other hand, is scalable with the number of agents as it does not list all possible
coalitions.
Decentralized approaches for task assignment are introduced in [5, 13, 14, 23]. A
game-theoretic task assignment strategy is introduced in [14] to assign tasks to a team of
homogeneous robots with social inhibition. In [5], multiple tasks are assigned to a team
of homogeneous robots. The authors develop of a continuous abstraction of the team by
modeling population fractions and defining the task allocation problem as the selection
of rates of robot switching from and to each task. In [13], the authors extend the method
in [5] with a wireless communication-free quorum sensing mechanism in order to reduce
task assignment time. In [23], a decentralized approach for heterogeneous robot swarms is
introduced. The approach computes optimal rates at which the robots must switch between
the different tasks. These rates, in turn, are used to compute probabilities that determine
stochastic control policies of each robot. However, a common shortcoming of these decen-
tralized approaches is that they assume that the desired behavior is specified as a function
of the distribution of agents across the tasks.
Auction or market-based methods also provide solutions to the MRTA problem involv-
ing single-task robots [9, 11, 20, 33]. In [11], the robot responsible for any given task is
the robot who discovers the task. Once discovered, the robot holds an auction to recruit
other robots into a coalition. [20] introduce combinatorial bidding to form coalitions and
provides explicit cooperation mechanism for robots to form coalitions and bid for tasks.
A homogeneous task assignment algorithm for robot soccer is presented in [33]. Sensed
information from the robots are shared to compute a shared potential function that would
help the robots move in a coordinated manner. We refer readers to [9] for a survey of mar-
ket-based approaches applied to multi-robot coordination. A common trait of auction or
market-based methods is that they require extensive communication for bidding and scale
poorly with the number of robots in the team. Further, the methods discussed thus far are
limited to either single-robot tasks or single-task robots. In contrast, STRATA considers
agents capable of performing tasks that require coordination between multiple agents.
Our work falls under the category of Single-Task Robots Multi-Robot Tasks Instanta-
neous Assignment (ST-MR-IA) problem, also known as the coalition formation problem
[10]. In other words, we are interested in assigning a team of agents to several tasks, each
requiring several agents. The assignment type is instantaneous since our task assignment
does not reason about future task assignments or scheduling constraints. The ST-MR-IA
is an instance of the set-partitioning problem in combinatorial optimization and is known
to be strongly NP-Hard [10]. Albeit not developed for MRTA, a few efficient approximate
solutions have been proposed for the set partitioning problem [2, 12]. Based on prior work
in set partitioning problems, centralized and distributed algorithms to solve a ST-MR-IA
problem have been proposed in [29, 30]. The method in [30] has also been adapted to be
more efficient by reducing the required communication and discouraging imbalanced coali-
tions [34]. A method for coalition formation is introduced in [26] by building a solution to
a task by dynamically connecting a network of behaviors associated with individual robots.
13
A limitation of most of the existing approaches is that the desired behavior is assumed
to be specified in terms of optimal agent distribution. A notable exception to this generali-
zation is the framework introduced in [28], which is capable of receiving the task require-
ments provided in the form a desired trait distribution cross tasks. We take a position simi-
lar to [28], and do not assume that the desired distribution of agents is known. Another
similarity between STRATA and [28] is being suitable for a decentralized implementation.
Thus, both approaches are scalable in the number of agents and their capabilities, and are
robust to changes in the agent population.
While STRATA shares several similarities with [28], there are a number of notable
relative advantages. First, our species-trait model is continuous, while [28] uses a binary
model. Second, we differentiate between cumulative and non-cumulative traits. Third,
the framework in [28] utilizes a deterministic model of traits. In contrast, we consider the
inherent variability in the agent’ traits, thereby capturing the variations at both species and
agent levels. Finally, while the diversity measures introduced in [28] are limited to binary
trait models, our measures are compatible with continuous-space models.
3 Modeling framework
In this section, we introduce the various elements of STRATA that enables task assign-
ments in large heterogeneous teams. Assigning tasks to the different agents in the team
requires reasoning about their complementary traits and the limited resources of the team.
STRATA handles this challenge using (1) a stochastic trait model that describes the capa-
bilities of each species in the team along with the corresponding variance, (2) an task
graph that describes the dynamics and constraints associated with agents traversing the
task graph, and (3) a agent distribution model that specifies how agents are distributed
across the various tasks. Together, these models explain the combinations of capabilities
that are currently available at each task.
Based on the above mentioned models, we formulate and solve a constrained optimiza-
tion problem to distribute the agents across the different tasks to satisfy certain trait-based
task requirements. Specifically, we compute the transition rates on the task graph, which
in turn dictate how task assignments vary as a function of time such that the desired trait
distribution is achieved and maintained as quickly as possible. Further, our optimization
explicitly reasons about the expected variance of the trait distribution.
Throughout the paper, we illustrate STRATA using a running example of a task assign-
ment problem. We will progressively build the example as we introduce the different parts
of the framework.
3.1 Trait model
First, we introduce the details of our proposed stochastic trait model.
3.1.1 Base model
Consider a heterogeneous team of agents. We assume that each agent is a member of a

particular species. The number of species S ∈ ℕ is finite, and the number of agents in
the sth species is denoted by Ns. Thus, the total number of agents in the team is given by
13
∑
N = Ss=1 Ns We define each species by its abilities (traits). Specifically, the traits of each
species are defined as follows
q(s) ≜ [q(s)
1
, q(s)
2
, … , q(s)
U
], ∀s = 1, 2, … , S (1)
where q(s)
u ∈ ℝ+
is the uth trait of the sth species, and U is the number of traits. Thus, the
traits of the team is defined by a S × U species-trait matrix Q = [q(1) , … , q(S) ]T ∈ ℝ+S×U ,
T T
with each row corresponding to one species and each column corresponding to one trait.
3.1.2 Stochastic traits
To capture the natural variability in the capabilities of each species, we main-

tain a stochastic summary of each species’ traits. Specifically, each element of
Q is assumed to be an independent Gaussian random variable, q(s) 2 .
u ∼ N(𝜇su , 𝜎su )
Thus, a vector random variable with all the traits of all the species can be written as
q = [q(1) , q(2) , … , q(S) ] ∼ N(𝜇q , 𝛴q ), with its mean given by
𝜇q = [𝜇q(1) , … , 𝜇q(S) ] ∈ ℝSU

+ (2)
where 𝜇q(s) = [𝜇s1 , … , 𝜇sU ] ∈ ℝU contains the expected trait values of the sth species, and
its covariance given by the following block-diagonal matrix
𝛴q = diag([𝛴q(1) , … , 𝛴q(S) ]) ∈ ℝSU×SU

+ (3)
where 𝛴q(s) = diag([𝜎s1

2 2
, … , 𝜎sU ]) ∈ ℝU×U
+ is the diagonal covariance matrix associated
with the s species. The expected trait values can be rewritten in the form of the expected
th
species-trait matrix 𝜇Q = [𝜇qT(1) , … , 𝜇qT(S) ]T ∈ ℝS×U

+ as follows
⎡𝜇11 ⋯ 𝜇1U ⎤
𝝁Q = ⎢ ⋮ ⋱ ⋮⎥ (4)
⎢ ⎥
⎣𝜇S1 ⋯ 𝜇SU ⎦
Similarly, the non-zero diagonal elements of the covariance matrix can be rewritten in
matrix form as
⎡𝜎11
2
⋯ 2
𝜎1U ⎤
⎢
VarQ = ⋮ ⋱ ⋮⎥ (5)
⎢ 2 2 ⎥
⎣𝜎S1 ⋯ 𝜎SU ⎦
While the above stochastic model can be directly specified, it can also be learned directly
from data. Given the trait values of each agent in the team, clustering algorithms, such as
the Gaussian mixture model (GMM), can be used to automatically find Gaussian clusters,
each of which can be interpreted as a probabilistic representation of a species in the trait
space.
Example Consider an example scenario in which the team is made up of four species, each
consisting of 25 agents. Each species is characterized by the following four traits: (1) cov-
erage area (m2 ): a function of sensing capabilities, (2) area of influence (m2 ): a function of
actuation capabilities, (3) number of health packs: a function of payload capacity, and (4)
13
Fig. 2 The task graph associated

with our example task assign-
ment problem
ammunition: another function of payload capacity. Let the expected value of the species-
trait matrix and the corresponding matrix of variances for our example team be given by
⎡ 50 15 20 140⎤ ⎡3 1 1.5 5.6⎤

⎢150 10 10 0 ⎥ ⎢2 1.5 0.5 0⎥
𝝁Q = ⎢
175 0 25 60 ⎥
, VarQ = ⎢
1 0 2.4 8.7⎥ (6)
⎢ ⎥ ⎢ ⎥
⎣200 35 30 140⎦ ⎣6 2.3 3.9 9.2⎦
Note that STRATA allows for modeling traits of different orders of magnitude. Further, for
the same trait, the variation observed in each species is different. For instance, consider the
ammunition trait (4th columns of 𝜇Q and VarQ). The distribution of this trait is consider-
ably different in each species. Specifically, while Species 1 has the largest average units of
ammunition (140), it also has the smallest variance (5.6). On the other hand, Species 3 has
considerably lower units of ammunition (60) while its variance (8.7) is considerably higher
than that of Species 1. Encoding these aspects of each species enables STRATA to reason
about the various trade-offs involved in recruiting agents to meet the task requirements.
◻
3.2 Task graph
Given the trait model from the previous section, we require the team to accomplish M
tasks. We model the topology of the tasks using a strongly connected graph 𝔾 = (E, V).
The vertices V represent the M tasks, and the edges E connect tasks such that the exist-
ence of an edge between two tasks represents the agents’ ability to switch between them.
For each species, we aim to optimize the transition rate kij(s) for every edge in E , such that
0 < kij(s) < kij,max
(s)
.
The transition rate kij(s) defines the rate at which an agent from species s currently per-
forming task i switches to task j. The limits on transition rates can help incorporate realistic
constraints, such as the time required to travel between physical locations and to change the
tools required to perform tasks. The transition rates implicitly dictate how the distribution
of agents across tasks evolves in time. Note that, to account for the differences among spe-
cies, the transition rates and their limits are defined separately for each species.
Example Let our example problem involve 5 tasks and the task graph is shown in Fig. 2.
Note that the graph is not fully connected. This reflects the restrictions on how the agents
13
can switch between tasks. For instance, let each task be carried out in a different physi-
cal location. The presence (absence) of an edge between any two tasks implies that it is
(not) possible for the agents to move between the two tasks. STRATA explicitly takes these
restrictions into consideration when distributing agents across the task graph. ◻
3.3 Agent distribution
With the capabilities of the team and the tasks defined, the modeling of individual agents and
their assignments remains. The distribution of agents from species s across the M tasks at time
t is defined by x(s) (t) = [x1(s) (t), x2(s) (t), … xM
(s)
(t)]T ∈ ℕM . Thus the distribution of the whole
team across the tasks at time t can be described using a abstract state information matrix
X(t) = [x(1) (t), x(2) (t), … , x(S) (t)] ∈ ℕM×S.
Example Let us assume that the initial distribution of agents, perhaps a result of initial
deployment or earlier task requirements, is given by
⎡25 0 0 0⎤
⎢0 25 0 0⎥
X(0) = ⎢ 0 0 25 0 ⎥. (7)
⎢ ⎥
⎢0 0 0 25⎥
⎣0 0 0 0⎦
Thus, initially, all the 25 agents from Species 1 are assigned to Task 1, all agents from
Species 2 to Task 2, and so on. Further, no agents are assigned to Task 5. Note that each
column adds up to the number of agents in the corresponding species. ◻
As in [28], the time evolution of the number of agents from Species s at Task i is explained
by the following dynamical system
∑ (s) (s)
ẋ i(s) (t) = kji xj (t) − kij(s) xi(s) (t)
(8)
∀j|(i,j)∈E
and thus the dynamics of each species’ abstract state information can be computed as
ẋ (s) (t) = K (s) x(s) (t), ∀s = 1, 2, … , S (9)

where K (s) ∈ ℝM×M
+
is the rate matrix of species s, defined as follows
⎧ k(s) , if i ≠ j, (i, j) ∈ E
⎪ ji
Kij(s) = ⎨ 0, if i ≠ j, (i, j) ∉ E (10)
⎪ − ∑M (s)
if i = j
⎩ i=1,(i,j)∈E kij ,
The solution of the dynamics in (9) for sth species can be written as
(11)
(s) 𝜏
x(s) (𝜏) = eK x(s) (0)
Thus, the time evolution of the abstract state information is given by
13
∑
S
(s) 𝜏
X(𝜏) = eK z(s) (0) (12)
s=1
where z(s) (0) = X(0) ⊙ (1 ⋅ es ) ∈ ℕM×S , 1 is an M-dimensional vector of ones, and es is the
S-dimensional unit vector with its sth element equal to one.
3.4 Trait aggregation and distribution
Finally, we represent the trait distribution across the tasks by the trait distribution matrix
Y(t) ∈ ℝM×U
+
and is computed as
Y(t) = X(t)Q (13)
Thus, each column of Y(t) represents the aggregated amounts of the corresponding trait
available at each task at time t. Put another way, Y(t) represents the aggregation of various
traits assigned to each task at time t.
Note that the stochastic nature of Q results in the elements of Y(t) being random vari-
ables. The expected value of Y(t) can be computed as follows
𝝁Y (t) = X(t)𝝁Q (14)
and since the elements of Q are independent random variables, the closed form expression
for the variance of each element of Y can be captured in the following matrix
VarY (t) = (X(t) ⊙ X(t)) VarQ (15)
where ⊙ denotes the Hadamard (entry-wise) product. Furthermore, the covariance between
any two elements of Y is given by
� ∑S
(x(s) xk(s) 𝜎sj2 ), if j = l
Cov{Y ij , Y kl } = s=1 i (16)
0, otherwise
Example The expected value of the species-trait matrix and the initial abstract state infor-
mation of our example problem are defined in (6) and (7), respectively. Thus, the expected
values of the initial trait distribution and the associated variances can be computed using
(14) and (15), and are thus given by
⎡1250 375 500 3500⎤ ⎡1875 625 937.5 3500 ⎤

⎢3750 250 250 0 ⎥ ⎢1250 937.5 312.5 0 ⎥
𝝁Y (0) = ⎢4375 0 625 1500⎥ VarY (0) = ⎢ 625 0 1500 5437.5⎥
⎢ ⎥ ⎢ ⎥
⎢5000 875 750 3500⎥ ⎢3750 1437.5 2437.5 5750 ⎥
⎣ 0 0 0 0 ⎦ ⎣ 0 0 0 0 ⎦
The above matrices explain how the team’s capabilities are expected to be distributed
across the different tasks. ◻
13
3.5 Cumulative versus non‑cumulative traits
Indeed, not all traits are suitable for aggregation. To handle this fact, we explicitly differen-
tiate between two kinds of traits: cumulative and non-cumulative traits. While examples of
cumulative traits include ammunition, equipment, and coverage area, those of non-cumulative
traits include speed and special skills. We model cumulative traits as continuous variables
(i.e., q(s)
i
∈ ℝ+ , ∀i ∈ C, where C ⊆ {1, 2, … , U} is the set of indices corresponding to cumu-
lative traits). To handle non-cumulative traits, we approximate them as binary variables (i.e.,
q(s)
j
∈ {0, 1}, ∀j ∈ {1, 2, … , U} ⧵ C).
Unlike prior methods that only consider the existence of capabilities to define binary traits,
STRATA assigns binary values to non-cumulative traits based on the following rule
{
1, if 𝜇si ≥ qmin
𝜇si =
0, otherwise
i
(17)
where qmin
i
is a user-defined minimum acceptable value for the ith trait. The binary repre-
sentation of non-cumulative traits captures information about whether the agents of each
species posses the minimum required capabilities. Further, when aggregated to form 𝝁Y
(Sect. 3.4), the binary representation of non-cumulative traits provides the total number of
agents that meet the minimum requirements.
Example Let us expand our example, which had four cumulative traits, to include a new
non-cumulative trait: speed (m/s). Let the expected speed trait for each species be given by
𝜇q(5) = [8, 2, 5, 10]. In order to represent the non-cumulative speed trait in the binary form,
let the minimum acceptable value for speed be qmin 5
= 7 m/s. The matrix denoting the new
species-trait distribution will contain an additional column and is given by
⎡ 50 15 20 140 1⎤
⎢150 10 10 0 0⎥
𝝁�Q =⎢
175 0 25 60 0⎥ (18)
⎢ ⎥
⎣200 35 30 140 1⎦
Note that the average speeds of Species 2 (2 m/s) and 3 (5 m/s) are lower than the mini-
mum requirement of 7 m/s miles. Thus, Species 2 and 3 are considered to not meet the
minimum requirements for speed and are assigned zeros for the same trait.
Given an assignment X(0), similar to cumulative traits, the effects of the new non-cumula-
tive trait can be observed by computing the new expected a task-trait distribution using (14),
and is thus given by
⎡1250 375 500 3500 25⎤

⎢3750 250 250 0 0⎥
𝝁�Y (0) = ⎢4375 0 625 1500 0⎥ (19)
⎢ ⎥
⎢5000 875 750 3500 25⎥
⎣ 0 0 0 0 0⎦
Note that, while the first four columns of the task-trait distribution (that correspond to
cumulative traits) remain unchanged, the last column (that corresponds to the non-cumu-
lative speed trait) now represents the number of agents assigned to each task that meet the
minimum requirement. ◻
13
4 Problem formulation
Based on the modeling framework described in Sect. 3, this section considers the problem
of task assignment that achieves a desired trait distribution across tasks. Specifically, we
wish to find the transition rates K (s) for each species such that the expected trait distribution
over tasks 𝜇Y (t), defined in (13), reaches the desired trait distribution Y ∗ as quickly as pos-
sible while minimizing the expected variance in the trait distribution.
We express the problem as the following optimization problem
∗
𝜏 ∗ , K (s) = arg min
(s)
𝜏 (20)
𝜏,K
s.t. X(𝜏)𝜇Q ∈ G(Y ∗ ) (21)
‖VarY (𝜏)‖F ≤ 𝜖var (22)

where 𝜖var is the threshold used to limit the variance in Y(𝜏), G(Y ∗ ) ∶ ℝM×U
+ → 𝛺, named
the goal function, is a function that defines the set of admissible expected trait distribu-
tion matrices 𝛺. Note that the constraint in (22) helps minimize the expected variance,
and thereby, maximize the chances that the actual trait distribution Y(𝜏 ∗ ) meets the goal
function.
We consider two goal functions:
1. Exact matching: G1 (Y ∗ ) = {𝜇Y |Y ∗ = 𝜇Y }

2. Minimum matching: G2 (Y ∗ ) = {𝜇Y |Y ∗ ⪯ 𝜇Y }
where ⪯ denotes the element-wise less-than-or-equal-to operator. While goal function G1

requires achieving the exact desired trait distribution, goal function G2 requires the trait dis-
tribution be greater than or equal to the desired trait distribution. In other words, G1 does not
allow any deviation from the desired trait distribution, and G2 allows for over-provisioning.
Example Let the desired trait distribution for our example be given by
⎡ 0 0 0 0 0⎤
⎢1250 375 500 3500 25⎥
Y ∗ = ⎢3750 250 250 0 0 ⎥. (23)
⎢ ⎥
⎢4375 0 625 1500 0⎥
⎣5000 675 750 3500 25⎦
Note that the expected initial trait distribution 𝜇Y� (0) is defined in (19). Thus, the task
assignment algorithm is required to compute the transition rates K (s) such that the team’s
∗
expected trait distribution satisfies the chosen goal function as quickly as possible. ◻
13
5 Diversity measures
Large heterogeneous teams with multiple species might result in capabilities that are
complementary and or redundant. We study the properties of the average species-trait
matrix 𝜇Q to understand the similarities and variations among the species of a given
team. Measures of team diversity were defined in [28] for species defined by binary
traits. In this section, we define diversity measures for species defined by continuous
traits. We define two measures of trait diversity for a given team, one for each of the two
goal functions defined in Sect. 4. To this end, we utilize the following definitions.
Definition 1 Minspecies: In a team described by an average species-trait matrix 𝜇Q, a min-

species set is a subset of rows of 𝜇Q with minimal cardinality, such that the system can still
achieve the goal G(Y ∗ ).
Definition 2 Minspecies cardinality: The cardinality of the minspecies set is defined as the
Minspecies cardinality and is represented by the function DG ∶ ℝS×U
+ → ℕ+ that takes the
average species-trait matrix 𝜇Q as the input and returns the minimum number of species
required to achieve the goal G(Y ∗ ).
5.1 Eigenspecies
First, we define a diversity measure related to the exact matching goal, G1.
Proposition 1 The cardinality of eigenspecies (the minspecies corresponding to goal

function G1) is computed as follows
DG1 = min |M1 | (24)
∑
s.t. 𝛼s̃s 𝜇q(s) = 𝜇q(̃s) , ∀̃s ∉ M1 , ∀𝛼s̃s ∈ ℕ
(25)
s∈M1
where M1 is a subset of all the species in the team, | ⋅ | denotes the cardinality, and ℕ is the
set of all non-negative integers.
Proof The expected species-trait matrix can be factorized as 𝜇Q = A𝜇̂ Q, such that
𝜇q = A𝜇̂ Q, where A ∈ ℕS×|M1 | and 𝜇̂ Q ∈ ℝ|M1 |×U . Now, Y ∗ = X∗ 𝜇Q = X∗ A𝜇̂ Q = X̂ 𝜇̂ Q
where X̂ = X∗ A. Thus, there exists an agent distribution X̂ that can achieve the goal G1 with
only a subset of the species, defined using the minimal species-trait matrix 𝜇̂ Q. ◻
Thus, M1 contains the minimal set of species that can exactly match any desired
trait distribution without recruiting agents from species not in M1, and DG1 denotes the
number of species that form M1. Note that weighting factors of the sum are restricted
to be natural numbers. The motivation behind this restriction is that, when aggregat-
ing traits, the weighting factor corresponds to the number of agents we are considering
when aggregating traits.
13
Example For the team in our example, the average species-trait matrix is defined in (6).
Note that the sum of the first row rows is equal to the last row. Further, no other rows
are equal to the weighted (by natural numbers) sum of the remaining rows. Specifically,
𝜇q(1) + 𝜇q(2) = 𝜇q(4). The average species-trait matrix can thus be factorized as 𝜇Q = A𝜇̂ Q,
where
⎡1 0 0⎤
⎡ 50 15 20 140 1⎤
⎢0 1 0⎥ ⎢150
A=⎢ 0⎥ (26)
1⎥ Q ⎢
𝜇
̂ = 10 10 0
⎢
0 0
⎥ ⎥
⎣1 1 0⎦ ⎣175 0 25 60 0⎦
Thus, M1 = {1, 2, 3} and consequently DG1 = 3. In other words, the traits of only one spe-
cies (Species 4) can be exactly matched by aggregating the traits of other species (Species
1 and 2). ◻
5.2 Coverspecies
Next, we define a diversity measure for the minimum matching goal, G2.
Proposition 2 The cardinality of coverspecies (the minspecies corresponding to goal

function G2) is computed as follows
DG2 = min |M2 | (27)
∑
s.t. 𝛼s̃s 𝜇q(s) ⪰ 𝜇q(̃s) , ∀̃s ∉ M2 , ∀𝛼s̃s ∈ ℕ
(28)
s∈M2
Proof The expected species-trait matrix can be factorized as 𝜇Q = A𝜇̂ Q, such that
𝜇Q ⪯ A𝜇̂ Q, where A ∈ ℕS×|M2 |, 𝜇̂ Q ∈ ℝ|M2 |×U . Now, Y ∗ ⪯ X∗ 𝜇Q ⪯ X∗ A𝜇̂ Q. Thus, there
exists an agent distribution X̂ = X∗ A that can achieve the goal G2 with only a subset of the
species, defined using the species-trait matrix 𝜇̂ Q. ◻
Thus, M2 contains the minimal set of species that can satisfy (with potential over-pro-
vision) any desired trait distribution without recruiting agents from species not in M2, and
DG2 is the number of species that form such a minimal set.
Example For the team in our example, the average species-trait matrix is defined in (6).
Note that each element of the last row is larger or equal to the corresponding elements
in every other row. The average species-trait matrix can thus be factorized as 𝜇Q ⪯ A𝜇̂ Q,
where
⎡1⎤
⎢1⎥ � �
A=⎢ ⎥
1
𝜇̂ Q = 200 35 30 140 1 (29)
⎢ ⎥
⎣1⎦
Thus, M2 = {4} and consequently DG2 = 1. In other words, the traits of three species (Spe-
cies 1,2, and 3) can be minimum matched by the traits of one species (Species 4). ◻
13
6 Solution approach
This section details the proposed solution to the optimization problem defined in (20)–(21).
Our solution computes the transition rates K (s) without assuming knowledge of the optimal
∗
agent distribution X∗.
6.1 Optimization criteria
We begin by considering the time evolution of average trait distribution over the tasks. To this
end, we combine (14) and (11), yielding
∑
S
(s) 𝜏
𝝁Y (𝜏, K (1,..,S) , x(s) (0)) = eK x(s) (0) 𝜇q(s) (30)
s=1
In order to satisfy the goal function constraint, as defined in (21), we impose constraints
on two error functions. The first error function computes the trait distribution error and is
defined separately for each goal function as follows:
G
E1 1 (𝜏, K (1,..,S) , X(0)) =‖Y ∗ − 𝝁Y (𝜏)‖2F (31)
G
E1 2 (𝜏, K (1,..,S) , X(0)) =‖ max[(Y ∗ − 𝝁Y (𝜏)), 0]‖2F (32)
where ‖ ⋅ ‖F denotes the Frobenius norm of a matrix. Note that we have omitted the
dependence of 𝜇Y (⋅) on the transition rates and initial conditions for brevity. The second
error function measures the deviation from the steady state agent trait distribution and is
defined as follows for both goal functions:
�
S
(s) 𝜏
E2 (𝜏, K (1,2,..,S) , X(0)) = ‖eK x(s) (0)
s=1 (33)
K (s) (𝜏+𝜈) (s)
− e x (0)‖22
The first error function E1 (for both goal functions) penalizes the system when the trait
distribution at time 𝜏 does not satisfy the appropriate goal, and the error function E2 penal-
izes the system if its trait distribution does not reach steady state at time 𝜏 . Thus, enforcing
upper bounds on these error functions guarantees a certain minimum level of performance.
Further, as noted in (22), the expected variation in the achieved trait distribution, VarY (𝜏),
is also considered in the optimization.
6.2 Optimization problem
Based on the definitions in Sects. 6.1, we reformulate the optimization problem in (20)–(22)
for goal G1 as follows
∗
𝜏 ∗ , K (s) = arg min
(s)
𝜏 (34)
𝜏,K
13
G
s.t. E1 1 (𝜏, K (1,..,S) , X(0)) ≤ 𝜖1 (35)
E2 (𝜏, K (1,..,S) , X(0)) ≤ 𝜖2 (36)
‖VarY (𝜏, K (1,..,S) , X(0))‖2F ≤ 𝜖var (37)
kij(s) ≤ kij,max
(s)
, ∀i, j = {1, … , M}, ∀s = {1, .., S} (38)
𝜏>0 (39)
where 𝜖1, 𝜖2, and 𝜖var are user-defined positive scalars. Note that the optimization
problem for goal G2 is identical except that we replace the constraint in (35) with
G
E1 2 (𝜏, K (1,..,S) , X(0)) ≤ 𝜖1.
Note that the solution to the optimization problem in (34)–(39) guarantees minimum
levels of performance, both in terms of achieving and maintaining the appropriate goal, as
defined by the arbitrary constants 𝜖1 and 𝜖2, respectively. The constraint in (37) helps ensure
that the expected variance of the achieved trait distribution is below a desired threshold.
Thus, for each task, the constraint in (37) encourages the system to recruit agents who pos-
sess traits (required for the task) with relatively low variance, there by increasing the odds
of the actual trait distribution satisfying the specified goal.
6.3 Analytical gradients
To efficiently solve the the optimization problem in (34)–(39), we derive and utilize the
analytical gradients of all the constraints with respect to the decision variables. In this sec-
tion we define the analytical gradients of constraints defined in (35)–(39) with respect to
the unknowns 𝜏 and K (s). We refer the readers to [28] for closed-form expressions of the
G G
derivatives of E1 1, E1 2 and E2 with respect to both K (s) and 𝜏.
We adapt the closed-form expressions for derivatives of the matrix exponential [16], and
use the chain rule to derive the derivatives of ‖VarY ‖2F with respect K (s) and 𝜏 as follows2
𝜕‖VarY ‖2F 𝜕‖VarY ‖2F 𝜕eK (s) 𝜏 𝜕K (s) 𝜏

= (40)
𝜕K (s) 𝜕eK (s) 𝜏 𝜕K (s) 𝜏 𝜕K (s)
𝜕‖VarY ‖2F 𝜕‖VarY ‖2F 𝜕eK (s) 𝜏 𝜕K (s) 𝜏

= (41)
𝜕𝜏 𝜕eK (s) 𝜏 𝜕K (s) 𝜏 𝜕𝜏
where
𝜕‖VarY ‖2F � �
= 4 (VarY VarTQ ) ⊙ X(t) z(s) (0) (42)
𝜕eK (s) 𝜏
Thus, the closed form expressions of the derivatives are given by
2
We drop the arguments of VarY for brevity.
13
𝜕‖VarY ‖2F
= (V (s) )−T B(s) (V (s) )T 𝜏 (43)
𝜕K (s)
𝜕‖VarY ‖2F �
S
� �
= 1T (V (s) )−T B(s) (V (s) )T K (s) 1 (44)
𝜕𝜏 s=1
where K (s) = V (s) D(s) (V (s) )−1 is the eigenvalue decomposition of K (s), D(s) = diag(d1 , … , dM )
is the diagonal matrix with the eigenvalues of K (s), B(s) is a M × M matrix defined as
� �
𝜕‖Var Y ‖ 2
F
B(s) = (V (s) )T (V (s) )−T ⊙ W (s) (45)
𝜕eK (s) 𝜏
and W(𝜏) is a M × M matrix with its klth element is given by

{ dk 𝜏 dl 𝜏
(e −e )
(s) , k≠l
Wkl =
d
dk 𝜏−dl 𝜏
𝜏
(46)
ek , k=l
6.4 Decentralized online implementation
We note that it is possible to realize a decentralized implementation of the proposed opti-

mization problem in (34)–(39). Indeed, prior work has demonstrated that one can compute
the state information X locally, and enable online adaptation (re-optimization) of the transi-
tion rates as the state information is updated [3, 15, 24, 28]. Our approach, while introduc-
ing new capabilities, directly inherits these advantages from prior work.
7 Experimental evaluation
We evaluate STRATA using two sets of experiments. In both experiments we compare

STRATA’s performance with that of a baseline. Our baseline method is a bootstrapped ver-
sion of the binary-trait-based method introduced in [28].3
7.1 Baseline and metrics
Since the baseline algorithm [28] requires the desired trait distribution to be speci-
fied in the binary trait space and only considers binary species-trait distributions, we
make the following modifications to the baseline. We define a binary species-trait matrix
to be Q̄ = sign(𝜇Q ), where sign(⋅) is the signum operator applied to each element of its
matrix argument. We also define a modified desired trait distribution for the baseline:
Ȳ = ⌊Y ∗ ⊘ 𝜇Y ⌋, where ⌊⋅⌋ is the floor function applied to each element of a matrix, ⊘
refers to Hadamard (element-wise) division, 𝜇Y = [𝜇Y1 ⊙ 1M , … , 𝜇YU ⊙ 1M ], 𝜇Yi is the mean
desired value of the ith trait computed across all species, and 1M is a M-dimensional vec-
tor of ones. We implemented both the baseline and STRATA on a computer running Intel
Core i7-4770K with 16 GB of memory.
3
Source code acquired from https://github.com/amandaprorok/diversity.git.
13
Fig. 3 The four error measures used to quantify the proportion of trait mismatch
The task assignment performance of each method is evaluated in terms of four meas-
ures of percentage trait mismatch, as defined in Fig. 3. As illustrated, the metrics measure
performance in two scenarios: when the actual traits of the agents are (1) assumed to be
known and equal to the average of the corresponding trait (constant trait), and (2) assumed
to be unknown and sampled based on the trait distribution (trait variation). Additionally,
in each scenario, the performance is measured in terms of both exact and minimum trait
matching, irrespective of the optimization goal.
7.2 Simulation
In the first set of experiments, we study the performances of STRATA and the baseline
in terms of matching the desired trait requirements for a large heterogeneous team. To
this end, we simulate a task assignment problem with M = 8 nodes (tasks), U = 5 traits
(3 cumulative and 2 non cumulative traits), and S = 5 species (each with 200 agents). We
present the results computed from 100 independent simulation runs.
In each run, we make the following design choices. The task graph along with its con-
nections is randomly generated. The initial and final agent distributions, X(0) and X∗, are
uniformly randomly generated. Based on the obtained X∗, a desired trait distribution Y ∗
is computed for each run. The expected value of the species-trait matrix is chosen to be
𝜇Q = [a, a, a, b, b]T , where each element of a ∈ ℝU +
is sampled from a uniform distribution:
ai ∼ U(0, 10), and each element of U-dimensional b is sampled from a discrete uniform
distribution: bi ∼ U{0, 1}. Each element of VarQ is sampled from a uniform distribution:
U(0, 2). An example initial and desired trait distribution is illustrated in Fig. 4. The maxi-
mum transition rates kij,max
(s)
, ∀i, j, s are chosen to be 0.02 S−1. The thresholds 𝜖1 and 𝜖2 are
both chosen so as to be equivalent to 5% of ‖Y ∗ ‖F.
To ensure a fair comparison, we limit both STRATA and the baseline framework to a
maximum of 20 meta iterations of the basin hopping algorithm during each run. In order to
measure 𝛿G1 (Q) and 𝛿G2 (Q) for each run, 10 samples of the trait-species matrix Q are gener-
ated to compute Y(Q)
̂ .
7.2.1 Exact trait matching
First, we compute the transition rates according to both STRATA and the binary trait
framework [28] with respect to the function Goal G1. STRATA is found to converge
during 79 of the 100 simulation runs and the binary trait framework during 10 runs. In
Fig 5, we present the performances of both frameworks by plotting the errors (defined
in Fig. 3) as functions of time. Note that the error plots for each method reflect the
error measures computed only across the converged runs.
13
Fig. 4 The initial (left) and desired (right) team configurations from an exemplary run with eight tasks
(nodes) and four traits. The bar plots denote the trait distribution at each task and the edges represent the
possibility of switching between the corresponding tasks
Fig. 5 Comparison of the performances of STRATA and the baseline [28] (binary) framework when opti-
mizing for exact matching (G1). The performance of each framework is quantified in terms of four measures
of percentage trait mismatch
As shown in Fig. 5a, b, STRATA consistently performs better than the baseline in
terms of deterministic performance, as measured by both 𝛿G1 (𝜇Q ) and 𝛿G2 (𝜇Q ) . Further,
as shown in Fig. 5c, d, when the agents’ traits are randomly sampled, STRATA per-
forms better than the baseline on average. The stochastic nature of the species-trait
matrix forces the errors to be larger than zero.
13
Fig. 6 Comparison of the performances of STRATA and the baseline [28] (binary) framework when opti-
mizing for minimum trait matching (G2). The performance of each framework is quantified in terms of four
measures of percentage trait mismatch
7.2.2 Minimum trait matching
Next, we compute the transition rates according to both STRATA and the binary trait
model [28] with respect to the function Goal G2. STRATA is found to converge during
85 of the 100 simulation runs and the binary trait framework during 16 runs. In Fig 6, we
present the performances of both frameworks by plotting the errors (defined in Fig. 3) as
functions of time.
STRATA consistently performs better than the baseline when optimizing to satisfy min-
imum trait distribution, as measured by 𝛿G2 (𝜇Q ). On average, STRATA performs better than
the baseline when considering stochastic species-trait matrix, as measured by 𝛿G2 (Q). These
assertions are supported by the plots in Fig. 6b, d. In Fig. 6a, c, the baseline exhibits high
error and variance in terms of both 𝛿G1 (𝜇Q ) and 𝛿G1 (Q). This implies that when optimizes
for G2, the binary trait model, unlike STRATA, results in a high level of over-provisioning.
7.2.3 Discussion
As demonstrated by the results above, reasoning about stochastic traits results in consist-
ently satisfying complex task requirements. Stochastic trait models outperform binary trait
models for a number a reasons. Firstly, binary trait models are incapable of reasoning about
requirements in the continuous trait space. Secondly, to construct the modified (binary)
trait distribution Ȳ , one is required to consider, at minimum, the average value of each trait
in the team. In the process, however, the binary trait model ignores all variations both at
the species and individual levels. Indeed, the advantages of considering these variations are
reflected in terms of more accurate assignment of agents in Figs. 5 and 6.
Given that the results above reflect only the performance from converged runs, it is impor-
tant to investigate the ratio of converged runs to total number of runs. As seen in Fig. 7,
STRATA successfully converged to a solution in significantly ( p < 0.001) more runs than the
binary trait framework for both exact trait matching (G1) and minimum trait matching (G2).
This observation demonstrates that considering stochastic and continuous trait models over
binary models is considerably more likely to satisfy complex trait requirements.
13
STRATA
Trait Matching
Baseline [28]
Exact
*
Trait Matching
Minimum
0 10 20 30 40 50 60 70 80 90 100
Number of Converged Runs
Fig. 7 Number of converged runs for each algorithm out of 100 simulation runs
7.3 Capture the flag
In this section, we study the question: Does STRATA improve higher-level team perfor-
mance? To this end, we quantify the effect of STRATA on team performance in a capture
the flag (CTF) game. Note that we do not explicitly model the adversarial elements of the
game, and focus only on effectively assigning agents to roles, based on an empirically-
identified ideal trait distribution Y ∗. We built the environment using the Unity 3D game
engine.
Table 1 Specifications of the # Species # agents # Tasks # Traits Role assignment

teams implemented in the per spe-
capture the flag environment cies
Team A 4 3 3 4 STRATA
Team B 4 3 3 4 Baseline [28]
Team C 4 3 3 4 Random
13
Wins Draws Losses

*
Baseline vs. Random 377 27 96
*
STRATA vs. Random 382 30 88
*
STRATA vs. Baseline 281 48 171
Fig. 8 Relative performances of random task assignment, the baseline framework [28] and STRATA across
500 runs of the capture-the-flag game. An asterisk indicates the statistically significant ( p < 0.001) differ-
ence between the proportion of wins to losses in each match-up
7.3.1 Experimental design and results
We compare the performances of three teams, named A, B, and C. The details pertaining
to each team are listed in Table 1. The three tasks (roles) in the game are defend, attack,
and heal, in that order. The four traits are speed, viewing distance, health, and ammuni-
tion, in that order. We consider speed and viewing distance are non-cumulative traits, and
health and ammunition as cumulative traits. Similar to the experiments in simulation, the
maximum transition rates kij,max
(s)
, ∀ i, j, s are chosen to be 0.02 S−1, and the thresholds 𝜖1
and 𝜖2 are both chosen so as to be equivalent to 5% of ‖Y ∗ ‖F . The average time to optimize
the transition rates for the game with M = 3, U = 4, and S = 4 is around 0.88 ms for the
baseline and 0.97 ms for STRATA.
The baseline task assignment strategy, similar to the experiment in Sect. 7.2, is a boot-
strapped version of the binary-trait-based method introduced in [28]. For the random task
assignment strategy, each agent is assigned uniformly randomly to one of the three roles.
Note that both the algorithms are provided with identical teams, consisting of 12 agents.
Thus, any variation in performance is limited to the task assignment strategy used by each
team and the inherent randomness of the game.
The traits of the agents are sampled from the following stochastic species-trait matrix
⎡1.5 15 90 40⎤ ⎡0.35 5 10 3⎤

⎢1.5 30 60 40⎥ ⎢0.35 5 10 3⎥
𝜇Q = ⎢
30⎥
VarQ = ⎢
3 15 80 0.35 5 10 3⎥
⎢ ⎥ ⎢ ⎥
⎣3 30 350 30⎦ ⎣0.35 5 10 3⎦
The minimum acceptable value for the non-cumulative traits are chosen to be as follows:
q(1)
min
= 0 m/s for speed and q(2)
min
= 10 m for viewing distance. The desired trait distribution
is designed to be
⎡2 2 120 80 ⎤
Y ∗ = ⎢6 6 340 200⎥
⎢ ⎥
⎣4 4 320 140⎦
13
All games are played with two teams at a time, one versus another. We compare the perfor-
mances of each team against the other two teams. We consider a team to have won a game
if the team captures the opponent’s flag and brings it back to the starting position. If neither
team is able to capture and bring back the opponent’s flag in 120 s, then the team with the
highest number of active agents is considered the winner. Lastly, if both teams retain the
same number of active agents after 120 s, the game is considered to have ended in a draw.
The relative performances of all three approaches are illustrated in Fig. 8.
7.3.2 Discussion
As shown in Fig. 8, given appropriate Ȳ and Y ∗, both the baseline framework and STRATA
are more likely to win against random task assignment. However, when compared head-to-
head with the same Ȳ and Y ∗, STRATA is more likely to win against the baseline frame-
work. Further, based on the z-test, we find that the proportions of wins are statistically
significantly ( p < 0.001) higher than those of losses in all three conditions. These obser-
vations are likely due to the fact that the baseline framework implicitly reasons about the
average trait values when corresponding the modified task requirements Ȳ that is compat-
ible to the binary trait space. This type of reasoning, while limited, is still more effec-
tive than not reasoning about any of the factors that influence team performance. However,
similar to our observations in the simulated experiments, reasoning about continuous trait
models along with inter- and intra- species variations is not considerably more effective
than reasoning about binary approximations of traits. Thus, STRATA’s ability to reason
about the traits and task requirements translates to high-level team performance.
8 Conclusion
We presented STRATA, a unified framework capable of effective task assignments in large

teams of heterogeneous agents. The members of the team are modeled as belonging to dif-
ferent species, each defined by a set of its capabilities. STRATA models capabilities in
the continuous space and explicitly takes into account both species-level and agent-level
variations. Further, we quantified the diversity of a given team by introducing two separate
notions of minspecies, each specifying the minimal subset of species necessary to achieve
the corresponding goal. Finally, we illustrated the necessity and effectiveness of STRATA
using two sets of experiments. The experimental results demonstrate that STRATA (1)
successfully distributes a large heterogeneous team to meet complex task requirements,
(2) consistently performs better than the baseline framework that only considers binary
traits, and (3) results in improved higher-level team performance in a simulated game of
capture-the-flag.
Our proposed approach and its limitations have revealed exciting avenues for future
research. Firstly, STRATA can be extended to consider the importance of different tasks
or traits, resulting in preferential or importance-based task assignment. Secondly, given
the complexity of the optimization problem, it is yet unclear how to provide guarantees
on the optimality of the solution. Thirdly, to minimize risk, STRATA reduces variance in
expected trait distribution by choosing the most consistent species for tasks. However, it
might beneficial in scenarios with very limited resources to consider maximizing trait vari-
ance in the interest of the possibility of higher rewards. Finally, we have introduced new
13
measures of diversity based in a comprehensive class of trait models. It would be inter-

esting to explore how to effectively utilize these measures in order study the trade-offs
involved in the synthesis and composition of effective multi-agent teams.
Acknowledgements We would like to thank the anonymous reviews for their constructive feedback that
greatly helped improve the quality of this article. This work was supported by the Army Research Lab under
Grant W911NF-17-2-0181 (DCIST CRA).
References
1. Albrecht, S. V., & Stone, P. (2018). Autonomous agents modelling other agents: A comprehensive sur-
vey and open problems. Artificial Intelligence, 258, 66–95.
2. Atamtürk, A., Nemhauser, G. L., & Savelsbergh, M. W. (1996). A combined lagrangian, linear pro-
gramming, and implication heuristic for large-scale set partitioning problems. Journal of Heuristics,
1(2), 247–259.
3. Bandyopadhyay, S., Chung, S. J., & Hadaegh, F. Y. (2017). Probabilistic and distributed control of a
large-scale swarm of autonomous agents. IEEE Transactions on Robotics, 33(5), 1103–1123.
4. Beni, G. (2004). From swarm intelligence to swarm robotics. In International Workshop on Swarm
Robotics (pp. 1–9). Berlin: Springer.
5. Berman, S., Halász, Á., Hsieh, M. A., & Kumar, V. (2009). Optimized stochastic policies for task allo-
cation in swarms of robots. IEEE Transactions on Robotics, 25(4), 927–937.
6. Bordini, R. H., Dastani, M., Dix, J., & Seghrouchni, A. E. F. (2007). Programming multi-agent-sys-
tems. In 4th international workshop, revised and invited papers (Vol. 4411). New York: Springer.
7. Cohen, L., Uras, T., Kumar, T.S., Xu, H., Ayanian, N., & Koenig, S. (2016) Improved solvers for
bounded-suboptimal multi-agent path finding. In International joint conferences on artificial intelli-
gence (IJCAI) (pp. 3067–3074)
8. DeCostanza, A. H., Marathe, A. R., Bohannon, A., Evans, A. W., Palazzolo, E. T., Metcalfe, J. S.,
et al. (2018). Enhancing human-agent teaming with individualized, adaptive technologies: A discus-
sion of critical scientific questions. Technical report, US Army Research Laboratory Aberdeen Proving
Ground United States.
9. Dias, M. B., Zlot, R., Kalra, N., & Stentz, A. (2006). Market-based multirobot coordination: A survey
and analysis. Proceedings of the IEEE, 94(7), 1257–1270.
10. Gerkey, B. P., & Matarić, M. J. (2004). A formal analysis and taxonomy of task allocation in multi-
robot systems. The International Journal of Robotics Research, 23(9), 939–954.
11. Guerrero, J., & Oliver, G. (2003). Multi-robot task allocation strategies using auction-like mecha-
nisms. Artificial Research and Development in Frontiers in Artificial Intelligence and Applications,
100, 111–122.
12. Hoffman, K. L., & Padberg, M. (1993). Solving airline crew scheduling problems by branch-and-cut.
Management Science, 39(6), 657–682.
13. Hsieh, M. A., Halász, Á., Berman, S., & Kumar, V. (2008). Biologically inspired redistribution of a
swarm of robots among multiple sites. Swarm Intelligence, 2(2–4), 121–141.
14. Jang, I., Shin, H. S., & Tsourdos, A. (2018a). Anonymous hedonic game for task allocation in a large-
scale multiple agent system. IEEE Transactions on Robotics, 34(6), 1534–1548.
15. Jang, I., Shin, H. S., & Tsourdos, A. (2018b). Local information-based control for probabilistic swarm
distribution guidance. Swarm Intelligence, 12(4), 327–359.
16. Kalbfleisch, J., & Lawless, J. F. (1985). The analysis of panel data under a markov assumption. Journal
of the American Statistical Association, 80(392), 863–871.
17. Khamis, A., Hussein, A., & Elmogy, A. (2015). Multi-robot task allocation: A review of the state-of-
the-art. In Cooperative robots and sensor networks 2015 (pp. 31–51). Berlin; Springer
18. Korsah, G. A., Stentz, A., & Dias, M. B. (2013). A comprehensive taxonomy for multi-robot task allo-
cation. The International Journal of Robotics Research, 32(12), 1495–1512.
19. Li, J., Esteban-Fernàndez de Àvila, B., Gao, W., Zhang, L., & Wang, J. (2017). Micro/nanorobots for
biomedicine: Delivery, surgery, sensing, and detoxification. Science Robotics. https://doi.org/10.1126/
scirobotics.aam6431.
20. Lin, L., & Zheng, Z. (2005). Combinatorial bids based multi-robot task allocation method. In IEEE
international conference on robotics and automation (ICRA), IEEE (pp. 1145–1150).
13
21. Ma, H., & Koenig, S. (2016). Optimal target assignment and path finding for teams of agents. In Inter-
national conference on autonomous agents & Multiagent Systems (AAMAS), international foundation
for autonomous agents and multiagent systems (pp. 1144–1152).
22. MacAlpine, P., Price, E., & Stone, P. (2015). SCRAM: Scalable collision-avoiding role assignment
with minimal-makespan for formational positioning. In AAAI conference on artificial intelligence.
23. Matthey, L., Berman, S., & Kumar, V. (2009). Stochastic strategies for a swarm robotic assembly sys-
tem. In International conference on robotics and automation (ICRA), IEEE (pp. 1953–1958).
24. Milam, M. B., Franz, R., Hauser, J. E., & Murray, R. M. (2005). Receding horizon control of vectored
thrust flight experiment. IEE Proceedings-Control Theory and Applications, 152(3), 340–348.
25. Olfati-Saber, R., Fax, J. A., & Murray, R. M. (2007). Consensus and cooperation in networked multi-
agent systems. Proceedings of the IEEE, 95(1), 215–233.
26. Parker, L. E., & Tang, F. (2006). Building multirobot coalitions through automated task solution syn-
thesis. Proceedings of the IEEE, 94(7), 1289–1305.
27. Petchey, O. L., & Gaston, K. J. (2002). Functional diversity (fd), species richness and community com-
position. Ecology Letters, 5(3), 402–411.
28. Prorok, A., Hsieh, M. A., & Kumar, V. (2017). The impact of diversity on optimal control policies for
heterogeneous robot swarms. IEEE Trans Robotics, 33(2), 346–358.
29. Shehory, O., & Kraus, S. (1995). A kernel-oriented model for autonomous-agent coalition-formation
in general environments. In Australian workshop on distributed artificial intelligence (pp. 31–45). Ber-
lin: Springer.
30. Shehory, O., & Kraus, S. (1998). Methods for task allocation via agent coalition formation. Artificial
Intelligence, 101(1), 165–200.
31. Shkurti, F., Xu, A., Meghjani, M., Higuera, J. C. G., Girdhar, Y., Giguere, P., et al. (2012). Multi-
domain monitoring of marine environments using a heterogeneous robot team. In International confer-
ence on intelligent robots and systems (IROS), IEEE/RSJ (pp. 1747–1753).
32. Tokekar, P., Vander Hook, J., Mulla, D., & Isler, V. (2016). Sensor planning for a symbiotic uav and
UGV system for precision agriculture. IEEE Transactions on Robotics, 32(6), 1498–1511.
33. Vail, D., & Veloso, M. (2003). Multi-robot dynamic role assignment and coordination through shared
potential fields. Multi-robot Systems, 2, 87–98.
34. Vig, L., & Adams, J. A. (2006). Multi-robot coalition formation. IEEE Transactions on Robotics,
22(4), 637–649.
35. Werfel, J., Petersen, K., & Nagpal, R. (2014). Designing collective behavior in a termite-inspired robot
construction team. Science, 343(6172), 754–758.
36. Wurman, P. R., D’Andrea, R., & Mountz, M. (2008). Coordinating hundreds of cooperative, autono-
mous vehicles in warehouses. AI Magazine, 29(1), 9.
Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and
institutional affiliations.
13

STRATA: Unified Framework For Task Assignments in Large Teams of Heterogeneous Agents

Uploaded by

Copyright:

Available Formats

STRATA: Unified Framework For Task Assignments in Large Teams of Heterogeneous Agents

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

STRATA: Unified Framework For Task Assignments in Large Teams of Heterogeneous Agents

Uploaded by

Copyright:

Available Formats

Autonomous Agents and Multi-Agent Systems (2020) 34:38

STRATA: unified framework for task assignments in large

Harish Ravichandar1 · Kenneth Shaw1 · Sonia Chernova1

Published online: 13 May 2020

Keywords Multi-agent systems · Task assignment · Heterogeneous agents

We evaluate STRATA using detailed simulations and a capture-the-flag game environment.

First, we introduce the details of our proposed stochastic trait model.

Consider a heterogeneous team of agents. We assume that each agent is a member of a

To capture the natural variability in the capabilities of each species, we main-

𝜇q = [𝜇q(1) , … , 𝜇q(S) ] ∈ ℝSU

𝛴q = diag([𝛴q(1) , … , 𝛴q(S) ]) ∈ ℝSU×SU

where 𝛴q(s) = diag([𝜎s1

species-trait matrix 𝜇Q = [𝜇qT(1) , … , 𝜇qT(S) ]T ∈ ℝS×U

Fig. 2 The task graph associated

⎡ 50 15 20 140⎤ ⎡3 1 1.5 5.6⎤

ẋ (s) (t) = K (s) x(s) (t), ∀s = 1, 2, … , S (9)

3.4 Trait aggregation and distribution

⎡1250 375 500 3500⎤ ⎡1875 625 937.5 3500 ⎤

3.5 Cumulative versus non‑cumulative traits

⎡1250 375 500 3500 25⎤

s.t. X(𝜏)𝜇Q ∈ G(Y ∗ ) (21)

‖VarY (𝜏)‖F ≤ 𝜖var (22)

1. Exact matching: G1 (Y ∗ ) = {𝜇Y |Y ∗ = 𝜇Y }

where ⪯ denotes the element-wise less-than-or-equal-to operator. While goal function G1

Definition 1 Minspecies: In a team described by an average species-trait matrix 𝜇Q, a min-

Proposition 1 The cardinality of eigenspecies (the minspecies corresponding to goal

Proposition 2 The cardinality of coverspecies (the minspecies corresponding to goal

agent distribution X∗.

E2 (𝜏, K (1,..,S) , X(0)) ≤ 𝜖2 (36)

‖VarY (𝜏, K (1,..,S) , X(0))‖2F ≤ 𝜖var (37)

𝜕‖VarY ‖2F 𝜕‖VarY ‖2F 𝜕eK (s) 𝜏 𝜕K (s) 𝜏

𝜕‖VarY ‖2F 𝜕‖VarY ‖2F 𝜕eK (s) 𝜏 𝜕K (s) 𝜏

and W(𝜏) is a M × M matrix with its klth element is given by

6.4 Decentralized online implementation

We note that it is possible to realize a decentralized implementation of the proposed opti-

We evaluate STRATA using two sets of experiments. In both experiments we compare

7.1 Baseline and metrics

7.2.1 Exact trait matching

7.2.2 Minimum trait matching

7.3 Capture the flag

Table 1 Specifications of the # Species # agents # Tasks # Traits Role assignment

Wins Draws Losses

7.3.1 Experimental design and results

⎡1.5 15 90 40⎤ ⎡0.35 5 10 3⎤

We presented STRATA, a unified framework capable of effective task assignments in large

measures of diversity based in a comprehensive class of trait models. It would be inter-

You might also like

3.4 Trait aggregation and distribution

3.5 Cumulative versus non‑cumulative traits

6.4 Decentralized online implementation

7.1 Baseline and metrics

7.2.1 Exact trait matching

7.2.2 Minimum trait matching

7.3 Capture the flag

7.3.1 Experimental design and results