Masma – Lab 2

Reactive Agents

The architecture of an agent system is an arrangement of the components which determine

the agents' behaviors and decision-making processes. When an agent has access to a
model of its environment and behaves by reasoning and analyzing such a model its
architecture is deliberative. Conversely, an agent with a reactive architecture behaves
according to external stimuli and whatever data it can gather locally, without having global
information or a complete representation of the world it inhabits.
An agent's decision making process is achieved through a set of task accomplishing
behaviors. These behaviors may be sequential or they may trigger simultaneously
depending on the design of the agents' architecture. Most commonly, the modules
comprising a reactive architecture are arranged into a subsumption hierarchy, where the
agents' behaviors are divided among multiple layers. Lower layers in the hierarchy are able
to inhibit higher layers, therefore the lower the layer, the higher its priority. Higher layers
typically contain abstract behaviors. The basic, formal model of a subsumption hierarchy is
as follows:
- The see function, which represents the agent’s perceptual ability, is assumed to
remain unchanged. However, in implemented subsumption architecture systems,
there is assumed to be quite tight coupling between perception and action – raw
sensor input is not processed or transformed much, and there is certainly no
attempt to transform images to symbolic representations.
- The decision function action is realized through a set of behaviours, together with
an inhibition relation holding between these behaviours. A behaviour is a pair
(c, a ) , where c ⊆ P is a set of percepts called the condition, and a ∈ A is an

action. A behaviour (c, a ) will fire when the environment is in state s ∈ S iff

see(s ) ∈ c . Let Beh = {(c, a ) | c ⊆ P and a} be the set of all such rules.
- Associated with an agent’s set of behaviour rules R ∈ Beh is a binary inhibition
relation on the set of behaviours: π ⊆ R × R . This relation is assumed to be a
total ordering on R (i.e., it is transitive, irreflexive, and antisymmetric). We write
b1πb2 if (b1 , b2 ) ∈ π and read this as "b1 inhibits b2", that is, b1 is lower in the
hierarchy than b2, and will hence get priority over b2. The action function is then
defined as follows:

1
Masma – Lab 2

- Action selection begins by first computing the set fired of all behaviours that fire
(5). Then, each behaviour (c, a ) that fires is checked, to determine whether there
is some other higher priority behaviour that fires. If not, then the action part of the
behaviour, a, is returned as the selected action (8). If no behaviour fires, then the
distinguished action null will be returned, indicating that no action has been
chosen.
The overall time complexity of the subsumption action function is no worse than
( )
O n 2 where n is the larger of the number of behaviours or number of percepts. Thus, even
with the naive algorithm above, decision making is tractable. In practice, the decision making
logic can be encoded into hardware, giving constant decision time. For modern hardware,
this means that an agent can be guaranteed to select an action within nano-seconds. This
computational simplicity is the strength of the subsumption architecture.

Example:

Consider an environment containing a set of explorer agents and their home base. Multiple
resources are scattered throughout the environment. The explorer agents are tasked with
finding the resources and bringing them back to the home base.
The explorer agents are reactive, meaning that they don't have a complete representation of
the environment, thus their decisions are based on what they encounter locally (for example,
if they encounter an obstacle they will avoid it, if they encounter a resource they will pick it
up etc.).

2
 Masma – Lab 2

Resource

Explorer agent
carrying resource

Home base

Explorer agent
searching

The rules of the explorer agents are expressed as reactive actions, as follows:

- R1: if detect an obstacle then change direction

- R2: if carrying resources and at the base then drop resources
- R3: if carrying resources and not at the base then travel toward the base
- R4: if detected a resource then pick it up
- R5: move randomly (for agents with nothing better to do)

The behaviors are arranged in the following hierarchy:

R1 – R2 – R3 – R4 – R5

For implementation details, see the example provided along with this document.

Problem

In the previous example, the resources are randomly and uniformly placed throughout the
environment. A great number of randomly-spawning agents with the above-described
behavior is rather suitable for gathering such resources. However, in practice, the resources
are usually clustered (multiple samples are located in the same region) as shown below:

3
Masma – Lab 2

Random, independently-searching agents would not be effective in tracking grouped

resources, so it makes sense to have the agents collaborate. We model the collaboration
thusly:
- Initially, all grid cells have a weight (an importance) of 0.

- Whenever an agent finds a resource and returns to base, it will reinforce the path
taken (it will add 2 to the weight of each grid cell it encounters on its way back to
the base)

- Whenever a searching agent encounters a reinforced path (a grid cell with a

weight > 0), it will move from that grid cell in a direction away from the base and
it will decrease the importance of that grid cell (it will subtract 1 from the weight of
the grid cell)

We therefore add the following rules to the behavior of a collaborating agent:

- R6: If carrying samples and not at the base then increase path weight by 2 and
travel toward base.
- R7: If detect path with weight > 0 decrease path weight by 1 and travel away from
base.

4
Masma – Lab 2

The behaviors are now arranged in the following hierarchy:

R1 – R2 – R6 – R4 – R7 – R5

Whenever an agent finds a resource in a certain location, it reinforces the path from the
location to the base, since there may be other resources clustered in the immediate
neighborhood of that location. Whenever a non-carrying agent encountes a reinforced path it
tends to follow it in hope that it may find a resource cluster. At the same time, the importance
of that path decreases as more and more searching agents travel along it.

The following image illustrates the weights of each path. The darker the shade of red of a
grid cell, the greater the chance that travelling on it will lead to more resources.

Tasks:
1. Study the example application provided with the documentation.

2. Modify the application so that the resources are clustered (grouped together) and not
evenly spread throughout the entire grid.

3. Implement the collaborative agent behavior as described above.

4. Measure the performance in terms of the total number of moves by all agents until all
resources are collected, and compare the results in the following cases:

- Agents are not collaborative and the resources are uniformly distributed (the
default in our example)

- Agents are not collaborative and the resources are grouped together (for
instance, the are only generated in the upper-left quadrant of the grid)

- Agents collaborate and the resources are grouped together