Agent and Multi-Agent Systems:

architectures and reasoning

Introduction to MAS: definitions and implementation of a
plateforme

Khaled Belahcène, Wassila Ouerdane, Nicolas Sabouret

27.02.2023
Agenda

1. Introduction to MAS: definitions and implementation of a plateforme

2. Multiagent simulation : preys and predators
3. Interaction mechanisms : models and implementation
4. Argumentation-based dialogue I: Practical session
5. Argumentation-based dialogue II: what is argumentation?
6. Argumentation-based dialogue III: what is an argumentation-based
protocol?
7. Argumentation-based dialogue IV: Practical session

1
Evaluation

• Project 1 (courses 2 and 3) → 25 %

MultiAgent Based Simulation
Deadline: 10/03/2023 (23h59)

• Project 2 (courses 4 to 7) → 75 %
Argumentation-based Dialogue
Dealdine: 22/04/2023 (23h59)

All practical work will be done in Python

2
Table of contents

1. Motivations: Multi-Agent Systems and AI

2. A brief history of MAS

3. What is an agent?

4. Properties and Definitions

5. The mesa platform for Python

3
Motivations: Multi-Agent
Systems and AI
Motivations

Artificial Intelligence (AI)

”Designing and building machines that do things that would require
intelligence if performed by humans” [ Marvin Minsky].

Source: https://humanoides.fr/robot- appris- jouer- echecs- tout- seul/

4
Motivations

Artificial Intelligence (AI)

”Designing and building machines that do things that would require
intelligence if performed by humans” [ Marvin Minsky].

Source: https://humanoides.fr/robot- appris- jouer- echecs- tout- seul/

AI is not about
7 Building intelligent machines
7 Building machines that think

4
Artificial Intelligence: several definitions!

Reproduce cognition

The automation of activities that we Study mental faculties through the use of com-
associate with human thinking, such putational models (McDermott, 1985)
as decision-making, problem-solving,
learning. (Bellman, 1978) Perceive, Reason and Act (Winston, 1992)

Rationnality

Machines that perform functions that

require intelligence when performed by Intelligent behavior in artifacts (Nilsson, 1998)
humans (Kurzweil, 1990)

Make computer do things at which, at

the moment, people are better (Rich &
Knight, 1991)

Reproduce acts

5
Artificial Intelligence: several definitions!

There is more than one approach to AI. . .

Machine Learning and Neural Networks exist since the 1960s!

6
Artificial Intelligence: several definitions!

There is more than one approach to AI. . .

Machine Learning and Neural Networks exist since the 1960s!

1. Rule based systems explicit representation of knowledge

• Ask a domain expert how he makes the decision
• Model this decision process in a computer program
• Formal models (Knowledge Representation and Reasoning)
• Pros: explainable AI (not always)
• Cons: the bottomless pit of exceptions. . .

6
Artificial Intelligence: several definitions!

There is more than one approach to AI. . .

Machine Learning and Neural Networks exist since the 1960s!

1. Rule based systems explicit representation of knowledge

• Ask a domain expert how he makes the decision
• Model this decision process in a computer program
• Formal models (Knowledge Representation and Reasoning)
• Pros: explainable AI (not always)
• Cons: the bottomless pit of exceptions. . .

2. ML based systems knowledge implicit in data

• Ask an expert what is important to make the decision (features)
• Find a corpus of (possibly annotated) data that cover the situation
• Write a ML program and train it on the data
• Pros: success on many difficult AI tasks
• Cons: not personalized, hardly explainable, not always possible (data
required) 6
Artificial Intelligence: several definitions!

MAS in all this?

3 Reproduce cognition
3 Rational behaviours (but not only)
3 Rule-based models with explicit knowledge (but not only)

Ü Reproduce collective intelligence

Agents achieve things together, the result emerges from their interactions

7
A brief history of MAS
A brief history of Multiagent Systems

• Multiagent Systems appeared in the 1990s

• At the frontier between:
• software engineering,
• distributed computing,
• artificial intelligence.
with some contributions from human sciences!

8
MAS as seen by a software engineer

• Improve the speed, the reusability, the readability, the security of code, . . . ?
• From Object to Service:
• A piece of code, possibly a thread, should be provided with some
machine-understandable interface;
• Allows other code to use it without a priori knowledge of its
operational content.
• Software engineers rely on interaction protocols to define how the different
services should be assembled, how the information has to be exchanged
between them, so as to produce some expected result.

9
MAS as seen by a software engineer

Multiagent system
• Platforms to support the development of such service-oriented software
• Coordination entities capable to interpreting protocols.

The OASIS Web Services Coordination standard (version 1.1, 2007)

What is important (for a software engineer)?

• Make it possible to assemble separate components or threads by designing
well-formed interaction protocols

10
MAS as seen by a distributed systems engineer

• Different processes located on different physical systems

• All connected by some network infrastructure
• Interact with each other using some message passing mechanism

Ü The agents (or processes) can run on one single computer but:
• they have independent memory spaces,
• they have independent runtimes ( They cannot rely on synchronous interactions).

Counter-example (synchronous code)

1 def compute (x ,y , z ) :
2 ... computation here ...
3 return result
4
5 def ca l l ing_function () :
6 ... beginning of code ...
7 res = compute (1 ,3 , ’ hello ’)
8 ... end of code ...

calling function waits for the compute function to end before the code continues 11
MAS as seen by a distributed systems engineer

In Distributed systems, several problems can occur:

even when assuming a perfectly operational infrastructure

• The called process can not be running, or even not be accessible in

the system;
• The called process can not answer, whatever the reason;
• When it answers, there is no guarantee on the delay before it arrives;

programming a piece of software by taking into account the fact that

the other piece of code run in a completely asynchronous manner.

12
MAS as seen by a distributed systems engineer

Consequences
• Don’t wait for the answer: agents must carry on their activity while
a possible answer is computed by another agent;
• Use timeouts: agents must deal with the absence of answers.

What is important (for a DS engineer)?

• Deal with asynchronous interactions and possible errors

and support the distribution of the system among different physical supports
(optional: most MAS architectures run on a single machine)

13
MAS as seen by an Artificial Intelligence engineer

• Artificial Intelligence is about having machine solve problems (through com-

putation) that human beings solve with their intelligence.
• A subfiled of AI: Planning
• design of actions and changes
• solve problems that consist in finding a sequence of actions to go
from a given state to another.
• The STanford Research Institute Problem Solver (STRIPS) is one of the
first (and most famous) planner.

14
MAS as seen by an Artificial Intelligence engineer

1
2 Initial state : At ( A ) , Level ( low ) , BoxAt ( C ) , BananasAt ( B )
3 Goal state : Have ( bananas )
4
5 Actions :
6 // move from X to Y
7 _Move (X , Y ) _
8 Preconditions : At ( X ) , Level ( low )
9 Post conditi ons : not At ( X ) , At ( Y )
10
11 // climb up on the box
12 _ClimbUp ( Location ) _
13 Preconditions : At ( Location ) , BoxAt ( Location ) , Level ( low )
14 Post conditi ons : Level ( high ) , not Level ( low )
15
16 // climb down from the box
17 _ClimbDown ( Location ) _
18 Preconditions : At ( Location ) , BoxAt ( Location ) , Level ( high )
19 Post conditi ons : Level ( low ) , not Level ( high )
20
21 // move monkey and box from X to Y
22 _MoveBox (X , Y ) _
23 Preconditions : At ( X ) , BoxAt ( X ) , Level ( low )
24 Post conditi ons : BoxAt ( Y ) , not BoxAt ( X ) , At ( Y ) , not At ( X )
25
26 // take the bananas
27 _TakeBananas ( Location ) _
28 Preconditions : At ( Location ) , BananasAt ( Location ) , Level ( high )
29 Post conditi ons : Have ( bananas )

A solution: Move(A,C), MoveBox(C,B), ClimbUp(B), TakeBananas(B) (automatically is an NP-hard problem). 15

MAS as seen by an Artificial Intelligence engineer

From Single Intelligenec to Collective Intelligence

• BDI (Belief, Desire, Intention) logic: modeling actions and changes, plan-
ning
programming systems that simulate human decision making
with action selection based on its perception of the environment
• Social Animals Behaviour: solution emerges from the interactions between
the individuals
programming systems that interact with each other to compute a solu-
tion.
e.g. Ant Colony Optimisation algorithm (Marco Dorigo, 1992)

What is important (for an AI engineer)?

• Solve problems in a distributed manner, using interactions and automated
reasoning

16
To be remembered

An agent is :

• a software process (or part of a process, or possibly attached to some

physical components e.g. in robotics. . . )
• with some encapsulated data (i.e. other agents/processes can’t modify
directly this data),
• capable of providing a predefined set of services (its actions)
• and capable of reasoning about its actions and the other agents’ actions
and to coordinate with the other agents so as to participate in a collective
problem solving.

17
To be remembered

• A MAS is a group of agents that share a common environment and that

act in a distributed manner (the agents runtime are supposed to be asyn-
chronous) to solve a problem for a user.
• A MAS always has a predefined purpose, and it is important to define the
expected outcomes, should it be a solution to a concrete problem (e.g. a
path in a TSP problem) or a set of observable properties in a multi-agent
based simulation.

18
Some Practical Application of MAS

MAS Applied to Fault Detection

• SurferLab is a joint research lab formed by Bombardier Transport and the

university of Valencienne, France.
• MAS is used for diagnosis of trains with the Jade platform.
• The multi-agent systems is capable to detect faulty cars so as to correct
them faster1 .

1 Antoine Le Mortellec, Joffrey Clarhaut, Yves Sallez, Thierry Berger, Damien Trentesaux,

Embedded holonic fault diagnosis of complex transportation systems, Engineering Applications of

Artificial Intelligence, Volume 26, Issue 1, 2013 19
Some Practical Application of MAS

MAS Applied to the Study of Electrical Consumption

• The R&D branch of EDF has been working on a MAS for the simulation
of electrical consumption in households. This applied research project is
called SMACH2 .
• The MAS platform is used for the generation of realistic load curves in
answer to prospective studies: market research & pricing, new production
means, new consumption habits, etc.

2 https://www.youtube.com/watch?v=Hyc7XX4vEjw&t=17s

20
Some Practical Application of MAS

MAS Applied to Taxi Fleet Management

• The company Magenta Technology has developed in 2008 a MAS-based

technology for the management of taxi fleets.
• This software solution is now being used by many taxi companies, such
as Green Tomato Cars or Blackberry Cars, two major UK companies in
London, UK.

21
Some Practical Application of MAS

MAS for Crowd Simulation in Movies

• Massive Software was originally developed for use in Peter Jackson’s The
Lord Of The Rings film trilogy.
• Massive has become the leading software for crowd related visual effects
and autonomous character animation.
• It combines MultiAgent Systems with Graphical Design.

22
Some Practical Application of MAS

MASA

23
Subjects–AAMAS 2022/2023

• KnowLedger – A Multi-Agent System Blockchain for Smart Cities Data

• two main autonomous entities, with distinct behaviors. Worker Agents: responsible for
collecting and processing the data that are gathered from heterogeneous sources (data
capture behavior). Ledger Agents: responsible for building and maintaining the ledger.
They receive pre-processed data from worker ones, create transactions, and then mine
the blocks (mining behavior).
• https://www.youtube.com/watch?v=3IIbirYfNAg

• Cellulan World: Interactive platform to learn swarm behaviors

• an educational framework consisting of a swarm of virtual and/or physical agents
(robots) as a game-based learning environment to introduce the underlying rules of
swarm behaviors.

24
Subjects–AAMAS 2022/2023

• An Agent-Based Simulator for Maritime Transport Decarbonisation

• Simulating civil emergency evacuation with Inverse Generative Social Sci-
ence
• Towards Agent-Based Market Analysis of Urban Housing Balance
• Learning a Robust Multiagent Driving Policy for Traffic Congestion Reduc-
tion
• Dynamic Cyber-Physical System Security Planning with Multiple Agents
using Attack Graphs
• Mixed agent-based compartmental (MAC) framework for multi-disease mod-
eling
• Joint Communication and Motion Planning for Cobots.
• ...

25
What is an agent?
PRS architecture [Georgeff and Lansky 87]

• PRS: Procedural Resoning System

Agent
• The environment: characterized by a (partially)
Monitor Sensors Perceptions
From signal to knowledge. . . observable state:
Beliefs
Goals Beliefs • it can be modified,
(desires) (knowledge base)
Environment • some of the variable values can be
Reasoning Reasoner Agents are situated observed by the agents.
Plans Intentions a set of observation functions and a set of
Actions modification functions.
Command
Generator
Actuators Actions & interactions • Agent: sensors (to access the observation func-
From knowledge to signal. . . tion ) + actuators (to access the modification
→ Procedural Loop functions).

27
PRS architecture [Georgeff and Lansky 87]

1 environment_variable = 0
2
3 def o p e r a t i o n _ i n c r e a s e _ v a r i a b l e () :
4 global e n v i r o n m e n t _ v a r i a b l e
5 e n v i r o n m e n t _ v a r i a b l e += 1
6
7 def p e r c e p t i o n _ g e t _ v a r i a b l e () :
8 global e n v i r o n m e n t _ v a r i a b l e
9 return e n v i r o n m e n t _ v a r i a b l e
10
11 def agent ( preferred_val u e ) :
12 v = p e r c e p t i o n _ g e t _ v a r i a b l e () # this is a sensor
13 while (v < preferred_v al ue ) :
14 o p e r a t i o n _ i n c r e a s e _ v a r i a b l e () # this is an actuator
15 v = p e r c e p t i o n _ g e t _ v a r i a b l e ()
16

28
PRS architecture [Georgeff and Lansky 87]

• Procedural Loop: an agent runs continuously in

Agent three steps:
Monitor Sensors Perceptions
From signal to knowledge. . . 1. Perception of the environment
Beliefs
Goals Beliefs 2. Deliberation = selection of the action (i.e.
(desires) (knowledge base)
transforms the result of the perception

Environment
Reasoning Reasoner Agents are situated into internal variables, called beliefs. It
then combines these with a set of goals
Plans Intentions

Actions
and a set of plans so as to decide for some
intention that will be turn into a concrete
Command
Actuators Actions & interactions
Generator action in the environment.)
From knowledge to signal. . .
3. Action on the environment
→ Procedural Loop

29
PRS architecture [Georgeff and Lansky 87]

• Procedural Loop: an agent runs continuously in

Agent three steps:
Monitor Sensors Perceptions
From signal to knowledge. . . 1. Perception of the environment
Beliefs
Goals Beliefs 2. Deliberation = selection of the action (i.e.
(desires) (knowledge base)
transforms the result of the perception

Environment
Reasoning Reasoner Agents are situated into internal variables, called beliefs. It
then combines these with a set of goals
Plans Intentions

Actions
and a set of plans so as to decide for some
intention that will be turn into a concrete
Command
Actuators Actions & interactions
Generator action in the environment.)
From knowledge to signal. . .
3. Action on the environment
→ Procedural Loop

To keep in mind: an agent

• Is situated in an environment: it can perceive and act;

• Continuously performs a procedural loop: perception, deliberation, action.

29
An agent: remark

There is not ”the” definition of an agent

[Russel and Norvig,1995]
”An agent is anything that can be viewed as perceiving its environment
through sensors and acting upon that environment through effectors.”

30
An agent: remark

There is not ”the” definition of an agent

[Russel and Norvig,1995]
”An agent is anything that can be viewed as perceiving its environment
through sensors and acting upon that environment through effectors.”

[Wooldridge & Jennings, 1995]

”An agent is a computer system that is situated in some environment,
and that is capable of autonomous action in this environment in order to
meet its design objectives”
...

30
Agent Models

• Reactive agents: do not make use of any internal variable

directly wire perceptions to a specific action (or set of actions)

31
Agent Models

• Reactive agents: do not make use of any internal variable

directly wire perceptions to a specific action (or set of actions)
• Cognitive agents:
• internal variables and data
• beliefs to store the perceptions memory + decisions based on this
memory

31
Agent Models

• Reactive agents: do not make use of any internal variable

directly wire perceptions to a specific action (or set of actions)
• Cognitive agents:
• internal variables and data
• beliefs to store the perceptions memory + decisions based on this
memory
• must be capable of handling with action failures.

31
Agent Models

• Reactive agents: do not make use of any internal variable

directly wire perceptions to a specific action (or set of actions)
• Cognitive agents:
• internal variables and data
• beliefs to store the perceptions memory + decisions based on this
memory
• must be capable of handling with action failures.
• “explicit planning” is not mandatory (e.g. hard-wired ”plans”: the agent only
checks its current state of beliefs and search for a possible action)
Different levels of “reasoning” about the beliefs: simple deduction,
belief revision, diagnosis, ...

STRIPS example

Move(A,C), MoveBox(C,B), ClimbUp(B), TakeBananas(B)

Note that this plan can fail in a MAS

31
Agent Models

• Reactive agents: do not make use of any internal variable

directly wire perceptions to a specific action (or set of actions)
• Cognitive agents:
• internal variables and data
• beliefs to store the perceptions memory + decisions based on this
memory
• must be capable of handling with action failures.
• “explicit planning” is not mandatory (e.g. hard-wired ”plans”: the agent only
checks its current state of beliefs and search for a possible action)
Different levels of “reasoning” about the beliefs: simple deduction,
belief revision, diagnosis, ...
• “explicit planning”: agents that have explicit goals and that perform
any sort of computation (from simple plan selection in a predefined list
to complex planing algorithms such as SATPLAN) Rational agents.

31
To be remembered

According to the PRS model that is at the core of agent modeling, an agent:

• is situated in the environment (it can perceive and act);

• might have some internal data called beliefs and these are encapsulated
within the agent (they are not accessible to other agents or to the environ-
ment);
• runs with a procedural loop (perceive, decide for an action, act) in an
asynchronous w.r.t other agents;
• in the decision phase, it can do any sort of reasoning, from simple reac-
tion (reactive agents) to more complex reasoning on the beliefs (cognitive
agents);
• agents that have explicit goals and planning procedures in the deliberation
phase are called rational agents.

32
Agent and MAS: Exercices

Open the notebook: IntroductionMAS 2023 Student.ipynb.

Let us begin with two very simple agents in an environment

1 from time import sleep

2
3 class Environment :
4 def act ( self , message ) :
5 print ( message )
6 def perceive ( self ) :
7 pass
8
9 class Agent :
10 def __init__ ( self , name , env ) :
11 self . name = name
12 self . env = env
13 def p ro ce d ur al _l o op ( self ) :
14 while True :
15 self . env . act ( " Agent " + self . name + " says hello ! " )
16 sleep (0.1)
17
18 class Runtime :
19 def __init__ ( self ) :
20 e = Environment ()
21 ( Agent ( " Alice " ,e ) ) . pr oc e du ra l _l oo p ()
22 ( Agent ( " Bob " ,e ) ) . p ro c ed ur a l_ lo op ()
23
24 Runtime ()
25

33
Agent and MAS: Exercices

Questions
• Why does this not behave as a multiagent system?
• What is the problem?

34
Agent and MAS: Exercices

Questions
• Why does this not behave as a multiagent system?
• What is the problem?

Answers

• Running this code produces a continuous output of “Agent Alice says

hello!”.
• It does not behave as a MAS because the agents do not run
asynchronously.
• The reason is that the runtime never leaves the procedural loop of agent
Alice.

34
Agent and MAS: Exercices

Questions
To overcome the above limitation, two solutions can be considered.
The first one is to write a scheduler, i.e. a piece of code that calls the
procedural loops of all agents, one after the other.

• Modify the previous code so that Runtime creates two agents and
calls a single-step procedural loop for all agents.

35
Answers. Version1: with home made scheduler

1
2 from time import sleep
3
4 class Environment :
5 def act ( self , message ) :
6 print ( message )
7 def perceive ( self ) :
8 pass
9
10 class Agent :
11 def __init__ ( self , name , env ) :
12 self . name = name
13 self . env = env
14 def p ro ce d ur al _l o op ( self ) :
15 self . env . act ( " Agent " + self . name + " says hello ! " )
16 sleep (0.1)
17
18 class Runtime :
19 def __init__ ( self ) :
20 e = Environment ()
21 a = Agent ( " Alice " ,e )
22 b = Agent ( " Bob " ,e )
23 while True :
24 a . p ro ce du r al _l oo p ()
25 b . p ro ce du r al _l oo p ()
26
27 Runtime ()
28

36
Answers. Version1: with home made scheduler

1. This MAS verifies a specific property, which is that the procedural loop of
all agents is performed at each time step: all agents run at the same
“speed”.
This is called a synchronous MAS. While the agent has its own runtime,
interleaved with the one of the other agents (which corresponds to the
specification of a MAS) these runtimes are synchronised.
2. The agents procedural loops are always invoked in the same order, which
is not a valid hypothesis in a MAS. Agents should never use that property.

If you want to avoid this, you can simply modify the Runtime class as
follows:

37
Answers. Version1: with home made scheduler – Avoiding the
same order

1
2 from time import sleep
3
4 class Environment :
5 def act ( self , message ) :
6 print ( message )
7 def perceive ( self ) :
8 pass
9
10 class Agent :
11 def __init__ ( self , name , env ) :
12 self . name = name
13 self . env = env
14 def p ro ce d ur al _l o op ( self ) :
15 self . env . act ( " Agent " + self . name + " says hello ! " )
16 sleep (0.1)
17
18 class Runtime :
19 def __init__ ( self ) :
20 e = Environment ()
21 agents = [ Agent ( " Alice " , e ) , Agent ( " Bob " , e ) ]
22 while True :
23 shuffle ( agents )
24 for a in agents :
25 a . pr o ce du r al _l oo p ()
26 Runtime ()
27

Note that this new version still is a synchronous MAS. 38

Agent and MAS: Exercices (cont.)

Questions
The second solution to have agent’s procedural loop interleave is to rely
on the Operating System’s multitasking mechanism (threads): each
agent must be a different thread.

• Modify the initial code so that Agents are threads which Runtime
starts when creating the agents.

39
Answers. Version2: with Threads

1 from time import sleep

2 from threading import Thread
3
4 class Environment :
5 def act ( self , message ) :
6 print ( message )
7
8 def perceive ( self ) :
9 pass
10
11 class Agent ( Thread ) :
12 def __init__ ( self , name , env ) :
13 Thread . __init__ ( self )
14 self . name = name
15 self . env = env
16
17 def run ( self ) :
18 while True :
19 self . pr oc e du ra l_ l oo p ()
20
21 def p ro ce d ur al _l o op ( self ) :
22 self . env . act ( " Agent " + self . name + " says hello ! " )
23 sleep (0.1)
24
25 class Runtime :
26 def __init__ ( self ) :
27 e = Environment ()
28 a = Agent ( " Alice " , e )
29 b = Agent ( " Bob " , e )
30 a . start ()
31 b . start ()
32 40
33 Runtime ()
Agent and MAS: Exercices

Questions
Do we have a multiagent system yet?

41
Agent and MAS: Exercices

Questions
Do we have a multiagent system yet?

Answers

• The agents in the preceding example are not really situated since the
perception phase does nothing.
• They use encapsulated data (the agent’s name), but we can’t really call
these a set of beliefs since they are not connected to the perception.
• Although they have asynchronous runtimes, with a procedural loop, we
cannot really call them agents, even not reactive agents. In order to write
reactive agents, we want the deliberation phase, in the procedural loop, to
depend on the perceptions.

41
Agent and MAS: Exercices

Questions
Write a new multi-agent system in which the environment has some
variable that the agents can perceive.
Write two reactive agents: the first one increases the variable by a
random value when it is even, the other one when it is odd.

42
Answers. Two agents and an environment’s variable

1
2 from time import sleep
3 from random import *
4 from threading import Thread
5
6 class Environment :
7 v = 0
8
9 def increase ( self , name ) :
10 x = randint (1 ,4)
11 print ( " Agent " + name + " increase the value by " + str ( x ) )
12 self . v = self . v + x
13 print ( " --> " + str ( self . v ) )
14
15 def perceive ( self ) :
16 return self . v
17
18 class Agent ( Thread ) :
19 def __init__ ( self , name , env ) :
20 Thread . __init__ ( self )
21 self . name = name
22 self . env = env
23
24 def run ( self ) :
25 while True :
26 self . pr oc e du ra l_ l oo p ()
27
28 def p ro ce d ur al _l o op ( self ) :
29 self . b = self . env . perceive ()
30 self . act ()
31 sleep ( uniform (0.1 ,0.5) )
32 43
Answers. Two agents and an environment’s variable (next)

1
2
3 class AgentOdd ( Agent ) :
4 def act ( self ) :
5 if self . b % 2 != 0:
6 self . env . increase ( self . name )
7
8 class AgentEven ( Agent ) :
9 def act ( self ) :
10 if self . b % 2 == 0:
11 self . env . increase ( self . name )
12
13 class Runtime :
14 def __init__ ( self ) :
15 e = Environment ()
16 a = AgentOdd ( " Alice " , e )
17 b = AgentEven ( " Bob " , e )
18 a . start ()
19 b . start ()
20
21 Runtime ()
22

44
Concurrent Modifications

Writing asynchronous MAS with threads requires to consider possible

concurrent modifications. In the general case, you must consider a
possible interruption of the following sequence of code:

1 self . b = self . env . perceive () # in Agent . p r oc ed u ra l_ lo o p ()

2 if co ndition_on_b : # in AgentX . act ()
3 x = randint (1 ,4) # in Environment . increase () -- 4
lines
4 print ( " Agent " + name + " increase the value by " + str ( x ) )
5 self . v = self . v +
6 print ( " --> " + str ( self . v ) )
7

45
Concurrent Modifications

Writing asynchronous MAS with threads requires to consider possible

concurrent modifications. In the general case, you must consider a
possible interruption of the following sequence of code:

1 self . b = self . env . perceive () # in Agent . p r oc ed u ra l_ lo o p ()

2 if co ndition_on_b : # in AgentX . act ()
3 x = randint (1 ,4) # in Environment . increase () -- 4
lines
4 print ( " Agent " + name + " increase the value by " + str ( x ) )
5 self . v = self . v +
6 print ( " --> " + str ( self . v ) )
7

Questions
• What do you expect to be a problem? Explain why.
• What is a possible solution?

45
Concurrent Modifications: Answers

• the scheduler interrupts the code between line 1 and 2, or even between
line 2 and 3, and passes the “handle” to the other agent:
the first agent will perform an increase operation on a variable that
might not be of the right parity. This is perfectly correct according to a
MAS specification!

46
Concurrent Modifications: Answers

• the scheduler interrupts the code between line 1 and 2, or even between
line 2 and 3, and passes the “handle” to the other agent:
the first agent will perform an increase operation on a variable that
might not be of the right parity. This is perfectly correct according to a
MAS specification!
• the scheduler stops in the middle of the Environment.increase() method.
This is a problem because the calling agent could perfectly assume this
method to be atomic, uninterruptible. This is what you expect when you
perform operations in an environment.
Property of a MAS: actions in the environment must be atomic

46
Concurrent Modifications: Answers

• the scheduler interrupts the code between line 1 and 2, or even between
line 2 and 3, and passes the “handle” to the other agent:
the first agent will perform an increase operation on a variable that
might not be of the right parity. This is perfectly correct according to a
MAS specification!
• the scheduler stops in the middle of the Environment.increase() method.
This is a problem because the calling agent could perfectly assume this
method to be atomic, uninterruptible. This is what you expect when you
perform operations in an environment.
Property of a MAS: actions in the environment must be atomic

1 from threading import Lock

2 l = Lock ()
3 with l :
4 ... concurrent code ...
5

When writing an asynchronous MAS with threads, the environment operations should be protected
by some lock.
46
Properties and Definitions
Properties of an environment [Russel &Norvig 2003]

Properties of an environment [Russel &Norvig 2003]

• Fully observable vs. partially observable – can agents obtain complete and
correct information about the state of the world?
• Deterministic vs. stochastic –Do actions have guaranteed and uniquely
defined effects? Independent episodes (ML algorithms can be used to
build Markov Decision Processes representations of the MAS behavior.)?
• Static vs. dynamic – Does the environment change by processes beyond
agent control (e.g. ant colony simulation in which pheromones dropped by
the agents in the environment need to spread and evaporate with time)?
• Discrete vs. continuous – Is the number of actions and percepts fixed and
finite?

47
Properties of the Agents in the MAS

Autonomous agents
The independence of one agent’s behavior (i.e. action selection) with
respect to some part of the system.

1. At the agent level, pro-action is the first degree of autonomy (i.e.

independently from any specific signal from the environment).
e.g. an agent that plays hide and seek and that begins seeking after
ending a countdown.
2. At the interaction level, autonomy consists in having agents that do not
necessarily accept other agent’s requests (ignoring messages because they
are considered irrelevant; not answering immediately while it could have;
agent answers but refuses the request; etc.).
3. At the MAS level, there is no autonomy: agents are not autonomous
w.r.t. the system. On the contrary, they perform what they are designed
for. They follow well-defined protocols so that the global task is achieved.
Otherwise, there is some misconception in the system.

48
Properties of the Agents in the MAS

Loosely vs tightly coupled?

1. In tightly coupled MAS, the developer has a complete view of the MAS,
of possible actions performed by other agents. In other words, it can use
some of this information to design action mechanism in a more efficient
manner.
2. In a loosely coupled MAS, you must make no assumption on the design
of other agents in the MAS. This means that you must define very robust
behaviours to avoid deadlocks, infinite loops or unnecessary waiting times
in communication.

49
Properties of the Agents in the MAS

Open MAS?

• a specific kind of system in which you assume that agents can enter and
leave the system at any time during the execution.
• requires programmers to design specific mechanism to ensure that
messages are not left unanswered, that some feature in the system will
not disappear when not expected, etc.
• Concrete examples of open MAS are ant colonies or prey-predator
simulations

50
Properties of the Agents in the MAS

Distributed systems?

1. Conceptual level: it implements distribution principles: encapsulated data,

accessed via perception and action methods only, using a procedural loop
that interleaves the perceptions and actions of agents.
Netlogo or Gamma
2. Software level: agents get separated in different threads or processes
(depending on the language and implementation). These threads
generally run on a single computer.
Repast Symphony
3. Physical level: specific code to connect the abstract actions and
perceptions to physical operations, message passing, etc. (e.g. a network
of smart sensors that exchange information with each other)
The Java Agent Development Environment (Jade) , The Robot OS
(ROS)

51
The mesa platform for Python
The mesa platform for Python

Open Notebook: money model Students.ipynb

• Mesa is a Python framework for agent-based modeling.

• The running example will be the implementation of a simple
Agent-based Computational Economics (ACE) model that simulates
distributions of money, income, and wealth in society using
statistical physics3 .

3 https://arxiv.org/abs/cond-mat/0211175

52