Scribd
Upload
34 views3 pages

CLAI Syllabus

Uploaded by

Adam
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
34 views3 pages

CLAI Syllabus

Uploaded by

Adam
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

Collaborative AI

Spring 2024
Class location: Taub 4
Lecture Meeting time: Wednesday 14:30-16:30
Tutorial: Wednesday 16:30-17:30
Teaching Staff:

Instructor: Sarah Keren [email protected]


Office Hours: By appointment

TA: Itay Segev [email protected]


Office Hours: By appointment

Assignments Andrey Elashkin [email protected]


Office Hours: By appointment

Prerequisites: 236501 (Introduction to Artificial Intelligence).


Co-requisites: NA
Courses Without Credit: NA

Credits: 3

Study hours per week: Lecture-2 Tutorial-1 (Lectures and tutorials will be in English)

Course Goals and Description

Historically, most AI research has dealt with the canonical setting consisting of a single agent
confronting a possibly non-social environment. Among the frameworks that do account for
multiple agents, most efforts have been devoted to adversarial settings and zero-sum games, in
which achievements can be made only at the expense of others. While such settings are
relatively easy to model and benchmark, they are rare in the real world. Instead, recent global
events and technological advancements have given rise to the need for AI agents that can
interact and collaborate with each other and with humans in dynamic and uncertain
environments. In accordance with this agenda, the course will be dedicated to understanding
the science of collaborative and cooperative AI. The focus will be on promoting efficient
behaviors of single AI agents and on promoting cooperative behaviors between agents, even
when their incentives are not fully aligned.
In order to promote effective collaborations between self-interested and possibly partially
informed agents, there is a need to understand their decision-making procedures. The course
will therefore start by covering the main common AI approaches to sequential decision making
under uncertainty for a single agent. This includes both planning, for settings in which the agent
has a complete model of the underlying environment, and reinforcement learning (RL), for

1
settings in which the model of the environment is only partially known. After gaining a deep
understanding of the different elements that influence the decision making of an agent that is
assumed to be alone in the environment, we will explore the different complexities that arise in
the presence of other agents. Focusing on non-adversarial settings and sequential social
dilemmas, we will explore different methods that have been suggested in the literature for
computing joint policies in settings with limited resources, limited and noisy communication,
and individual reward functions.

The course will include learning the theoretical aspects of various single-agent and multi-agent
AI frameworks, as well as practical work with different systems, using Python based
implementations. Students will be required to run various single-agent and multi-agent
planning, RL and deep-RL algorithms on different domains and analyze the performance of the
different approaches.

Learning Outcomes

- Knowledge of various AI frameworks for modeling single-agent and multi-agent settings.


- Understanding the theoretical guarantees and limitations of different AI algorithms for
single and multi-agent planning and RL.
- Acquiring practical experience using AI tools and implementing them in various AI
domains.
- Experience in analyzing the performance of different AI approaches.
- Offering new algorithms for collaborative AI.

Course Content/Topics

The tools and models we will explore are based in a variety of AI fields including automated
planning, sequential decision making under uncertainty, model-based reasoning, game theory,
multi-agent systems, reinforcement learning and more.

The course agenda is as follows.

• Single-agent planning:
o Classical planning.
o Planning in fully observable stochastic environments.
o Planning in partially observable stochastic environments.
• Single-agent reinforcement learning (RL)
o Tabular methods vs. approximate methods.
o Policy gradient vs. value-based methods.
o Model-based vs. model free RL.
• Multi-agent planning:
o Planning in adversarial, cooperative, and collaborative multi-agent settings.
o Communication and resource allocation in multi-agent systems.
• Multi-agent RL:
o Learning in the presence of others.
o Emergent coordination and cooperation.

2
o Efficient communication in noisy partially known environments.
o Automated design of AI environments to promote collaboration.

Assignments and Grading Procedures

The course will require writing a tutorial, a quiz and a final project.

The final project will involve one or more of the following:

• suggesting a novel multi-agent benchmark (or a novel adaptation of an existing


benchmark)
• offering a novel algorithm (or a novel adaptation of an existing approach) for a
cooperative setting
• analyzing the formal properties of the suggested approach
• evaluating the suggested approach on a set of domains

Grading:
20% - tutorial
10% - quiz (magen)
70%-project

80% physical attendance in class is mandatory.

Text book(s) and/or other materials

• Reinforcement Learning: An Introduction (second edition). By Richard S. Sutton and


Andrew G. Barto. 2015. MIT Press.
• Multi-Agent Reinforcement Learning Foundations and Modern Approaches. By Filippos
Christianos and Stefano V. Albrecht. 2024. MIT Press.
• Rules of Encounter: Designing Conventions for Automated Negotiation. By Jeffrey S.
Rosenschein and Gilad Zlotkin. 2014. MIT Press
• Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations by Yoav
Shoham and Kevin Leyton-Brown. Cambridge University Press. 2008
• An Introduction to MultiAgent Systems by Michael Wooldgridge John. Wiley & Sons.
2009.
• A concise introduction to models and methods for automated planning by Hector
Geffner and Blai Bonet. 2013 Synthesis Lectures on Artificial Intelligence and Machine
Learning. Morgan Claypool Publishers.

You might also like