Models

In many cases, agents need access to LLM model services such as OpenAI, Azure OpenAI,
or local models. Since there are many different providers with different APIs, autogen-core
implements a protocol for model clients and autogen-ext implements a set of model clients
for popular model services. AgentChat can use these model clients to interact with model
services.

This section provides a quick overview of available model clients. For more details on how to
use them directly, please refer to Model Clients in the Core API documentation.

Note

See ChatCompletionCache for a caching wrapper to use with the following clients.

OpenAI
To access OpenAI models, install the openai extension, which allows you to use the
OpenAIChatCompletionClient.

pip install "autogen-ext[openai]"

You will also need to obtain an API key from OpenAI.

from autogen_ext.models.openai import OpenAIChatCompletionClient

openai_model_client = OpenAIChatCompletionClient(
model="gpt-4o-2024-08-06",
# api_key="sk-...", # Optional if you have an OPENAI_API_KEY environment variable set.
)
To test the model client, you can use the following code:

from autogen_core.models import UserMessage

result = await openai_model_client.create([UserMessage(content="What is the capital of

France?", source="user")])
print(result)
CreateResult(finish_reason='stop', content='The capital of France is Paris.',
usage=RequestUsage(prompt_tokens=15, completion_tokens=7), cached=False,
logprobs=None)
Note

You can use this client with models hosted on OpenAI-compatible endpoints, however, we
have not tested this functionality. See OpenAIChatCompletionClient for more information.

Azure OpenAI
Similarly, install the azure and openai extensions to use the
AzureOpenAIChatCompletionClient.

pip install "autogen-ext[openai,azure]"

To use the client, you need to provide your deployment id, Azure Cognitive Services
endpoint, api version, and model capabilities. For authentication, you can either provide an
API key or an Azure Active Directory (AAD) token credential.

The following code snippet shows how to use AAD authentication. The identity used must be
assigned the Cognitive Services OpenAI User role.

from autogen_ext.models.openai import AzureOpenAIChatCompletionClient

from azure.identity import DefaultAzureCredential, get_bearer_token_provider

# Create the token provider

token_provider = get_bearer_token_provider(DefaultAzureCredential(),
"https://cognitiveservices.azure.com/.default")

az_model_client = AzureOpenAIChatCompletionClient(
azure_deployment="{your-azure-deployment}",
model="{model-name, such as gpt-4o}",
api_version="2024-06-01",
azure_endpoint="https://{your-custom-endpoint}.openai.azure.com/",
azure_ad_token_provider=token_provider, # Optional if you choose key-based
authentication.
# api_key="sk-...", # For key-based authentication.
)
See here for how to use the Azure client directly or for more information.

Azure AI Foundry
Azure AI Foundry (previously known as Azure AI Studio) offers models hosted on Azure. To
use those models, you use the AzureAIChatCompletionClient.

You need to install the azure extra to use this client.

pip install "autogen-ext[azure]"

Below is an example of using this client with the Phi-4 model from GitHub Marketplace.

import os

from autogen_core.models import UserMessage

from autogen_ext.models.azure import AzureAIChatCompletionClient
from azure.core.credentials import AzureKeyCredential

client = AzureAIChatCompletionClient(
model="Phi-4",
endpoint="https://models.inference.ai.azure.com",
# To authenticate with the model you will need to generate a personal access token (PAT)
in your GitHub settings.
# Create your PAT token by following instructions here:
https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-
your-personal-access-tokens
 credential=AzureKeyCredential(os.environ["GITHUB_TOKEN"]),
model_info={
"json_output": False,
"function_calling": False,
"vision": False,
"family": "unknown",
},
)

result = await client.create([UserMessage(content="What is the capital of France?",

source="user")])
print(result)
finish_reason='stop' content='The capital of France is Paris.'
usage=RequestUsage(prompt_tokens=14, completion_tokens=8) cached=False
logprobs=None
Ollama
Ollama is a local model server that can run models locally on your machine.

Currently, we recommend using the OpenAIChatCompletionClient to interact with Ollama

server.

Note

Small local models are typically not as capable as larger models on the cloud. For some
tasks they may not perform as well and the output may be suprising.

from autogen_core.models import UserMessage

from autogen_ext.models.openai import OpenAIChatCompletionClient

model_client = OpenAIChatCompletionClient(
model="llama3.2:latest",
base_url="http://localhost:11434/v1",
api_key="placeholder",
model_info={
"vision": False,
"function_calling": True,
"json_output": False,
"family": "unknown",
},
)

response = await model_client.create([UserMessage(content="What is the capital of

France?", source="user")])
print(response)
finish_reason='unknown' content='The capital of France is Paris.'
usage=RequestUsage(prompt
Messages
In AutoGen AgentChat, messages facilitate communication and information exchange with
other agents, orchestrators, and applications. AgentChat supports various message types,
each designed for specific purposes.

Types of Messages
At a high level, messages in AgentChat can be categorized into two types: agent-agent
messages and an agent’s internal events and messages.

Agent-Agent Messages
AgentChat supports many message types for agent-to-agent communication. They belong to
the union type ChatMessage. This message type allows both text and multimodal
communication and subsumes other message types, such as TextMessage or
MultiModalMessage.

For example, the following code snippet demonstrates how to create a text message, which
accepts a string content and a string source:

from autogen_agentchat.messages import TextMessage

text_message = TextMessage(content="Hello, world!", source="User")

Similarly, the following code snippet demonstrates how to create a multimodal message,
which accepts a list of strings or Image objects:

from io import BytesIO

import requests
from autogen_agentchat.messages import MultiModalMessage
from autogen_core import Image as AGImage
from PIL import Image

pil_image = Image.open(BytesIO(requests.get("https://picsum.photos/300/200").content))
img = AGImage(pil_image)
multi_modal_message = MultiModalMessage(content=["Can you describe the content of this
image?", img], source="User")
img

The TextMessage and MultiModalMessage we have created can be passed to agents

directly via the on_messages method, or as tasks given to a team run() method. Messages
are also used in the responses of an agent. We will explain these in more detail in Agents
and Teams.

Internal Events
AgentChat also supports the concept of events - messages that are internal to an agent.
These messages are used to communicate events and information on actions within the
agent itself, and belong to the union type AgentEvent.
Examples of these include ToolCallRequestEvent, which indicates that a request was made
to call a tool, and ToolCallExecutionEvent, which contains the results of tool calls.

Typically, events are created by the agent itself and are contained in the inner_messages
field of the Response returned from on_messages. If you are building a custom agent and
have events that you want to communicate to other entities (e.g., a UI), you can include
these in the inner_messages field of the Response. We will show examples of this in
Custom Agents.

You can read about the full set of messages supported in AgentChat in the messages
module.

Agents
AutoGen AgentChat provides a set of preset Agents, each with variations in how an agent
might respond to messages. All agents share the following attributes and methods:

name: The unique name of the agent.

description: The description of the agent in text.

on_messages(): Send the agent a sequence of ChatMessage get a Response. It is

important to note that agents are expected to be stateful and this method is expected to be
called with new messages, not the complete history.

on_messages_stream(): Same as on_messages() but returns an iterator of AgentEvent or

ChatMessage followed by a Response as the last item.

on_reset(): Reset the agent to its initial state.

run() and run_stream(): convenience methods that call on_messages() and

on_messages_stream() respectively but offer the same interface as Teams.

See autogen_agentchat.messages for more information on AgentChat message types.

Assistant Agent
AssistantAgent is a built-in agent that uses a language model and has the ability to use
tools.

from autogen_agentchat.agents import AssistantAgent

from autogen_agentchat.messages import TextMessage
from autogen_agentchat.ui import Console
from autogen_core import CancellationToken
from autogen_ext.models.openai import OpenAIChatCompletionClient
# Define a tool that searches the web for information.
async def web_search(query: str) -> str:
"""Find information on the web"""
return "AutoGen is a programming framework for building multi-agent applications."
# Create an agent that uses the OpenAI GPT-4o model.
model_client = OpenAIChatCompletionClient(
model="gpt-4o",
# api_key="YOUR_API_KEY",
)
agent = AssistantAgent(
name="assistant",
model_client=model_client,
tools=[web_search],
system_message="Use tools to solve tasks.",
)
Getting Responses
We can use the on_messages() method to get the agent response to a given message.

async def assistant_run() -> None:

response = await agent.on_messages(
[TextMessage(content="Find information on AutoGen", source="user")],
cancellation_token=CancellationToken(),
)
print(response.inner_messages)
print(response.chat_message)

# Use asyncio.run(assistant_run()) when running in a script.

await assistant_run()
[ToolCallRequestEvent(source='assistant',
models_usage=RequestUsage(prompt_tokens=598, completion_tokens=16),
content=[FunctionCall(id='call_9UWYM1CgE3ZbnJcSJavNDB79',
arguments='{"query":"AutoGen"}', name='web_search')], type='ToolCallRequestEvent'),
ToolCallExecutionEvent(source='assistant', models_usage=None,
content=[FunctionExecutionResult(content='AutoGen is a programming framework for
building multi-agent applications.', call_id='call_9UWYM1CgE3ZbnJcSJavNDB79',
is_error=False)], type='ToolCallExecutionEvent')]
source='assistant' models_usage=None content='AutoGen is a programming framework for
building multi-agent applications.' type='ToolCallSummaryMessage'
The call to the on_messages() method returns a Response that contains the agent’s final
response in the chat_message attribute, as well as a list of inner messages in the
inner_messages attribute, which stores the agent’s “thought process” that led to the final
response.

Note

It is important to note that on_messages() will update the internal state of the agent – it will
add the messages to the agent’s history. So you should call this method with new
messages. You should not repeatedly call this method with the same messages or the
complete history.
Note

Unlike in v0.2 AgentChat, the tools are executed by the same agent directly within the same
call to on_messages(). By default, the agent will return the result of the tool call as the final
response.

You can also call the run() method, which is a convenience method that calls
on_messages(). It follows the same interface as Teams and returns a TaskResult object.

Multi-Modal Input
The AssistantAgent can handle multi-modal input by providing the input as a
MultiModalMessage.

from io import BytesIO

import PIL
import requests
from autogen_agentchat.messages import MultiModalMessage
from autogen_core import Image

# Create a multi-modal message with random image and text.

pil_image =
PIL.Image.open(BytesIO(requests.get("https://picsum.photos/300/200").content))
img = Image(pil_image)
multi_modal_message = MultiModalMessage(content=["Can you describe the content of this
image?", img], source="user")
img

# Use asyncio.run(...) when running in a script.

response = await agent.on_messages([multi_modal_message], CancellationToken())
print(response.chat_message.content)
The image depicts a vintage car, likely from the 1930s or 1940s, with a sleek, classic design.
The car seems to be customized or well-maintained, as indicated by its shiny exterior and
lowered stance. It has a prominent grille and round headlights. There's a license plate on the
front with the text "FARMER BOY." The setting appears to be a street with old-style buildings
in the background, suggesting a historical or retro theme.
You can also use MultiModalMessage as a task input to the run() method.

Streaming Messages
We can also stream each message as it is generated by the agent by using the
on_messages_stream() method, and use Console to print the messages as they appear to
the console.

async def assistant_run_stream() -> None:

# Option 1: read each message from the stream (as shown in the previous example).
# async for message in agent.on_messages_stream(
# [TextMessage(content="Find information on AutoGen", source="user")],
# cancellation_token=CancellationToken(),
 # ):
# print(message)

# Option 2: use Console to print all messages as they appear.

await Console(
agent.on_messages_stream(
[TextMessage(content="Find information on AutoGen", source="user")],
cancellation_token=CancellationToken(),
),
output_stats=True, # Enable stats printing.
)

# Use asyncio.run(assistant_run_stream()) when running in a script.

await assistant_run_stream()
---------- assistant ----------
[FunctionCall(id='call_fSp5iTGVm2FKw5NIvfECSqNd', arguments='{"query":"AutoGen
information"}', name='web_search')]
[Prompt tokens: 61, Completion tokens: 16]
---------- assistant ----------
[FunctionExecutionResult(content='AutoGen is a programming framework for building multi-
agent applications.', call_id='call_fSp5iTGVm2FKw5NIvfECSqNd')]
---------- assistant ----------
AutoGen is a programming framework designed for building multi-agent applications. If you
need more detailed information or specific aspects about AutoGen, feel free to ask!
[Prompt tokens: 93, Completion tokens: 32]
---------- Summary ----------
Number of inner messages: 2
Total prompt tokens: 154
Total completion tokens: 48
Duration: 4.30 seconds
The on_messages_stream() method returns an asynchronous generator that yields each
inner message generated by the agent, with the final item being the response message in
the chat_message attribute.

From the messages, you can observe that the assistant agent utilized the web_search tool
to gather information and responded based on the search results.

You can also use run_stream() to get the same streaming behavior as
on_messages_stream(). It follows the same interface as Teams.

Using Tools
Large Language Models (LLMs) are typically limited to generating text or code responses.
However, many complex tasks benefit from the ability to use external tools that perform
specific actions, such as fetching data from APIs or databases.

To address this limitation, modern LLMs can now accept a list of available tool schemas
(descriptions of tools and their arguments) and generate a tool call message. This capability
is known as Tool Calling or Function Calling and is becoming a popular pattern in building
intelligent agent-based applications. Refer to the documentation from OpenAI and Anthropic
for more information about tool calling in LLMs.

In AgentChat, the AssistantAgent can use tools to perform specific actions. The web_search
tool is one such tool that allows the assistant agent to search the web for information. A
custom tool can be a Python function or a subclass of the BaseTool.

By default, when AssistantAgent executes a tool, it will return the tool’s output as a string in
ToolCallSummaryMessage in its response. If your tool does not return a well-formed string in
natural language, you can add a reflection step to have the model summarize the tool’s
output, by setting the reflect_on_tool_use=True parameter in the AssistantAgent constructor.

Langchain Tools
In addition to custom tools, you can also use tools from the Langchain library by wrapping
them in LangChainToolAdapter.

import pandas as pd
from autogen_ext.tools.langchain import LangChainToolAdapter
from langchain_experimental.tools.python.tool import PythonAstREPLTool

df = pd.read_csv("https://raw.githubusercontent.com/pandas-dev/pandas/main/doc/data/
titanic.csv")
tool = LangChainToolAdapter(PythonAstREPLTool(locals={"df": df}))
model_client = OpenAIChatCompletionClient(model="gpt-4o")
agent = AssistantAgent(
"assistant", tools=[tool], model_client=model_client, system_message="Use the `df`
variable to access the dataset."
)
await Console(
agent.on_messages_stream(
[TextMessage(content="What's the average age of the passengers?", source="user")],
CancellationToken()
),
output_stats=True,
)
---------- assistant ----------
[FunctionCall(id='call_BEYRkf53nBS1G2uG60wHP0zf',
arguments='{"query":"df[\'Age\'].mean()"}', name='python_repl_ast')]
[Prompt tokens: 111, Completion tokens: 22]
---------- assistant ----------
[FunctionExecutionResult(content='29.69911764705882',
call_id='call_BEYRkf53nBS1G2uG60wHP0zf')]
---------- assistant ----------
29.69911764705882
---------- Summary ----------
Number of inner messages: 2
Total prompt tokens: 111
Total completion tokens: 22
Duration: 0.62 seconds
Response(chat_message=ToolCallSummaryMessage(source='assistant',
models_usage=None, content='29.69911764705882', type='ToolCallSummaryMessage'),
inner_messages=[ToolCallRequestEvent(source='assistant',
models_usage=RequestUsage(prompt_tokens=111, completion_tokens=22),
content=[FunctionCall(id='call_BEYRkf53nBS1G2uG60wHP0zf',
arguments='{"query":"df[\'Age\'].mean()"}', name='python_repl_ast')],
type='ToolCallRequestEvent'), ToolCallExecutionEvent(source='assistant',
models_usage=None, content=[FunctionExecutionResult(content='29.69911764705882',
call_id='call_BEYRkf53nBS1G2uG60wHP0zf')], type='ToolCallExecutionEvent')])
Parallel Tool Calls
Some models support parallel tool calls, which can be useful for tasks that require multiple
tools to be called simultaneously. By default, if the model client produces multiple tool calls,
AssistantAgent will call the tools in parallel.

You may want to disable parallel tool calls when the tools have side effects that may
interfere with each other, or, when agent behavior needs to be consistent across different
models. This should be done at the model client level.

For OpenAIChatCompletionClient and AzureOpenAIChatCompletionClient, set

parallel_tool_calls=False to disable parallel tool calls.

model_client_no_parallel_tool_call = OpenAIChatCompletionClient(
model="gpt-4o",
parallel_tool_calls=False, # type: ignore
)
agent_no_parallel_tool_call = AssistantAgent(
name="assistant",
model_client=model_client_no_parallel_tool_call,
tools=[web_search],
system_message="Use tools to solve tasks.",
)
Structured Output
Structured output allows models to return structured JSON text with pre-defined schema
provided by the application. Different from JSON-mode, the schema can be provided as a
Pydantic BaseModel class, which can also be used to validate the output.

Note

Structured output is only available for models that support it. It also requires the model client
to support structured output as well. Currently, the OpenAIChatCompletionClient and
AzureOpenAIChatCompletionClient support structured output.

Structured output is also useful for incorporating Chain-of-Thought reasoning in the agent’s
responses. See the example below for how to use structured output with the assistant agent.

from typing import Literal

from pydantic import BaseModel

# The response format for the agent as a Pydantic base model.

class AgentResponse(BaseModel):
thoughts: str
response: Literal["happy", "sad", "neutral"]

# Create an agent that uses the OpenAI GPT-4o model with the custom response format.
model_client = OpenAIChatCompletionClient(
model="gpt-4o",
response_format=AgentResponse, # type: ignore
)
agent = AssistantAgent(
"assistant",
model_client=model_client,
system_message="Categorize the input as happy, sad, or neutral following the JSON
format.",
)

await Console(agent.run_stream(task="I am happy."))

---------- user ----------
I am happy.
---------- assistant ----------
{"thoughts":"The user explicitly states that they are happy.","response":"happy"}
TaskResult(messages=[TextMessage(source='user', models_usage=None, content='I am
happy.', type='TextMessage'), TextMessage(source='assistant',
models_usage=RequestUsage(prompt_tokens=89, completion_tokens=18),
content='{"thoughts":"The user explicitly states that they are happy.","response":"happy"}',
type='TextMessage')], stop_reason=None)
Streaming Tokens
You can stream the tokens generated by the model client by setting
model_client_stream=True. This will cause the agent to yield
ModelClientStreamingChunkEvent messages in on_messages_stream() and run_stream().

The underlying model API must support streaming tokens for this to work. Please check with
your model provider to see if this is supported.

model_client = OpenAIChatCompletionClient(model="gpt-4o")

streaming_assistant = AssistantAgent(
name="assistant",
model_client=model_client,
system_message="You are a helpful assistant.",
model_client_stream=True, # Enable streaming tokens.
)
# Use an async function and asyncio.run() in a script.
async for message in streaming_assistant.on_messages_stream( # type: ignore
[TextMessage(content="Name two cities in South America", source="user")],
cancellation_token=CancellationToken(),
):
print(message)
source='assistant' models_usage=None content='Two'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' cities'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' in'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' South'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' America'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' are'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' Buenos'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' Aires'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' in'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' Argentina'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' and'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' São'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' Paulo'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' in'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' Brazil'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content='.' type='ModelClientStreamingChunkEvent'
Response(chat_message=TextMessage(source='assistant',
models_usage=RequestUsage(prompt_tokens=0, completion_tokens=0), content='Two
cities in South America are Buenos Aires in Argentina and São Paulo in Brazil.',
type='TextMessage'), inner_messages=[])
You can see the streaming chunks in the output above. The chunks are generated by the
model client and are yielded by the agent as they are received. The final response, the
concatenation of all the chunks, is yielded right after the last chunk.

Similarly, run_stream() will also yield the same streaming chunks, followed by a full text
message right after the last chunk.
async for message in streaming_assistant.run_stream(task="Name two cities in North
America."): # type: ignore
print(message)
source='user' models_usage=None content='Name two cities in North America.'
type='TextMessage'
source='assistant' models_usage=None content='Two'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' cities'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' in'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' North'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' America'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' are'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' New'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' York'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' City'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' in'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' the'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' United'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' States'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' and'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' Toronto'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' in'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content=' Canada'
type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=None content='.' type='ModelClientStreamingChunkEvent'
source='assistant' models_usage=RequestUsage(prompt_tokens=0, completion_tokens=0)
content='Two cities in North America are New York City in the United States and Toronto in
Canada.' type='TextMessage'
TaskResult(messages=[TextMessage(source='user', models_usage=None, content='Name
two cities in North America.', type='TextMessage'), TextMessage(source='assistant',
models_usage=RequestUsage(prompt_tokens=0, completion_tokens=0), content='Two
cities in North America are New York City in the United States and Toronto in Canada.',
type='TextMessage')], stop_reason=None)
Using Model Context
AssistantAgent has a model_context parameter that can be used to pass in a
ChatCompletionContext object. This allows the agent to use different model contexts, such
as BufferedChatCompletionContext to limit the context sent to the model.

By default, AssistantAgent uses the UnboundedChatCompletionContext which sends the full

conversation history to the model. To limit the context to the last n messages, you can use
the BufferedChatCompletionContext.

from autogen_core.model_context import BufferedChatCompletionContext

# Create an agent that uses only the last 5 messages in the context to generate responses.
agent = AssistantAgent(
name="assistant",
model_client=model_client,
tools=[web_search],
system_message="Use tools to solve tasks.",
model_context=BufferedChatCompletionContext(buffer_size=5), # Only use the last 5
messages in the context.
)
Other Preset Agents
The following preset agents are available:

UserProxyAgent: An agent that takes user input returns it as responses.

CodeExecutorAgent: An agent that can execute code.

OpenAIAssistantAgent: An agent that is backed by an OpenAI Assistant, with ability to use

custom tools.

MultimodalWebSurfer: A multi-modal agent that can search the web and visit web pages for
information.

FileSurfer: An agent that can search and browse local files for information.

VideoSurfer: An agent that can watch videos for information.

Teams
In this section you’ll learn how to create a multi-agent team (or simply team) using AutoGen.
A team is a group of agents that work together to achieve a common goal.

We’ll first show you how to create and run a team. We’ll then explain how to observe the
team’s behavior, which is crucial for debugging and understanding the team’s performance,
and common operations to control the team’s behavior.

Note

When should you use a team? Teams are for complex tasks that require collaboration and
diverse expertise. However, they also demand more scaffolding to steer compared to single
agents. While AutoGen simplifies the process of working with teams, start with a single
agent for simpler tasks, and transition to a multi-agent team when a single agent proves
inadequate. Ensure that you have optimized your single agent with the appropriate tools and
instructions before moving to a team-based approach.

Creating a Team
RoundRobinGroupChat is a simple yet effective team configuration where all agents share
the same context and take turns responding in a round-robin fashion. Each agent, during its
turn, broadcasts its response to all other agents, ensuring that the entire team maintains a
consistent context.

We will begin by creating a team with two AssistantAgent and a TextMentionTermination

condition that stops the team when a specific word is detected in the agent’s response.

The two-agent team implements the reflection pattern, a multi-agent design pattern where a
critic agent evaluates the responses of a primary agent. Learn more about the reflection
pattern using the Core API.

import asyncio

from autogen_agentchat.agents import AssistantAgent

from autogen_agentchat.base import TaskResult
from autogen_agentchat.conditions import ExternalTermination, TextMentionTermination
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.ui import Console
from autogen_core import CancellationToken
from autogen_ext.models.openai import OpenAIChatCompletionClient

# Create an OpenAI model client.

model_client = OpenAIChatCompletionClient(
model="gpt-4o-2024-08-06",
# api_key="sk-...", # Optional if you have an OPENAI_API_KEY env variable set.
)

# Create the primary agent.

primary_agent = AssistantAgent(
 "primary",
model_client=model_client,
system_message="You are a helpful AI assistant.",
)

# Create the critic agent.

critic_agent = AssistantAgent(
"critic",
model_client=model_client,
system_message="Provide constructive feedback. Respond with 'APPROVE' to when
your feedbacks are addressed.",
)

# Define a termination condition that stops the task if the critic approves.
text_termination = TextMentionTermination("APPROVE")

# Create a team with the primary and critic agents.

team = RoundRobinGroupChat([primary_agent, critic_agent],
termination_condition=text_termination)
Running a Team
Let’s call the run() method to start the team with a task.

# Use `asyncio.run(...)` when running in a script.

result = await team.run(task="Write a short poem about the fall season.")
print(result)
TaskResult(messages=[TextMessage(source='user', models_usage=None, content='Write a
short poem about the fall season.', type='TextMessage'), TextMessage(source='primary',
models_usage=RequestUsage(prompt_tokens=28, completion_tokens=109),
content="Leaves of amber, gold, and rust, \nDance upon the gentle gust. \nCrisp air
whispers tales of old, \nAs daylight wanes, the night grows bold. \n\nPumpkin patch and
apple treats, \nLaughter in the street repeats. \nSweaters warm and fires aglow, \nIt's time
for nature's vibrant show. \n\nThe harvest moon ascends the sky, \nWhile geese in
formation start to fly. \nAutumn speaks in colors bright, \nA fleeting grace, a pure delight. ",
type='TextMessage'), TextMessage(source='critic',
models_usage=RequestUsage(prompt_tokens=154, completion_tokens=200),
content='Your poem beautifully captures the essence of the fall season with vivid imagery
and a rhythmic flow. The use of descriptive language like "amber, gold, and rust" effectively
paints a visual picture of the changing leaves. Phrases such as "crisp air whispers tales of
old" and "daylight wanes, the night grows bold" add a poetic touch by incorporating seasonal
characteristics.\n\nHowever, you might consider exploring other sensory details to deepen
the reader\'s immersion. For example, mentioning the sound of crunching leaves underfoot
or the scent of cinnamon and spices in the air could enhance the sensory experience.\n\
nAdditionally, while the mention of "pumpkin patch and apple treats" is evocative of fall,
expanding on these elements or including more personal experiences or emotions
associated with the season might make the poem more relatable and engaging.\n\nOverall,
you\'ve crafted a lovely poem that celebrates the beauty and traditions of autumn with grace
and warmth. A few tweaks to include multisensory details could elevate it even further.',
type='TextMessage'), TextMessage(source='primary',
models_usage=RequestUsage(prompt_tokens=347, completion_tokens=178),
content="Thank you for the thoughtful feedback. Here's a revised version of the poem with
additional sensory details:\n\nLeaves of amber, gold, and rust, \nDance upon the gentle
gust. \nCrisp air whispers tales of old, \nAs daylight wanes, the night grows bold. \n\
nCrunch beneath the wandering feet, \nA melody of autumn's beat. \nCinnamon and spices
blend, \nIn every breeze, nostalgia sends. \n\nPumpkin patch and apple treats, \nLaughter
in the street repeats. \nSweaters warm and fires aglow, \nIt's time for nature's vibrant show.
\n\nThe harvest moon ascends the sky, \nWhile geese in formation start to fly. \nAutumn
speaks in colors bright, \nA fleeting grace, a pure delight. \n\nI hope this version resonates
even more with the spirit of fall. Thank you again for your suggestions!",
type='TextMessage'), TextMessage(source='critic',
models_usage=RequestUsage(prompt_tokens=542, completion_tokens=3),
content='APPROVE', type='TextMessage')], stop_reason="Text 'APPROVE' mentioned")
The team runs the agents until the termination condition was met. In this case, the team ran
agents following a round-robin order until the the termination condition was met when the
word “APPROVE” was detected in the agent’s response. When the team stops, it returns a
TaskResult object with all the messages produced by the agents in the team.

Observing a Team
Similar to the agent’s on_messages_stream() method, you can stream the team’s messages
while it is running by calling the run_stream() method. This method returns a generator that
yields messages produced by the agents in the team as they are generated, with the final
item being the TaskResult object.

# When running inside a script, use a async main function and call it from `asyncio.run(...)`.
await team.reset() # Reset the team for a new task.
async for message in team.run_stream(task="Write a short poem about the fall season."): #
type: ignore
if isinstance(message, TaskResult):
print("Stop Reason:", message.stop_reason)
else:
print(message)
source='user' models_usage=None content='Write a short poem about the fall season.'
type='TextMessage'
source='primary' models_usage=RequestUsage(prompt_tokens=28,
completion_tokens=105) content="Leaves descend in golden dance, \nWhispering secrets
as they fall, \nCrisp air brings a gentle trance, \nHeralding Autumn's call. \n\nPumpkins
glow with orange light, \nFields wear a cloak of amber hue, \nDays retreat to longer night, \
nSkies shift to deeper blue. \n\nWinds carry scents of earth and pine, \nSweaters wrap us,
warm and tight, \nNature's canvas, bold design, \nIn Fall's embrace, we find delight. "
type='TextMessage'
source='critic' models_usage=RequestUsage(prompt_tokens=150, completion_tokens=226)
content='Your poem beautifully captures the essence of fall with vivid imagery and a
soothing rhythm. The imagery of leaves descending, pumpkins glowing, and fields cloaked in
amber hues effectively paints a picture of the autumn season. The use of contrasting
elements like "Days retreat to longer night" and "Sweaters wrap us, warm and tight" provides
a nice balance between the cold and warmth associated with the season. Additionally, the
personification of autumn through phrases like "Autumn\'s call" and "Nature\'s canvas, bold
design" adds depth to the depiction of fall.\n\nTo enhance the poem further, you might
consider focusing on the soundscape of fall, such as the rustling of leaves or the distant call
of migrating birds, to engage readers\' auditory senses. Also, varying the line lengths slightly
could add a dynamic flow to the reading experience.\n\nOverall, your poem is engaging and
effectively encapsulates the beauty and transition of fall. With a few adjustments to explore
other sensory details, it could become even more immersive. \n\nIf you incorporate some of
these suggestions or find another way to expand the sensory experience, please share your
update!' type='TextMessage'
source='primary' models_usage=RequestUsage(prompt_tokens=369,
completion_tokens=143) content="Thank you for the thoughtful critique and suggestions.
Here's a revised version of the poem with added attention to auditory senses and varied line
lengths:\n\nLeaves descend in golden dance, \nWhisper secrets in their fall, \nBreezes hum
a gentle trance, \nHeralding Autumn's call. \n\nPumpkins glow with orange light, \nAmber
fields beneath wide skies, \nDays retreat to longer night, \nChill winds and distant cries. \n\
nRustling whispers of the trees, \nSweaters wrap us, snug and tight, \nNature's canvas,
bold and free, \nIn Fall's embrace, pure delight. \n\nI appreciate your feedback and hope
this version better captures the sensory richness of the season!" type='TextMessage'
source='critic' models_usage=RequestUsage(prompt_tokens=529, completion_tokens=160)
content='Your revised poem is a beautiful enhancement of the original. By incorporating
auditory elements such as "Breezes hum" and "Rustling whispers of the trees," you\'ve
added an engaging soundscape that draws the reader deeper into the experience of fall.
The varied line lengths work well to create a more dynamic rhythm throughout the poem,
adding interest and variety to each stanza.\n\nThe succinct, yet vivid, lines of "Chill winds
and distant cries" wonderfully evoke the atmosphere of the season, adding a touch of
mystery and depth. The final stanza wraps up the poem nicely, celebrating the complete
sensory embrace of fall with lines like "Nature\'s canvas, bold and free."\n\nYou\'ve
successfully infused more sensory richness into the poem, enhancing its overall emotional
and atmospheric impact. Great job on the revisions!\n\nAPPROVE' type='TextMessage'
Stop Reason: Text 'APPROVE' mentioned
As demonstrated in the example above, you can determine the reason why the team
stopped by checking the stop_reason attribute.

The Console() method provides a convenient way to print messages to the console with
proper formatting.

await team.reset() # Reset the team for a new task.

await Console(team.run_stream(task="Write a short poem about the fall season.")) #
Stream the messages to the console.
---------- user ----------
Write a short poem about the fall season.
---------- primary ----------
Golden leaves in crisp air dance,
Whispering tales as they prance.
Amber hues paint the ground,
Nature's symphony all around.

Sweaters hug with tender grace,

While pumpkins smile, a warm embrace.
Chill winds hum through towering trees,
A vibrant tapestry in the breeze.

Harvest moons in twilight glow,

Casting magic on fields below.
Fall's embrace, a gentle call,
To savor beauty before snowfalls.
[Prompt tokens: 28, Completion tokens: 99]
---------- critic ----------
Your poem beautifully captures the essence of the fall season, creating a vivid and cozy
atmosphere. The imagery of golden leaves and amber hues paints a picturesque scene that
many can easily relate to. I particularly appreciate the personification of pumpkins and the
gentle embrace of sweaters, which adds warmth to your verses.

To enhance the poem further, you might consider adding more sensory details to make the
reader feel even more immersed in the experience. For example, including specific sounds,
scents, or textures could deepen the connection to autumn's ambiance. Additionally, you
could explore the emotional transitions as the season prepares for winter to provide a
reflective element to the piece.

Overall, it's a lovely and evocative depiction of fall, evoking feelings of comfort and
appreciation for nature's changing beauty. Great work!
[Prompt tokens: 144, Completion tokens: 157]
---------- primary ----------
Thank you for your thoughtful feedback! I'm glad you enjoyed the imagery and warmth in the
poem. To enhance the sensory experience and emotional depth, here's a revised version
incorporating your suggestions:

---

Golden leaves in crisp air dance,

Whispering tales as they prance.
Amber hues paint the crunchy ground,
Nature's symphony all around.

Sweaters hug with tender grace,

While pumpkins grin, a warm embrace.
Chill winds hum through towering trees,
Crackling fires warm the breeze.

Apples in the orchard's glow,

Sweet cider scents that overflow.
Crunch of paths beneath our feet,
Cinnamon spice and toasty heat.

Harvest moons in twilight's glow,

Casting magic on fields below.
Fall's embrace, a gentle call,
Reflects on life's inevitable thaw.

---

I hope this version enhances the sensory and emotional elements of the season. Thank you
again for your insights!
[Prompt tokens: 294, Completion tokens: 195]
---------- critic ----------
APPROVE
[Prompt tokens: 506, Completion tokens: 4]
---------- Summary ----------
Number of messages: 5
Finish reason: Text 'APPROVE' mentioned
Total prompt tokens: 972
Total completion tokens: 455
Duration: 11.78 seconds
TaskResult(messages=[TextMessage(source='user', models_usage=None, content='Write a
short poem about the fall season.', type='TextMessage'), TextMessage(source='primary',
models_usage=RequestUsage(prompt_tokens=28, completion_tokens=99),
content="Golden leaves in crisp air dance, \nWhispering tales as they prance. \nAmber
hues paint the ground, \nNature's symphony all around. \n\nSweaters hug with tender
grace, \nWhile pumpkins smile, a warm embrace. \nChill winds hum through towering trees,
\nA vibrant tapestry in the breeze. \n\nHarvest moons in twilight glow, \nCasting magic on
fields below. \nFall's embrace, a gentle call, \nTo savor beauty before snowfalls. ",
type='TextMessage'), TextMessage(source='critic',
models_usage=RequestUsage(prompt_tokens=144, completion_tokens=157),
content="Your poem beautifully captures the essence of the fall season, creating a vivid and
cozy atmosphere. The imagery of golden leaves and amber hues paints a picturesque scene
that many can easily relate to. I particularly appreciate the personification of pumpkins and
the gentle embrace of sweaters, which adds warmth to your verses. \n\nTo enhance the
poem further, you might consider adding more sensory details to make the reader feel even
more immersed in the experience. For example, including specific sounds, scents, or
textures could deepen the connection to autumn's ambiance. Additionally, you could explore
the emotional transitions as the season prepares for winter to provide a reflective element to
the piece.\n\nOverall, it's a lovely and evocative depiction of fall, evoking feelings of comfort
and appreciation for nature's changing beauty. Great work!", type='TextMessage'),
TextMessage(source='primary', models_usage=RequestUsage(prompt_tokens=294,
completion_tokens=195), content="Thank you for your thoughtful feedback! I'm glad you
enjoyed the imagery and warmth in the poem. To enhance the sensory experience and
emotional depth, here's a revised version incorporating your suggestions:\n\n---\n\nGolden
leaves in crisp air dance, \nWhispering tales as they prance. \nAmber hues paint the
crunchy ground, \nNature's symphony all around. \n\nSweaters hug with tender grace, \
nWhile pumpkins grin, a warm embrace. \nChill winds hum through towering trees, \
nCrackling fires warm the breeze. \n\nApples in the orchard's glow, \nSweet cider scents
that overflow. \nCrunch of paths beneath our feet, \nCinnamon spice and toasty heat. \n\
nHarvest moons in twilight's glow, \nCasting magic on fields below. \nFall's embrace, a
gentle call, \nReflects on life's inevitable thaw. \n\n--- \n\nI hope this version enhances the
sensory and emotional elements of the season. Thank you again for your insights!",
type='TextMessage'), TextMessage(source='critic',
models_usage=RequestUsage(prompt_tokens=506, completion_tokens=4),
content='APPROVE', type='TextMessage')], stop_reason="Text 'APPROVE' mentioned")
Resetting a Team
You can reset the team by calling the reset() method. This method will clear the team’s state,
including all agents. It will call the each agent’s on_reset() method to clear the agent’s state.

await team.reset() # Reset the team for the next run.

It is usually a good idea to reset the team if the next task is not related to the previous task.
However, if the next task is related to the previous task, you don’t need to reset and you can
instead resume the team.

Stopping a Team
Apart from automatic termination conditions such as TextMentionTermination that stops the
team based on the internal state of the team, you can also stop the team from outside by
using the ExternalTermination.

Calling set() on ExternalTermination will stop the team when the current agent’s turn is over.
Thus, the team may not stop immediately. This allows the current agent to finish its turn and
broadcast the final message to the team before the team stops, keeping the team’s state
consistent.

# Create a new team with an external termination condition.

external_termination = ExternalTermination()
team = RoundRobinGroupChat(
[primary_agent, critic_agent],
termination_condition=external_termination | text_termination, # Use the bitwise OR
operator to combine conditions.
)

# Run the team in a background task.

run = asyncio.create_task(Console(team.run_stream(task="Write a short poem about the fall
season.")))

# Wait for some time.

await asyncio.sleep(0.1)

# Stop the team.

external_termination.set()

# Wait for the team to finish.

await run
---------- user ----------
Write a short poem about the fall season.
---------- primary ----------
Leaves of amber, gold, and red,
Gently drifting from trees overhead.
Whispers of wind through the crisp, cool air,
Nature's canvas painted with care.

Harvest moons and evenings that chill,

Fields of plenty on every hill.
Sweaters wrapped tight as twilight nears,
Fall's charming embrace, as warm as it appears.

Pumpkins aglow with autumn's light,

Harvest feasts and stars so bright.
In every leaf and breeze that calls,
We find the magic of glorious fall.
[Prompt tokens: 28, Completion tokens: 114]
---------- Summary ----------
Number of messages: 2
Finish reason: External termination requested
Total prompt tokens: 28
Total completion tokens: 114
Duration: 1.71 seconds
TaskResult(messages=[TextMessage(source='user', models_usage=None, content='Write a
short poem about the fall season.', type='TextMessage'), TextMessage(source='primary',
models_usage=RequestUsage(prompt_tokens=28, completion_tokens=114),
content="Leaves of amber, gold, and red, \nGently drifting from trees overhead. \nWhispers
of wind through the crisp, cool air, \nNature's canvas painted with care. \n\nHarvest moons
and evenings that chill, \nFields of plenty on every hill. \nSweaters wrapped tight as twilight
nears, \nFall's charming embrace, as warm as it appears. \n\nPumpkins aglow with
autumn's light, \nHarvest feasts and stars so bright. \nIn every leaf and breeze that calls, \
nWe find the magic of glorious fall. ", type='TextMessage')], stop_reason='External
termination requested')
From the ouput above, you can see the team stopped because the external termination
condition was met, but the speaking agent was able to finish its turn before the team
stopped.

Resuming a Team
Teams are stateful and maintains the conversation history and context after each run, unless
you reset the team.

You can resume a team to continue from where it left off by calling the run() or run_stream()
method again without a new task. RoundRobinGroupChat will continue from the next agent
in the round-robin order.

await Console(team.run_stream()) # Resume the team to continue the last task.

---------- critic ----------
This poem beautifully captures the essence of the fall season with vivid imagery and a
soothing rhythm. The descriptions of the changing leaves, cool air, and various autumn
traditions make it easy for readers to envision and feel the charm of fall. Here are a few
suggestions to enhance its impact:
1. **Structure Variation**: Consider breaking some lines with a hyphen or ellipsis for
dramatic effect or emphasis. For instance, “Sweaters wrapped tight as twilight nears— /
Fall’s charming embrace, as warm as it appears."

2. **Sensory Details**: While the poem already evokes visual and tactile senses,
incorporating other senses such as sound or smell could deepen the immersion. For
example, include the scent of wood smoke or the crunch of leaves underfoot.

3. **Metaphorical Language**: Adding metaphors or similes can further enrich the imagery.
For example, you might compare the leaves falling to a golden rain or the chill in the air to a
gentle whisper.

Overall, it’s a lovely depiction of fall. These suggestions are minor tweaks that might elevate
the reader's experience even further. Nice work!

Let me know if these feedbacks are addressed.

[Prompt tokens: 159, Completion tokens: 237]
---------- primary ----------
Thank you for the thoughtful feedback! Here’s a revised version, incorporating your
suggestions:

Leaves of amber, gold—drifting like dreams,

A golden rain from trees’ canopies.
Whispers of wind—a gentle breath,
Nature’s scented tapestry embracing earth.

Harvest moons rise as evenings chill,

Fields of plenty paint every hill.
Sweaters wrapped tight as twilight nears—
Fall’s embrace, warm as whispered years.

Pumpkins aglow with autumn’s light,

Crackling leaves underfoot in flight.
In every leaf and breeze that calls,
We find the magic of glorious fall.

I hope these changes enhance the imagery and sensory experience. Thank you again for
your feedback!
[Prompt tokens: 389, Completion tokens: 150]
---------- critic ----------
Your revisions have made the poem even more evocative and immersive. The use of
sensory details, such as "whispers of wind" and "crackling leaves," beautifully enriches the
poem, engaging multiple senses. The metaphorical language, like "a golden rain from trees’
canopies" and "Fall’s embrace, warm as whispered years," adds depth and enhances the
emotional warmth of the poem. The structural variation with the inclusion of dashes
effectively adds emphasis and flow.
Overall, these changes bring greater vibrancy and life to the poem, allowing readers to truly
experience the wonders of fall. Excellent work on the revisions!

APPROVE
[Prompt tokens: 556, Completion tokens: 132]
---------- Summary ----------
Number of messages: 3
Finish reason: Text 'APPROVE' mentioned
Total prompt tokens: 1104
Total completion tokens: 519
Duration: 9.79 seconds
TaskResult(messages=[TextMessage(source='critic',
models_usage=RequestUsage(prompt_tokens=159, completion_tokens=237), content='This
poem beautifully captures the essence of the fall season with vivid imagery and a soothing
rhythm. The descriptions of the changing leaves, cool air, and various autumn traditions
make it easy for readers to envision and feel the charm of fall. Here are a few suggestions to
enhance its impact:\n\n1. **Structure Variation**: Consider breaking some lines with a
hyphen or ellipsis for dramatic effect or emphasis. For instance, “Sweaters wrapped tight as
twilight nears— / Fall’s charming embrace, as warm as it appears."\n\n2. **Sensory
Details**: While the poem already evokes visual and tactile senses, incorporating other
senses such as sound or smell could deepen the immersion. For example, include the scent
of wood smoke or the crunch of leaves underfoot.\n\n3. **Metaphorical Language**: Adding
metaphors or similes can further enrich the imagery. For example, you might compare the
leaves falling to a golden rain or the chill in the air to a gentle whisper.\n\nOverall, it’s a
lovely depiction of fall. These suggestions are minor tweaks that might elevate the reader\'s
experience even further. Nice work!\n\nLet me know if these feedbacks are addressed.',
type='TextMessage'), TextMessage(source='primary',
models_usage=RequestUsage(prompt_tokens=389, completion_tokens=150),
content='Thank you for the thoughtful feedback! Here’s a revised version, incorporating your
suggestions: \n\nLeaves of amber, gold—drifting like dreams, \nA golden rain from trees’
canopies. \nWhispers of wind—a gentle breath, \nNature’s scented tapestry embracing
earth. \n\nHarvest moons rise as evenings chill, \nFields of plenty paint every hill. \
nSweaters wrapped tight as twilight nears— \nFall’s embrace, warm as whispered years. \
n\nPumpkins aglow with autumn’s light, \nCrackling leaves underfoot in flight. \nIn every
leaf and breeze that calls, \nWe find the magic of glorious fall. \n\nI hope these changes
enhance the imagery and sensory experience. Thank you again for your feedback!',
type='TextMessage'), TextMessage(source='critic',
models_usage=RequestUsage(prompt_tokens=556, completion_tokens=132),
content='Your revisions have made the poem even more evocative and immersive. The use
of sensory details, such as "whispers of wind" and "crackling leaves," beautifully enriches the
poem, engaging multiple senses. The metaphorical language, like "a golden rain from trees’
canopies" and "Fall’s embrace, warm as whispered years," adds depth and enhances the
emotional warmth of the poem. The structural variation with the inclusion of dashes
effectively adds emphasis and flow. \n\nOverall, these changes bring greater vibrancy and
life to the poem, allowing readers to truly experience the wonders of fall. Excellent work on
the revisions!\n\nAPPROVE', type='TextMessage')], stop_reason="Text 'APPROVE'
mentioned")
You can see the team resumed from where it left off in the output above, and the first
message is from the next agent after the last agent that spoke before the team stopped.

Let’s resume the team again with a new task while keeping the context about the previous
task.

# The new task is to translate the same poem to Chinese Tang-style poetry.
await Console(team.run_stream(task="将这首诗用中文唐诗风格写一遍。"))
---------- user ----------
将这首诗用中文唐诗风格写一遍。
---------- primary ----------
朔风轻拂叶飘金，
枝上斜阳染秋林。
满山丰收人欢喜，
月明归途衣渐紧。

南瓜影映灯火中，
落叶沙沙伴归程。
片片秋意随风起，
秋韵悠悠心自明。
[Prompt tokens: 700, Completion tokens: 77]
---------- critic ----------
这首改编的唐诗风格诗作成功地保留了原诗的意境与情感，体现出秋季特有的氛围和美感。通过“朔风轻拂叶飘金”、“枝上斜阳染秋林”等意象，生动地
描绘出了秋天的景色，与唐诗中的自然意境相呼应。且“月明归途衣渐紧”、“落叶沙沙伴归程”让人感受到秋天的安宁与温暖。

通过这些诗句，读者能够感受到秋天的惬意与宁静，勾起丰收与团圆的画面，是一次成功的翻译改编。

APPROVE
[Prompt tokens: 794, Completion tokens: 161]
---------- Summary ----------
Number of messages: 3
Finish reason: Text 'APPROVE' mentioned
Total prompt tokens: 1494
Total completion tokens: 238
Duration: 3.89 seconds
TaskResult(messages=[TextMessage(source='user', models_usage=None,
content='将这首诗用中文唐诗风格写一遍。', type='TextMessage'),
TextMessage(source='primary',
models_usage=RequestUsage(prompt_tokens=700, completion_tokens=77),
content='朔风轻拂叶飘金， \n 枝上斜阳染秋林。 \n 满山丰收人欢喜， \n 月明归途衣渐紧。 \n\n 南瓜影映灯火中， \n 落叶沙
沙伴归程。 \n 片片秋意随风起， \n 秋韵悠悠心自明。 ', type='TextMessage'),
TextMessage(source='critic', models_usage=RequestUsage(prompt_tokens=794,
completion_tokens=161), content='这首改编的唐诗风格诗作成功地保留了原诗的意境与情感，体现出秋季特有的氛围和美
感。通过“朔风轻拂叶飘金”、“枝上斜阳染秋林”等意象，生动地描绘出了秋天的景色，与唐诗中的自然意境相呼应。且“月明归途衣渐紧”、“落叶沙沙
伴归程”让人感受到秋天的安宁与温暖。\n\n 通过这些诗句，读者能够感受到秋天的惬意与宁静，勾起丰收与团圆的画面，是一次成功的翻译改编。\
n\nAPPROVE', type='TextMessage')], stop_reason="Text 'APPROVE' mentioned")
Aborting a Team
You can abort a call to run() or run_stream() during execution by setting a
CancellationToken passed to the cancellation_token parameter.

Different from stopping a team, aborting a team will immediately stop the team and raise a
CancelledError exception.

Note

The caller will get a CancelledError exception when the team is aborted.

# Create a cancellation token.

cancellation_token = CancellationToken()

# Use another coroutine to run the team.

run = asyncio.create_task(
team.run(
task="Translate the poem to Spanish.",
cancellation_token=cancellation_token,
)
)

# Cancel the run.

cancellation_token.cancel()

try:
result = await run # This will raise a CancelledError.
except asyncio.CancelledError:
print("Task was cancelled.")
Task was cancelled.
Human-in-the-Loop
In the previous section Teams, we have seen how to create, observe, and control a team of
agents. This section will focus on how to interact with the team from your application, and
provide human feedback to the team.

There are two main ways to interact with the team from your application:

During a team’s run – execution of run() or run_stream(), provide feedback through a

UserProxyAgent.

Once the run terminates, provide feedback through input to the next call to run() or
run_stream().

Providing Feedback During a Run

The UserProxyAgent is a special built-in agent that acts as a proxy for a user to provide
feedback to the team.

To use the UserProxyAgent, you can create an instance of it and include it in the team
before running the team. The team will decide when to call the UserProxyAgent to ask for
feedback from the user.

The following diagram illustrates how you can use UserProxyAgent to get feedback from the
user during a team’s run:

human-in-the-loop-user-proxy

The bold arrows indicates the flow of control during a team’s run: when the team calls the
UserProxyAgent, it transfers the control to the application/user, and waits for the feedback;
once the feedback is provided, the control is transferred back to the team and the team
continues its execution.

Note

When UserProxyAgent is called during a run, it blocks the execution of the team until the
user provides feedback or errors out. This will hold up the team’s progress and put the team
in an unstable state that cannot be saved or resumed.

Due to the blocking nature of this approach, it is recommended to use it only for short
interactions that require immediate feedback from the user, such as asking for approval or
disapproval with a button click, or an alert requiring immediate attention otherwise failing the
task.

Here is an example of how to use the UserProxyAgent in a RoundRobinGroupChat for a

poetry generation task:

from autogen_agentchat.agents import AssistantAgent, UserProxyAgent

from autogen_agentchat.conditions import TextMentionTermination
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.ui import Console
from autogen_ext.models.openai import OpenAIChatCompletionClient

# Create the agents.

model_client = OpenAIChatCompletionClient(model="gpt-4o-mini")
assistant = AssistantAgent("assistant", model_client=model_client)
user_proxy = UserProxyAgent("user_proxy", input_func=input) # Use input() to get user
input from console.

# Create the termination condition which will end the conversation when the user says
"APPROVE".
termination = TextMentionTermination("APPROVE")

# Create the team.

team = RoundRobinGroupChat([assistant, user_proxy], termination_condition=termination)

# Run the conversation and stream to the console.

stream = team.run_stream(task="Write a 4-line poem about the ocean.")
# Use asyncio.run(...) when running in a script.
await Console(stream)
---------- user ----------
Write a 4-line poem about the ocean.
---------- assistant ----------
Waves whisper secrets to the shore’s embrace,
A dance of blue under the sun's warm grace.
Endless horizons where dreams take flight,
The ocean's heart glimmers, a canvas of light.
TERMINATE
[Prompt tokens: 46, Completion tokens: 49]
---------- user_proxy ----------
APPROVE
---------- Summary ----------
Number of messages: 3
Finish reason: Text 'APPROVE' mentioned
Total prompt tokens: 46
Total completion tokens: 49
Duration: 6.64 seconds
TaskResult(messages=[TextMessage(source='user', models_usage=None, content='Write a
4-line poem about the ocean.', type='TextMessage'), TextMessage(source='assistant',
models_usage=RequestUsage(prompt_tokens=46, completion_tokens=49),
content="Waves whisper secrets to the shore’s embrace, \nA dance of blue under the sun's
warm grace. \nEndless horizons where dreams take flight, \nThe ocean's heart glimmers, a
canvas of light. \nTERMINATE", type='TextMessage'), TextMessage(source='user_proxy',
models_usage=None, content='APPROVE', type='TextMessage')], stop_reason="Text
'APPROVE' mentioned")
From the console output, you can see the team solicited feedback from the user through
user_proxy to approve the generated poem.
You can provide your own input function to the UserProxyAgent to customize the feedback
process. For example, when the team is running as a web service, you can use a custom
input function to wait for message from a web socket connection. The following code snippet
shows an example of custom input function when using the FastAPI web framework:

@app.websocket("/ws/chat")
async def chat(websocket: WebSocket):
await websocket.accept()

async def _user_input(prompt: str, cancellation_token: CancellationToken | None) -> str:

data = await websocket.receive_json() # Wait for user message from websocket.
message = TextMessage.model_validate(data) # Assume user message is a
TextMessage.
return message.content

# Create user proxy with custom input function

# Run the team with the user proxy
# ...
See the AgentChat FastAPI sample for a complete example.

Providing Feedback to the Next Run

Often times, an application or a user interacts with the team of agents in an interactive loop:
the team runs until termination, the application or user provides feedback, and the team runs
again with the feedback.

This approach is useful in a persisted session with asynchronous communication between

the team and the application/user: Once a team finishes a run, the application saves the
state of the team, puts it in a persistent storage, and resumes the team when the feedback
arrives.

Note

For how to save and load the state of a team, please refer to Managing State. This section
will focus on the feedback mechanisms.

The following diagram illustrates the flow of control in this approach:

human-in-the-loop-termination

There are two ways to implement this approach:

Set the maximum number of turns so that the team always stops after the specified number
of turns.

Use termination conditions such as TextMentionTermination and HandoffTermination to

allow the team to decide when to stop and give control back, given the team’s internal state.

You can use both methods together to achieve your desired behavior.
Using Max Turns
This method allows you to pause the team for user input by setting a maximum number of
turns. For instance, you can configure the team to stop after the first agent responds by
setting max_turns to 1. This is particularly useful in scenarios where continuous user
engagement is required, such as in a chatbot.

To implement this, set the max_turns parameter in the RoundRobinGroupChat() constructor.

team = RoundRobinGroupChat([...], max_turns=1)

Once the team stops, the turn count will be reset. When you resume the team, it will start
from 0 again. However, the team’s internal state will be preserved, for example, the
RoundRobinGroupChat will resume from the next agent in the list with the same
conversation history.

Note

max_turn is specific to the team class and is currently only supported by

RoundRobinGroupChat, SelectorGroupChat, and Swarm. When used with termination
conditions, the team will stop when either condition is met.

Here is an example of how to use max_turns in a RoundRobinGroupChat for a poetry

generation task with a maximum of 1 turn:

from autogen_agentchat.agents import AssistantAgent

from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.ui import Console
from autogen_ext.models.openai import OpenAIChatCompletionClient

# Create the agents.

model_client = OpenAIChatCompletionClient(model="gpt-4o-mini")
assistant = AssistantAgent("assistant", model_client=model_client)

# Create the team setting a maximum number of turns to 1.

team = RoundRobinGroupChat([assistant], max_turns=1)

task = "Write a 4-line poem about the ocean."

while True:
# Run the conversation and stream to the console.
stream = team.run_stream(task=task)
# Use asyncio.run(...) when running in a script.
await Console(stream)
# Get the user response.
task = input("Enter your feedback (type 'exit' to leave): ")
if task.lower().strip() == "exit":
break
---------- user ----------
Write a 4-line poem about the ocean.
---------- assistant ----------
Endless waves in a dance with the shore,
Whispers of secrets in tales from the roar,
Beneath the vast sky, where horizons blend,
The ocean’s embrace is a timeless friend.
TERMINATE
[Prompt tokens: 46, Completion tokens: 48]
---------- Summary ----------
Number of messages: 2
Finish reason: Maximum number of turns 1 reached.
Total prompt tokens: 46
Total completion tokens: 48
Duration: 1.63 seconds
---------- user ----------
Can you make it about a person and its relationship with the ocean
---------- assistant ----------
She walks along the tide, where dreams intertwine,
With every crashing wave, her heart feels aligned,
In the ocean's embrace, her worries dissolve,
A symphony of solace, where her spirit evolves.
TERMINATE
[Prompt tokens: 117, Completion tokens: 49]
---------- Summary ----------
Number of messages: 2
Finish reason: Maximum number of turns 1 reached.
Total prompt tokens: 117
Total completion tokens: 49
Duration: 1.21 seconds
You can see that the team stopped immediately after one agent responded.

Using Termination Conditions

We have already seen several examples of termination conditions in the previous sections.
In this section, we focus on HandoffTermination which stops the team when an agent sends
a HandoffMessage message.

Let’s create a team with a single AssistantAgent agent with a handoff setting, and run the
team with a task that requires additional input from the user because the agent doesn’t have
relevant tools to continue processing the task.

Note

The model used with AssistantAgent must support tool call to use the handoff feature.

from autogen_agentchat.agents import AssistantAgent

from autogen_agentchat.base import Handoff
from autogen_agentchat.conditions import HandoffTermination, TextMentionTermination
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.ui import Console
from autogen_ext.models.openai import OpenAIChatCompletionClient

# Create an OpenAI model client.

model_client = OpenAIChatCompletionClient(
model="gpt-4o",
# api_key="sk-...", # Optional if you have an OPENAI_API_KEY env variable set.
)

# Create a lazy assistant agent that always hands off to the user.
lazy_agent = AssistantAgent(
"lazy_assistant",
model_client=model_client,
handoffs=[Handoff(target="user", message="Transfer to user.")],
system_message="If you cannot complete the task, transfer to user. Otherwise, when
finished, respond with 'TERMINATE'.",
)

# Define a termination condition that checks for handoff messages.

handoff_termination = HandoffTermination(target="user")
# Define a termination condition that checks for a specific text mention.
text_termination = TextMentionTermination("TERMINATE")

# Create a single-agent team with the lazy assistant and both termination conditions.
lazy_agent_team = RoundRobinGroupChat([lazy_agent],
termination_condition=handoff_termination | text_termination)

# Run the team and stream to the console.

task = "What is the weather in New York?"
await Console(lazy_agent_team.run_stream(task=task), output_stats=True)
---------- user ----------
What is the weather in New York?
---------- lazy_assistant ----------
[FunctionCall(id='call_EAcMgrLGHdLw0e7iJGoMgxuu', arguments='{}',
name='transfer_to_user')]
[Prompt tokens: 69, Completion tokens: 12]
---------- lazy_assistant ----------
[FunctionExecutionResult(content='Transfer to user.',
call_id='call_EAcMgrLGHdLw0e7iJGoMgxuu')]
---------- lazy_assistant ----------
Transfer to user.
---------- Summary ----------
Number of messages: 4
Finish reason: Handoff to user from lazy_assistant detected.
Total prompt tokens: 69
Total completion tokens: 12
Duration: 0.69 seconds
TaskResult(messages=[TextMessage(source='user', models_usage=None, content='What is
the weather in New York?', type='TextMessage'),
ToolCallRequestEvent(source='lazy_assistant',
models_usage=RequestUsage(prompt_tokens=69, completion_tokens=12),
content=[FunctionCall(id='call_EAcMgrLGHdLw0e7iJGoMgxuu', arguments='{}',
name='transfer_to_user')], type='ToolCallRequestEvent'),
ToolCallExecutionEvent(source='lazy_assistant', models_usage=None,
content=[FunctionExecutionResult(content='Transfer to user.',
call_id='call_EAcMgrLGHdLw0e7iJGoMgxuu')], type='ToolCallExecutionEvent'),
HandoffMessage(source='lazy_assistant', models_usage=None, target='user',
content='Transfer to user.', context=[], type='HandoffMessage')], stop_reason='Handoff to
user from lazy_assistant detected.')
You can see the team stopped due to the handoff message was detected. Let’s continue the
team by providing the information the agent needs.

await Console(lazy_agent_team.run_stream(task="The weather in New York is sunny."))

---------- user ----------
The weather in New York is sunny.
---------- lazy_assistant ----------
Great! Enjoy the sunny weather in New York! Is there anything else you'd like to know?
---------- lazy_assistant ----------
TERMINATE
TaskResult(messages=[TextMessage(source='user', models_usage=None, content='The
weather in New York is sunny.', type='TextMessage'), TextMessage(source='lazy_assistant',
models_usage=RequestUsage(prompt_tokens=110, completion_tokens=21),
content="Great! Enjoy the sunny weather in New York! Is there anything else you'd like to
know?", type='TextMessage'), TextMessage(source='lazy_assistant',
models_usage=RequestUsage(prompt_tokens=137, completion_tokens=5),
content='TERMINATE', type='TextMessage')], stop_reason="Text 'TERMINATE'
mentioned")
You can see the team continued after the user provided the information.
Termination
In the previous section, we explored how to define agents, and organize them into teams
that can solve tasks. However, a run can go on forever, and in many cases, we need to
know when to stop them. This is the role of the termination condition.

AgentChat supports several termination condition by providing a base TerminationCondition

class and several implementations that inherit from it.

A termination condition is a callable that takes a sequence of AgentEvent or ChatMessage

objects since the last time the condition was called, and returns a StopMessage if the
conversation should be terminated, or None otherwise. Once a termination condition has
been reached, it must be reset by calling reset() before it can be used again.

Some important things to note about termination conditions:

They are stateful but reset automatically after each run (run() or run_stream()) is finished.

They can be combined using the AND and OR operators.

Note

For group chat teams (i.e., RoundRobinGroupChat, SelectorGroupChat, and Swarm), the
termination condition is called after each agent responds. While a response may contain
multiple inner messages, the team calls its termination condition just once for all the
messages from a single response. So the condition is called with the “delta sequence” of
messages since the last time it was called.

Built-In Termination Conditions:

MaxMessageTermination: Stops after a specified number of messages have been produced,

including both agent and task messages.

TextMentionTermination: Stops when specific text or string is mentioned in a message (e.g.,

“TERMINATE”).

TokenUsageTermination: Stops when a certain number of prompt or completion tokens are

used. This requires the agents to report token usage in their messages.

TimeoutTermination: Stops after a specified duration in seconds.

HandoffTermination: Stops when a handoff to a specific target is requested. Handoff

messages can be used to build patterns such as Swarm. This is useful when you want to
pause the run and allow application or user to provide input when an agent hands off to
them.

SourceMatchTermination: Stops after a specific agent responds.

ExternalTermination: Enables programmatic control of termination from outside the run. This
is useful for UI integration (e.g., “Stop” buttons in chat interfaces).

StopMessageTermination: Stops when a StopMessage is produced by an agent.

To demonstrate the characteristics of termination conditions, we’ll create a team consisting

of two agents: a primary agent responsible for text generation and a critic agent that reviews
and provides feedback on the generated text.

from autogen_agentchat.agents import AssistantAgent

from autogen_agentchat.conditions import MaxMessageTermination,
TextMentionTermination
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.ui import Console
from autogen_ext.models.openai import OpenAIChatCompletionClient

model_client = OpenAIChatCompletionClient(
model="gpt-4o",
temperature=1,
# api_key="sk-...", # Optional if you have an OPENAI_API_KEY env variable set.
)

# Create the primary agent.

primary_agent = AssistantAgent(
"primary",
model_client=model_client,
system_message="You are a helpful AI assistant.",
)

# Create the critic agent.

critic_agent = AssistantAgent(
"critic",
model_client=model_client,
system_message="Provide constructive feedback for every message. Respond with
'APPROVE' to when your feedbacks are addressed.",
)
Let’s explore how termination conditions automatically reset after each run or run_stream
call, allowing the team to resume its conversation from where it left off.

max_msg_termination = MaxMessageTermination(max_messages=3)
round_robin_team = RoundRobinGroupChat([primary_agent, critic_agent],
termination_condition=max_msg_termination)

# Use asyncio.run(...) if you are running this script as a standalone script.

await Console(round_robin_team.run_stream(task="Write a unique, Haiku about the weather
in Paris"))
---------- user ----------
Write a unique, Haiku about the weather in Paris
---------- primary ----------
Gentle rain whispers,
Cobblestones glisten softly—
Paris dreams in gray.
[Prompt tokens: 30, Completion tokens: 19]
---------- critic ----------
The Haiku captures the essence of a rainy day in Paris beautifully, and the imagery is vivid.
However, it's important to ensure the use of the traditional 5-7-5 syllable structure for Haikus.
Your current Haiku lines are composed of 4-7-5 syllables, which slightly deviates from the
form. Consider revising the first line to fit the structure.

For example:
Soft rain whispers down,
Cobblestones glisten softly —
Paris dreams in gray.

This revision maintains the essence of your original lines while adhering to the traditional
Haiku structure.
[Prompt tokens: 70, Completion tokens: 120]
---------- Summary ----------
Number of messages: 3
Finish reason: Maximum number of messages 3 reached, current message count: 3
Total prompt tokens: 100
Total completion tokens: 139
Duration: 3.34 seconds
TaskResult(messages=[TextMessage(source='user', models_usage=None, content='Write a
unique, Haiku about the weather in Paris'), TextMessage(source='primary',
models_usage=RequestUsage(prompt_tokens=30, completion_tokens=19), content='Gentle
rain whispers, \nCobblestones glisten softly— \nParis dreams in gray.'),
TextMessage(source='critic', models_usage=RequestUsage(prompt_tokens=70,
completion_tokens=120), content="The Haiku captures the essence of a rainy day in Paris
beautifully, and the imagery is vivid. However, it's important to ensure the use of the
traditional 5-7-5 syllable structure for Haikus. Your current Haiku lines are composed of 4-7-
5 syllables, which slightly deviates from the form. Consider revising the first line to fit the
structure.\n\nFor example:\nSoft rain whispers down, \nCobblestones glisten softly — \
nParis dreams in gray.\n\nThis revision maintains the essence of your original lines while
adhering to the traditional Haiku structure.")], stop_reason='Maximum number of messages
3 reached, current message count: 3')
The conversation stopped after reaching the maximum message limit. Since the primary
agent didn’t get to respond to the feedback, let’s continue the conversation.

# Use asyncio.run(...) if you are running this script as a standalone script.

await Console(round_robin_team.run_stream())
---------- primary ----------
Thank you for your feedback. Here is the revised Haiku:

Soft rain whispers down,

Cobblestones glisten softly —
Paris dreams in gray.
[Prompt tokens: 181, Completion tokens: 32]
---------- critic ----------
The revised Haiku now follows the traditional 5-7-5 syllable pattern, and it still beautifully
captures the atmospheric mood of Paris in the rain. The imagery and flow are both clear and
evocative. Well done on making the adjustment!

APPROVE
[Prompt tokens: 234, Completion tokens: 54]
---------- primary ----------
Thank you for your kind words and approval. I'm glad the revision meets your expectations
and captures the essence of Paris. If you have any more requests or need further
assistance, feel free to ask!
[Prompt tokens: 279, Completion tokens: 39]
---------- Summary ----------
Number of messages: 3
Finish reason: Maximum number of messages 3 reached, current message count: 3
Total prompt tokens: 694
Total completion tokens: 125
Duration: 6.43 seconds
TaskResult(messages=[TextMessage(source='primary',
models_usage=RequestUsage(prompt_tokens=181, completion_tokens=32),
content='Thank you for your feedback. Here is the revised Haiku:\n\nSoft rain whispers
down, \nCobblestones glisten softly — \nParis dreams in gray.'),
TextMessage(source='critic', models_usage=RequestUsage(prompt_tokens=234,
completion_tokens=54), content='The revised Haiku now follows the traditional 5-7-5 syllable
pattern, and it still beautifully captures the atmospheric mood of Paris in the rain. The
imagery and flow are both clear and evocative. Well done on making the adjustment! \n\
nAPPROVE'), TextMessage(source='primary',
models_usage=RequestUsage(prompt_tokens=279, completion_tokens=39),
content="Thank you for your kind words and approval. I'm glad the revision meets your
expectations and captures the essence of Paris. If you have any more requests or need
further assistance, feel free to ask!")], stop_reason='Maximum number of messages 3
reached, current message count: 3')
The team continued from where it left off, allowing the primary agent to respond to the
feedback.

Next, let’s show how termination conditions can be combined using the AND (&) and OR (|)
operators to create more complex termination logic. For example, we’ll create a team that
stops either after 10 messages are generated or when the critic agent approves a message.

max_msg_termination = MaxMessageTermination(max_messages=10)
text_termination = TextMentionTermination("APPROVE")
combined_termination = max_msg_termination | text_termination

round_robin_team = RoundRobinGroupChat([primary_agent, critic_agent],

termination_condition=combined_termination)
# Use asyncio.run(...) if you are running this script as a standalone script.
await Console(round_robin_team.run_stream(task="Write a unique, Haiku about the weather
in Paris"))
---------- user ----------
Write a unique, Haiku about the weather in Paris
---------- primary ----------
Spring breeze gently hums,
Cherry blossoms in full bloom—
Paris wakes to life.
[Prompt tokens: 467, Completion tokens: 19]
---------- critic ----------
The Haiku beautifully captures the awakening of Paris in the spring. The imagery of a gentle
spring breeze and cherry blossoms in full bloom effectively conveys the rejuvenating feel of
the season. The final line, "Paris wakes to life," encapsulates the renewed energy and
vibrancy of the city. The Haiku adheres to the 5-7-5 syllable structure and portrays a vivid
seasonal transformation in a concise and poetic manner. Excellent work!

APPROVE
[Prompt tokens: 746, Completion tokens: 93]
---------- Summary ----------
Number of messages: 3
Finish reason: Text 'APPROVE' mentioned
Total prompt tokens: 1213
Total completion tokens: 112
Duration: 2.75 seconds
TaskResult(messages=[TextMessage(source='user', models_usage=None, content='Write a
unique, Haiku about the weather in Paris'), TextMessage(source='primary',
models_usage=RequestUsage(prompt_tokens=467, completion_tokens=19),
content='Spring breeze gently hums, \nCherry blossoms in full bloom— \nParis wakes to
life.'), TextMessage(source='critic', models_usage=RequestUsage(prompt_tokens=746,
completion_tokens=93), content='The Haiku beautifully captures the awakening of Paris in
the spring. The imagery of a gentle spring breeze and cherry blossoms in full bloom
effectively conveys the rejuvenating feel of the season. The final line, "Paris wakes to life,"
encapsulates the renewed energy and vibrancy of the city. The Haiku adheres to the 5-7-5
syllable structure and portrays a vivid seasonal transformation in a concise and poetic
manner. Excellent work!\n\nAPPROVE')], stop_reason="Text 'APPROVE' mentioned")
The conversation stopped after the critic agent approved the message, although it could
have also stopped if 10 messages were generated.

Alternatively, if we want to stop the run only when both conditions are met, we can use the
AND (&) operator.

combined_termination = max_msg_termination & text_termination

Managing State
So far, we have discussed how to build components in a multi-agent application - agents,
teams, termination conditions. In many cases, it is useful to save the state of these
components to disk and load them back later. This is particularly useful in a web application
where stateless endpoints respond to requests and need to load the state of the application
from persistent storage.

In this notebook, we will discuss how to save and load the state of agents, teams, and
termination conditions.

Saving and Loading Agents

We can get the state of an agent by calling save_state() method on an AssistantAgent.

from autogen_agentchat.agents import AssistantAgent

from autogen_agentchat.conditions import MaxMessageTermination
from autogen_agentchat.messages import TextMessage
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.ui import Console
from autogen_core import CancellationToken
from autogen_ext.models.openai import OpenAIChatCompletionClient

assistant_agent = AssistantAgent(
name="assistant_agent",
system_message="You are a helpful assistant",
model_client=OpenAIChatCompletionClient(
model="gpt-4o-2024-08-06",
# api_key="YOUR_API_KEY",
),
)

# Use asyncio.run(...) when running in a script.

response = await assistant_agent.on_messages(
[TextMessage(content="Write a 3 line poem on lake tangayika", source="user")],
CancellationToken()
)
print(response.chat_message.content)
In Tanganyika's embrace so wide and deep,
Ancient waters cradle secrets they keep,
Echoes of time where horizons sleep.
agent_state = await assistant_agent.save_state()
print(agent_state)
{'type': 'AssistantAgentState', 'version': '1.0.0', 'llm_messages': [{'content': 'Write a 3 line
poem on lake tangayika', 'source': 'user', 'type': 'UserMessage'}, {'content': "In Tanganyika's
embrace so wide and deep, \nAncient waters cradle secrets they keep, \nEchoes of time
where horizons sleep. ", 'source': 'assistant_agent', 'type': 'AssistantMessage'}]}
new_assistant_agent = AssistantAgent(
name="assistant_agent",
system_message="You are a helpful assistant",
 model_client=OpenAIChatCompletionClient(
model="gpt-4o-2024-08-06",
),
)
await new_assistant_agent.load_state(agent_state)

# Use asyncio.run(...) when running in a script.

response = await new_assistant_agent.on_messages(
[TextMessage(content="What was the last line of the previous poem you wrote",
source="user")], CancellationToken()
)
print(response.chat_message.content)
The last line of the poem was: "Echoes of time where horizons sleep."
Note

For AssistantAgent, its state consists of the model_context. If your write your own custom
agent, consider overriding the save_state() and load_state() methods to customize the
behavior. The default implementations save and load an empty state.

Saving and Loading Teams

We can get the state of a team by calling save_state method on the team and load it back by
calling load_state method on the team.

When we call save_state on a team, it saves the state of all the agents in the team.

We will begin by creating a simple RoundRobinGroupChat team with a single agent and ask
it to write a poem.

# Define a team.
assistant_agent = AssistantAgent(
name="assistant_agent",
system_message="You are a helpful assistant",
model_client=OpenAIChatCompletionClient(
model="gpt-4o-2024-08-06",
),
)
agent_team = RoundRobinGroupChat([assistant_agent],
termination_condition=MaxMessageTermination(max_messages=2))

# Run the team and stream messages to the console.

stream = agent_team.run_stream(task="Write a beautiful poem 3-line about lake tangayika")

# Use asyncio.run(...) when running in a script.

await Console(stream)

# Save the state of the agent team.

team_state = await agent_team.save_state()
---------- user ----------
Write a beautiful poem 3-line about lake tangayika
---------- assistant_agent ----------
In Tanganyika's gleam, beneath the azure skies,
Whispers of ancient waters, in tranquil guise,
Nature's mirror, where dreams and serenity lie.
[Prompt tokens: 29, Completion tokens: 34]
---------- Summary ----------
Number of messages: 2
Finish reason: Maximum number of messages 2 reached, current message count: 2
Total prompt tokens: 29
Total completion tokens: 34
Duration: 0.71 seconds
If we reset the team (simulating instantiation of the team), and ask the question What was
the last line of the poem you wrote?, we see that the team is unable to accomplish this as
there is no reference to the previous run.

await agent_team.reset()
stream = agent_team.run_stream(task="What was the last line of the poem you wrote?")
await Console(stream)
---------- user ----------
What was the last line of the poem you wrote?
---------- assistant_agent ----------
I'm sorry, but I am unable to recall or access previous interactions, including any specific
poem I may have composed in our past conversations. If you like, I can write a new poem for
you.
[Prompt tokens: 28, Completion tokens: 40]
---------- Summary ----------
Number of messages: 2
Finish reason: Maximum number of messages 2 reached, current message count: 2
Total prompt tokens: 28
Total completion tokens: 40
Duration: 0.70 seconds
TaskResult(messages=[TextMessage(source='user', models_usage=None, content='What
was the last line of the poem you wrote?', type='TextMessage'),
TextMessage(source='assistant_agent', models_usage=RequestUsage(prompt_tokens=28,
completion_tokens=40), content="I'm sorry, but I am unable to recall or access previous
interactions, including any specific poem I may have composed in our past conversations. If
you like, I can write a new poem for you.", type='TextMessage')], stop_reason='Maximum
number of messages 2 reached, current message count: 2')
Next, we load the state of the team and ask the same question. We see that the team is able
to accurately return the last line of the poem it wrote.

print(team_state)

# Load team state.

await agent_team.load_state(team_state)
stream = agent_team.run_stream(task="What was the last line of the poem you wrote?")
await Console(stream)
{'type': 'TeamState', 'version': '1.0.0', 'agent_states': {'group_chat_manager/a55364ad-86fd-
46ab-9449-dcb5260b1e06': {'type': 'RoundRobinManagerState', 'version': '1.0.0',
'message_thread': [{'source': 'user', 'models_usage': None, 'content': 'Write a beautiful poem
3-line about lake tangayika', 'type': 'TextMessage'}, {'source': 'assistant_agent',
'models_usage': {'prompt_tokens': 29, 'completion_tokens': 34}, 'content': "In Tanganyika's
gleam, beneath the azure skies, \nWhispers of ancient waters, in tranquil guise, \nNature's
mirror, where dreams and serenity lie.", 'type': 'TextMessage'}], 'current_turn': 0,
'next_speaker_index': 0}, 'collect_output_messages/a55364ad-86fd-46ab-9449-
dcb5260b1e06': {}, 'assistant_agent/a55364ad-86fd-46ab-9449-dcb5260b1e06': {'type':
'ChatAgentContainerState', 'version': '1.0.0', 'agent_state': {'type': 'AssistantAgentState',
'version': '1.0.0', 'llm_messages': [{'content': 'Write a beautiful poem 3-line about lake
tangayika', 'source': 'user', 'type': 'UserMessage'}, {'content': "In Tanganyika's gleam,
beneath the azure skies, \nWhispers of ancient waters, in tranquil guise, \nNature's mirror,
where dreams and serenity lie.", 'source': 'assistant_agent', 'type': 'AssistantMessage'}]},
'message_buffer': []}}, 'team_id': 'a55364ad-86fd-46ab-9449-dcb5260b1e06'}
---------- user ----------
What was the last line of the poem you wrote?
---------- assistant_agent ----------
The last line of the poem I wrote is:
"Nature's mirror, where dreams and serenity lie."
[Prompt tokens: 86, Completion tokens: 22]
---------- Summary ----------
Number of messages: 2
Finish reason: Maximum number of messages 2 reached, current message count: 2
Total prompt tokens: 86
Total completion tokens: 22
Duration: 0.96 seconds
TaskResult(messages=[TextMessage(source='user', models_usage=None, content='What
was the last line of the poem you wrote?', type='TextMessage'),
TextMessage(source='assistant_agent', models_usage=RequestUsage(prompt_tokens=86,
completion_tokens=22), content='The last line of the poem I wrote is: \n"Nature\'s mirror,
where dreams and serenity lie."', type='TextMessage')], stop_reason='Maximum number of
messages 2 reached, current message count: 2')
Persisting State (File or Database)
In many cases, we may want to persist the state of the team to disk (or a database) and load
it back later. State is a dictionary that can be serialized to a file or written to a database.

import json

## save state to disk

with open("coding/team_state.json", "w") as f:

json.dump(team_state, f)

## load state from disk

with open("coding/team_state.json", "r") as f:
team_state = json.load(f)
new_agent_team = RoundRobinGroupChat([assistant_agent],
termination_condition=MaxMessageTermination(max_messages=2))
await new_agent_team.load_state(team_state)
stream = new_agent_team.run_stream(task="What was the last line of the poem you
wrote?")
await Console(stream)
---------- user ----------
What was the last line of the poem you wrote?
---------- assistant_agent ----------
The last line of the poem I wrote is:
"Nature's mirror, where dreams and serenity lie."
[Prompt tokens: 86, Completion tokens: 22]
---------- Summary ----------
Number of messages: 2
Finish reason: Maximum number of messages 2 reached, current message count: 2
Total prompt tokens: 86
Total completion tokens: 22
Duration: 0.72 seconds
TaskResult(messages=[TextMessage(source='user', models_usage=None, content='What
was the la