10-2-appendix-tool-use

April 9, 2024

1 Appendix 10.2: Tool Use

• Lesson
• Exercises
• Example Playground

1.1 Setup
Run the following setup cell to load your API key and establish the get_completion helper function.

[ ]: !pip install anthropic

# Import python's built-in regular expression library

import re
import anthropic

# Retrieve the API_KEY variable from the IPython store

%store -r API_KEY

client = anthropic.Anthropic(api_key=API_KEY)

# Rewrittten to call Claude 3 Sonnet, which is generally better at tool use,␣

↪and include stop_sequences

def get_completion(messages, system_prompt="", prefill="",stop_sequences=None):

message = client.messages.create(
model="claude-3-sonnet-20240229",
max_tokens=2000,
temperature=0.0,
system=system_prompt,
messages=messages,
stop_sequences=stop_sequences
)
return message.content[0].text

1
 1.2 Lesson
While it might seem conceptually complex at first, tool use, a.k.a. function calling, is actually
quite simple! You already know all the skills necessary to implement tool use, which is really just
a combination of substitution and prompt chaining.
In previous substitution exercises, we substituted text into prompts. With tool use, we substitute
tool or function results into prompts. Claude can’t literally call or access tools and functions.
Instead, we have Claude: 1. Output the tool name and arguments it wants to call 2. Halt any
further response generation while the tool is called 3. Then we reprompt with the appended tool
results
Function calling is useful because it expands Claude’s capabilities and enables Claude to handle
much more complex, multi-step tasks. Some examples of functions you can give Claude: - Calculator
- Word counter - SQL database querying and data retrieval - Weather API
You can get Claude to do tool use by combining these two elements:
1. A system prompt, in which we give Claude an explanation of the concept of tool use as well
as a detailed descriptive list of the tools it has access to
2. The control logic with which to orchestrate and execute Claude’s tool use requests

1.2.1 Tool use roadmap

This lesson teaches our current tool use format. However, we will be updating and improving tool
use functionality in the near future, including: * A more streamlined format for function definitions
and calls * More robust error handilgj and edge case coverage * Tighter integration with the rest of
our API * Better reliability and performance, especially for more complex tool use tasks

1.2.2 Examples
To enable tool use in Claude, we start with the system prompt. In this special tool use system
prompt, wet tell Claude: * The basic premise of tool use and what it entails * How Claude can call
and use the tools it’s been given * A detailed list of tools it has access to in this specific scenario
Here’s the first part of the system prompt, explaining tool use to Claude. This part of the system
prompt is generalizable across all instances of prompting Claude for tool use. The tool calling struc-
ture we’re giving Claude (<function_calls> [...] </function_calls>) is a structure Claude
has been specifically trained to use, so we recommend that you stick with this.
[ ]: system_prompt_tools_general_explanation = """You have access to a set of␣
↪functions you can use to answer the user's question. This includes access to␣

↪a

sandboxed computing environment. You do NOT currently have the ability to␣
↪inspect files or interact with external

resources, except by invoking the below functions.

You can invoke one or more functions by writing a "<function_calls>" block like␣
↪the following as part of your

reply to the user:

<function_calls>

2
 <invoke name="$FUNCTION_NAME">
<antml:parameter name="$PARAMETER_NAME">$PARAMETER_VALUE</parameter>
...
</invoke>
<nvoke name="$FUNCTION_NAME2">
...
</invoke>
</function_calls>

String and scalar parameters should be specified as is, while lists and objects␣
↪should use JSON format. Note that

spaces for string values are not stripped. The output is not expected to be␣
↪valid XML and is parsed with regular

expressions.

The output and/or any errors will appear in a subsequent "<function_results>"␣

↪block, and remain there as part of

your reply to the user.

You may then continue composing the rest of your reply to the user, respond to␣
↪any errors, or make further function

calls as appropriate.
If a "<function_results>" does NOT appear after your function calls, then they␣
↪are likely malformatted and not

recognized as a call."""

Here’s the second part of the system prompt, which defines the exact tools Claude has access to
in this specific situation. In this example, we will be giving Claude a calculator tool, which takes
three parameters: two operands and an operator.
Then we combine the two parts of the system prompt.
[ ]: system_prompt_tools_specific_tools = """Here are the functions available in␣
↪JSONSchema format:

<tools>
<tool_description>
<tool_name>calculator</tool_name>
<description>
Calculator function for doing basic arithmetic.
Supports addition, subtraction, multiplication
</description>
<parameters>
<parameter>
<name>first_operand</name>
<type>int</type>
<description>First operand (before the operator)</description>
</parameter>
<parameter>

3
 <name>second_operand</name>
<type>int</type>
<description>Second operand (after the operator)</description>
</parameter>
<parameter>
<name>operator</name>
<type>str</type>
<description>The operation to perform. Must be either +, -, *, or /</
↪description>

</parameter>
</parameters>
</tool_description>
</tools>
"""

system_prompt = system_prompt_tools_general_explanation +␣
↪system_prompt_tools_specific_tools

Now we can give Claude a question that requires use of the calculator tool. We will use
<function_calls\> in stop_sequences to detect if and when Claude calls the function.
[ ]: multiplication_message = {
"role": "user",
"content": "Multiply 1,984,135 by 9,343,116"
}

stop_sequences = ["</function_calls>"]

# Get Claude's response

function_calling_response = get_completion([multiplication_message],␣
↪system_prompt=system_prompt, stop_sequences=stop_sequences)

print(function_calling_response)

Now, we can extract out the parameters from Claude’s function call and actually run the function
on Claude’s behalf.
First we’ll define the function’s code.
[ ]: def do_pairwise_arithmetic(num1, num2, operation):
if operation == '+':
return num1 + num2
elif operation == "-":
return num1 - num2
elif operation == "*":
return num1 * num2
elif operation == "/":
return num1 / num2
else:

4
 return "Error: Operation not supported."

Then we’ll extract the parameters from Claude’s function call response. If all the parameters exist,
we run the calculator tool.
[ ]: def find_parameter(message, parameter_name):
parameter_start_string = f"name=\"{parameter_name}\">"
start = message.index(parameter_start_string)
if start == -1:
return None
if start > 0:
start = start + len(parameter_start_string)
end = start
while message[end] != "<":
end += 1
return message[start:end]

first_operand = find_parameter(function_calling_response, "first_operand")

second_operand = find_parameter(function_calling_response, "second_operand")
operator = find_parameter(function_calling_response, "operator")

if first_operand and second_operand and operator:

result = do_pairwise_arithmetic(int(first_operand), int(second_operand),␣
↪operator)

print("---------------- RESULT ----------------")

print(f"{result:,}")

Now that we have a result, we have to properly format that result so that when we pass it back to
Claude, Claude understands what tool that result is in relation to. There is a set format for this
that Claude has been trained to recognize:
<function_results>
<result>
<tool_name>{TOOL_NAME}</tool_name>
<stdout>
{TOOL_RESULT}
</stdout>
</result>
</function_results>
Run the cell below to format the above tool result into this structure.
[ ]: def construct_successful_function_run_injection_prompt(invoke_results):
constructed_prompt = (
"<function_results>\n"
+ '\n'.join(
f"<result>\n<tool_name>{res['tool_name']}</
↪tool_name>\n<stdout>\n{res['tool_result']}\n</stdout>\n</result>"

for res in invoke_results

5
 ) + "\n</function_results>"
)

return constructed_prompt

formatted_results = [{
'tool_name': 'do_pairwise_arithmetic',
'tool_result': result
}]
function_results =␣
↪construct_successful_function_run_injection_prompt(formatted_results)

print(function_results)

Now all we have to do is send this result back to Claude by appending the result to the same
message chain as before, and we’re good!
[ ]: full_first_response = function_calling_response + "</function_calls>"

# Construct the full conversation

messages = [multiplication_message,
{
"role": "assistant",
"content": full_first_response
},
{
"role": "user",
"content": function_results
}]

# Print Claude's response

final_response = get_completion(messages, system_prompt=system_prompt,␣
↪stop_sequences=stop_sequences)

print("------------- FINAL RESULT -------------")

print(final_response)

Congratulations on running an entire tool use chain end to end!

Now what if we give Claude a question that doesn’t that doesn’t require using the given tool at
all?
[ ]: non_multiplication_message = {
"role": "user",
"content": "Tell me the capital of France."
}

stop_sequences = ["</function_calls>"]

# Get Claude's response

6
 function_calling_response = get_completion([non_multiplication_message],␣
↪system_prompt=system_prompt, stop_sequences=stop_sequences)

print(function_calling_response)

Success! As you can see, Claude knew not to call the function when it wasn’t needed.
If you would like to experiment with the lesson prompts without changing any content above, scroll
all the way to the bottom of the lesson notebook to visit the Example Playground.

1.3 Exercises
• Exercise 10.2.1 - SQL

1.3.1 Exercise 10.2.1 - SQL

In this exercise, you’ll be writing a tool use prompt for querying and writing to the world’s smallest
“database”. Here’s the initialized database, which is really just a dictionary.
[ ]: db = {
"users": [
{"id": 1, "name": "Alice", "email": "[email protected]"},
{"id": 2, "name": "Bob", "email": "[email protected]"},
{"id": 3, "name": "Charlie", "email": "[email protected]"}
],
"products": [
{"id": 1, "name": "Widget", "price": 9.99},
{"id": 2, "name": "Gadget", "price": 14.99},
{"id": 3, "name": "Doohickey", "price": 19.99}
]
}

And here is the code for the functions that write to and from the database.
[ ]: def get_user(user_id):
for user in db["users"]:
if user["id"] == user_id:
return user
return None

def get_product(product_id):
for product in db["products"]:
if product["id"] == product_id:
return product
return None

def add_user(name, email):

user_id = len(db["users"]) + 1

7
 user = {"id": user_id, "name": name, "email": email}
db["users"].append(user)
return user

def add_product(name, price):

product_id = len(db["products"]) + 1
product = {"id": product_id, "name": name, "price": price}
db["products"].append(product)
return product

To solve the exercise, start by defining a system prompt like

system_prompt_tools_specific_tools above. Make sure to include the name and de-
scription of each tool, along with the name and type and description of each parameter for each
function. We’ve given you some starting scaffolding below.
[ ]: system_prompt_tools_specific_tools_sql = """
"""

system_prompt = system_prompt_tools_general_explanation +␣
↪system_prompt_tools_specific_tools_sql

When you’re ready, you can try out your tool definition system prompt on the examples below.
Just run the below cell!
[ ]: examples = [
"Add a user to the database named Deborah.",
"Add a product to the database named Thingo",
"Tell me the name of User 2",
"Tell me the name of Product 3"
]

for example in examples:

message = {
"role": "user",
"content": example
}

# Get & print Claude's response

function_calling_response = get_completion([message],␣
↪system_prompt=system_prompt, stop_sequences=stop_sequences)

print(example, "\n----------\n\n", function_calling_response,␣

↪"\n*********\n*********\n*********\n\n")

If you did it right, the function calling messages should call the add_user, add_product, get_user,
and get_product functions correctly.
For extra credit, add some code cells and write parameter-parsing code. Then call the functions
with the parameters Claude gives you to see the state of the “database” after the call.

8
 � If you want to see a possible solution, run the cell below!
[ ]: from hints import exercise_10_2_1_solution; print(exercise_10_2_1_solution)

1.3.2 Congrats!
Congratulations on learning tool use and function calling! Head over to the last appendix section
if you would like to learn more about search & RAG.

1.4 Example Playground

This is an area for you to experiment freely with the prompt examples shown in this lesson and
tweak prompts to see how it may affect Claude’s responses.
[ ]: system_prompt_tools_general_explanation = """You have access to a set of␣
↪functions you can use to answer the user's question. This includes access to␣

↪a

sandboxed computing environment. You do NOT currently have the ability to␣
↪inspect files or interact with external

resources, except by invoking the below functions.

You can invoke one or more functions by writing a "<function_calls>" block like␣
↪the following as part of your

reply to the user:

<function_calls>
<invoke name="$FUNCTION_NAME">
<antml:parameter name="$PARAMETER_NAME">$PARAMETER_VALUE</parameter>
...
</invoke>
<nvoke name="$FUNCTION_NAME2">
...
</invoke>
</function_calls>

String and scalar parameters should be specified as is, while lists and objects␣
↪should use JSON format. Note that

spaces for string values are not stripped. The output is not expected to be␣
↪valid XML and is parsed with regular

expressions.

The output and/or any errors will appear in a subsequent "<function_results>"␣

↪block, and remain there as part of

your reply to the user.

You may then continue composing the rest of your reply to the user, respond to␣
↪any errors, or make further function

calls as appropriate.

9
 If a "<function_results>" does NOT appear after your function calls, then they␣
↪are likely malformatted and not

recognized as a call."""

[ ]: system_prompt_tools_specific_tools = """Here are the functions available in␣

↪JSONSchema format:

<tools>
<tool_description>
<tool_name>calculator</tool_name>
<description>
Calculator function for doing basic arithmetic.
Supports addition, subtraction, multiplication
</description>
<parameters>
<parameter>
<name>first_operand</name>
<type>int</type>
<description>First operand (before the operator)</description>
</parameter>
<parameter>
<name>second_operand</name>
<type>int</type>
<description>Second operand (after the operator)</description>
</parameter>
<parameter>
<name>operator</name>
<type>str</type>
<description>The operation to perform. Must be either +, -, *, or /</
↪description>

</parameter>
</parameters>
</tool_description>
</tools>
"""

system_prompt = system_prompt_tools_general_explanation +␣
↪system_prompt_tools_specific_tools

[ ]: multiplication_message = {
"role": "user",
"content": "Multiply 1,984,135 by 9,343,116"
}

stop_sequences = ["</function_calls>"]

# Get Claude's response

10
 function_calling_response = get_completion([multiplication_message],␣
↪system_prompt=system_prompt, stop_sequences=stop_sequences)

print(function_calling_response)

[ ]: def do_pairwise_arithmetic(num1, num2, operation):

if operation == '+':
return num1 + num2
elif operation == "-":
return num1 - num2
elif operation == "*":
return num1 * num2
elif operation == "/":
return num1 / num2
else:
return "Error: Operation not supported."

[ ]: def find_parameter(message, parameter_name):

parameter_start_string = f"name=\"{parameter_name}\">"
start = message.index(parameter_start_string)
if start == -1:
return None
if start > 0:
start = start + len(parameter_start_string)
end = start
while message[end] != "<":
end += 1
return message[start:end]

first_operand = find_parameter(function_calling_response, "first_operand")

second_operand = find_parameter(function_calling_response, "second_operand")
operator = find_parameter(function_calling_response, "operator")

if first_operand and second_operand and operator:

result = do_pairwise_arithmetic(int(first_operand), int(second_operand),␣
↪operator)

print("---------------- RESULT ----------------")

print(f"{result:,}")

[ ]: def construct_successful_function_run_injection_prompt(invoke_results):
constructed_prompt = (
"<function_results>\n"
+ '\n'.join(
f"<result>\n<tool_name>{res['tool_name']}</
↪tool_name>\n<stdout>\n{res['tool_result']}\n</stdout>\n</result>"

for res in invoke_results

) + "\n</function_results>"
)

11
 return constructed_prompt

formatted_results = [{
'tool_name': 'do_pairwise_arithmetic',
'tool_result': result
}]
function_results =␣
↪construct_successful_function_run_injection_prompt(formatted_results)

print(function_results)

[ ]: full_first_response = function_calling_response + "</function_calls>"

# Construct the full conversation

messages = [multiplication_message,
{
"role": "assistant",
"content": full_first_response
},
{
"role": "user",
"content": function_results
}]

# Print Claude's response

final_response = get_completion(messages, system_prompt=system_prompt,␣
↪stop_sequences=stop_sequences)

print("------------- FINAL RESULT -------------")

print(final_response)

[ ]: non_multiplication_message = {
"role": "user",
"content": "Tell me the capital of France."
}

stop_sequences = ["</function_calls>"]

# Get Claude's response

function_calling_response = get_completion([non_multiplication_message],␣
↪system_prompt=system_prompt, stop_sequences=stop_sequences)

print(function_calling_response)

12