Chapter 05 · Tool Use & Function Calling
An LLM without tools is a philosopher. An LLM with tools is an engineer. This is the chapter where your agent stops talking and starts doing.
You built an agent in Chapter 4. It can hold a conversation, maintain state across turns, and reason about complex topics. Ask it about Tokyo travel and it will give you a thoughtful, detailed answer — the best neighborhoods, the must-see temples, the optimal time to visit.
There is one problem. Every fact in that answer came from the model's training data, which is months old. Your agent does not know the current weather in Tokyo. It cannot tell you what a flight costs right now. It has no idea whether the restaurant it just recommended closed last week. It is working from a frozen snapshot of the world, and that snapshot is getting staler by the day.
Here is the same query, before and after tools.
Without tools:
User: "I'm flying to Tokyo next Tuesday. What's the weather
forecast? And find me a flight under $800 from SFO."
Agent: "Tokyo in late July is typically hot and humid, with
temperatures around 30°C (86°F) and a chance of
afternoon thunderstorms. As for flights, prices from
SFO to NRT typically range from $700 to $1,200
depending on the season and how far in advance you
book. I'd recommend checking Google Flights or Kayak
for current pricing."
That answer is articulate, well-structured, and completely useless for making an actual decision. The weather is a seasonal generalization. The flight prices are a historical range. The agent cannot tell you whether to pack a raincoat or whether you should book now or wait.
With tools:
User: "I'm flying to Tokyo next Tuesday. What's the weather
forecast? And find me a flight under $800 from SFO."
Agent: [calls get_weather(city="Tokyo", date="2026-08-04")]
→ "29°C, 65% humidity, scattered thunderstorms after 3 PM.
Pack a light rain jacket."
Agent: [calls search_flights(origin="SFO", destination="NRT",
date="2026-08-04", max_price=800)]
→ "Found 3 flights under $800:
- Zipair: $520, depart 10:30 AM, 10h 45m, 1 stop
- United: $745, depart 11:15 AM, 11h 20m, direct
- JAL: $790, depart 1:05 PM, 11h 05m, direct"
Agent: "Here's what I found. Tokyo next Tuesday will be 29°C
with scattered thunderstorms after 3 PM — bring a rain
jacket. For flights from SFO under $800, there are three
options. The Zipair flight at $520 is the cheapest but
has a stop. United at $745 is direct and a solid choice.
JAL at $790 is also direct and typically has better
service. If you want the best value, I'd go with United
— direct flight, reasonable price, and you'll arrive
before the storms hit."
That is the difference. The first agent is a philosopher — it can reason about the world but cannot touch it. The second agent is an engineer — it reaches out, gets real data, and acts on it. This chapter is about building the second agent.
Section 1: How Tool Calling Works
The tool-calling protocol is simple enough to describe in a paragraph, but the implications take a chapter to unpack. Here is the protocol:
You send the model a prompt, a conversation history, and a list of tool definitions. The model looks at everything and makes a binary decision: respond with text, or respond with a tool call. If it is a tool call, you execute the function and send the result back. The model sees the result and decides again: text or another tool call. This repeats until the model produces text — the final answer.
The critical word in that description is you. You execute the function. Not the model. The model has never run a line of code in its existence. It produces tokens that describe which function to call and with what arguments. Your code reads those tokens, calls the function, and feeds the result back into the conversation.
The LLM does not execute tools. It requests tool execution. Your code executes them. This separation is the foundation of security, control, and reliability in agent systems.
Here is the flow visualized for a single tool call:
┌─────────────────────────────────────────────────────────┐
│ User: "What's the weather in Tokyo?" │
└────────────────────────┬────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────┐
│ You send to LLM: │
│ - System prompt │
│ - User message │
│ - Tool definitions (including get_weather) │
└────────────────────────┬────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────┐
│ LLM responds: │
│ tool_call( │
│ name="get_weather", │
│ arguments={"city": "Tokyo"} │
│ ) │
└────────────────────────┬────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────┐
│ Your code executes: │
│ result = get_weather(city="Tokyo") │
│ → "22°C, partly cloudy, humidity 55%" │
└────────────────────────┬────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────┐
│ You send back to LLM: │
│ role: "tool" │
│ content: "22°C, partly cloudy, humidity 55%" │
└────────────────────────┬────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────┐
│ LLM responds: │
│ "The weather in Tokyo is 22°C and partly cloudy │
│ with 55% humidity." │
└─────────────────────────────────────────────────────────┘
The model never touched a weather API. It never made an HTTP request. It produced a structured output that said "call get_weather with city=Tokyo," and your code did the rest. This is not a limitation — it is a design choice. It means you control what gets executed, how it gets executed, and what the model sees afterward.
Terminology
You will see three terms used interchangeably across providers: tool calling, function calling, and tool use. They all mean the same thing: the model produces a structured request to invoke a function, and your code executes it. OpenAI calls it "function calling." Anthropic calls it "tool use." The concept is identical. This book uses "tool calling" and "tool use" interchangeably.
Section 2: Defining Tools
A tool definition has three parts: a name, a description, and a parameters schema. The name is how your code identifies the function. The description is how the model decides when to use it. The parameters schema tells the model what arguments to pass.
Here is the thing most developers get wrong on their first attempt: the description is a prompt. It is not documentation for humans. It is instructions for a language model that is trying to decide, in real time, whether this function is the right one to call. Write it accordingly.
Bad description:
"description": "Web search function."
The model learns nothing from this. When should it use this function? What kind of results does it return? What makes it different from the other 15 tools you defined?
Good description:
"description": "Search the web for current information. Use this when you need facts that may have changed since your training data cutoff, or when the user asks about recent events, news, prices, weather, or any time-sensitive information. Returns a list of results with titles, URLs, and snippets."
Now the model knows what this tool does, when to reach for it, and what it will get back. The description is doing real work — it is guiding the model's decision-making.
The same principle applies to parameter descriptions. Every parameter is an opportunity to tell the model exactly what you expect.
Bad:
"city": {"type": "string", "description": "city"}
Good:
"city": {
"type": "string",
"description": "The city name in English, e.g. 'Tokyo' not '東京', 'New York' not 'NYC'. Include the country if ambiguous, e.g. 'Paris, France' not just 'Paris'."
}
The good description prevents the model from passing non-English characters, ambiguous abbreviations, or city names that could refer to multiple places. It costs you ten extra seconds of typing and saves you hours of debugging.
The Complete Tool Definition Schema
Here are four complete tool definitions. Study the descriptions. Notice how each one tells the model not just what the tool does, but when to use it and what to expect.
TOOLS = [
{
"type": "function",
"function": {
"name": "web_search",
"description": (
"Search the web for current information. Use this when you need "
"facts that may have changed since your training data, or when the "
"user asks about recent events, news, prices, weather, or any "
"time-sensitive information. Returns up to 10 results with titles, "
"URLs, and text snippets."
),
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": (
"The search query. Be specific. Use keywords, not full "
"sentences. Example: 'Tokyo weather forecast August 2026' "
"not 'What is the weather going to be like in Tokyo next month?'"
)
},
"num_results": {
"type": "integer",
"description": "Number of results to return, between 1 and 10. Default 5.",
"default": 5
}
},
"required": ["query"]
}
}
},
{
"type": "function",
"function": {
"name": "calculator",
"description": (
"Evaluate a mathematical expression. Use this for arithmetic, "
"percentages, unit conversions, and any calculation more complex "
"than mental math. Supports +, -, *, /, **, %, and parentheses. "
"Example expressions: '850 * 0.15', '(100 + 50) / 3', '2**10'."
),
"parameters": {
"type": "object",
"properties": {
"expression": {
"type": "string",
"description": (
"A mathematical expression using numbers and operators "
"(+, -, *, /, **, %). No variables, no function calls. "
"Example: '4500 * 1.08' to calculate 8% tax on $4,500."
)
}
},
"required": ["expression"]
}
}
},
{
"type": "function",
"function": {
"name": "get_weather",
"description": (
"Get the current weather or forecast for a city. Use this when the "
"user asks about temperature, conditions, humidity, wind, or "
"precipitation. Returns temperature, conditions, humidity, wind speed, "
"and a short forecast summary."
),
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": (
"City name in English. Use the full, standard name: "
"'San Francisco' not 'SF', 'New York' not 'NYC'. "
"Include country if ambiguous."
)
},
"units": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature unit. Default is celsius."
}
},
"required": ["city"]
}
}
},
{
"type": "function",
"function": {
"name": "query_database",
"description": (
"Run a SQL query against the local SQLite database. Use this when "
"the user asks about data stored in the database — customer records, "
"sales figures, inventory, or any structured data. Only SELECT queries "
"are allowed. Returns results as a list of rows."
),
"parameters": {
"type": "object",
"properties": {
"sql": {
"type": "string",
"description": (
"A SELECT SQL query to execute. Use parameterized syntax "
"with ? placeholders for values. Example: "
"'SELECT * FROM customers WHERE city = ?' with params=['Tokyo']. "
"Only SELECT statements are permitted."
)
},
"params": {
"type": "array",
"items": {"type": "string"},
"description": "Parameters to substitute into the query's ? placeholders, in order."
}
},
"required": ["sql"]
}
}
}
]
Common Mistakes in Tool Definitions
Missing descriptions. A tool with no description is invisible to the model. It will never be called because the model has no idea what it does. Every tool needs a description. Every parameter needs a description.
Wrong types. If a parameter is a number, use "type": "number" or "type": "integer", not "type": "string". The model uses the type to decide what values are valid. If you mark a price as a string, the model might pass "about fifty dollars" instead of 50.
Ambiguous parameter names. A parameter called "input" tells the model nothing. A parameter called "search_query" tells the model exactly what to put there. Name parameters for what they represent, not for their position in the function signature.
No required fields. If a parameter is essential, mark it as required. The model will sometimes omit optional parameters even when they would be useful. Required fields force the model to provide them.
Overloading one tool. A single tool called "do_everything" with 15 parameters is worse than five focused tools with three parameters each. The model has to reason about which parameters to use in which combination. Give it smaller, sharper tools.
Section 3: Integrating Tools into the Agent Loop
In Chapter 4, your agent loop was simple: send a message, get a response, append both to history, repeat. Adding tools changes the loop in one fundamental way: the model's response is no longer always text. Sometimes it is a tool call, and when it is, you have to execute the tool and feed the result back before the conversation can continue.
Here is the updated loop:
1. Send prompt + conversation + tool definitions to LLM
2. If LLM returns text → it's the final answer. Done.
3. If LLM returns tool_call(s) → execute each tool, add results
to conversation, go to step 1.
That is the entire change. Three steps instead of two. But step 3 is where everything interesting happens — parallel tool calls, error handling, iteration limits, and result formatting. Let us build it.
The Complete Tool-Using Agent Loop
import json
from openai import OpenAI
client = OpenAI()
class ToolAgent:
def __init__(self, system_prompt, tools, max_iterations=10):
self.system_prompt = system_prompt
self.tools = tools
self.tool_map = {t["function"]["name"]: t for t in tools}
self.max_iterations = max_iterations
def run(self, user_message):
messages = [
{"role": "system", "content": self.system_prompt},
{"role": "user", "content": user_message}
]
for iteration in range(self.max_iterations):
response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
tools=self.tools,
tool_choice="auto"
)
message = response.choices[0].message
# Case 1: Model produced text — we're done.
if message.content and not message.tool_calls:
return message.content
# Case 2: Model wants to call tools.
if message.tool_calls:
# Add the assistant message (with tool calls) to history.
messages.append({
"role": "assistant",
"content": message.content,
"tool_calls": [
{
"id": tc.id,
"type": "function",
"function": {
"name": tc.function.name,
"arguments": tc.function.arguments
}
}
for tc in message.tool_calls
]
})
# Execute each tool call and add results.
for tool_call in message.tool_calls:
tool_name = tool_call.function.name
tool_args = json.loads(tool_call.function.arguments)
print(f" [TOOL] {tool_name}({tool_args})")
result = self._execute_tool(tool_name, tool_args)
messages.append({
"role": "tool",
"tool_call_id": tool_call.id,
"content": result
})
# Loop continues — model sees tool results and decides next step.
return "Agent exceeded maximum iterations without producing a final answer."
def _execute_tool(self, name, args):
"""Execute a tool by name. Override or extend this in subclasses."""
if name == "calculator":
return self._safe_eval(args["expression"])
elif name == "web_search":
return self._web_search(args["query"], args.get("num_results", 5))
elif name == "get_weather":
return self._get_weather(args["city"], args.get("units", "celsius"))
elif name == "query_database":
return self._query_db(args["sql"], args.get("params", []))
else:
return f"Error: Unknown tool '{name}'. Available tools: {list(self.tool_map.keys())}"
def _safe_eval(self, expression):
"""Safely evaluate a mathematical expression."""
import ast
import operator
allowed_ops = {
ast.Add: operator.add,
ast.Sub: operator.sub,
ast.Mult: operator.mul,
ast.Div: operator.truediv,
ast.Pow: operator.pow,
ast.Mod: operator.mod,
ast.USub: operator.neg,
}
def _eval(node):
if isinstance(node, ast.Expression):
return _eval(node.body)
elif isinstance(node, ast.Constant):
return node.value
elif isinstance(node, ast.BinOp):
op = allowed_ops.get(type(node.op))
if op is None:
raise ValueError(f"Operator not allowed: {type(node.op).__name__}")
return op(_eval(node.left), _eval(node.right))
elif isinstance(node, ast.UnaryOp):
op = allowed_ops.get(type(node.op))
if op is None:
raise ValueError(f"Unary operator not allowed: {type(node.op).__name__}")
return op(_eval(node.operand))
else:
raise ValueError(f"Expression type not allowed: {type(node).__name__}")
try:
tree = ast.parse(expression, mode="eval")
return str(_eval(tree))
except Exception as e:
return f"Error evaluating expression: {e}"
def _web_search(self, query, num_results):
"""Stub — real implementation in Section 6."""
return f"[Web search for '{query}' would return {num_results} results]"
def _get_weather(self, city, units):
"""Stub — real implementation in Section 6."""
return f"[Weather for {city} in {units} would be fetched here]"
def _query_db(self, sql, params):
"""Stub — real implementation in Section 6."""
return f"[Database query: {sql} with params {params}]"
That is 120 lines. It handles the full tool-calling protocol: sending tool definitions, detecting tool calls, executing them, formatting results, and looping until the model produces a final answer. The max_iterations guard prevents infinite loops — if the model calls tools 10 times without producing text, the agent stops and reports the failure.
Parallel Tool Calls
The model can request multiple tools in a single response. When the user asks "What is the weather in Tokyo and what is the weather in London?", the model can call get_weather twice in parallel rather than sequentially. The loop above handles this naturally — it iterates over message.tool_calls, which is a list. All tool results are added to the conversation before the next model call, so the model sees all results at once.
The Tool Result Format
The format of tool results matters. The model reads the result and uses it to decide what to do next. If the result is a raw JSON blob with 50 fields, the model has to parse it and extract what is relevant. If the result is a clean, human-readable summary, the model can use it immediately.
A good tool result is:
- Concise. Include the information the model needs, not everything the API returned.
- Structured. Use clear formatting — bullet points, labeled fields, or a short paragraph.
- Actionable. The model should be able to use the result directly in its response to the user.
For example, a weather API might return 200 lines of JSON with hourly forecasts, UV indices, and pollen counts. Your tool function should extract the relevant fields and return a clean summary:
Tokyo: 22°C, partly cloudy, humidity 55%, wind 12 km/h.
Forecast: clearing by evening. High of 24°C, low of 18°C.
A Multi-Tool Trace
Here is what the loop looks like in practice. The user asks a question that requires two tools:
User: "What's the weather in Tokyo, and what's 15% of the
average daily budget of ¥12,000?"
--- Iteration 1 ---
[TOOL] get_weather({'city': 'Tokyo', 'units': 'celsius'})
[TOOL] calculator({'expression': '12000 * 0.15'})
--- Iteration 2 ---
Model sees:
- Weather result: "Tokyo: 22°C, partly cloudy, humidity 55%"
- Calculator result: "1800.0"
Model responds:
"The weather in Tokyo is 22°C and partly cloudy with 55%
humidity. 15% of your ¥12,000 daily budget is ¥1,800."
Two tool calls in one iteration. The model saw both results, synthesized them, and produced a single coherent answer. This is the pattern that makes tool-using agents powerful — the model decides what information it needs, gathers it in parallel, and weaves it together.
Section 4: Tool Selection Strategies
The model decides which tool to call. But you decide which tools the model can see. This is a lever you can pull, and pulling it correctly is the difference between an agent that reliably calls the right tool and one that gets lost in a sea of options.
The "Too Many Tools" Problem
Give the model 5 tools and it will choose correctly most of the time. Give it 50 tools and it will get confused. The descriptions blur together. The model calls search_customer_database when it should call search_product_catalog. It calls a tool that almost fits instead of the one that exactly fits. It calls nothing at all because the choice is overwhelming.
The solution is not to give the model fewer tools. The solution is to give the model the right tools for the current context.
Strategy 1: Tool Filtering
Before sending the request, filter the tool list based on the user's query. If the user is asking about weather, include the weather tool and exclude the database tool. If the user is asking about sales data, include the database tool and exclude the weather tool.
def filter_tools(query, all_tools):
"""Simple keyword-based tool filtering."""
query_lower = query.lower()
relevant = []
if any(w in query_lower for w in ["weather", "temperature", "forecast", "rain"]):
relevant.append("get_weather")
if any(w in query_lower for w in ["search", "find", "look up", "news", "current"]):
relevant.append("web_search")
if any(w in query_lower for w in ["calculate", "math", "percent", "sum", "total"]):
relevant.append("calculator")
if any(w in query_lower for w in ["database", "sql", "query", "customer", "sales"]):
relevant.append("query_database")
# If nothing matched, include all tools as fallback.
if not relevant:
return all_tools
return [t for t in all_tools if t["function"]["name"] in relevant]
This is crude but effective. A more sophisticated version uses embeddings to find semantically similar tools, or a fast classifier model to select tools. The principle is the same: reduce the choice set to what is relevant.
Strategy 2: The Router Pattern
Use a fast, cheap model to decide which tools to include, then use a powerful, expensive model to use them.
def route_and_execute(query, all_tools, fast_model, powerful_model):
# Step 1: Fast model selects relevant tools.
route_prompt = (
"Given the user's query, which of these tools are relevant? "
"Return a JSON list of tool names.\n\n"
f"Query: {query}\n\n"
f"Available tools: {[t['function']['name'] for t in all_tools]}"
)
route_response = fast_model.chat(route_prompt)
relevant_names = json.loads(route_response)
# Step 2: Filter tools.
relevant_tools = [
t for t in all_tools if t["function"]["name"] in relevant_names
]
# Step 3: Powerful model uses the filtered tools.
return powerful_model.chat(query, tools=relevant_tools)
The fast model costs a fraction of a cent per call. The powerful model costs more but only has to reason about 3-5 tools instead of 50. The total cost is lower and the accuracy is higher.
Strategy 3: Tool Categories
Group tools into categories and include categories based on context. Instead of 50 individual tools, the model sees 5 categories, each containing related tools. The model picks a category, and a second pass resolves the specific tool.
This is more complex to implement but scales to hundreds of tools. It is the pattern used by large production agent systems.
Controlling Tool Choice Behavior
The tool_choice parameter controls whether the model must call a tool, may call a tool, or must not call a tool.
"auto" (default): The model decides. It can produce text or a tool call. Use this when the user's query might or might not need a tool — you trust the model to figure it out.
"required": The model must call at least one tool. Use this when you know the task requires tool use — for example, when the user's query is being routed through a system that always needs a function call. If the model would normally just answer, it will instead call a tool, even if the tool call is unnecessary. This can force the model into awkward tool calls, so use it sparingly.
"none": The model must not call any tools. Use this when you want a pure text response — for example, when generating a summary of a conversation that has already gathered all necessary data.
Forcing a specific tool: You can set tool_choice to a specific tool name. The model will call that tool and no other. Use this when you know exactly which tool should be used — for example, in a pipeline where step 3 is always a database query.
# Force the model to use the calculator.
response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
tools=tools,
tool_choice={"type": "function", "function": {"name": "calculator"}}
)
The general rule: use "auto" unless you have a specific reason not to. The model is better at deciding when to use tools than most developers give it credit for. Only constrain it when you have evidence that the unconstrained behavior is wrong.
Section 5: Error Handling in Tool Use
Tools fail. APIs return 500s. Networks time out. The model hallucinates a tool name that does not exist. The model passes a string where a number is expected. The model calls the same failing tool six times in a row.
Your agent must handle all of these gracefully. The pattern is the same in every case: catch the error, format it as a tool result, and let the model decide what to do next. The model is surprisingly good at recovering from errors when you give it clear error messages.
Failure Mode 1: Unknown Tool
The model invents a tool name. It happens — the model hallucinates "send_email" when you never defined that tool.
def _execute_tool(self, name, args):
if name not in self.tool_map:
available = ", ".join(self.tool_map.keys())
return (
f"Error: Tool '{name}' does not exist. "
f"Available tools: {available}. "
f"Please use one of the available tools or answer without tools."
)
# ... proceed with execution
The model sees this error and typically apologizes, then either calls the correct tool or answers without tools. The key is telling it what is available, not just what is not.
Failure Mode 2: Invalid Parameters
The model passes a string where a number is expected, or omits a required parameter, or passes a value outside the valid range.
def _execute_tool(self, name, args):
tool_def = self.tool_map[name]
required = tool_def["function"]["parameters"].get("required", [])
# Check required parameters.
missing = [p for p in required if p not in args]
if missing:
return (
f"Error: Missing required parameters: {missing}. "
f"Please provide values for these parameters and try again."
)
# Validate types (basic check).
props = tool_def["function"]["parameters"].get("properties", {})
for param_name, param_value in args.items():
if param_name in props:
expected_type = props[param_name].get("type")
if expected_type == "integer" and not isinstance(param_value, int):
return (
f"Error: Parameter '{param_name}' must be an integer. "
f"Received: {type(param_value).__name__} = {param_value}. "
f"Please provide an integer value."
)
if expected_type == "number" and not isinstance(param_value, (int, float)):
return (
f"Error: Parameter '{param_name}' must be a number. "
f"Received: {type(param_value).__name__} = {param_value}."
)
# ... proceed with execution
The model sees the validation error, corrects its mistake, and tries again. This is one of the most important patterns in agent engineering: validate before executing, and return structured errors the model can act on.
Failure Mode 3: Tool Execution Failure
The API is down. The network times out. The database is locked. The tool function itself fails.
def _execute_tool(self, name, args):
try:
if name == "web_search":
return self._web_search(args["query"], args.get("num_results", 5))
elif name == "get_weather":
return self._get_weather(args["city"], args.get("units", "celsius"))
# ... other tools
except TimeoutError:
return (
f"Error: The {name} tool timed out. The external service may be "
f"unavailable. Try a different approach or inform the user that "
f"this data is currently unavailable."
)
except Exception as e:
return (
f"Error: The {name} tool failed with: {str(e)}. "
f"If this is a transient error, you can retry with different "
f"parameters. If the error persists, use an alternative approach."
)
The model reads the error, understands that the tool is unavailable, and adapts. It might try a different tool, reformulate the query, or tell the user that the information is currently unavailable. This is the error-as-result pattern: every failure becomes information the model can reason about.
Failure Mode 4: Tool-Calling Loop
The model calls a tool, gets a result, calls the same tool again with slightly different parameters, gets a similar result, and keeps going. It is stuck in a loop.
class ToolAgent:
def __init__(self, system_prompt, tools, max_iterations=10):
# ...
self.max_iterations = max_iterations
self.consecutive_same_tool = 0
self.last_tool_called = None
def run(self, user_message):
# ...
for iteration in range(self.max_iterations):
# ...
if message.tool_calls:
tool_name = message.tool_calls[0].function.name
# Detect loops: same tool called repeatedly.
if tool_name == self.last_tool_called:
self.consecutive_same_tool += 1
else:
self.consecutive_same_tool = 1
self.last_tool_called = tool_name
if self.consecutive_same_tool > 3:
messages.append({
"role": "user",
"content": (
f"You have called '{tool_name}' {self.consecutive_same_tool} "
f"times in a row without making progress. Stop calling this "
f"tool and either use a different tool, synthesize what you "
f"have, or tell the user what you know so far."
)
})
self.consecutive_same_tool = 0
continue # Skip tool execution, let model reconsider.
# ...
The loop detector injects a message telling the model to stop and reconsider. This breaks the cycle without crashing the agent. The model reads the nudge, realizes it is stuck, and changes course.
Failure Mode 5: Model Never Calls a Tool
The user asks a question that clearly requires a tool — "What is the current stock price of AAPL?" — and the model answers from its training data instead of calling the search tool.
The fix is tool_choice: "required" for queries that you know need tools. But you do not always know in advance. A better approach is a post-hoc check:
def run(self, user_message):
# First attempt: let the model decide.
response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
tools=self.tools,
tool_choice="auto"
)
message = response.choices[0].message
# If the model answered without tools but the answer contains
# hedging language, force a tool call.
hedging_phrases = [
"as of my knowledge cutoff",
"as of my training data",
"I don't have real-time",
"I cannot access current",
"you should check",
]
if message.content and not message.tool_calls:
if any(phrase in message.content.lower() for phrase in hedging_phrases):
# Model knows it is guessing. Force tool use.
response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
tools=self.tools,
tool_choice="required"
)
# ... process the forced tool call
This is a heuristic, not a guarantee. But it catches the most common case: the model knows it is working from stale data and says so, but does not take the initiative to call a tool. Forcing the call gives the user real data instead of a hedged guess.
Section 6: Building Real Tools
The stubs in the agent loop above return placeholder strings. Now you will build three real tools that actually do something: a web search tool, a calculator tool, and a database query tool.
Tool 1: Web Search
You will use the Brave Search API. It is free for up to 2,000 queries per month, requires no credit card for the free tier, and returns clean, structured results. Sign up at https://brave.com/search/api/ to get an API key.
import os
import requests
def web_search(query, num_results=5):
"""
Search the web using the Brave Search API.
Args:
query: The search query string.
num_results: Number of results to return (1-10).
Returns:
A formatted string with search results.
"""
api_key = os.environ.get("BRAVE_API_KEY")
if not api_key:
return "Error: BRAVE_API_KEY environment variable not set."
try:
response = requests.get(
"https://api.search.brave.com/res/v1/web/search",
headers={
"Accept": "application/json",
"Accept-Encoding": "gzip",
"X-Subscription-Token": api_key,
},
params={
"q": query,
"count": min(num_results, 10),
},
timeout=10,
)
response.raise_for_status()
data = response.json()
results = data.get("web", {}).get("results", [])
if not results:
return f"No results found for '{query}'."
lines = []
for i, r in enumerate(results[:num_results], 1):
title = r.get("title", "No title")
url = r.get("url", "No URL")
description = r.get("description", "No description")
lines.append(f"{i}. {title}\n URL: {url}\n {description}\n")
return "\n".join(lines)
except requests.Timeout:
return f"Error: Search timed out for query '{query}'. Try a more specific query."
except requests.RequestException as e:
return f"Error: Search failed: {str(e)}"
The function is straightforward: make an HTTP request, parse the JSON, format the results. The formatting is the part that matters for the agent — the model needs clean, scannable results it can use immediately.
Tool 2: Calculator
The calculator tool from Section 3 used ast.parse for safe evaluation. Here is why that matters and how it works.
import ast
import operator
import math
def safe_calculator(expression):
"""
Safely evaluate a mathematical expression.
Uses Python's AST module to parse the expression and evaluate it
with a whitelist of allowed operators. This prevents arbitrary code
execution — only mathematical operations are permitted.
Why not eval()? eval(expression) executes arbitrary Python code.
If the model produces expression="__import__('os').system('rm -rf /')",
eval() will run it. The AST approach only allows the operators we
explicitly whitelist.
"""
# Operators we allow.
allowed_ops = {
ast.Add: operator.add,
ast.Sub: operator.sub,
ast.Mult: operator.mul,
ast.Div: operator.truediv,
ast.Pow: operator.pow,
ast.Mod: operator.mod,
ast.FloorDiv: operator.floordiv,
ast.USub: operator.neg, # Unary minus: -5
ast.UAdd: operator.pos, # Unary plus: +5
}
# Functions we allow.
allowed_funcs = {
"abs": abs,
"round": round,
"min": min,
"max": max,
"sqrt": math.sqrt,
"log": math.log,
"log10": math.log10,
"sin": math.sin,
"cos": math.cos,
"tan": math.tan,
}
def _eval_node(node):
if isinstance(node, ast.Expression):
return _eval_node(node.body)
elif isinstance(node, ast.Constant):
return node.value
elif isinstance(node, ast.BinOp):
op = allowed_ops.get(type(node.op))
if op is None:
raise ValueError(f"Operator not allowed: {type(node.op).__name__}")
return op(_eval_node(node.left), _eval_node(node.right))
elif isinstance(node, ast.UnaryOp):
op = allowed_ops.get(type(node.op))
if op is None:
raise ValueError(f"Unary operator not allowed: {type(node.op).__name__}")
return op(_eval_node(node.operand))
elif isinstance(node, ast.Call):
func_name = node.func.id if isinstance(node.func, ast.Name) else None
if func_name not in allowed_funcs:
raise ValueError(f"Function not allowed: {func_name}")
args = [_eval_node(a) for a in node.args]
return allowed_funcs[func_name](*args)
else:
raise ValueError(f"Expression type not allowed: {type(node).__name__}")
try:
tree = ast.parse(expression.strip(), mode="eval")
result = _eval_node(tree)
# Format the result nicely.
if isinstance(result, float):
if result == int(result):
return str(int(result))
return f"{result:.6g}"
return str(result)
except SyntaxError as e:
return f"Error: Invalid expression syntax: {e}"
except ZeroDivisionError:
return "Error: Division by zero."
except Exception as e:
return f"Error: {e}"
The AST approach is a whitelist, not a blacklist. It starts by rejecting everything, then explicitly allows only the operations you have approved. A blacklist approach — "block these dangerous functions" — always misses something. The whitelist is exhaustive by design.
Tool 3: Database Query
The database tool lets the agent run SQL queries against a local SQLite database. The critical constraint: only SELECT queries. No INSERT, UPDATE, DELETE, DROP, or ALTER.
import sqlite3
# Global connection — in production, use a connection pool.
_db_connection = None
def get_db():
"""Get or create the database connection."""
global _db_connection
if _db_connection is None:
_db_connection = sqlite3.connect("agent_data.db")
_db_connection.row_factory = sqlite3.Row
return _db_connection
def query_database(sql, params=None):
"""
Execute a read-only SQL query against the local database.
Only SELECT statements are allowed. Parameters are passed using
parameterized queries (? placeholders) to prevent SQL injection.
Args:
sql: A SELECT SQL statement with optional ? placeholders.
params: List of values to substitute for placeholders.
Returns:
Formatted query results as a string.
"""
# Reject non-SELECT queries.
sql_stripped = sql.strip().upper()
if not sql_stripped.startswith("SELECT"):
return (
f"Error: Only SELECT queries are allowed. "
f"Received: {sql[:50]}... "
f"Use a SELECT statement to read data."
)
# Reject multiple statements (prevents "SELECT 1; DROP TABLE users;").
if ";" in sql.strip().rstrip(";"):
return (
"Error: Multiple SQL statements are not allowed. "
"Provide a single SELECT statement."
)
if params is None:
params = []
try:
conn = get_db()
cursor = conn.cursor()
cursor.execute(sql, params)
rows = cursor.fetchall()
if not rows:
return "Query returned no results."
# Format results as a table.
columns = [desc[0] for desc in cursor.description]
lines = []
# Header.
lines.append(" | ".join(columns))
lines.append("-" * len(lines[0]))
# Rows (limit to 20 to avoid flooding the context).
for row in rows[:20]:
values = [str(row[c]) for c in columns]
lines.append(" | ".join(values))
if len(rows) > 20:
lines.append(f"... and {len(rows) - 20} more rows.")
return "\n".join(lines)
except sqlite3.Error as e:
return f"Error: Database query failed: {e}"
Three security measures in this function:
-
SELECT-only enforcement. The function checks that the query starts with SELECT and rejects anything else. The model might hallucinate a DELETE statement — this check stops it.
-
Multi-statement prevention. A semicolon in the query (outside the trailing one) indicates multiple statements. The function rejects these to prevent
"SELECT 1; DROP TABLE users;"attacks. -
Parameterized queries. Values are passed separately from the SQL string using
?placeholders. This prevents SQL injection even if the model produces malicious parameter values. The database driver handles escaping.
A Complex Multi-Tool Trace
Here is the agent using all three tools to answer a compound query:
User: "Find the population of Tokyo, calculate what 15% of
that is, and search for recent news about Tokyo's
population growth."
--- Iteration 1 ---
[TOOL] web_search({'query': 'Tokyo population 2026'})
[TOOL] web_search({'query': 'Tokyo population growth news 2026'})
--- Iteration 2 ---
Model sees search results:
1. Tokyo population estimated at 14.18 million (2026)
2. Tokyo population growth slows, aging accelerates - Japan Times
[TOOL] calculator({'expression': '14180000 * 0.15'})
--- Iteration 3 ---
Model sees calculator result: 2127000.0
Model responds:
"Tokyo's population is approximately 14.18 million as of 2026.
15% of that is 2,127,000 people.
Regarding population growth, recent news indicates that
Tokyo's growth is slowing while its population ages. The
Japan Times reports that the city's demographic shift is
accelerating, with the proportion of residents over 65
reaching record levels. This mirrors Japan's broader
demographic challenges."
Three tool calls across two iterations. The model searched for two things in parallel, then did a calculation, then synthesized everything into a coherent answer. This is not a scripted workflow — the model decided what to search for, when to calculate, and how to combine the results.
Section 7: The Tool-Use Agent — Complete
Here is the complete agent, with all three real tools integrated. This is the agent you will extend in every subsequent chapter.
import ast
import json
import math
import operator
import os
import sqlite3
import requests
from openai import OpenAI
# ---------------------------------------------------------------------------
# Tool Implementations
# ---------------------------------------------------------------------------
def web_search(query, num_results=5):
"""Search the web using Brave Search API."""
api_key = os.environ.get("BRAVE_API_KEY")
if not api_key:
return "Error: BRAVE_API_KEY not set. Set the environment variable to use web search."
try:
response = requests.get(
"https://api.search.brave.com/res/v1/web/search",
headers={
"Accept": "application/json",
"Accept-Encoding": "gzip",
"X-Subscription-Token": api_key,
},
params={"q": query, "count": min(num_results, 10)},
timeout=10,
)
response.raise_for_status()
data = response.json()
results = data.get("web", {}).get("results", [])
if not results:
return f"No results found for '{query}'."
lines = []
for i, r in enumerate(results[:num_results], 1):
title = r.get("title", "No title")
url = r.get("url", "No URL")
desc = r.get("description", "No description")
lines.append(f"{i}. {title}\n URL: {url}\n {desc}\n")
return "\n".join(lines)
except requests.Timeout:
return f"Error: Search timed out for '{query}'."
except requests.RequestException as e:
return f"Error: Search failed: {e}"
def safe_calculator(expression):
"""Safely evaluate a mathematical expression using AST whitelist."""
allowed_ops = {
ast.Add: operator.add, ast.Sub: operator.sub,
ast.Mult: operator.mul, ast.Div: operator.truediv,
ast.Pow: operator.pow, ast.Mod: operator.mod,
ast.FloorDiv: operator.floordiv,
ast.USub: operator.neg, ast.UAdd: operator.pos,
}
allowed_funcs = {
"abs": abs, "round": round, "min": min, "max": max,
"sqrt": math.sqrt, "log": math.log, "log10": math.log10,
"sin": math.sin, "cos": math.cos, "tan": math.tan,
}
def _eval_node(node):
if isinstance(node, ast.Expression):
return _eval_node(node.body)
elif isinstance(node, ast.Constant):
return node.value
elif isinstance(node, ast.BinOp):
op = allowed_ops.get(type(node.op))
if op is None:
raise ValueError(f"Operator not allowed: {type(node.op).__name__}")
return op(_eval_node(node.left), _eval_node(node.right))
elif isinstance(node, ast.UnaryOp):
op = allowed_ops.get(type(node.op))
if op is None:
raise ValueError(f"Unary operator not allowed: {type(node.op).__name__}")
return op(_eval_node(node.operand))
elif isinstance(node, ast.Call):
func_name = node.func.id if isinstance(node.func, ast.Name) else None
if func_name not in allowed_funcs:
raise ValueError(f"Function not allowed: {func_name}")
args = [_eval_node(a) for a in node.args]
return allowed_funcs[func_name](*args)
else:
raise ValueError(f"Expression type not allowed: {type(node).__name__}")
try:
tree = ast.parse(expression.strip(), mode="eval")
result = _eval_node(tree)
if isinstance(result, float):
if result == int(result):
return str(int(result))
return f"{result:.6g}"
return str(result)
except SyntaxError as e:
return f"Error: Invalid expression syntax: {e}"
except ZeroDivisionError:
return "Error: Division by zero."
except Exception as e:
return f"Error: {e}"
_db_connection = None
def get_db():
global _db_connection
if _db_connection is None:
_db_connection = sqlite3.connect("agent_data.db")
_db_connection.row_factory = sqlite3.Row
return _db_connection
def query_database(sql, params=None):
"""Execute a read-only SQL query."""
sql_stripped = sql.strip().upper()
if not sql_stripped.startswith("SELECT"):
return f"Error: Only SELECT queries allowed. Received: {sql[:50]}..."
if ";" in sql.strip().rstrip(";"):
return "Error: Multiple SQL statements not allowed."
if params is None:
params = []
try:
conn = get_db()
cursor = conn.cursor()
cursor.execute(sql, params)
rows = cursor.fetchall()
if not rows:
return "Query returned no results."
columns = [desc[0] for desc in cursor.description]
lines = [" | ".join(columns), "-" * len(lines[0])]
for row in rows[:20]:
lines.append(" | ".join(str(row[c]) for c in columns))
if len(rows) > 20:
lines.append(f"... and {len(rows) - 20} more rows.")
return "\n".join(lines)
except sqlite3.Error as e:
return f"Error: Database query failed: {e}"
# ---------------------------------------------------------------------------
# Tool Definitions
# ---------------------------------------------------------------------------
TOOLS = [
{
"type": "function",
"function": {
"name": "web_search",
"description": (
"Search the web for current information. Use when you need facts "
"that may have changed since your training data, or when the user "
"asks about recent events, news, prices, or time-sensitive data. "
"Returns results with titles, URLs, and snippets."
),
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "Search query. Use keywords, not full sentences."
},
"num_results": {
"type": "integer",
"description": "Number of results (1-10). Default 5."
}
},
"required": ["query"]
}
}
},
{
"type": "function",
"function": {
"name": "calculator",
"description": (
"Evaluate a mathematical expression. Use for arithmetic, percentages, "
"conversions, and any calculation beyond mental math. Supports "
"+, -, *, /, **, %, sqrt, log, sin, cos, tan, abs, round, min, max."
),
"parameters": {
"type": "object",
"properties": {
"expression": {
"type": "string",
"description": "Math expression, e.g. '4500 * 1.08'."
}
},
"required": ["expression"]
}
}
},
{
"type": "function",
"function": {
"name": "query_database",
"description": (
"Run a SELECT query against the local SQLite database. Use when the "
"user asks about stored data. Only SELECT queries allowed."
),
"parameters": {
"type": "object",
"properties": {
"sql": {
"type": "string",
"description": "SELECT SQL query with ? placeholders for values."
},
"params": {
"type": "array",
"items": {"type": "string"},
"description": "Values for ? placeholders, in order."
}
},
"required": ["sql"]
}
}
}
]
# ---------------------------------------------------------------------------
# Agent
# ---------------------------------------------------------------------------
class ToolAgent:
def __init__(self, system_prompt, tools=None, max_iterations=10):
self.system_prompt = system_prompt
self.tools = tools or TOOLS
self.tool_map = {t["function"]["name"]: t for t in self.tools}
self.max_iterations = max_iterations
self.client = OpenAI()
def run(self, user_message):
messages = [
{"role": "system", "content": self.system_prompt},
{"role": "user", "content": user_message}
]
last_tool = None
same_tool_count = 0
for iteration in range(self.max_iterations):
response = self.client.chat.completions.create(
model="gpt-4o",
messages=messages,
tools=self.tools,
tool_choice="auto"
)
message = response.choices[0].message
# Text response — done.
if message.content and not message.tool_calls:
return message.content
# Tool calls.
if message.tool_calls:
# Loop detection.
current_tool = message.tool_calls[0].function.name
if current_tool == last_tool:
same_tool_count += 1
else:
same_tool_count = 1
last_tool = current_tool
if same_tool_count > 3:
messages.append({
"role": "user",
"content": (
f"You have called '{current_tool}' {same_tool_count} "
f"times in a row. Stop and use a different approach."
)
})
same_tool_count = 0
continue
# Add assistant message with tool calls.
messages.append({
"role": "assistant",
"content": message.content,
"tool_calls": [
{
"id": tc.id,
"type": "function",
"function": {
"name": tc.function.name,
"arguments": tc.function.arguments
}
}
for tc in message.tool_calls
]
})
# Execute each tool.
for tc in message.tool_calls:
name = tc.function.name
args = json.loads(tc.function.arguments)
print(f" [{iteration+1}] {name}({args})")
result = self._execute_tool(name, args)
messages.append({
"role": "tool",
"tool_call_id": tc.id,
"content": result
})
return "Agent exceeded maximum iterations."
def _execute_tool(self, name, args):
if name not in self.tool_map:
return (
f"Error: Tool '{name}' does not exist. "
f"Available: {', '.join(self.tool_map.keys())}."
)
tool_def = self.tool_map[name]
required = tool_def["function"]["parameters"].get("required", [])
missing = [p for p in required if p not in args]
if missing:
return f"Error: Missing required parameters: {missing}."
try:
if name == "web_search":
return web_search(args["query"], args.get("num_results", 5))
elif name == "calculator":
return safe_calculator(args["expression"])
elif name == "query_database":
return query_database(args["sql"], args.get("params", []))
else:
return f"Error: No handler for tool '{name}'."
except Exception as e:
return f"Error: Tool '{name}' failed: {e}"
# ---------------------------------------------------------------------------
# Run It
# ---------------------------------------------------------------------------
if __name__ == "__main__":
agent = ToolAgent(
system_prompt=(
"You are a helpful assistant with access to tools. "
"Use tools to get current, accurate information. "
"Always prefer tool results over your training data when they conflict. "
"If a tool fails, try an alternative approach or tell the user what you know."
)
)
query = (
"What is the current population of Tokyo? Calculate 15% of that number. "
"Then search for recent news about Tokyo's population trends."
)
print(f"User: {query}\n")
answer = agent.run(query)
print(f"\nAgent: {answer}")
That is the complete tool-using agent. 150 lines of Python, three real tools, error handling, loop detection, and parameter validation. It searches the web, runs calculations, queries databases, and synthesizes results into coherent answers. It is not a demo — it is a working system you can run right now.
The Turn
Your agent is no longer a philosopher. It can search the web for current information. It can run calculations safely. It can query databases. It can check the weather, look up flight prices, and find restaurant reviews. It can reach out into the world and bring back real data.
This is the threshold. Everything before this chapter was preparation — understanding the model, writing prompts, building the loop. Everything after this chapter builds on tool use. Memory systems store tool results. Planning agents chain tool calls into multi-step workflows. Multi-agent systems delegate tool execution to specialized agents. Tool use is the foundation that every advanced capability rests on.
But there is a problem, and you have probably already noticed it. Close your terminal and open it again. Run the agent. Ask it about Tokyo. It searches the web, calculates, and answers — just like before. But it does not remember that you already asked about Tokyo five minutes ago. It does not remember that you prefer celsius over fahrenheit. It does not remember that the last search for flights returned three options and you asked it to filter for direct flights only.
Every conversation starts from zero. Every query is a blank slate. The agent can reach out into the world, but it cannot remember what it found there.
Looking Ahead
Your agent can now use tools. It can search the web, run calculations, and query databases. But close your terminal and open it again — everything is gone. The agent remembers nothing.
How do you give an agent memory that persists across conversations, across days, across deployments? How do you make it remember that the user prefers metric units, that the last three searches for "Tokyo hotels" returned the same irrelevant results, that the database query from yesterday is still valid today? How do you build an agent that learns from experience instead of starting over every single time?
That is the next chapter.