An AI agent is a loop. You send a language model a conversation plus a list of tools it may call. The model replies with either a final answer or one or more tool call requests. Your code runs those tools, appends the results to the conversation, and sends the whole thing back. Repeat until the model stops asking for tools or a limit you set is reached. That loop is about twenty lines of code, and every agent framework in existence is a wrapper around it.
What makes agents hard is not the loop. It is the five things around it: tool definitions the model can choose from correctly, error results the model can recover from instead of crashing on, budgets that stop a runaway loop before it stops your credit card, approval gates in front of anything that writes, sends, deletes, or spends, and traces you can actually read at 2am when a run went sideways. Skip any one of them and you have a demo, not a system.
Key Takeaways
- The loop is trivial; the boundary is not. Everything expensive happens where the model touches your code.
- Tool descriptions are prompt engineering. The model chooses almost entirely from the name and description, so say when to call, not just what it does.
- Errors belong in the conversation. Return failures as tool results with an error flag so the model can adapt; do not throw them out of the loop.
- Four ceilings, not one. Steps, tokens, wall-clock, and money each fail differently, so cap each one and log which fired.
This guide walks the whole path: the loop in full, tool schema design, the error path, budgets, human gates, context management on long runs, what to log, the failure modes nobody puts in the demo video, and an honest comparison against frameworks and against not building an agent at all. Code uses the Anthropic Python SDK, but the shapes are the same wherever tool calling exists.
The Agent Loop, Written Out in Full
Three facts drive the loop. The model never runs your tools; it only asks. Every request carries the entire conversation, because the API is stateless. And the field that decides whether to loop again is stop_reason, not the presence of text.
Install the SDK and set a key, then the loop reads top to bottom with no framework involved.
pip install anthropic
export ANTHROPIC_API_KEY="sk-ant-..."
import anthropic
client = anthropic.Anthropic()
MAX_STEPS = 12
def run_agent(user_input, tools, dispatch):
messages = [{"role": "user", "content": user_input}]
for step in range(MAX_STEPS):
response = client.messages.create(
model="claude-opus-5",
max_tokens=8000,
system="You are a support agent. Use tools to look things up. "
"Never guess an order status.",
tools=tools,
messages=messages,
)
# Append the assistant turn VERBATIM, including tool_use blocks.
messages.append({"role": "assistant", "content": response.content})
if response.stop_reason == "pause_turn":
continue # server-side tool paused; resend to resume
if response.stop_reason != "tool_use":
return response # end_turn, max_tokens, refusal, stop_sequence
# Run every requested tool, then return ALL results in ONE message.
results = []
for block in response.content:
if block.type == "tool_use":
results.append(dispatch(block))
messages.append({"role": "user", "content": results})
raise RuntimeError(f"step budget exhausted after {MAX_STEPS} turns")
Four details in that code are load bearing, and each one is a bug people ship.
Append response.content, not the text. The assistant turn contains the tool_use blocks the API needs to match your results against. Extract just the text and the next request is rejected or, worse, silently loses the model's reasoning.
All parallel tool results go in a single user message. A model may request three tools at once. Splitting those three results across three messages trains it to stop making parallel calls, which quietly doubles your latency over time.
Branch on stop_reason. There are six values worth handling: end_turn (done), tool_use (loop again), max_tokens (truncated, raise the cap or stream), stop_sequence, pause_turn (a server-side tool hit its own iteration limit and the turn resumes if you resend), and refusal (the model declined). Code that reads response.content[0].text unconditionally breaks the first time a refusal returns an empty content array.
The loop needs an exit that is not the model's decision. The for range is the crudest possible budget, and it is still better than while True. Section four replaces it with something real.
Tool Definitions: The Schema Is the Contract
A tool definition is three fields: a name, a description, and a JSON Schema for the inputs. The model sees only those three things when deciding whether to call, so the description is prompt engineering rather than documentation.
SEARCH_ORDERS = {
"name": "search_orders",
# Say WHEN to call it, not only what it does.
"description": (
"Search the order database by customer email or order number. "
"Call this whenever the user mentions an order, invoice, refund, "
"or shipment by number or by the customer's email address. "
"Returns at most 50 rows, newest first."
),
"input_schema": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "An email address or an order number like SO-10493.",
},
"status": {
"type": "string",
"enum": ["open", "shipped", "cancelled"],
"description": "Optional filter. Omit to search all statuses.",
},
"limit": {"type": "integer", "description": "Rows to return, 1 to 50."},
},
"required": ["query"],
"additionalProperties": False,
},
# strict guarantees the arguments validate against the schema.
# It requires additionalProperties: false and a required list.
"strict": True,
}
Four rules produce tools a model uses well, and they are worth more than any prompt tuning you do later.
Describe the trigger
"Searches the DB" loses to "Call this when the user mentions an account, order, or invoice by number." The second sentence is what does the routing.
Constrain with enums
A free-text status field invites invented values. An enum makes the wrong call impossible instead of merely unlikely.
Keep the surface small
Every tool schema is serialized into every request, on every turn. One well-typed tool beats four near-duplicates on both cost and accuracy.
Return prose, not dumps
The result string enters the model's context verbatim. Fifty kilobytes of JSON for a two-line answer is the fastest way to blow a context budget.
Tool count is a tax you pay per turn. Five connected servers with fifteen tools each means seventy-five schemas in front of every message, which costs tokens and measurably degrades selection accuracy. If your tool list is growing past a couple of dozen, that is the signal to reach for dynamic tool discovery rather than a longer array. Our Model Context Protocol guide covers the standard way to package tools so the same implementation works across applications, and the structured outputs guide covers the schema half of the problem in depth.
Tool Results and Error Recovery
A tool result is a block with the matching tool_use_id, a content string, and an optional is_error flag. Failures belong in that block, not in a raised exception, because a returned error reaches the model and a thrown one does not.
This is the single most consequential decision in the whole build. When a tool fails and you return the failure in band, the model can retry with corrected arguments, fall back to a different tool, or tell the user plainly what went wrong. When you let the exception escape, the turn dies and every bit of context the model accumulated dies with it.
import json, logging
TOOL_TIMEOUT_S = 20
def dispatch(block):
"""Run one tool_use block and return a tool_result block."""
handler = HANDLERS.get(block.name)
if handler is None:
return _error(block, f"No tool named {block.name}. "
f"Available: {', '.join(HANDLERS)}.")
try:
out = handler(**block.input, timeout=TOOL_TIMEOUT_S)
return {
"type": "tool_result",
"tool_use_id": block.id,
"content": out if isinstance(out, str) else json.dumps(out)[:6000],
}
except ValueError as e:
# Bad arguments. Tell the model exactly how to fix the call.
return _error(block, f"Invalid arguments: {e}. "
"Order numbers look like SO-10493.")
except TimeoutError:
return _error(block, "The lookup timed out after 20s. "
"Try a narrower query or a smaller limit.")
except Exception as e:
logging.exception("tool %s failed", block.name)
return _error(block, f"The tool failed: {type(e).__name__}. "
"Do not retry this call unchanged.")
def _error(block, message):
return {
"type": "tool_result",
"tool_use_id": block.id,
"content": message,
"is_error": True,
}
Write error messages for the model, not the log
"Error 500" tells the model nothing and it will retry the identical call. "Invalid arguments: limit must be 1 to 50, got 500" gets a corrected call on the next turn. State what failed, what was wrong with the input, and what a valid call looks like. Where a retry cannot help, say so explicitly so the model stops trying and reports back instead.
Two failures do belong outside the loop: an expired or missing credential, and a tool that has already failed the same way three times. Neither is something the model can reason its way out of.
One more thing the dispatch above quietly does right: every tool result is truncated. An unbounded result is a context leak with a friendly face, and one oversized database dump can end a run that would otherwise have finished.
Budgets: Four Ceilings Every Agent Needs
A step cap alone is not a budget. An agent can exhaust a month of spend in eight expensive steps, or run for forty cheap ones and blow a latency SLA. Cap steps, tokens, wall-clock time, and money separately, and record which ceiling fired.
import time
from dataclasses import dataclass, field
# Published Claude API rates, USD per million tokens (Aug 2026).
PRICES = {
"claude-opus-5": (5.00, 25.00),
"claude-sonnet-5": (3.00, 15.00),
"claude-haiku-4-5": (1.00, 5.00),
}
@dataclass
class Budget:
max_steps: int = 25
max_output_tokens: int = 120_000
max_seconds: float = 180.0
max_usd: float = 2.00
steps: int = 0
output_tokens: int = 0
usd: float = 0.0
started: float = field(default_factory=time.monotonic)
def charge(self, model, usage):
inp, out = PRICES[model]
self.steps += 1
self.output_tokens += usage.output_tokens
self.usd += (usage.input_tokens * inp + usage.output_tokens * out) / 1_000_000
def exceeded(self):
if self.steps >= self.max_steps: return "steps"
if self.output_tokens >= self.max_output_tokens: return "tokens"
if time.monotonic() - self.started >= self.max_seconds: return "clock"
if self.usd >= self.max_usd: return "spend"
return None
Call budget.charge(model, response.usage) after each API response and check budget.exceeded() at the top of each iteration. Which ceiling fires is diagnostic on its own. Hitting the step cap usually means the agent is stuck in a retry cycle. Hitting the token cap usually means a tool is returning too much. Hitting the clock usually means one slow tool is dominating. Hitting spend usually means the model is too large for the job.
There is a second, softer lever worth knowing. The Messages API supports a task budget, which tells the model how many tokens it has for the whole agentic run so it can pace itself and finish gracefully instead of being cut off mid-thought. It is a suggestion the model can see, unlike max_tokens, which is an enforced per-response cap it cannot. The minimum total is 20,000 tokens, and it goes inside output_config:
with client.beta.messages.stream(
model="claude-opus-5",
max_tokens=64000,
betas=["task-budgets-2026-03-13"],
output_config={
"effort": "high", # low | medium | high | xhigh | max
"task_budget": {"type": "tokens", "total": 64000},
},
tools=tools,
messages=messages,
) as stream:
response = stream.get_final_message()
Use both. The task budget makes the agent wrap up on its own; your hard ceilings catch the case where it does not. Note the streaming call: any request with a large max_tokens should stream, or it risks an HTTP timeout on a long turn.
Human-in-the-Loop Gates
Split your tools into read and write. Read tools run automatically. Write tools pause the loop, surface the exact arguments to a human, and run only on approval. A denial is not an exception; it is a tool result the model reads and adapts to.
Reads, with a rate limit
Search, fetch, list, read a file, run a query against a read-only replica. Reversible, cheap, and invisible if wrong.
Still bound them: a read tool that pulls a million rows is a denial-of-service against your own context window.
Anything that changes the world
Send an email, post to an API, write a file, delete a record, refund a charge, merge a branch, run an unbounded shell command.
Show the arguments, not the intent. "Send email" is not a decision; "Send to 4,812 recipients" is.
WRITE_TOOLS = {"send_email", "issue_refund", "delete_record"}
def gated_dispatch(block, approve):
if block.name in WRITE_TOOLS:
decision = approve(block.name, block.input) # your UI, Slack, or CLI prompt
if not decision.allowed:
# A denial is a normal result. Explain WHY so the model adapts
# instead of reissuing the identical call on the next turn.
return {
"type": "tool_result",
"tool_use_id": block.id,
"content": f"A human declined this action: {decision.reason}. "
"Do not retry it. Continue without this step or "
"report what you would need instead.",
"is_error": True,
}
return dispatch(block)
Two things make gates survivable in production rather than merely correct. First, batch them: an agent that pauses eleven separate times will be approved blindly by the third pause, so collect the writes and present them together where the task allows. Second, make the pause durable. If your approval step lives in process memory, a deploy in the middle of a wait loses the run. Persist the pending state keyed by an ID and resume from storage.
Everything a tool returns becomes model input
If an agent reads issues, emails, web pages, or documents, then anyone who can write to those surfaces can write instructions into your model's context. This is the reason blanket auto-approval on write tools is unsafe no matter how good your system prompt is: the attacker's text and your instructions arrive through the same channel. Keep write-capable tools behind explicit approval, and treat every retrieved document as untrusted input. Our prompt injection guide covers the attack patterns in detail.
Context Management on Long Runs
Every turn resends the entire conversation. A forty-step run pays for step one forty times, so the two levers that matter on long runs are caching what repeats and pruning what no longer earns its place.
Prompt caching is a prefix match: the request renders as tools, then system, then messages, and any byte change invalidates everything after it. Put a cache breakpoint at the end of the stable part and keep volatile content (timestamps, session IDs, per-request notes) after it. Cache reads bill at roughly a tenth of the input rate, so on an agent that reuses a large system prompt across dozens of turns this is the difference between viable and not.
response = client.messages.create(
model="claude-opus-5",
max_tokens=8000,
system=[{
"type": "text",
"text": BIG_STABLE_SYSTEM_PROMPT, # no datetime.now() in here
"cache_control": {"type": "ephemeral"},
}],
tools=tools, # deterministic order; sort it
messages=messages,
)
# If this stays 0 across repeated runs, something in the prefix is changing.
print(response.usage.cache_read_input_tokens)
The classic silent invalidator is a timestamp or a UUID interpolated into the system prompt. It changes the prefix on every single request, and no cache marker anywhere downstream can save you. Sort your tool list too, since a set iteration order that varies between processes has the same effect.
Pruning is the other half. Old tool results go stale fast: once the agent has moved on, that 4,000-token database dump from step three is pure cost. The API can clear old tool results for you rather than making you rewrite history by hand.
response = client.beta.messages.create(
model="claude-opus-5",
max_tokens=8000,
betas=["context-management-2025-06-27"],
context_management={"edits": [{"type": "clear_tool_uses_20250919"}]},
tools=tools,
messages=messages,
)
Clearing removes old turns; compaction summarizes them instead, which is what you want when the conversation itself carries information the agent still needs. They are different features with different beta flags, and reaching for the wrong one is a common mix-up. If your agent needs facts that outlive a single run, that is not a context problem at all: it is retrieval, and the RAG pipeline guide is the right starting point.
Observability: The Trace Is the Product
When an agent misbehaves, the answer is almost always visible in the sequence of tool calls. Log every step as a structured record with the run ID, the step number, the tool name, the arguments, the duration, the outcome, and the token usage. Then you can answer "what did it actually do" without rerunning anything.
import json, time, uuid
def traced(run_id, step, block, fn):
t0 = time.monotonic()
result = fn(block)
print(json.dumps({
"run_id": run_id,
"step": step,
"tool": block.name,
"args": redact(block.input), # never log raw PII or secrets
"ms": round((time.monotonic() - t0) * 1000),
"error": bool(result.get("is_error")),
"result_chars": len(result["content"]),
}))
return result
Record the request ID from each API response alongside it, and log the four token counters (input_tokens, output_tokens, cache_creation_input_tokens, cache_read_input_tokens) per step rather than per run. Total prompt size is the sum of the three input counters, so an agent whose input_tokens looks suspiciously small is probably reading heavily from cache, which is good news you would otherwise misread as a bug.
Four numbers are worth a dashboard: steps per completed task (rising means the agent is thrashing), tool error rate (rising means a dependency drifted), cache hit rate (falling means something started varying in your prefix), and cost per completed task (the only number that decides whether this ships). Pair those with an offline evaluation set of real tasks and expected outcomes, because production traces tell you what happened while evals tell you whether a change helped. The LLM evaluation guide covers how to build that set.
The Honest Failure Modes of Long-Horizon Autonomy
Agents fail in recognizable ways, and every one of them is worse the longer the run. None of these are solved problems in 2026. Design as if each will happen, because on a long enough horizon each one does.
The six failures you will actually hit
- Compounding error. A per-step success rate of 95% is 36% over twenty steps if the errors are independent. Long chains need verification at checkpoints, not just at the end.
- Loops that look like progress. The agent retries a failing call with cosmetic variations, burning budget while appearing busy. Detect repeated identical calls and break the cycle in your code.
- Goal drift. Deep into a run the original request is buried under tool output, and the agent optimizes for the subtask in front of it. Restating the goal periodically helps; so does keeping runs short.
- Confident fabrication. When a tool returns nothing useful, models sometimes report a plausible answer instead of a failure. Require claims to cite a tool result, and make "I could not determine this" an acceptable output.
- Silent partial completion. Three of four subtasks done, reported as finished. Verify against the specification rather than the agent's own summary.
- Injection through retrieved content. Covered in section five, and the risk grows with every additional data source the agent can read.
The practical consequence is that autonomy is a dial, not a switch. The systems that work in production are the boring ones: short runs, narrow tool surfaces, checkpoints where a human or a deterministic check confirms state, and a hard preference for reversible actions. An agent that runs unattended for an hour makes a better demo and a worse product than one that does five minutes of work and hands back something a person can verify in thirty seconds.
Hand-Rolled Loop vs Frameworks vs Hosted Runtimes
The loop is not the hard part, so a framework's value is in what it adds around the loop: persistence, resumability, branching control flow, and tracing. Pick by which of those you actually need.
| Dimension | Hand-rolled loop | SDK tool runner | Graph framework | Hosted agent runtime |
|---|---|---|---|---|
| Code you write | The loop plus your tools | Just the tool functions | Nodes, edges, and state schema | Config plus your tool results |
| Control over each turn | Total | Per-turn hooks | Explicit, at node boundaries | Limited to what the API exposes |
| Durable state across restarts | You build it | You build it | Built in (checkpointing) | Built in (server-side sessions) |
| Tool sandbox | Your infrastructure | Your infrastructure | Your infrastructure | Provider-hosted container |
| Debuggability | Highest: it is your code | High | Good, with a graph to read | Depends on the trace UI |
| Lock-in | None beyond the API | SDK-level | Framework-level | Highest |
| Best for | Learning, and most production agents under ~10 tools | Custom-tool agents without loop boilerplate | Branching workflows that must pause, resume, and replay | Long-running jobs where you do not want to host the sandbox |
Concretely: start hand-rolled, because you will understand your own failure modes far faster. Move to an SDK tool runner when the loop boilerplate is duplicated across services and you want approval hooks without maintaining them yourself. Move to a graph framework such as LangGraph when you need durable checkpoints, resumable interrupts, and control flow that branches on state rather than on the model's next token. Use LangChain when the value you want is the integration library rather than the orchestration. Consider a hosted runtime when sessions run for hours and you would rather not operate the execution sandbox at all; several providers now offer one in beta, at the cost of running your agent inside someone else's process model.
These layers stack rather than compete. Frameworks consume tool servers as tool sources, so the integration boundary and the orchestration layer are separate decisions you can make independently.
When Not to Build an Agent
Agency costs latency, money, and predictability. It is worth paying when the right next step genuinely depends on what the last step returned. It is a bad trade when you already know the steps.
Reach for plain code instead when
- The sequence is fixed. If your pipeline always calls the same three endpoints in the same order, that is a workflow. Write the function. It is faster, cheaper, testable, and it never invents a fourth step.
- One retrieval answers the question. Search, then generate. A tool-calling loop adds turns to a problem that a single call solves.
- Latency is the product. Every loop iteration is a full model round trip. On an interactive path, one well-prompted call beats three agentic ones.
- Errors are expensive and hard to reverse. Unless every write sits behind approval, the cost of a wrong action exceeds what autonomy buys you.
- You cannot describe success. If you cannot write down what a correct outcome looks like, you cannot evaluate the agent, and you will be shipping on vibes.
The four questions worth asking before you build: is the task multi-step and hard to fully specify in advance; does the outcome justify the extra cost and latency; is the model actually capable at this task type; and can errors be caught and recovered from. A no to any of those is a signal to stay at the simpler tier. A single well-constructed model call, or a workflow of a few of them, handles far more real work than the current enthusiasm suggests, and it handles it with a stack trace you can read.
The bottom line: write the loop yourself first, spend your effort on tool descriptions and the error path, bound every run with four ceilings, gate every write, log every step, and keep runs short. That gets you an agent you can debug. Everything after that is tuning.
Frequently Asked Questions
What is an AI agent, in technical terms?
An AI agent is a loop around a language model that can call tools. You send the model a conversation plus a list of tool definitions. The model replies with either a final answer or one or more tool call requests. Your code executes those tools, appends the results to the conversation as tool result blocks, and sends the whole thing back. The loop ends when the model stops requesting tools or when a limit you set is reached. Everything else in an agent, including memory, planning, and multi-agent delegation, is built on top of that loop.
How many steps should an AI agent be allowed to take?
Set a hard step cap and treat hitting it as a failure worth investigating, not a normal outcome. For a focused task such as answering a support question with two or three lookups, 10 steps is generous. For a coding or research agent that reads many files, 40 to 60 is a reasonable ceiling. Pair the step cap with a token ceiling, a wall-clock deadline, and a spend limit, because a loop can burn a budget in ten expensive steps or in sixty cheap ones. Log which ceiling fired, since that tells you whether the agent was stuck or merely slow.
How should an agent handle a tool that fails?
Return the failure to the model as a tool result marked with an error flag, rather than raising an exception out of the loop. A returned error reaches the model, which can retry with different arguments, try another tool, or tell the user what went wrong. A thrown exception ends the turn and the model never learns anything. Write the error message for the model as a reader: say what failed, what was wrong with the input if anything, and what a valid call would look like. Reserve hard exceptions for failures the model cannot act on, such as an expired credential.
When should you not build an AI agent?
Skip the agent when the sequence of steps is known in advance. If your pipeline always calls the same three endpoints in the same order, that is a workflow, and ordinary code runs it faster, cheaper, and more predictably. Skip it when the task is a single retrieval plus an answer, where search and one model call are enough. Skip it when errors are expensive and hard to reverse, unless every write is gated behind human approval. Agency is worth its cost when the task is genuinely open ended and the right next step depends on what the previous step returned.
References: Anthropic tool use documentation, prompt caching, and context editing. Model IDs and published rates current as of August 2026; check the docs before pinning either.
Explore More Guides
- How to Build an AI Agent from Scratch (Step-by-Step)
- AI Agents Explained: What They Are and Why They Matter
- Claude API Guide: Tool Use, Streaming, and the Request Loop
- Model Context Protocol (MCP): A Complete Guide
- LangGraph: Durable, Resumable Agent Workflows
- LLM Evaluation: Building an Eval Set That Catches Regressions
- Pydantic AI: Typed Agents in Python