Skip to content

AI agents: what they are and how they work

  • A working definition of an AI agent, and how it differs from a single model call or a fixed workflow.
  • The agent loop: how a model asks for a tool, how your code runs it, and how the result flows back.
  • The five parts every agent has: model, tools, instructions, state and stop rules.
  • A runnable loop with a step budget and an approval gate, and the same loop driven by Claude.
  • How agents fail in production, and when a simpler design is the better choice.

This page is about one agent and its mechanics. For autonomy levels, orchestration patterns (routing, orchestrator-workers, evaluator loops, multi-agent systems) and frameworks, read Agentic AI next.

An AI agent is a program in which a language model decides, step by step, which action to take next, your code carries the action out with a tool, and the result goes back to the model until the goal is met or a limit stops it. Anthropic’s widely cited definition separates workflows, “systems where LLMs and tools are orchestrated through predefined code paths”, from agents, “systems where LLMs dynamically direct their own processes and tool usage” (Building effective agents, December 2024).

The model never touches your systems directly. It produces a structured request (“call get_order with order_id: A1001”); your application decides whether to run it, runs it, and returns the result. That separation is where permissions, approvals, budgets and logging live, and it is the most important idea on this page.

A single model call is enough when everything the model needs is in the prompt and the answer is text. Real support, coding and operations tasks break that assumption:

  • The information is elsewhere. Order status, stock levels and log lines live in databases and APIs, and change by the minute.
  • The path is not known in advance. Whether a customer gets a refund, a replacement or an escalation depends on what the order lookup returns.
  • Work has side effects. Opening a return or issuing a refund changes the world, and some changes cannot be undone.
  • Steps fail. APIs time out, records are missing, and the model sometimes picks the wrong tool.

You can handle the first two with retrieval and hard-coded branches. An agent becomes worth its cost when the number and order of steps genuinely depend on intermediate results, so you cannot write the branches ahead of time.

Every agent, from a ten-line script to a coding agent that runs for hours, repeats the same cycle:

  1. Send the goal, the instructions, the tool definitions and the conversation so far to the model.
  2. Read the decision. The model either answers or asks for one or more tool calls, each with a tool name and JSON arguments.
  3. Act. Your code validates the request, checks permissions, runs the tool and captures the result or the error.
  4. Observe. Append the model’s request and the tool result to the conversation.
  5. Repeat or stop. Loop back to step 1, unless the model has answered, a budget is spent or a rule says stop.

This is the “reason, then act, then observe” pattern described in the ReAct paper (Yao et al., 2022), and it is what provider tool-calling APIs implement. On the Claude Messages API, a response with stop_reason: "tool_use" carries tool_use blocks; your code replies with tool_result blocks; end_turn means the model has finished (Tool use with Claude). OpenAI’s Responses API and Google’s Gemini API use the same shape under different names (function calls and function results).

Part What it is Design questions
Model The LLM that reads the context and chooses the next action Which tier, which reasoning effort, what it costs per completed task
Tools Functions the model may request, each with a name, a description and a JSON Schema for its arguments Which actions are exposed, which are read-only, which need approval
Instructions The system prompt: role, policies, when to use which tool, when to stop and ask Are the rules testable? Do tool descriptions say when not to use the tool?
State The conversation so far, plus anything stored outside it (a task list, a checkpoint, long-term memory) What survives a crash? What is trimmed when the context fills up?
Stop rules The final answer, a step or token budget, a timeout, an approval that was declined What happens when a budget runs out mid-task?

Two of these deserve more weight than they usually get.

Tool descriptions are prompts. The model chooses tools by reading their names and descriptions. A vague description (“handles orders”) produces wrong calls; a precise one (“Look up an order by ID. Returns status, total and carrier. Does not change anything.”) produces right ones. Small, single-purpose tools with strict schemas beat one large tool with a mode argument.

The context window is the working memory. Every tool result is appended to the conversation and re-sent on the next call, so a chatty tool that returns 50 KB of JSON makes every later step slower and costlier, and pushes earlier instructions further from the model’s attention. Return only what the model needs, and trim or summarize old results on long runs.

Worked example: an OrdersPortal returns agent

Section titled “Worked example: an OrdersPortal returns agent”

The example domain is an invented shop, OrdersPortal. A customer writes “Order A1001 arrived in the wrong size. Please start a return.” The agent has two tools: get_order (read-only) and open_return (changes data, so it needs approval).

The first sample replaces the model with a scripted function so the mechanics are visible and the output is repeatable. Everything else is what a production loop has: a tool registry, an approval gate, error results instead of exceptions, and a step budget. It needs only Python 3.10 or later (run with Python 3.11.18): save it as agent_loop.py and run python agent_loop.py.

# agent_loop.py - the agent loop with a scripted stand-in for the model
import json
# --- Tools: ordinary functions the application owns -----------------------
ORDERS = {"A1001": {"status": "delivered", "total": 64.00, "carrier": "ParcelCo"}}
def get_order(order_id: str) -> dict:
order = ORDERS.get(order_id)
return order if order else {"error": f"order {order_id} not found"}
def open_return(order_id: str, reason: str) -> dict:
return {"return_id": f"R-{order_id}", "label_sent": True}
TOOLS = {
"get_order": {"fn": get_order, "needs_approval": False},
"open_return": {"fn": open_return, "needs_approval": True},
}
# --- The "model": scripted here; a real agent calls an LLM API ------------
def fake_model(messages: list[dict]) -> dict:
"""Return either a tool call or a final answer, based on what it has seen."""
seen = [m for m in messages if m["role"] == "tool"]
if not seen:
return {"tool": "get_order", "args": {"order_id": "A1001"}}
last = json.loads(seen[-1]["content"])
if last.get("status") == "delivered":
return {"tool": "open_return", "args": {"order_id": "A1001", "reason": "wrong size"}}
if "return_id" in last:
return {"final": f"Return {last['return_id']} is open and a label was emailed."}
return {"final": "I could not complete the return; a person will follow up."}
def approve(tool: str, args: dict) -> bool:
print(f" approval requested: {tool} {args} -> approved")
return True # a real system waits for a person or a policy engine
# --- The loop ------------------------------------------------------------
def run_agent(goal: str, max_steps: int = 5) -> str:
messages = [{"role": "user", "content": goal}]
for step in range(1, max_steps + 1):
decision = fake_model(messages)
if "final" in decision:
print(f"step {step}: final answer")
return decision["final"]
name, args = decision["tool"], decision["args"]
print(f"step {step}: call {name}({args})")
tool = TOOLS.get(name)
if tool is None:
result = {"error": f"unknown tool {name}"}
elif tool["needs_approval"] and not approve(name, args):
result = {"error": "action declined by reviewer"}
else:
result = tool["fn"](**args)
messages.append({"role": "tool", "name": name, "content": json.dumps(result)})
return "Stopped: step budget used up before the goal was reached."
print(run_agent("Order A1001 arrived in the wrong size. Please start a return."))
print(run_agent("Order A1001 arrived in the wrong size. Please start a return.", max_steps=1))

Output:

step 1: call get_order({'order_id': 'A1001'})
step 2: call open_return({'order_id': 'A1001', 'reason': 'wrong size'})
approval requested: open_return {'order_id': 'A1001', 'reason': 'wrong size'} -> approved
step 3: final answer
Return R-A1001 is open and a label was emailed.
step 1: call get_order({'order_id': 'A1001'})
Stopped: step budget used up before the goal was reached.

Three details carry over to every real agent:

  • Unknown tools and declined actions become results, not crashes. The model reads the error on the next step and can recover, for example by asking the customer for a different order number.
  • Approval is enforced in code, not in the prompt. A prompt that says “always ask before opening a return” is a request; the needs_approval check is a guarantee.
  • The budget ends the run with an honest message. The second call shows what the customer sees when the agent runs out of steps, instead of a loop that never ends.

Swapping the scripted function for a real model changes only the decision step. This version uses the Anthropic Python SDK and the Messages API. It requires an API key and was not executed against the API; it was run with anthropic 2.8.0 against a mocked HTTP transport that returned two tool_use responses and an end_turn response, and it sent three requests, returned each tool result in one user message, and printed the final text.

# claude_loop.py - the same loop driven by Claude through the Messages API
# Requires ANTHROPIC_API_KEY; pip install anthropic
import json
import anthropic
client = anthropic.Anthropic()
TOOLS = [
{
"name": "get_order",
"description": "Look up an OrdersPortal order by ID. Returns status, total and carrier.",
"input_schema": {
"type": "object",
"properties": {"order_id": {"type": "string", "description": "Order ID such as A1001"}},
"required": ["order_id"],
"additionalProperties": False,
},
"strict": True,
},
{
"name": "open_return",
"description": "Open a return for a delivered order and email a return label. Needs approval.",
"input_schema": {
"type": "object",
"properties": {
"order_id": {"type": "string"},
"reason": {"type": "string"},
},
"required": ["order_id", "reason"],
"additionalProperties": False,
},
"strict": True,
},
]
def execute(name: str, args: dict) -> tuple[str, bool]:
"""Run a tool; return (content, is_error). Replace with real implementations."""
if name == "get_order":
return json.dumps({"status": "delivered", "total": 64.00}), False
if name == "open_return":
return json.dumps({"return_id": "R-" + args["order_id"]}), False
return f"unknown tool {name}", True
def run_agent(goal: str, max_steps: int = 8) -> str:
messages = [{"role": "user", "content": goal}]
for _ in range(max_steps):
response = client.messages.create(
model="claude-opus-5-5",
max_tokens=16000,
system="You are the OrdersPortal support agent. Use the tools; never guess order data.",
tools=TOOLS,
messages=messages,
)
if response.stop_reason == "refusal":
return "Declined by the model's safety checks."
if response.stop_reason == "max_tokens":
return "Response cut off: raise max_tokens."
messages.append({"role": "assistant", "content": response.content})
if response.stop_reason != "tool_use": # end_turn: the model has answered
return "".join(b.text for b in response.content if b.type == "text")
results = []
for block in response.content:
if block.type == "tool_use":
content, is_error = execute(block.name, block.input)
results.append({"type": "tool_result", "tool_use_id": block.id,
"content": content, "is_error": is_error})
messages.append({"role": "user", "content": results}) # all results in one message
return "Stopped: step budget used up."
if __name__ == "__main__":
print(run_agent("Order A1001 arrived in the wrong size. Please start a return."))

What changed and why:

  • Tools are JSON Schema. strict: True with additionalProperties: False makes the API guarantee that tool arguments match the schema, so execute does not have to defend against a missing order_id.
  • The whole assistant turn is appended, not just its text, because the next request must contain the tool_use blocks that the tool_result blocks refer to by tool_use_id.
  • All results from one turn go back in one message. The model may ask for several tools at once; returning them together keeps parallel calls working.
  • Stop reasons are checked before content. A refusal or max_tokens response is not a usable answer. Put the approval gate from the first sample inside execute before going live.

claude-opus-5-5 is Anthropic’s recommended default model in November 2026; see Anthropic Claude models for the lineup. The Anthropic SDK also offers a beta tool runner that writes this loop for you; writing it once by hand is the best way to understand what such helpers do.

Seen from the agent’s side, each alternative gives up model-driven control flow in exchange for predictability.

If you choose instead You gain over an agent You give up Choose it when
A single model call (with retrieval if needed) One request, low latency, easy to test Multi-step work and actions The answer depends only on text you can put in the prompt
A fixed workflow (code calls the model at set points) Predictable path, cost and latency; ordinary unit tests Adapting the path to results You can draw the flowchart in advance
A deterministic program No model cost, exact behavior Handling free-form input and ambiguity The rules are fully known
A larger agent system (several agents, long runs) Parallel work, separation of roles Simplicity; each agent adds calls and hand-offs One agent with good tools has been measured and falls short (see Agentic AI)
  • Wrong tool, plausible arguments. The model calls open_return for an order that is not yet delivered. Validate preconditions inside the tool, not only in the prompt.
  • Loops. The model retries a failing call with the same arguments. Count repeated identical calls and stop after two or three, alongside the overall step budget.
  • Invented data. Without a tool result to read, a model may fill in an order status. Instruct it to use tools for facts, and return explicit “not found” results rather than empty ones.
  • Context growth. Long runs push early instructions out of focus and raise cost per step. Trim old tool results, summarize, or store state outside the conversation.
  • Prompt injection through tool results. A customer email or web page the agent reads can contain instructions. Treat tool output as data, keep sensitive actions behind approval, and give the agent the narrowest credentials that work. Risks of agentic AI covers this in depth.
  • Side effects that repeat. A retry after a timeout can open two returns. Give write tools idempotency keys.
  • Invisible reasoning. When a run goes wrong you need every request, tool call, result, token count and stop reason. Log them per step from the first day, with the trace ID in the customer record.

Measure agents on cost and success per completed task, not per request: a cheaper model that needs twice as many steps is not cheaper.

  • The task fits in one call. Summaries, classification, extraction and drafting rarely need a loop.
  • The steps are known. If a flowchart describes the work, write the flowchart as code and call the model at the steps that need language.
  • Errors are expensive and hard to detect. Payments, legal notices and production changes need a human decision, with the agent preparing the work.
  • You cannot observe or evaluate it. Without traces and a graded test set, you cannot tell whether a change made the agent better or worse.

1. Why do tool descriptions and schemas matter as much as the system prompt? The model chooses tools and fills arguments by reading their names, descriptions and JSON Schemas, which are sent with every request. Vague descriptions cause wrong tool choices; loose schemas let malformed or missing arguments through. Small, single-purpose tools with precise descriptions (including when not to use them) and strict schemas make the agent more reliable without touching the prompt.

2. Walk through what happens when a model “calls a tool”. The model does not execute anything. It returns a structured request with a tool name and JSON arguments (on Claude, a tool_use block and stop_reason: "tool_use"). Your application validates it, checks permissions, runs the function, and sends back a result tied to the request’s ID. The model then reads that result as part of the conversation on its next call.

3. Why should failed tool calls be returned to the model instead of raised as exceptions? Because the model can often recover: try a different argument, ask the user for missing information, or report the problem honestly. An exception ends the run and loses that chance. Mark the result as an error (is_error: true on Claude) so the model knows it failed.

4. How do you stop an agent from running forever or overspending? Combine a step budget, a token or cost budget, a wall-clock timeout, and detection of repeated identical calls. Each limit should end the run with a clear status rather than a crash, and the limits belong in code, not only in the prompt.

5. Senior scenario: a returns agent occasionally opens two returns for the same order. How do you investigate and fix it? Pull the traces for affected orders and check whether the model asked twice, or the application executed once and retried after a timeout. If the model asked twice, the first result was probably ambiguous; return the existing return ID so the model sees the action already happened. If the retry layer duplicated it, add an idempotency key derived from the order ID and reason so the returns service rejects duplicates. In both cases make open_return check for an existing open return first, add a test case that replays the trace, and alert on duplicates.

  • An agent is a loop: the model chooses an action, your code performs it, and the result goes back to the model.
  • The model only requests actions; permissions, approvals, budgets and logging live in your code.
  • Tool names, descriptions and schemas are part of the prompt; make them small and precise.
  • Return errors as results, bound every loop, and make write actions idempotent.
  • Prefer a single call or a fixed workflow when it works; reach for an agent when the path depends on results.