Agentic AI: autonomy, orchestration patterns and frameworks
What you’ll learn
Section titled “What you’ll learn”- What “agentic” means as a property of a system, and how to decide how much autonomy to give it.
- The orchestration patterns that structure agentic systems: prompt chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer, and multi-agent designs.
- Where people belong in the loop, and how to enforce it.
- What the main frameworks provide in November 2026, and how to choose between them.
- The hard parts in production, and when a simpler design is the better answer.
This page assumes you know how a single agent works. If not, read AI agents first: it covers the model-tool loop, tool calling and the parts of one agent.
Summary
Section titled “Summary”Agentic AI describes systems in which language models do more than answer: they plan, call tools, observe results and keep working toward a goal over several steps, with some freedom to decide what to do next. “Agentic” is a dial rather than a switch. At one end, code fixes every step and the model fills in text; at the other, the model chooses the steps, the tools and when it is finished.
Most production systems sit in the middle. They combine a few fixed workflow patterns with agent steps where the path genuinely depends on results, put people at the points where a mistake is expensive, and enforce budgets in code. Frameworks such as LangGraph, Microsoft Agent Framework, CrewAI and the provider SDKs package these patterns, but the design decisions are the same whichever you pick.
Context and forces: how much autonomy?
Section titled “Context and forces: how much autonomy?”Autonomy buys flexibility and costs predictability. Before choosing a pattern, answer four questions (adapted from Anthropic’s guidance in Building effective agents):
- Can the steps be written down in advance? If yes, a workflow is cheaper, faster and easier to test.
- Is the outcome worth the extra cost and latency? Agentic runs make many model calls; a ticket worth a few cents cannot pay for twenty.
- Can the model actually do the task? Measure on real cases before designing around it.
- Can mistakes be caught and undone? Tests, review, drafts and rollbacks make autonomy safe. Irreversible actions need a person or a hard rule.
The site uses the five levels from Spectrum of agency, which treats this dial in detail:
| Level | What the model decides | Example in OrdersPortal |
|---|---|---|
| 0. No agency | Nothing; it fills fixed slots | Summarize a ticket for a support agent to read |
| 1. Routed workflow | Which predefined path an input takes | Route a ticket to billing, delivery or returns |
| 2. Tool-using assistant | Which tool to call, from a bounded set, within one request | Look up the order and the return policy, then draft a reply |
| 3. Autonomous agent | The plan and the sequence of steps, over many turns | Investigate a delivery dispute across tracking, photos and earlier tickets, then propose a refund for approval |
| 4. Multi-agent system | Which agent works on what, and when to continue | A triage agent hands disputes to an investigator agent and a policy-review agent |
Levels 0 to 2 are workflows or close to it; levels 3 and 4 are agents. Autonomy is separate from authority (what an action may change): a level-3 agent that only reads is safer than a level-1 workflow that moves money.
The design
Section titled “The design”The loop and its components, briefly
Section titled “The loop and its components, briefly”Every agentic system is built from the loop described on AI agents: the model chooses an action, code runs it, the result goes back. Around that loop sit the same components at every scale: a model, tools, instructions, state (short-term context and longer-lived memory) and stop rules. Anthropic calls the basic unit an augmented LLM: a model with retrieval, tools and memory. Agentic architecture is the question of how many of these units you have, and who decides how they connect.
Orchestration patterns
Section titled “Orchestration patterns”The first five patterns are workflows: code decides how the steps connect, and models do the work inside each step. The names follow Anthropic’s December 2024 article, and most frameworks implement the same ideas under their own names.
| Pattern | How it works | Use it when | Watch out for |
|---|---|---|---|
| Prompt chaining | Step A’s output is step B’s input, with checks between steps | A task splits into fixed stages (extract, then draft, then translate) | Errors in early steps flow through; add a check gate between stages |
| Routing | A classifier sends each input to a specialized prompt, tool set or model | Inputs fall into distinct categories that need different handling | Misroutes are silent; log the route and measure its accuracy |
| Parallelization | Independent subtasks run at once (sectioning), or the same task runs several times and the results are combined (voting) | Subtasks do not depend on each other, or you need confidence from agreement | Cost multiplies; combining conflicting results needs a rule |
| Orchestrator-workers | A model breaks the task into subtasks at run time, hands them to workers, and merges the results | You cannot know the subtasks in advance (which files to change, which sources to read) | The orchestrator can over-split; cap the number of workers |
| Evaluator-optimizer | One model drafts, another evaluates against criteria, and the draft is revised | Clear quality criteria exist and revision measurably helps | Endless revision; bound the rounds and escalate on failure |
Two further patterns give the model more control:
- Single agent with tools (ReAct style). The model alternates between reasoning and tool calls until it is done. This is the default “agent” and covers more ground than people expect; see AI agents.
- Plan-and-execute. The model writes an explicit plan first, then works through it, revising the plan when a step fails. The written plan is useful for review and for resuming after a crash.
Multi-agent systems
Section titled “Multi-agent systems”A multi-agent system is orchestrator-workers or routing where the workers are themselves agents with their own instructions, tools and context. The common shapes are:
- Supervisor (subagents). A lead agent calls specialist agents as if they were tools, and every result comes back through it. Easiest to trace and reason about.
- Handoffs. Control passes from one agent to another, for example from triage to billing, and the active agent talks to the user directly.
- Shared workspace. Agents read and write a common store (a task board, a repository), coordinated by rules rather than by a single lead.
Multi-agent designs help in three situations: the instructions or tool list for one agent have grown too large to follow reliably; subtasks can run in parallel with separate context windows, as in broad research; or roles need different permissions. They cost more model calls, add hand-offs where context gets lost, and make failures harder to trace. Start with one agent and good tools, measure it, and split only where the measurement shows a gap. Two worked examples on this site: LangGraph for graph-based designs and Multi-agent pipeline for a staged content pipeline.
Protocols are starting to standardize the plumbing: the Model Context Protocol connects agents to tools and data, and the Agent2Agent (A2A) protocol, which Google donated to the Linux Foundation in 2025, describes how separately built agents exchange tasks. Check each protocol’s current specification before depending on it.
Worked example: an OrdersPortal returns workflow
Section titled “Worked example: an OrdersPortal returns workflow”A returns ticket for the invented shop OrdersPortal combines three patterns: routing decides whether the ticket is about returns, two parallel workers fetch the return policy and the order, and an evaluator loop checks the draft reply against a rule (“always tell the customer the next step”) before it is sent. A run-wide budget caps model calls across all steps.
The model calls are stubs so the control flow and the output are repeatable; each llm(...) call is where a real system calls a model API. Run it with Python 3.10 or later (tested with 3.11.18): save as patterns.py and run python patterns.py.
# patterns.py - routing, parallel workers and an evaluator loop, with stub model callsimport asyncio
CALLS = {"count": 0}MAX_CALLS = 10 # run-wide budget shared by every step
async def llm(role: str, text: str) -> str: """Stand-in for a model call. A real system calls an LLM API here.""" CALLS["count"] += 1 if CALLS["count"] > MAX_CALLS: raise RuntimeError("model-call budget exhausted") await asyncio.sleep(0.01) if role == "router": return "returns" if "return" in text.lower() else "general" if role == "policy": return "Returns accepted within 30 days if unworn." if role == "order": return "Order A1001: delivered 9 days ago, 64.00." if role == "writer": draft = "You can return it. " + text return draft if "label" in text else draft + " (no next step given)" if role == "reviewer": return "PASS" if "(no next step" not in text else "FAIL: tell the customer how to get a label" raise ValueError(role)
async def handle(ticket: str) -> str: route = await llm("router", ticket) # 1. routing if route != "returns": return "sent to the general queue" policy, order = await asyncio.gather( # 2. parallel workers llm("policy", ticket), llm("order", ticket)) facts = f"{policy} {order}" feedback = "" for attempt in range(1, 3): # 3. evaluator loop, max 2 rounds draft = await llm("writer", facts + feedback) verdict = await llm("reviewer", draft) print(f" draft {attempt}: {verdict}") if verdict == "PASS": return draft feedback = " A prepaid label is in your account under Returns." return "escalated to a person after 2 failed reviews"
reply = asyncio.run(handle("A1001 is the wrong size, can I return it?"))print("reply:", reply)print("model calls used:", CALLS["count"], "of", MAX_CALLS)Output:
draft 1: FAIL: tell the customer how to get a label draft 2: PASSreply: You can return it. Returns accepted within 30 days if unworn. Order A1001: delivered 9 days ago, 64.00. A prepaid label is in your account under Returns.model calls used: 7 of 10What the example shows:
- Most of the system is a workflow. Only the content of each step comes from a model. That makes cost predictable (seven calls here) and each stage testable on its own.
- Every loop has a bound and an exit. The evaluator gets two rounds, then the ticket goes to a person; the budget stops the whole run if anything misbehaves.
- The evaluator needs a checkable rule. “Tell the customer the next step” can be graded; “write a good reply” cannot.
When a delivery dispute needs open-ended investigation (carrier tracking, photos, earlier tickets), that branch becomes an agent with its own tools, and the workflow treats it as one step with a budget.
Human in the loop
Section titled “Human in the loop”Decide where people take part from the cost of a mistake, not from how confident the model sounds:
- Approve before acting for irreversible or expensive actions: payments and refunds over a threshold, emails to customers, deletions, production changes.
- Review the output where errors are cheap to fix before release: drafts, pull requests, reports.
- Escalate on failure: when an evaluator rejects a draft twice, a budget runs out, or a tool keeps failing.
- Audit after the fact for low-risk, high-volume actions, with sampling and alerts.
Enforce approval in code: the tool that issues a refund checks for an approval record, whatever the prompt says. The approval request should show the proposed action, the evidence the agent used and the expected effect, so the reviewer is not rubber-stamping. Durable runtimes help here because a run may wait hours for an answer; LangGraph’s interrupt() and Microsoft Agent Framework’s workflow checkpoints both persist state while waiting.
Frameworks
Section titled “Frameworks”All of these provide a tool-calling loop; they differ in how they express orchestration and state. Versions are the latest on PyPI on November 16, 2026.
| Framework | Model of the work | Notable for | Python package |
|---|---|---|---|
| LangGraph | Explicit graph of state, nodes and edges | Checkpoints, interrupts, resumable long runs; model-agnostic | langgraph 1.2.22 |
| LangChain | Prebuilt agent (create_agent) on top of LangGraph, plus integrations |
Fastest start for a standard tool-calling agent | langchain 1.5.0 |
| CrewAI | Role-based agents assigned tasks in a crew, plus event-driven flows | Readable team-style definitions | crewai 1.15.41 |
| Microsoft Agent Framework | Agents plus typed, graph-based workflows with checkpointing | Microsoft’s direct successor to Semantic Kernel and AutoGen; Python and .NET | agent-framework 1.26.0 |
| OpenAI Agents SDK | Agents, handoffs and guardrails | Lightweight multi-agent handoffs | openai-agents 0.24.0 |
| Google Agent Development Kit (ADK) | Agents composed into sequential, parallel and loop workflows | Integration with Gemini and Google Cloud | google-adk 2.14.1 |
| Claude Agent SDK | The Claude Code agent loop as a library, with built-in file, shell and web tools | Coding and file-based agents with little setup | claude-agent-sdk 0.2.182 |
| Strands Agents | Model-driven agent loop from AWS | AWS integrations, including Bedrock | strands-agents 1.65.0 |
How to choose:
- Start without a framework if your agent is one loop with a few tools; the loop on AI agents is about forty lines.
- Pick for the state model you need. Long-running, resumable runs with approvals point to LangGraph or Microsoft Agent Framework workflows; short request-response agents do not need either.
- Prefer what your platform already supports for tracing, deployment and identity. A framework your operations team cannot observe is a liability.
- Check the release cadence. Several of these packages ship weekly, and pre-1.0 versions (
openai-agents,claude-agent-sdk) can change APIs between minor releases. Pin versions and read changelogs before upgrading.
Where agentic AI gets hard
Section titled “Where agentic AI gets hard”- Compounding errors. A wrong reading in step two becomes an accepted fact by step six. Validate intermediate results, and keep early steps deterministic where you can.
- Cost and latency. Calls multiply with every loop, worker and revision. Track cost and success per completed task, set budgets per run, and cache stable prompt prefixes.
- State and recovery. Long runs crash, time out and get redeployed. Checkpoint state, make side effects idempotent, and plan for runs that are in flight during a schema change.
- Security. Anything an agent reads can carry instructions (prompt injection), and every tool is a permission. Give each agent the narrowest credentials that work, treat tool output as data, and gate actions. OWASP lists prompt injection first and “excessive agency” third in its Top 10 for LLM Applications 2026; see Risks of agentic AI.
- Evaluation. Multi-step, non-deterministic behavior needs scenario tests with graded outcomes, replayed traces and regression suites; see AI harness engineering.
- Observability. You need a trace per run: every model call, tool call, result, token count, decision and approval, linked to the business record.
- Governance. Someone must own each agent’s permissions, budget and incident response; see AI governance.
When not to use it
Section titled “When not to use it”- A single call with retrieval answers the question. Most Q&A and drafting tasks.
- The process is fixed and known. Write it as a workflow or ordinary code and call a model only where language is involved.
- Mistakes are costly and hard to detect, and you cannot put a person or a hard check in front of them.
- You cannot measure it. Without a graded set of real cases, you cannot tell whether more autonomy helped.
- Multi-agent, specifically: when one agent with well-designed tools has not yet been tried and measured.
Interview questions
Section titled “Interview questions”1. A level-2 support assistant works well. Product asks to let it “just handle” refunds end to end. What changes in the design? That request raises both autonomy (from one bounded request to multi-step runs) and authority (from reading to moving money), and they should be decided separately. Moving to level 3 needs step and cost budgets, full traces and scenario-based evaluation. Adding refund authority needs an amount limit enforced in the refund service, approval above it, idempotency keys and an audit trail. Often the right answer is level 3 autonomy for the investigation with refunds still behind approval.
2. When would you use orchestrator-workers instead of parallelization? When the subtasks cannot be known in advance. Parallelization runs a fixed set of independent steps; orchestrator-workers lets a model decide at run time which subtasks exist, such as which files a code change touches, and then merges the results.
3. How do you keep an evaluator-optimizer loop from running forever or approving bad output? Give the evaluator explicit, checkable criteria, bound the number of rounds, and escalate to a person when the limit is hit. Check the evaluator itself against human-graded examples, and use a different prompt (or model) from the writer so it does not simply agree with its own style.
4. Senior scenario: product wants a team of six agents (triage, billing, shipping, returns, tone reviewer, supervisor) for OrdersPortal support. How do you respond? Start from the baseline: one agent with well-scoped tools, measured on a graded set of real tickets for resolution rate, cost and latency. Then look at where it fails. If failures come from too many tools or conflicting instructions, split along those lines, usually into a router plus two or three specialists, not six. Put refunds behind approval in code, cap total calls per ticket, pass each specialist only the context it needs, and trace every hand-off. Ship the split only if it beats the baseline on the same test set; otherwise you have bought cost and complexity for nothing.
5. Senior scenario: an agent that files carrier disputes overnight starts producing duplicates after a framework upgrade. What do you check? Whether retries or resume behavior changed: a step re-executed after a checkpoint, or a tool retried after a timeout. Compare traces before and after the upgrade, pin the previous version while investigating, add idempotency keys to the filing tool, and add the duplicate case to the regression suite before upgrading again.
Key takeaways
Section titled “Key takeaways”- Agentic is a dial: choose the least autonomy that does the job.
- Most production systems are workflows with a few agent steps; the five workflow patterns cover most needs.
- Multi-agent designs are a cost; add agents only when a measured single-agent baseline falls short.
- Put people where mistakes are expensive, and enforce approvals and budgets in code.
- Frameworks package the patterns; choose by state model, observability and release stability, and pin versions.
Related pages
Section titled “Related pages”- AI agents: how one agent works
- Spectrum of agency: choosing the level of autonomy
- LangGraph, LangChain and CrewAI
- Multi-agent pipeline
- RAG vs agentic RAG
- Model Context Protocol
- Risks of agentic AI and AI governance
- AI harness engineering
Official references
Section titled “Official references”- Anthropic, Building effective agents (December 19, 2024): workflow patterns and when to use agents
- LangGraph overview and multi-agent patterns
- Microsoft Learn, Microsoft Agent Framework overview and migration from AutoGen
- CrewAI documentation
- OpenAI Agents SDK (Python) on GitHub
- Google Agent Development Kit (Python) on GitHub
- Claude Agent SDK overview
- Strands Agents on GitHub
- Agent2Agent (A2A) protocol on GitHub (Linux Foundation project)
- OWASP GenAI Security Project, Top 10 for LLM Applications 2026 (published August 4, 2026)
- PyPI pages for the versions above: langgraph, langchain, crewai, agent-framework, openai-agents, google-adk, claude-agent-sdk, strands-agents