Multi-agent pipelines: stages, checks and handoffs
What you’ll learn
Section titled “What you’ll learn”- What a multi-agent pipeline is, and how it differs from a single agent and from a free-form agent team.
- How to design the handoffs between stages so information is not lost or invented.
- Why the most important stage is often a deterministic checker, not another model.
- How to bound a writer-critic loop, and what to do when it does not converge.
- How to choose between plain code, LangGraph and CrewAI to run the pipeline.
Summary
Section titled “Summary”A multi-agent pipeline splits a job into a fixed sequence of stages, each handled by a narrowly prompted model call (an “agent”) with its own instructions and output format: research, draft, check, score, revise, approve, publish. The order is decided by your code, not by the models. That makes it a workflow in Anthropic’s terminology: models and tools orchestrated through predefined code paths. The design combines two patterns from Anthropic’s “Building effective agents”: prompt chaining (each step works on the previous step’s output) and evaluator-optimizer (one call generates, another evaluates, in a loop).
Pipelines work because narrow prompts are easier to write, test and improve than one prompt that does everything. They fail when handoffs drop facts, when one model is trusted to grade another without evidence, and when loops are unbounded. The rest of this page is about those three problems.
The framework pages show the same ideas in a specific runtime: LangGraph for explicit state, loops and approval interrupts, and CrewAI for role-based crews inside flows. This page stays framework-free so the design is visible.
Context and forces
Section titled “Context and forces”The classic example is content production: research a topic, brainstorm an angle, write a draft, score it, rewrite until it is good enough, then publish. Each step is something a model can do, but no single prompt does all of them well. The research instructions crowd out the style rules, and nobody can tell which part of a long prompt caused a bad result.
Splitting the job helps, and introduces new forces:
- Every handoff is lossy. The writer sees the researcher’s summary, not the sources. Numbers, caveats and attribution disappear between stages.
- Models invent plausibly. A writer asked for “release notes” will happily add a feature nobody shipped.
- Model judges are weak evidence. A scorer model shares the writer’s blind spots and is easily satisfied by confident text.
- Loops cost money. Every rewrite is a writer call plus a scorer call, and a loop that never converges burns budget indefinitely.
- Publishing is irreversible. Once a post, email or release note is out, a mistake is public.
The design
Section titled “The design”inputs ─► [research] ─► facts with IDs ─► [write] ─► draft + cited IDs ▲ │ │ ▼ feedback ◄── [check (code)] ─► [score (model)] │ │ rewrite (max N) ◄── score < bar ───────┤ │ score ≥ bar [human gate] ◄────┘ │ approved ▼ [publish]The design rules behind the diagram:
- Typed contracts between stages. Each stage returns a structured object, not prose: research returns facts with IDs, the writer returns text plus the IDs it relied on. Validate every handoff; reject and retry on a schema failure instead of passing garbage downstream.
- Code before models. Anything that can be checked deterministically (citations exist, numbers match the source, length, banned words, links resolve) is checked by code. The model scorer then judges only what code cannot: tone, clarity, usefulness.
- Hard failures cap the score. If the checker finds an invented fact, the draft fails regardless of how well it reads.
- A bounded loop with an exit that is not “publish anyway”. After N rewrites, stop and route to a person. A pipeline that publishes its best failed attempt has no quality bar.
- A human gate before irreversible actions, at least until measured quality earns removal for low-risk categories.
- A run record. One object carries the call budget, the trace of stages and the inputs and outputs of each step, so a bad result can be explained.
Running the sample
Section titled “Running the sample”The sample is plain Python 3.10 or later (tested with Python 3.11; no packages needed). The model calls are scripted stubs, so it runs deterministically without an API key; in production each stub would be one model call with its own prompt and output schema. Save it as release_notes_pipeline.py and run python3 release_notes_pipeline.py.
Worked example: OrdersPortal release notes
Section titled “Worked example: OrdersPortal release notes”An invented shop, OrdersPortal, publishes customer-facing release notes every week. The pipeline reads merged pull requests, drafts notes, checks them, scores them, revises if needed, and waits for an editor before publishing. The scripted writer’s first draft contains a classic failure: an invented feature (“same-day delivery”) and no citations.
# release_notes_pipeline.py - a multi-agent pipeline with typed handoffs, a checker, a bounded# rewrite loop and a human gate. Model calls are scripted stubs, so it runs deterministically.import refrom dataclasses import dataclass, field
# ---- Contracts between stages -------------------------------------------------@dataclass(frozen=True)class Change: # output of the research stage: facts with an ID id: str summary: str customer_visible: bool
@dataclassclass Draft: # output of the writer: text plus the change IDs it cites text: str cited: list[str]
@dataclassclass Review: # output of the scorer score: int feedback: str
@dataclassclass Run: # shared run record: budget and trace budget_calls: int = 8 calls: int = 0 trace: list[str] = field(default_factory=list)
def spend(self, stage: str) -> None: self.calls += 1 self.trace.append(stage) if self.calls > self.budget_calls: raise RuntimeError("call budget exceeded")
# ---- Stubbed "agents" (each would be one model call with its own prompt) -----def research(run: Run) -> list[Change]: run.spend("research") # in production: read merged pull requests and tickets return [Change("PR-412", "Order history loads twice as fast", True), Change("PR-415", "Refactor of the tax module", False), Change("PR-419", "Customers can download invoices as PDF", True)]
WRITER_SCRIPT = iter([ Draft("Order history is faster. You can now download invoices as PDF. " "We also added same-day delivery.", ["PR-412", "PR-419"]), Draft("Order history now loads about twice as fast (PR-412). " "You can download any invoice as a PDF from the order page (PR-419).", ["PR-412", "PR-419"]),])def write(run: Run, changes: list[Change], feedback: str = "") -> Draft: run.spend("write" if not feedback else "rewrite") return next(WRITER_SCRIPT)
def check(changes: list[Change], draft: Draft) -> list[str]: """Deterministic checker: no model call. Every sentence must cite a known, visible change.""" visible = {c.id for c in changes if c.customer_visible} problems = [f"unknown or internal change cited: {i}" for i in draft.cited if i not in visible] for sentence in filter(None, (s.strip() for s in draft.text.split("."))): if not re.search(r"PR-\d+", sentence): problems.append(f"uncited claim: '{sentence}'") return problems
def score(run: Run, draft: Draft, problems: list[str]) -> Review: run.spend("score") # a model grades tone and clarity against a rubric if problems: # hard failures from the checker cap the score return Review(3, "; ".join(problems)) return Review(9, "clear and specific")
def human_gate(draft: Draft) -> bool: return True # in production: pause and wait for an editor's approval
# ---- Orchestration --------------------------------------------------------------def pipeline(max_rewrites: int = 2) -> None: run = Run() changes = research(run) draft = write(run, changes) for attempt in range(max_rewrites + 1): problems = check(changes, draft) review = score(run, draft, problems) print(f"attempt {attempt}: score={review.score} feedback={review.feedback}") if review.score >= 8: break if attempt == max_rewrites: print("stopped: rewrite limit reached, sent to an editor instead") return draft = write(run, changes, review.feedback) if human_gate(draft): print("PUBLISHED:", draft.text) print(f"calls={run.calls} trace={' > '.join(run.trace)}")
pipeline()Output:
attempt 0: score=3 feedback=uncited claim: 'Order history is faster'; uncited claim: 'You can now download invoices as PDF'; uncited claim: 'We also added same-day delivery'attempt 1: score=9 feedback=clear and specificPUBLISHED: Order history now loads about twice as fast (PR-412). You can download any invoice as a PDF from the order page (PR-419).calls=5 trace=research > write > score > rewrite > scoreWhat the example shows:
- The checker caught the invented feature without a model. “Same-day delivery” has no pull request behind it, so it cannot cite one. Grounding rules that code can enforce are the cheapest defense against fabrication.
- Internal changes stay internal. PR-415, a tax refactor, is marked not customer-visible; citing it would be flagged.
- Feedback is specific. The rewrite receives the exact sentences that failed, which converges faster than “please improve”.
- The run is explainable. Five calls, in a known order, within a budget of eight.
The same shape fits many jobs:
| Use case | Research input | Writer output | Deterministic checks | Gate |
|---|---|---|---|---|
| Release notes | Merged pull requests, tickets | Customer-facing notes | Every claim cites a visible change | Editor |
| Technical blog or LinkedIn post | Docs, papers, internal data | Article draft | Facts and figures trace to sources, length, links resolve | Author |
| API documentation | Code, tests, OpenAPI spec | Reference pages | Every endpoint and parameter exists in the spec | Maintainer |
| Support macros | Policy documents | Reply templates | Policy section cited, no promises outside policy | Support lead |
| Code review summary | Diff, test results | Review comments | Each comment points at a changed line | Developer |
Implementing it in a framework
Section titled “Implementing it in a framework”The pipeline above is about 100 lines of plain code, and for many teams that is enough. Reach for a framework when you need what it provides, not for the vocabulary:
- LangGraph when runs are long or must survive restarts: its checkpointers persist state between steps,
interrupt()implements the human gate as a real pause that can resume tomorrow, and conditional edges express the rewrite loop. Set the recursion limit explicitly so a non-converging loop stops. - CrewAI when the stages map naturally to roles and you want a flow to own routing and state, with crews only where reasoning is needed. Task guardrails are the natural home for the deterministic checker.
- Plain code plus a job queue when the pipeline is short, synchronous and runs on a schedule. You keep full control of prompts and the smallest dependency surface.
Whichever you choose, keep the checker as code you can unit-test, and keep the stage contracts as schemas (Pydantic models or JSON Schema) shared by every stage.
Trade-offs and alternatives
Section titled “Trade-offs and alternatives”| Design | Gain | Cost | Choose it when |
|---|---|---|---|
| One prompt, one call | Cheapest, fastest, simplest to trace | Long prompt, hard to debug, weak on multi-step work | The job is small and the output easy to check |
| Fixed multi-agent pipeline (this page) | Narrow prompts, testable stages, predictable cost | More calls and latency; handoffs can lose information | The steps are known in advance and quality must be controlled |
| Orchestrator with dynamic workers | Adapts to inputs whose subtasks cannot be predicted | Much higher and more variable cost; harder to test and trace | Open-ended research or tasks whose shape depends on the input |
| Single agent with tools | One reasoning context, no handoff loss | The agent decides its own steps; less control over order | Interactive tasks where the model must choose what to do next |
Anthropic’s description of its own multi-agent research system (June 2025) reported that multi-agent runs used far more tokens than single-agent chats, and that the approach paid off for broad, parallelizable research. Treat that as a reason to measure, not to assume, before adding agents.
Failure modes and operations
Section titled “Failure modes and operations”- Fabrication across a handoff. The writer adds facts not in the research. Fix with citation IDs in the contract and a checker that rejects unsupported claims.
- Telephone-game loss. Each summary drops detail. Pass structured facts, not prose, and give later stages access to the original sources when needed.
- Scorer leniency or drift. The model judge approves weak drafts, or its scores shift after a model upgrade. Calibrate the scorer against human ratings on a fixed set, and re-calibrate on every model change.
- Non-converging loops. Rewrites oscillate between two flawed drafts. Cap the loop, send specific feedback, and escalate to a person at the cap.
- Cost and latency creep. Calls multiply by stages and rewrites. Track calls and tokens per run and alarm on outliers, not just averages.
- Cascading retries. A transient error in stage four restarts the whole run. Persist stage outputs so a retry resumes where it failed.
- Prompt injection through research inputs. Ticket text or web pages can carry instructions that later stages obey. Treat research outputs as data, strip instructions, and keep publishing behind the gate.
- Silent partial publishing. A publish step that fails midway leaves some channels updated. Make publishing idempotent with a run ID.
When not to use it
Section titled “When not to use it”- One good prompt passes your evaluation. More agents add cost without adding quality.
- The work is deterministic. Formatting, templating and data transformation belong in ordinary code.
- The output is needed in real time. Several sequential model calls add seconds; interactive features rarely tolerate it.
- You cannot define what “good” means. Without a checker and a rubric, a scorer loop only adds cost and false confidence. Define acceptance criteria first.
Interview questions
Section titled “Interview questions”1. What is the difference between a multi-agent pipeline and an autonomous agent? A pipeline is a workflow: code fixes the order of stages, and each model call does one narrow job. An autonomous agent decides its own next step and tool calls in a loop. Pipelines are more predictable and testable; agents are more flexible for open-ended tasks.
2. Why put a deterministic checker before the model scorer? Code checks are cheap, reproducible and cannot be talked out of a rule. They catch the failures that matter most, such as invented facts or missing citations, and leave the model scorer to judge what code cannot. A model judging a model alone shares its blind spots.
3. How do you stop a writer-critic loop from running forever, and what happens at the limit? Cap the number of rewrites and the total call budget per run, and send specific feedback so each rewrite has a chance to converge. At the cap, stop and route the draft and the feedback history to a person. Publishing the last failed draft defeats the purpose of the scorer.
4. Senior scenario: OrdersPortal’s release-notes pipeline has run for three months. Editors now approve 98% of drafts unchanged, and the business asks to remove the human gate. What do you require first? Evidence that the 98% is real quality, not rubber-stamping: sample approved notes for a blind second review, and measure post-publication corrections and support tickets about notes. Break the approval rate down by category, because “minor UI change” may be safe while “pricing” or “security” is not. Remove the gate per category, starting with the lowest-risk ones, and keep it for anything that mentions money, security or data. Add compensating controls before removing it: the deterministic checker blocks publication on any failure, a delayed-publish window allows an editor to cancel, every note is reversible, and a weekly sample review continues. Monitor scorer calibration after each model upgrade, because the evidence for removing the gate was collected on the old model.
5. Senior scenario: the pipeline’s monthly cost tripled after a model upgrade, with no change in traffic. How do you investigate? Compare per-stage traces before and after: calls per run, rewrites per run and tokens per call. The usual causes are a scorer that became stricter (more rewrites), a model that writes longer outputs or reasons more by default, or a schema failure that triggers retries. Fix the cause, not the symptom: re-calibrate the scorer threshold on the fixed evaluation set, set explicit output limits and effort per stage, and use a smaller model for stages that only extract or format. Add a per-run cost alarm so the next regression shows up in a day, not a month.
Key takeaways
Section titled “Key takeaways”- A multi-agent pipeline is a code-ordered workflow of narrow model calls: prompt chaining plus an evaluator-optimizer loop.
- Design the handoffs: typed contracts with source IDs, validated at every stage.
- Check with code first; let a model score only what code cannot.
- Bound every loop, escalate to a person at the limit, and gate irreversible actions.
- Use plain code until you need persistence, interrupts or role-based crews, then pick LangGraph or CrewAI.
Related pages
Section titled “Related pages”- LangGraph
- CrewAI
- AI agents overview
- The spectrum of agency
- RAG vs agentic RAG
- Agentic AI risks
- Automated evaluation pipeline
Official references
Section titled “Official references”- Anthropic, Building effective agents (December 2024): workflows versus agents, prompt chaining, evaluator-optimizer, orchestrator-workers
- Anthropic, How we built our multi-agent research system (June 2025): token cost and when multi-agent designs pay off
- LangChain, Multi-agent patterns and LangGraph interrupts
- CrewAI, Flows and task guardrails