Skip to content

LangChain: models, chains, agents and RAG

  • What LangChain is in its 1.x form, and how langchain, langchain-core, the provider packages, LangGraph and LangSmith relate.
  • How to compose a prompt, a model and a parser with the | operator, and what invoke, stream and batch do.
  • How create_agent runs a tool-calling loop, and how middleware bounds it.
  • How a retrieval-augmented generation (RAG) pipeline is assembled from an embedding model, a vector store and a retriever.
  • Where the abstraction costs more than it gives, and what to use instead.

LangChain is an open-source framework, in Python and JavaScript/TypeScript, for building applications on large language models. It gives you one interface over many model providers, a set of composable parts (prompts, tools, retrievers, output parsers) and a ready-made agent loop. Since version 1.0 the langchain package is deliberately small and centered on create_agent; older chain classes moved to a separate langchain-classic package. LangChain agents run on LangGraph, and LangSmith is the companion service for tracing and evaluation.

Use it when provider independence, integrations or the agent harness save you real work. Skip it when one SDK call does the job.

Package or product Role
langchain-core Base interfaces: messages, prompts, tools, the Runnable protocol, fake models for tests
langchain The agent harness (create_agent), middleware, and init_chat_model
Provider packages (langchain-anthropic, langchain-openai, langchain-google-genai, …) One package per model provider or data store
langchain-classic Legacy chains, retrievers and the hub client from before 1.0
LangGraph The lower-level graph runtime that agents execute on: state, persistence, interrupts
LangSmith Hosted tracing, evaluation and deployment

If you are reading older tutorials: AgentExecutor, create_react_agent with a prompt pulled from the hub, and LLMChain are pre-1.0 APIs. They now live in langchain-classic. New code should use create_agent.

The samples were run with Python 3.11, langchain 1.5.0, langchain-core 1.8.1, langgraph 1.2.22 and numpy 2.4.6, installed with pip install langchain numpy (langchain 1.5 requires Python 3.11 or later). NumPy is needed only by the RAG sample: the in-memory vector store’s similarity search imports it, and without it the sample fails with ImportError: cosine_similarity requires numpy. They use the fake chat models that ship in langchain-core, so they need no API key and print the same output every time. Save each as a .py file and run it with python.

To call a real model, install a provider package and replace the fake model. This fragment requires an API key and was not executed:

# Fragment: requires ANTHROPIC_API_KEY in the environment; not executed
# pip install "langchain[anthropic]"
from langchain.chat_models import init_chat_model
model = init_chat_model("anthropic:claude-sonnet-5-5")

init_chat_model takes a provider:model string, so the provider can be changed through configuration instead of code.

The example domain for this page is a support assistant for an invented shop, OrdersPortal.

Every LangChain component implements the Runnable interface. The | operator joins Runnables into a pipeline (the LangChain Expression Language, LCEL), and the pipeline is itself a Runnable.

# lc_chain.py - prompt | model | parser, with a fake model so no API key is needed
from langchain_core.language_models.fake_chat_models import GenericFakeChatModel
from langchain_core.messages import AIMessage
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
prompt = ChatPromptTemplate.from_messages([
("system", "You are the OrdersPortal support assistant. Answer in one sentence."),
("human", "{question}"),
])
# The fake model replays these replies in order instead of calling a provider.
model = GenericFakeChatModel(messages=iter([
AIMessage(content="Standard delivery takes three to five business days."),
AIMessage(content="Returns are free within thirty days."),
AIMessage(content="Yes, you can change the address until the order ships."),
]))
chain = prompt | model | StrOutputParser()
print(prompt.invoke({"question": "How long is delivery?"}).to_messages()[1])
print(chain.invoke({"question": "How long is delivery?"}))
for chunk in chain.stream({"question": "Are returns free?"}):
print(chunk, end="|")
print()
print(chain.batch([{"question": "Can I change my address?"}]))

Output:

content='How long is delivery?' additional_kwargs={} response_metadata={}
Standard delivery takes three to five business days.
Returns| |are| |free| |within| |thirty| |days.|
['Yes, you can change the address until the order ships.']

What to notice:

  • The prompt template turns a dictionary into a list of messages. The model turns messages into an AIMessage. The parser turns that into a plain string.
  • Nothing runs until you call invoke (one input), stream (chunks as they arrive) or batch (several inputs, run concurrently). Async versions (ainvoke, astream) exist for each.
  • Because the model is just another Runnable, a test can swap in a fake one. That is the cheapest way to unit-test prompt wiring.

An agent is a model called in a loop: the model either answers or asks for a tool, the harness runs the tool and feeds the result back, and the loop repeats until the model answers. create_agent builds that loop and returns a compiled LangGraph graph.

# lc_agent.py - create_agent with one tool; the model is scripted so the run is deterministic
from langchain.agents import create_agent
from langchain.agents.middleware import ModelCallLimitMiddleware
from langchain_core.language_models.fake_chat_models import GenericFakeChatModel
from langchain_core.messages import AIMessage
from langchain_core.tools import tool
ORDERS = {"A1001": "shipped, arriving Thursday", "A1002": "awaiting payment"}
@tool
def get_order_status(order_id: str) -> str:
"""Look up the delivery status of an OrdersPortal order by its ID."""
return ORDERS.get(order_id, "order not found")
class ScriptedModel(GenericFakeChatModel):
"""Stand-in for a real chat model: accepts tools and replays scripted messages."""
def bind_tools(self, tools, **kwargs):
return self
model = ScriptedModel(messages=iter([
# Turn 1: the "model" asks for a tool call.
AIMessage(content="", tool_calls=[
{"name": "get_order_status", "args": {"order_id": "A1001"}, "id": "call_1"}
]),
# Turn 2: with the tool result in context, it answers.
AIMessage(content="Order A1001 has shipped and arrives on Thursday."),
]))
agent = create_agent(
model=model,
tools=[get_order_status],
system_prompt="You are the OrdersPortal support assistant.",
middleware=[ModelCallLimitMiddleware(run_limit=5)], # hard stop for runaway loops
)
result = agent.invoke({"messages": [{"role": "user", "content": "Where is order A1001?"}]})
for message in result["messages"]:
print(f"{message.type:>5}: {message.content or message.tool_calls}")

Output:

human: Where is order A1001?
ai: [{'name': 'get_order_status', 'args': {'order_id': 'A1001'}, 'id': 'call_1', 'type': 'tool_call'}]
tool: shipped, arriving Thursday
ai: Order A1001 has shipped and arrives on Thursday.

The four messages are the whole mechanism. With a real model, pass a string such as "anthropic:claude-sonnet-5-5" as model; the tool’s name, type hints and docstring become the schema the model sees, so write the docstring for the model, not for yourself.

Two features matter in production:

  • Middleware hooks into the loop. langchain.agents.middleware in 1.5 includes ModelCallLimitMiddleware and ToolCallLimitMiddleware (cost and loop bounds), ModelFallbackMiddleware, SummarizationMiddleware (compress long histories), PIIMiddleware and HumanInTheLoopMiddleware (approval before selected tools run).
  • Memory is a LangGraph checkpointer. Pass checkpointer=InMemorySaver() (from langgraph.checkpoint.memory) and call the agent with {"configurable": {"thread_id": "customer-42"}}; later calls with the same thread ID continue the conversation. Use a database-backed checkpointer outside development. See LangGraph.

Retrieval-augmented generation pipeline: help articles are chunked, embedded and stored in a vector store; each question retrieves matching chunks, which are passed with the question to the chat model to produce an answer with sources.

RAG answers from your documents: embed the documents, store the vectors, retrieve the closest chunks for each question, and put them in the prompt. This sample uses a toy embedding class so it runs offline, and needs numpy installed next to langchain (see Running the samples); the echo model prints the final prompt, which is exactly what a real model would receive.

# lc_rag.py - retrieval-augmented generation with in-memory parts and no network calls
import hashlib
import math
import re
from langchain_core.documents import Document
from langchain_core.embeddings import Embeddings
from langchain_core.language_models.fake_chat_models import ParrotFakeChatModel
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.vectorstores import InMemoryVectorStore
class WordHashEmbeddings(Embeddings):
"""Toy bag-of-words embedding. Replace with a real embedding model in production."""
def _embed(self, text: str) -> list[float]:
vector = [0.0] * 256
for word in re.findall(r"[a-z]+", text.lower()):
vector[int(hashlib.md5(word.encode()).hexdigest(), 16) % 256] += 1.0
norm = math.sqrt(sum(v * v for v in vector)) or 1.0
return [v / norm for v in vector]
def embed_documents(self, texts: list[str]) -> list[list[float]]:
return [self._embed(t) for t in texts]
def embed_query(self, text: str) -> list[float]:
return self._embed(text)
policies = [
Document(page_content="Returns are free within 30 days of delivery.", metadata={"source": "returns.md"}),
Document(page_content="Standard delivery takes 3 to 5 business days.", metadata={"source": "delivery.md"}),
Document(page_content="Gift cards cannot be exchanged for cash.", metadata={"source": "gift-cards.md"}),
]
store = InMemoryVectorStore.from_documents(policies, WordHashEmbeddings())
retriever = store.as_retriever(search_kwargs={"k": 1})
def format_docs(docs: list[Document]) -> str:
return "\n".join(f"[{d.metadata['source']}] {d.page_content}" for d in docs)
prompt = ChatPromptTemplate.from_template(
"Answer from the context only. If it is not there, say you do not know.\n"
"Context:\n{context}\n\nQuestion: {question}"
)
# ParrotFakeChatModel echoes the final prompt, which shows exactly what a real model would receive.
rag_chain = (
{"context": retriever | format_docs, "question": RunnablePassthrough()}
| prompt
| ParrotFakeChatModel()
| StrOutputParser()
)
print(rag_chain.invoke("How many days do I have for returns?"))

Output:

Answer from the context only. If it is not there, say you do not know.
Context:
[returns.md] Returns are free within 30 days of delivery.
Question: How many days do I have for returns?

In a real system the same shape holds, with a document loader and text splitter in front, a provider embedding model, and a persistent vector store. The quality of the answer is decided mostly before the model is called: by chunking, by the embedding model and by how many chunks you retrieve. Test retrieval on its own (does the right chunk come back?) before judging answers.

Agent bugs are rarely visible from the final answer. LangSmith records each model call, tool call, token count and latency as a trace. Tracing is switched on with environment variables and needs no code change:

Terminal window
# Requires a LangSmith account and API key; not executed
export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY="..." # read from your secret store, never commit it

LangSmith is a hosted commercial service with a free tier; check its data-handling terms before sending customer content. LangChain also emits standard callbacks, so other tracing backends can be used.

Seen from LangChain, each alternative is a decision to give up part of what LangChain provides, or to add something it does not.

If you choose instead You gain over LangChain You give up from LangChain Choose it when
The provider SDK directly Fewer layers, every provider feature on release day, simpler stack traces The provider-neutral model interface, the ready-made tool loop, middleware and integrations; you write retries and history yourself One provider, one or two call sites, no agent loop
LangGraph directly Explicit state, branches, parallel steps and durable pauses that you design create_agent’s defaults: you build the loop and wire middleware-like concerns as nodes The control flow, not the tool loop, is the hard part
CrewAI A role-and-task model that reads like a team plan, with delegation built in Fine-grained control of each step and the breadth of LangChain’s integrations Role-based multi-agent pipelines where the team metaphor fits
A small internal interface you own No third-party release cadence to track Integrations and community components Provider abstraction is the only thing you need
  • Following pre-1.0 tutorials. Imports such as from langchain.agents import AgentExecutor or from langchain import hub fail on langchain 1.x. Use create_agent, or install langchain-classic to keep old code running while you migrate.
  • No loop bound. A model that keeps calling tools burns tokens until something stops it. Set a model-call or tool-call limit and a timeout.
  • Vague tool descriptions. The model picks tools from the name and docstring. “Search stuff” produces wrong calls; say what the tool returns and when to use it.
  • Unpinned versions. The provider packages and core release often. Pin versions and upgrade deliberately, with your test suite.
  • Treating retrieval as solved. Wrong answers in RAG are usually retrieval misses. Log the retrieved chunks and evaluate them separately.
  • Prompt injection through retrieved text or tool output. Anything the model reads can contain instructions. Keep destructive tools behind approval, and give tools the narrowest permissions that work.
  • Unbounded history. Conversation state grows on every turn. Trim or summarize it, or cost and latency rise until the context limit is hit.
  • Testing only against live models. Slow, costly and non-deterministic. Use the fake models for wiring tests and a small graded test set against the real model for quality.
  • A single prompt or a fixed two-step call. The provider SDK is shorter and has fewer moving parts.
  • You need a brand-new provider feature immediately. A wrapper can lag behind the native SDK or expose the feature differently.
  • Latency-critical paths where every layer is measured. The overhead is small next to a model call, but the indirection makes profiling harder.
  • A team that cannot track a fast-moving dependency. If nobody will own upgrades, fewer dependencies is the safer design.
  • A workflow that is mostly deterministic. Ordinary code with one model call inside is easier to test than an agent.

1. What does the | operator do in LangChain? It composes Runnables into a sequence: the output of the left side becomes the input of the right. The result is also a Runnable, so it gains invoke, stream, batch and their async forms without extra code. A dictionary in the pipeline runs its values in parallel and passes a dictionary on.

2. How does a tool-calling agent decide when to stop? Each model response either contains tool calls or does not. The harness executes any tool calls, appends the results as tool messages and calls the model again. When a response has no tool calls it is the final answer. In practice you add an external bound (a call limit or timeout), because the model’s own stopping decision is not guaranteed.

3. When would you drop from create_agent to a custom LangGraph graph? When the control flow stops being “model, tools, repeat”. Typical signals: steps that must run in a fixed order regardless of what the model says (validate, then approve, then execute); parallel branches whose results are merged; a human approval that must pause for hours and resume from a checkpoint at a specific point; different models or prompts per step; or retries and fallbacks that depend on state rather than on the last message. Middleware covers a lot (call limits, summarization, approval before a tool), so first check whether one middleware hook solves it. If you find yourself encoding a state machine in prompts or middleware, write the graph: it makes the flow explicit, testable node by node, and visible in traces. create_agent itself returns a LangGraph graph, so an agent can become one node of the larger graph instead of being rewritten.

4. Your RAG assistant gives confident wrong answers. How do you debug it? Separate retrieval from generation. For a set of failing questions, inspect the retrieved chunks: if the right passage is missing, fix chunking, the embedding model, the number of results, or add keyword or hybrid search. If the passage is present and the answer is still wrong, tighten the prompt (answer only from context, cite the source) and check for conflicting chunks. Then keep those questions as a regression set.

5. Senior scenario: a team wants to move a support bot from one model provider to another within a month. The bot is built on direct SDK calls scattered across services. What do you propose? First define the acceptance test: a graded set of real conversations with expected outcomes, so “works on the new provider” is measurable. Then introduce one seam for model access, either a thin internal interface or LangChain’s chat model interface, and move the call sites behind it without changing behavior. Run both providers against the test set, compare quality, latency and cost, and adjust prompts per provider where needed, since prompts do not transfer perfectly. Roll out behind a flag with a fallback to the old provider. I would use LangChain here only if the team also needs its agent loop or integrations; for provider abstraction alone, a small interface they own may be less to maintain.

  • LangChain 1.x is a small core: a standard model interface, composable Runnables and the create_agent harness.
  • Agents run on LangGraph; memory is a checkpointer, and behavior is customized with middleware.
  • Fake models make chains and agents testable without an API key.
  • RAG quality is mostly retrieval quality.
  • Bound every agent loop, pin versions and trace runs.
  • For one simple call, use the provider SDK.