Skip to content

AI integrations: connecting models to apps and data

  • The seven integration patterns that connect a language model to real applications and data, and when each fits.
  • How to define a tool once and expose it to more than one provider’s API, with a tested example.
  • Where MCP, retrieval, frameworks and gateways sit relative to plain API calls.
  • The security, cost and reliability concerns that every integration shares.

A model on its own only turns text into text. Integrations give it context (your data, documents and records) and reach (the ability to act in your systems). There are a small number of patterns: call the model directly, constrain its output to a schema, let it call your functions, connect it to tools through the Model Context Protocol, feed it retrieved documents, build on a framework, or route everything through a gateway. Most production systems combine several.

The pages below go deep on each building block. This page explains how they fit together and how to pick.

Every integration answers the same questions:

  • Where does the context come from? Typed into the prompt, retrieved from a search index, or fetched by the model through a tool.
  • Who decides what happens next? Your code (a fixed integration) or the model (tool calling and agents).
  • What may the model change? Nothing (read-only), drafts for a person, or real records.
  • How many models and providers? One provider through its SDK, or several behind an abstraction.
  • Who else needs the integration? One application, or many assistants and editors that should share the same tools.
Pattern What it is Use it when Read more
1. Direct API call Your code sends a prompt to a provider’s API and uses the reply Summaries, drafts, classification inside an existing feature API tokens, Python and OpenAI, Python and Gemini
2. Structured output The API constrains the reply to a JSON Schema you supply The result feeds code: extraction, routing, form filling Provider docs; LLM model selection
3. Tool (function) calling You describe functions; the model asks to call them and your code runs them The model needs live data or must act AI agents
4. Model Context Protocol Tools and data exposed by an MCP server, usable by any MCP-capable host The same tools should serve several assistants, editors or agents Model Context Protocol
5. Retrieval (RAG) Relevant passages are searched and added to the prompt Answers must come from your documents RAG vs agentic RAG, LangChain
6. Frameworks Libraries that package models, tools, retrieval and agent loops You want building blocks, durable state or multi-agent orchestration LangChain, LangGraph, CrewAI, Agentic AI
7. Gateway A service between your apps and providers that holds keys, routes, logs and enforces limits Several apps or providers; central cost control and audit AI governance

A few distinctions trip people up:

  • Structured output versus tool calling. If you only need data in a fixed shape, ask for structured output; tool calling is for when the model must choose an action or fetch something.
  • Tool calling versus MCP. Tool calling is the API mechanism; MCP is a protocol for packaging and sharing tools. An MCP host still turns MCP tools into tool calls for the model.
  • Retrieval versus tools. Classic retrieval always runs before the model; a retrieval tool lets the model search when, and as often as, it decides. The second is “agentic RAG”, with more flexibility and more cost.

The invented shop OrdersPortal wants its “where is my order” function available to assistants built on different providers. Tool definitions are JSON Schema in every major API, but the wrapping differs: Anthropic’s Messages API uses input_schema, while OpenAI’s Responses API uses a flat function tool with parameters. Defining the tool once in code and generating each provider’s format keeps descriptions and schemas from drifting apart.

Run with Python 3.10 or later (tested with 3.11.18); no packages are needed. Save as tool_adapter.py and run python tool_adapter.py. The generated shapes were checked against the tool parameter types in the anthropic 2.8.0 and openai 3.51.0 Python SDKs.

# tool_adapter.py - define a tool once, export it for two providers' tool-calling APIs
import inspect
import json
from typing import get_type_hints
TYPE_MAP = {str: "string", int: "integer", float: "number", bool: "boolean"}
def tool_schema(fn) -> dict:
"""Build a JSON Schema for a function's parameters from its type hints."""
hints = get_type_hints(fn)
params = inspect.signature(fn).parameters
return {
"type": "object",
"properties": {name: {"type": TYPE_MAP[hints[name]]} for name in params},
"required": [name for name, p in params.items() if p.default is inspect.Parameter.empty],
"additionalProperties": False,
}
def for_anthropic(fn) -> dict: # Messages API "tools" entry
return {"name": fn.__name__, "description": inspect.getdoc(fn),
"input_schema": tool_schema(fn), "strict": True}
def for_openai(fn) -> dict: # Responses API function tool
return {"type": "function", "name": fn.__name__, "description": inspect.getdoc(fn),
"parameters": tool_schema(fn), "strict": True}
def get_order_status(order_id: str) -> dict:
"""Look up an OrdersPortal order by ID and return its status and carrier. Read-only."""
return {"order_id": order_id, "status": "in_transit", "carrier": "ParcelCo"}
print(json.dumps(for_anthropic(get_order_status), indent=2))
print(json.dumps(for_openai(get_order_status), indent=2))

Output:

{
"name": "get_order_status",
"description": "Look up an OrdersPortal order by ID and return its status and carrier. Read-only.",
"input_schema": {
"type": "object",
"properties": {
"order_id": {
"type": "string"
}
},
"required": [
"order_id"
],
"additionalProperties": false
},
"strict": true
}
{
"type": "function",
"name": "get_order_status",
"description": "Look up an OrdersPortal order by ID and return its status and carrier. Read-only.",
"parameters": {
"type": "object",
"properties": {
"order_id": {
"type": "string"
}
},
"required": [
"order_id"
],
"additionalProperties": false
},
"strict": true
}

What to take from it:

  • The docstring is the prompt. The model chooses tools from their descriptions, so “Read-only” and the return fields are there for the model, not for humans.
  • Strict schemas need additionalProperties: false and a required list. Both providers’ strict modes rely on them to guarantee arguments that match.
  • This adapter handles flat, simple parameters only. For nested objects, enums and descriptions per field, generate schemas from Pydantic models or use a framework’s tool decorator. When the same tool must serve many hosts (editors, chat apps, agents), publish it as an MCP server instead of maintaining adapters.
Choice Gain Cost
Provider SDK directly Newest features first, least abstraction, clear errors Code tied to one provider’s request shapes
Thin internal adapter (as above) Portability for the parts you share; you control it You maintain it as APIs change
Framework (LangChain, LangGraph, CrewAI) Many integrations and patterns ready-made Another dependency with its own release pace and abstractions
MCP server One tool implementation for every MCP host A separate process or service to secure and operate
Gateway Central keys, routing, fallback, logging and budgets A shared component in every request path; must be highly available

A common, durable layering: provider SDKs inside one small client library, tools published as MCP servers when several hosts need them, a gateway once more than a few applications call models, and frameworks only where their orchestration features earn their keep.

  • Leaked keys. Keys in source code, front-end bundles or logs. Keep them server-side in a secret store, scope them per application, and rotate them (API tokens).
  • Tools with too much power. A generic “run SQL” or “call any URL” tool turns every prompt injection into a breach. Expose narrow, read-only tools by default and gate writes (Risks of agentic AI).
  • Schema drift. A tool’s implementation changes but its description does not, and the model calls it wrongly. Generate schemas from code and test them.
  • Unbounded context. Retrieval or tools return huge payloads, raising cost and pushing instructions out of focus. Limit result sizes and return only needed fields.
  • Provider outages and retirements. Every provider has incidents and retires models. Use timeouts, retries with backoff, fallbacks for critical paths, and track retirement dates (AI models in 2026).
  • No trace. Without logs of prompts, tool calls and results, nobody can explain a bad answer. Log per request with data rules applied (AI governance).
  • The task is deterministic. Validation, calculations and lookups belong in ordinary code.
  • You cannot tolerate occasional wrong answers and cannot put a check or a person after the model.
  • The data may not leave your environment and no approved hosted option or local model fits.

1. When do you choose structured output over tool calling? When you need the model’s answer in a fixed shape and no action or lookup is involved, such as extracting fields from an email. Tool calling is for when the model must decide to fetch data or act. Structured output is simpler: one request, no loop.

2. What does MCP add on top of provider tool calling? A standard way to package tools and data as a server that any MCP-capable host can use, so a tool written once works in several assistants, editors and agents. The model still sees ordinary tool definitions; MCP standardizes discovery, transport and authorization between host and server.

3. Why put a gateway in front of model providers? To centralize keys, enforce budgets and rate limits, log requests consistently, route between models and providers, and fail over during outages. The cost is a shared component in every request path, so it needs the same availability and security standards as any core service.

4. Senior scenario: five OrdersPortal teams each call two providers with their own keys, prompts and logging. Costs are rising and an audit is coming. What do you propose? Introduce a gateway and a shared client library: keys move to the gateway, each team gets a scoped credential and budget, and every request is logged with team, model and token usage. Publish shared tools (order lookup, customer profile) as MCP servers with read-only defaults instead of five private copies. Add evaluation sets per use case so model changes can be tested, and produce the inventory the audit needs from the gateway’s logs. Migrate team by team, starting with the highest spend.

  • Seven patterns cover almost every integration: direct call, structured output, tool calling, MCP, retrieval, frameworks and gateways.
  • Choose by where context comes from, who decides the next step, and what the model may change.
  • Define tools once in code and generate provider formats, or publish them through MCP when many hosts need them.
  • Keep keys server-side, tools narrow, payloads small and every request traced.