Microsoft small language models: Phi and Aion
What you’ll learn
Section titled “What you’ll learn”- What Microsoft’s small language models are, and where each one runs: Azure, your own hardware, the browser or Windows itself.
- The current Phi-4 models in Microsoft Foundry, with their context and output limits.
- How Phi Silica works on Windows today, and what its replacement by Aion Instruct means for apps.
- How to design a local-first feature that falls back to a larger model only when it has to.
- Where small models fail, and when a hosted fast tier is the better choice.
Summary
Section titled “Summary”A small language model (SLM) trades breadth for size: a few billion to about fifteen billion parameters, small enough to run on a laptop, a phone-class device or a single modest GPU, and cheap enough to run at very high volume. Microsoft has two SLM lines. Phi is the open-weight family, published on Hugging Face and offered in Microsoft Foundry and Foundry Local. Phi Silica, and its successor Aion, are on-device models built into Windows and called through the Windows AI APIs.
In November 2026 the Phi-4 family (Phi-4, Phi-4-mini-instruct, Phi-4-multimodal-instruct, Phi-4-reasoning and Phi-4-mini-reasoning) is the current Phi generation in Foundry. On Windows, Aion Instruct begins replacing Phi Silica on Insider devices this month and on retail devices in January 2027. Aion 1.0 Plan, announced at Build 2026, is a 14-billion-parameter on-device model for reasoning and tool calling.
The general LLM-versus-SLM decision is on LLM model selection. This page is the Microsoft-specific detail. Microsoft’s larger in-house models are on Microsoft MAI models.
Where Microsoft’s small models run
Section titled “Where Microsoft’s small models run”| Surface | Models (November 2026) | Who runs the hardware | Access |
|---|---|---|---|
| Microsoft Foundry | Phi-4, Phi-4-mini-instruct, Phi-4-multimodal-instruct, Phi-4-reasoning, Phi-4-mini-reasoning | Azure | Deployed endpoint, billed through Azure |
| Foundry Local | Phi-4, Phi-4-mini (aliases such as phi-4-mini pick the CPU, CUDA or NPU variant for the machine), plus other publishers’ models |
Your PC or server | Local OpenAI-compatible REST server, CLI and SDKs (preview) |
| Windows AI APIs | Phi Silica today; Aion Instruct from November 2026 (Insider) and January 2027 (retail) | The user’s Windows PC | Windows App SDK APIs |
| Microsoft Edge Prompt API | Phi-4-mini, built into Edge | The user’s device | JavaScript in a web page (developer preview) |
| Hugging Face | Phi open weights | Wherever you deploy them | Download under the model card’s license |
The surfaces differ more than the models do. Foundry gives you an SLA-backed endpoint with no hardware to manage. Foundry Local and Hugging Face give you the weights on your hardware. The Windows and Edge APIs give you a model that is already on the user’s device, but on Microsoft’s schedule, not yours.
The Phi-4 family in Foundry
Section titled “The Phi-4 family in Foundry”Limits as listed in the Foundry model catalog, November 2026.
| Model | Input | Max input | Max output | Tool calling | Typical use |
|---|---|---|---|---|---|
Phi-4 |
Text | 16,384 tokens | 16,384 tokens | No | General reasoning and chat at 14B parameters |
Phi-4-mini-instruct |
Text | 131,072 tokens | 4,096 tokens | No | Fast instruction following, classification, extraction, long input with short output |
Phi-4-multimodal-instruct |
Text, images, audio | 131,072 tokens | 4,096 tokens | No | Speech, image and text in one small model |
Phi-4-reasoning |
Text (English) | 32,768 tokens | 32,768 tokens | No | Step-by-step math and logic; returns reasoning content |
Phi-4-mini-reasoning |
Text (English) | 128,000 tokens | 128,000 tokens | No | Reasoning at the mini size |
Three things in this table shape designs:
- No tool calling in the Foundry listings. If your feature needs function calls, either parse structured output yourself or use a model that supports tools.
- Long input, short output on the mini models. 131K tokens in but 4,096 out suits summarizing or classifying a long document, not writing one.
- Reasoning variants are English-focused and spend output tokens on reasoning, so they are slower and more expensive per answer than the instruct variants.
Self-hosting changes the numbers. Foundry Local’s vLLM reference, for example, lists a maximum context of 93,520 tokens for Phi-4-mini-instruct on an NVIDIA A10 with about 7.8 GB of GPU memory. Size from the runtime’s limits, not the model card’s.
Earlier Phi generations (for context)
Release months and sizes as published by Microsoft at each release. Phi-4 models supersede these for new work; check the Foundry catalog before relying on an older model’s availability.
| Generation | Models | Sizes | What it showed |
|---|---|---|---|
| Phi-1, Phi-1.5 (2023) | Phi-1, Phi-1.5 | 1.3B | Carefully curated “textbook-quality” training data lets a very small model do well on code and common-sense tasks |
| Phi-2 (December 2023) | Phi-2 | 2.7B | The approach scaled to general language and reasoning |
| Phi-3 (April to May 2024) | mini, small, medium, vision | 3.8B, 7B, 14B, 4.2B | The first production family, with 4K/8K and 128K context variants and ONNX builds for devices |
| Phi-3.5 (August 2024) | mini, MoE, vision | 3.8B, 41.9B total (mixture of experts), 4.2B | Longer context everywhere, a mixture-of-experts variant and multi-frame vision |
| Phi-4 (December 2024 to April 2025) | Phi-4, mini, multimodal, reasoning, mini-reasoning | 14B, 3.8B, 5.6B, 14B, 3.8B | Reasoning became a design goal; one model for speech, vision and text |
Phi Silica and Aion on Windows
Section titled “Phi Silica and Aion on Windows”Phi Silica is an SLM tuned for Windows. It runs on the NPU of Copilot+ PCs (40+ TOPS) and, since an expansion, on the GPU of supported non-Copilot+ Windows 11 devices. Apps reach it through the Windows AI APIs in the Windows App SDK: text generation plus built-in summarize, rewrite and text-to-table skills, LoRA adapters for customization, and adjustable content filtering. It uses speculative decoding: a smaller draft model proposes tokens and the main model validates them in parallel. Phi Silica is a Limited Access Feature (an unlock token is required) and is not available in China.
Aion Instruct replaces it. Microsoft’s published schedule:
- November 2026: Aion Instruct rolls out to Windows Insider Preview devices. Phi Silica stays present; a controlled feature rollout decides which model is active, and a registry key lets developers test both side by side.
- January 2027: Aion Instruct rolls out to retail devices and Phi Silica is removed.
Aion Instruct does not need the Limited Access token. LoRA adapters trained for Phi Silica must be retrained for Aion Instruct with the Foundry Toolkit. A standalone sideloadable package is available for testing and LoRA training; Microsoft states that this preview package is not for customer deployment.
Aion 1.0 Plan is the larger sibling: a 14-billion-parameter on-device model for reasoning and tool calling in local agentic workflows. The application stays in control of which tools exist, what permissions they have and which actions need approval.
Running the sample
Section titled “Running the sample”The sample is plain Python (tested with Python 3.11; no packages needed). The two model calls are stubs, so the routing logic runs deterministically without downloading a model. In production, local_slm would call a Phi model through Foundry Local or the Windows AI APIs, and cloud_llm a hosted model. Save it as slm_router.py and run python3 slm_router.py.
Worked example: local-first classification for OrdersPortal
Section titled “Worked example: local-first classification for OrdersPortal”An invented shop, OrdersPortal, runs a returns desk app on Windows PCs in its warehouses. Staff type the customer’s return reason, and the app labels it so the parcel goes to the right bench. The network is unreliable, labels must appear instantly, and the reasons often contain customer email addresses that should not leave the site. That profile suits a small model on the device, with a larger model as an occasional backstop.
# slm_router.py - route OrdersPortal return reasons: local small model first, cloud model only when needed# The two model calls are stubs so the routing logic runs deterministically without a model.from dataclasses import dataclass
LABELS = {"damaged", "wrong_item", "late", "changed_mind", "other"}
@dataclassclass Result: label: str confidence: float source: str
def local_slm(text: str) -> Result: """Stand-in for a Phi model on the device (Foundry Local, Windows AI APIs, ONNX Runtime).""" t = text.lower() if "broken" in t or "cracked" in t: return Result("damaged", 0.94, "local") if "wrong size" in t or "not what i ordered" in t: return Result("wrong_item", 0.91, "local") if "arrived after" in t: return Result("late", 0.88, "local") return Result("other", 0.41, "local") # unsure: low confidence
def cloud_llm(text: str) -> Result: """Stand-in for a larger hosted model, used only when the device is online.""" print(" sent to cloud:", text) return Result("changed_mind", 0.86, "cloud")
def redact(text: str) -> str: """Strip obvious personal data before anything leaves the device.""" return " ".join("<email>" if "@" in w else w for w in text.split())
def classify(text: str, online: bool, threshold: float = 0.75) -> Result: first = local_slm(text) if first.label not in LABELS: # never trust free text from a model first = Result("other", 0.0, "local") if first.confidence >= threshold: return first if not online: return Result(first.label, first.confidence, "local-queued-for-review") return cloud_llm(redact(text))
reasons = [ "Mug arrived cracked on one side", "Sent the wrong size, not what I ordered", "Found it cheaper elsewhere, contact me at sam@example.test", "Found it cheaper elsewhere",]for i, text in enumerate(reasons): online = i != 3 # the last one is classified offline r = classify(text, online) print(f"{r.label:<13} {r.confidence:.2f} {r.source:<24} {text}")Output:
damaged 0.94 local Mug arrived cracked on one sidewrong_item 0.91 local Sent the wrong size, not what I ordered sent to cloud: Found it cheaper elsewhere, contact me at <email>changed_mind 0.86 cloud Found it cheaper elsewhere, contact me at sam@example.testother 0.41 local-queued-for-review Found it cheaper elsewhereThe design decisions behind the code:
- The small model handles the common case. Most return reasons fall into a few clear labels, which is exactly the narrow, repetitive work SLMs do well.
- Confidence decides escalation, and the threshold is a tuned number. Derive it from a labeled set of real reasons, measuring how often the local label is wrong at each confidence level.
- Redaction happens before the cloud call, never after. Only escalated text leaves the device.
- Offline is a first-class path. When the device is offline, an uncertain label is kept and queued for review instead of blocking the desk.
- Model output is validated. A label outside the allowed set becomes
other, whatever the model says.
Trade-offs
Section titled “Trade-offs”| Factor | Phi in Foundry | Phi on your hardware (Foundry Local, ONNX, vLLM) | Phi Silica / Aion on Windows | Hosted fast tier (another provider) |
|---|---|---|---|---|
| Data location | Azure region | Your machines | The user’s PC | Provider’s cloud |
| Latency | Network plus inference | Local inference only | Local inference only | Network plus inference |
| Operations | Managed | You patch, scale and monitor | Microsoft ships and updates the model | Managed |
| Model control | Choose the version | Full: pin, quantize, fine-tune | None: the OS decides the model | Choose the version |
| Quality ceiling | Phi-4 class | Phi-4 class, lower if quantized hard | Tuned for device tasks | Usually higher than Phi at similar cost |
| Works offline | No | Yes | Yes | No |
Failure modes and operations
Section titled “Failure modes and operations”- The platform model changes under you. Phi Silica is removed from retail Windows in January 2027. Apps that depend on it, and LoRA adapters trained for it, need testing and retraining on Aion Instruct before then.
- Hardware variety. A Windows AI API can be unavailable on a given PC (no supported NPU or GPU, wrong region, missing token). Check availability at run time and have a fallback; never assume the model is there.
- Quantization and runtime limits. A 4-bit build on a laptop is not the model you evaluated in Foundry. Evaluate the exact build, runtime and context limit you ship.
- Short output limits. The mini models stop at 4,096 output tokens in Foundry. Long generations get cut off; check the finish reason.
- License assumptions. Phi weights are published with a license on each model card. Read the card for the exact model and version before you redistribute weights or fine-tuned derivatives.
- Confidence that is not calibrated. Small models can be confidently wrong. If you route on confidence, measure calibration on your own data and re-check it after any model or prompt change.
- First-run downloads. Foundry Local and the Edge Prompt API download models on first use; Edge needs about 20 GB free and an unmetered connection. Plan the first-run experience and consent.
When not to use a small model
Section titled “When not to use a small model”- The task needs judgment across messy, open-ended input, such as reviewing contracts or debugging production incidents. Use a frontier or balanced LLM.
- You need tool calling and the model listing says no. Use a model with native function calling rather than parsing free text.
- A hosted fast tier already passes your evaluation at an acceptable price and data may leave the device. You avoid running and updating models yourself.
- You cannot manage the fleet. Running models on hundreds of devices means updates, telemetry and support for each hardware class.
Interview questions
Section titled “Interview questions”1. When would you choose an SLM over an LLM? When the task is narrow and repetitive (classification, extraction, routing, short rewrites), when latency or cost per call dominates, or when data must stay on the device or work offline. The choice is proven on a graded test set, not assumed from the model size.
2. What is the difference between Phi-4-mini-instruct and Phi Silica? Phi-4-mini-instruct is an open-weight model you deploy yourself or call in Foundry, with the version under your control. Phi Silica is a Windows-tuned model shipped in the OS and reached through the Windows AI APIs; Microsoft controls the version and hardware support, and it is being replaced by Aion Instruct.
3. Your local model scores well in testing but poorly on users’ laptops. What do you check? That the shipped build matches the evaluated one (quantization level, runtime, execution provider), that the context limit on the device is not truncating input, that the prompt template is the one the model expects, and that the test set reflects real user input. Then evaluate per hardware class.
4. Senior scenario: OrdersPortal wants on-device classification on a mixed fleet of 600 Windows PCs, some Copilot+ and some older, half often offline, ahead of the Phi Silica to Aion change. Design it. Put one interface in front of the model, with providers behind it chosen at run time. On each device, probe in order: the Windows AI API (Aion Instruct where it is active, Phi Silica until January 2027), then a Foundry Local Phi build matched to the hardware, then a tiny rules-based fallback that labels the obvious cases and queues the rest. Ship an evaluation harness with the app: a fixed set of labeled reasons run on first start and after every OS or model update, reporting accuracy and latency per device class to telemetry, so a regression after the Aion rollout shows up as data, not complaints. Keep LoRA adapters in a pipeline that can retrain for each base model, and test Aion side by side with the registry switch on Insider devices now. Calibrate a confidence threshold per provider, because each model’s scores mean different things. Redact before any cloud escalation, and keep the offline queue durable so nothing is lost when a laptop is closed.
Key takeaways
Section titled “Key takeaways”- Microsoft’s SLMs come in two lines: open-weight Phi (Foundry, Foundry Local, Hugging Face) and on-device Windows models (Phi Silica, now Aion).
- The Phi-4 family is current in Foundry; the mini models take 131K tokens in but return at most 4,096, and none list tool calling there.
- Aion Instruct replaces Phi Silica: Insider devices from November 2026, retail from January 2027, with LoRA adapters retrained.
- Design local-first features with a confidence threshold, redaction before escalation and an offline path.
- Evaluate the exact build and hardware you ship, and re-evaluate when the platform model changes.
Related pages
Section titled “Related pages”- LLM model selection
- Microsoft MAI models
- Running LLMs locally
- AI on Docker and Kubernetes
- AI and machine learning basics
Official references
Section titled “Official references”Checked on November 16, 2026.
- Foundry Models from partners and community: Microsoft (Phi model limits)
- Region availability for Foundry Models sold by Azure
- Get started with Foundry Local and Foundry Local CLI
- What are Windows AI APIs?
- Get started with Phi Silica (Aion Instruct transition)
- Platform card: Phi Silica
- Run AI at the edge on Windows IoT Enterprise (Aion 1.0 Plan)
- Microsoft Edge: Prompt API with Phi-4-mini
- Phi-4-mini-instruct model card (Hugging Face)