Microservices communication: gateway, discovery, breakers
When a monolith is split into services, method calls become network calls. Three problems appear at once. Clients now face many services with different addresses, versions and security rules. Services move: instances start, stop and scale, so hard-coded URLs break. And remote calls fail or slow down, and one struggling dependency can drag every caller down with it. The patterns on this page answer those three problems: an API gateway gives clients one entry point, service discovery turns logical names into current addresses, and circuit breakers stop callers hammering a dependency that is already failing.
The samples follow an invented OrdersPortal system with orders, catalog and inventory services. They are .NET 10 file-based programs (dotnet run file.cs) that start their own local services on 127.0.0.1, so they need no containers. All were run with .NET SDK 10.0.115. For asynchronous, message-based communication, see messaging patterns.
Topics on this page
Section titled “Topics on this page”| Pattern | Solves | Typical tools |
|---|---|---|
| API gateway | One public entry point; routing, auth, rate limiting and aggregation at the edge. | YARP, Azure API Management, Amazon API Gateway, Kong, Ocelot |
| Service discovery | Finding healthy instances of a service by name. | Kubernetes Services and DNS, Consul, Eureka, Microsoft.Extensions.ServiceDiscovery |
| Circuit breaker | Failing fast when a dependency is unhealthy, and recovering automatically. | Polly 8, Microsoft.Extensions.Http.Resilience, service mesh outlier detection |
API gateway
Section titled “API gateway”An API gateway is a reverse proxy that sits between clients and your services. Clients call one host (api.ordersportal.example.test); the gateway routes /orders/* to the orders service and /products/* to catalog, so internal addresses, service boundaries and refactorings stay hidden. Because every request passes through it, the gateway is the natural place for edge concerns: TLS termination, authentication and API keys, rate limiting, request size limits, CORS, response caching and request logging. A variant, the backend for frontend (BFF), is a gateway per client type (web, mobile) that can also aggregate several service calls into one response shaped for that client.
In .NET, YARP (Yet Another Reverse Proxy) is Microsoft’s open-source reverse proxy library: you host it in an ordinary ASP.NET Core app, so you can add any middleware, and configure routes and clusters from appsettings.json or code. Managed gateways such as Azure API Management or Amazon API Gateway add developer portals, subscription keys, quotas and analytics without code. Ocelot is an older open-source .NET gateway library, and Kong, Tyk and Apigee are popular platform-independent gateways.
#:sdk Microsoft.NET.Sdk.Web#:package Yarp.ReverseProxy@2.3.0#:property PublishAot=falseusing Yarp.ReverseProxy.Configuration;
// Two backend services, each on its own port (normally separate deployments).var orders = StartService("orders", "http://127.0.0.1:5131", app => app.MapGet("/api/orders/{id:int}", (int id) => new { id, status = "Shipped" }));var catalog = StartService("catalog", "http://127.0.0.1:5132", app => app.MapGet("/api/products/{sku}", (string sku) => new { sku, price = 49.99m }));
// The gateway: one public entry point that routes by path and adds cross-cutting rules.var builder = WebApplication.CreateBuilder();builder.Logging.ClearProviders();builder.WebHost.UseUrls("http://127.0.0.1:5130");builder.Services.AddReverseProxy().LoadFromMemory( routes: [ new RouteConfig { RouteId = "orders", ClusterId = "orders", Match = new() { Path = "/orders/{**rest}" }, Transforms = [new Dictionary<string, string> { ["PathPattern"] = "/api/orders/{**rest}" }] }, new RouteConfig { RouteId = "catalog", ClusterId = "catalog", Match = new() { Path = "/products/{**rest}" }, Transforms = [new Dictionary<string, string> { ["PathPattern"] = "/api/products/{**rest}" }] }, ], clusters: [ new ClusterConfig { ClusterId = "orders", Destinations = new Dictionary<string, DestinationConfig> { ["o1"] = new() { Address = "http://127.0.0.1:5131" } } }, new ClusterConfig { ClusterId = "catalog", Destinations = new Dictionary<string, DestinationConfig> { ["c1"] = new() { Address = "http://127.0.0.1:5132" } } }, ]);var gateway = builder.Build();gateway.Use(async (ctx, next) => // example cross-cutting rule at the edge{ if (!ctx.Request.Headers.ContainsKey("X-Api-Key")) { ctx.Response.StatusCode = 401; return; } await next(ctx);});gateway.MapReverseProxy();await gateway.StartAsync();
using var http = new HttpClient { BaseAddress = new Uri("http://127.0.0.1:5130") };http.DefaultRequestHeaders.Add("X-Api-Key", "demo-key");Console.WriteLine($"GET /orders/1001 -> {await http.GetStringAsync("/orders/1001")}");Console.WriteLine($"GET /products/KB-101 -> {await http.GetStringAsync("/products/KB-101")}");using var anonymous = new HttpClient();Console.WriteLine($"No API key -> {(int)(await anonymous.GetAsync("http://127.0.0.1:5130/orders/1001")).StatusCode}");
await gateway.StopAsync(); await orders.StopAsync(); await catalog.StopAsync();
static WebApplication StartService(string name, string url, Action<WebApplication> map){ var b = WebApplication.CreateBuilder(); b.Logging.ClearProviders(); b.WebHost.UseUrls(url); var app = b.Build(); map(app); app.StartAsync().GetAwaiter().GetResult(); return app;}Output:
GET /orders/1001 -> {"id":1001,"status":"Shipped"}GET /products/KB-101 -> {"sku":"KB-101","price":49.99}No API key -> 401Pitfalls:
- Business logic in the gateway. Pricing rules or order validation in gateway code create a new monolith that every team must change. Keep the gateway to routing and edge policy.
- A single point of failure. Run several gateway instances behind a load balancer and give the gateway its own timeouts and resilience to backends.
- Trusting the gateway alone. Services should still authenticate requests (validate the forwarded token) so a misconfigured route or internal caller cannot bypass security.
Service discovery
Section titled “Service discovery”Service discovery maps a logical name (catalog) to the network addresses of its healthy instances. With client-side discovery, the caller asks a registry (Consul, Eureka) for instances and load-balances between them itself. With server-side discovery, the caller sends the request to a stable address and the platform routes it: a Kubernetes Service named catalog gets a stable virtual IP and a DNS name (catalog.default.svc.cluster.local, or just catalog inside the same namespace), and kube-proxy spreads connections across ready pods. Azure Container Apps and most service meshes work the same way. Registries only route to instances that pass health checks, which in ASP.NET Core come from AddHealthChecks() and MapHealthChecks("/health"); Kubernetes uses readiness probes for the same purpose.
Microsoft.Extensions.ServiceDiscovery (part of the .NET Aspire stack, usable in any .NET app) lets code use names such as http://catalog and resolves them from configuration, DNS SRV records or a pass-through to the platform, so the same code works on a laptop (configuration) and in Kubernetes (platform DNS). A scheme list such as https+http://catalog prefers HTTPS and falls back to HTTP.
#:sdk Microsoft.NET.Sdk.Web#:package Microsoft.Extensions.ServiceDiscovery@10.11.0#:property PublishAot=false
// Two instances of the catalog service, as an orchestrator would start them.var instanceA = await StartCatalogAsync("http://127.0.0.1:5141", "catalog-a");var instanceB = await StartCatalogAsync("http://127.0.0.1:5142", "catalog-b");
// The caller: knows the logical name "catalog", never the addresses.var builder = Host.CreateApplicationBuilder();builder.Logging.ClearProviders();builder.Configuration.AddInMemoryCollection(new Dictionary<string, string?>{ // In Kubernetes or .NET Aspire this comes from the platform; here from configuration. ["Services:catalog:http:0"] = "http://127.0.0.1:5141", ["Services:catalog:http:1"] = "http://127.0.0.1:5142",});builder.Services.AddServiceDiscovery();builder.Services.AddHttpClient<CatalogClient>(c => c.BaseAddress = new Uri("http://catalog")) .AddServiceDiscovery();using var host = builder.Build();
var client = host.Services.GetRequiredService<CatalogClient>();for (int i = 0; i < 4; i++) Console.WriteLine(await client.GetPriceAsync("KB-101"));
await instanceA.StopAsync(); await instanceB.StopAsync();
static async Task<WebApplication> StartCatalogAsync(string url, string instance){ var b = WebApplication.CreateBuilder(); b.Logging.ClearProviders(); b.WebHost.UseUrls(url); var app = b.Build(); app.MapGet("/api/products/{sku}/price", (string sku) => $"{sku} 49.99 (served by {instance})"); await app.StartAsync(); return app;}
public sealed class CatalogClient(HttpClient http){ public Task<string> GetPriceAsync(string sku) => http.GetStringAsync($"/api/products/{sku}/price");}Output (requests are spread across the two instances; which one answers first can vary):
KB-101 49.99 (served by catalog-b)KB-101 49.99 (served by catalog-a)KB-101 49.99 (served by catalog-b)KB-101 49.99 (served by catalog-a)Pitfalls:
- Liveness and readiness confused. A health endpoint that checks the database used as a liveness probe makes Kubernetes restart every pod during a database blip. Use readiness for “can serve traffic” and keep liveness to “the process is not stuck”.
- Caching addresses forever.
HttpClientinstances created once with resolved IPs, or long-lived connections, keep talking to removed instances. UseIHttpClientFactory, and setPooledConnectionLifetimewhen relying on DNS. - Adding a registry you do not need. On Kubernetes or Container Apps, platform DNS is usually enough; Consul or Eureka add another stateful system to operate.
Circuit breaker
Section titled “Circuit breaker”A circuit breaker wraps calls to a dependency and tracks failures. While closed, calls pass through. When failures cross a threshold (in Polly 8, a failure ratio over a sampling window with a minimum number of calls), it opens: calls fail immediately with BrokenCircuitException without touching the dependency, which protects the caller’s threads and connections and gives the dependency room to recover. After the break duration it goes half-open and lets a trial call through: success closes it, failure opens it again. Pair it with a fallback (cached data, a default, a degraded feature) so users see something useful while it is open.
In .NET use Polly 8’s resilience pipelines, or for HttpClient the Microsoft.Extensions.Http.Resilience package: AddStandardResilienceHandler() adds a tested pipeline of rate limiter, total timeout, retry, circuit breaker and per-attempt timeout (outermost to innermost). In that standard order the retry sits outside the breaker, so each attempt counts towards opening it and an open breaker makes remaining retries fail fast. The older Microsoft.Extensions.Http.Polly package and Polly v7 Policy API are deprecated for new code.
#:package Polly.Core@8.9.1using Polly;using Polly.CircuitBreaker;using Polly.Fallback;using Polly.Timeout;
var time = new FakeTimeProvider(); // lets the sample move the clock instead of sleepingvar inventory = new FlakyInventoryService();
var pipeline = new ResiliencePipelineBuilder<int>{ TimeProvider = time,} .AddFallback(new FallbackStrategyOptions<int> { // Outermost: when everything else gives up, answer from cache instead of failing the page. ShouldHandle = new PredicateBuilder<int>().Handle<BrokenCircuitException>().Handle<HttpRequestException>(), FallbackAction = _ => Outcome.FromResultAsValueTask(-1), }) .AddCircuitBreaker(new CircuitBreakerStrategyOptions<int> { FailureRatio = 0.5, // open when at least half of the calls fail... MinimumThroughput = 4, // ...out of at least 4 calls... SamplingDuration = TimeSpan.FromSeconds(30),// ...within a 30 second window BreakDuration = TimeSpan.FromSeconds(15), // stay open for 15 seconds, then try one call ShouldHandle = new PredicateBuilder<int>().Handle<HttpRequestException>(), OnOpened = a => { Console.WriteLine($" breaker OPEN for {a.BreakDuration.TotalSeconds}s"); return default; }, OnHalfOpened = _ => { Console.WriteLine(" breaker HALF-OPEN: trial call"); return default; }, OnClosed = _ => { Console.WriteLine(" breaker CLOSED"); return default; }, }) .AddTimeout(TimeSpan.FromSeconds(2)) // innermost: bound each attempt .Build();
async Task CallAsync(string label){ int stock = await pipeline.ExecuteAsync(async ct => await inventory.GetStockAsync("KB-101", ct)); Console.WriteLine($"{label}: {(stock < 0 ? "fallback (cached value shown)" : $"stock {stock}")} [real calls so far: {inventory.Calls}]");}
inventory.Healthy = false;for (int i = 1; i <= 6; i++) await CallAsync($"call {i}");
time.Advance(TimeSpan.FromSeconds(16)); // break duration elapses; the service has recoveredinventory.Healthy = true;await CallAsync("call 7");await CallAsync("call 8");
public sealed class FlakyInventoryService{ public bool Healthy { get; set; } = true; public int Calls { get; private set; } public Task<int> GetStockAsync(string sku, CancellationToken ct) { Calls++; return Healthy ? Task.FromResult(42) : Task.FromException<int>(new HttpRequestException("503 from inventory")); }}
public sealed class FakeTimeProvider : TimeProvider{ private DateTimeOffset _now = new(2026, 11, 18, 9, 0, 0, TimeSpan.Zero); public override DateTimeOffset GetUtcNow() => _now; public void Advance(TimeSpan by) => _now += by;}Output:
call 1: fallback (cached value shown) [real calls so far: 1]call 2: fallback (cached value shown) [real calls so far: 2]call 3: fallback (cached value shown) [real calls so far: 3] breaker OPEN for 15scall 4: fallback (cached value shown) [real calls so far: 4]call 5: fallback (cached value shown) [real calls so far: 4]call 6: fallback (cached value shown) [real calls so far: 4] breaker HALF-OPEN: trial call breaker CLOSEDcall 7: stock 42 [real calls so far: 5]call 8: stock 42 [real calls so far: 6]The breaker opened on the fourth failure (four calls, 100% failing), so calls 5 and 6 never reached the inventory service. After the break duration the trial call succeeded and the circuit closed. For an HttpClient, the equivalent is one line: builder.Services.AddHttpClient<InventoryClient>().AddStandardResilienceHandler();.
Pitfalls:
- Retrying everything. Retries multiply load on a struggling service and duplicate non-idempotent requests; disable retries for
POST(options.Retry.DisableForUnsafeHttpMethods()) unless the operation is idempotent, and use exponential back-off with jitter. - One breaker for many dependencies. A shared breaker opens for all of them when one fails. Use one pipeline per dependency (the standard handler can select by URL authority).
- Timeouts longer than the caller’s. If the gateway gives up after 10 seconds but the service waits 30 seconds per attempt, work continues for nobody. Set the total timeout below the caller’s budget.
Choosing between them
Section titled “Choosing between them”These patterns combine rather than compete. A typical path for one request: the client calls the gateway, which authenticates and routes by path; the gateway and every service-to-service HttpClient resolve targets by name through service discovery (often just Kubernetes DNS); and every outgoing call goes through a resilience pipeline with timeouts, limited retries and a circuit breaker. Prefer the platform’s built-ins first (managed gateway or ingress, platform DNS, a service mesh’s outlier detection) and add libraries where you need code-level control. If a call does not need an immediate answer, avoid the synchronous chain entirely and send a message instead.
Interview questions
Section titled “Interview questions”What belongs in an API gateway and what does not? Edge concerns that apply to many services: routing, TLS, authentication, rate limiting, CORS, caching and request logging. Business rules and data transformation specific to one domain do not; they belong in the services, otherwise the gateway becomes a shared bottleneck owned by everyone.
What is the difference between client-side and server-side discovery? In client-side discovery the caller queries a registry and picks an instance itself; in server-side discovery the caller uses a stable address and a load balancer or the platform (a Kubernetes Service) picks the instance.
Explain the three states of a circuit breaker. Closed passes calls and counts failures; open fails calls immediately for the break duration; half-open allows trial calls and returns to closed on success or open on failure.
How should retries and circuit breakers be ordered? In Microsoft’s standard resilience handler the order is total timeout, retry, circuit breaker, attempt timeout: each attempt is timed and counted by the breaker, and once it opens the retry stops quickly because the breaker rejects attempts immediately. Retries must be limited, use back-off with jitter, and be disabled for non-idempotent operations.
Senior scenario: the OrdersPortal checkout page times out whenever the recommendations service is slow. What do you change? Decide which dependencies are critical: recommendations is optional. Give its client a short total timeout well under the page budget, a circuit breaker and a fallback (empty list or cached recommendations), and no retries on the hot path. Isolate it with a concurrency limiter (bulkhead) so slow calls cannot exhaust connections used by payment and inventory. Better still, load recommendations asynchronously in the UI or precompute them from events. Add metrics on breaker state and dependency latency, and alert on breaker openings.
Related pages
Section titled “Related pages”- Microservices overview
- Messaging patterns
- ASP.NET Core Web API: controllers, routing and pipeline
- Architecture patterns: event-driven architecture
- Structural patterns: decorator
Official references
Section titled “Official references”- API gateway pattern versus direct client-to-microservice communication, .NET microservices guide
- YARP documentation
- Service discovery in .NET
- Kubernetes: DNS for Services and Pods
- Introduction to resilient app development in .NET
- Build resilient HTTP apps: standard resilience handler
- Circuit Breaker pattern, Azure Architecture Center
- Polly documentation: circuit breaker strategy