Skip to content

Python ChainMap: Layered Dictionary Lookups

  • How collections.ChainMap chains several mappings into a single read view.
  • How layered config lookups give you defaults with per-call overrides without copying.
  • How new_child and parents build layered scopes without mutation.
  • When ChainMap beats dict.update, and the performance trade-off it carries.

collections.ChainMap groups multiple mappings into one object that reads like a single dictionary. Lookups search each mapping in order and return the first hit; writes, deletes, and new keys go to the first mapping only. It is the standard-library answer to layered configuration: a chain of “environment, config file, defaults” resolves with one lookup and no data copied. ChainMap is a view, so changes to the underlying dicts show through immediately.

Layered configuration is everywhere in a service. OrdersPortal reads a setting from the environment first, then from a config file, then from built-in defaults. The naive fix merges everything into one dict with repeated update calls, which copies the data and loses the layering: you can no longer tell which layer a value came from, and a change to a default does not appear until you rebuild the merged dict.

Merged approach:

defaults = {"port": 8000, "debug": False, "region": "us"}
config_file = {"port": 9000, "region": "eu"}
environment = {"debug": True}
merged = {}
merged.update(defaults)
merged.update(config_file)
merged.update(environment)
print(merged)
# {'port': 9000, 'debug': True, 'region': 'eu'}

This works, but every update copies values, and merged is now a snapshot. If defaults changes later, merged does not reflect it.

A ChainMap takes mappings as arguments and searches them left to right. Reading a key returns the first mapping that has it; reading a missing key raises KeyError. Writing a key sets it on the first mapping; deleting a key removes it from the first mapping only, and raises if the key is absent there even if a later map has it.

from collections import ChainMap
defaults = {"port": 8000, "debug": False, "region": "us"}
config_file = {"port": 9000, "region": "eu"}
environment = {"debug": True}
config = ChainMap(environment, config_file, defaults)
print(config["port"]) # 9000 <- config_file wins over defaults
print(config["debug"]) # True <- environment wins
print(config["region"]) # 'eu'
print(list(config)) # ['debug', 'port', 'region'] <- union, first-seen order

The order of the arguments is the search order: the first mapping is checked first and is also the target for writes. list(config) yields every key across all maps, with duplicates kept in first-seen order.

new_child returns a new ChainMap with a fresh empty map prepended, so writes land there and leave the originals untouched. parents drops the first map, giving you the chain below the current layer. Both return new ChainMaps rather than mutating, which keeps the layering composable.

from collections import ChainMap
defaults = {"port": 8000, "debug": False, "region": "us"}
config_file = {"port": 9000, "region": "eu"}
environment = {"debug": True}
config = ChainMap(environment, config_file, defaults)
# a per-request override layer that does not pollute config_file
request = config.new_child({"port": 7000})
print(request["port"]) # 7000 <- the new layer wins
print(config["port"]) # 9000 <- the original is untouched
print(request.parents["port"]) # 9000 <- parents drops the new layer

Because new_child and parents are pure operations, you can build a stack of scopes for a request, a transaction, or a transaction-with-nested-override, each independent.

OrdersPortal resolves connection settings from three layers and adds a per-request override for a dry-run preview, all without copying the base configuration.

from collections import ChainMap
# three layers: environment beats config_file beats defaults
defaults = {"host": "orders.example.test", "port": 8000, "timeout": 30, "dry_run": False}
config_file = {"port": 9000, "timeout": 60}
environment = {"host": "staging.orders.example.test"}
base = ChainMap(environment, config_file, defaults)
def resolve(request_overrides):
# each call gets its own override layer; base is never mutated
cfg = base.new_child(request_overrides)
return {k: cfg[k] for k in ("host", "port", "timeout", "dry_run")}
print(resolve({"dry_run": True}))
# {'host': 'staging.orders.example.test', 'port': 9000, 'timeout': 60, 'dry_run': True}
print(resolve({}))
# {'host': 'staging.orders.example.test', 'port': 9000, 'timeout': 60, 'dry_run': False}
print(resolve({})["dry_run"])
# False <- the override layer was empty, so it did not stick

Each call to resolve builds a fresh override layer with new_child, so a dry_run override in one request never leaks into another. The base chain is never written to, which is the whole point of layering.

  • Keep writes off the shared base. Use new_child for per-request or per-transaction overrides so concurrent requests cannot clobber each other through the first map.
  • Resolve once, not in a hot loop. A ChainMap lookup scans maps until it hits, so resolving the same keys repeatedly is slower than a flat dict. Build a resolved dict once and pass that around.
  • Treat it as a read view. Reads are cheap and always current; writes and deletes are surprising because they touch only the first map. If your code mutates a lot, you probably want a plain dict.
  • Writing a key and expecting it in every layer. chain[key] = value sets it on the first map only. The other maps are untouched, which is correct but surprising if you expected a copy.
  • Deleting a key that a later layer has. del chain[key] removes it from the first map and raises KeyError if it is not there, even though key in chain is True from a deeper map.
  • Assuming independence from the underlying dicts. ChainMap is a view. Mutate defaults and the chain sees the new value immediately, which is great for live config but a trap for code that assumed a snapshot.
  • Using ChainMap for heavy writes. It is optimized for layered reads. A pattern of frequent writes and deletes is clearer and faster with a plain dict.
  • Confusing search order with argument order. The first argument is searched first and is the write target. Reversing the arguments inverts the precedence.
Need Use
Layered defaults with per-scope overrides, read-mostly ChainMap
A single flat configuration you mutate freely dict
A one-time merge of several sources into a snapshot dict.update
Many lookups against a resolved config in a hot path resolve to a dict first

ChainMap wins when the layering itself is the feature: you want the environment to override the file, and you want to add and remove override layers without rebuilding. For a one-time merge you will mutate afterward, dict.update is simpler.

collections.ChainMap arrived in Python 3.3. The new_child and parents methods have been available since then. The examples here run on Python 3.9 and later. ChainMap takes any mapping, not just dict, so it works with os.environ, types.MappingProxyType, and custom mappings.

1. How does a ChainMap lookup differ from a merged dict? A merged dict copies every key into one mapping at merge time. A ChainMap keeps the mappings separate and scans them in order on each lookup, returning the first hit. The ChainMap is a live view; the merged dict is a snapshot.

2. Where do writes go, and why? Writes and deletes go to the first mapping only. ChainMap treats the first map as the active layer, so updates do not silently change the defaults or the config file underneath.

3. What is new_child for? It returns a new ChainMap with a fresh empty map prepended, so writes land there and leave the originals untouched. It is the way to add a temporary override layer without mutating the base.

4. Senior scenario: a shared ChainMap config is intermittently returning wrong values across concurrent requests. What happened? A request wrote an override directly to the base chain’s first map instead of calling new_child, so concurrent requests now see each other’s overrides through the shared mapping. The fix is to give each request its own override layer with new_child and never write to the shared base, or to resolve the config to an immutable dict per request.

  • ChainMap chains mappings into one read view; the first map is searched first and is the write target.
  • new_child adds a temporary override layer; parents drops the top layer. Both are pure.
  • It is a live view: changes to the underlying dicts show through immediately.
  • Use it for layered, read-mostly config; use a plain dict for heavy writes.