Python ChainMap: Layered Dictionary Lookups
What you’ll learn
Section titled “What you’ll learn”- How
collections.ChainMapchains several mappings into a single read view. - How layered config lookups give you defaults with per-call overrides without copying.
- How
new_childandparentsbuild layered scopes without mutation. - When ChainMap beats
dict.update, and the performance trade-off it carries.
Summary
Section titled “Summary”collections.ChainMap groups multiple mappings into one object that reads like a single dictionary. Lookups search each mapping in order and return the first hit; writes, deletes, and new keys go to the first mapping only. It is the standard-library answer to layered configuration: a chain of “environment, config file, defaults” resolves with one lookup and no data copied. ChainMap is a view, so changes to the underlying dicts show through immediately.
The problem it solves
Section titled “The problem it solves”Layered configuration is everywhere in a service. OrdersPortal reads a setting from the environment first, then from a config file, then from built-in defaults. The naive fix merges everything into one dict with repeated update calls, which copies the data and loses the layering: you can no longer tell which layer a value came from, and a change to a default does not appear until you rebuild the merged dict.
Merged approach:
defaults = {"port": 8000, "debug": False, "region": "us"}config_file = {"port": 9000, "region": "eu"}environment = {"debug": True}
merged = {}merged.update(defaults)merged.update(config_file)merged.update(environment)print(merged)# {'port': 9000, 'debug': True, 'region': 'eu'}This works, but every update copies values, and merged is now a snapshot. If defaults changes later, merged does not reflect it.
How it works
Section titled “How it works”A ChainMap takes mappings as arguments and searches them left to right. Reading a key returns the first mapping that has it; reading a missing key raises KeyError. Writing a key sets it on the first mapping; deleting a key removes it from the first mapping only, and raises if the key is absent there even if a later map has it.
from collections import ChainMap
defaults = {"port": 8000, "debug": False, "region": "us"}config_file = {"port": 9000, "region": "eu"}environment = {"debug": True}
config = ChainMap(environment, config_file, defaults)print(config["port"]) # 9000 <- config_file wins over defaultsprint(config["debug"]) # True <- environment winsprint(config["region"]) # 'eu'print(list(config)) # ['debug', 'port', 'region'] <- union, first-seen orderThe order of the arguments is the search order: the first mapping is checked first and is also the target for writes. list(config) yields every key across all maps, with duplicates kept in first-seen order.
new_child returns a new ChainMap with a fresh empty map prepended, so writes land there and leave the originals untouched. parents drops the first map, giving you the chain below the current layer. Both return new ChainMaps rather than mutating, which keeps the layering composable.
from collections import ChainMap
defaults = {"port": 8000, "debug": False, "region": "us"}config_file = {"port": 9000, "region": "eu"}environment = {"debug": True}config = ChainMap(environment, config_file, defaults)
# a per-request override layer that does not pollute config_filerequest = config.new_child({"port": 7000})print(request["port"]) # 7000 <- the new layer winsprint(config["port"]) # 9000 <- the original is untouchedprint(request.parents["port"]) # 9000 <- parents drops the new layerBecause new_child and parents are pure operations, you can build a stack of scopes for a request, a transaction, or a transaction-with-nested-override, each independent.
Realistic example
Section titled “Realistic example”OrdersPortal resolves connection settings from three layers and adds a per-request override for a dry-run preview, all without copying the base configuration.
from collections import ChainMap
# three layers: environment beats config_file beats defaultsdefaults = {"host": "orders.example.test", "port": 8000, "timeout": 30, "dry_run": False}config_file = {"port": 9000, "timeout": 60}environment = {"host": "staging.orders.example.test"}
base = ChainMap(environment, config_file, defaults)
def resolve(request_overrides): # each call gets its own override layer; base is never mutated cfg = base.new_child(request_overrides) return {k: cfg[k] for k in ("host", "port", "timeout", "dry_run")}
print(resolve({"dry_run": True}))# {'host': 'staging.orders.example.test', 'port': 9000, 'timeout': 60, 'dry_run': True}print(resolve({}))# {'host': 'staging.orders.example.test', 'port': 9000, 'timeout': 60, 'dry_run': False}print(resolve({})["dry_run"])# False <- the override layer was empty, so it did not stickEach call to resolve builds a fresh override layer with new_child, so a dry_run override in one request never leaks into another. The base chain is never written to, which is the whole point of layering.
Production patterns
Section titled “Production patterns”- Keep writes off the shared base. Use
new_childfor per-request or per-transaction overrides so concurrent requests cannot clobber each other through the first map. - Resolve once, not in a hot loop. A ChainMap lookup scans maps until it hits, so resolving the same keys repeatedly is slower than a flat dict. Build a resolved dict once and pass that around.
- Treat it as a read view. Reads are cheap and always current; writes and deletes are surprising because they touch only the first map. If your code mutates a lot, you probably want a plain dict.
Common mistakes
Section titled “Common mistakes”- Writing a key and expecting it in every layer.
chain[key] = valuesets it on the first map only. The other maps are untouched, which is correct but surprising if you expected a copy. - Deleting a key that a later layer has.
del chain[key]removes it from the first map and raisesKeyErrorif it is not there, even thoughkey in chainis True from a deeper map. - Assuming independence from the underlying dicts. ChainMap is a view. Mutate
defaultsand the chain sees the new value immediately, which is great for live config but a trap for code that assumed a snapshot. - Using ChainMap for heavy writes. It is optimized for layered reads. A pattern of frequent writes and deletes is clearer and faster with a plain dict.
- Confusing search order with argument order. The first argument is searched first and is the write target. Reversing the arguments inverts the precedence.
When to use it and when not to
Section titled “When to use it and when not to”| Need | Use |
|---|---|
| Layered defaults with per-scope overrides, read-mostly | ChainMap |
| A single flat configuration you mutate freely | dict |
| A one-time merge of several sources into a snapshot | dict.update |
| Many lookups against a resolved config in a hot path | resolve to a dict first |
ChainMap wins when the layering itself is the feature: you want the environment to override the file, and you want to add and remove override layers without rebuilding. For a one-time merge you will mutate afterward, dict.update is simpler.
Compatibility
Section titled “Compatibility”collections.ChainMap arrived in Python 3.3. The new_child and parents methods have been available since then. The examples here run on Python 3.9 and later. ChainMap takes any mapping, not just dict, so it works with os.environ, types.MappingProxyType, and custom mappings.
Interview questions
Section titled “Interview questions”1. How does a ChainMap lookup differ from a merged dict? A merged dict copies every key into one mapping at merge time. A ChainMap keeps the mappings separate and scans them in order on each lookup, returning the first hit. The ChainMap is a live view; the merged dict is a snapshot.
2. Where do writes go, and why? Writes and deletes go to the first mapping only. ChainMap treats the first map as the active layer, so updates do not silently change the defaults or the config file underneath.
3. What is new_child for?
It returns a new ChainMap with a fresh empty map prepended, so writes land there and leave the originals untouched. It is the way to add a temporary override layer without mutating the base.
4. Senior scenario: a shared ChainMap config is intermittently returning wrong values across concurrent requests. What happened?
A request wrote an override directly to the base chain’s first map instead of calling new_child, so concurrent requests now see each other’s overrides through the shared mapping. The fix is to give each request its own override layer with new_child and never write to the shared base, or to resolve the config to an immutable dict per request.
Key takeaways
Section titled “Key takeaways”- ChainMap chains mappings into one read view; the first map is searched first and is the write target.
new_childadds a temporary override layer;parentsdrops the top layer. Both are pure.- It is a live view: changes to the underlying dicts show through immediately.
- Use it for layered, read-mostly config; use a plain dict for heavy writes.
Related pages
Section titled “Related pages”- the collections module — the family ChainMap belongs to.
- dictionaries — the mappings ChainMap chains.
- ordereddict — when order, not layering, is what you need.
Official references
Section titled “Official references”- Python 3 —
collections.ChainMap— the class, its methods, and examples. - Python 3 —
types.MappingProxyType— a read-only mapping view, often used inside a chain. - PEP 411 — ChainMap examples — the recipes section in the standard library docs.