Python weak references: caches and observers without leaks
What you’ll learn
Section titled “What you’ll learn”- What a weak reference is and how it differs from an ordinary (strong) reference.
- How to use
weakref.ref,proxy,WeakValueDictionary,WeakKeyDictionary,WeakSet,WeakMethodandfinalize. - Which objects can be weakly referenced, and why
int,str,tupleand__slots__classes often cannot. - Why a weak cache can have a hit rate of zero, and why a weak reference to a bound method dies at once.
- When a weak reference is the right ownership signal and when an LRU cache or explicit cleanup is better.
Summary
Section titled “Summary”A normal reference keeps an object alive: CPython frees an object when its reference count reaches zero. A weak reference, created with the weakref module, refers to an object without adding to that count. While something else holds the object, the weak reference can return it; once the last strong reference disappears, the object is freed and the weak reference reports that it is gone (calling it returns None). Weak references have been in the standard library since Python 2.1 (PEP 205) and are fully current in Python 3.14.
The problem it solves
Section titled “The problem it solves”A cache keyed by customer ID, a registry of open sessions or an event bus that stores callbacks all hold strong references, so every object they have seen stays alive until someone removes it. That is the most common Python “memory leak”: nothing is lost, it is just never let go.
In OrdersPortal, the request handler loads Customer objects and wants to share one instance between concurrent pieces of work on the same customer. A plain dict does that, but it also keeps every customer alive for the life of the process. A WeakValueDictionary shares the instance only while something is using it.
Running the samples: each sample is a complete script. Save it as a .py file and run it with Python 3.14 (python3 sample.py). The outputs shown were produced with CPython 3.14.6. The exact moment an object is freed is a CPython implementation detail (reference counting); other implementations such as PyPy may free it later.
import weakref
class Customer: def __init__(self, customer_id: str) -> None: self.customer_id = customer_id
def __repr__(self) -> str: return f"Customer({self.customer_id!r})"
def load_customer(customer_id: str) -> Customer: print(f" loading {customer_id} from the database") return Customer(customer_id)
# Before: a plain dict keeps every customer alive for the life of the process.strong_cache: dict[str, Customer] = {}
# After: entries vanish once no request is using that customer any more.weak_cache: weakref.WeakValueDictionary[str, Customer] = weakref.WeakValueDictionary()
def get_customer(customer_id: str) -> Customer: customer = weak_cache.get(customer_id) if customer is None: customer = load_customer(customer_id) weak_cache[customer_id] = customer return customer
c = Customer("C-0")strong_cache["C-0"] = cdel cprint("strong cache still holds:", list(strong_cache.values()))
a = get_customer("C-1") # loadsb = get_customer("C-1") # cache hit while `a` is aliveprint(a is b, len(weak_cache))del a, b # last strong references goneprint(len(weak_cache))get_customer("C-1") # loads againOutput:
strong cache still holds: [Customer('C-0')] loading C-1 from the databaseTrue 10 loading C-1 from the databaseThe strong cache still holds C-0 after the program dropped it. The weak cache shared one C-1 instance while it was in use and forgot it when the last user let go. Note the single get followed by a test of the result: a separate key in weak_cache check can race with collection.
How it works
Section titled “How it works”weakref.ref(obj, callback=None) returns a reference object. Call it to get the referent back, or None once it has been collected. The optional callback receives the dead reference object, not the referent, which no longer exists. weakref.proxy(obj) returns an object that forwards attribute access and operators to the referent and raises ReferenceError once the referent is gone.
import weakref
class Cart: pass
cart = Cart()r = weakref.ref(cart, lambda ref: print(" callback: cart collected"))print(r() is cart, weakref.getweakrefcount(cart))
p = weakref.proxy(cart)p.total = 42.0 # behaves like the objectprint(cart.total)
del cart # callback runs immediately in CPythonprint(r())try: p.totalexcept ReferenceError as exc: print("ReferenceError:", exc)
for value in (1, "ORD-1", (1, 2), [1], {"a": 1}): try: weakref.ref(value) except TypeError as exc: print("TypeError:", exc)
class Tags(list): # a list subclass can be weakly referenced passtags = Tags(["gift"])print(weakref.ref(tags)() is tags)
class Slotted: __slots__ = ("sku",)class SlottedWeak: __slots__ = ("sku", "__weakref__")for cls in (Slotted, SlottedWeak): try: weakref.ref(cls()) print(cls.__name__, "ok") except TypeError as exc: print(cls.__name__, "TypeError:", exc)Output:
True 142.0 callback: cart collectedNoneReferenceError: weakly-referenced object no longer existsTypeError: cannot create weak reference to 'int' objectTypeError: cannot create weak reference to 'str' objectTypeError: cannot create weak reference to 'tuple' objectTypeError: cannot create weak reference to 'list' objectTypeError: cannot create weak reference to 'dict' objectTrueSlotted TypeError: cannot create weak reference to 'Slotted' objectSlottedWeak okWhich objects support weak references:
- Yes: instances of ordinary classes, functions written in Python, bound methods, sets and frozensets, generators, classes, sockets,
arrays,deques, compiled regular expressions and code objects. - Only through a subclass:
listanddict. A trivial subclass gains a__weakref__slot. - No, even when subclassed (CPython):
intandtuple. Plainstrcannot be weakly referenced either, although astrsubclass can. __slots__classes: only if"__weakref__"is listed in__slots__. Adding it costs one pointer per instance; see__slots__.
The weak containers build on ref:
import weakref
class Request: def __init__(self, path: str) -> None: self.path = path
# Attach metadata to objects you do not own, without leaking them.timings: weakref.WeakKeyDictionary[Request, float] = weakref.WeakKeyDictionary()open_requests: weakref.WeakSet[Request] = weakref.WeakSet()
req = Request("/orders/ORD-4001")timings[req] = 0.042open_requests.add(req)print(timings[req], len(open_requests))
del reqprint(len(timings), len(open_requests))Output:
0.042 10 0WeakValueDictionaryholds values weakly: the entry disappears when the value dies. Use it for identity maps and shared-instance caches.WeakKeyDictionaryholds keys weakly: the entry disappears when the key dies. Use it to attach data to objects without adding attributes to them. Keys are still compared by==andhash, so two equal keys share one entry.WeakSetis a set of weak references, handy for “all live instances of X”.WeakMethod(3.4) references a bound method correctly, andfinalize(3.4) registers a cleanup function that runs when the object is collected or, by default, at interpreter exit.
Realistic example
Section titled “Realistic example”An OrdersPortal event bus notifies dashboards when an order changes. Dashboards come and go as users open and close browser tabs; the bus should not be the thing that keeps a closed dashboard alive.
import weakreffrom collections.abc import Callable
class OrderEvents: """OrdersPortal event bus that does not keep subscribers alive."""
def __init__(self) -> None: self._subscribers: list[weakref.WeakMethod] = []
def subscribe(self, handler: Callable[[str], None]) -> None: # weakref.ref(handler) would die at once: a bound method is a new, # temporary object on every attribute access. WeakMethod tracks # the instance and the function separately. self._subscribers.append(weakref.WeakMethod(handler))
def publish(self, order_id: str) -> None: live = [] for ref in self._subscribers: handler = ref() if handler is not None: handler(order_id) live.append(ref) self._subscribers = live # prune dead entries
class Dashboard: def __init__(self, name: str) -> None: self.name = name weakref.finalize(self, print, f" {name} dashboard closed")
def on_order(self, order_id: str) -> None: print(f" {self.name} saw {order_id}")
events = OrderEvents()sales, ops = Dashboard("sales"), Dashboard("ops")events.subscribe(sales.on_order)events.subscribe(ops.on_order)
events.publish("ORD-3001")del ops # user closed the ops tabevents.publish("ORD-3002")print("subscribers left:", len(events._subscribers))
m = weakref.ref(sales.on_order) # the classic mistakeprint("plain ref to bound method:", m())Output:
sales saw ORD-3001 ops saw ORD-3001 ops dashboard closed sales saw ORD-3002subscribers left: 1plain ref to bound method: None sales dashboard closedWeakMethod stores weak references to the instance and its function and rebuilds the bound method on demand. The finalize callback is given print and a string, not a bound method of self, which would keep the object alive forever. The last line comes from finalize running at interpreter exit for the dashboard still alive (the default, atexit=True).
Common mistakes
Section titled “Common mistakes”- Expecting a weak cache to keep things cached. A
WeakValueDictionaryonly remembers objects someone else is holding. If each request loads, uses and drops a customer, every lookup misses (the first sample shows this). For a cache that keeps recently used values, usefunctools.lru_cacheor a bounded dict, optionally in front of a weak identity map. - Weakly referencing a bound method.
weakref.ref(obj.method)is dead immediately because the bound method object is temporary. UseWeakMethod. - Checking liveness, then dereferencing.
if r() is not None: r().save()can fail if the object dies between the two calls, especially with threads. Call once:obj = r(), then testobj. - Letting a callback or finalizer capture the object. A lambda or bound method that refers to
selfcreates a strong reference and the object is never collected. - Iterating a weak container while objects die. Entries can disappear during iteration. Copy first (
list(weak_set)) when the loop body may drop the last reference. - Relying on immediate collection. Objects in a reference cycle, objects on PyPy, and in the free-threaded build some objects shared between threads may be freed later than you expect. Tests that need it can call
gc.collect().
When to use it and when not to
Section titled “When to use it and when not to”Use a weak reference when the code holding it is an observer, not an owner: identity maps, registries of live objects, back-references from child to parent, metadata attached to objects you do not control, and subscriber lists. Do not use one to make a cache “memory safe” when what you really want is eviction by size or age; that needs a bounded cache.
| Need | Strong dict |
WeakValueDictionary |
WeakKeyDictionary |
lru_cache / bounded dict |
|---|---|---|---|---|
| Keeps values alive | Yes | No | Yes (values), keys weak | Yes, up to maxsize |
| Entry removed when | You delete it | Value is collected | Key is collected | Evicted by policy |
| Hit rate if callers drop results | High | Zero | Not applicable | High |
Works with int/str values |
Yes | No | Values yes, keys no | Yes |
| Typical use | Owned data | Identity map, shared instances | Per-object metadata | Memoizing results |
For parent-child structures, a weak back-reference (self.parent = weakref.ref(parent)) avoids a reference cycle so the tree is freed by reference counting instead of waiting for the cycle collector.
Compatibility
Section titled “Compatibility”| Version | Change |
|---|---|
| 2.1 | weakref module added (PEP 205). |
| 3.2 | Weak references to threading.Lock and code objects. |
| 3.4 | WeakMethod, finalize, and ref.__callback__. |
| 3.8 | Proxies support @ and @=. |
| 3.9 | WeakKeyDictionary and WeakValueDictionary support | and |= (PEP 584). |
The API is the same in the free-threaded build of 3.13 and 3.14; only the timing of collection can differ.
Interview questions
Section titled “Interview questions”1. Why does weakref.ref(obj.handler)() return None immediately?
obj.handler creates a new bound method object on each access. Nothing else holds that object, so it is freed as soon as the expression ends. weakref.WeakMethod stores weak references to obj and the underlying function instead.
2. Why can you weakly reference a subclass of dict but not a dict?
Weak reference support needs a slot in the object to hold the list of weak references. Built-in dict and list omit it to save memory; a Python subclass adds a __weakref__ slot automatically (unless it declares __slots__ without it).
3. Senior scenario: memory in the OrdersPortal API grows steadily, and a heap profile points at a module-level registry of ReportJob objects that is only appended to. How do you fix it?
First decide who owns a job. If the registry exists only to list running jobs for an admin page, it should not own them: switch it to a WeakSet (or WeakValueDictionary keyed by job ID) so finished jobs disappear when the worker drops them. If the registry is the owner (the admin page must show finished jobs), weak references would lose data; add an explicit lifecycle instead, such as removal on completion plus a bounded history. Then check for hidden strong references that would defeat the weak registry: callbacks that close over the job, exceptions with tracebacks stored on the job, and lru_cache on methods (which holds self). Confirm with tracemalloc snapshots before and after.
4. When would you choose finalize over __del__?
finalize does not require changing the class, can be registered by code that does not own the class, can be called or detached explicitly, runs at exit by default, and avoids the pitfalls of __del__ (resurrection, and exceptions that are only printed). The callback must not reference the object.
Key takeaways
Section titled “Key takeaways”- A weak reference lets you find an object without being a reason it stays alive.
- Call a
refonce and test the result; useWeakMethodfor bound methods andfinalizefor cleanup. WeakValueDictionaryis an identity map, not an eviction policy: if nobody else holds the value, it is gone.int,str,tuple, plainlist/dictand__slots__classes without__weakref__cannot be weakly referenced.- Hidden strong references (closures, caches, tracebacks) defeat weak registries; verify with a heap profile.
Related pages
Section titled “Related pages”- Memory management: reference counting and the cycle collector that weak references interact with.
__slots__: why slotted classes need"__weakref__"to be weakly referenceable.- lru_cache: the bounded cache to use when you want values kept.
- Context managers: deterministic cleanup, usually preferable to finalizers.
- collections module: the regular containers the weak containers mirror.