Skip to content

Python weak references: caches and observers without leaks

  • What a weak reference is and how it differs from an ordinary (strong) reference.
  • How to use weakref.ref, proxy, WeakValueDictionary, WeakKeyDictionary, WeakSet, WeakMethod and finalize.
  • Which objects can be weakly referenced, and why int, str, tuple and __slots__ classes often cannot.
  • Why a weak cache can have a hit rate of zero, and why a weak reference to a bound method dies at once.
  • When a weak reference is the right ownership signal and when an LRU cache or explicit cleanup is better.

A normal reference keeps an object alive: CPython frees an object when its reference count reaches zero. A weak reference, created with the weakref module, refers to an object without adding to that count. While something else holds the object, the weak reference can return it; once the last strong reference disappears, the object is freed and the weak reference reports that it is gone (calling it returns None). Weak references have been in the standard library since Python 2.1 (PEP 205) and are fully current in Python 3.14.

A cache keyed by customer ID, a registry of open sessions or an event bus that stores callbacks all hold strong references, so every object they have seen stays alive until someone removes it. That is the most common Python “memory leak”: nothing is lost, it is just never let go.

In OrdersPortal, the request handler loads Customer objects and wants to share one instance between concurrent pieces of work on the same customer. A plain dict does that, but it also keeps every customer alive for the life of the process. A WeakValueDictionary shares the instance only while something is using it.

Running the samples: each sample is a complete script. Save it as a .py file and run it with Python 3.14 (python3 sample.py). The outputs shown were produced with CPython 3.14.6. The exact moment an object is freed is a CPython implementation detail (reference counting); other implementations such as PyPy may free it later.

import weakref
class Customer:
def __init__(self, customer_id: str) -> None:
self.customer_id = customer_id
def __repr__(self) -> str:
return f"Customer({self.customer_id!r})"
def load_customer(customer_id: str) -> Customer:
print(f" loading {customer_id} from the database")
return Customer(customer_id)
# Before: a plain dict keeps every customer alive for the life of the process.
strong_cache: dict[str, Customer] = {}
# After: entries vanish once no request is using that customer any more.
weak_cache: weakref.WeakValueDictionary[str, Customer] = weakref.WeakValueDictionary()
def get_customer(customer_id: str) -> Customer:
customer = weak_cache.get(customer_id)
if customer is None:
customer = load_customer(customer_id)
weak_cache[customer_id] = customer
return customer
c = Customer("C-0")
strong_cache["C-0"] = c
del c
print("strong cache still holds:", list(strong_cache.values()))
a = get_customer("C-1") # loads
b = get_customer("C-1") # cache hit while `a` is alive
print(a is b, len(weak_cache))
del a, b # last strong references gone
print(len(weak_cache))
get_customer("C-1") # loads again

Output:

strong cache still holds: [Customer('C-0')]
loading C-1 from the database
True 1
0
loading C-1 from the database

The strong cache still holds C-0 after the program dropped it. The weak cache shared one C-1 instance while it was in use and forgot it when the last user let go. Note the single get followed by a test of the result: a separate key in weak_cache check can race with collection.

weakref.ref(obj, callback=None) returns a reference object. Call it to get the referent back, or None once it has been collected. The optional callback receives the dead reference object, not the referent, which no longer exists. weakref.proxy(obj) returns an object that forwards attribute access and operators to the referent and raises ReferenceError once the referent is gone.

import weakref
class Cart:
pass
cart = Cart()
r = weakref.ref(cart, lambda ref: print(" callback: cart collected"))
print(r() is cart, weakref.getweakrefcount(cart))
p = weakref.proxy(cart)
p.total = 42.0 # behaves like the object
print(cart.total)
del cart # callback runs immediately in CPython
print(r())
try:
p.total
except ReferenceError as exc:
print("ReferenceError:", exc)
for value in (1, "ORD-1", (1, 2), [1], {"a": 1}):
try:
weakref.ref(value)
except TypeError as exc:
print("TypeError:", exc)
class Tags(list): # a list subclass can be weakly referenced
pass
tags = Tags(["gift"])
print(weakref.ref(tags)() is tags)
class Slotted:
__slots__ = ("sku",)
class SlottedWeak:
__slots__ = ("sku", "__weakref__")
for cls in (Slotted, SlottedWeak):
try:
weakref.ref(cls())
print(cls.__name__, "ok")
except TypeError as exc:
print(cls.__name__, "TypeError:", exc)

Output:

True 1
42.0
callback: cart collected
None
ReferenceError: weakly-referenced object no longer exists
TypeError: cannot create weak reference to 'int' object
TypeError: cannot create weak reference to 'str' object
TypeError: cannot create weak reference to 'tuple' object
TypeError: cannot create weak reference to 'list' object
TypeError: cannot create weak reference to 'dict' object
True
Slotted TypeError: cannot create weak reference to 'Slotted' object
SlottedWeak ok

Which objects support weak references:

  • Yes: instances of ordinary classes, functions written in Python, bound methods, sets and frozensets, generators, classes, sockets, arrays, deques, compiled regular expressions and code objects.
  • Only through a subclass: list and dict. A trivial subclass gains a __weakref__ slot.
  • No, even when subclassed (CPython): int and tuple. Plain str cannot be weakly referenced either, although a str subclass can.
  • __slots__ classes: only if "__weakref__" is listed in __slots__. Adding it costs one pointer per instance; see __slots__.

The weak containers build on ref:

import weakref
class Request:
def __init__(self, path: str) -> None:
self.path = path
# Attach metadata to objects you do not own, without leaking them.
timings: weakref.WeakKeyDictionary[Request, float] = weakref.WeakKeyDictionary()
open_requests: weakref.WeakSet[Request] = weakref.WeakSet()
req = Request("/orders/ORD-4001")
timings[req] = 0.042
open_requests.add(req)
print(timings[req], len(open_requests))
del req
print(len(timings), len(open_requests))

Output:

0.042 1
0 0
  • WeakValueDictionary holds values weakly: the entry disappears when the value dies. Use it for identity maps and shared-instance caches.
  • WeakKeyDictionary holds keys weakly: the entry disappears when the key dies. Use it to attach data to objects without adding attributes to them. Keys are still compared by == and hash, so two equal keys share one entry.
  • WeakSet is a set of weak references, handy for “all live instances of X”.
  • WeakMethod (3.4) references a bound method correctly, and finalize (3.4) registers a cleanup function that runs when the object is collected or, by default, at interpreter exit.

An OrdersPortal event bus notifies dashboards when an order changes. Dashboards come and go as users open and close browser tabs; the bus should not be the thing that keeps a closed dashboard alive.

import weakref
from collections.abc import Callable
class OrderEvents:
"""OrdersPortal event bus that does not keep subscribers alive."""
def __init__(self) -> None:
self._subscribers: list[weakref.WeakMethod] = []
def subscribe(self, handler: Callable[[str], None]) -> None:
# weakref.ref(handler) would die at once: a bound method is a new,
# temporary object on every attribute access. WeakMethod tracks
# the instance and the function separately.
self._subscribers.append(weakref.WeakMethod(handler))
def publish(self, order_id: str) -> None:
live = []
for ref in self._subscribers:
handler = ref()
if handler is not None:
handler(order_id)
live.append(ref)
self._subscribers = live # prune dead entries
class Dashboard:
def __init__(self, name: str) -> None:
self.name = name
weakref.finalize(self, print, f" {name} dashboard closed")
def on_order(self, order_id: str) -> None:
print(f" {self.name} saw {order_id}")
events = OrderEvents()
sales, ops = Dashboard("sales"), Dashboard("ops")
events.subscribe(sales.on_order)
events.subscribe(ops.on_order)
events.publish("ORD-3001")
del ops # user closed the ops tab
events.publish("ORD-3002")
print("subscribers left:", len(events._subscribers))
m = weakref.ref(sales.on_order) # the classic mistake
print("plain ref to bound method:", m())

Output:

sales saw ORD-3001
ops saw ORD-3001
ops dashboard closed
sales saw ORD-3002
subscribers left: 1
plain ref to bound method: None
sales dashboard closed

WeakMethod stores weak references to the instance and its function and rebuilds the bound method on demand. The finalize callback is given print and a string, not a bound method of self, which would keep the object alive forever. The last line comes from finalize running at interpreter exit for the dashboard still alive (the default, atexit=True).

  • Expecting a weak cache to keep things cached. A WeakValueDictionary only remembers objects someone else is holding. If each request loads, uses and drops a customer, every lookup misses (the first sample shows this). For a cache that keeps recently used values, use functools.lru_cache or a bounded dict, optionally in front of a weak identity map.
  • Weakly referencing a bound method. weakref.ref(obj.method) is dead immediately because the bound method object is temporary. Use WeakMethod.
  • Checking liveness, then dereferencing. if r() is not None: r().save() can fail if the object dies between the two calls, especially with threads. Call once: obj = r(), then test obj.
  • Letting a callback or finalizer capture the object. A lambda or bound method that refers to self creates a strong reference and the object is never collected.
  • Iterating a weak container while objects die. Entries can disappear during iteration. Copy first (list(weak_set)) when the loop body may drop the last reference.
  • Relying on immediate collection. Objects in a reference cycle, objects on PyPy, and in the free-threaded build some objects shared between threads may be freed later than you expect. Tests that need it can call gc.collect().

Use a weak reference when the code holding it is an observer, not an owner: identity maps, registries of live objects, back-references from child to parent, metadata attached to objects you do not control, and subscriber lists. Do not use one to make a cache “memory safe” when what you really want is eviction by size or age; that needs a bounded cache.

Need Strong dict WeakValueDictionary WeakKeyDictionary lru_cache / bounded dict
Keeps values alive Yes No Yes (values), keys weak Yes, up to maxsize
Entry removed when You delete it Value is collected Key is collected Evicted by policy
Hit rate if callers drop results High Zero Not applicable High
Works with int/str values Yes No Values yes, keys no Yes
Typical use Owned data Identity map, shared instances Per-object metadata Memoizing results

For parent-child structures, a weak back-reference (self.parent = weakref.ref(parent)) avoids a reference cycle so the tree is freed by reference counting instead of waiting for the cycle collector.

Version Change
2.1 weakref module added (PEP 205).
3.2 Weak references to threading.Lock and code objects.
3.4 WeakMethod, finalize, and ref.__callback__.
3.8 Proxies support @ and @=.
3.9 WeakKeyDictionary and WeakValueDictionary support | and |= (PEP 584).

The API is the same in the free-threaded build of 3.13 and 3.14; only the timing of collection can differ.

1. Why does weakref.ref(obj.handler)() return None immediately? obj.handler creates a new bound method object on each access. Nothing else holds that object, so it is freed as soon as the expression ends. weakref.WeakMethod stores weak references to obj and the underlying function instead.

2. Why can you weakly reference a subclass of dict but not a dict? Weak reference support needs a slot in the object to hold the list of weak references. Built-in dict and list omit it to save memory; a Python subclass adds a __weakref__ slot automatically (unless it declares __slots__ without it).

3. Senior scenario: memory in the OrdersPortal API grows steadily, and a heap profile points at a module-level registry of ReportJob objects that is only appended to. How do you fix it? First decide who owns a job. If the registry exists only to list running jobs for an admin page, it should not own them: switch it to a WeakSet (or WeakValueDictionary keyed by job ID) so finished jobs disappear when the worker drops them. If the registry is the owner (the admin page must show finished jobs), weak references would lose data; add an explicit lifecycle instead, such as removal on completion plus a bounded history. Then check for hidden strong references that would defeat the weak registry: callbacks that close over the job, exceptions with tracebacks stored on the job, and lru_cache on methods (which holds self). Confirm with tracemalloc snapshots before and after.

4. When would you choose finalize over __del__? finalize does not require changing the class, can be registered by code that does not own the class, can be called or detached explicitly, runs at exit by default, and avoids the pitfalls of __del__ (resurrection, and exceptions that are only printed). The callback must not reference the object.

  • A weak reference lets you find an object without being a reason it stays alive.
  • Call a ref once and test the result; use WeakMethod for bound methods and finalize for cleanup.
  • WeakValueDictionary is an identity map, not an eviction policy: if nobody else holds the value, it is gone.
  • int, str, tuple, plain list/dict and __slots__ classes without __weakref__ cannot be weakly referenced.
  • Hidden strong references (closures, caches, tracebacks) defeat weak registries; verify with a heap profile.
  • Memory management: reference counting and the cycle collector that weak references interact with.
  • __slots__: why slotted classes need "__weakref__" to be weakly referenceable.
  • lru_cache: the bounded cache to use when you want values kept.
  • Context managers: deterministic cleanup, usually preferable to finalizers.
  • collections module: the regular containers the weak containers mirror.