Python Context Managers: with Statement and contextlib
What you’ll learn
Section titled “What you’ll learn”- What a context manager does and why the
withstatement matters. - Using
withfor files, locks, and database connections. - Writing a custom context manager with
contextlib.contextmanager. - Writing a class-based context manager with
__enter__and__exit__.
Prerequisites
Section titled “Prerequisites”- Python 3.9+ installed
- Familiarity with exceptions (exceptions tutorial)
What is a context manager?
Section titled “What is a context manager?”A context manager is an object that defines setup and teardown behavior. The with statement calls the setup on entry and guarantees the teardown runs on exit — even if an exception is raised in between.
The most common example:
# Without a context manager — easy to forget close(), or miss it on exceptionf = open("orders.txt", encoding="utf-8")data = f.read()f.close() # only runs if read() didn't raise
# With a context manager — close() is guaranteedwith open("orders.txt", encoding="utf-8") as f: data = f.read()# f is closed here, whether or not read() raisedThe with block calls f.__enter__() on entry (which returns the file object itself, bound to f), and f.__exit__(...) on exit.
Step 1 — Files
Section titled “Step 1 — Files”# Readingwith open("orders.txt", encoding="utf-8") as f: for line in f: print(line.rstrip())
# Writingwith open("report.txt", "w", encoding="utf-8") as f: f.write("Order Report\n") f.write("=============\n")
# Multiple files in one withwith open("input.txt", encoding="utf-8") as fin, \ open("output.txt", "w", encoding="utf-8") as fout: for line in fin: fout.write(line.upper())Step 2 — Locks for thread safety
Section titled “Step 2 — Locks for thread safety”import threading
lock = threading.Lock()counter = 0
def increment(): global counter with lock: # Only one thread can be here at a time counter += 1
threads = [threading.Thread(target=increment) for _ in range(1000)]for t in threads: t.start()for t in threads: t.join()
print(counter) # 1000 — safe from race conditionsWithout the lock, concurrent increments would produce a result less than 1000 due to race conditions.
Step 3 — Write a custom context manager with contextlib
Section titled “Step 3 — Write a custom context manager with contextlib”For simple cases, the contextlib.contextmanager decorator is the easiest way to write a context manager. Everything before yield is setup; everything after is teardown.
from contextlib import contextmanagerimport time
@contextmanagerdef timed(label: str): """Context manager that prints how long the block took.""" start = time.perf_counter() try: yield # the with block runs here finally: elapsed = (time.perf_counter() - start) * 1000 print(f"{label}: {elapsed:.1f}ms")
with timed("process orders"): total = sum(range(1_000_000))# process orders: 42.3msThe try/finally ensures the timing is printed even if the block raises an exception.
A more practical example — a temporary directory that is cleaned up on exit:
from contextlib import contextmanagerfrom pathlib import Pathimport tempfileimport shutil
@contextmanagerdef temp_directory(): """Create a temporary directory and delete it when done.""" path = Path(tempfile.mkdtemp()) try: yield path finally: shutil.rmtree(path)
with temp_directory() as tmp: (tmp / "work.txt").write_text("temporary data") print(list(tmp.iterdir())) # [PosixPath('/tmp/xyz/work.txt')]# tmp/ is deleted hereStep 4 — Class-based context manager
Section titled “Step 4 — Class-based context manager”For more complex cases or when you need to hold state across __enter__ and __exit__, use a class:
class DatabaseConnection: """Manages a database connection lifecycle."""
def __init__(self, url: str) -> None: self.url = url self._conn = None
def __enter__(self): print(f"Connecting to {self.url}") self._conn = self._connect() return self._conn # the value bound to 'as' in the with statement
def __exit__(self, exc_type, exc_val, exc_tb): if self._conn: if exc_type is not None: print(f"Rolling back due to {exc_type.__name__}") self._conn.rollback() else: self._conn.commit() self._conn.close() print("Connection closed") return False # False = do not suppress the exception
def _connect(self): # In practice, use a real DB library return type("FakeConn", (), {"rollback": lambda s: None, "commit": lambda s: None, "close": lambda s: None, "execute": lambda s, q: None})()
with DatabaseConnection("postgresql://localhost/orders") as conn: conn.execute("UPDATE orders SET status='confirmed' WHERE id='ORD-001'")# Connecting to postgresql://localhost/orders# Connection closed
# If an exception is raised inside the with block:try: with DatabaseConnection("postgresql://localhost/orders") as conn: conn.execute("UPDATE ...") raise ValueError("Something went wrong")except ValueError: pass# Connecting to postgresql://localhost/orders# Rolling back due to ValueError# Connection closedThe __exit__ parameters are:
exc_type— the exception class, orNoneif no exceptionexc_val— the exception instanceexc_tb— the traceback object
Return True from __exit__ to suppress the exception; return False (or nothing) to let it propagate.
Step 5 — contextlib utilities
Section titled “Step 5 — contextlib utilities”from contextlib import suppress, nullcontext
# suppress: ignore specific exceptionswith suppress(FileNotFoundError): Path("nonexistent.txt").unlink() # no error if file doesn't exist
# nullcontext: a no-op context manager (useful for conditional wrapping)def process(lock=None): ctx = lock if lock is not None else nullcontext() with ctx: # critical section if lock is provided, otherwise proceeds normally passComplete working example
Section titled “Complete working example”from contextlib import contextmanagerfrom pathlib import Pathimport csv
@contextmanagerdef csv_writer(path: str): """Open a CSV file for writing and flush/close it on exit.""" f = open(path, "w", newline="", encoding="utf-8") writer = csv.DictWriter(f, fieldnames=["order_id", "customer", "amount"]) writer.writeheader() try: yield writer finally: f.flush() f.close() print(f"Wrote {Path(path).name}")
with csv_writer("orders.csv") as writer: writer.writerow({"order_id": "ORD-001", "customer": "BensonCo", "amount": 149.99}) writer.writerow({"order_id": "ORD-002", "customer": "NovaCorp", "amount": 270.00})# Wrote orders.csvRun with python example.py (Python 3.9+).
Common mistakes
Section titled “Common mistakes”- Not using
withfor files. A forgottenclose()call causes resource leaks that manifest asToo many open filesunder load. - Returning
Truefrom__exit__unintentionally. Returning any truthy value suppresses exceptions. ReturnFalseorNoneunless you explicitly intend to swallow them. - Putting cleanup after
yieldwithouttry/finally. If the with block raises, code afteryielddoes not run unless it is insidefinally.
Next steps
Section titled “Next steps”- File I/O tutorial — context managers for files in depth
- Context managers feature guide —
contextlibfull reference - Threading tutorial — thread locks and synchronization
Official references
Section titled “Official references”- Python docs: contextlib
- PEP 343 — the
withstatement