threading vs multiprocessing in Python 3.14
What you’ll learn
Section titled “What you’ll learn”- How
threadingandmultiprocessingdiffer in execution model, memory sharing and cost. - Why threads speed up I/O-bound work but not CPU-bound work on the default (GIL) build of CPython.
- What changed in Python 3.13 and 3.14: the free-threaded build (PEP 703, PEP 779) and subinterpreters (
concurrent.interpreters,InterpreterPoolExecutor). - Why the default
multiprocessingstart method on Linux changed toforkserverin 3.14, and what breaks. - How to choose, and how to pick the right pool at run time.
Summary
Section titled “Summary”threading runs several threads inside one process. They share memory directly, but on the default CPython build the global interpreter lock (GIL) lets only one thread execute Python bytecode at a time, so threads help while code waits on I/O and do not help pure-Python computation. multiprocessing runs separate OS processes, each with its own GIL: real CPU parallelism, paid for in startup time and pickling. Python 3.14 adds a supported (optional) free-threaded build and standard-library subinterpreters.
The problem it solves
Section titled “The problem it solves”A script that calls a remote API for each OrdersPortal order spends almost all its time waiting. Sequentially, the waits add up; threads let them overlap.
Running the samples: each sample is a complete script. Save it as a .py file and run it with Python 3.14 (python3 sample.py). Outputs come from CPython 3.14.6 on a 4-vCPU Linux container; free-threaded results come from the python3.14t executable of the same release. All timings are machine-dependent and vary from run to run; compare the ratios, not the numbers. time.sleep stands in for network calls.
import timefrom concurrent.futures import ThreadPoolExecutor
def fetch_tracking(order_id: str) -> str: """Stand-in for an HTTPS call to a carrier API: 0.5 s of waiting.""" time.sleep(0.5) return f"{order_id}: in transit"
order_ids = [f"ORD-{n}" for n in range(1001, 1009)]
start = time.perf_counter()sequential = [fetch_tracking(o) for o in order_ids]print(f"sequential: {time.perf_counter() - start:.2f} s")
start = time.perf_counter()with ThreadPoolExecutor(max_workers=8) as pool: threaded = list(pool.map(fetch_tracking, order_ids))print(f"8 threads: {time.perf_counter() - start:.2f} s")print(threaded[:2], threaded == sequential)Output (timings machine-dependent):
sequential: 4.00 s8 threads: 0.50 s['ORD-1001: in transit', 'ORD-1002: in transit'] TrueBlocking calls such as socket reads, file I/O and time.sleep release the GIL while they wait, so eight threads finish in about the time of one call. The rest of the page uses concurrent.futures, which gives threads, processes and interpreters the same Executor interface.
How it works
Section titled “How it works”| Threads | Processes | Subinterpreters (3.14) | |
|---|---|---|---|
| Lives in | One process | Separate OS processes | One process, isolated interpreters |
| Memory | Shared | Separate; data is pickled | Separate; data is pickled or explicitly shared |
| Parallel Python bytecode | Only on the free-threaded build | Yes | Yes (one GIL per interpreter) |
| Startup cost | Low | High | Medium (not yet optimized) |
| Main risk | Races on shared state | Pickling, start-method surprises | Extension modules not yet compatible |
CPU-bound work on each build
Section titled “CPU-bound work on each build”The same pure-Python function, run four times, through each executor:
import sysimport sysconfigimport timefrom concurrent.futures import ( InterpreterPoolExecutor, ProcessPoolExecutor, ThreadPoolExecutor,)
def score_orders(seed: int) -> int: """CPU-bound stand-in: a pure-Python fraud score over 10 million steps.""" total = seed for i in range(10_000_000): total = (total * 31 + i) % 1_000_003 return total
def run(label, executor_cls, batches): start = time.perf_counter() with executor_cls(max_workers=4) as pool: results = list(pool.map(score_orders, batches)) print(f"{label:<12} {time.perf_counter() - start:5.2f} s") return results
if __name__ == "__main__": gil = "disabled" if not sys._is_gil_enabled() else "enabled" ft = sysconfig.get_config_var("Py_GIL_DISABLED") print(f"Python {sys.version.split()[0]}, free-threaded build: {bool(ft)}, GIL {gil}") batches = [1, 2, 3, 4] start = time.perf_counter() expected = [score_orders(b) for b in batches] print(f"{'sequential':<12} {time.perf_counter() - start:5.2f} s") for label, cls in [("threads", ThreadPoolExecutor), ("processes", ProcessPoolExecutor), ("interpreters", InterpreterPoolExecutor)]: assert run(label, cls, batches) == expectedOutput with python3.14 and with python3.14t (timings machine-dependent):
Python 3.14.6, free-threaded build: False, GIL enabledsequential 1.76 sthreads 1.54 sprocesses 0.56 sinterpreters 0.43 sPython 3.14.6, free-threaded build: True, GIL disabledsequential 1.60 sthreads 0.46 sprocesses 0.69 sinterpreters 0.74 sOn the GIL build, threads give no speedup for CPU work, while processes and subinterpreters run on several cores. On the free-threaded build, plain threads scale too, without pickling or extra interpreters. A second run on the same machine took 0.75 s for free-threaded threads and 1.01 s for interpreters, so treat these numbers as an illustration, not a benchmark.
The free-threaded build (3.13 and 3.14)
Section titled “The free-threaded build (3.13 and 3.14)”PEP 703 made the GIL optional. Python 3.13 shipped the free-threaded build as experimental, as a separate executable (python3.13t). In Python 3.14, PEP 779 moved it to officially supported but still optional (“phase II”): the default build still has a GIL, and making free-threading the default is undecided. What’s New in 3.14 puts the single-threaded overhead at roughly 5 to 10%, depending on platform and compiler. Check a running interpreter with sysconfig.get_config_var("Py_GIL_DISABLED") (the build) and sys._is_gil_enabled() (the GIL can be re-enabled with PYTHON_GIL=1, -X gil=1, or automatically when an extension module that is not marked free-threading-safe is imported, which prints a warning).
Subinterpreters (3.14)
Section titled “Subinterpreters (3.14)”PEP 734 added concurrent.interpreters: several isolated interpreters in one process, each with its own GIL since Python 3.12 (PEP 684). InterpreterPoolExecutor wraps them in the familiar executor interface.
from concurrent import interpreters
def with_tax(total: float) -> float: return round(total * 1.2, 2)
interp = interpreters.create()print(interp.call(with_tax, 59.9)) # runs in the other interpreter, this thread
interp.exec("ORDERS_SEEN = 1")interp.exec("print('isolated module state:', ORDERS_SEEN)")print("main sees ORDERS_SEEN?", "ORDERS_SEEN" in globals())
worker = interp.call_in_thread(with_tax, 10.0) # returns a threading.Threadworker.join()interp.close()Output:
71.88isolated module state: 1main sees ORDERS_SEEN? FalseAn interpreter provides isolation, not concurrency: call runs in the current thread, and parallelism comes from running interpreters in several threads. The 3.14 documentation lists current limits: slow startup, extra memory per interpreter, few ways to share objects, and many third-party extension modules without support yet.
Start methods and pickling
Section titled “Start methods and pickling”Every argument and result sent to a process pool is pickled, and each worker re-imports your main module. In 3.14 the default start method on Linux and other non-macOS Unix platforms changed from fork to forkserver; Windows and macOS keep spawn.
import multiprocessing as mpfrom concurrent.futures import ProcessPoolExecutor
TAX_RATE = 0.2 # module-level constant: re-imported in every child process
def with_tax(total: float) -> float: return round(total * (1 + TAX_RATE), 2)
if __name__ == "__main__": print("default start method:", mp.get_start_method()) with ProcessPoolExecutor(max_workers=2) as pool: print(list(pool.map(with_tax, [10.0, 59.9]))) try: print(list(pool.map(lambda t: t * 2, [10.0]))) except Exception as exc: print(type(exc).__name__ + ":", exc)Output (Linux; the memory address varies):
default start method: forkserver[12.0, 71.88]PicklingError: Can't pickle <function <lambda> at 0x7fdf9d698880>: it's not found as __main__.<lambda>Code that relied on fork copying mutated module state into children, or that passed unpicklable objects, worked on Linux before 3.14 and fails now. Request mp.get_context("fork") explicitly only if you need it and the parent has no threads; since 3.12, os.fork() in a process that Python detects has several threads emits a DeprecationWarning.
Sharing state between threads
Section titled “Sharing state between threads”The GIL never made compound operations atomic. A read-modify-write such as self.reserved += 1 can interleave between threads; a threading.Lock makes it safe.
import threading
class StockCounter: def __init__(self) -> None: self.reserved = 0 self._lock = threading.Lock()
def reserve_unsafe(self) -> None: current = self.reserved # read current += 1 # modify self.reserved = current # write: another thread may have written in between
def reserve_safe(self) -> None: with self._lock: self.reserved += 1
def hammer(method, threads=8, calls=100_000): workers = [threading.Thread(target=lambda: [method() for _ in range(calls)]) for _ in range(threads)] for w in workers: w.start() for w in workers: w.join()
for name in ("reserve_unsafe", "reserve_safe"): counter = StockCounter() hammer(getattr(counter, name)) print(f"{name}: {counter.reserved:,} of 800,000")Output with python3.14t (the lost count varies per run):
reserve_unsafe: 248,997 of 800,000reserve_safe: 800,000 of 800,000On the GIL build this run happened to print 800,000 twice. That is scheduling luck: the unsafe version is a bug on either build; free-threading just exposes it more often.
Realistic example
Section titled “Realistic example”OrdersPortal’s nightly reconciliation fetches settlements from a payment provider (I/O-bound) and then computes a risk score for each (CPU-bound). The I/O stage always uses threads; the CPU stage picks threads on a free-threaded interpreter and processes otherwise.
"""OrdersPortal nightly reconciliation: I/O stage, then CPU stage."""import osimport sysimport timefrom concurrent.futures import Executor, ProcessPoolExecutor, ThreadPoolExecutor
def fetch_settlement(order_id: str) -> tuple[str, int]: time.sleep(0.2) # stand-in for a payment-provider API call return order_id, int(order_id[-4:]) # (order, amount in cents)
def risk_score(item: tuple[str, int]) -> tuple[str, int]: order_id, cents = item total = cents for i in range(2_000_000): # pure-Python CPU work total = (total * 31 + i) % 1_000_003 return order_id, total % 100
def cpu_executor(workers: int) -> Executor: """Pick the cheapest pool that gives real parallelism on this interpreter.""" if not sys._is_gil_enabled(): return ThreadPoolExecutor(workers) # free-threaded build return ProcessPoolExecutor(workers) # safest default with a GIL
if __name__ == "__main__": order_ids = [f"ORD-{n}" for n in range(1001, 1013)] workers = min(4, os.cpu_count() or 1)
start = time.perf_counter() with ThreadPoolExecutor(max_workers=12) as io_pool: # I/O: many threads settlements = list(io_pool.map(fetch_settlement, order_ids)) print(f"fetched {len(settlements)} settlements in {time.perf_counter() - start:.2f} s")
start = time.perf_counter() with cpu_executor(workers) as cpu_pool: scores = dict(cpu_pool.map(risk_score, settlements)) print(f"scored with {type(cpu_pool).__name__} in {time.perf_counter() - start:.2f} s") print("flagged:", sorted(o for o, s in scores.items() if s > 80))Output with python3.14, then python3.14t (timings machine-dependent):
fetched 12 settlements in 0.21 sscored with ProcessPoolExecutor in 0.34 sflagged: ['ORD-1005', 'ORD-1006']fetched 12 settlements in 0.21 sscored with ThreadPoolExecutor in 0.30 sflagged: ['ORD-1005', 'ORD-1006']risk_score is a module-level function over picklable tuples, so it works with either pool, and with InterpreterPoolExecutor when every imported extension supports subinterpreters.
Common mistakes
Section titled “Common mistakes”- Using threads for pure-Python CPU work on the GIL build. It adds overhead without parallelism. Use processes, subinterpreters or the free-threaded build, or move the hot loop into a library that releases the GIL (NumPy, compression, hashing).
- Assuming the GIL makes your code thread-safe. It protects the interpreter’s internals, not your invariants. Guard shared mutable state with a
Lock, or pass data through aqueue.Queue. - Forgetting
if __name__ == "__main__":. Withspawnandforkserver, children import the main module; unguarded pool creation fails or recurses. - Sending unpicklable work to a process pool. Lambdas, nested functions, open connections and locks cannot be pickled. Use module-level functions and plain data.
- Relying on
forksemantics after 3.14. Children no longer inherit mutated globals on Linux by default. Pass state explicitly or use aninitializer. - Sending large objects per task. Pickling big data to every worker can cost more than the work. Send IDs or paths, or use
multiprocessing.shared_memory. - Oversubscribing. Four processes that each start a multithreaded math library can create dozens of busy threads on four cores. Size pools to the CPUs you actually have.
When to use it and when not to
Section titled “When to use it and when not to”| Workload | First choice | Why | Avoid |
|---|---|---|---|
| Many network or disk waits (tens to hundreds) | ThreadPoolExecutor |
Waits release the GIL; cheap, shared memory | Processes (pure overhead) |
| Thousands of concurrent sockets | asyncio |
One thread, no per-connection thread | One thread per connection |
| Pure-Python CPU work, GIL build | ProcessPoolExecutor |
One GIL per process, mature | Threads |
| Pure-Python CPU work, all dependencies support it | InterpreterPoolExecutor (3.14) |
Lighter than processes | Extensions without subinterpreter support |
| CPU work on a free-threaded build | ThreadPoolExecutor |
Real parallelism with shared memory | Unsynchronized shared state |
| CPU work in NumPy or another GIL-releasing library | Threads | Library code runs in parallel already | Processes that copy large arrays |
| Crash isolation, memory limits per task | Processes | A crash or leak dies with the worker | Threads, subinterpreters |
Skip concurrency when the bottleneck is a database or an API rate limit: more workers only queue at the same place.
Compatibility
Section titled “Compatibility”| Version | Change |
|---|---|
| 2.6 | multiprocessing added. |
| 3.2 | concurrent.futures (ThreadPoolExecutor, ProcessPoolExecutor). |
| 3.8 | macOS default start method changed to spawn. |
| 3.12 | Per-interpreter GIL (PEP 684), C API only. |
| 3.13 | Experimental free-threaded build (PEP 703), python3.13t; sys._is_gil_enabled(). |
| 3.14 | Free-threaded build officially supported but optional (PEP 779); concurrent.interpreters (PEP 734) and InterpreterPoolExecutor; forkserver is the default start method on Unix except macOS. |
Interview questions
Section titled “Interview questions”1. Why do threads speed up downloads but not image resizing written in pure Python? On the GIL build only one thread runs bytecode at a time. Blocking I/O releases the GIL, so downloads overlap; pure-Python computation holds it, so threads take turns. Resizing in a C library that releases the GIL, or on a free-threaded build, does run in parallel.
2. A job that worked on Python 3.13 on Linux fails on 3.14 with a PicklingError or sees empty globals in workers. What changed?
The default start method moved from fork to forkserver. Workers now start from a clean server process, so arguments must be picklable and module-level state set by the parent after import is not copied. Fix the code to pass state explicitly; only fall back to get_context("fork") deliberately.
3. What is the difference between a subinterpreter and a process? Both isolate Python state and need data to be copied or explicitly shared. Subinterpreters live in one OS process, so they start faster and use fewer resources, but they share a crash domain and need extension modules that support multiple interpreters.
4. Senior scenario: OrdersPortal’s report service is CPU-bound, runs on a 16-core container, and uses a C extension for PDF rendering. The team wants to adopt the free-threaded build. How do you evaluate it?
Confirm every compiled dependency ships free-threaded wheels; otherwise importing it re-enables the GIL (with a warning), which sys._is_gil_enabled() will show. Audit shared mutable state, because races that the GIL hid become visible. Benchmark the real workload on both builds: single-threaded code runs a few percent slower, so the gain depends on how much work is parallel. Keep the executor choice behind one function, as in the realistic example, so the service can fall back to processes, and roll out on a canary first.
Key takeaways
Section titled “Key takeaways”- I/O-bound: threads (or
asyncioat very high concurrency). CPU-bound on the GIL build: processes or, in 3.14, subinterpreters. - The 3.14 free-threaded build is supported but optional; it lets threads use every core but exposes races.
- Process pools pickle everything and, since 3.14 on Linux, start workers with
forkserver. - The GIL never made your own read-modify-write code safe; use locks or queues.
- Measure on your hardware; timings vary between machines and runs.
Related pages
Section titled “Related pages”- The GIL: what the global interpreter lock protects and why it exists.
- async/await: single-threaded concurrency for large numbers of I/O waits.
- Memory management: reference counting, the reason the GIL was simple to keep.
- Context managers:
withfor executors and locks. - Generators: streaming work items into a pool without building huge lists.
Official references
Section titled “Official references”- threading (Python 3.14 docs)
- multiprocessing: contexts and start methods (Python 3.14 docs)
- concurrent.futures (Python 3.14 docs)
- concurrent.interpreters (Python 3.14 docs)
- Python support for free threading (Python 3.14 HOWTO)
- What’s New in Python 3.14: free-threaded mode, PEP 734, multiprocessing
- What’s New in Python 3.13: free-threaded CPython
- PEP 703: Making the GIL Optional, PEP 779: Supported status for free-threaded Python, PEP 734: Multiple Interpreters in the Stdlib