Skip to content

threading vs multiprocessing in Python 3.14

  • How threading and multiprocessing differ in execution model, memory sharing and cost.
  • Why threads speed up I/O-bound work but not CPU-bound work on the default (GIL) build of CPython.
  • What changed in Python 3.13 and 3.14: the free-threaded build (PEP 703, PEP 779) and subinterpreters (concurrent.interpreters, InterpreterPoolExecutor).
  • Why the default multiprocessing start method on Linux changed to forkserver in 3.14, and what breaks.
  • How to choose, and how to pick the right pool at run time.

threading runs several threads inside one process. They share memory directly, but on the default CPython build the global interpreter lock (GIL) lets only one thread execute Python bytecode at a time, so threads help while code waits on I/O and do not help pure-Python computation. multiprocessing runs separate OS processes, each with its own GIL: real CPU parallelism, paid for in startup time and pickling. Python 3.14 adds a supported (optional) free-threaded build and standard-library subinterpreters.

A script that calls a remote API for each OrdersPortal order spends almost all its time waiting. Sequentially, the waits add up; threads let them overlap.

Running the samples: each sample is a complete script. Save it as a .py file and run it with Python 3.14 (python3 sample.py). Outputs come from CPython 3.14.6 on a 4-vCPU Linux container; free-threaded results come from the python3.14t executable of the same release. All timings are machine-dependent and vary from run to run; compare the ratios, not the numbers. time.sleep stands in for network calls.

import time
from concurrent.futures import ThreadPoolExecutor
def fetch_tracking(order_id: str) -> str:
"""Stand-in for an HTTPS call to a carrier API: 0.5 s of waiting."""
time.sleep(0.5)
return f"{order_id}: in transit"
order_ids = [f"ORD-{n}" for n in range(1001, 1009)]
start = time.perf_counter()
sequential = [fetch_tracking(o) for o in order_ids]
print(f"sequential: {time.perf_counter() - start:.2f} s")
start = time.perf_counter()
with ThreadPoolExecutor(max_workers=8) as pool:
threaded = list(pool.map(fetch_tracking, order_ids))
print(f"8 threads: {time.perf_counter() - start:.2f} s")
print(threaded[:2], threaded == sequential)

Output (timings machine-dependent):

sequential: 4.00 s
8 threads: 0.50 s
['ORD-1001: in transit', 'ORD-1002: in transit'] True

Blocking calls such as socket reads, file I/O and time.sleep release the GIL while they wait, so eight threads finish in about the time of one call. The rest of the page uses concurrent.futures, which gives threads, processes and interpreters the same Executor interface.

Threads Processes Subinterpreters (3.14)
Lives in One process Separate OS processes One process, isolated interpreters
Memory Shared Separate; data is pickled Separate; data is pickled or explicitly shared
Parallel Python bytecode Only on the free-threaded build Yes Yes (one GIL per interpreter)
Startup cost Low High Medium (not yet optimized)
Main risk Races on shared state Pickling, start-method surprises Extension modules not yet compatible

The same pure-Python function, run four times, through each executor:

import sys
import sysconfig
import time
from concurrent.futures import (
InterpreterPoolExecutor, ProcessPoolExecutor, ThreadPoolExecutor,
)
def score_orders(seed: int) -> int:
"""CPU-bound stand-in: a pure-Python fraud score over 10 million steps."""
total = seed
for i in range(10_000_000):
total = (total * 31 + i) % 1_000_003
return total
def run(label, executor_cls, batches):
start = time.perf_counter()
with executor_cls(max_workers=4) as pool:
results = list(pool.map(score_orders, batches))
print(f"{label:<12} {time.perf_counter() - start:5.2f} s")
return results
if __name__ == "__main__":
gil = "disabled" if not sys._is_gil_enabled() else "enabled"
ft = sysconfig.get_config_var("Py_GIL_DISABLED")
print(f"Python {sys.version.split()[0]}, free-threaded build: {bool(ft)}, GIL {gil}")
batches = [1, 2, 3, 4]
start = time.perf_counter()
expected = [score_orders(b) for b in batches]
print(f"{'sequential':<12} {time.perf_counter() - start:5.2f} s")
for label, cls in [("threads", ThreadPoolExecutor),
("processes", ProcessPoolExecutor),
("interpreters", InterpreterPoolExecutor)]:
assert run(label, cls, batches) == expected

Output with python3.14 and with python3.14t (timings machine-dependent):

Python 3.14.6, free-threaded build: False, GIL enabled
sequential 1.76 s
threads 1.54 s
processes 0.56 s
interpreters 0.43 s
Python 3.14.6, free-threaded build: True, GIL disabled
sequential 1.60 s
threads 0.46 s
processes 0.69 s
interpreters 0.74 s

On the GIL build, threads give no speedup for CPU work, while processes and subinterpreters run on several cores. On the free-threaded build, plain threads scale too, without pickling or extra interpreters. A second run on the same machine took 0.75 s for free-threaded threads and 1.01 s for interpreters, so treat these numbers as an illustration, not a benchmark.

PEP 703 made the GIL optional. Python 3.13 shipped the free-threaded build as experimental, as a separate executable (python3.13t). In Python 3.14, PEP 779 moved it to officially supported but still optional (“phase II”): the default build still has a GIL, and making free-threading the default is undecided. What’s New in 3.14 puts the single-threaded overhead at roughly 5 to 10%, depending on platform and compiler. Check a running interpreter with sysconfig.get_config_var("Py_GIL_DISABLED") (the build) and sys._is_gil_enabled() (the GIL can be re-enabled with PYTHON_GIL=1, -X gil=1, or automatically when an extension module that is not marked free-threading-safe is imported, which prints a warning).

PEP 734 added concurrent.interpreters: several isolated interpreters in one process, each with its own GIL since Python 3.12 (PEP 684). InterpreterPoolExecutor wraps them in the familiar executor interface.

from concurrent import interpreters
def with_tax(total: float) -> float:
return round(total * 1.2, 2)
interp = interpreters.create()
print(interp.call(with_tax, 59.9)) # runs in the other interpreter, this thread
interp.exec("ORDERS_SEEN = 1")
interp.exec("print('isolated module state:', ORDERS_SEEN)")
print("main sees ORDERS_SEEN?", "ORDERS_SEEN" in globals())
worker = interp.call_in_thread(with_tax, 10.0) # returns a threading.Thread
worker.join()
interp.close()

Output:

71.88
isolated module state: 1
main sees ORDERS_SEEN? False

An interpreter provides isolation, not concurrency: call runs in the current thread, and parallelism comes from running interpreters in several threads. The 3.14 documentation lists current limits: slow startup, extra memory per interpreter, few ways to share objects, and many third-party extension modules without support yet.

Every argument and result sent to a process pool is pickled, and each worker re-imports your main module. In 3.14 the default start method on Linux and other non-macOS Unix platforms changed from fork to forkserver; Windows and macOS keep spawn.

import multiprocessing as mp
from concurrent.futures import ProcessPoolExecutor
TAX_RATE = 0.2 # module-level constant: re-imported in every child process
def with_tax(total: float) -> float:
return round(total * (1 + TAX_RATE), 2)
if __name__ == "__main__":
print("default start method:", mp.get_start_method())
with ProcessPoolExecutor(max_workers=2) as pool:
print(list(pool.map(with_tax, [10.0, 59.9])))
try:
print(list(pool.map(lambda t: t * 2, [10.0])))
except Exception as exc:
print(type(exc).__name__ + ":", exc)

Output (Linux; the memory address varies):

default start method: forkserver
[12.0, 71.88]
PicklingError: Can't pickle <function <lambda> at 0x7fdf9d698880>: it's not found as __main__.<lambda>

Code that relied on fork copying mutated module state into children, or that passed unpicklable objects, worked on Linux before 3.14 and fails now. Request mp.get_context("fork") explicitly only if you need it and the parent has no threads; since 3.12, os.fork() in a process that Python detects has several threads emits a DeprecationWarning.

The GIL never made compound operations atomic. A read-modify-write such as self.reserved += 1 can interleave between threads; a threading.Lock makes it safe.

import threading
class StockCounter:
def __init__(self) -> None:
self.reserved = 0
self._lock = threading.Lock()
def reserve_unsafe(self) -> None:
current = self.reserved # read
current += 1 # modify
self.reserved = current # write: another thread may have written in between
def reserve_safe(self) -> None:
with self._lock:
self.reserved += 1
def hammer(method, threads=8, calls=100_000):
workers = [threading.Thread(target=lambda: [method() for _ in range(calls)])
for _ in range(threads)]
for w in workers:
w.start()
for w in workers:
w.join()
for name in ("reserve_unsafe", "reserve_safe"):
counter = StockCounter()
hammer(getattr(counter, name))
print(f"{name}: {counter.reserved:,} of 800,000")

Output with python3.14t (the lost count varies per run):

reserve_unsafe: 248,997 of 800,000
reserve_safe: 800,000 of 800,000

On the GIL build this run happened to print 800,000 twice. That is scheduling luck: the unsafe version is a bug on either build; free-threading just exposes it more often.

OrdersPortal’s nightly reconciliation fetches settlements from a payment provider (I/O-bound) and then computes a risk score for each (CPU-bound). The I/O stage always uses threads; the CPU stage picks threads on a free-threaded interpreter and processes otherwise.

"""OrdersPortal nightly reconciliation: I/O stage, then CPU stage."""
import os
import sys
import time
from concurrent.futures import Executor, ProcessPoolExecutor, ThreadPoolExecutor
def fetch_settlement(order_id: str) -> tuple[str, int]:
time.sleep(0.2) # stand-in for a payment-provider API call
return order_id, int(order_id[-4:]) # (order, amount in cents)
def risk_score(item: tuple[str, int]) -> tuple[str, int]:
order_id, cents = item
total = cents
for i in range(2_000_000): # pure-Python CPU work
total = (total * 31 + i) % 1_000_003
return order_id, total % 100
def cpu_executor(workers: int) -> Executor:
"""Pick the cheapest pool that gives real parallelism on this interpreter."""
if not sys._is_gil_enabled():
return ThreadPoolExecutor(workers) # free-threaded build
return ProcessPoolExecutor(workers) # safest default with a GIL
if __name__ == "__main__":
order_ids = [f"ORD-{n}" for n in range(1001, 1013)]
workers = min(4, os.cpu_count() or 1)
start = time.perf_counter()
with ThreadPoolExecutor(max_workers=12) as io_pool: # I/O: many threads
settlements = list(io_pool.map(fetch_settlement, order_ids))
print(f"fetched {len(settlements)} settlements in {time.perf_counter() - start:.2f} s")
start = time.perf_counter()
with cpu_executor(workers) as cpu_pool:
scores = dict(cpu_pool.map(risk_score, settlements))
print(f"scored with {type(cpu_pool).__name__} in {time.perf_counter() - start:.2f} s")
print("flagged:", sorted(o for o, s in scores.items() if s > 80))

Output with python3.14, then python3.14t (timings machine-dependent):

fetched 12 settlements in 0.21 s
scored with ProcessPoolExecutor in 0.34 s
flagged: ['ORD-1005', 'ORD-1006']
fetched 12 settlements in 0.21 s
scored with ThreadPoolExecutor in 0.30 s
flagged: ['ORD-1005', 'ORD-1006']

risk_score is a module-level function over picklable tuples, so it works with either pool, and with InterpreterPoolExecutor when every imported extension supports subinterpreters.

  • Using threads for pure-Python CPU work on the GIL build. It adds overhead without parallelism. Use processes, subinterpreters or the free-threaded build, or move the hot loop into a library that releases the GIL (NumPy, compression, hashing).
  • Assuming the GIL makes your code thread-safe. It protects the interpreter’s internals, not your invariants. Guard shared mutable state with a Lock, or pass data through a queue.Queue.
  • Forgetting if __name__ == "__main__":. With spawn and forkserver, children import the main module; unguarded pool creation fails or recurses.
  • Sending unpicklable work to a process pool. Lambdas, nested functions, open connections and locks cannot be pickled. Use module-level functions and plain data.
  • Relying on fork semantics after 3.14. Children no longer inherit mutated globals on Linux by default. Pass state explicitly or use an initializer.
  • Sending large objects per task. Pickling big data to every worker can cost more than the work. Send IDs or paths, or use multiprocessing.shared_memory.
  • Oversubscribing. Four processes that each start a multithreaded math library can create dozens of busy threads on four cores. Size pools to the CPUs you actually have.
Workload First choice Why Avoid
Many network or disk waits (tens to hundreds) ThreadPoolExecutor Waits release the GIL; cheap, shared memory Processes (pure overhead)
Thousands of concurrent sockets asyncio One thread, no per-connection thread One thread per connection
Pure-Python CPU work, GIL build ProcessPoolExecutor One GIL per process, mature Threads
Pure-Python CPU work, all dependencies support it InterpreterPoolExecutor (3.14) Lighter than processes Extensions without subinterpreter support
CPU work on a free-threaded build ThreadPoolExecutor Real parallelism with shared memory Unsynchronized shared state
CPU work in NumPy or another GIL-releasing library Threads Library code runs in parallel already Processes that copy large arrays
Crash isolation, memory limits per task Processes A crash or leak dies with the worker Threads, subinterpreters

Skip concurrency when the bottleneck is a database or an API rate limit: more workers only queue at the same place.

Version Change
2.6 multiprocessing added.
3.2 concurrent.futures (ThreadPoolExecutor, ProcessPoolExecutor).
3.8 macOS default start method changed to spawn.
3.12 Per-interpreter GIL (PEP 684), C API only.
3.13 Experimental free-threaded build (PEP 703), python3.13t; sys._is_gil_enabled().
3.14 Free-threaded build officially supported but optional (PEP 779); concurrent.interpreters (PEP 734) and InterpreterPoolExecutor; forkserver is the default start method on Unix except macOS.

1. Why do threads speed up downloads but not image resizing written in pure Python? On the GIL build only one thread runs bytecode at a time. Blocking I/O releases the GIL, so downloads overlap; pure-Python computation holds it, so threads take turns. Resizing in a C library that releases the GIL, or on a free-threaded build, does run in parallel.

2. A job that worked on Python 3.13 on Linux fails on 3.14 with a PicklingError or sees empty globals in workers. What changed? The default start method moved from fork to forkserver. Workers now start from a clean server process, so arguments must be picklable and module-level state set by the parent after import is not copied. Fix the code to pass state explicitly; only fall back to get_context("fork") deliberately.

3. What is the difference between a subinterpreter and a process? Both isolate Python state and need data to be copied or explicitly shared. Subinterpreters live in one OS process, so they start faster and use fewer resources, but they share a crash domain and need extension modules that support multiple interpreters.

4. Senior scenario: OrdersPortal’s report service is CPU-bound, runs on a 16-core container, and uses a C extension for PDF rendering. The team wants to adopt the free-threaded build. How do you evaluate it? Confirm every compiled dependency ships free-threaded wheels; otherwise importing it re-enables the GIL (with a warning), which sys._is_gil_enabled() will show. Audit shared mutable state, because races that the GIL hid become visible. Benchmark the real workload on both builds: single-threaded code runs a few percent slower, so the gain depends on how much work is parallel. Keep the executor choice behind one function, as in the realistic example, so the service can fall back to processes, and roll out on a canary first.

  • I/O-bound: threads (or asyncio at very high concurrency). CPU-bound on the GIL build: processes or, in 3.14, subinterpreters.
  • The 3.14 free-threaded build is supported but optional; it lets threads use every core but exposes races.
  • Process pools pickle everything and, since 3.14 on Linux, start workers with forkserver.
  • The GIL never made your own read-modify-write code safe; use locks or queues.
  • Measure on your hardware; timings vary between machines and runs.
  • The GIL: what the global interpreter lock protects and why it exists.
  • async/await: single-threaded concurrency for large numbers of I/O waits.
  • Memory management: reference counting, the reason the GIL was simple to keep.
  • Context managers: with for executors and locks.
  • Generators: streaming work items into a pool without building huge lists.