Skip to content

Python list comprehensions: build lists in one expression

  • How [expr for x in it if cond] maps to an ordinary for loop, including multiple for clauses.
  • The difference between a filter (if at the end) and a conditional expression (a if c else b at the front).
  • Why loop variables do not leak, and why a comprehension in a class body cannot see class attributes.
  • The dict, set and generator expression forms, and which to pick.
  • What “comprehensions are faster” really means, and when a plain loop is clearer.

A list comprehension builds a new list from an iterable in one expression: [expression for item in iterable if condition]. It was added in Python 2.0 (PEP 202). The same syntax produces sets and dicts ({...}) and lazy generator expressions ((...)). Since Python 3.0 the loop variable stays private to the comprehension, and since Python 3.12 list, set and dict comprehensions are inlined into the surrounding function rather than run as a hidden function call (PEP 709).

Building a list with a loop takes four lines and a mutable accumulator: create an empty list, loop, test, append. In the OrdersPortal reporting job, that pattern appears for every report column. A comprehension states the result directly: “the IDs of the paid orders”.

Running the samples: each sample is a complete script. Save it as a .py file and run it with Python 3.14 (python3 sample.py). The outputs shown were produced with CPython 3.14.6; one sample marked as Python 3.15 was run on 3.15.0b4.

orders = [
{"id": "A-1001", "status": "paid", "total": 98.00},
{"id": "A-1002", "status": "cancelled", "total": 19.50},
{"id": "A-1003", "status": "paid", "total": 141.00},
]
# Before: an empty list, a loop, a condition and append().
paid_ids = []
for order in orders:
if order["status"] == "paid":
paid_ids.append(order["id"])
print(paid_ids)
# After: one expression that says what the list contains.
paid_ids = [order["id"] for order in orders if order["status"] == "paid"]
print(paid_ids)

Output:

['A-1001', 'A-1003']
['A-1001', 'A-1003']

The comprehension also cannot forget the append or append twice in two branches.

The language reference describes a comprehension as a single expression followed by at least one for clause and zero or more for or if clauses. Read the clauses left to right as nested blocks, then put the leading expression in the innermost block.

skus = ["KB-101", "MS-220", "HD-330", "KB-101"]
# Clauses nest left to right, like the equivalent for/if blocks.
sizes = ["S", "M"]
print([f"{sku}/{size}" for sku in skus[:2] for size in sizes])
# A filter (if at the end) drops items; a conditional expression maps them.
print([s for s in skus if s.startswith("KB")])
print([s if s.startswith("KB") else "other" for s in skus])
# Flatten a list of lists: outer loop first.
baskets = [["KB-101", "MS-220"], [], ["HD-330"]]
print([sku for basket in baskets for sku in basket])
# The same syntax builds sets, dicts and lazy generators.
print(sorted({s[:2] for s in skus})) # sets are unordered
print({s: len(s) for s in skus})
gen = (len(s) for s in skus)
print(gen.__class__.__name__, sum(gen))
# Loop variables do not leak into the enclosing scope.
sku = "unchanged"
_ = [sku for sku in skus]
print(sku)

Output:

['KB-101/S', 'KB-101/M', 'MS-220/S', 'MS-220/M']
['KB-101', 'KB-101']
['KB-101', 'other', 'other', 'KB-101']
['KB-101', 'MS-220', 'HD-330']
['HD', 'KB', 'MS']
{'KB-101': 6, 'MS-220': 6, 'HD-330': 6}
generator 24
unchanged

Key points:

  • Filter versus mapping. A trailing if decides whether an item is included and cannot have an else. A conditional expression at the front decides what each item becomes, so every item is kept.
  • Multiple for clauses run in the order written. [sku for basket in baskets for sku in basket] reads as “for basket in baskets: for sku in basket: append sku”.
  • Four forms. [...] builds a list, {x ...} a set, {k: v ...} a dict (duplicate keys keep the last value), and (...) a generator expression that produces items lazily. There is no tuple comprehension; tuple(x for x in it) builds one.
  • Scope. Apart from the leftmost iterable, a comprehension runs in its own implicit scope, so sku outside is untouched. PEP 709 inlining (3.12) kept this isolation; it only removed the separate frame from tracebacks and profiles.
  • Leftmost iterable is evaluated first and once, in the enclosing scope. Later for clauses and all if conditions run inside the comprehension.
  • yield is not allowed inside a comprehension, and an async for or await is allowed only inside an async def function (Python 3.6+).

Because the comprehension’s scope skips over a class body, names defined in the class are visible only in the leftmost iterable:

class OrderReport:
statuses = ["paid", "cancelled", "refunded"]
prefix = "status:"
# The leftmost iterable is evaluated in the class body: fine.
upper = [s.upper() for s in statuses]
try:
# Other names are looked up in a nested scope that skips the class body.
labels = [prefix + s for s in statuses]
except NameError as exc:
error = str(exc)
print(OrderReport.upper)
print(OrderReport.error)

Output:

['PAID', 'CANCELLED', 'REFUNDED']
name 'prefix' is not defined

Move the comprehension into a method or a module-level function, or pass the value through the leftmost iterable.

The OrdersPortal daily report needs per-order totals, units sold per SKU, the regions with paid orders and total revenue. Each column uses the form that fits: a list comprehension to filter, a dict comprehension for a lookup table, a nested comprehension to flatten order lines, a set comprehension for distinct values, and a generator expression for a single aggregate. The accumulation into a counter stays a plain loop.

from collections import Counter
from dataclasses import dataclass
@dataclass(frozen=True)
class Line:
sku: str
qty: int
unit_price: float
@dataclass(frozen=True)
class Order:
id: str
status: str
region: str
lines: list[Line]
ORDERS = [
Order("A-1001", "paid", "EU", [Line("KB-101", 2, 49.0), Line("MS-220", 1, 19.5)]),
Order("A-1002", "cancelled", "US", [Line("HD-330", 1, 35.25)]),
Order("A-1003", "paid", "US", [Line("HD-330", 4, 35.25)]),
Order("A-1004", "paid", "EU", []),
]
def order_total(order: Order) -> float:
return sum(line.qty * line.unit_price for line in order.lines)
paid = [o for o in ORDERS if o.status == "paid" and o.lines]
# dict comprehension: order id -> total
totals = {o.id: order_total(o) for o in paid}
# flatten nested lines (outer loop first)
all_lines = [line for o in paid for line in o.lines]
# accumulating into a counter is a job for a plain loop
units = Counter()
for line in all_lines:
units[line.sku] += line.qty
# set comprehension: which regions had paid orders
regions = sorted({o.region for o in paid})
# generator expression: no intermediate list for a single aggregate
revenue = sum(order_total(o) for o in paid)
print(totals)
print(dict(units))
print(regions)
print(f"revenue {revenue:.2f}")

Output:

{'A-1001': 117.5, 'A-1003': 141.0}
{'KB-101': 2, 'MS-220': 1, 'HD-330': 4}
['EU', 'US']
revenue 258.50

Python 3.15, currently in development, adds unpacking in comprehensions (PEP 798), so the flattening step can be written [*o.lines for o in paid]. This fragment was checked on 3.15.0b4 and is a SyntaxError on 3.14:

# Python 3.15+ only
baskets = [["KB-101", "MS-220"], [], ["HD-330"]]
print([*basket for basket in baskets]) # ['KB-101', 'MS-220', 'HD-330']

Performance, measured. The speed advantage of a comprehension comes from skipping the out.append attribute lookup and call on every item. It is real but modest, and the work done per item usually dominates. This script compares the two on a simple filter-and-map:

import timeit
setup = "data = list(range(10_000))"
loop = """
out = []
for x in data:
if x % 3 == 0:
out.append(x * 2)
"""
comp = "out = [x * 2 for x in data if x % 3 == 0]"
for name, stmt in [("for loop", loop), ("comprehension", comp)]:
best = min(timeit.repeat(stmt, setup, number=200, repeat=5))
print(f"{name:14} {best * 1000 / 200:.3f} ms per run")

Output on one machine with CPython 3.14.6 (timings vary by machine and run):

for loop 0.233 ms per run
comprehension 0.225 ms per run

On the same machine CPython 3.11 showed a larger gap (0.222 ms versus 0.195 ms). Choose comprehensions for clarity; do not rewrite readable loops for speed without profiling.

Memory. A list comprehension materializes every item. If the result feeds one consumer (sum, max, any, "".join, a for loop), pass a generator expression instead: sum(x.total for x in orders) holds one item at a time. any() and all() also short-circuit on a generator, which a list cannot do because the list is fully built first.

  • Using a comprehension for side effects. [print(o) for o in orders] builds a list of None values. Use a for loop when you are not keeping the result.
  • Putting else on the filter. [x for x in xs if c else y] is a syntax error. Move the conditional to the front: [x if c else y for x in xs].
  • Reversing nested for clauses. [sku for sku in basket for basket in baskets] fails because basket is not defined yet. Write the outer loop first.
  • Building a huge list for one aggregate. sum([o.total for o in orders]) allocates a list; sum(o.total for o in orders) does not.
  • Packing too much logic in. Three for clauses with two filters and a function call is harder to read and debug than a short loop or a helper function.

Use a comprehension when the result is a new collection derived item by item from existing data, and the whole thing fits on one or two readable lines.

Situation Prefer
Transform or filter items into a new list List comprehension
Distinct values Set comprehension
Lookup table keyed by a field Dict comprehension
Feed sum, any, all, max, join Generator expression
Accumulate, count, or update existing objects for loop (or Counter, defaultdict)
Side effects only (print, write, send) for loop
Error handling per item, several steps, or more than two nested clauses for loop or a helper function
Just apply an existing function list(map(func, items)) is equally clear
Feature Python version
List comprehensions 2.0 (PEP 202)
Generator expressions 2.4 (PEP 289)
Dict and set comprehensions 2.7 and 3.0 (PEP 274)
Loop variable no longer leaks 3.0
Asynchronous comprehensions 3.6 (PEP 530)
Comprehension inlining 3.12 (PEP 709)
Unpacking ([*x for x in xs]) 3.15, in development (PEP 798)

1. What is the difference between [x for x in xs if x > 0] and [x if x > 0 else 0 for x in xs]? The first is a filter: items that fail the test are dropped, so the result can be shorter. The second is a conditional expression: every item is kept and mapped to either x or 0, so the result has the same length as xs.

2. How do you flatten a list of lists with a comprehension, and what order do the clauses go in? [item for sub in nested for item in sub]. The clauses appear in the same order as nested for statements would, outer loop first. On Python 3.15 you can also write [*sub for sub in nested].

3. When would you use a generator expression instead of a list comprehension? When the values are consumed once, in order, by something like sum, any, max or a for loop. It avoids allocating the whole list and lets any/all stop early. Use a list when you need len, indexing or several passes.

4. Why does a comprehension in a class body raise NameError for a class attribute? Comprehensions run in their own scope, and class bodies are not enclosing scopes for nested scopes. Only the leftmost iterable is evaluated in the class body itself. Move the logic into a method or a function.

5. Senior scenario: a data pipeline step is a nested comprehension that parses, validates and transforms 20 million records into a list, and it is now running out of memory and is hard to debug. How do you restructure it? Split it into named generator functions (parse, validate, transform) chained lazily so only one record is in memory at a time, and write results in batches (for example with itertools.batched) instead of collecting a list. Put per-record error handling and logging in the validation step, where a comprehension has no room for try/except. Keep small, side-effect-free comprehensions inside each step where they stay readable, and profile before and after so the change is measured.

  • [expr for x in it if cond] is a list built item by item; clauses read left to right as nested blocks.
  • A trailing if filters; a if c else b at the front maps.
  • Loop variables stay private; in a class body, only the leftmost iterable sees class attributes.
  • Use set, dict and generator forms where they fit; use a generator for single-pass aggregates.
  • Comprehensions are a little faster than append loops, but choose them for clarity, and switch to a loop for side effects, accumulation or complex logic.