PYTHON / ITERATORS, GENERATORS, AND COMPREHENSIONS
Comprehension patterns beyond lists
Build set, dict, and nested comprehensions with confidence, and predict exactly when duplicates get collapsed silently.
What you will learn
- Pick the container by bracket: [] list, {expr} set, {k: v} dict, () generator expression
- Read multiple for clauses like nested loops: the leftmost clause is the outer loop
- Expect a dict comprehension to keep the last value written for each repeated key
- Put a bare if after the for clause; use 'a if c else b' only in the element expression
Understanding Comprehension patterns beyond lists
Every comprehension in Python uses the same engine: a chain of for and if clauses that feeds one element expression, evaluated once per surviving item. What changes is the container you ask for. Square brackets collect into a list, braces around a single expression build a set, braces around a key: value pair build a dict, and parentheses give a generator expression instead of a concrete container.
Set and dict comprehensions differ from list comprehensions in a way that goes beyond syntax: they hash their elements or keys. A set comprehension drops repeated elements, and a dict comprehension behaves like repeated d[key] = value in iteration order, so a repeated key keeps the last value and quietly discards the earlier ones. That makes them ideal for normalising and deduplicating messy input, and a source of hard-to-see bugs when you assumed your key expression was unique — len(result) simply comes out smaller with no error.
Two for clauses in a row are not the same as one comprehension inside another. Written left to right, the clauses nest outer to inner, so [x for row in matrix for x in row] flattens; a comprehension placed in the element expression, as in [[...] for row in matrix], produces a nested structure. Later clauses may use names bound by earlier ones (that is why 'for row in matrix for x in row' works and the reverse does not), and each loop variable lives in the comprehension's own scope, so it never overwrites a same-named variable outside.
words = ["Ada", "grace", "ADA", "Alan", "grace", "Edsger"]
unique = {w.lower() for w in words} # set: duplicates vanish
canonical = {w.lower(): w for w in words} # dict: last spelling wins
pairs = [(a, b) for a in "AB" for b in (1, 2)] # left clause = outer loop
grid = [[r * c for c in range(1, 4)] for r in range(1, 3)]
print(len(words), len(unique))
print(sorted(unique))
print(sorted(canonical.items()))
print(pairs)
print(grid)The brackets and the shape of the element expression decide which container a comprehension builds, and the hashing containers (set, dict) silently collapse duplicates.
Worked examples
Filtering rows versus choosing values
Shows the difference between an if clause that removes items and a conditional expression that only changes the value.
stock = {"apple": 0, "pear": 12, "fig": 3, "plum": 0}
in_stock = {name: n for name, n in stock.items() if n}
labels = {name: ("out" if n == 0 else "ok") for name, n in stock.items()}
print(in_stock)
print(labels)
print(len(in_stock), len(labels))Example explained
Line 1`for name, n in stock.items()` unpacks each pair, so both parts are available to the key and value expressions.
Line 2The trailing `if n` is a filter: keys with a zero count never reach the dict, which is why in_stock has 2 entries.
Line 3The `"out" if n == 0 else "ok"` form sits inside the value expression, so every key survives and only the value changes.
Line 4The result dict follows the iteration order of stock, since dicts keep insertion order.
Flatten, transpose, collect
Contrasts two for clauses (flattening) with a comprehension nested in the element expression (transposing), plus a set comprehension for membership tests.
matrix = [[1, 2, 3], [4, 5, 6]]
flat = [x for row in matrix for x in row]
transposed = [[row[i] for row in matrix] for i in range(3)]
evens = {x for row in matrix for x in row if x % 2 == 0}
print(flat)
print(transposed)
print(sorted(evens), 4 in evens)Example explained
Line 1In `flat`, the second clause iterates the `row` bound by the first, so the element expression runs once per number: six items, one flat list.
Line 2In `transposed`, the inner comprehension is the element expression, so each pass over `range(3)` appends a whole new list.
Line 3`evens` combines two for clauses with a filter and hashes the results, giving a container built for `in` tests rather than for order.
Line 4`sorted(evens)` is used for printing because a set has no meaningful display order to rely on.
Which bracket gives what
Demonstrates that parentheses do not make a tuple, and that the loop variable stays inside the comprehension.
items = ["a", "bb", "ccc"]
not_a_tuple = (len(s) for s in items)
print(type(not_a_tuple).__name__)
print(tuple(len(s) for s in items))
print(frozenset(len(s) for s in items) == frozenset({1, 2, 3}))
x = "untouched"
sizes = [len(x) for x in items]
print(sizes, x)Example explained
Line 1Parentheses produce a generator object, not a tuple, so there is no such thing as a tuple comprehension.
Line 2`tuple(...)` and `frozenset(...)` consume that generator to get the immutable container you actually wanted.
Line 3The comprehension's `x` lives in its own scope, so the outer `x` still prints as 'untouched' afterwards.
When collapsing keys hides data
Shows a dict comprehension losing entries to key collisions and the aggregating alternative.
sales = [("north", 10), ("south", 4), ("north", 7), ("east", 3)]
last_only = {region: amount for region, amount in sales}
totals = {}
for region, amount in sales:
totals[region] = totals.get(region, 0) + amount
print(last_only)
print(totals)
print(sum(last_only.values()), sum(totals.values()))Example explained
Line 1The comprehension assigns 'north' twice, and the second assignment overwrites 10 with 7 without any warning.
Line 2The totals sum shows the damage: 14 instead of 24, a silent data loss rather than an exception.
Line 3A comprehension cannot accumulate across items, so aggregation needs an explicit loop (or collections.Counter).
Important notes
Set elements and dict keys must be hashable, so `{row for row in matrix}` over lists raises TypeError: unhashable type: 'list' — convert to tuples first.
`{}` is an empty dict, not an empty set; use `set()` when you need an empty set.
Common mistakes
Writing `{w.lower() for w in words}` when a mapping was intended: you get a set, and the first lookup like result['ada'] fails with TypeError: 'set' object is not subscriptable.
Putting the condition before the for, as in `[x if x % 2 == 0 for x in nums]`: that is a SyntaxError, because a bare if belongs after the for and a conditional expression in the element slot requires an else.
Assuming a dict comprehension keeps one entry per source item: repeated keys collapse silently, so totals and lengths come out wrong with no traceback to point at.
Try it yourself
Change, predict, then run
Given emails = ['A@x.com', 'b@Y.com', 'a@x.com', 'c@z.com'], build a set comprehension of the lowercased addresses and a dict comprehension mapping lowercased domain to lowercased local part. Print both (sorted for the set) and explain in a comment why the dict has fewer entries than the input list.
Open the Python workspaceCheck your understanding
What does {len(w): w for w in ['one', 'two', 'six', 'ten']} evaluate to, and why?
- {3: 'ten'} — all four keys are equal, and each later item overwrites the previous value
- {3: 'one'} — the first value for a key is kept and later duplicates are ignored
- {3: ['one', 'two', 'six', 'ten']} — values for the same key are collected into a list
- It raises ValueError because the key 3 is produced four times
Show answer
A dict comprehension is equivalent to repeating d[key] = value in iteration order, so the last assignment for key 3 wins and 'ten' survives. 'one' would only be kept if duplicates were ignored, which dict assignment never does; nothing groups values automatically and duplicate keys are not an error.