PYTHON / FILE HANDLING
The with statement and context managers
Use with to guarantee cleanup, read the __enter__/__exit__ protocol, and build your own context managers as classes or generators.
What you will learn
- Trace the exact call order of __enter__, the block body, and __exit__
- Write a class context manager and control what the as name is bound to
- Build a context manager from a generator with @contextmanager and try/finally
- Decide when __exit__ should suppress an exception instead of letting it propagate
Understanding The with statement and context managers
A with statement is a compiler-enforced pairing of setup and teardown. Before the block runs, Python looks up __enter__ on the object and calls it; whatever that method returns is what the as name is bound to. When the block ends, for any reason at all, Python calls __exit__ on the same object. That 'any reason' is the whole point: a return, a break, a raised exception, or falling off the end all take the same path out, so the teardown cannot be skipped by a code path you forgot about.
__exit__ is not called with zero arguments. It receives three: the exception type, the exception instance, and the traceback, all None if the block finished normally. That signature is why a context manager can react to failure, for example rolling back instead of committing. The return value matters too: a truthy return tells Python the exception was handled and should stop propagating, while None or False lets it continue rising. Since a method with no return statement returns None, normal cleanup code propagates exceptions by default, which is what you almost always want.
Writing the two dunder methods is boilerplate when the resource is simple, so contextlib.contextmanager turns a generator into a context manager. Everything before yield is the __enter__ body, the value you yield becomes the as value, and everything after yield is __exit__. When the block raises, the exception is thrown back into the generator at the yield point, so code after yield only runs if you wrap the yield in try/finally. Without that finally, an exception in the body skips your cleanup entirely.
class Tracer:
def __init__(self, name):
self.name = name
def __enter__(self):
print('enter', self.name)
return self.name.upper()
def __exit__(self, exc_type, exc_value, tb):
print('exit', self.name, '| exc_type =', exc_type)
return False
with Tracer('outer') as label:
print('body sees', label)
try:
with Tracer('risky') as label:
raise ValueError('boom')
except ValueError as err:
print('caught', err)
print('label still bound:', label)
with calls __enter__ before the block and __exit__ after it no matter how the block exits, which is what makes cleanup guaranteed rather than hoped for.
Worked examples
A generator-based context manager that restores state
Shows how @contextmanager splits setup and teardown around yield, and why the try/finally is required.
from contextlib import contextmanager
contextmanager
def temp_setting(store, key, value):
old = store.get(key)
store[key] = value
print('set', key, '->', value)
try:
yield store
finally:
if old is None:
del store[key]
else:
store[key] = old
print('restored', key, '->', old)
config = {'mode': 'prod'}
with temp_setting(config, 'mode', 'debug') as current:
print('inside:', current)
print('after:', config)
try:
with temp_setting(config, 'retries', 5):
raise RuntimeError('body failed')
except RuntimeError as err:
print('caught:', err)
print('final:', config)
Example explained
Line 1The value after yield is what the as name receives, so current is the same dict object as config.
Line 2The finally block is the teardown half; it runs on the normal exit and again when RuntimeError is thrown into the generator.
Line 3'restored retries -> None' prints before 'caught: body failed' because teardown happens while the exception is still travelling outward.
Line 4old is None for a key that did not exist, so the key is deleted rather than reset, leaving config exactly as it started.
Suppressing an exception from __exit__
Demonstrates that a truthy return from __exit__ stops the exception, and that a falsy return lets it through.
class Catching:
def __init__(self, *types):
self.types = types
self.error = None
def __enter__(self):
return self
def __exit__(self, exc_type, exc_value, tb):
if exc_type is not None and issubclass(exc_type, self.types):
self.error = exc_value
return True
return False
box = Catching(ZeroDivisionError)
with box:
print(1 / 0)
print('never reached')
print('survived, error was:', box.error)
try:
with box:
raise KeyError('missing')
except KeyError as err:
print('not suppressed:', err)
Example explained
Line 11 / 0 raises immediately, so 'never reached' never prints, yet execution continues after the with block.
Line 2return True inside __exit__ tells Python the ZeroDivisionError is handled, which is why the next line runs normally.
Line 3KeyError is not in self.types, so __exit__ returns False and the exception reaches the except clause.
Line 4The manager can store exc_value because __exit__ is handed the exception instance itself, not just its type.
Several managers in one with statement
Shows that a comma-separated with is equivalent to nesting, and that teardown happens in reverse order.
class Resource:
def __init__(self, name):
self.name = name
def __enter__(self):
print('open', self.name)
return self
def __exit__(self, exc_type, exc_value, tb):
print('close', self.name)
with Resource('a') as first, Resource('b') as second:
print('using', first.name, 'and', second.name)
Example explained
Line 1The managers are entered left to right, so 'open a' precedes 'open b'.
Line 2Teardown is last in, first out, because the second with is conceptually nested inside the first.
Line 3That ordering matters when the later resource depends on the earlier one still being alive.
Line 4__exit__ here has no return statement, so it returns None and any exception from the body keeps propagating.
Important notes
A file object works as a context manager only once; reusing the same object in a second with raises ValueError because __enter__ checks that it is still open.
contextlib also ships ready-made managers such as suppress for ignoring chosen exceptions and ExitStack for entering a number of managers decided at runtime.
Common mistakes
Using the as variable after the block ends. The name is still bound because with is not a new scope, but the resource is closed, so a file read raises ValueError: I/O operation on closed file.
Copying return True into __exit__ from an example. Every exception raised in the body is then silently discarded, and bugs in the block become invisible.
Calling yield in a @contextmanager generator without try/finally. When the body raises, the cleanup lines after yield never execute and the resource is left open.
Try it yourself
Change, predict, then run
Write a class-based context manager Guard whose __exit__ prints 'cleanup' and suppresses only ZeroDivisionError, then run it twice: once around 1 / 0 and once around int('x'), and confirm 'cleanup' prints both times while only the ValueError escapes.
Open the Python workspaceCheck your understanding
A context manager's __exit__ logs a message and has no return statement. The with body raises a KeyError. What happens?
- The message is logged and the KeyError continues propagating to the caller
- The message is logged and the KeyError is discarded, since __exit__ finished without error
- __exit__ is skipped because the block did not finish normally
- Python re-runs the block once and then raises the KeyError
Show answer
__exit__ always runs on the way out, and a method with no return statement returns None, which is falsy, so Python re-raises the exception. Option 2 is tempting because __exit__ is handed the exception details, but receiving them is not the same as handling them; only a truthy return value suppresses the exception.