Python test mechanics: pytest fixtures, assertions and lifecycle
Why you care
Every production bug that passes CI undetected passed because the test either did not exist, or existed but was ordered to depend on state set by a previous test — and nobody noticed until the run happened in a different order. Product company engineers are expected to maintain test suites that could run in any sequence. Interviewers ask about fixture scope because the answer tells them whether you have been bitten by shared mutable state, which is the most common source of order-dependent test failures in Python projects. This card owns the mechanics — lifecycle, scope, assertions, and teardown guarantees. The ETL-specific application is in Test an ETL job through failure and replay.
Explain like I'm 5
A test suite is like a hotel that runs a banquet for many guests. The kitchen is stocked once before any guests arrive (setUpClass). Each table gets its own clean tablecloth before that course begins (setUp), the food is served (the test body runs), and the tablecloth is cleared away (tearDown). After the last guest leaves, the kitchen is shut down (tearDownClass). If one guest leaves crumbs on the tablecloth and setUp does not replace it, the next guest finds yesterday's mess — and the failure report blames the wrong course.
Core concept
Python has two test frameworks you will encounter: unittest (stdlib, ships with Python) and pytest (third-party, industry standard). Both use the same arrange-act-assert mental model.
The unittest lifecycle runs in this fixed order:
setUpModuleonce before any class in the modulesetUpClassonce per class (decorated@classmethod)setUponce before each individual test method- the test method itself
tearDownonce after each test method, even if the test raisedtearDownClassonce after all methods in the classtearDownModuleonce after all classes
Teardown guarantee: tearDown runs even when the test body raises. setUpClass teardown (tearDownClass) is skipped only if setUpClass itself raises — which means resources allocated there are not released on setup failure.
pytest fixtures replace the setUp pattern: any test function that names a fixture as a parameter receives its return value. The scope argument controls how often the fixture is created: "function" (default, one per test), "class", "module", or "session" (once per pytest process). A yield fixture replaces tearDown: code after yield runs on cleanup.
Decision rule: use setUp/tearDown (or function-scoped fixtures) for state that must be fresh per test — database connections, mutable containers, temporary files. Use setUpClass (or class/module-scoped fixtures) only for expensive read-only resources — a compiled regex, a loaded static file, an in-memory SQLite schema that no test mutates. Never share mutable objects across tests unless every test reinitialises the relevant fields in setUp.
Minimal example Lifecycle order logged to a list
import unittest
import io
log = []
class T(unittest.TestCase):
@classmethod
def setUpClass(cls):
log.append('setUpClass')
def setUp(self):
log.append('setUp')
def test_a(self):
log.append('test_a')
def test_b(self):
log.append('test_b')
def tearDown(self):
log.append('tearDown')
@classmethod
def tearDownClass(cls):
log.append('tearDownClass')
suite = unittest.TestLoader().loadTestsFromTestCase(T)
unittest.TextTestRunner(stream=io.StringIO(), verbosity=0).run(suite)
print(log)['setUpClass', 'setUp', 'test_a', 'tearDown', 'setUp', 'test_b', 'tearDown', 'tearDownClass']In production Settlement loader with per-test fresh database
import sqlite3
import csv
import io
import unittest
FIXTURE_CSV = 'settlement_id,amount\nS001,10.50\nS002,200.00\n'
def parse_paise(s):
return round(float(s) * 100)
def load_settlements(csv_body, conn):
conn.execute(
'CREATE TABLE IF NOT EXISTS settlements (id TEXT, paise INT)'
)
for row in csv.DictReader(io.StringIO(csv_body)):
conn.execute(
'INSERT INTO settlements VALUES (?,?)',
(row['settlement_id'], parse_paise(row['amount'])),
)
conn.commit()
class SettlementLoaderTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
# Compiled once: each test re-reads this string, not the S3 file.
cls.csv_body = FIXTURE_CSV
def setUp(self):
# Fresh DB per test: mutations in one test cannot bleed into another.
self.conn = sqlite3.connect(':memory:')
def tearDown(self):
self.conn.close()
def test_row_count(self):
load_settlements(self.csv_body, self.conn)
n = self.conn.execute(
'SELECT COUNT(*) FROM settlements'
).fetchone()[0]
self.assertEqual(n, 2)
def test_paise_conversion(self):
load_settlements(self.csv_body, self.conn)
row = self.conn.execute(
'SELECT paise FROM settlements WHERE id=?', ('S001',)
).fetchone()
self.assertEqual(row[0], 1050)
buf = io.StringIO()
runner = unittest.TextTestRunner(stream=buf, verbosity=2)
suite = unittest.TestLoader().loadTestsFromTestCase(SettlementLoaderTests)
result = runner.run(suite)
print('tests run:', result.testsRun, 'failures:', len(result.failures))tests run: 2 failures: 0See it happen setUp and tearDown wrap every individual test
- Called once before any test method in the class runs.
- setUpClass finishes; shared class-level state is now available via cls.*.
- Called before test_a. A fresh SQLite connection is created here.
- The test exercises the loader; it mutates its own isolated DB.
- Control returns regardless of whether the test passed or raised.
- Connection closed. Any state from test_a is gone.
- Called again before test_b — a clean connection, no memory of test_a.
- test_b sees only the state it creates in this setUp call.
In your world
The situation
Your Jenkins job runs a suite of three test methods. Two of them share a module-level list that the first test appends to. The second and third tests pass locally when run in alphabetical order but fail on CI because CI runs them in a different order.
What actually breaks
The build is green one day and red the next with no code change. The failure message is AssertionError: 3 != 2 — a count is wrong, but the count changes by run order, not by logic.
What you'd do on Monday
Move the list into setUp. Each test gets its own fresh copy. If it must be shared, make it read-only or reset it at the start of each setUp.
import unittest
import io
class OrderSafeTests(unittest.TestCase):
def setUp(self):
self.log = [] # fresh list every test
def test_first(self):
self.log.append('first')
self.assertEqual(len(self.log), 1)
def test_second(self):
self.log.append('second')
self.assertEqual(len(self.log), 1)
buf = io.StringIO()
result = unittest.TextTestRunner(stream=buf, verbosity=0).run(
unittest.TestLoader().loadTestsFromTestCase(OrderSafeTests)
)
print('tests run:', result.testsRun, 'failures:', len(result.failures))tests run: 2 failures: 0Imagine you work at Shopify
The numbers
50,000 tests × 80 ms average each = 50,000 × 0.08 s = 4,000 s ≈ 67 min wall-clock with a single worker. With 8 parallel workers: 4,000 s ÷ 8 = 500 s ≈ 8 min.
What the Scenario-A answer does here
When fixtures at class or module scope hold mutable shared state — a DB seeded once per class, a cache populated in setUpClass — parallel workers can interleave reads and writes. Parallelism makes hidden order-dependency crash tests that ran fine sequentially.
What flips at this scale
Every mutable fixture that parallel tests touch must be scoped to the function level (one instance per test), or the mutation must be behind a transaction that is rolled back in teardown. The expensive setup stays at the class level but must produce read-only objects. That contract — expensive shared setup, isolated mutable teardown — is what forces a clean separation between seed data and working data.
The trade-off you'd defend
I would defend function-scoped mutable fixtures over class-scoped shared ones, trading a small amount of setup overhead for the ability to run the full suite in any order with any concurrency setting.
Illustrative; Shopify's internal CI architecture is not published.
pytest fixtures vs unittest setUp
| pytest fixture | unittest setUp / tearDown | |
|---|---|---|
| Scope control | function / class / module / session | class or module level only |
| Teardown | yield in the fixture body | tearDown method |
| Sharing between tests | implicit via parameter name | explicit method call |
| Stdlib | no (requires pip install pytest) | yes |
| Run command | pytest | python -m unittest |
Gotchas
Shared mutable fixture causes order-dependent failures
AssertionError: 5 != 2 on CI; passes locally. The number changes when the test runner changes the execution order.
A list or dict initialised in setUpClass is appended to by setUp or the test body, so each subsequent test inherits the previous test's mutations.
Move the mutable object into setUp so it is reinitialised before each test.
tearDownClass skipped on setUpClass failure
A resource allocated at the start of setUpClass (a temp directory, a socket) is never released if setUpClass raises halfway through.
CPython only calls tearDownClass if setUpClass succeeds. Any partial allocation made before the error leaks.
Use a try/finally inside setUpClass to release partial allocations, or use addClassCleanup (available from Python 3.8+) which always fires.
assertRaises only checks the exception type, not the message
The test passes even though the function raised for a completely different reason than expected.
self.assertRaises(ValueError, func, arg) (and the with form) only checks that the named type was raised; it does not inspect the message. Two different code paths that both raise ValueError are indistinguishable to assertRaises alone.
Use with self.assertRaises(ValueError) as ctx: and then check self.assertIn('expected text', str(ctx.exception)) after the block, or use self.assertRaisesRegex(ValueError, r'expected text', func, arg) to assert both type and message in one call.
Tricky questions
No — an unhandled exception in the test skips teardown, which is why resources leak.
tearDown always runs after the test body, whether the test passed, failed an assertion, or raised an unexpected exception. The only lifecycle method that skips teardown when it itself fails is setUpClass: if setUpClass raises, tearDownClass is not called.
Yes — the test runner calls setUp and then the test method regardless.
If setUp raises, the test method is not called and neither is tearDown. The test is marked as an error, not a failure. This means any resource allocated in setUp before the exception is leaked unless you use addCleanup.
Scenario question
You're handed a test suite of 400 tests that passes reliably when run with python -m unittest discover, but fails intermittently with pytest --randomly-seed=12345. Fix it without changing how the tests are structured.
Model answer
I'd start by running pytest -v --randomly-seed=12345 and collecting the first failing test. I'd then ask: what global or class-level state could test N-1 have mutated that test N depends on? The typical culprits are class-level lists, dicts, or database tables seeded in setUpClass. I'd grep for assignments to cls.* that happen in any method other than setUpClass. The fix is to move the mutable object into setUp, which runs once per test. The trade-off is a small overhead increase for cheap fixtures — acceptable for deterministic results.
Interview corner
CommonWhat is the difference between setUpClass and setUp?
TrickIf setUp raises, is tearDown called?
addCleanup — it registers a cleanup callable before anything can go wrong, and it always fires.”CommonWhat does assertRaises actually check?
ValueError for completely different reasons, and assertRaises passes either way. I always follow it with self.assertIn('expected text', str(ctx.exception)), or I use assertRaisesRegex to check both type and message in one assertion.”Experience checkHow would you structure fixtures for tests that share an expensive S3 download?
Memory hook
A test suite is like a hotel banquet: the kitchen is stocked once (setUpClass), each table gets a clean cloth before every course (setUp), the meal is served (the test), and the cloth is cleared away (tearDown). If setUp does not replace the cloth, the next test inherits the previous guest's mess.
Put every mutable fixture in setUp — never setUpClass — so tests can run in any order.
Self-check
When is setUpClass called relative to setUp?
setUpClass is called once before the first test method in the class; setUp is called before every test method.
What happens if setUp raises?
The test method is not called and tearDown is not called either. The test is reported as an error, not a failure.
Can tearDown release resources safely even when the test fails?
Yes. tearDown runs after every test method regardless of whether the test passed, failed an assertion, or raised an unexpected exception.
What assertion method should you use to verify an exception message?
Use
with self.assertRaises(ExcType) as ctx:and thenself.assertIn('expected text', str(ctx.exception)).What does
iter(tc) is tcindicate about a test case?Nothing — that identity check applies to iterators (see Iterator protocol, exhaustion and pull model). A test case is not an iterator.
Write a test class with a class-level mutable list initialised in setUpClass. Add two tests that each append to that list and assert len == 1. Run with python -m unittest (alphabetical order). Then swap the test method names so they run in reverse order. Paste the failure output showing which assertion breaks and at which count. Then move the list to setUp and confirm both orderings pass.