Testing strategy: unit, integration, contract and e2e
Why you care
Every backend job description says "strong testing practices," and every interview asks some version of "walk me through your test pyramid." The useful skill is not naming four layers on a whiteboard; it's knowing which layer would have caught last week's incident, and why it didn't. Call something a "unit test" when it actually opens a real socket, and you get a suite that is slow, flaky, and still cannot answer "is this API safe to deploy." Interviewers ask this because it is the fastest way to tell whether you have operated a CI pipeline or only written tests inside one repo in isolation.
Explain like I'm 5
Think of proving a bridge is safe before opening it. A unit test is checking one bolt in a vice: fast, and it tells you that bolt is fine, but nothing about the bridge. An integration test is bolting two real beams together and loading them: slower, and it tells you the joint holds. A contract test is checking that the blueprints of two independently built sections agree, without building either section. An end-to-end test is driving a real truck across the finished bridge: the most convincing signal, and the one you can afford to run the least often.
Core concept
Four boundaries, each answering a different question:
- Unit test: one function or class, every collaborator faked or absent. Answers "is this logic correct in isolation." Milliseconds. Tells you nothing about wiring.
- Integration test: real collaborators (a real database, a real queue, or a close-enough fake). Answers "do these pieces actually talk to each other correctly." Seconds, sometimes more.
- Contract test: checks the API or schema shape two independently deployed services agree on, without booting the other service. Answers "will my change break a consumer I don't control." Runs in each side's own pipeline.
- End-to-end (e2e) test: drives the real, deployed system through a real user flow. Answers "does the whole thing work," at the highest cost and the least stability.
The classic test pyramid (many unit, fewer integration, fewest e2e) is a bet about where most bugs live and how expensive feedback is at each layer, not a mandated ratio. Its critics point out real counter-cases: a service that is mostly thin glue between other services needs more integration tests than unit tests (sometimes drawn as a "honeycomb" or "trophy" shape), and a monorepo with many independently deployed consumers needs contract tests as a first-class layer, not an afterthought squeezed between integration and e2e.
The decision rule: pick the layer that answers the specific question you need answered, at the cheapest cost that still tells the truth. Use unit tests for logic with many input combinations. Use integration tests to prove your own wiring against a real dependency. Use contract tests at every boundary you do not fully control end to end. Reserve e2e for the handful of flows where "the whole system worked, for real" is the only acceptable evidence. Pytest fixtures, assertions and parametrize mechanics for actually writing any of these are covered separately in the Python testing chapter (start there for syntax); this card is about which boundary to test, not how to call assert.
Minimal example Same test count, very different CI time
UNIT_S, INTEGRATION_S, E2E_S = 0.02, 2.0, 90.0
pyramid = {"unit": 700, "integration": 150, "e2e": 10}
inverted = {"unit": 10, "integration": 150, "e2e": 700}
def suite_time(mix):
return mix["unit"] * UNIT_S + mix["integration"] * INTEGRATION_S + mix["e2e"] * E2E_S
for name, mix in [("pyramid-shaped", pyramid), ("inverted (e2e-heavy)", inverted)]:
total = suite_time(mix)
print(f"{name}: {sum(mix.values())} tests, {total:,.0f}s ({total / 60:.1f} min)")pyramid-shaped: 860 tests, 1,214s (20.2 min)
inverted (e2e-heavy): 860 tests, 63,300s (1055.0 min)In production Partition a suite by boundary, not by folder name
import unittest
def tag(*labels):
def decorator(fn):
fn.tags = labels
return fn
return decorator
class SettlementTests(unittest.TestCase):
@tag("unit")
def test_fee_calculation_is_pure(self):
self.assertEqual(round(1000 * 0.02), 20)
@tag("integration")
def test_insert_and_read_back_from_real_db(self):
import sqlite3
conn = sqlite3.connect(":memory:")
conn.execute("CREATE TABLE t (id INTEGER)")
conn.execute("INSERT INTO t VALUES (1)")
self.assertEqual(conn.execute("SELECT COUNT(*) FROM t").fetchone()[0], 1)
@tag("e2e")
def test_full_settlement_flow_through_the_api(self):
self.skipTest("would call a deployed staging environment")
def suite_with_tags(test_case_cls, wanted):
loader = unittest.TestLoader()
suite = unittest.TestSuite()
for name in loader.getTestCaseNames(test_case_cls):
method = getattr(test_case_cls, name)
if wanted & set(getattr(method, "tags", ())):
suite.addTest(test_case_cls(name))
return suite
fast_gate = suite_with_tags(SettlementTests, {"unit"})
full_suite = suite_with_tags(SettlementTests, {"unit", "integration", "e2e"})
print("fast pre-commit gate runs:", fast_gate.countTestCases(), "test(s)")
print("full nightly suite runs:", full_suite.countTestCases(), "test(s)")
result = unittest.TextTestRunner(verbosity=0).run(fast_gate)
print("fast gate failures:", len(result.failures) + len(result.errors))fast pre-commit gate runs: 1 test(s)
full nightly suite runs: 3 test(s)
fast gate failures: 0See it happen Speed and confidence trade off in opposite directions
- Unit tests check one function or class in isolation: fast, but they say nothing about how pieces talk to each other.
- Integration tests exercise real collaborators to prove the wiring works, at roughly 100 times the cost per test.
- Contract tests check the API shape both independently deployed sides agree on, without booting the other service, so they scale across many teams.
- End-to-end tests drive the real deployed system through a real flow: the most convincing signal, and the slowest and most brittle.
- Speed and confidence trade off in opposite directions as you move right; nothing here is both instant and fully convincing.
- A suite that is mostly e2e does not buy more confidence for the same bugs; it buys a much slower, flakier signal.
In your world
The situation
Your team's Jenkins job has one stage called "unit tests." Forty of its test methods do conn = psycopg2.connect(TEST_DSN) in setUp, hit a shared test database, and take about 3 seconds each once connection overhead is counted.
What actually breaks
The "unit" stage takes 10 minutes. Nobody runs it before pushing because 10 minutes feels like a coffee break, not a sanity check, so regressions in that code land on main and get caught only when the nightly build fails, a day later than they should have.
What you'd do on Monday
Reclassify by what each test touches, not by its folder name. Anything opening a real socket or connection is integration, not unit, and it moves to a stage that runs on merge, not on every save:
def touches_network_or_db(test_body_uses_connect):
return "integration" if test_body_uses_connect else "unit"
tests = [
("test_fee_calculation", False),
("test_insert_then_read_from_db", True),
("test_parse_settlement_row", False),
]
for name, uses_connect in tests:
print(name, "->", touches_network_or_db(uses_connect))test_fee_calculation -> unit
test_insert_then_read_from_db -> integration
test_parse_settlement_row -> unitImagine you work at Google
The numbers
Illustrative: a service with 5,000 fast tests at 20ms, 400 integration tests at 2s, and 30 e2e tests at 90s, run serially, costs 5,000×0.02 + 400×2 + 30×90 = 100 + 800 + 2,700 = 3,600s, about 60 minutes of feedback for a single commit.
What the Scenario-A answer does here
The Scenario A fix (classify by boundary) still holds, but "run the whole reclassified suite on every commit anyway" does not: 60 minutes of feedback per push, multiplied across hundreds of commits a day, either queues commits for hours or burns a proportional amount of CI compute just waiting.
What flips at this scale
At this scale you split the gate by timing, not just by boundary: a pre-submit gate runs only the sub-2-second layer (roughly 100s here) and blocks the merge; the integration and e2e layers run post-submit or nightly, sharded across workers, and a failure there pages an owner instead of blocking every other commit behind it.
The trade-off you'd defend
I would defend a fast, mandatory pre-submit gate plus a slower, parallel post-submit gate over one serial "everything, every time" pipeline, trading immediate blocking on integration/e2e regressions for a feedback loop developers will actually wait for.
Illustrative figures chosen to make the arithmetic visible; not Google-published numbers.
Gotchas
"Unit" tests that open a real connection
A stage named "unit tests" takes minutes, not seconds, and developers skip running it locally.
The tests touch a real database, queue or network call, which makes them integration tests wearing a unit-test label.
Reclassify by what the test body actually touches, and move real-collaborator tests to their own, less-frequent stage.
A contract change breaks a consumer nobody tested against
Every layer's tests pass, the deploy goes green, and a downstream service still starts 500ing.
No contract test exists between two independently deployed services, so a renamed or retyped field is invisible to both pipelines.
Add a provider/consumer contract test at that boundary instead of relying on integration or e2e coverage to notice.
A flaky e2e test gets re-run until it's green
The e2e stage is red about one run in ten, and the habit is to click "re-run" without reading why.
The layer meant to give the most confidence is instead training the team to distrust red builds.
Treat e2e flakiness as a defect to fix or quarantine with an owner, not noise to route around.
Tricky questions
Yes, unit tests are cheap, so write as many as possible.
A service that is mostly thin calls to other services has little pure logic to unit test; it needs more integration and contract coverage than unit tests, which is exactly the "honeycomb" shape critics of the strict pyramid point to.
Yes, it's an integration test with less setup.
A contract test can run without the other service ever running, against a recorded or shared schema. That is the point: it decouples two teams' pipelines, which an integration test — which needs the real collaborator up — cannot do.
Scenario question
You're handed a service where "tests" means one enormous e2e suite that clicks through a staging environment, takes 40 minutes, and fails about twice a week for reasons nobody has time to chase. Where do you start?
Model answer
I would not add more e2e tests. I would first sort the 40-minute suite's coverage by what it is actually proving, and for every check that is really "does this one function compute the right number," pull it down into a fast unit test. For every check that is "do our own two services agree on a call," I would add an integration test against a real (or disposable) dependency. Only the handful of checks that need the full, deployed stack to mean anything would stay e2e. The trade-off is upfront work to triage 40 minutes of tests, but the payoff is a pre-submit gate that runs in seconds and an e2e suite small enough that failures get investigated instead of re-run.
Interview corner
CommonExplain the test pyramid and when it doesn't apply.
TrickIs 90% unit test coverage on a thin API gateway a good sign?
Experience checkTell me about a test suite that was slow for a bad reason.
DesignHow would you structure CI stages for a growing monorepo?
Memory hook
A unit test is one bolt in a vice; an e2e test is driving the truck across the finished bridge.
Pick the layer that answers your actual question, at the cheapest cost that still tells the truth.
Self-check
What question does a contract test answer that an integration test cannot answer cheaply?
"Will my change break a consumer I don't control," without needing that consumer's service running at all.
Why is the test pyramid a bet, not a rule?
It assumes most bugs live in isolated logic and integration is expensive; a thin, glue-heavy service breaks that assumption.
In the production example, why does the fast gate run only one test?
Because only
test_fee_calculation_is_pureis tagged "unit"; the other two touch a real database or a deployed environment.
List the last 15 tests you wrote or touched. For each, name the boundary it actually crosses (unit, integration, contract, e2e) based on what it touches, not its file name. Report how many were mislabeled, and re-time your suite split into a "fast" group and everything else.