Every ETL script you have written lives inside a process you kick off with Jenkins; this chapter shows you how to expose the same logic over HTTP so a frontend, a mobile app, or another service can call it without waiting for a scheduled job. By the end you can build a typed FastAPI service, wire dependencies and authorisation, match async code to actual I/O, and paginate results without scanning the whole table.
Difficulty
Priority
Show
Priority
Difficulty
Level
py-12-01Card 1 of 6
Build routes around request and response contracts
The first thing a hiring manager asks when they look at a Python backend candidate's code is "where are the contracts?" A route that accepts a raw dict and returns another raw dict has no contract: it loads garbage, returns garbage, and silently strips fields that callers expected. FastAPI with Pydantic models enforces a contract on every request and response automatically, at the framework level, before your handler runs. Interviewers ask about request and response models because they predict whether you have shipped a breaking API change and blamed the client, or whether you caught it at the contract boundary before it left your service.
Explain like I'm 5
Imagine a bank teller window with a fixed deposit form. You fill in exactly the fields the form demands — no more, no less. If you leave "amount" blank, the form bounces back before the teller ever sees it. When the teller stamps your receipt, only the fields printed on the receipt template appear; whatever you scribbled in the margins is not on the receipt. The form is your OrderIn model; the receipt template is your OrderOut model.
Core concept
A request model is a Pydantic BaseModel that FastAPI uses to parse, validate and coerce the incoming JSON body. A response model is a BaseModel (or a type alias) that FastAPI uses to serialise the return value and to generate the OpenAPI schema. The two are deliberately separate because what you read from the client is not always what you expose back.
Key rules:
Annotate the body parameter with the model class: async def create_order(body: OrderIn).
Set response_model=OrderOut on the route decorator; FastAPI validates the return value against it.
Extra fields in the request body are silently ignored (by default). Extra fields on the return value are silently stripped before the client sees them.
Use Annotated[int, Field(gt=0)] for inline constraints rather than a @field_validator; it is clearer and composes with Depends.
Return the response status with status_code=201 on @app.post for creates; do not hard-code it in a JSONResponse.
Decision rule. Use a separate In and Out model whenever the stored or returned data contains fields you must not expose (passwords, internal IDs, audit timestamps). Use a single model only when input and output are truly identical, which is rare in production code.
Minimal example Contract bounces an invalid request before the handler runs
PythonDataclass stand-in for Pydantic (stdlib only)runs
from dataclasses import dataclass
@dataclassclassOrderIn:
item: str
quantity: int@dataclassclassOrderOut:
id: int
item: str
total_price: floatdefcreate_order(data: OrderIn) ->OrderOut:
if data.quantity <=0:
raiseValueError(f"quantity must be > 0, got {data.quantity}")
returnOrderOut(id=1, item=data.item, total_price=round(data.quantity *9.99, 2))
order =create_order(OrderIn(item="apple", quantity=3))
print(f"id={order.id} item={order.item} total={order.total_price}")
try:
create_order(OrderIn(item="apple", quantity=-1))
exceptValueErroras e:
print(f"ValueError: {e}")
Output
id=1 item=apple total=29.97
ValueError: quantity must be > 0, got -1
In production Multi-field contract with nested items (stdlib only)
PythonTwo-level contract: outer order + line itemsruns
from dataclasses import dataclass
from typing importList@dataclassclassLineItem:
sku: str
qty: int
unit_price: float@dataclassclassOrderIn:
customer_id: str
items: List[LineItem]
defvalidate(self) ->None:
ifnotself.customer_id:
raiseValueError("customer_id required")
ifnotself.items:
raiseValueError("order must have at least one item")
for item inself.items:
if item.qty <=0:
raiseValueError(f"qty must be > 0 for sku {item.sku}")
@dataclassclassOrderOut:
order_id: str
customer_id: str
total: float
item_count: intdefprocess_order(raw: dict) ->OrderOut:
items = [LineItem(**i) for i in raw.get("items", [])]
order =OrderIn(customer_id=raw.get("customer_id", ""), items=items)
order.validate()
total =round(sum(i.qty * i.unit_price for i in order.items), 2)
returnOrderOut(
order_id="ord-001",
customer_id=order.customer_id,
total=total,
item_count=len(items),
)
good = {
"customer_id": "c-42",
"items": [
{"sku": "SKU-A", "qty": 2, "unit_price": 149.5},
{"sku": "SKU-B", "qty": 1, "unit_price": 99.0},
],
}
result =process_order(good)
print(f"order_id={result.order_id} total={result.total} items={result.item_count}")
bad = {"customer_id": "c-42", "items": [{"sku": "SKU-A", "qty": -1, "unit_price": 149.5}]}
try:
process_order(bad)
exceptValueErroras e:
print(f"ValueError: {e}")
Output
order_id=ord-001 total=398.0 items=2
ValueError: qty must be > 0 for sku SKU-A
The real FastAPI equivalent of process_order is a route with body: OrderIn and response_model=OrderOut. FastAPI calls Pydantic's validation automatically before your handler runs; there is no validate() method to call.
PythonNeeds FastAPI + Pydantic installed
from typing importAnnotatedfrom fastapi importFastAPI, status
from pydantic importBaseModel, ConfigDict, Field
app =FastAPI()
classLineItem(BaseModel):
sku: str
qty: Annotated[int, Field(gt=0)]
unit_price: floatclassOrderIn(BaseModel):
customer_id: str
items: list[LineItem]
classOrderOut(BaseModel):
model_config =ConfigDict(from_attributes=True)
order_id: str
customer_id: str
total: float
item_count: int@app.post("/orders", response_model=OrderOut, status_code=status.HTTP_201_CREATED)
asyncdefcreate_order(body: OrderIn) ->OrderOut:
total =round(sum(i.qty * i.unit_price for i in body.items), 2)
returnOrderOut(
order_id="ord-001",
customer_id=body.customer_id,
total=total,
item_count=len(body.items),
)
Scenario A
In your world
The situation
Your Jenkins job reads orders from Postgres, validates them, and exposes a POST /orders endpoint. A colleague wrote the route to accept request.json() directly and return order.__dict__. Finance calls: why does the API return _sa_instance_state and hashed_password?
What actually breaks
The raw __dict__ of a SQLAlchemy model includes internal ORM attributes and every column, including ones you never intended to expose. Every response leaks 3 internal fields (_sa_instance_state, hashed_password, audit_created_at); Jenkins is green while a DAST scanner flags 3 critical data-exposure findings on the first run.
What you'd do on Monday
Define an OrderOut Pydantic model with only the fields callers need, add model_config = ConfigDict(from_attributes=True) so Pydantic can read ORM attributes directly, set response_model=OrderOut on the route, and return the ORM object directly — FastAPI calls OrderOut.model_validate(orm_obj) automatically. Without from_attributes=True, returning an ORM object raises ValidationError because Pydantic cannot read attributes from a non-dict by default.
Scenario B
Imagine you work at Zepto
40k orders/min
The numbers
Illustrative: 40,000 orders/min during the 10-minute peak delivery window. Each POST body ≈ 800 bytes → 40,000 × 800 B = 32,000,000 B = 32 MB/min of request body per pod. Pydantic V2's Rust-backed validator handles roughly 100k–500k validations/s on a modern CPU (benchmarks vary by model complexity; label this illustrative). At 40,000/min ≈ 667/s, validation is not the bottleneck — DB writes are.
What the Scenario-A answer does here
The Scenario A fix (define OrderIn and OrderOut) still holds at scale. The issue that emerges is schema versioning: at 40,000 orders/min you cannot deploy a breaking change to OrderOut (rename a field) without a coordinated cutover. Half the clients are still on the old schema when you push.
What flips at this scale
At this rate you need at least two API versions live simultaneously. response_model=OrderOut must remain stable across a deploy window; additive changes (new optional fields) are safe, but removing or renaming fields requires a versioned path (/v2/orders) with a deprecation window. Pydantic's model_config = {"extra": "ignore"} is your guard: clients sending extra fields do not break the server.
The trade-off you'd defend
I would defend strict response_model validation (no raw dict returns) over flexibility, trading slightly slower iteration for never leaking internal fields to external clients. The one exception is an internal admin API where the consumer is the same team.
Illustrative numbers chosen to make the arithmetic visible; not Zepto-published figures.
FastAPI vs Flask vs Django REST
Concern
FastAPI
Flask
Django REST Framework
Request validation
Pydantic, automatic before handler
Manual (marshmallow or hand-rolled)
Serializer classes
Response schema
response_model= on decorator
Manual jsonify
Serializer to_representation
OpenAPI docs
Auto-generated at /docs
Flask-RESTX or similar plugin
drf-spectacular plugin
Async native
Yes (async def or def)
Requires Quart or Async-Flask
Partial (ASGI via channels)
Learning curve
Low (Pydantic + type hints)
Low (minimal framework)
High (ORM-first opinions)
FastAPI wins for new greenfield services where you want automatic docs and types. Django REST makes sense when you already have a Django ORM project. Flask is a neutral base with no opinions.
Gotchas
Response model strips fields you expected to see
Symptom
The client receives a JSON object missing a field you added to the return dict.
Cause
response_model=OrderOut strips any key not declared in OrderOut. The return dict can have a hundred keys; only those in the model appear.
Fix
Add the field to OrderOut. If the field is optional, use field: str | None = None.
Missing response_model leaks internal columns
Symptom
The API response includes hashed_password, _sa_instance_state, or updated_at fields the client was never meant to see.
Cause
Without response_model, FastAPI returns whatever jsonable_encoder can serialise from your return value.
Fix
Always set response_model= on every production route. If no response is sent (204), use response_class=Response and status_code=204.
Pydantic V2 model_validate vs V1 parse_obj
Symptom
AttributeError: type object 'OrderIn' has no attribute 'parse_obj' after upgrading Pydantic.
Cause
Pydantic V2 renamed parse_obj to model_validate.
Fix
Replace OrderIn.parse_obj(data) with OrderIn.model_validate(data). The signature is identical.
Tricky questions
Does response_model validate the handler's return value?
Most people say
No, it only controls the OpenAPI docs schema.
Actually
Yes, FastAPI passes the return value through jsonable_encoder and validates it against the response_model. If a required field is missing, FastAPI raises a ResponseValidationError (500 in production). Turn on app = FastAPI(debug=True) locally to see the full message.
Can you reuse one Pydantic model for both request and response?
Most people say
Yes, it keeps the code DRY.
Actually
You can, but it usually breaks at the first iteration: your request model might have a password field you must never return, or your response adds an id that does not belong in the request. Separate models are the correct default.
Scenario question
You're handed a POST /settlements endpoint that accepts a raw dict, loads it into Postgres, and returns the ORM object's __dict__. Finance notices that the API response contains _sa_instance_state and internal_memo. How do you fix it without rewriting the handler logic?
Model answer
I'd define a SettlementOut Pydantic model containing only the fields finance should see, add model_config = ConfigDict(from_attributes=True) to it so Pydantic can read ORM attributes directly, set response_model=SettlementOut on the decorator, and return the ORM object — FastAPI calls SettlementOut.model_validate(orm_obj) automatically. Without from_attributes=True, returning an ORM object raises ValidationError because Pydantic treats it as a non-dict. The trade-off is that I now have to keep SettlementOut in sync with schema changes — a minor maintenance cost that is worth never leaking internal state.
Interview corner
CommonWhat is the difference between a request model and a response model in FastAPI?
“I'd say a request model parses and validates the incoming JSON, while a response model serialises and filters the outgoing JSON. They're typically separate because what you accept from a client usually contains fields you must never send back, like passwords or internal IDs.”
TrickIf you return a dict with extra keys from a route that has response_model=OrderOut, what happens?
“I'd expect FastAPI to silently strip the extra keys — they're excluded by the response model. The client never sees them, and no error is raised. That behaviour is why missing response_model is dangerous, not having one.”
Experience checkHow has strict response modelling saved you in a real project?
“In my ETL work I once returned a SQLAlchemy model's __dict__ from a route and accidentally exposed a hashed_password field during testing. After that I added response_model= on every route as a team rule; the CI linter checks for it now.”
CommonHow do you add a field constraint like quantity > 0 with Pydantic V2?
“I'd use Annotated[int, Field(gt=0)] as the type annotation on the field. Pydantic V2 evaluates it before the handler runs and returns a 422 with a clear message if the constraint fails.”
Memory hook
A bank teller window with a fixed deposit form: you fill in what the form demands, nothing more, and you get back only what the receipt prints. Your OrderIn is the form; your OrderOut is the receipt.
If you remember only one thing:
Always set response_model= on every route — it is the only thing standing between your ORM internals and your API clients.
Self-check
What HTTP status code should a POST /orders route return on success, and how do you set it?
201 Created. Set status_code=status.HTTP_201_CREATED on the @app.post decorator.
What happens to a request body field that is not declared in the Pydantic request model?
It is silently ignored by default. Pydantic V2's default model_config has extra = "ignore".
What error does FastAPI return when a required field is missing from the request body?
A 422 Unprocessable Entity with a JSON body listing the missing fields and their locations.
How do you convert a Pydantic V1 parse_obj call to V2?
Replace MyModel.parse_obj(data) with MyModel.model_validate(data).
Prove it
Write a SettlementIn model with amount: float (must be > 0) and currency: str (must be exactly 3 uppercase letters). Write a SettlementOut model with id: int, amount: float, currency: str, and status: str = "pending". Write a process function that validates a SettlementIn and returns a SettlementOut. Call it with a valid dict and two invalid dicts (negative amount, short currency). Paste the three outputs.
A route that embeds its own token-parsing and DB-session-opening code is untestable: you cannot swap out the real DB for a fixture without monkey-patching. FastAPI's dependency injection lets you declare what a route needs (a verified user, an open session, a config value) as a function signature, and FastAPI resolves the whole graph before your handler runs. Interviewers ask about dependency injection because it predicts whether you can write a handler that has a single responsibility, and whether you can test it without a running database.
Explain like I'm 5
A building has a reception desk. To enter the executive meeting room (your route handler), you first show your badge to reception (a dependency). Reception checks the badge, logs your name, and either waves you in or sends you back. You do not need to know how reception checks badges — the handler only knows that by the time it runs, someone has already verified the visitor is allowed. Swapping the badge reader for a test stub at the desk changes nothing about the meeting room itself.
Core concept
A dependency in FastAPI is any callable (function, class, or yield-based function) declared with Depends() in a route parameter. FastAPI builds a DAG of dependencies, evaluates them in order (leaves first), and passes the resolved values to your handler as keyword arguments.
Key rules:
Declare dependencies with Annotated: user: Annotated[User, Depends(get_current_user)].
Deduplication (use_cache=True default) applies within one request's dependency graph: if two sub-dependencies in the same request both declare Depends(get_db), FastAPI calls get_db once and shares the result. Two separate requests always produce independent calls.
A dependency that yields is a context manager: code before yield runs before the handler; code after yield runs after the response is sent.
FastAPI runs async def dependencies on the event loop and def dependencies in the thread pool, regardless of whether the route itself is async def or def.
Raise HTTPException inside a dependency to short-circuit the whole chain; the handler never runs.
Decision rule. Use Depends for any logic that belongs to more than one route: authentication, rate limiting, DB sessions, feature flags. Do not use it for logic that belongs to exactly one route — that is just a helper function.
Under the hood
FastAPI builds the dependency graph at import time (when the route is decorated) and evaluates it at request time. The graph is a directed acyclic graph: if require_admin depends on get_current_user, which depends on get_db, FastAPI calls get_db first, passes its result to get_current_user, and passes that result to require_admin. The handler receives the final resolved value. Dependencies that yield are wrapped in contextlib.asynccontextmanager or contextlib.contextmanager internally; FastAPI drives the teardown phase (after yield) after the response is fully sent to the client. If a dependency raises in its teardown, FastAPI logs it but cannot send another error response — the response is already gone.
Minimal example Dependency chain with plain functions (stdlib only)
{'order_id': 7, 'fetched_by': 1}
PermissionError: need 'admin', have 'reader'
PermissionError: unauthenticated
In production FastAPI route with Depends-wired authorisation
PythonNeeds FastAPI + Pydantic installed
from typing importAnnotatedfrom fastapi importDepends, FastAPI, HTTPException, status
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
security =HTTPBearer()
app =FastAPI()
defget_db() ->dict:
# In production: yield an AsyncSession from AsyncSessionLocal()return {"kind": "sqlite", "path": ":memory:"}
defget_current_user(
creds: Annotated[HTTPAuthorizationCredentials, Depends(security)],
db: Annotated[dict, Depends(get_db)],
) ->dict:
if creds.credentials !="valid-token":
raiseHTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="invalid token",
headers={"WWW-Authenticate": "Bearer"},
)
return {"id": 1, "role": "admin"}
defrequire_admin(
user: Annotated[dict, Depends(get_current_user)],
) ->dict:
if user["role"] !="admin":
raiseHTTPException(status_code=status.HTTP_403_FORBIDDEN)
return user
@app.get("/orders/{order_id}")
asyncdefget_order(
order_id: int,
user: Annotated[dict, Depends(require_admin)],
db: Annotated[dict, Depends(get_db)],
) ->dict:
# db is the same instance as the one passed to get_current_user —# FastAPI evaluates get_db once per request.return {"order_id": order_id, "fetched_by": user["id"]}
See it happen How FastAPI resolves the dependency graph
A request arrives; FastAPI reads the route's dependency list before calling the handler.
FastAPI starts from the leaves of the dependency graph — get_db has no sub-dependencies, so it runs first.
get_db returns (or yields) a session; FastAPI caches it for this request.
get_current_user depends on get_db; FastAPI injects the cached session automatically.
If the token is invalid, a 401 is raised here and the chain stops — the handler never runs.
require_admin depends on get_current_user; FastAPI injects the resolved user.
If the role is wrong, a 403 stops the chain here.
All dependencies resolved; the handler runs with clean, validated arguments.
Handler returns; FastAPI runs the teardown phase of any yield-based dependencies in reverse order.
Notice: the handler never calls get_db() or get_current_user() directly; it receives already-resolved values, which makes it fully testable with plain dicts.
Scenario A
In your world
The situation
Your route opens its own psycopg2 connection, parses the JWT by hand, and checks the role — all inside the handler function. A second route does the same. Three months later, the JWT secret rotates and you update the logic in only one of the two routes.
What actually breaks
Half the API enforces the new secret; half still accepts old tokens. Nobody notices until a security audit.
What you'd do on Monday
Extract get_current_user as a standalone function and declare it with Depends() on both routes. Now one change propagates everywhere, and you can test it in isolation by calling get_current_user("bad-token", db={}) in a unit test without spinning up a server.
Scenario B
Imagine you work at CRED
3M concurrent users at peak
The numbers
Illustrative: 3M concurrent users, each sending roughly 10 messages per minute → 3,000,000 × 10 / 60 = 500,000 auth checks per second. By Little's Law, with a 5 ms DB lookup per check: 500,000 checks/s × 0.005 s/check = 2,500 DB connections in flight at any instant. A typical production pool has 20–50 connections; 2,500 concurrent connections saturates it 50–125× over, making the DB the bottleneck, not the application.
What the Scenario-A answer does here
The Scenario A fix (centralise auth in a dependency) still prevents the duplicate-logic bug. What breaks at scale is the implementation: a DB lookup on every message at 500,000/s fills the pool immediately and queues every request behind a connection checkout.
What flips at this scale
At this rate, auth shifts from a DB lookup to an in-process JWT signature verification using a cached public key. The dependency's interface stays the same — it still returns a User dict or raises 401 — but the body now calls jwt.decode(token, public_key) instead of hitting a users table. Cache the public key at startup via lifespan; refresh it on a 1-hour timer.
The trade-off you'd defend
I would defend stateless JWT verification over session-based DB lookup, trading the ability to invalidate a token immediately for a constant-time auth check that never touches the DB pool. For high-value operations (password change, purchase confirmation), I would add a short-lived blacklist in Redis.
Illustrative numbers; not CRED-published figures.
Gotchas
Async session dependency used in a sync route
Symptom
MissingGreenlet: greenlet_spawn has not been called or a similar error when an AsyncSession is used inside a def (sync) route.
Cause
FastAPI runs async def dependencies on the event loop and def dependencies in the thread pool. An AsyncSession requires an event loop context that is not available in a thread-pool worker.
Fix
Use the synchronous Session (from sessionmaker) in def routes and AsyncSession (from async_sessionmaker) in async def routes. Keep session type and route type aligned.
Depends() bypassed in tests
Symptom
Your test calls the route but the dependency still opens a real DB connection, ignoring the test override.
Cause
You patched the dependency function but did not use app.dependency_overrides.
Fix
In tests, set app.dependency_overrides[get_db] = lambda: test_db_session before calling the test client. Remove the override after the test.
Teardown code after yield runs even on error
Symptom
An exception in the handler triggers a second unrelated error inside the dependency's teardown (the code after yield).
Cause
FastAPI always runs dependency teardown, whether the handler succeeded or failed.
Fix
Wrap the teardown in try/except so that a secondary error does not mask the primary one. This is the same rule as finally blocks in regular Python.
Tricky questions
Are dependencies re-evaluated on every request?
Most people say
No, they are evaluated once at startup.
Actually
They are evaluated once per request (not per handler call that uses them, but per request). Two routes depending on get_db in the same request share one get_db result. Two consecutive requests each get a fresh call.
Can a dependency depend on a path parameter?
Most people say
No, dependencies can only receive other dependencies or the request object.
Actually
A dependency can declare path parameters directly: def get_order_owner(order_id: int, db: Annotated[dict, Depends(get_db)]). FastAPI matches order_id from the path and injects it as a regular argument.
Scenario question
You're handed an API where every route repeats the same 15-line JWT verification block. A security team asks you to add IP allowlist checking to every authenticated endpoint in one day. How do you do it without touching every handler?
Model answer
I'd extract the JWT verification into a get_current_user dependency and add the IP allowlist check there. The change is in one function; every route that depends on it picks it up automatically. The trade-off is that the IP check now runs even for endpoints that used to be publicly accessible — I'd add a separate require_ip_allowlist dependency and apply it only to the authenticated routes, so the two concerns stay composable and independently testable.
Interview corner
CommonWhat is the difference between Depends and a regular function call inside a route handler?
“I'd say the key difference is that FastAPI evaluates Depends before your handler runs and caches the result per request, so two places in the same request that depend on get_db share one session. A direct call inside the handler would create a second session, skip the caching, and make the handler untestable without a real DB.”
TrickIf two routes both depend on get_db, how many times does get_db run per request?
“I'd say once per request, not twice. FastAPI caches the result of each unique Depends() instance within a request scope. The session is shared and torn down once, after the response is sent.”
Experience checkHow do you test a route that has a database dependency?
“I'd override the dependency using app.dependency_overrides[get_db] = lambda: my_test_session in the test setup, then call the test client normally. The route handler never knows it's getting a test session instead of a real one, which means the test is a real integration test without touching production data.”
DesignWhen would you not use Depends for auth?
“I'd skip Depends for a public read-only endpoint where auth is optional rather than required — adding it forces every caller to send a token. Instead, I'd use a single optional dependency that returns None for unauthenticated requests and a user dict otherwise, so the handler can branch on it.”
Memory hook
The reception desk checks your badge before you enter the meeting room; the meeting room never has to ask who you are. Your dependency is the desk, your handler is the room.
If you remember only one thing:
Declare what a route needs via Depends; never put auth or session logic inside the handler itself.
Self-check
What does app.dependency_overrides do in tests?
It replaces a dependency callable with a test substitute for the duration of the test, so the real DB is never touched.
What happens if you raise HTTPException inside a dependency?
FastAPI short-circuits the entire chain: the handler never runs, and the client receives the HTTP status and detail from the exception.
Why must a yield-based dependency's teardown code be wrapped in try/except?
Because FastAPI always runs teardown, even after a handler exception. A bare teardown can raise a secondary error that masks the original one.
What is the FastAPI V2 way to declare a dependency on a route parameter?
Use Annotated[Type, Depends(fn)] as the type annotation on the parameter, e.g., user: Annotated[User, Depends(get_current_user)].
Prove it
Write a get_rate_limiter dependency that maintains a per-client counter using a plain dict in memory. If the same client_id query parameter appears more than 3 times (within the test run), raise PermissionError("rate limit exceeded"). Call a handle_request(client_id, limiter) function (with limiter = get_rate_limiter()) four times with the same client_id and paste the output, showing the first three succeed and the fourth raises.
Every senior engineer who has debugged a FastAPI service under load has hit the same wall: the service is async def everywhere, yet a single slow database call freezes all other in-flight requests for 200 ms. The cause is always the same — blocking I/O called from inside an async function without await. Getting async right means understanding exactly when the event loop is released and when it is held. Interviewers ask about async endpoints because the answer predicts whether you will know the difference between a service that handles 200 req/s and one that handles 20 req/s.
Explain like I'm 5
A waiter in a busy restaurant. When you take a table's order and head to the kitchen, you hand the ticket to the cook and then walk to the next table rather than standing in the kitchen waiting. That walk to the next table is await asyncio.sleep(0) — you are voluntarily releasing your attention so someone else can be served. If instead you stand frozen in the kitchen until the food is ready (calling time.sleep), every other table waits for you to come back. The food arrives at the same speed either way; the difference is how many tables you can serve in parallel.
Core concept
In FastAPI, a route can be declared async def or def:
async def runs on the event loop thread. It must only await true coroutines (asyncio-native I/O, asyncio.sleep, asyncio.to_thread). Any call to time.sleep, a synchronous requests.get, or a blocking cursor in psycopg2 occupies the event loop for its entire duration — no other coroutine can run during that time.
def routes run in a thread-pool executor. FastAPI calls them with run_in_executor automatically, so a blocking time.sleep inside a def route does not freeze the event loop.
The correct rule: use async def when the route awaits true async I/O (httpx, asyncpg, aiofiles, asyncio.to_thread). Use def when the route calls a synchronous library and you have not wrapped it in asyncio.to_thread. Mixing async def with a blocking library is the worst of both: you get the complexity of async with the throughput of sync.
Decision rule. If all your I/O calls have await in front of them, use async def. If any I/O call is a blocking library with no async wrapper, either keep the route def (FastAPI will thread-pool it) or wrap the blocking call with asyncio.to_thread.
Minimal example Blocking vs async tasks under asyncio.gather
The event log shows the difference: with time.sleep, A runs to completion before B starts. With await asyncio.sleep, both start before either finishes.
In production asyncio.gather for concurrent I/O in a route
PythonFetching user and cart concurrently with asyncio.gatherruns
The real FastAPI version replaces asyncio.sleep(0) with actual await session.execute(stmt) calls using an async SQLAlchemy session. The structure — gather the independent queries, unpack, merge — stays identical.
Your new FastAPI service pulls a supplier catalogue from a legacy requests.get call, wrapped in an async def endpoint.
What actually breaks
Under load, all requests queue behind the one that is waiting for the HTTP response. A 500 ms catalogue fetch freezes the event loop for 500 ms; during that window, 500 other requests time out.
What you'd do on Monday
Either change the route to def (FastAPI will run it in a thread pool) or replace requests.get with httpx.AsyncClient and await client.get(...). The simplest Monday morning fix is def because it requires one word change and zero new dependencies.
Scenario B
Imagine you work at Ola
200k ride requests/min
The numbers
Illustrative: 200,000 requests/min ÷ 60 ≈ 3,333 req/s. Each ride request needs three independent I/O calls: GPS lookup (20 ms), driver availability check (20 ms), ETA calculation (20 ms). Serially: 3 × 20 ms = 60 ms per request; at 3,333 req/s, the service needs 3,333 × 0.060 = 200 concurrent workers. With asyncio.gather on all three: max(20, 20, 20) = 20 ms per request; the service needs 3,333 × 0.020 ≈ 67 concurrent workers — a 3× reduction in concurrency requirement.
What the Scenario-A answer does here
The Scenario A fix (change sync route to async) only helps if the underlying I/O calls are also async. Calling requests.get three times in async def with no await still runs them serially and still holds the event loop.
What flips at this scale
At 3,333 req/s, you cannot afford serial I/O per request. The design change is asyncio.gather for truly independent calls, combined with an async HTTP client (httpx) or async DB driver (asyncpg). The event loop model scales until you hit a CPU-bound calculation (route matching, fare computation); that part should run in asyncio.to_thread.
The trade-off you'd defend
I would defend asyncio.gather for independent I/O over sequential await calls, trading slightly more complex error handling (you must decide whether one gather failure cancels others) for a 3× latency improvement on routes with multiple I/O steps.
Illustrative numbers; not Ola-published figures.
async def vs def endpoints in FastAPI
Scenario
Use async def
Use def
All I/O is asyncio-native (asyncpg, httpx, aiofiles)
Yes
No
I/O is a blocking library (psycopg2, requests)
No (use asyncio.to_thread or switch to def)
Yes
CPU-bound logic only
No (blocks event loop)
Yes (thread pool isolates it)
Calling another async function with await
Required
Not possible
Gotchas
Blocking library call in async def freezes all requests
Symptom
Under concurrent load, all requests queue behind the slow one; the service appears single-threaded despite being async.
Cause
time.sleep, requests.get, or a synchronous DB driver called from async def holds the event loop for its entire duration.
Fix
Change the route to def (FastAPI auto-threads it) or wrap the blocking call with await asyncio.to_thread(fn, *args).
asyncio.run() inside a running event loop raises RuntimeError
Symptom
RuntimeError: This event loop is already running when calling asyncio.run() inside a FastAPI route or a Jupyter notebook.
Cause
asyncio.run() creates a new event loop; calling it inside an existing one is not allowed.
Fix
Inside an async context, await the coroutine directly. In Jupyter, use nest_asyncio.apply() as a workaround.
Making a CPU-bound route async def makes it slower
Symptom
Adding async def to a route that does heavy computation increases latency under load compared to def.
Cause
async def runs on the event loop thread; heavy computation blocks it for all other requests. A def route runs in a thread-pool, so the event loop stays free.
Fix
Keep CPU-bound routes as def, or use asyncio.to_thread to offload the computation explicitly.
Tricky questions
Does async def make an endpoint faster by default?
Most people say
Yes, async is always faster than sync in FastAPI.
Actually
async def is only faster than def when the endpoint actually awaits true async I/O. Without any await, an async def endpoint runs synchronously on the event loop thread and blocks other requests. A regular def at least gets threaded by FastAPI.
Can you mix await and time.sleep in the same async function?
Most people say
Yes, because await before time.sleep releases the loop first.
Actually
The await before time.sleep releases the event loop to run other coroutines up to that point. Once time.sleep starts, it blocks the loop again for its duration. The two are not additive — time.sleep always blocks, regardless of what await ran before it.
Scenario question
You're handed an async def FastAPI route that calls requests.get(legacy_api_url) for each incoming request. Under moderate load the service freezes periodically for 400–600 ms. Requests pile up and some time out. How do you diagnose and fix it?
Model answer
I'd immediately suspect requests.get is blocking the event loop. I'd add a quick timing log around the call to confirm the 400–600 ms stall comes from that call, not from our code. The fix has two options: change the route from async def to def so FastAPI runs it in a thread pool — that is the smallest change and lowest risk; or replace requests.get with await httpx.AsyncClient().get(...) so it genuinely yields. I'd prefer the def route for legacy endpoints and httpx for new ones. The trade-off is that def routes consume a thread per request, which caps out around a few hundred concurrent requests; httpx scales further but requires migrating the HTTP client.
Interview corner
CommonWhat is the difference between async def and def routes in FastAPI?
“I'd say an async def route runs on the event loop thread and must only call async-native I/O; a def route runs in a thread-pool executor, so blocking calls inside it don't freeze other requests. The wrong move is using async def with a blocking library — that's the worst of both.”
TrickIf you put time.sleep(1) inside an async def route, what happens to other concurrent requests?
“I'd expect them all to queue. time.sleep holds the event loop for one full second, so no other coroutine can run during that time. The server appears frozen to every other client.”
Experience checkWhen have you seen async used incorrectly in a service you worked on?
“In my ETL work I converted a route to async def hoping to improve throughput, but the route still called psycopg2 synchronously. Latency got worse because the event loop was now blocked and other requests were starving. The fix was to change it back to def while I migrated the DB driver to asyncpg.”
DesignWhen would you use asyncio.to_thread versus just switching to a def route?
“I'd use asyncio.to_thread when one step in an otherwise async function is a blocking call — say, reading a config file or calling a legacy SDK. Switching the whole route to def is simpler but trades the async benefits for every other part of the handler. asyncio.to_thread lets you keep the route async and isolate only the blocking part.”
Memory hook
A waiter who stands frozen in the kitchen until the food arrives serves one table at a time. time.sleep inside async def is that waiter — the event loop waits at the kitchen window while every other table goes unserved.
If you remember only one thing:
async def without await does not buy concurrency; it just blocks the event loop instead of a thread.
Self-check
What does FastAPI do with a def (sync) route under load?
It runs the route in a thread-pool executor, so the event loop is not blocked and other async requests can proceed concurrently.
How do you call a blocking function from inside an async def route without blocking the event loop?
await asyncio.to_thread(blocking_fn, *args) — it runs the function in a thread pool and awaits the result.
What is the observable symptom of time.sleep inside an async def route under concurrent load?
All concurrent requests queue behind the one that is sleeping; the service appears single-threaded and latency spikes for all clients.
Prove it
Write two async functions, slow_sync (calls time.sleep(0.05)) and slow_async (calls await asyncio.sleep(0.05)). Use asyncio.gather to run two copies of each and log the start/end events as in the minimal example. Show that slow_sync serialises and slow_async interleaves. Paste the two event logs.
A route handler has one job: produce a response. Every cross-cutting concern that should apply to all routes — logging request latency, attaching a correlation ID, catching unhandled exceptions before they leak stack traces to clients — belongs in middleware or an exception handler, not in the handler itself. Background tasks are a third layer: work that must happen after a response is sent, like firing a webhook or updating a cache, without making the client wait for it. Interviewers ask about middleware because the answer reveals whether you understand where code belongs architecturally, and whether you know the difference between work the client must wait for and work the client should not.
Explain like I'm 5
An airport security checkpoint. Before you reach your gate (the route handler), every passenger passes through the same checkpoint (middleware). The checkpoint does not care which flight you're on — it applies the same rules to everyone. If you fail the check, you're turned back before you ever reach the gate. Once you land (response is sent), ground crew (background tasks) refuels the plane and files paperwork — the passenger is already off.
Core concept
FastAPI (via Starlette) supports three patterns for cross-cutting behaviour:
Middleware wraps every request and response. It receives the raw Request, calls await call_next(request) to invoke the handler stack, and then post-processes the Response. Use it for: correlation IDs, timing, CORS, rate limiting, request logging.
Exception handlers (@app.exception_handler(ExcType)) run when a handler or dependency raises a specific exception type. Use them for: mapping domain exceptions to HTTP status codes, standardising error response shapes, preventing stack traces from reaching the client.
Background tasks run after the response is sent. Add them with background_tasks.add_task(fn, *args) inside a route handler. Async task functions are awaited on the event loop; plain def task functions run in the thread pool — both execute after the HTTP response bytes are flushed. Use them for: webhook calls, cache invalidation, audit log writes that do not affect the response.
Decision rule. Middleware for work that runs on every request before or after the handler. Exception handlers for turning known error types into clean HTTP responses. Background tasks for work the client must not wait for. If work must complete before the response, it is not a background task — it belongs in the handler.
Under the hood
Starlette's middleware stack is a chain of ASGI apps, each wrapping the next (@app.middleware("http") wraps your function in BaseHTTPMiddleware). Each app.add_middleware(MW) call inserts MW at the front of the internal list, so the last middleware added becomes the outermost layer — it runs first on the way in and last on the way out. When you call await call_next(request), you hand control to the next link in the chain, which eventually reaches the router and the handler. The Response you get back from call_next is a streaming response; reading its body in middleware consumes the stream. If you must inspect the body (for request/response logging), you must drain response.body_iterator into bytes yourself and return a new Response built from them, because the streaming response from call_next has no .body() to call. This has a performance cost on large responses. Starlette's @app.exception_handler uses the same chain: an unhandled exception propagates up through middleware until a handler catches it or it becomes a 500.
Minimal example Middleware chain with plain decorator wrappers (stdlib only)
PythonRequest flows through timing → auth → handler and backruns
The real FastAPI equivalent uses @app.middleware("http") for correlation_middleware and BackgroundTasks.add_task(audit_log, cid, path) injected into the route via Depends.
See it happen Middleware stack — request in, response out
The request arrives and enters the outermost middleware first.
Timing middleware records the start time and passes the request to auth middleware.
Auth middleware checks the token. Valid token: passes to handler.
Handler produces a 200 response and returns it to auth middleware.
The response unwinds through auth, then timing, which records elapsed time.
Bad token path: auth short-circuits with 401 before the handler runs.
Notice: middleware wraps the handler — the same code runs on the way in and the way out, which is why it is the right place for timing and for attaching response headers.
Scenario A
In your world
The situation
Your FastAPI service raises KeyError when a required config value is missing. The stack trace propagates to the client as a 500 with a JSON body containing the full Python traceback and the contents of os.environ.
What actually breaks
Every one of the 47 routes exposes the full Python traceback on any unhandled error; Jenkins is green while a DAST scan reports 47 critical data-exposure findings on its first run. DB connection strings and environment variable dumps are now in your client-facing JSON.
What you'd do on Monday
Add an @app.exception_handler(Exception) that catches any unhandled exception, logs the full traceback internally, and returns {"error": "internal server error"} with a 500 status. Specific exceptions like KeyError or ValueError get their own handlers that return a 400 with a safe message.
Scenario B
Imagine you work at Nykaa
5M daily sessions during a sale
The numbers
Illustrative: 5M sessions/day ÷ 86,400 s ≈ 58 req/s average. During a 2-hour peak sale window with 60% of traffic concentrated: 5M × 0.6 ÷ 7,200 ≈ 417 req/s peak. At 417 req/s, a correlation-ID middleware that generates a uuid.uuid4() per request adds ~417 UUID generations per second — negligible. A middleware that reads the full request body (for logging) and then re-wraps it adds ~1–5 ms per request; at 417 req/s that is 417–2,085 ms of added latency per second — measurable.
What the Scenario-A answer does here
The Scenario A fix (add exception handler so stack traces never reach clients) still holds at scale. What breaks is careless middleware: a middleware that awaits an external call (a logging service, a rate-limit counter in a slow Redis node) on every request adds that latency to every single request.
What flips at this scale
At 417 req/s, middleware must be sub-millisecond on the hot path. External calls (rate-limit checks, audit logs) move to background tasks or async fire-and-forget writes to a local queue (like a Redis stream). The middleware only attaches a correlation ID and records the start time; everything else is deferred.
The trade-off you'd defend
I would defend keeping middleware synchronous and under 1 ms over rich middleware that does external calls, trading full audit coverage in the request path for consistent sub-millisecond overhead. Fire-and-forget to a local queue gives near-full coverage with bounded latency impact.
Illustrative numbers; not Nykaa-published figures.
Gotchas
Background tasks cannot modify the response
Symptom
A background task updates a field you expected to see in the response, but the client receives the old value.
Cause
Background tasks run after the response bytes are flushed to the client. By then, the response object is immutable from the client's perspective.
Fix
Move any computation whose result the client needs into the handler, not a background task. Background tasks are for work the client should not wait for.
Middleware that calls await response.body() buffers the whole response
Symptom
Memory usage spikes on large file downloads or streaming endpoints; latency increases for all responses.
Cause
await response.body() reads and buffers the entire response body before passing it on.
Fix
Only read the body in middleware when you must (e.g., for HMAC validation). Otherwise, access only the headers or pass through without touching the body.
Exception handler swallows the traceback
Symptom
Production sees a 500 but the logs show no traceback; debugging is impossible.
Cause
The exception handler catches the exception but does not log it.
Fix
Always logger.exception("unhandled error", exc_info=True) inside the handler before returning the sanitised response. The client gets the safe message; your logs get the full traceback.
Tricky questions
Does middleware run for 404 routes (paths that do not exist)?
Most people say
No, middleware only runs when a route matches.
Actually
Middleware runs for every HTTP request, including 404s. FastAPI raises HTTPException(404) inside the router, which middleware sees on the way back out. If your middleware assumes a route matched, it will fail on 404s.
Do background tasks run if the handler raises an exception?
Most people say
Yes, anything already added with add_task still runs.
Actually
No, none run — not even ones added before the raise. FastAPI attaches the background task list to the response only when the endpoint returns normally. If the handler raises (including HTTPException), the exception handler builds an error response with no background tasks attached. If the side effect must happen, move it before the raise, or catch the error and return a normal response.
Scenario question
You're handed a service where every route manually generates a UUID request ID, logs request start/end, and catches exceptions with a try/except block. There are 40 routes. A teammate asks you to add response-time tracking. How do you do it without touching the 40 handlers?
Model answer
I'd add a single timing middleware that attaches a UUID to request.state.correlation_id, calls call_next, measures elapsed time, and sets it on the response header. One place, 40 routes covered, zero handler changes. The trade-off is that all 40 routes now pay the cost of UUID generation and header mutation, but those operations are sub-millisecond, so it is worth the uniformity. I'd also add a single @app.exception_handler(Exception) to centralise error formatting and remove the per-route try/except blocks.
Interview corner
CommonWhat is the difference between middleware and an exception handler in FastAPI?
“I'd say middleware wraps every request regardless of whether an exception occurs; it runs before the handler and after. An exception handler runs only when a specific exception escapes from the handler or a dependency. You use middleware for logging and headers; you use exception handlers for mapping domain errors to HTTP responses.”
TrickCan a background task modify the response before the client receives it?
“I'd say no — background tasks run after the response is already sent. If you need to change the response, the work must happen inside the handler, not in a background task.”
Experience checkWhen have you used background tasks in production?
“I've used them for webhook delivery — the client's payment result should not be delayed by our call to a third-party webhook URL, which might take 200 ms. The handler saves the event to the DB, schedules the webhook call as a background task, and returns 200 immediately. The task fires after the response is flushed.”
DesignHow do you add rate limiting to every route in a FastAPI app?
“I'd add a middleware that reads the client's IP or API key from the request headers, checks an in-process counter or Redis, and returns 429 if the limit is exceeded before calling call_next. I would not put rate limiting in each handler, because it would be duplicated and could be forgotten on new routes.”
Memory hook
An airport security checkpoint runs before every gate, not inside each gate. Middleware is the checkpoint; the handler is the gate.
If you remember only one thing:
Middleware belongs to every request; exception handlers belong to specific error types; background tasks belong after the response is sent.
Self-check
What does await call_next(request) do inside a middleware?
It passes the request to the next middleware in the stack (or the router if there are no more middleware layers) and awaits the response.
Why should background tasks not be used for work the client is waiting for?
Because background tasks run after the response is sent; the client has already received the reply and cannot see anything the task does.
What is the risk of an exception handler that does not log the exception?
The application swallows the traceback silently, making production debugging impossible.
Prove it
Write a function add_request_id_middleware(handler) that wraps a handler function, generates a UUID, attaches it to the request dict as request_id, calls the handler, and appends "x-request-id": <uuid> to the response. Call it with a handler that returns {"status": 200}. Show that the response contains the x-request-id header and that two consecutive calls produce different IDs.
Every Python backend eventually connects to a relational database. SQLAlchemy 2.x is the de-facto ORM, but it introduces a mental model — Unit of Work, identity map, session lifecycle — that is completely different from writing raw SQL. The most common production incidents involving SQLAlchemy are not query bugs; they are session lifecycle bugs: a shared session across concurrent tasks, a session that was never rolled back after a failed flush, or an Alembic migration that locked a 50-million-row table. Interviewers ask about SQLAlchemy sessions because the answer predicts whether you have shipped a silent data corruption bug or a 3 AM pager alert from a hung migration.
Explain like I'm 5
A bank transaction. You open a ledger (Session), write entries (add objects, update fields), and either seal it (commit) or shred your draft (rollback). While the ledger is open, every change you write exists only in your private draft — nothing is committed to the bank's permanent books yet. A flush writes your draft to a holding area the DB can see but has not confirmed; a commit confirms it. If the ledger catches fire mid-entry (an exception), you cannot just start writing on the next page in the same ledger — you must either shred the draft (rollback) and open a fresh ledger, or every subsequent write will be tainted by the failed draft.
Core concept
SQLAlchemy 2.x splits into two layers:
Core — SQL expression language (select, insert, update). Direct SQL, no ORM. ORM — DeclarativeBase, Mapped, mapped_column, Session, AsyncSession. Objects mapped to rows.
Key session rules:
One Session per request. Never share a Session between concurrent async tasks; a Session is not thread-safe or task-safe.
session.add(obj) marks an object for INSERT; it does not write to the DB yet.
session.flush() sends pending SQL to the DB within the current transaction. Rows are visible to the same session but not committed.
session.commit() commits the transaction. All flushed and pending changes become permanent.
After any exception that reaches the session, call session.rollback() before using the session again. SQLAlchemy marks a session as expired after a failed flush and will raise InvalidRequestError on subsequent use without a rollback.
Use scalars() in SQLAlchemy 2.x to get ORM objects from a select statement: (await session.scalars(select(Order))).all().
Decision rule. Use AsyncSession with async_sessionmaker in async FastAPI routes. Use synchronous Session in sync routes, scripts, and Alembic migrations. Never mix them.
Under the hood
SQLAlchemy's Unit of Work pattern is the mechanism behind session.add and session.commit. When you call session.flush(), SQLAlchemy's mapper computes the correct INSERT/UPDATE/DELETE order by topological-sorting the object graph (foreign-key dependencies first). The identity map inside the session is a dict from (mapper, pk) to object; session.get(Order, 42) returns the cached object if it has been loaded in this session, without a DB round-trip. That caching is the source of a subtle bug: two concurrent requests sharing one session can read stale data from each other's identity map. Alembic stores migration history in an alembic_version table; alembic upgrade head is idempotent but does not automatically ensure downtime-free deploys — adding a NOT NULL column without a default will fail on a live table unless you use a three-step migration (add nullable, backfill, add NOT NULL constraint).
Minimal example Session-per-request pattern with sqlite3 (stand-in for SQLAlchemy)
PythonContext manager guarantees commit-or-rollback on every requestruns
import sqlite3
from contextlib import contextmanager
@contextmanagerdefget_session():
"""One connection per request; commits on clean exit, rolls back on exception."""
con = sqlite3.connect(":memory:")
con.execute("CREATE TABLE orders (id INTEGER PRIMARY KEY, amount REAL)")
try:
yield con
con.commit()
exceptException:
con.rollback()
raisefinally:
con.close()
withget_session() as db:
db.execute("INSERT INTO orders VALUES (1, 250.0)")
db.execute("INSERT INTO orders VALUES (2, 90.0)")
try:
withget_session() as db:
db.execute("INSERT INTO orders VALUES (3, 400.0)")
raiseValueError("downstream validation failed")
exceptValueErroras e:
print(f"caught: {e}")
print("session closed and rolled back; next request gets a fresh one")
Output
caught: downstream validation failed
session closed and rolled back; next request gets a fresh one
In production AsyncSession via Depends in a FastAPI route
Note: lifespan (added in FastAPI 0.93) is the replacement for @app.on_event; current FastAPI marks on_event deprecated. Use lifespan in all new FastAPI code.
See it happen AsyncSession lifecycle per request
FastAPI calls get_session at the start of the dependency resolution for this request.
A new AsyncSession is created; no DB connection is taken from the pool yet.
The session object is yielded to the handler — the dependency pauses here.
FastAPI injects the session into the handler as a dependency argument.
The handler issues a query; the session acquires a connection from the pool and executes it.
The SQL is sent to the database.
Rows return to the session's identity map.
The handler finishes; control returns to the dependency's teardown code (after yield).
Clean exit: the transaction is committed and the connection is returned to the pool.
On exception: rollback fires before the session is closed, preventing a poisoned connection.
Notice: the session is opened and closed on every request — never shared between requests — so each request starts with a clean transaction state.
Scenario A
In your world
The situation
A shared AsyncSession is created at module level and reused across all requests. Two concurrent requests both add an order; the second request raises an integrity error.
What actually breaks
1 bad write poisons the shared session; 100% of subsequent requests fail with sqlalchemy.exc.InvalidRequestError: Can't reconnect until invalid transaction is rolled back on every DB operation — a total service outage until the process restarts. Jenkins stays green because the test suite uses a different session.
What you'd do on Monday
Create one session per request using a yield-based dependency with a try/except that calls await session.rollback() on any exception. The module-level session is the root cause; remove it entirely.
Scenario B
Imagine you work at Airbnb
1M search queries/hr
The numbers
Illustrative: 1M queries/hr ÷ 3,600 s ≈ 278 queries/s. Each query holds a session for roughly 100 ms (DB round-trip + Python overhead). Average concurrent sessions = 278 × 0.1 = 27.8 ≈ 28 sessions. A pool of 30 connections covers peak comfortably. If sessions are leaked (the dependency yields but teardown never fires because of a framework mis-configuration), pool size grows to 278 connections in 1 second and the connection pool exhausts.
What the Scenario-A answer does here
The Scenario A fix (session per request with rollback on error) prevents the corrupted session. What breaks at this query rate is a lazy-loaded relationship: if the Order ORM object has a listing relationship and accessing order.listing triggers a second DB query for every item in a 1,000-item page result, you get 1,000 extra queries per search (N+1 problem).
What flips at this scale
At 278 queries/s, N+1 queries turn 28 average concurrent sessions into 28,000. The design fix is selectinload or joinedload on the ORM query, loading all relationships in one additional query. With scalars() + selectinload, the session issues exactly 2 queries per page, not N+1.
The trade-off you'd defend
I would defend explicit selectinload on every relationship used in a list response over lazy loading, trading slightly more verbose query construction for predictable N+1-free DB load. The one exception is a detail endpoint that always loads exactly one object with all its relationships.
Illustrative numbers; not Airbnb-published figures.
Gotchas
Shared AsyncSession across concurrent tasks
Symptom
sqlalchemy.exc.InvalidRequestError: A transaction is already begun on this Session or corrupted data where one task sees another task's uncommitted changes.
Cause
AsyncSession is not safe to share across concurrent coroutines. Its transaction state is mutable and unsynchronised.
Fix
One session per request via a yield-based dependency. Never store a session in a module-level variable.
Failed flush leaves the session unusable without rollback
Symptom
After catching an IntegrityError, the next DB operation raises sqlalchemy.exc.InvalidRequestError: Can't reconnect until invalid transaction is rolled back.
Cause
A failed flush marks the session's transaction as rolled-back on the DB side, but the Python Session object does not know until you call rollback().
Fix
Always call await session.rollback() in the except block before attempting further use. The safest pattern is in the dependency's teardown: except Exception: await session.rollback(); raise.
Using on_event instead of lifespan
Symptom
DeprecationWarning: on_event is deprecated when running the application.
Cause
@app.on_event("startup") and @app.on_event("shutdown") are deprecated in current FastAPI; lifespan is the replacement.
Fix
Replace with a lifespan async context manager passed to FastAPI(lifespan=lifespan). The yield separates startup (before) from shutdown (after).
Tricky questions
Does session.add(obj) write to the database immediately?
Most people say
Yes, it inserts the row as soon as you call it.
Actually
session.add(obj) marks the object as pending in the Unit of Work. The INSERT happens at the next flush() or commit(). If you raise an exception before committing, nothing is written to the DB.
Is SQLAlchemy 2.x's Query API still valid?
Most people say
Yes, session.query(Order).filter(...) still works.
Actually
The legacy Query API still works in 2.x but is soft-deprecated. The 2.x way is select(Order).where(...) with session.scalars(stmt). New code should always use the 2.x style.
Scenario question
You're handed a FastAPI service that stores a global AsyncSession at module level, initialised at startup. Under load, the service starts returning 500s for all DB operations after any single failed request. How do you diagnose and fix it?
Model answer
I'd immediately suspect the shared session is in a failed transaction state. After one request triggers an IntegrityError or any other DB exception, the session is poisoned — subsequent requests keep using it and hit InvalidRequestError. I'd add a one-line fix: call await session.rollback() in a middleware exception handler. But the permanent fix is to remove the module-level session entirely and replace it with a yield-based dependency that creates and closes a session per request. The trade-off is slightly higher overhead (one connection checkout per request), but with a connection pool of 20–50 connections that cost is negligible.
Interview corner
CommonWhat is the difference between session.flush() and session.commit()?
“I'd say flush writes pending changes to the DB within the current transaction — the rows are visible to the same session but not to other connections. Commit ends the transaction and makes the changes permanent and visible to everyone. If you want to get the auto-generated ID of a new row without committing, you flush and then read the object's id attribute.”
TrickWhat happens if you share one AsyncSession across two concurrent async tasks?
“I'd say you get a race condition. Both tasks share the same transaction state and identity map; one task's flush can interfere with the other's, leading to corrupted data or an InvalidRequestError. Sessions are designed to be one-per-request or one-per-task.”
Experience checkHave you hit the N+1 query problem with SQLAlchemy?
“In my ETL work I once iterated over a list of orders and accessed order.supplier.name in a loop. SQLAlchemy lazy-loaded each supplier with a separate SELECT, turning a 1,000-row result into 1,001 queries. The fix was selectinload(Order.supplier) on the initial query, which loaded all suppliers in one extra query.”
DesignHow do you write a zero-downtime Alembic migration that adds a NOT NULL column?
“I'd use a three-step migration: first add the column as nullable with no default; second, deploy app code that writes the new column on every insert and update; third, after all existing rows are backfilled, run a second migration to add the NOT NULL constraint. Doing it in one step on a live table with millions of rows locks the table for the duration of the backfill.”
Memory hook
You open a bank ledger, write your entries, and either seal it or shred the draft. A poisoned ledger is one where the bank's system already rejected your last entry, but you kept writing on the same page instead of starting fresh.
If you remember only one thing:
One session per request; always rollback before reuse after any exception.
Self-check
What does session.scalars(select(Order)).all() return in SQLAlchemy 2.x?
A list of Order ORM objects. scalars() unwraps the row tuples to return the first (and only) column, which is the ORM object when select(Order) is used.
Why should you never share an AsyncSession between concurrent tasks?
Because AsyncSession is not task-safe: its transaction state and identity map are mutable and unsynchronised, so concurrent access causes data corruption and InvalidRequestError.
What replaces @app.on_event("startup") in current FastAPI?
An async context manager passed to FastAPI(lifespan=lifespan), where code before yield runs at startup and code after yield runs at shutdown.
What does expire_on_commit=False do in async_sessionmaker?
It prevents SQLAlchemy from expiring (clearing the cache of) all objects after a commit, so you can still access their attributes after committing without triggering a lazy load. Useful in async code where a lazy load would require an active session.
Prove it
Write a get_session context manager using sqlite3.connect(":memory:") that creates an orders table, yields the connection, commits on clean exit, and rolls back on exception. Call it twice: first with two valid inserts, then with a valid insert followed by raise ValueError("bad data"). After the second call, open a third session, count the rows, and show that only the first session's rows are there. Paste the output.
A route that returns all rows from a table works fine in development with 50 rows. It times out and OOMs in production with 10 million rows. Pagination is not optional — it is the contract between your API and every client. Offset pagination is the default that every tutorial shows and that breaks silently under load; keyset pagination is the version that actually scales. Interviewers ask about pagination because it signals whether you have shipped a "works on my machine, 50 rows" API into production and learned the lesson, or whether you know the lesson in advance.
Explain like I'm 5
A library catalog with a page marker. Returning all two million books at once would crash the catalog terminal. Instead, the librarian gives you 20 results and a bookmark (a cursor) to the last book on the page. Next time you come, you show the bookmark; the librarian picks up exactly where they left off. Showing up and saying "give me books number 50,001 to 50,020" (offset) makes the librarian silently count through 50,000 books before starting — every time.
Core concept
Two pagination strategies:
Offset pagination: SELECT ... LIMIT 20 OFFSET 50000. The DB scans from row 0, discards 50,000 rows, returns 20. Cost grows linearly with page number. Consistent only within a single transaction; concurrent inserts or deletes shift subsequent pages.
Keyset pagination (also called cursor pagination): SELECT ... WHERE id > 50020 ORDER BY id LIMIT 20. The DB uses the index on id to seek directly to the cursor and scans 20 rows. Cost is O(log n) for the seek regardless of page depth. Stable under concurrent inserts or deletes before the cursor.
Key rules:
The cursor column must be indexed, unique, and monotonically ordered. id (auto-increment) is the canonical choice.
The client sends after_id=<last_seen_id> as a query parameter; the first page uses after_id=0.
You cannot jump to "page N" with keyset; you can only move forward. For "next page" / "load more" UIs, keyset is always correct. For "jump to page 500" UIs, offset is required — but add an index and limit the maximum allowed offset.
Include a next_cursor field in the response so clients do not need to know the cursor scheme.
Project structure rule. Split routes into routers/ (FastAPI APIRouter instances), services/ (business logic functions), and repositories/ (DB query functions). A route calls a service; a service calls a repository. This keeps routes thin, services testable, and queries swappable.
Minimal example Keyset vs offset stability under deletion
PythonOffset skips a row after deletion; keyset stays stableruns
rows =list(range(1, 11)) # row ids 1..10
page1 = rows[0:3] # offset page 1: ids 1,2,3
cursor = page1[-1] # cursor = 3 (last seen id)print("page 1:", page1)
# id=3 is deleted between page 1 and page 2 fetches
rows_after_delete = [r for r in rows if r !=3]
p2_offset = rows_after_delete[3:6] # offset stays at 3
p2_keyset = [r for r in rows_after_delete if r > cursor][:3] # cursor stays at 3print("offset p2 (id 4 skipped):", p2_offset)
print("keyset p2 (stable): ", p2_keyset)
In production Keyset pagination with sqlite3 (stand-in for SQLAlchemy)
PythonKeyset vs offset with a live table and a mid-page deletionruns
import sqlite3
defmake_db() -> sqlite3.Connection:
con = sqlite3.connect(":memory:")
con.execute("CREATE TABLE listings (id INTEGER PRIMARY KEY, city TEXT)")
data = [(i, f"city_{i % 5}") for i inrange(1, 51)]
con.executemany("INSERT INTO listings VALUES (?,?)", data)
con.commit()
return con
defoffset_page(con: sqlite3.Connection, city: str, page: int, size: int=5) ->list[int]:
return [
r[0]
for r in con.execute(
"SELECT id FROM listings WHERE city=? ORDER BY id LIMIT ? OFFSET ?",
(city, size, (page -1) * size),
).fetchall()
]
defkeyset_page(
con: sqlite3.Connection, city: str, after_id: int, size: int=5
) ->list[int]:
return [
r[0]
for r in con.execute(
"SELECT id FROM listings WHERE city=? AND id>? ORDER BY id LIMIT ?",
(city, after_id, size),
).fetchall()
]
con =make_db()
p1 =offset_page(con, "city_1", page=1)
cursor = p1[-1]
print("page 1:", p1)
# id=6 is deleted (city_1 row) between page fetches
con.execute("DELETE FROM listings WHERE id=6")
con.commit()
p2_off =offset_page(con, "city_1", page=2)
p2_ks =keyset_page(con, "city_1", after_id=cursor)
print("offset p2 (id 26 skipped):", p2_off)
print("keyset p2 (stable): ", p2_ks)
The real FastAPI + SQLAlchemy version replaces sqlite3 with session.scalars(select(Listing).where(Listing.city == city, Listing.id > after_id).order_by(Listing.id).limit(size)).
Your GET /settlements route returns SELECT * FROM settlements with no limit. It works fine in development against a 200-row test table.
What actually breaks
In production, the settlements table has 8 million rows. The route loads all rows into memory (about 4 GB), times out after 30 seconds, and returns a 504 to the client.
What you'd do on Monday
Add after_id: int = 0 and limit: int = 100 query parameters. Change the query to WHERE id > after_id ORDER BY id LIMIT limit. Return next_cursor in the response so clients know whether more pages exist.
Scenario B
Imagine you work at Booking.com
15M search queries/hr
The numbers
Illustrative: 15M queries/hr ÷ 3,600 ≈ 4,167 queries/s. A user browsing to page 2,500 with page_size=20 triggers a query with OFFSET 50,000. The DB scans rows 0 through 50,019 and returns 20. Cost: 4,167 × 50,020 ≈ 208,000,000 row reads per second at that page depth — enough to saturate a DB cluster. With keyset pagination the same page is WHERE id > cursor ORDER BY id LIMIT 20: the index seek is O(log n), and each query reads exactly 20 rows. Cost: 4,167 × 20 ≈ 83,400 row reads per second — three orders of magnitude lower.
What the Scenario-A answer does here
The Scenario A fix (add pagination to a previously unbounded endpoint) stops the OOM. Offset pagination still works for shallow pages (1–10), where the scanned rows number in the hundreds, not thousands.
What flips at this scale
At 4,167 queries/s, deep offset pagination is a DB saturation attack from your own clients. Keyset pagination is the switch. The constraint it introduces is that clients cannot jump to "page 2,500" — they must follow next_cursor. For a travel search UI where users scroll through results, that is acceptable. For a bulk export that needs all rows, use a streaming query or a pre-generated export file.
The trade-off you'd defend
I would defend keyset pagination over offset for any table expected to exceed 100k rows, trading the ability to jump to arbitrary pages for O(log n) query cost and stable result sets. For admin tools that genuinely need "jump to page N", I would add a separate endpoint with explicit offset and a maximum depth of 200 pages.
Illustrative numbers; not Booking.com-published figures.
Offset vs keyset vs cursor token
Concern
Offset (OFFSET N)
Keyset (id > cursor)
Cursor token (opaque)
Deep-page cost
O(N) rows scanned
O(log N) index seek
O(log N)
Stable under inserts/deletes
No — rows can drift or repeat
Yes — cursor is an absolute position
Yes
Can jump to page N
Yes
No
No
Client complexity
Page number in URL
after_id in URL
Opaque token in URL
Requires indexed sort column
No
Yes
Yes
A cursor token is a keyset cursor encoded as a base64 string so the client cannot reverse-engineer it. Use it when you want to hide the ID from the URL or when the sort key is multi-column.
Gotchas
Keyset pagination requires an indexed sort column
Symptom
Keyset queries are slow despite the WHERE id > cursor clause.
Cause
id is not indexed (unusual, but possible with a composite or natural key table), or you are sorting by a non-indexed column like created_at.
Fix
Ensure the cursor column has a B-tree index. If sorting by created_at, add a composite index on (created_at, id) and use both columns in the keyset clause to handle ties.
Returning all rows when limit is missing
Symptom
GET /orders without a limit parameter loads 10M rows into memory.
Cause
The route has no default limit and the ORM query has no .limit().
Fix
Set a default and a maximum: limit: int = Query(default=20, le=200). The le=200 constraint rejects requests for more than 200 rows and returns a 422.
Offset drift on concurrent writes
Symptom
A client fetching pages 1, 2, 3 in sequence sees the same row on two consecutive pages, or skips a row entirely.
Cause
Rows are inserted or deleted between page fetches, shifting the offset window.
Fix
Switch to keyset pagination. If offset is required, wrap the pagination in a DB transaction with REPEATABLE READ isolation (PostgreSQL supports this; note it locks more rows for longer).
Tricky questions
Is keyset pagination always faster than offset?
Most people say
Yes, keyset is always faster.
Actually
Keyset is faster for deep pages (hundreds or thousands), where offset scans many rows to be discarded. For page 1 or page 2 on a small table, offset and keyset are equivalent in cost. Keyset also requires a unique indexed sort column, which offset does not.
Can you use keyset pagination with ORDER BY amount DESC?
Most people say
No, keyset only works with auto-increment IDs.
Actually
Keyset works with any indexed, orderable column. For ORDER BY amount DESC, the clause becomes WHERE amount < last_seen_amount AND id < last_seen_id (the second condition handles ties when two rows have the same amount). A composite index on (amount DESC, id DESC) is required for O(log n) seeks.
Scenario question
You're handed a GET /orders endpoint that returns 50k rows per call, each with a full order object. The frontend team says it is too slow. How do you fix it without changing the frontend?
Model answer
I'd add server-side pagination immediately: a default limit=20 with a maximum of limit=200, and a keyset cursor via after_id=0. For the frontend to work without changes, I'd check whether it is calling the API with no parameters — if so, I'd start by adding the default limit server-side, which fixes the immediate OOM without requiring a frontend deploy. Once that is live, I'd work with the frontend team to adopt the next_cursor response field for progressive loading. The trade-off is that adding a limit silently changes the API contract: clients expecting all rows will only see the first page. I'd version the endpoint (/v2/orders) rather than silently breaking /orders.
Interview corner
CommonWhat is the difference between offset pagination and keyset pagination?
“I'd say offset pagination uses OFFSET N and scans from row zero every time, so deep pages are expensive and rows can drift if the table changes between requests. Keyset uses a cursor — typically the last seen ID — and seeks directly to that position using the index, so cost is O(log n) regardless of page depth and results are stable.”
TrickWhy does offset pagination sometimes skip a row or return a duplicate?
“I'd say it happens because a row is inserted or deleted between two page fetches. If row 50 is deleted while a client is fetching page 3, offset skips what was row 51 — it is now row 50 and falls into the previous page. Keyset avoids this because the cursor is an absolute position, not a relative count.”
Experience checkHow have you structured a large FastAPI project?
“I'd separate routes into routers/, business logic into services/, and DB queries into repositories/. Routes only call services; services call repositories. That separation keeps routes thin, services unit-testable with mocked repositories, and makes it easy to swap the DB layer without touching business logic.”
DesignWhen would you still use offset pagination?
“I'd use it for admin tools where operators genuinely need to jump to page N, or for reports where the result set is bounded and small enough that deep-page cost is under 1 ms. I'd always add a maximum page depth to prevent accidental full-table scans.”
Memory hook
The library catalog gives you a bookmark, not a page number. A page number makes the librarian count from the beginning every time; a bookmark lets them open the book at the exact place you left off.
If you remember only one thing:
Use keyset pagination with after_id for any table you expect to grow beyond 100k rows.
Self-check
What does the next_cursor field in a keyset pagination response contain?
The ID (or other cursor key) of the last item on the current page. The client sends it as after_id on the next request.
Why does offset pagination produce duplicate or skipped rows under concurrent writes?
Because the OFFSET N clause counts rows from the start of the result set at query time; if rows are inserted or deleted between fetches, the count shifts.
What index does a keyset query WHERE id > 50 ORDER BY id LIMIT 20 require?
A B-tree index on id. Most databases create this automatically for the primary key.
Why is limit: int = Query(default=20, le=200) better than limit: int = 20?
Query(le=200) enforces a maximum; plain limit: int = 20 only provides a default. A client can send limit=1000000 and the query will return a million rows.
Prove it
Write two Python functions, offset_page(rows, page, size) and keyset_page(rows, after_id, size), operating on a list of integers 1..100. Fetch page 1 with both methods. Then delete id=3 from the list and fetch page 2 with both methods. Show that offset skips id=4 and keyset returns id=4. Paste the four output lines.