Shelf · Python · Chapter 8

Chapter 8: Concurrency & async

Concurrency in Python is three separate tools — threads, processes, and async — each solving a different problem; this chapter teaches you when to reach for each, how the event loop moves work forward, and why a blocking call inside an async function is silent sabotage. By the end you can design a concurrent service, cancel tasks cleanly, and defend why async would hurt a specific job rather than help it.

Difficulty
Priority
py-08-01Card 1 of 6

Runtime concurrency: threads, processes, async and GIL

HardP3 · Role-dependentTalk about it14 min
gilthreadsprocessesasyncconcurrencydecision

Why you care

The Python concurrency question comes up in almost every backend interview at a product company, and getting it wrong costs the offer: not because the interviewer expects you to have memorised the CPython source, but because the answer predicts whether you will reach for async on a CPU-bound job and wonder why it got slower. In a TCS ETL role you probably never had to choose; the script ran sequentially and that was fine. Product companies run 50k concurrent requests and need you to know why threads do not double CPU throughput, what the GIL prevents, and when async is the only sane choice.

Explain like I'm 5

Imagine a commercial kitchen with one stove (the GIL). Three chefs (threads) share the kitchen: only one can cook at a time. If a chef is waiting for the oven timer to ring — a network call, a disk read — they step away from the stove and another chef can use it. That is why threads help for I/O: the stove is free while you wait. CPU work is different: the chef using the stove just stands there, and the other chefs queue behind, so two chefs doing pure CPU work take longer than one — the queue adds overhead with no gain. Processes are separate restaurants, each with their own stove: truly parallel. Async is one chef so organised that she never stands idle: she starts a sauce, hands the pan off with "await", chops vegetables while waiting, and returns the moment the oven signals done.

Core concept

Three primitives, three problems:

  • Threads (threading, concurrent.futures.ThreadPoolExecutor) share heap memory and overlap I/O-bound work because the GIL is released during OS syscalls. They do not parallelise pure Python CPU work: only one thread holds the GIL and executes bytecode at a time.
  • Processes (multiprocessing, concurrent.futures.ProcessPoolExecutor) each have their own interpreter and their own GIL: true CPU parallelism. The cost is startup time (~50–100 ms per process, illustrative) and pickling every argument and result.
  • Async (asyncio) keeps everything on one thread. A coroutine yields at every await; the loop picks the next ready task. It is the most memory-efficient model for large numbers of concurrent I/O operations because there is no OS thread per connection. It gives you nothing for CPU-bound work.

Decision rule. Use threads when you have a few blocking calls and want overlap without library rewrites. Use processes when the work is CPU-bound pure Python. Use async when you have many concurrent I/O operations you can control end-to-end, and every dependency in the call chain has an async-compatible client. If you are unsure and the job is small, stay synchronous: a plain for loop is easier to debug than a stalled event loop.

Under the hood

The GIL is a mutex in CPython's interpreter that protects reference-count operations. CPython tracks how many names point to each object with a plain integer; two threads decrementing that counter simultaneously without a lock would free an object while another thread was using it. The GIL is held for the lifetime of a bytecode instruction and released before every OS syscall — file read, socket write, time.sleep — and reacquired on return. Every 5 ms (sys.getswitchinterval()) CPython signals other threads to compete; the running thread finishes its current bytecode and may yield. This signal-and-yield is what causes CPU-bound threads to be slower than serial code: extra context switches with no parallelism.

Python 3.13 added an experimental free-threaded build (a separate python3.13t binary; PYTHON_GIL=0 keeps the GIL off in it). Python 3.14 makes it officially supported via PEP 779, but it remains a separate build — the standard GIL interpreter is still the default and the one shipped by most platforms. Extension modules that access Python objects without their own internal locking automatically re-enable the GIL with a warning. Production use depends on your full dependency tree being updated.

Minimal example A coroutine object is not a running task

PythonWhat type you get when you call an async functionruns
import asyncio, warnings

async def upload(key):
    return f'uploaded {key}'

# Calling an async function returns a coroutine object; it does not run
coro = upload('orders.csv')
print('type:', type(coro).__name__)

# Forgetting await: Python warns at garbage-collection time
with warnings.catch_warnings(record=True) as w:
    warnings.simplefilter('always')
    dead = upload('settlements.csv')
    del dead
    import gc; gc.collect()
    print('warning:', w[0].category.__name__ if w else 'none')

# Run it properly
result = asyncio.run(upload('audit.csv'))
print('result:', result)
coro.close()
Output
type: coroutine
warning: RuntimeWarning
result: uploaded audit.csv

In production Parallel S3 downloads with a thread pool

A Jenkins ETL job downloads five daily settlement files from S3 and inserts them into Postgres. Replacing the sequential loop with ThreadPoolExecutor halves wall-clock time because the GIL is released during network I/O. sqlite3 stands in for Postgres; file reads simulate the S3 download.

PythonThreadPoolExecutor for parallel file loads (sqlite3 stands in for Postgres)runs
import sqlite3, csv, concurrent.futures
from pathlib import Path

for i in range(1, 6):
    Path(f'settlement_{i}.csv').write_text(
        'id,amount\n' + '\n'.join(f'{i*100+j},{j*10}' for j in range(1, 4))
    )

db = sqlite3.connect(':memory:')
db.execute('CREATE TABLE settlements (file TEXT, id INT, amount INT)')

def load_file(filename):
    with open(filename, newline='') as f:
        return [(filename, int(r['id']), int(r['amount'])) for r in csv.DictReader(f)]

files = [f'settlement_{i}.csv' for i in range(1, 6)]

with concurrent.futures.ThreadPoolExecutor(max_workers=5) as pool:
    all_batches = list(pool.map(load_file, files))

for batch in all_batches:
    db.executemany('INSERT INTO settlements VALUES (?,?,?)', batch)
db.commit()

count = db.execute('SELECT COUNT(*) FROM settlements').fetchone()[0]
print('rows loaded:', count)
print('files processed:', len(all_batches))
Output
rows loaded: 15
files processed: 5

See it happen Three concurrency models on one diagram

holdswaitsreleases on I/Oown GILsresumeawaitresumeawaitGILthread 1thread 2I/O wait(GIL free)process 1process 2event loopcoro Acoro B
  1. Two threads share one GIL: only one executes Python bytecode at a time.
  2. Thread 1 hits a network call, releases the GIL; thread 2 runs Python while thread 1 waits for I/O.
  3. Processes each have their own GIL and truly run in parallel on separate CPU cores.
  4. Async: one event loop thread. Coro A runs until it awaits, then hands control back to the loop.
  5. The loop immediately picks up coro B. Both coroutines make progress with zero OS threads beyond one.
Notice: async avoids the GIL entirely by staying single-threaded; it is not a third way to share one GIL.
Scenario A

In your world

The situation

Your Jenkins job downloads five daily pricing CSV files from S3 sequentially, parses each, and inserts into a Postgres staging table. Total runtime is 8 minutes, mostly network-wait during S3 reads.

What actually breaks

Finance waits 8 minutes every morning for the staging table. You try two parallel Jenkins jobs but hit a file-naming collision on the shared download path: zero rows load, build failure.

What you'd do on Monday

Replace the sequential loop with ThreadPoolExecutor. Each thread downloads and parses one file; the GIL is released during S3 reads so all five overlap, cutting runtime from 8 minutes to roughly 2 (illustrative). Stream directly into memory to avoid collision:

PythonParallel S3 fetch then bulk insert (needs boto3 + psycopg)
import concurrent.futures, boto3, csv, io, psycopg

def fetch_and_parse(key):
    body = boto3.client('s3').get_object(Bucket='pricing', Key=key)['Body'].read()
    return [row for row in csv.DictReader(io.StringIO(body.decode()))]

keys = ['cat_a.csv', 'cat_b.csv', 'cat_c.csv', 'cat_d.csv', 'cat_e.csv']
with concurrent.futures.ThreadPoolExecutor(max_workers=5) as pool:
    batches = list(pool.map(fetch_and_parse, keys))

with psycopg.connect(DSN) as conn:
    for rows in batches:
        conn.executemany(INSERT_SQL, rows)
Scenario B

Imagine you work at Zomato

50k concurrent API requests at lunch peak

The numbers

50,000 concurrent HTTP requests arrive at the 1–2 PM lunch window. Each request enters a Python handler that makes three downstream I/O calls (menu service, inventory, personalisation) averaging 100 ms each. Sequential per request: 3 × 100 ms = 300 ms. One OS thread per request: 50,000 threads × 64 KB minimum stack = 3.2 GB stack allocation. The OS scheduler managing 50,000 runnable threads adds 20–30% CPU overhead from context switches alone (illustrative), even when all threads are idle waiting for the same three downstream services.

What the Scenario-A answer does here

The Scenario A fix — switch from sequential to ThreadPoolExecutor — works well up to a few hundred concurrent requests. At 50k the thread count approaches the OS limit (typically 32k–64k on Linux without ulimit tuning), and the memory pressure from thread stacks turns a latency problem into an OOM crash.

What flips at this scale

At this scale, async is the design. One event loop handles all 50,000 handlers concurrently; each await yields control while the handler waits. Thread overhead: one OS thread. Memory per waiting handler: one task object (~few KB) instead of a 64 KB stack.

The trade-off you'd defend

I would defend async over threads for a high-concurrency request handler, trading the higher refactor cost and the need for async-compatible clients (httpx, asyncpg) for an order-of-magnitude reduction in memory footprint. For an internal tool handling ten concurrent requests, I would keep threads: simpler to test and no library rewrites.

Illustrative; 50k concurrent sessions and 64 KB stack sizes are representative of production-scale web services, not Zomato-published figures.

Threads vs Processes vs Async

AxisThreadsProcessesAsync
GILsharedeach has ownsingle thread, no contention
CPU-bound workno speedup (GIL)true parallelismno speedup
I/O-bound workoverlaps (GIL released)overlapsoptimal (many tasks, one thread)
Startup costlow (microseconds)high (~50–100 ms, illustrative)low (coroutine object)
Memory per unit~64 KB stack (OS)separate interpreter~few KB per task
Shared heapyesno (pickle boundary)yes

Gotchas

A blocking call inside an async function stalls the entire event loop

Symptom

An async handler takes 2 seconds per request; asyncio.get_event_loop().is_running() is True but no other coroutine makes progress during those 2 seconds.

Cause

Calling time.sleep(), a synchronous requests.get(), or any blocking OS call directly inside async def does not yield to the event loop. The loop is stuck until the call returns.

Fix

Replace with the async equivalent (asyncio.sleep, httpx.AsyncClient) or offload with await asyncio.to_thread(blocking_fn, arg) (Isolate blocking work with executors).

Threads do not speed up pure Python CPU work

Symptom

Adding a second thread to a CPU-bound loop makes total runtime longer, not shorter.

Cause

The GIL serialises bytecode execution; the second thread adds context-switch overhead with no parallel progress.

Fix

Use ProcessPoolExecutor for CPU-bound work, or switch to a C-extension that releases the GIL internally (NumPy operations, hashlib, zlib).

asyncio.get_event_loop() in a new thread raises RuntimeError

Symptom

Code that works in the main thread raises RuntimeError: There is no current event loop in thread 'ThreadPoolExecutor-0_0'.

Cause

Python 3.10+ no longer creates an event loop on demand in threads that did not start with asyncio.run.

Fix

Call asyncio.run(coro()) to create and run a new loop in that thread, or use asyncio.to_thread() from within an already-running loop (Isolate blocking work with executors).

Tricky questions

Do threads in Python truly run in parallel?
Most people say

Yes — that is what threading is for.

Actually

For pure Python code, no. The GIL means one thread runs bytecodes at a time. Threads do overlap for I/O waits and for work in C extensions that release the GIL. Whether you get parallelism depends on what the thread does, not just that it exists.

Is async always faster than threads for I/O work?
Most people say

Yes — async is the modern choice, so it must be better.

Actually

For a small number of concurrent operations the difference is negligible. Async wins at scale — thousands of connections — because it avoids OS thread-scheduling overhead and per-thread stack memory. For ten requests, a thread pool is as fast and simpler.

Scenario question

You're handed a Python microservice that makes three external API calls per incoming request using the synchronous requests library. It handles 200 requests per second and is hitting its CPU ceiling. Where is the bottleneck and what do you change?

Model answer

I'd profile first to confirm where the CPU goes. If the three API calls dominate latency, the service is I/O-bound, not CPU-bound — the "CPU ceiling" is thread context-switch overhead from each request holding a thread across three serial waits. I'd switch to httpx.AsyncClient with asyncio.gather to fan out the three calls in parallel per request (Choose gather or a task group), cutting latency from three serial times to one. If a call can't be made async, I'd isolate it with asyncio.to_thread (Isolate blocking work with executors), not rewrite everything.

Interview corner

CommonWhat is the GIL and why does it matter for concurrency?
“I'd explain it as a lock Python uses to protect reference-count bookkeeping — only one thread executes bytecodes at a time. In practice, threads overlap I/O-bound work fine but give you no speedup on CPU-bound loops. The trade-off is that Python's memory model is simple and thread-safe by default, which is why the GIL survived so long.”
TrickWhy is calling an async function without await a silent bug?
“I'd say the function returns a coroutine object, not its result. The body never runs. Python only warns you when the object is garbage-collected — so it's easy to ship code that silently does nothing and shows no exception.”
CommonWhen would you choose processes over threads?
“I'd choose processes when the work is CPU-bound pure Python — route optimisation, data transforms, image resizing — because each process has its own GIL and runs truly in parallel. The cost is higher startup time and the need to pickle everything across the boundary, so I'd keep task granularity coarse.”
DesignHow would you design a service handling 10k concurrent WebSocket connections?
“I'd use asyncio with a framework like FastAPI or websockets. Each connection is a coroutine, not a thread — 10k coroutines cost a few MB versus several hundred MB of OS stacks. Any blocking DB call gets offloaded via asyncio.to_thread (Isolate blocking work with executors).”

Memory hook

The kitchen image holds everything: one stove (GIL) means two chef-threads take turns, not doubling kitchen output. A second restaurant (process) is the only way to truly cook in parallel. Async is one very organised chef who never stands idle at the oven.

If you remember only one thing:

Threads overlap I/O; processes parallelise CPU; async handles many concurrent I/O tasks on a single thread — never confuse which tool is which.

Self-check

  1. Can two Python threads execute Python bytecode at the same time?

    No. The GIL allows only one thread to execute bytecodes at a time. They can overlap when one is blocked on I/O.

  2. What does calling an async function return?

    A coroutine object. The function body does not run until the coroutine is awaited inside a running event loop.

  3. What does asyncio.run(coro()) return?

    The coroutine's own return value; it creates a new event loop, runs it to completion, and closes it.

  4. If you call time.sleep(5) inside an async def, what happens to the event loop?

    The loop blocks for 5 seconds. All other tasks also stop progressing. Use await asyncio.sleep(5) instead.

  5. When does ProcessPoolExecutor outperform ThreadPoolExecutor?

    When the work is CPU-bound pure Python, where the GIL would prevent thread parallelism.

Prove it

Write a bench.py file with:

PythonCPU-bound thread vs serial — GIL effectruns
import concurrent.futures, time

def cpu_sum(n):
    return sum(i * i for i in range(n))

N, TASKS = 500_000, 4

t0 = time.perf_counter()
for _ in range(TASKS):
    cpu_sum(N)
serial_s = time.perf_counter() - t0

t0 = time.perf_counter()
with concurrent.futures.ThreadPoolExecutor(max_workers=TASKS) as pool:
    list(pool.map(cpu_sum, [N] * TASKS))
thread_s = time.perf_counter() - t0

print('threads slower than serial:', thread_s > serial_s)
Output
threads slower than serial: True

Record the ratio thread_s / serial_s. Then add a ProcessPoolExecutor version inside a if __name__ == '__main__': block in bench.py and run python bench.py. Record the process-pool ratio. Explain in one sentence why the process result differs.

py-08-02Card 2 of 6

Follow the event loop through await

MediumP3 · Role-dependentKnow cold12 min
event-loopawaitcoroutineschedulingcooperative

Why you care

Half the async bugs you will encounter in production come from one misunderstanding: await does not mean "run this now in parallel". It means "run this coroutine to the next suspension point and let the event loop decide what runs next". If you cannot trace exactly what the loop does between two await expressions, you will write async code that is secretly sequential, wonder why it is no faster than a for loop, or worse, write code that deadlocks when two coroutines each wait for the other to advance. Interviewers ask this to separate people who have used asyncio.run from people who understand what happens inside it.

Explain like I'm 5

Think of a restaurant ticket rail. The event loop is one waiter who clips new orders to the rail (schedules tasks) and then works through them one at a time. The chef calls out "waiting on the oven" (hits an await) and the waiter moves to the next ticket on the rail without standing idle at that station. When the oven signals ready, the waiter comes back to finish that order. Only one ticket is being actively worked at any moment, but all tickets are in flight — that is how one thread handles hundreds of concurrent "orders".

Core concept

Two concepts that beginners merge but must keep separate:

  • Creating a coroutine is just an object construction: calling async_fn() returns a coroutine object immediately and runs zero code from the function body. Nothing is scheduled. Nothing is waiting.
  • Scheduling a coroutine happens in two ways: await coro runs it inline, suspending the current coroutine until it finishes (sequential); asyncio.create_task(coro) registers it on the running event loop so it can make progress concurrently with the current coroutine.

The event loop's inner cycle: (1) pick any task whose I/O is ready or whose previous await has returned; (2) run that task until it hits an await; (3) register any new waitable produced by the await; (4) repeat. "Cooperative" means each task must explicitly yield via await; if it never does, no other task runs.

asyncio.run(coro) is the entry point from synchronous code: it creates a new event loop, runs coro as the root task, drives the loop until the root task is done, closes the loop, and returns the result. You get exactly one event loop per thread at a time.

Decision rule. Use await coro() when you want sequential execution and need the result before moving on. Use asyncio.create_task(coro()) (or asyncio.gather) when you want the coroutine to make progress concurrently with the current task. Use asyncio.run only at the top level — not inside an already-running loop.

Minimal example Sequential vs task-concurrent execution order

Pythoncreate_task interleaves; plain await does notruns
import asyncio

order = []

async def step(name):
    order.append(f'{name}:start')
    await asyncio.sleep(0)   # yield back to the loop
    order.append(f'{name}:done')

async def sequential():
    await step('A')   # A runs completely before B starts
    await step('B')

async def concurrent():
    t1 = asyncio.create_task(step('X'))   # X is scheduled
    t2 = asyncio.create_task(step('Y'))   # Y is scheduled
    await t1                              # run both, collect X's result
    await t2                              # collect Y's result

asyncio.run(sequential())
print('sequential:', order)

order.clear()
asyncio.run(concurrent())
print('concurrent:', order)
Output
sequential: ['A:start', 'A:done', 'B:start', 'B:done']
concurrent: ['X:start', 'Y:start', 'X:done', 'Y:done']

In production Concurrent cart price fetches across customer sessions

Three customer sessions each request prices for their cart items. Fetching all sessions concurrently with create_task means the event loop services all three carts while they wait for the pricing service to respond.

PythonConcurrent pricing requests for three shopping cartsruns
import asyncio

PRICES = {'apples': 45, 'bread': 28, 'milk': 52, 'rice': 65, 'oil': 120}

async def fetch_price(session_id, item):
    await asyncio.sleep(0)   # simulates async HTTP call to pricing service
    return {'session': session_id, 'item': item, 'price': PRICES.get(item, 0)}

async def get_cart_total(session_id, cart_items):
    tasks = [asyncio.create_task(fetch_price(session_id, item)) for item in cart_items]
    results = await asyncio.gather(*tasks)
    return session_id, sum(r['price'] for r in results)

async def main():
    sessions = [
        ('s001', ['apples', 'bread']),
        ('s002', ['milk', 'rice', 'oil']),
        ('s003', ['bread']),
    ]
    session_tasks = [asyncio.create_task(get_cart_total(sid, items)) for sid, items in sessions]
    for t in session_tasks:
        sid, total = await t
        print(f'{sid}: total={total}')

asyncio.run(main())
Output
s001: total=73
s002: total=237
s003: total=28

See it happen The event loop working through two tasks

event looptask Atask Bcreate_task(fetch_price('apples'))create_task(fetch_price('bread'))resumeawait asyncio.sleep(0)resumeawait asyncio.sleep(0)sleep done, resume
  1. Task A is registered on the loop. It does not run yet.
  2. Task B is also registered. Two tasks are now in the loop's ready queue.
  3. The loop picks task A and runs it until the first await.
  4. Task A suspends and hands control back. It is now waiting for the sleep to resolve.
  5. With A suspended the loop immediately runs task B until its first await.
  6. Task B also suspends. Both tasks are waiting; the loop checks what is ready next.
  7. Task A's sleep resolved first. The loop resumes A; it returns its result. Task B follows the same pattern.
Notice: both tasks start before either finishes — interleaving happens at every await, not just at task creation.
Scenario A

In your world

The situation

Your Jenkins pipeline calls a vendor REST API to validate each of 500 product IDs before inserting them into Postgres. The validation is a requests.get call, one per ID, and the job takes 12 minutes because the calls run sequentially.

What actually breaks

You rewrite the loop using async def and asyncio.run, but it is still 12 minutes. Profiling shows the requests are still sequential: you called requests.get(url) (synchronous) inside the coroutine without await instead of replacing it with an async HTTP client.

What you'd do on Monday

Replace requests.get with httpx.AsyncClient.get and fan out all 500 requests with asyncio.gather. If httpx is not available, wrap requests.get with asyncio.to_thread (Isolate blocking work with executors) to release the event loop while the blocking call runs:

PythonFan-out validation (needs httpx)
import asyncio, httpx

async def validate(client, product_id):
    r = await client.get(f'https://vendor/api/validate/{product_id}')
    return product_id, r.json()['valid']

async def main(product_ids):
    async with httpx.AsyncClient() as client:
        tasks = [asyncio.create_task(validate(client, pid)) for pid in product_ids]
        return await asyncio.gather(*tasks)
Scenario B

Imagine you work at Uber

1.25M driver location events per second

The numbers

5M active rides, driver app pings location every 4 seconds = 5,000,000 ÷ 4 = 1,250,000 events/second. One async location service process handles 10,000 events/second. Each event: read from Kafka + write to Redis = 2 I/O calls × 5 ms each = 10 ms of waiting. Sequential handling: 10,000 × 10 ms = 100 seconds to process one second's worth of events — impossibly behind. Async: 10,000 events all in-flight simultaneously; the loop wakes each as its I/O resolves; all 10,000 finish in roughly 10 ms (the max single I/O latency, not their sum).

What the Scenario-A answer does here

Understanding the sequential vs create_task distinction is still necessary. A naive async loop that awaits each event handler individually before starting the next handles exactly one event at a time and is no faster than synchronous code.

What flips at this scale

The design change is to schedule all 10,000 handlers as tasks before awaiting any of them. asyncio.gather or a TaskGroup does this in one expression. The flip point is: a single await coro() inside a loop is sequential; create_task(coro()) inside a loop followed by a single gather is concurrent.

The trade-off you'd defend

I would defend a task-per-event model over batching events into worker threads, trading slightly more complex cancellation handling for zero OS thread overhead at 10k concurrency. The one catch is that any blocking call inside a handler immediately serialises the whole loop (Runtime concurrency: threads, processes, async and GIL).

Illustrative numbers; Uber's reported scale and 4-second ping interval are public, but the exact per-service event rates and I/O latencies are not.

Gotchas

Calling asyncio.run inside a running event loop raises RuntimeError

Symptom

RuntimeError: This event loop is already running when you call asyncio.run(coro()) from inside an async def function.

Cause

asyncio.run creates a new event loop; you cannot nest running loops.

Fix

Use await coro() or asyncio.create_task(coro()) from inside an already-running loop. Reserve asyncio.run for the top-level entry point in synchronous code.

await coro() without create_task is secretly sequential

Symptom

Your async rewrite is no faster than the original for loop; there is no interleaving.

Cause

await coro() runs that coroutine to completion before the current coroutine resumes. If you do this inside a loop, every iteration waits for the previous one to finish.

Fix

Call asyncio.create_task(coro()) inside the loop, collect the tasks, then await asyncio.gather(*tasks) once after the loop.

A coroutine that is never awaited produces a RuntimeWarning at GC time

Symptom

RuntimeWarning: coroutine 'fn' was never awaited appears in logs with no obvious traceback.

Cause

You called async_fn() but neither awaited the result nor passed it to create_task. The coroutine was collected by the garbage collector without running.

Fix

Always either await or create_task any coroutine you create. Enable -W error::RuntimeWarning in tests to catch this immediately.

Tricky questions

Does await asyncio.sleep(0) actually sleep?
Most people say

Yes, it pauses for a very short time.

Actually

It yields control to the event loop for exactly one iteration and then resumes. The purpose is to give other ready tasks a chance to run. The actual time elapsed is near zero (the loop scheduling overhead). It is the standard way to write a coroutine that is cooperative without doing real I/O.

Does asyncio.create_task(coro()) start the coroutine immediately?
Most people say

Yes, the coroutine starts as soon as you call create_task.

Actually

The coroutine is scheduled: it is placed in the loop's ready queue. It will not start until the currently running coroutine yields at the next await. You will see this in the minimal: both create_task calls happen before either task prints "start".

Scenario question

You're handed an async function that loops over 200 URLs and fetches each with await session.get(url) inside the loop body. It takes 40 seconds. The senior engineer says "just make it concurrent." What do you change?

Model answer

I'd change the loop to collect tasks first, then await them all together. Instead of await session.get(url) inside the loop — which is sequential — I'd do tasks = [asyncio.create_task(fetch(session, url)) for url in urls] and then results = await asyncio.gather(*tasks). The trade-off is that all 200 requests are now in-flight simultaneously, which is exactly what we want for I/O-bound HTTP work, but it also means 200 open connections at once. If the server has a rate limit or a connection cap, I'd use a asyncio.Semaphore to cap concurrency at a safe number like 20.

Interview corner

CommonWhat is the difference between awaiting a coroutine and creating a task?
“I'd say awaiting a coroutine runs it inline — the current coroutine blocks until it finishes. Creating a task schedules it on the event loop so it can make progress concurrently with the code that created it. That distinction is the entire difference between sequential and concurrent async.”
TrickCan you have more than one event loop running in the same thread?
“I'd say no — at most one event loop can run per thread at a time. asyncio.run enforces this: it raises RuntimeError if a loop is already running in that thread. That is also why you cannot call asyncio.run from inside an async def.”
Experience checkWhy is await asyncio.sleep(0) sometimes necessary in a tight async loop?
“I'd use it when a coroutine has CPU work to do and no natural I/O awaits in its body. Without at least one await, the coroutine holds the loop indefinitely, blocking all other tasks. sleep(0) yields for one loop iteration with near-zero wall-clock cost. In practice, if I'm doing CPU work I'd question whether async is the right tool at all (Explain when async is the wrong answer).”

Memory hook

The ticket rail image: the event loop is one waiter who clips orders to the rail (schedules tasks) and works through them one at a time. The chef calling "waiting on the oven" (await) tells the waiter to move on; when the oven signals ready, the waiter comes back.

If you remember only one thing:

await coro() is sequential; create_task(coro()) followed by gather is concurrent — one character of difference, an order of magnitude of performance difference.

Self-check

  1. What does calling async_fn() return?

    A coroutine object. The function body does not execute.

  2. What is the difference between await coro() and asyncio.create_task(coro())?

    await is sequential — the current task blocks until the coroutine finishes. create_task schedules the coroutine concurrently so both can make progress.

  3. What does asyncio.run(main()) do if called from inside a running event loop?

    It raises RuntimeError: This event loop is already running.

  4. Why might an async rewrite of a sequential loop have the same runtime as the original?

    If await coro() is used inside the loop, execution is still sequential — each iteration waits for the previous one before starting the next.

  5. What does await asyncio.sleep(0) do?

    It yields to the event loop for one iteration without waiting any real time, allowing other tasks to make progress.

Prove it

Start with the sequential cart pricing example from this card (three sessions, each with five items, asyncio.sleep(0.1) per item fetch). Measure runtime with time.perf_counter. Then rewrite to use asyncio.create_task plus asyncio.gather. Record the ratio. The concurrent version should be roughly (items_per_cart / items_per_session) × faster. Paste the two timing lines as your artefact and explain why the ratio is not infinite.

py-08-03Card 3 of 6

Choose gather or a task group

MediumP3 · Role-dependentKnow cold12 min
gathertaskgroupfan-outexception-groupconcurrent-tasks

Why you care

You will write fan-out patterns in almost every async service: fetch three upstream APIs for one request, validate a batch of records in parallel, or run N settlement checks before committing. Choosing between asyncio.gather and asyncio.TaskGroup determines whether a single failure tears down all sibling tasks or silently drops their results — and getting it wrong produces either ghost tasks consuming memory or partial results treated as complete. Interviewers ask this to test whether you understand structured concurrency and exception groups, both of which were added to make async code safer and are now the preferred pattern.

Explain like I'm 5

Two styles of team lead. The first (gather) says: "Here are three jobs. Call me when they're all done; if one fails, I'll take the exception but the others might or might not finish depending on timing." The second (TaskGroup) says: "I own these jobs. If one fails, I immediately cancel the rest and hand you one bundled report of everything that went wrong." For small, independent tasks where you want results in order: gather. For a group that lives or dies together: TaskGroup.

Core concept

asyncio.gather(*coros_or_tasks) runs all awaitables concurrently and returns results in input order.

  • By default, the first exception propagates immediately; the other tasks are not automatically cancelled (they keep running; their results are just discarded).
  • With return_exceptions=True, every task runs to completion; exceptions are returned as values in the result list — you inspect them yourself.
  • gather predates structured concurrency and does not enforce task ownership.

asyncio.TaskGroup (3.11+) is the structured alternative:

  • Tasks created inside async with TaskGroup() as tg: tg.create_task(coro()) are owned by the group.
  • If any task raises an exception, the group immediately cancels all siblings and then raises an ExceptionGroup containing all exceptions collected during cleanup.
  • Use except* ExcType as eg: to unwrap the ExceptionGroup.
  • The group's __aexit__ does not return until every child task is done (cancelled or completed), so there are no leaked tasks.

Decision rule. Use gather(return_exceptions=True) when you want all results regardless of partial failures and will inspect each. Use gather (default) for fire-and-forget fan-outs where one failure should abort the batch. Use TaskGroup whenever tasks share a lifetime — if one should not outlive the others, TaskGroup enforces that contract. Prefer TaskGroup in new code; it was designed to replace gather's ambiguous cancellation behaviour.

Minimal example gather vs TaskGroup on a failing task

Pythongather and TaskGroup handle exceptions differentlyruns
import asyncio

async def ok(n):
    await asyncio.sleep(0)
    return n

async def fail():
    await asyncio.sleep(0)
    raise ValueError('bad data')

async def main():
    # gather raises the first exception; results from ok() are discarded
    try:
        await asyncio.gather(ok(1), fail(), ok(3))
    except ValueError as e:
        print('gather exception:', e)

    # return_exceptions=True: all tasks run; exceptions become values
    results = await asyncio.gather(ok(1), fail(), ok(3), return_exceptions=True)
    print('return_exceptions types:', [type(r).__name__ for r in results])

    # TaskGroup: siblings are cancelled; raises ExceptionGroup
    try:
        async with asyncio.TaskGroup() as tg:
            t1 = tg.create_task(ok(10))
            tg.create_task(fail())
            t3 = tg.create_task(ok(30))
    except* ValueError as eg:
        print('TaskGroup ExceptionGroup:', [str(e) for e in eg.exceptions])
        print('t1.cancelled:', t1.cancelled(), 't1.done:', t1.done())

asyncio.run(main())
Output
gather exception: bad data
return_exceptions types: ['int', 'ValueError', 'int']
TaskGroup ExceptionGroup: ['bad data']
t1.cancelled: False t1.done: True

In production Payment verification fan-out at PhonePe

Each payment must clear three independent checks before it is settled. Using gather fans them out in parallel, cutting per-transaction latency from the sum of three call times to the maximum of any one.

PythonThree-way payment verification with gatherruns
import asyncio

async def bank_auth(txn_id):
    await asyncio.sleep(0)   # simulates ~80 ms bank round-trip
    return {'service': 'bank', 'txn': txn_id, 'approved': True}

async def fraud_check(txn_id):
    await asyncio.sleep(0)   # simulates ~50 ms
    return {'service': 'fraud', 'txn': txn_id, 'risk': 'low'}

async def ledger_post(txn_id):
    await asyncio.sleep(0)   # simulates ~30 ms
    return {'service': 'ledger', 'txn': txn_id, 'posted': True}

async def process_payment(txn_id):
    bank, fraud, ledger = await asyncio.gather(
        bank_auth(txn_id),
        fraud_check(txn_id),
        ledger_post(txn_id),
    )
    settled = bank['approved'] and fraud['risk'] == 'low' and ledger['posted']
    return {'txn': txn_id, 'status': 'settled' if settled else 'failed'}

async def main():
    results = await asyncio.gather(*[process_payment(t) for t in ['T001', 'T002', 'T003']])
    for r in results:
        print(r['txn'], r['status'])

asyncio.run(main())
Output
T001 settled
T002 settled
T003 settled
Scenario A

In your world

The situation

Your Jenkins job hits three internal REST endpoints for each of 200 product records to build a consolidated export: pricing service, inventory service, and supplier catalogue. They are called sequentially inside a for loop. The job takes 18 minutes.

What actually breaks

You add asyncio.gather inside the loop but you still see 18 minutes. The loop calls await process_record(id) sequentially — each call hits the three services in parallel internally, but the 200 loop iterations still run one at a time.

What you'd do on Monday

Move the fan-out outside the loop. Collect tasks for all 200 records, then gather once:

PythonFan-out all 200 records in one gather (needs httpx)
import asyncio, httpx

async def fetch_record(client, product_id):
    price, inv, supp = await asyncio.gather(
        client.get(f'{PRICING_URL}/{product_id}'),
        client.get(f'{INVENTORY_URL}/{product_id}'),
        client.get(f'{SUPPLIER_URL}/{product_id}'),
    )
    return build_row(product_id, price.json(), inv.json(), supp.json())

async def main(product_ids):
    async with httpx.AsyncClient() as client:
        tasks = [asyncio.create_task(fetch_record(client, pid)) for pid in product_ids]
        return await asyncio.gather(*tasks)
Scenario B

Imagine you work at PhonePe

1,000 transactions per second at peak

The numbers

1,000 transactions/second at peak. Each transaction fans out to 3 services: bank auth 80 ms + fraud 50 ms + ledger 30 ms. Sequential per transaction: 80 + 50 + 30 = 160 ms. With gather (parallel): max(80, 50, 30) = 80 ms. At 1,000 tx/sec: sequential total handling time = 1,000 × 160 ms = 160 s per second of transactions — 160× behind. With gather: 1,000 × 80 ms = 80 s per second — still 80× behind if handled serially. The full fix is concurrent transactions too: gather(*[process_payment(t) for t in batch]) collapses 1,000 × 80 ms to ~80 ms for the batch.

What the Scenario-A answer does here

The Scenario A fix (gather inside each process_payment) halves per-transaction latency, which helps. At 1,000 tx/sec the bottleneck shifts: if you still process transactions sequentially in a loop, you are now limited by gather's 80 ms floor × 1,000 = 80 s/s. You need concurrency at the transaction level too.

What flips at this scale

The design that scales is: gather the three service calls inside each transaction (per-transaction fan-out) AND run multiple transactions concurrently (cross-transaction fan-out via gather or TaskGroup over the batch). At this point the latency floor is set by the slowest single I/O call, not its count.

The trade-off you'd defend

I would defend TaskGroup over gather for the transaction-level fan-out once the team is comfortable with except* syntax. If one service call returns an internal error, TaskGroup immediately cancels the other two and raises a clean ExceptionGroup, preventing a partial settlement. gather with return_exceptions=True requires explicit per-result error inspection that is easy to forget.

Illustrative; PhonePe's public transaction volumes are cited in the billions per day, but per-service latencies and exact peak-second rates are not published.

gather vs TaskGroup

Axisasyncio.gatherasyncio.TaskGroup
Sibling cancellation on failureNo (default); yes with careful manual cancelYes — always
Exception type raisedFirst exception as-isExceptionGroup (requires except*)
return_exceptions optionYesNo
Task ownership / structured scopeNoYes — scope ends only when all tasks are done
Result accessOrdered return listVia task.result() on the task objects
Minimum Python version3.4+3.11+

Gotchas

gather does not cancel sibling tasks when one fails

Symptom

One task in a gather raises an exception, you catch it, but the remaining tasks keep running in the background and you see unexpected side effects or memory never freed.

Cause

gather by default does not cancel siblings; they run to completion even after the caller caught the first exception.

Fix

Pass return_exceptions=True and inspect each result, or switch to TaskGroup which guarantees cancellation and cleanup.

except ExceptionGroup catches the group but does not route its contents

Symptom

You catch except ExceptionGroup as eg: and get the group object, but you have to inspect eg.exceptions manually; trying except ValueError: below it never fires.

Cause

except ExceptionGroup: catches the group object itself (ExceptionGroup subclasses Exception) but does not split it apart. except* ExcType (3.11+) is what routes each contained exception to the matching handler by type.

Fix

Use except* ValueError as eg: to handle each contained ValueError separately. Reserve except ExceptionGroup: only when you want the whole bundle as a single value to log or re-raise.

gather results are in input order, not completion order

Symptom

You assume result[0] is the fastest task; it is not — it is the first task you passed.

Cause

gather preserves the order of its input arguments, regardless of which task finishes first.

Fix

Use gather if you need ordered results. If you need results in completion order, use asyncio.as_completed.

Tricky questions

Does asyncio.gather cancel siblings when one task raises?
Most people say

Yes — that is why it is safe to use for fan-out.

Actually

No, by default gather does not cancel siblings. The first exception propagates to the caller, but the remaining tasks continue running. Only TaskGroup guarantees cancellation of all siblings.

Can you mix coroutines and already-running tasks in gather?
Most people say

No — gather only accepts coroutines.

Actually

Yes. gather accepts coroutines, Task objects, and Future objects. It wraps bare coroutines in tasks automatically. The results list is in the order you passed them.

Scenario question

You're handed an async service that uses asyncio.gather to fan out to five upstream APIs. One API is flaky and occasionally returns a 500. After the fix, the caller sometimes gets partial results (4 valid, 1 exception) silently treated as all-valid. How did this happen and what do you change?

Model answer

I'd check whether return_exceptions=True was set. With that flag, the exception from the flaky API is returned as a value in the result list rather than raised; if the caller does not inspect each result for isinstance(r, Exception), it silently processes the exception object as data. I'd either remove return_exceptions=True and let the first failure propagate, or keep it but add explicit validation: if isinstance(r, Exception): raise RuntimeError(...). The trade-off is that raising fast aborts the entire fan-out on a single flaky call; if the five calls are independent (different services, different data), I'd keep return_exceptions=True and inspect — but never skip the inspection.

Interview corner

CommonWhat is the main difference between gather and TaskGroup?
“I'd say TaskGroup gives structured ownership: when one task fails, all siblings are automatically cancelled and the group raises an ExceptionGroup. gather does not cancel siblings by default, so you can end up with tasks running in the background after you've already handled the error. For new code I'd default to TaskGroup.”
TrickWhat is the type of exception raised by TaskGroup when a child task fails?
“I'd say it's an ExceptionGroup — a container that holds one or more child exceptions. You unwrap it with except* syntax (3.11+), not a plain except. Catching it with except Exception gives you the group object but loses the individual errors inside.”
Experience checkHow would you limit concurrency in a gather of 500 tasks?
“I'd use an asyncio.Semaphore. Create it with the desired concurrency count, then async with semaphore: inside each task before doing the I/O. The semaphore ensures at most N tasks hold an active slot at once, without changing the fan-out structure of the gather.”
DesignWhen would you pick gather over TaskGroup for production code?
“I'd pick gather when I need return_exceptions=True — collecting all results including failures into one list for bulk inspection. I'd also use it for truly independent tasks where partial success is acceptable. For anything where "the group fails together or succeeds together" — a payment fan-out, a batch write to multiple replicas — I'd use TaskGroup because the cancellation guarantee removes a class of partial-state bugs.”

Memory hook

Two team leads: gather says "call me when everyone's done, but if one fails I'll raise and the others keep running in the background." TaskGroup says "I own this team: one failure and I cancel everyone, then hand you one tidy bundle of all the errors."

If you remember only one thing:

Use TaskGroup when tasks share a lifetime — it cancels siblings on failure and never leaks a running task past the scope.

Self-check

  1. Does asyncio.gather cancel sibling tasks when one raises an exception?

    No, by default. Siblings keep running. Only TaskGroup automatically cancels all siblings.

  2. What syntax do you use to catch an ExceptionGroup?

    except* ExcType as eg: — note the *. A plain except ExceptionGroup works but does not unwrap the individual exceptions.

  3. What does gather(return_exceptions=True) change?

    Instead of raising the first exception, every task runs to completion and exceptions are returned as values in the result list, in input order.

  4. What is the minimum Python version for TaskGroup?

    3.11.

  5. Are gather results in completion order or input order?

    Input order — the result at index 0 corresponds to the first argument you passed, regardless of which finished first.

Prove it

Write an async function resilient_fetch(urls) that fetches all URLs concurrently and returns a list of (url, result_or_error) pairs — succeeding URLs as their result, failing ones as their exception message. Implement it twice: once with gather(return_exceptions=True) and once with TaskGroup. Confirm both return the same shape for a mix of succeeding and failing "calls" (simulate with coroutines that return or raise based on a condition). Paste both implementations and note which version required more defensive code.

py-08-04Card 4 of 6

Structured async cancellation and TaskGroup ownership

HardP3 · Role-dependentTalk about it14 min
cancellationcancellederrortaskgrouptimeoutstructured-concurrencycleanup

Why you care

Cancellation is the most mishandled part of asyncio code in production. If you swallow CancelledError by catching it without re-raising, a timeout that was supposed to abort a request silently turns into a full-length wait, leaking connections and leaving background tasks running past the scope that created them. If you never run cleanup in a finally block, a cancelled task leaves connections open, files locked, or partial writes committed. Interviewers ask about cancellation specifically because it is the gap between code that works in demos and code that survives production timeouts, upstream failures, and graceful shutdowns.

Explain like I'm 5

Think of a fire alarm evacuation. Every task (person) in a building must walk to the exit (finally cleanup) when the alarm sounds (CancelledError). A task that "hears" the alarm but ignores it (swallows CancelledError without re-raising) never checks out at the muster station — the fire marshal does not know the building is clear. The alarm keeps ringing. asyncio.timeout is the marshal who sounds the alarm after a given time; TaskGroup is the marshal who sounds it the moment any one person trips (raises an exception).

Core concept

CancelledError has been a direct subclass of BaseException since Python 3.8, not Exception. A bare except Exception: will not catch it; a bare except BaseException: will, but you must re-raise it unless you are absolutely sure you are handling the cancellation yourself.

Three rules:

  1. Always re-raise CancelledError (or let it propagate naturally). Catching it to do cleanup and then swallowing it makes the task appear "done" to the event loop while the requester who called cancel() believes the task was cancelled.
  2. Put cleanup in finally, not just in except asyncio.CancelledError. The finally block runs on both normal completion and cancellation, so you need only one cleanup path.
  3. task.cancel() injects CancelledError at the next await point, not immediately. If the task is executing CPU code with no await, the cancellation is deferred until the task next yields.

asyncio.timeout(seconds) (3.11+) is a context manager that cancels the enclosed scope after the deadline and converts the resulting CancelledError to TimeoutError. This is the standard way to add deadlines without writing cancel logic manually. Do not catch CancelledError inside an asyncio.timeout block and swallow it — you will break the timeout.

task.uncancel() (3.11+) decrements the pending cancellation count. Use it only when you are implementing a higher-level cancellation scope (like a library, not application code) and you are certain the CancelledError you caught was yours to consume.

Decision rule. Use asyncio.timeout for per-request deadlines. Use TaskGroup when a group of tasks should cancel together on the first failure. Never use task.uncancel() in application code unless you understand the internal cancellation counter.

Under the hood

task.cancel() does not immediately stop execution. It schedules a CancelledError to be thrown into the task at its next await point by setting an internal flag. When the task resumes at a await asyncio.sleep or similar, the event loop throws CancelledError into the generator frame of the coroutine instead of resuming it normally. This is exactly the generator.throw() mechanism (Delegate generators and exchange values) extended to coroutines.

asyncio.timeout works by wrapping the enclosed coroutine in an asyncio.Timeout object. On deadline, it calls the underlying task's cancel() with a deadline reason. On __aexit__, if the exit reason was the deadline, it converts the CancelledError into TimeoutError. If you catch and swallow CancelledError inside the with asyncio.timeout block, the Timeout.__aexit__ never sees the CancelledError it planted and cannot convert it — the timeout silently expires without effect.

In Python 3.13, TaskGroup cancellation handling was tightened: tasks cancelled by the group (due to a sibling failure) now track a separate "group cancellation" count distinct from external cancellations, so a task.cancel() from outside the group does not get absorbed by the group's internal cleanup. This matters if you nest TaskGroups or cancel group tasks from outside.

Minimal example Correct vs broken CancelledError handling

PythonSwallowing CancelledError breaks cancellationruns
import asyncio

cleanup_ran = []

async def correct():
    try:
        await asyncio.sleep(10)
    except asyncio.CancelledError:
        cleanup_ran.append('correct')
        raise   # re-raise so the task is truly cancelled

async def broken():
    try:
        await asyncio.sleep(10)
    except asyncio.CancelledError:
        cleanup_ran.append('broken')
        # NOT re-raising — the cancellation is silently swallowed

async def main():
    t1 = asyncio.create_task(correct())
    await asyncio.sleep(0)
    t1.cancel()
    try:
        await t1
    except asyncio.CancelledError:
        pass
    print('correct task cancelled:', t1.cancelled())

    t2 = asyncio.create_task(broken())
    await asyncio.sleep(0)
    t2.cancel()
    await t2   # no exception — broken swallowed it
    print('broken task cancelled:', t2.cancelled())

    print('cleanup ran:', cleanup_ran)

asyncio.run(main())
Output
correct task cancelled: True
broken task cancelled: False
cleanup ran: ['correct', 'broken']

In production Hotstar stream health checks with per-stream timeout

Each WebSocket stream gets a health ping with a 500 ms timeout. The finally block runs cleanup regardless of whether the ping succeeded or the timeout fired. Because CancelledError is re-raised inside check_stream, asyncio.timeout receives it and converts it to TimeoutError as designed.

PythonStream health check with timeout and guaranteed cleanupruns
import asyncio

closed_streams = []

async def check_stream(stream_id, latency_s):
    try:
        await asyncio.sleep(latency_s)   # simulates WebSocket ping round-trip
        return f'{stream_id}: alive'
    except asyncio.CancelledError:
        closed_streams.append(stream_id)   # log for metrics
        raise   # must re-raise so asyncio.timeout can convert to TimeoutError

async def check_with_timeout(stream_id, latency_s, timeout_s):
    try:
        async with asyncio.timeout(timeout_s):
            return await check_stream(stream_id, latency_s)
    except TimeoutError:
        return f'{stream_id}: timed_out'

async def main():
    results = await asyncio.gather(
        check_with_timeout('stream-1', 0.01, 0.5),
        check_with_timeout('stream-2', 10.0, 0.05),   # will time out
        check_with_timeout('stream-3', 0.01, 0.5),
    )
    print('results:', results)
    print('closed for cleanup:', closed_streams)

asyncio.run(main())
Output
results: ['stream-1: alive', 'stream-2: timed_out', 'stream-3: alive']
closed for cleanup: ['stream-2']

See it happen TaskGroup cancellation sequence

callerTaskGrouptask-slowtask-failasync with TaskGroup() as tgtg.create_task(slow_ok())tg.create_task(fail())raise ValueErrorcancel()CancelledError → finally runs → doneraise ExceptionGroup([ValueError])
  1. The caller enters the TaskGroup scope. The group will own all tasks created inside.
  2. task-slow is scheduled on the event loop. The group registers it as a child.
  3. task-fail is also scheduled. Both run concurrently until one finishes or fails.
  4. task-fail raises an exception. The group intercepts it before it reaches the caller.
  5. The group immediately cancels all remaining sibling tasks.
  6. task-slow receives CancelledError at its next await, runs finally cleanup, and reports done.
  7. Once all tasks are done, the group raises a single ExceptionGroup containing all collected exceptions.
Notice: no task leaks past the async with block — the group's __aexit__ waits for all cancellations to complete before raising.
Scenario A

In your world

The situation

Your async pipeline calls three partner APIs to enrich each payment record before writing to Postgres. One partner regularly times out. You added try/except Exception: around the slow call and log the error — but you forgot raise. The partner call has a 30-second socket timeout.

What actually breaks

Every time that partner times out, your pipeline waits 30 seconds per record instead of failing fast. The event loop is blocked for 30 seconds per slow record. At 100 slow records/hour, the pipeline is serially delayed by 50 minutes per hour. Jenkins shows green because the exception was caught and logged, not re-raised.

What you'd do on Monday

Replace the bare timeout with asyncio.timeout and re-raise CancelledError. Set the deadline to the SLA you can defend (e.g., 500 ms) and let TimeoutError propagate to the caller, which decides whether to skip or retry:

PythonPer-call timeout with proper cleanup (needs httpx)
import asyncio, httpx

async def enrich(client, payment_id):
    try:
        async with asyncio.timeout(0.5):   # 500 ms hard deadline
            resp = await client.get(f'{PARTNER_URL}/{payment_id}')
            return resp.json()
    except TimeoutError:
        return None   # caller decides: skip or retry
Scenario B

Imagine you work at Hotstar

10,000 concurrent stream health checks per service instance

The numbers

30M concurrent streams at IPL peak across a fleet. One health-check service instance manages 10,000 streams. Each stream gets a heartbeat check every 30 seconds = 10,000 ÷ 30 ≈ 333 checks/second. Each check has a 200 ms response timeout. At any instant: 333 checks/second × 200 ms window = ~67 checks in flight. A dead stream that is not cancelled holds a task for 200 ms. At 1% dead streams = 100 dead streams per 10,000. Without cancellation: 100 tasks × 200 ms = 20 task-seconds of leaked work per second — the loop is 20× overloaded with ghost tasks.

What the Scenario-A answer does here

The Scenario A fix (asyncio.timeout + re-raise) works at this scale for individual timeouts. The flip issue is scale of the fleet: 30M streams ÷ 10,000 per instance = 3,000 service instances. An outage that kills 10% of streams evenly = 10% × 10,000 = 1,000 dead streams per instance, hitting all 3,000 instances simultaneously. Each instance now handles 1,000 sudden timeouts at once, all triggering cleanup. If cleanup itself does I/O (e.g., notifying a session store), a CancelledError swallowing bug quietly skips that notification for every dead stream.

What flips at this scale

At fleet scale, every swallowed CancelledError is a metric gap: the session store shows streams as alive when they are dead, which delays failover and confuses downstream consumers. The design change is to make cleanup idempotent and always fire it — even if it means a brief extra write per cancellation.

The trade-off you'd defend

I would defend asyncio.timeout with re-raise and a finally block for session cleanup over a simpler "catch and ignore" approach, trading a small write per timeout for accurate fleet state. The one counter-argument is write amplification during mass outages — I'd cap cleanup retries and use a best-effort dead-letter queue rather than a blocking write.

Illustrative; Hotstar's 50M peak concurrent viewer count and fleet sizing are from public announcements; per-instance stream counts and cleanup costs are not published.

Gotchas

Swallowing CancelledError causes asyncio.timeout to silently fail

Symptom

Adding asyncio.timeout(1.0) to a call has no effect — the call still takes 5 seconds.

Cause

The code inside with asyncio.timeout catches asyncio.CancelledError and does not re-raise it. The timeout mechanism plants CancelledError; swallowing it hides the deadline from the timeout's __aexit__.

Fix

Never use except Exception: inside an asyncio.timeout block without re-raising CancelledError explicitly. Use finally for cleanup instead.

CancelledError is BaseException, not Exception

Symptom

A bare except Exception: logger wraps an async handler. Production timeouts are caught, logged, and then the task continues running for 30 seconds instead of aborting.

Cause

asyncio.CancelledError.__mro__ is [CancelledError, BaseException, object] — it bypasses except Exception.

Fix

Add an explicit except asyncio.CancelledError: raise before the broad except Exception: handler, or restructure to use finally for cleanup.

task.cancel() is deferred, not immediate

Symptom

You call task.cancel() and immediately check task.cancelled() — it returns False. A few milliseconds later it becomes True.

Cause

cancel() schedules a CancelledError to be thrown at the task's next await. If the task is executing CPU code, it has not reached that await yet.

Fix

Always await the task after cancelling it to ensure the cancellation has propagated: task.cancel(); try: await task; except asyncio.CancelledError: pass.

Tricky questions

Does catching CancelledError in a finally block suppress it?
Most people say

Yes — anything in a finally block runs, so catching CancelledError there also suppresses it.

Actually

finally runs on cancellation, but a plain finally does not catch CancelledError; it re-raises automatically. Only an explicit except asyncio.CancelledError: inside the finally (or inside a try within finally) would catch and potentially swallow it. The correct pattern is: finally: cleanup() with no except inside.

What does asyncio.timeout convert CancelledError into?
Most people say

It raises a plain TimeoutError from the asyncio module.

Actually

asyncio.timeout converts the CancelledError it planted into the built-in TimeoutError (the same class as concurrent.futures.TimeoutError, since 3.11 they are aliases). Any CancelledError that was not planted by that specific timeout scope is left as-is and propagates normally.

Scenario question

You're handed an async service where asyncio.wait_for(call(), timeout=2.0) was replaced with asyncio.timeout(2.0). After the change, some requests still take 10 seconds. The engineer says "I added a try/except around it." What went wrong?

Model answer

I'd look at whether the except inside the asyncio.timeout block catches asyncio.CancelledError. Since Python 3.8, CancelledError is a direct subclass of BaseException, not Exception, so except Exception: would not catch it. The more likely bug is except BaseException: log_and_continue(), or an explicit except asyncio.CancelledError: without re-raising. I'd inspect the except block and add except asyncio.CancelledError: raise before any broad handler. The trade-off is that surfacing swallowed cancellations as errors means the service may need a retry layer before the fix is safe to deploy.

Interview corner

CommonWhy must you re-raise CancelledError after catching it?
“I'd explain that CancelledError is the signal asyncio uses to stop a task cooperatively. If you catch it and don't re-raise, the task's caller — the code that called task.cancel() — believes the task was cancelled but the task actually completed normally. This breaks timeouts, TaskGroup cleanup, and graceful shutdown logic that depends on cancellation propagating.”
TrickIs CancelledError a subclass of Exception?
“I'd say no — since Python 3.8 its MRO is [CancelledError, BaseException, object]. It bypasses except Exception. This catches people out who wrap async handlers with a broad except Exception: log_error() and then wonder why their timeouts have no effect.”
Experience checkHow does asyncio.timeout work internally?
“I'd say it creates a deadline on the running task. When the deadline fires, it calls task.cancel() on the current task, which injects CancelledError at the next await. On __aexit__, if the exit was caused by the deadline's own CancelledError, it converts it to TimeoutError. If you swallow the CancelledError before __aexit__ sees it, the timeout effectively never fires.”
DesignHow would you add a per-request timeout to all calls in an async service?
“I'd wrap each request handler with async with asyncio.timeout(DEADLINE): and add a single except TimeoutError: at the top level of the handler to return a 504 to the client. I'd put resource cleanup (close connection, log metrics) in a finally block inside the handler, not in the except TimeoutError, so cleanup runs whether the request succeeded, failed, or timed out.”

Memory hook

The fire alarm image: every task must walk to the exit (finally cleanup) and check out (raise CancelledError) when the alarm sounds. Ignoring the alarm (swallowing CancelledError) leaves the fire marshal with an unknown headcount.

If you remember only one thing:

Always re-raise CancelledError; put cleanup in finally, not in the except block.

Self-check

  1. What is the base class of asyncio.CancelledError?

    BaseException, not Exception — this has been true since Python 3.8.

  2. When does task.cancel() actually throw CancelledError into the task?

    At the task's next await point — not immediately when cancel() is called.

  3. What does asyncio.timeout do with a CancelledError it did not plant?

    It leaves it as-is and lets it propagate. Only the CancelledError caused by its own deadline is converted to TimeoutError.

  4. Why should cleanup go in a finally block rather than in except CancelledError?

    A finally block runs on both normal completion and cancellation, so you need only one cleanup path. Putting cleanup only in except CancelledError means it does not run on exceptions other than cancellation.

  5. What happens if you swallow CancelledError inside an asyncio.timeout block?

    The timeout's __aexit__ never sees the CancelledError it planted and cannot convert it to TimeoutError. The call proceeds as if no timeout was set.

Prove it

Write a coroutine timed_fetch(delay, timeout) that simulates a network call taking delay seconds, wrapped in asyncio.timeout(timeout). First version: catch asyncio.CancelledError inside and swallow it. Second version: re-raise. Call both versions with delay=5, timeout=0.1 and measure elapsed time with time.perf_counter. Paste both runtimes. Explain in one sentence why they differ.

py-08-05Card 5 of 6

Isolate blocking work with executors

HardP3 · Role-dependentKnow cold13 min
executorto-threadblockingthread-pool-executorprocess-pool-executorasyncio

Why you care

Every async codebase eventually calls something that is not async: a legacy CSV parser, a PDF renderer, a synchronous database driver, a CPU-heavy compression function. Calling these directly inside async def stalls the entire event loop for the duration — every other coroutine freezes, every timeout starts counting against you. asyncio.to_thread and run_in_executor exist specifically to bridge blocking calls back into an async program safely. Not knowing this means you will either block the loop silently or refuse to use async at all because "it doesn't work with our libraries." Interviewers ask this to separate people who understand the event loop model from people who have only written small toy async scripts.

Explain like I'm 5

A kitchen chef (the event loop) can handle ten orders simultaneously as long as she never has to physically leave the building. But sometimes a recipe needs a rare ingredient from a remote warehouse (a blocking call). Instead of leaving the kitchen empty, she calls a delivery service (an executor) and keeps cooking other orders while the delivery is in transit. When the delivery arrives, the service notifies her (the future resolves) and she finishes that recipe. The chef never left; the kitchen never stopped.

Core concept

Two tools for isolating blocking work:

  • await asyncio.to_thread(fn, *args) (3.9+): runs fn(*args) in the default ThreadPoolExecutor managed by the event loop, returns a coroutine you can await. One line; the right default for most blocking I/O and library calls.
  • await loop.run_in_executor(executor, fn, *args): lower-level; lets you pass a custom ThreadPoolExecutor or ProcessPoolExecutor. Use when you need to control thread count or use processes.

For CPU-bound work that must truly parallelise:

  • Use ProcessPoolExecutor instead of ThreadPoolExecutor. Processes have their own GIL so CPU work runs in parallel, but every argument and result must be pickle-serialisable, and startup cost is ~50–100 ms per process (illustrative).

Decision rule. Use asyncio.to_thread for any blocking I/O call or synchronous library. Use ProcessPoolExecutor for CPU-bound pure Python work (compression, route optimisation, image processing). Never use ThreadPoolExecutor for CPU-bound work in an async context — the GIL prevents parallelism, and you have added complexity for no gain (Runtime concurrency: threads, processes, async and GIL).

Under the hood

asyncio.to_thread(fn, *args) is a thin wrapper around loop.run_in_executor(None, fn, *args). The None executor uses the event loop's own default ThreadPoolExecutor, which by default has min(32, os.cpu_count() + 4) threads. The function fn is submitted to the thread pool as a concurrent.futures.Future; the event loop wraps this in an asyncio.Future and registers a done-callback that calls loop.call_soon_threadsafe(future.set_result, result). That call-soon-threadsafe is the critical detail: worker threads cannot safely call the event loop directly; they must post the result through this thread-safe channel, which the loop picks up on its next iteration.

When using ProcessPoolExecutor, the child process start method determines how the new interpreter is created. Windows has only spawn (always has). macOS defaulted to spawn since Python 3.8. Linux/POSIX still defaults to fork in 3.12–3.13, but Python 3.14 changes the POSIX default to forkserver (not spawn) to avoid fork-safety bugs. With spawn or forkserver, the submitted function must be importable from a top-level module — you cannot use lambdas or locally-scoped functions — which is why process pool examples require a .py file with if __name__ == '__main__': rather than python -c.

Minimal example asyncio.to_thread offloads a blocking call

Pythonto_thread keeps the loop free while blocking I/O runs in a threadruns
import asyncio, time

def blocking_read(filename):
    time.sleep(0.05)   # simulates a synchronous file or DB read
    return f'data from {filename}'

async def main():
    # Both reads run in the thread pool while the event loop stays responsive
    results = await asyncio.gather(
        asyncio.to_thread(blocking_read, 'orders.csv'),
        asyncio.to_thread(blocking_read, 'settlements.csv'),
    )
    print('results:', results)
    print('ran in thread pool:', True)

asyncio.run(main())
Output
results: ['data from orders.csv', 'data from settlements.csv']
ran in thread pool: True

In production Parallel settlement file parsing with legacy sync library

A Grab delivery pipeline uses a legacy CSV library that cannot be made async. Three settlement files are parsed concurrently using asyncio.to_thread; the event loop stays free for heartbeat and health-check coroutines that run alongside.

PythonLegacy sync parser wrapped with to_thread (three files in parallel)runs
import asyncio, csv
from pathlib import Path

# --- fixture: write three settlement files ---
for i in range(1, 4):
    Path(f'settlements_{i}.csv').write_text(
        'txn_id,amount,currency\n' +
        '\n'.join(f'T{i}{j:03d},{j*100},INR' for j in range(1, 4))
    )

def parse_settlement_file(path):
    """Legacy sync CSV parser — cannot be made async."""
    with open(path, newline='') as f:
        return list(csv.DictReader(f))

async def main():
    files = [f'settlements_{i}.csv' for i in range(1, 4)]

    # All three files parsed concurrently in thread pool
    batches = await asyncio.gather(*[
        asyncio.to_thread(parse_settlement_file, f) for f in files
    ])

    total_rows = sum(len(b) for b in batches)
    total_amount = sum(int(r['amount']) for b in batches for r in b)
    print('files:', len(batches))
    print('total rows:', total_rows)
    print('total amount:', total_amount)

asyncio.run(main())
Output
files: 3
total rows: 9
total amount: 1800
Scenario A

In your world

The situation

Your async FastAPI service generates a PDF receipt after every completed order. You call a synchronous PDF library (reportlab) directly inside the endpoint's async def. The service handles 50 orders per second at peak.

What actually breaks

Under load, all response times degrade by 30–50 ms even for orders that do not need a receipt. The event loop is blocked for each PDF generation call. Profiling shows the PDF call takes 25–40 ms of CPU time per receipt; at 50/sec, the loop is blocked for up to 2 seconds in every second of clock time.

What you'd do on Monday

Move the PDF call to a thread pool with asyncio.to_thread. The loop stays free; receipt generation overlaps with other endpoint handling:

PythonPDF generation off-loaded to thread pool (needs reportlab + FastAPI)
import asyncio
from reportlab.pdfgen import canvas
import io

def generate_pdf_sync(order_id, items):
    buf = io.BytesIO()
    c = canvas.Canvas(buf)
    c.drawString(72, 800, f'Receipt #{order_id}')
    for i, item in enumerate(items):
        c.drawString(72, 780 - i * 20, item)
    c.save()
    return buf.getvalue()

async def generate_receipt(order_id, items):
    # Offload CPU+sync work; event loop is free during the call
    return await asyncio.to_thread(generate_pdf_sync, order_id, items)
Scenario B

Imagine you work at Grab

100 ride-completion receipts per second at peak

The numbers

Grab handles ~500,000 ride completions per day = ~6 per second on average, peak around 100/second during rush hour. Each PDF receipt: 30 ms of CPU (illustrative). Sequential in the async loop: 100 × 30 ms = 3,000 ms of CPU per second — the loop is 3× overloaded, no other coroutine runs. Thread pool (4 workers): 4 parallel PDFs × 30 ms = 4 receipts complete every 30 ms; to process 100 receipts: 100 ÷ 4 = 25 batches × 30 ms = 750 ms of wall clock. The event loop is free between batches to serve other requests.

What the Scenario-A answer does here

Knowing to use a thread pool solves the event loop blocking. The naive mistake at this scale is using an unbounded thread pool (max_workers=None, which defaults to 32 or more). 100 concurrent PDF tasks × 30 ms of CPU each on 8 cores = 12.5 CPU-seconds per second, which exceeds available CPU and causes thread starvation and context-switch overhead.

What flips at this scale

Bound the thread pool to max_workers = os.cpu_count() or slightly below. For pure CPU work, ProcessPoolExecutor provides true parallelism (each process has its own GIL); for mixed CPU+I/O work like PDF generation (which touches the filesystem), a bounded ThreadPoolExecutor is simpler and adequate.

The trade-off you'd defend

I would defend a bounded ThreadPoolExecutor for PDF generation over ProcessPoolExecutor, trading the simpler setup and no pickle overhead for the slight GIL contention on CPU work. If profiling showed the GIL was the bottleneck (threads blocked on each other, not on I/O), I would switch to processes — but I would measure first.

Illustrative; Grab's ride volumes are from public announcements, but per-second peaks and PDF generation times are not published.

Gotchas

Calling a blocking function directly inside async def stalls the loop

Symptom

Response latency for all endpoints spikes when one endpoint does a blocking call; no exceptions are raised; the loop appears healthy.

Cause

Any blocking call — requests.get, time.sleep, open().read() on a slow filesystem, pandas.read_csv — holds the event loop for its entire duration.

Fix

Wrap with await asyncio.to_thread(blocking_fn, *args) or replace with a native async library.

ProcessPoolExecutor requires picklable functions and a __main__ guard

Symptom

AttributeError: Can't get attribute 'fn' on <module '__main__'> or a BrokenProcessPool error when using ProcessPoolExecutor with a locally defined function.

Cause

Process workers use spawn or forkserver (Windows always uses spawn; macOS since 3.8; Linux/POSIX defaults to fork through 3.13 and forkserver from 3.14). With spawn/forkserver, workers reimport the module to find the function; lambdas and locally-scoped functions have no importable path.

Fix

Define functions at module level in a .py file, and protect the ProcessPoolExecutor call with if __name__ == '__main__':.

Thread pool exhaustion silently queues work

Symptom

asyncio.to_thread calls do not raise but take much longer than expected; ThreadPoolExecutor's work queue grows without bound.

Cause

The default thread pool (min(32, cpu_count + 4) workers) is full; new tasks are queued and wait for a free slot.

Fix

Set an explicit max_workers appropriate for your workload and monitor queue depth. For CPU-bound tasks, match max_workers to CPU count.

Tricky questions

Does asyncio.to_thread make a synchronous library "async"?
Most people say

Yes — wrapping a blocking call in to_thread makes it behave like a native async call.

Actually

It makes it non-blocking for the event loop — the library still runs synchronously in a thread. The event loop is free while the thread runs, but the thread is still blocking OS resources. If the library is not thread-safe, you will get races. to_thread is a bridge, not a conversion.

Can you use asyncio inside a function submitted to to_thread?
Most people say

Yes — the thread has access to the event loop.

Actually

The submitted function runs in a plain OS thread with no running event loop. asyncio.run(coro()) inside it would create a new loop; asyncio.get_event_loop() would raise RuntimeError: There is no current event loop in thread. If you need async work inside the thread, use asyncio.run explicitly or restructure so the async work happens in the event loop thread.

Scenario question

You're handed an async FastAPI service that calls pandas.read_csv(s3_url) directly in a route handler. The URL is a presigned S3 link; pandas opens it over HTTP. Latency is unpredictable. What is wrong and how do you fix it?

Model answer

I'd say the problem is double: pandas' HTTP call is synchronous and blocks the event loop for the entire download plus parse time, which could be seconds for a large file. I'd fix it in two steps. First, replace the pandas call with an async download: use httpx.AsyncClient to download the file bytes into memory, then run pandas.read_csv(io.BytesIO(bytes)) via asyncio.to_thread to keep the CPU-bound parse off the loop. The trade-off is that the file is held in memory twice — as bytes and as a DataFrame — which matters for large files. For very large CSVs I'd stream the download to a temp file asynchronously and pass the path to to_thread, trading slightly more code for bounded memory.

Interview corner

CommonWhat happens if you call time.sleep() inside an async function?
“I'd say it blocks the entire event loop for the sleep duration — every other coroutine also pauses. The fix is await asyncio.sleep(), which suspends only the current coroutine and lets the loop run other tasks.”
CommonWhen would you use asyncio.to_thread vs run_in_executor?
“I'd use asyncio.to_thread for most blocking calls — it's a one-liner and uses the loop's managed thread pool. I'd use run_in_executor when I need a custom pool with a specific max_workers, or when I need to pass a ProcessPoolExecutor for CPU-bound work. They do the same thing; to_thread is just the clean public API added in 3.9.”
Experience checkWhy does ProcessPoolExecutor fail when used with a locally-defined function in a script?
“I'd say process workers are spawned as fresh Python interpreters that reimport the module to find the function. A locally-defined function or lambda has no module-level name, so the worker can't find it. The fix is to define the function at module level in a .py file with a if __name__ == '__main__': guard around the pool call.”
DesignHow would you parallelise a CPU-heavy image resizing step in an async service?
“I'd isolate the resize function in a module-level function and submit it to a ProcessPoolExecutor via run_in_executor. The process pool gives true CPU parallelism because each process has its own GIL. I'd bound the pool to CPU count and accept that the event loop is free between submissions. The trade-off is that image bytes must be pickled across the process boundary — for very large images I'd use shared memory or write to a temp file instead.”

Memory hook

The delivery service image: the event loop is a chef who never leaves the kitchen. Blocking calls are trips to the warehouse. asyncio.to_thread calls a delivery service (a thread pool) — the chef keeps cooking, the delivery happens in parallel, and the ingredient arrives when the future resolves.

If you remember only one thing:

Offload every blocking call inside async def with await asyncio.to_thread(fn, *args), because a direct call freezes every other coroutine.

Self-check

  1. What does asyncio.to_thread actually do with the function you give it?

    It submits the function to a thread pool executor, wraps the result in an asyncio.Future, and returns a coroutine that resolves when the thread completes.

  2. Why does ProcessPoolExecutor require picklable arguments?

    Arguments and return values must cross a process boundary; Python uses pickle for that serialisation.

  3. What is the default number of threads in asyncio's managed thread pool?

    min(32, os.cpu_count() + 4).

  4. What goes wrong if you call asyncio.get_event_loop() inside a function submitted to to_thread?

    It raises RuntimeError: There is no current event loop in thread ... — the thread has no running event loop.

  5. What is the difference between ThreadPoolExecutor and ProcessPoolExecutor for CPU-bound work?

    ThreadPoolExecutor threads share the GIL — no CPU parallelism for pure Python. ProcessPoolExecutor processes each have their own GIL — true parallelism, at the cost of pickling and startup time.

Prove it

Write an async function that processes three large CSV files with a synchronous pandas read_csv call (or a time.sleep(0.1) simulation if pandas is not available). Version A: call the function directly in async def. Version B: wrap with asyncio.to_thread. Run both with time.perf_counter and record the elapsed time. Confirm that version B is roughly N × slower for N files in version A versus ~1× slower for version B (they overlap). Paste the two timings as your artefact.

py-08-06Card 6 of 6

Explain when async is the wrong answer

MediumP3 · Role-dependentTalk about it12 min
async-wrongcpu-boundcomplexitysequentialmeasurementdecision

Why you care

Async is the most over-applied pattern in Python today. Teams rewrite working scripts in asyncio because they heard it is "faster", spend a week debugging event loop errors, and end up with code that is harder to test and the same speed — or slower. The job consequence is wasted days and brittle code. The interview consequence is that a candidate who cannot explain when not to use async sounds like they learned it from a blog post, not from production experience. This card is about defending "I would not use async here, and here is why" — which is often the right answer and always the mature one.

Explain like I'm 5

A power drill is faster than a hand screwdriver for a hundred screws. But if you are putting together one flat-pack shelf, setting up the drill, choosing the right bit, charging the battery, and cleaning it up afterwards takes longer than just turning three screws by hand. Async is the power drill: indispensable at scale, overkill for a small job. The question is whether the setup cost exceeds the gain — and for CPU-bound work, sequential scripts, or codebases that depend on synchronous libraries, the answer is almost always: reach for the hand screwdriver.

Core concept

Async gives you concurrency, not speed. It lets one thread make progress on multiple I/O tasks at once. It does nothing for:

  • CPU-bound work: the event loop is one thread; async def process_row() does not run faster than def process_row() when the body is pure computation. Only processes give you CPU parallelism (Runtime concurrency: threads, processes, async and GIL).
  • Sequential scripts: a Jenkins ETL job that reads, transforms, and writes in order has nothing to gain from async. Wrapping it in asyncio.run adds an event loop setup and tear-down for zero benefit.
  • Synchronous dependency stacks: if your ORM is synchronous, your logging library is synchronous, and your main I/O bottleneck is a blocking DB driver, converting the application layer to async without converting the drivers produces async code that still blocks the loop (Isolate blocking work with executors).
  • Small I/O multiplexing: below ~10 concurrent connections, the overhead of managing tasks, futures and callbacks equals or exceeds the gain from overlap. A ThreadPoolExecutor with 4 threads is simpler and equally fast.

Decision rule. Ask three questions before adding async: (1) Does the job have many concurrent I/O operations that would overlap? (2) Are the libraries in the stack async-compatible? (3) Is the added complexity justified by a measured latency or throughput problem? If any answer is no, keep it synchronous.

Minimal example async def does not speed up CPU-only computation

PythonWrapping CPU work in async buys nothingruns
import asyncio

def compute_total(rows):
    return sum(r['amount'] for r in rows)

async def async_compute_total(rows):
    return sum(r['amount'] for r in rows)

rows = [{'amount': i} for i in range(1000)]

result_sync = compute_total(rows)
print('sync result:', result_sync)

result_async = asyncio.run(async_compute_total(rows))
print('async result:', result_async)
print('results equal:', result_sync == result_async)
print('async buys nothing for CPU-only work')
Output
sync result: 499500
async result: 499500
results equal: True
async buys nothing for CPU-only work

In production Nightly route optimisation that should stay synchronous

Delhivery's nightly batch assigns 20 packages to delivery routes using a pure Python algorithm. There is no I/O overlap needed: all data is already in memory, and the algorithm runs to completion before writing results. Adding async would add event loop overhead without benefit.

PythonRoute assignment: pure CPU, synchronous is correct hereruns
import csv
from pathlib import Path

def assign_route(package):
    """Pure CPU: nearest-depot bucketing. No I/O, no concurrency needed."""
    depot_lat, depot_lon = 12.97, 77.59
    dist = ((float(package['lat']) - depot_lat) ** 2 +
            (float(package['lon']) - depot_lon) ** 2) ** 0.5
    return {**package, 'route': int(dist * 100) % 10, 'depot': 'BLR'}

Path('packages.csv').write_text(
    'id,lat,lon,weight\n' +
    '\n'.join(f'P{i:04d},{12.9+i*0.01:.2f},{77.5+i*0.01:.2f},{i%5+1}' for i in range(20))
)

with open('packages.csv', newline='') as f:
    packages = list(csv.DictReader(f))

assigned = [assign_route(p) for p in packages]
routes = {}
for p in assigned:
    routes.setdefault(p['route'], []).append(p['id'])

print('packages assigned:', len(assigned))
print('routes created:', len(routes))
print('async would not help this job: all CPU, no I/O to overlap')
Output
packages assigned: 20
routes created: 9
async would not help this job: all CPU, no I/O to overlap
Scenario A

In your world

The situation

You have a nightly Jenkins job: read a settlements file from S3, apply a set of business rules to each row (purely Python arithmetic), write a summary CSV back to S3. A new engineer suggests rewriting it in asyncio because "async is faster".

What actually breaks

The rewrite compiles. At first, it looks the same speed. Then the team notices debugging is harder: exceptions from coroutines have longer tracebacks, test setup needs an event loop, and a library used for business logic (decimal operations) emits warnings when accessed from within the loop in tests.

What you'd do on Monday

Revert to synchronous code. The job is I/O-bound only at the S3 read/write boundaries — two operations total. Those two calls can be sped up with a thread (or by switching to an async S3 client), but the computation itself is CPU-bound and sequential. The right optimisation is to profile whether S3 I/O or computation is the bottleneck, then optimise that specific step — not rewrite the whole script in asyncio.

Scenario B

Imagine you work at Delhivery

2M packages per night

The numbers

2M packages per day, all processed in a nightly batch window. Route optimisation: pure Python graph algorithm, ~0.5 ms per package (illustrative) = 2,000,000 × 0.5 ms = 1,000 s ≈ 17 minutes total CPU time. If you add async: still 17 minutes — the event loop is one thread and does not parallelise CPU work. Using ProcessPoolExecutor with 8 cores: 1,000 s ÷ 8 = 125 s ≈ 2 minutes. Async saves 0 minutes; processes save 15 minutes.

What the Scenario-A answer does here

The temptation is to wrap the optimisation loop in asyncio.gather. This does not help because each task is CPU-bound; the event loop serialises them on one thread. You have added async complexity and task management overhead for zero throughput gain.

What flips at this scale

The flip is recognising that the bottleneck is CPU, not I/O. Once you know that, the answer is processes for parallelism, not async. And before adding processes, you should also profile whether the 17 minutes is actually a problem: if the batch runs overnight with a 6-hour window, 17 minutes is fine, and the correct answer is "don't change it."

The trade-off you'd defend

I would defend keeping this job synchronous and adding ProcessPoolExecutor only if the window constraint is measured to be a problem. The cost of async in this case is complexity with no measured benefit; the cost of processes is cross-process serialisation overhead, but the speedup is real and proportional to core count.

Illustrative; Delhivery's 2M+ daily shipment count is from public filings; per-package algorithm time and core count are not published.

Gotchas

Async does not help a CPU-bound script

Symptom

An async rewrite of a data transformation job runs at the same speed or slower; CPU utilisation is 100% on one core.

Cause

The event loop is single-threaded; async def wrapping around pure computation does not add parallelism.

Fix

Profile first. If CPU is the bottleneck, use ProcessPoolExecutor. If I/O is the bottleneck (DB calls, S3 reads), use async or thread-based overlap for those specific operations.

A synchronous dependency in the call stack blocks the loop

Symptom

The async service is 10× slower than expected even though all the application code is async def. Profiling shows time spent inside db.execute() which is a synchronous ORM call.

Cause

An async def function can still call synchronous blocking code. The ORM's synchronous DB driver blocks the loop on every query.

Fix

Switch to an async DB driver (asyncpg, aiosqlite) or wrap with asyncio.to_thread (Isolate blocking work with executors). Do not add async at the application layer without also addressing blocking dependencies.

Testing async code is more complex than testing synchronous code

Symptom

Unit tests that were trivial to write (assert fn(x) == y) now require asyncio.run(async_fn(x)) or a pytest-asyncio fixture; mocking becomes harder; test isolation breaks when an event loop leaks between tests.

Cause

Async functions return coroutines; test frameworks need explicit loop management.

Fix

If the function has no I/O and no concurrent behaviour, make it synchronous. Use async only where you need the event loop's concurrency features. Keep pure computation synchronous so it is easy to test.

Tricky questions

Does making a function async def make it run in a separate thread?
Most people say

Yes — async functions run concurrently in the background.

Actually

No. async def makes the function a coroutine. It runs on the same thread as the event loop, cooperatively. If you want a separate thread, use asyncio.to_thread (Isolate blocking work with executors). Async and threads are completely independent concepts.

Is an async web framework like FastAPI always faster than a sync one like Flask?
Most people say

Yes — async frameworks are the modern choice and always outperform sync ones.

Actually

It depends on the bottleneck. If the handlers do blocking DB calls or CPU work without proper executor offloading, an async framework can be slower than a well-tuned sync WSGI app with multiple workers. Async helps when you have many concurrent I/O operations that are properly non-blocking end-to-end.

Scenario question

You're handed a Python microservice that hits a single SQL database synchronously and serves 30 requests per second. The team wants to rewrite it in asyncio for "better performance." What questions do you ask before starting?

Model answer

I'd ask three things. First, where is the measured bottleneck: is it the Python process, the database, the network, or something else? If the DB is saturated, async won't help — the bottleneck is not in Python. Second, is the database driver async-compatible? If the team uses psycopg2, a naive async rewrite will still block the loop on every query; we would need to migrate to asyncpg or psycopg3's async mode. Third, what is the target throughput? At 30 requests/second, a gunicorn/gevent setup or simply adding more workers may be simpler and faster than a full async rewrite. The trade-off I'd defend: measure first, rewrite only if the measurement shows I/O concurrency is genuinely the constraint.

Interview corner

CommonName three cases where async is the wrong choice for Python.
“I'd say: CPU-bound work like data transforms or compression — async is one thread and won't parallelise them; a sequential script with one or two I/O calls where thread overhead beats async setup; and a codebase where the critical dependencies are synchronous-only and cannot be wrapped cleanly, so you'd end up blocking the loop anyway.”
Experience checkYou have a Python ETL script that takes 8 hours. Would you use async to speed it up?
“I'd profile first. If the 8 hours is dominated by synchronous DB inserts, I'd batch inserts and use a connection pool before touching async. If it's S3 downloads, I'd try threads first — simpler. I'd only reach for async if I had hundreds of concurrent I/O operations and the thread count was becoming a problem. Most likely the 8-hour job has a simpler fix.”
TrickIf async def does not speed up CPU work, why do async web frameworks exist?
“I'd say web frameworks use async for concurrency, not CPU speed. A web handler spends most of its time waiting for a database or external service response — that's I/O. Async lets one thread handle hundreds of handlers concurrently while they all wait for I/O, which is completely different from speeding up computation. The confusion comes from conflating concurrency (multiple tasks in progress) with parallelism (multiple tasks actually running simultaneously on hardware).”
DesignHow would you decide whether to migrate a legacy sync service to async?
“I'd measure three things: concurrent active requests at peak (is concurrency actually the bottleneck?), fraction of time spent in I/O vs CPU (only I/O benefits), and whether key dependencies have async support. If all three point to async being beneficial, I'd migrate incrementally — new endpoints first, with asyncio.to_thread wrapping unchanged sync dependencies — rather than a big-bang rewrite.”

Memory hook

The power drill image: async is the drill — powerful and right for a hundred screws, but the setup overhead beats the gain for one screw. CPU-bound jobs, sequential scripts, and sync-dependency stacks are the one-screw situations.

If you remember only one thing:

Ask whether your bottleneck is many concurrent I/O operations before adding async; if it is not, the complexity costs more than the benefit.

Self-check

  1. Does making a function async def make it run faster for CPU-bound work?

    No. The event loop is single-threaded; async def does not add CPU parallelism.

  2. Name two situations where async is the wrong tool.

    Any two of: CPU-bound computation; sequential scripts with one or two I/O calls; codebases with blocking synchronous dependencies that cannot be wrapped.

  3. What mechanism should you use instead of async for CPU-bound parallelism?

    ProcessPoolExecutor (or multiprocessing), which gives each worker its own Python interpreter and GIL.

  4. Why does converting the application layer to async without converting the DB driver not help?

    The sync DB driver still blocks the event loop on every query. The async wrapper is a coroutine, but the I/O inside it is blocking.

  5. What three questions should you ask before rewriting a service in asyncio?

    Whether there are many concurrent I/O operations, whether dependencies are async-compatible, and whether a measured performance problem actually requires async.

Prove it

Take a synchronous Python function that processes a list of 100k records (e.g., parsing date strings). Wrap it in an async def and call it with asyncio.run. Compare wall-clock time for both versions using time.perf_counter. Record whether async is faster, slower, or the same. Then add await asyncio.sleep(0) after every 10,000 records to yield to the loop. Record the time again. Write one sentence explaining what the sleep(0) version demonstrates about async overhead on CPU-bound work.