Names refer to objects
Why you care
Every "the job succeeded but the target table is empty" bug in Python ETL eventually traces to one mechanic: a variable name is not a box that holds a value, it is a label stuck to an object in memory. When you write rows = data you attach a second label to the same object, not copy the data. If you cannot tell the difference between rebinding a name and mutating an object, you will corrupt shared state, get no error, and spend your evening re-running a job that succeeded. Interviewers ask this on first-round screens because it filters out engineers who write defensive copy() calls everywhere from those who understand when a copy is actually needed.
Explain like I'm 5
Think of Python objects as balloons and variable names as strings tied to them. a = [1, 2, 3] inflates a balloon (the list) and ties the string a to it. b = a ties a second string, b, to the same balloon. Poking the balloon — appending to the list — is visible through both strings. Cutting string a and tying it to a new balloon (a = [99]) does not pop the first balloon; b still floats there, holding the original list.
Core concept
Every object in CPython has an identity (its memory address, returned by id()), a type, and a value. A name is a binding from a string key to an object reference inside the current namespace dictionary. Assignment evaluates the right side, finds or creates the object, and binds the name. No data moves.
a = exprcreates the object (if new) and binds the nameato it.b = abindsbto the same object. Two names, one object.a is breturnsTrue.- Mutating the object via either name (
a.append(x),b["k"] = v) changes the single underlying object. The change is immediately visible through every name bound to it. - Rebinding a name (
a = something_else) only moves thealabel; every other name is unaffected and still points at the original object.
The practical test: after b = a, ask a is b. If True, both names share one object. Appending to b also appends to a. The decision rule: use a second name for read-only access to a shared object; reach for an explicit copy (Choose shallow or deep copy) only when you need an independent value. Always document whether a function that receives a mutable container is allowed to mutate it, or must leave it intact.
Minimal example One list, two names, one mutation
a = [10, 20, 30]
b = a # b is a second label on the same list
b.append(40)
print(a) # mutation via b is visible through a
print(a is b) # same object in memory
a = [99] # rebind a to a new list; b is unaffected
print(b) # b still holds the original list
print(a is b) # now different objects[10, 20, 30, 40]
True
[10, 20, 30, 40]
FalseIn production Trade loader that tags each row in-place
A common ETL pattern adds audit fields to each dict as it is inserted. Any code that held a reference to the original list before loading will see those tags appear — useful when intentional, a surprise when not.
import csv, sqlite3
from pathlib import Path
Path("trades.csv").write_text("id,symbol,qty\n1,INFY,100\n2,TCS,50\n3,HDFC,200\n")
con = sqlite3.connect(":memory:")
con.execute("CREATE TABLE trades (id INT, symbol TEXT, qty INT)")
with open("trades.csv", newline="") as f:
rows = list(csv.DictReader(f))
original_first = rows[0] # hold a reference to the first dict before loading
def load_row(con, row):
con.execute("INSERT INTO trades VALUES (?,?,?)",
(int(row["id"]), row["symbol"], int(row["qty"])))
row["_loaded"] = True # mutates the shared dict object in-place
for row in rows:
load_row(con, row)
# original_first is rows[0] — same dict, so the tag is visible
print(original_first.get("_loaded"))
count = con.execute("SELECT COUNT(*) FROM trades").fetchone()[0]
print("rows in db:", count)
print(original_first is rows[0])True
rows in db: 3
TrueIn your world
The situation
Your Jenkins job reads settlements.csv from S3 into a list of dicts, passes the list to a validate() helper, and then calls load() on the same variable. The validate() function removes bad rows using list.remove() in a loop.
What actually breaks
Rows that fail validation are silently deleted from settlements before load() sees them. The audit table says 1,247 rows read; Postgres gets 1,201. Jenkins is green. Finance's overnight reconciliation fails three days later with a 46-row discrepancy.
What you'd do on Monday
The root cause is in-place deletion on a shared list. Either make validate() return a new list and rebind before passing it to load(), or assert count consistency between stages:
raw = read_from_s3(key)
valid = [r for r in raw if is_valid(r)] # new list, raw is untouched
rejected = len(raw) - len(valid)
log.info("validated: %d accepted, %d rejected", len(valid), rejected)
load(conn, valid)
assert_loaded_count(conn, expected=len(valid))Imagine you work at PhonePe
The numbers
Illustrative: 200 M transactions ÷ 86,400 s ≈ 2,300 txn/s average, with intraday peaks 5–8× that. A batch ETL holding 1 M dicts — each ~500 bytes in CPython with a small key set — uses roughly 500 MB of RAM per batch just for Python objects.
What the Scenario-A answer does here
Passing the same batch list through multiple pipeline stages and letting each stage mutate it in-place works at 10 k rows. At 1 M rows the first mutating stage corrupts data that later stages expect intact, and the failures are non-deterministic depending on stage execution order.
What flips at this scale
At this scale each pipeline stage receives an immutable view — a tuple of frozen dataclasses, or dicts that convention (or a type checker) forbids mutating — and produces a new collection. The no-mutation contract is enforced by type, not by code review. Any state that must persist across stages is written to a checkpoint store, not threaded through a shared Python object.
The trade-off you'd defend
I would defend immutable pipeline objects over in-place mutation, trading one extra allocation per stage for a bug class that disappears entirely. At 2,300 txn/s the allocation cost is negligible; the debugging cost of shared-state corruption at scale is not.
Illustrative numbers scaled to make the arithmetic visible; not PhonePe-published figures.
Gotchas
Reassigning a parameter inside a function does not change the caller's variable
After calling def reset(lst): lst = [], the caller's list is unchanged.
lst = [] rebinds the local name lst to a new empty list. The caller's name still points at the original object.
Either return the new value and rebind in the caller (rows = reset(rows)), or mutate in place (lst.clear()) and document which contract the function follows.
Appending inside a loop that uses two names for the same list doubles entries
The list grows twice as fast as expected, or an assert len(a) == len(b) passes unexpectedly.
Both names point at the same list. Appending through either name affects the shared object.
Confirm with a is b. Use list(a) or copy.copy(a) to make a separate object (Choose shallow or deep copy).
Tricky questions
a = b copy the value?Yes — after a = b, changing b won't affect a.
a = b makes both names point at the same object. Mutating the object through b (e.g., b.append(1)) is immediately visible through a. Only rebinding b to a different object leaves a pointing at the original.
No — functions receive copies.
Functions receive a reference to the same object. If the function calls lst.append() or lst.remove(), the caller's list is changed. Only rebinding the parameter name (lst = []) leaves the caller's variable unaffected.
Scenario question
You're handed an ETL helper def clean(rows) that trims whitespace from string fields and returns nothing. In tests with 10 dicts it works. In production you occasionally see untrimmed values reaching the loader. The function is called before the load step.
Model answer
I'd check whether a second code path appends new rows to the same batch list after clean() runs — those rows never go through the cleaner. I'd also verify that clean() mutates each dict in-place and is not accidentally creating new dicts. The fix is to ensure all rows are assembled before the clean step, and add an assertion inside the loader that rejects any row with leading or trailing whitespace. The trade-off of asserting inside the loader is that it couples two concerns, but it catches the bug even if the call order changes again.
Interview corner
CommonWhat is the difference between rebinding a name and mutating an object?
b = a, rebinding a = [] leaves b on the original list. But a.append(1) adds to the same list both names share.”TrickYou pass a list to a function. The function does lst = []. Does the caller's list change?
lst to a new empty list; the caller's variable still points at the original. If the intent was to empty the caller's list, you call lst.clear() or del lst[:], which mutates the object rather than rebinding the name.”Experience checkHas this caused a real bug for you?
DesignHow do you enforce that a pipeline stage does not mutate its input?
Memory hook
A variable name is a label tied to a balloon. b = a ties two strings to the same balloon; poking the balloon through either string pops it for everyone.
Assignment never copies. If you need independence, copy explicitly.
Self-check
After
b = a; a = [], what doesbcontain?The original list. Rebinding
aonly moves thealabel;bstill points at the original object.Name two ways to get an independent shallow copy of a flat list.
list(a)ora[:]. For nested data usecopy.deepcopy(a).What does
id()return in CPython?The memory address of the object. Two names with the same
id()share one object.What is the difference between
lst.clear()andlst = []?lst.clear()empties the original object in-place; every name bound to it sees an empty list.lst = []rebinds onlylst; other names retain the original contents.
In a script: create a = [[1], [2], [3]]. Bind b = a. Append 99 to a[0]. Print b[0] and b[0] is a[0]. Then set a = [[10], [20], [30]] and print b[0] again and a is b. Paste both outputs, then in one sentence explain why b[0] changed in the first step but b kept the original list after the second step.