Profiling, Complexity, and Memory: Production Python Systems

Outcome

Interpret a profile and replace repeated linear lookup with an indexed representation.

Why it matters

At this level, code is judged by how safely it changes under load, failure, and team ownership. The technique in this lesson makes an important boundary visible enough to test and review.

Concept

Complexity predicts growth; profiling shows where this workload spends time. Measure representative inputs, separate CPU from waiting, and include peak memory in the decision.

Read the code

def join_rows(agencies, awards):
    agency_by_id = {row["id"]: row["name"] for row in agencies}
    return [
        {"agency": agency_by_id.get(row["agency_id"], "Unknown"), "value": row["value"]}
        for row in awards
    ]

print(join_rows([{"id": 2, "name": "Transit"}], [{"agency_id": 2, "value": 8}]))

Read from the public behavior inward: identify the input boundary, the decision, and the observable result before studying syntax.

Predict the output

Predict the exact output before running the example.

Check your prediction

[{'agency': 'Transit', 'value': 8}]

It prints one joined record for Transit. Building the index once avoids scanning every agency for every award.

Modify the code

Change one valid input into the nearest invalid or overloaded case. Write down which layer should reject it and what the caller should observe.

Review the change

Keep the failure at the narrowest boundary that owns the rule. Preserve a stable return value or exception contract so callers do not need to inspect implementation details.

Debug the bug

A microbenchmark with tiny cached data can reward the wrong change. Use representative volume, warmup, multiple samples, and an end-to-end guardrail.

Try it yourself

Complete the focused implementation and run its deterministic checks.

Loading this exercise…

Practical challenge (optional)

Compare a nested-loop join and indexed join at three input sizes; report runtime and peak memory without claiming universal results.

Sign in to track your progress on this exercise.

AI collaboration

Checkpoint

What different questions do complexity analysis and a profiler answer?
What time-and-memory tradeoff does the agency index introduce?
Why can a tiny, warm-cache microbenchmark recommend the wrong change?

Answers

Complexity predicts how resource use grows; profiling measures where a representative run actually spends its resources.
It replaces repeated linear scans with average constant-time dictionary lookup while retaining an additional dictionary in memory.
It may omit realistic volume, I/O, allocation, cold-start, and end-to-end costs, so the measured bottleneck differs from production.

Sign in to track your progress on this exercise.

Summary and next step

You made the boundary explicit, predicted its behavior, tested a deterministic implementation, and examined its failure mode. Continue to the next lesson to combine this technique with a wider application or production constraint.