Writing Tests, and Fixing a Regression

By the end of this lesson you can write checks that state what a function must do, choose cases that genuinely catch mistakes, and add the specific test that stops a bug you just fixed from coming back.

Everything in Module 4 was arranged for this. Pure functions with clear contracts are testable; a script is not.

The immediate payoff is confidence in change. Once the shortlist rule has tests, you can alter it, or let an assistant alter it, and find out in one second whether anything else moved. Without them, every change is a small act of faith, and the cost of that compounds until people stop changing things at all.

There is a second payoff that arrives sooner than expected. Writing the cases forces you to decide what the function should do at the edges, and that decision is usually the part nobody had made. Half the bugs found while writing tests are found before the test is finished.

A test is a statement about behaviour that fails loudly when it stops being true. The smallest form is assert:

assert qualifies(record) is True, "a clearly qualifying record should qualify"

The message after the comma is not decoration. When the assertion fails, that text is what appears, so writing "a record worth exactly the minimum should qualify" turns a failure into a diagnosis. Write the message as the sentence you would say to a colleague, not as a restatement of the code.

Choose cases from four categories, and aim for at least one of each:

• Success: an unambiguous, ordinary input.
• Boundary: exactly on the threshold, one below, one above. Most rule bugs live here.
• Malformed: a missing field, an empty value, text where a number belongs.
• Regression: the exact input that caused a bug you already fixed.

A fixture is a named example input reused across cases. Giving one MATCH, another BOUNDARY, and another NO_CATEGORY makes each assertion read as a sentence and keeps the data out of the assertions.

When something breaks, the loop is: run the tests, read the failing message, form one hypothesis, make the smallest change, run again. If a previously passing test now fails, you have caused a regression, and the correct response is to undo the change rather than to adjust the test to match the new behaviour. A test edited to make it pass has stopped protecting anything.

assert is enough for short exercises and this in-page runner. On your own machine, pytest collects functions named test_... from files named test_...py, runs them all, and reports which failed, and the application in this repository uses Vitest for the same purpose in TypeScript. The tool changes; the four categories do not.

def response_days(record, default=None):
    """Return the response window in days, or default when it cannot be
    computed. Never raises."""
    posted = record.get("posted_days_ago")
    closes = record.get("closes_in_days")
    if posted is None or closes is None:
        return default
    return posted + closes


OPEN = {"posted_days_ago": 4, "closes_in_days": 26}
CLOSING_TODAY = {"posted_days_ago": 30, "closes_in_days": 0}
NO_CLOSE_DATE = {"posted_days_ago": 4}
EMPTY = {}

assert response_days(OPEN) == 30, "a normal record should sum both windows"
assert response_days(CLOSING_TODAY) == 30, "a zero closing window is still a valid number"
assert response_days(NO_CLOSE_DATE) is None, "a missing closing date has no computable window"
assert response_days(EMPTY) is None, "an empty record has no computable window"
assert response_days(EMPTY, default=0) == 0, "the caller's default should be honoured"

print("all 5 checks passed")

Five assertions covering four categories. OPEN is the success case. CLOSING_TODAY is a boundary: zero is a legitimate value and must not be confused with absence. NO_CLOSE_DATE and EMPTY are malformed inputs. The last case pins down the default parameter, which is part of the contract and therefore something a future change could break without touching any other assertion.

The CLOSING_TODAY case is the one worth stealing. Zero and None are both falsy, so a lazy implementation using if not closes: would treat a genuine zero as missing, and only a test that supplies zero on purpose would ever notice.

Predict the output of the program above.

all 5 checks passed

Change the two lines that read the record to posted = record.get("posted_days_ago") or 0 and closes = record.get("closes_in_days") or 0, and delete the if that returns the default. Predict which assertions now fail.

AssertionError: a missing closing date has no computable window

A colleague reports that the shortlist has started including closed opportunities. The rule was changed yesterday to add an urgency check, and the change came with a test.

def qualifies(record):
    urgent = record.get("closes_in_days", 99) <= 7
    active = record.get("status") == "Active"
    return urgent or active


CLOSED_URGENT = {"closes_in_days": 3, "status": "Closed"}

assert qualifies(CLOSED_URGENT) is True, "urgent records should be flagged"

print("checks passed")

return urgent and active

CLOSED_URGENT = {"closes_in_days": 3, "status": "Closed"}
OPEN_URGENT = {"closes_in_days": 3, "status": "Active"}

assert qualifies(CLOSED_URGENT) is False, "a closed record should never be shortlisted"
assert qualifies(OPEN_URGENT) is True, "an active record closing soon should be shortlisted"

A written assertion suite over the shortlist rule. One check fails. Read the message, repair the function, and confirm the rest still pass.

Loading this exercise…

Optional: add three more cases to the suite. A record one unit below the minimum, a record whose estimated_value is the string "310000" rather than a number, and a record whose set-aside is "total small business" in lower case. Decide what each one should do before you run it, then run it. At least one is likely to disagree with you, and that disagreement is a real design decision about normalisation that the capstone has to settle.

Sign in to track your progress on this exercise.

1. Why does an assertion message matter more than the assertion itself when something fails?
2. Name the four categories of test case this lesson recommends.
3. What is a regression test, and when do you write one?
4. What is wrong with editing a test so that it passes?

Sign in to track your progress on this exercise.

Tests state behaviour, messages diagnose failures, four categories of case catch most defects, a regression test is part of a fix, and a test edited to pass protects nothing. Module 6 gives the assistant real input: files, JSON, CSV, and the normalisation that turns published text into records these tests can rely on.