Framing a Task So the Result Can Be Checked

By the end of this lesson you can turn a vague request into a task with acceptance criteria specific enough that anyone, including you three weeks from now, can verify the result rather than judge it.

"Improve the matching rule" produces something. Whether it is right is then a matter of opinion, arrived at by reading the whole diff and forming an impression, which is slow and unreliable and gets worse as the change gets larger.

The same request with criteria attached, "a record with no published amount must return incomplete rather than no match, and the four existing tests must still pass", produces something you can check in seconds. The criteria did the review before the work started.

This is not really about AI. It is how a well-written ticket differs from a badly written one, and it has always paid. It simply matters more when the thing doing the work is fast, confident, and willing to fill any gap in the specification with a guess.

A well-framed task has five parts:

1. The goal, in one sentence, saying what should be true afterwards.
2. The boundary: which files may change, and which must not.
3. Acceptance criteria: specific, checkable statements, ideally as assertions.
4. The constraints: what must not change, what must keep passing, what patterns to follow.
5. The evidence required: the diff, the test output, an explanation in plain language.

A criterion is verifiable when two people would agree whether it is met without discussing it. "Handle missing data gracefully" is a judgement. "verdict({'amount': None}) returns 'incomplete'" is a criterion. Write them as assertions where you can, because then checking is running them.

The best time to write them is before the work exists. Criteria written afterwards describe what was built, which is how the test in Module 5 came to assert that a closed opportunity should be shortlisted.

Context is the second half. Supply the file being changed, the tests that must keep passing, the shape of the data, and the conventions to follow. Do not supply the whole repository, real records, credentials, or anything restricted. Too little context produces invented APIs; too much buries the request and increases what you must review.

Size is the third. A task is too large when you cannot state its criteria, when it touches parts of the system you cannot check, or when you would not be able to review the diff carefully. The right response to a large task is to split it, not to accept a change you will approve by scrolling.

One test of your own framing: if you cannot write the acceptance criteria, you do not yet know what you want, and no assistant can supply that.

A poorly framed request:

Make the matching rule better at handling edge cases.

The same work, framed:

Goal. verdict(record) must distinguish a record that cannot be judged from one that does not match.

Boundary. Change matching.py only. Do not change report.py or any fixture.

Acceptance criteria.

assert verdict({"set_aside": "Total Small Business"}) == "incomplete"
assert verdict({"amount": 310000}) == "incomplete"
assert verdict({"amount": 42000, "set_aside": "Total Small Business"}) == "no match"
assert verdict({"amount": 310000, "set_aside": "8(a)"}) == "no match"
assert verdict({"amount": 310000, "set_aside": "Total Small Business"}) == "match"

Copy

Constraints. The existing tests in test_matching.py must still pass. Keep the current function signature. No new dependencies.

Evidence. Show the diff, the full test output, and two sentences on what changed and why.

The second version is longer to write and shorter to check. Every criterion is executable, so "is it done" is answered by running them rather than by reading.

The boundary matters as much as the criteria. Without it, a plausible way to satisfy every assertion is to change the caller instead of the function, and the diff would still be green.

Suppose an assistant returns this implementation for the framed task. Predict which criteria pass.

def verdict(record, minimum=100000):
    if not record.get("amount"):
        return "incomplete"
    if record["amount"] < minimum:
        return "no match"
    if record.get("set_aside") != "Total Small Business":
        return "no match"
    return "match"

AssertionError

Add a sixth criterion to the framed task: assert verdict({"amount": 0, "set_aside": "Total Small Business"}) == "no match". Predict what the implementation above now does.

A task was framed like this:

Add a --since option so the report only includes notices posted on or after a date. Update the tests.

The assistant returned a diff that adds the option, adds a test, and also changes DEFAULT_LIMIT from 25 to 100, reformats report.py, and renames fetch_shortlist to get_shortlist with the call sites updated. Every test passes.

Implement verdict to satisfy exactly the five criteria below, no more and no less. You are on the receiving end of the framed task from this lesson.

Loading this exercise…

Optional: take the last change you made in this course and write the task you would have handed to someone else: goal, boundary, acceptance criteria as assertions, constraints, and required evidence. Then check whether your actual change satisfies every criterion you wrote. Writing the specification for work you have already done is the fastest way to discover which parts you never actually decided.

Sign in to track your progress on this exercise.

1. Name the five parts of a well-framed task.
2. What makes an acceptance criterion verifiable rather than a judgement?
3. Why should criteria be written before the work rather than after?
4. What are the two signs that a task is too large?

Sign in to track your progress on this exercise.

Goal, boundary, criteria, constraints, evidence; write the criteria as assertions and write them first; supply the context the task needs and nothing private; and split anything you could not review. Next: reading the diff that comes back, and finding the change nobody asked for.