Capstone: The Order Digest
Capstone Part Three: Handing It To Somebody Else
Packaging the workflow so it runs on a machine that is not yours, and writing the runbook that answers the four questions somebody has at three in the morning without you.
Lesson 18 of 18 in the recommended order · About 25 min (estimate)
On this page
Workflow automation glossary — terms and common confusions
- Dry run
- A run that reports every change it would make and makes none of them.
A dry run that still writes a log file, sends a message, or creates a folder is not a dry run.
- Idempotent
- Running it again leaves the same result as running it once.
"It did not crash the second time" is not the same property; check what the second run changed.
- Run record
- The stored facts about one execution: when it started, what it read, what it changed, how it ended.
A run record is not the log. The log is prose for a person; the record is data the next run reads.
- Transient failure
- A failure that a later identical attempt could succeed at, such as a timeout.
A rejected record and a dropped connection are not the same failure, and retrying the first one forever is a bug.
- Backoff
- Waiting longer between successive retries instead of retrying immediately.
Backoff without a maximum attempt count is an unbounded loop with a politeness delay.
- Request budget
- A hard cap on how many requests one run may make, checked before each request.
A page limit is not a budget if a retry can make extra requests the limit never counts.
- Quarantine
- Setting aside a record a run could not process, with the reason, so the run continues.
Quarantine is not "skip". A skipped record leaves no trace; a quarantined one is countable and reviewable.
- Reconciliation
- Showing that the counts in a report add up: read equals processed plus quarantined plus rejected.
A report whose totals cannot be reconciled is a summary of what the code believed, not of what happened.
- Trigger
- The event or time that causes a run to start.
Scheduling a trigger is a decision made on a machine, not something a browser lesson can install for you.
- Heartbeat
- A signal a healthy run emits, whose absence is itself the alert.
Alerting only on errors cannot detect the job that stopped running at all, which is the most common outage.
- Staleness
- How old the most recent successful run is, compared with how old it is allowed to be.
A green last run is not freshness. Ask when it ran, not whether it passed.
- Atomic replace
- Writing output to a temporary name and renaming it into place in one step.
A reader never sees a half-written file; an interrupted run leaves the previous output intact.
- Transport
- The injected callable that actually performs a request, separate from the client that interprets it.
A client that builds its own connection cannot be tested without a network, which is why the seam exists.
- In-memory filesystem
- The filesystem this course’s browser exercises operate on, which lives only inside the tab.
It behaves like a filesystem and is not your disk: nothing an exercise writes exists after the run ends.
Outcome
By the end of this lesson your project runs on a machine that is not the one it was written on, and it carries a runbook that answers, without you, how to run it, how to schedule it, how to tell whether last night worked, and what to do when it did not.
Why it matters
The handover test is simple and unforgiving: a colleague clones the project onto a clean machine and gets it running, scheduled, and monitored, without asking you anything.
Almost every project fails it the first time, and the reasons are dull rather than clever. An absolute path to a folder that exists only on the author's laptop. A dependency installed years ago and never written down. A settings value that lives in the author's shell profile. A schedule that was created by hand through a graphical interface and exists nowhere in the repository. Knowledge that the job must not be run before 06:30, held only in the author's head.
The consequence is not that the colleague is stuck. It is that you are the only person who can operate this thing, forever, including on the nights you are not available — which means the job is not really automated. It has a dependency on you.
A runbook is short. Four questions, a page, and it is worth more to whoever is woken up than every comment in the code.
Concept
What the project must carry.
- Every path in a settings file, none in the code. The inbox, the archive, the output folder, the state file. One file, read once, at the entry point.
- Pinned dependencies. A requirements file with exact versions and a note on which Python version it was run against. "It needs pandas" is not a dependency list.
- The fixtures and the tests. The recorded responses and the sample inputs are part of the project, not scratch files. Somebody must be able to run the tests before touching anything.
- No absolute paths anywhere in the source. They are the single most common reason a project runs for its author and nobody else.
- The schedule, as a command. A task created by clicking through a graphical interface exists on one machine and in nobody's memory. The
schtasksorcronline belongs in the runbook, where it can be read, copied, and diffed.
The runbook, four questions.
- How do I run it? The exact command, from the project root, with the dry run first. Include what a successful run prints.
- How do I schedule it? The literal command that creates the schedule, the interval, and the overlap policy. Also how to remove it, because somebody will need to.
- How do I know last night worked? Where the run record is, what a good one looks like, and the staleness rule — how old the most recent success may be before somebody should care.
- What do I do when it did not? A short list of the failures that have actually happened, each with its symptom and its recovery step. Two entries written from real incidents are worth more than twenty imagined ones.
Then two more that are not questions but belong on the page: what it changes (which folders it writes to, what it never touches) and what it is not — a plain statement that the report is a reporting aid rather than advice, so nobody downstream mistakes the output for a decision.
Make the runbook checkable. A document nobody can fail is a document that quietly rots. Keep the required section names in a list and check the file against it in the test suite: every required heading present, none of them empty, and the non-advice statement there. That check takes fifteen lines and it is why the runbook still describes reality in a year.
The rubric. The capstone in this course is measured against the acceptance criteria in capstone.yaml, phase by phase. They are written so two people would agree on whether each one holds — "the survey output is byte-identical across two runs" rather than "the survey is reliable". Work through them as a checklist, and where one does not hold, that is a defect list rather than a grade.
Read the code
Checking documentation the same way you check code:
REQUIRED = ["how to run", "how to schedule", "how to tell it worked", "how to recover"]
DISCLAIMER = "reporting aid"
def review(sections, required, min_words=12):
"""What is missing, what is too thin to be useful, and whether the statement is there."""
missing = [name for name in required if name not in sections]
thin = [
name
for name in required
if name in sections and len(sections[name].split()) < min_words
]
has_disclaimer = any(DISCLAIMER in text for text in sections.values())
return sorted(missing), sorted(thin), has_disclaimer
DRAFT = {
"how to run": "Run python run.py from the project root. It defaults to a dry run; add --apply to write.",
"how to schedule": "Use Task Scheduler.",
"how to tell it worked": "The run record in state/last-run.json has status completed and counts that reconcile.",
}
missing, thin, disclaimer = review(DRAFT, REQUIRED)
print("missing:", ", ".join(missing) or "none")
print("thin:", ", ".join(thin) or "none")
print("disclaimer:", "present" if disclaimer else "absent")
print("verdict:", "ready" if not missing and not thin and disclaimer else "not ready")
missing and thin are different findings and are reported separately: a section that is absent has been forgotten, while one that is three words long has been written by somebody who knew the answer and did not think it needed saying. The second is the more common failure and the harder one to notice.
min_words is a crude measure and it is honest about being one. It cannot tell whether a section is any good; it can tell that "Use Task Scheduler" is not an instruction.
Predict the output
Predict all four lines.
Check your prediction
missing: how to recover
thin: how to schedule
disclaimer: absent
verdict: not ready
Three findings from a runbook that looks, at a glance, mostly written. "How to recover" is absent entirely — the section people write last and need most. "Use Task Scheduler" is four words, which passes any check for presence and tells a reader nothing they did not already know. And no section carries the statement about what the report is not.
Modify the code
Drop the thin check, keeping only missing and the disclaimer.
What changes, and why
missing: how to recover
thin: none
disclaimer: absent
verdict: not ready
The verdict is unchanged, and the check has lost the finding it was most useful for. Add a recovery section reading "See the code" and a line containing the disclaimer, and this reports ready over a runbook that answers nothing.
Presence checks are easy to satisfy and easy to satisfy dishonestly, without anybody intending to be dishonest: somebody filling in a template writes a placeholder, means to come back, and does not. A crude length floor catches most of that, and the reason it is worth having despite being crude is that the alternative is a check that always passes.
Debug the bug
A project was handed over with this at the top of its only Python file, and a README saying "run the script".
INBOX = "C:\\Users\\dana\\Documents\\orders"
ARCHIVE = "C:\\Users\\dana\\Documents\\orders\\archive"
OUTPUT = "C:\\Users\\dana\\Desktop\\digest.txt"
BUDGET = 40
if __name__ == "__main__":
run(INBOX, ARCHIVE, OUTPUT, BUDGET, apply_changes=True)
What's actually wrong
- Three absolute paths naming one person's machine. Nobody else can run this at all, and the failure is a
FileNotFoundErrorthat tells the reader nothing about what to change. - The output goes to a desktop. A scheduled task running as a service account has no desktop, so the job fails under the scheduler while working perfectly when run by hand — which is the most confusing failure mode in this entire course.
apply_changes=Trueis hard-coded, so there is no way to run this without it changing things. The dry run built in the first module exists and cannot be reached.- The archive is inside the inbox. Whether that matters depends entirely on whether the collection step descends into subfolders, and nothing here says which — so the first run either archives correctly or archives its own archive, forever.
- No settings file, no requirements file, no tests, no runbook, and "run the script" as documentation.
- The schedule exists nowhere. It was created by hand on Dana's machine, and when that machine is rebuilt the job stops running and nothing notices — the exact outage the previous module described.
The handover-ready version:
order-digest/
run.py reads settings.toml, calls the pipeline, exits 0 / 1 / 2
settings.toml inbox, archive, output, state, budget, deadline, interval
requirements.txt pinned, with the Python version noted at the top
fixtures/ recorded inputs the tests run against
tests/
RUNBOOK.md how to run, schedule, check, recover; what it changes; what it is not
with run.py taking --settings so a second environment is a second file rather than an edit, and defaulting to the dry run so the dangerous mode has to be typed.
Try it yourself
Write review, which checks a runbook against a required-section list. It is run against two supplied runbooks: a draft with gaps and a finished one.
Loading this exercise…
Practical challenge (optional)
Optional, and the transfer task for this lesson, and the last one in this course: run the handover test for real.
Give the project to somebody else — or to yourself, on a different machine or in a fresh folder — with nothing but the repository and the runbook. Watch them, and write down every question they ask. Each question is a defect in the runbook, not in the person.
What a good answer looks like
The questions cluster. Most people are asked "which Python?" and "where does it write?" within the first two minutes, and both are settings-file answers that the runbook assumed.
The question worth waiting for is the fourth or fifth: something like "how would I know if this had stopped working?" If the runbook cannot answer it in one sentence naming a file and a threshold, the monitoring from the previous module exists in the code and not in anybody's hands.
And the honest observation about this whole exercise: you will not find these by re-reading your own runbook, because you cannot un-know what it does not say.
Sign in to track your progress on this exercise.
AI collaboration
Checkpoint
- What is the handover test, and what does failing it mean about the job?
- Why does an absolute path to a desktop folder break under a scheduler but work by hand?
- Why report a missing section and a thin section as separate findings?
- Why must the schedule itself live in the repository rather than only in the scheduler?
Answers
- Somebody else gets the project running, scheduled, and monitored from the repository and the runbook alone, without asking you anything. Failing it means the job still depends on you, including on the nights you are not available, which means it is not automated.
- A scheduled task running as a service account has no desktop and often no user profile at all, so the path does not exist for it. Run by hand it resolves fine, which makes it work in every test somebody performs by hand.
- They are different failures with different causes. A missing section was forgotten; a three-word section was written by somebody who knew the answer and did not think it needed saying, which is the more common and less visible problem.
- Because a task created by hand exists on one machine and in nobody's memory. When the machine is rebuilt the schedule is gone, the job stops running, and nothing fails — which is precisely the outage that goes unnoticed for months.
Sign in to track your progress on this exercise.
Summary and next step
Paths and thresholds in a settings file, dependencies pinned, fixtures and tests in the repository, the schedule written as a command, and no absolute path anywhere in the source. A runbook that answers how to run it, how to schedule it, how to tell whether last night worked, and what to do when it did not — plus what it changes and what it is not — checked in the test suite against a required-section list so it cannot quietly rot. Work the capstone acceptance criteria as a checklist and treat anything that does not hold as a defect list. That is the whole course: a job that runs without you, that you can hand to somebody else, and that says so honestly when it did not work.