Plan an Automation
Mapping a Workflow Before You Automate It
Turning a described piece of routine work into a map of trigger, steps, and handoffs, so the decision about what to automate is made on paper rather than discovered halfway through writing the script.
Lesson 1 of 18 in the recommended order · About 25 min (estimate)
On this page
Workflow automation glossary — terms and common confusions
- Dry run
- A run that reports every change it would make and makes none of them.
A dry run that still writes a log file, sends a message, or creates a folder is not a dry run.
- Idempotent
- Running it again leaves the same result as running it once.
"It did not crash the second time" is not the same property; check what the second run changed.
- Run record
- The stored facts about one execution: when it started, what it read, what it changed, how it ended.
A run record is not the log. The log is prose for a person; the record is data the next run reads.
- Transient failure
- A failure that a later identical attempt could succeed at, such as a timeout.
A rejected record and a dropped connection are not the same failure, and retrying the first one forever is a bug.
- Backoff
- Waiting longer between successive retries instead of retrying immediately.
Backoff without a maximum attempt count is an unbounded loop with a politeness delay.
- Request budget
- A hard cap on how many requests one run may make, checked before each request.
A page limit is not a budget if a retry can make extra requests the limit never counts.
- Quarantine
- Setting aside a record a run could not process, with the reason, so the run continues.
Quarantine is not "skip". A skipped record leaves no trace; a quarantined one is countable and reviewable.
- Reconciliation
- Showing that the counts in a report add up: read equals processed plus quarantined plus rejected.
A report whose totals cannot be reconciled is a summary of what the code believed, not of what happened.
- Trigger
- The event or time that causes a run to start.
Scheduling a trigger is a decision made on a machine, not something a browser lesson can install for you.
- Heartbeat
- A signal a healthy run emits, whose absence is itself the alert.
Alerting only on errors cannot detect the job that stopped running at all, which is the most common outage.
- Staleness
- How old the most recent successful run is, compared with how old it is allowed to be.
A green last run is not freshness. Ask when it ran, not whether it passed.
- Atomic replace
- Writing output to a temporary name and renaming it into place in one step.
A reader never sees a half-written file; an interrupted run leaves the previous output intact.
- Transport
- The injected callable that actually performs a request, separate from the client that interprets it.
A client that builds its own connection cannot be tested without a network, which is why the seam exists.
- In-memory filesystem
- The filesystem this course’s browser exercises operate on, which lives only inside the tab.
It behaves like a filesystem and is not your disk: nothing an exercise writes exists after the run ends.
Outcome
By the end of this lesson you can take a piece of routine work somebody describes to you in a paragraph and write it down as an ordered map: what starts it, what happens in order, who is responsible for each step, and the point where the work has to go back to a person. You can then say which part of that map is worth automating and defend the answer.
Why it matters
Most automations that get abandoned were not badly written. They automated the wrong thing.
The usual shape of the mistake: somebody automates the step that was annoying to do by hand, and the annoying step turns out to be the one that needed judgement. Now there is a script that makes a decision nobody reviews, and it is wrong in a way that only shows up in the quarterly numbers.
The other shape: the script does the easy part, and a person still has to open every file afterwards to finish the job. The work is not shorter, it is just split across two places.
Mapping first is cheap. It takes ten minutes and a text file, and it is the only point where changing your mind costs nothing. Once the map exists, "what should this automate?" stops being a matter of taste and becomes a question you can answer by pointing at a line.
Concept
A workflow map is an ordered list of steps. Each step names what happens and who is responsible for it. Two actors are enough: system, meaning a machine can do this from the information available, and person, meaning it needs a judgement, an approval, or knowledge that is not written down anywhere.
Four questions produce the map:
- What starts it? A time, a file arriving, a message, somebody asking. This is the trigger, and it is the first line of the map. A workflow with no identifiable trigger is not a workflow yet; it is a habit.
- What happens, in order? One line per step, in the order they actually happen, not the order they are usually described in.
- Who is responsible for each step?
systemorperson. Be strict about this. If a step needs somebody to weigh two things against each other, it is apersonstep even when a rule usually predicts the answer. - What comes out? The thing that exists at the end that did not exist at the start: a file, a message, a row in a table, a decision recorded somewhere.
The interesting line in any map is the handoff: the first person step. Earlier steps are candidates for automation only when the machine actually has the needed inputs and authority. At the handoff, this proposed unattended run pauses and produces something for a person to act on.
That gives a rule worth writing down:
Automate up to the handoff. Make the handoff easy. Do not automate through a person's decision without that decision and the required authorization.
"Make the handoff easy" is the part that gets skipped, and it is usually where the value is. If the automation stops and hands a person a list of four invoices with the reason each one needs attention, that person's job went from an hour to five minutes — without any step of theirs being replaced.
Two more things the map tells you before you write any code:
- Steps below the handoff stay pending in this unattended run. They can run only after the person's decision and any required authorization. A separate job can use the completed decision as its trigger; a designed workflow could instead pause and resume. This example sketches the separate-job option, not a universal architecture.
- A step with external effects is worth marking. Reading can also expose private information; writing, sending, deleting, and paying can require authorization and a recovery plan. Mark those boundaries before deciding what this run may do.
Read the code
Here is a workflow somebody described in a sentence: "Every month I check which memberships are about to lapse, work out which of those people are worth a personal note rather than the standard reminder, and then send the reminders."
Written down as a map:
RENEWALS = [
{"step": "First of the month, the membership export is refreshed", "actor": "system"},
{"step": "Find every membership expiring in the next 30 days", "actor": "system"},
{"step": "Group them by how long the person has been a member", "actor": "system"},
{"step": "Choose who gets a personal note instead of the standard reminder", "actor": "person"},
{"step": "Send the reminders", "actor": "system"},
]
def handoff_index(workflow):
"""Position of the first step a person is responsible for, or None."""
for position, step in enumerate(workflow):
if step["actor"] == "person":
return position
return None
position = handoff_index(RENEWALS)
print("steps in the map:", len(RENEWALS))
print("handoff at:", position)
print("handoff step:", RENEWALS[position]["step"])
print("automatable now:", position)
print("blocked until the decision:", len(RENEWALS) - position - 1)
handoff_index returns a position rather than the step itself, because the position is what splits the map into three parts: before, at, and after. Returning None for "no person step anywhere" is deliberate and is not the same as returning 0; a workflow with no human decision in it is a real and useful case, and confusing it with "the decision is the very first step" would put the split in exactly the wrong place.
Note what the last line counts. One step sits after the decision — sending the reminders. It is labelled system, but this proposed unattended job must not send before a person has chosen the list and sending is authorized. A separate send job could start from that completed choice, or a designed workflow could pause and resume. Neither action is implemented by this example.
Predict the output
Predict all five printed lines.
Check your prediction
steps in the map: 5
handoff at: 3
handoff step: Choose who gets a personal note instead of the standard reminder
automatable now: 3
blocked until the decision: 1
The three entries before the handoff are the scheduled refresh trigger, find, and group. The map counts them as a prefix; it does not prove this job itself controls the export refresh. The fourth entry is the person's choice. The fifth, sending, stays pending in this proposed unattended run until that choice and the required authorization.
Renewal map: prepare, decide, then wait to send
Step 1 · System
First of the month, the membership export is refreshed
Trigger in the map; this example does not prove the job owns the refresh
Step 2 · System
Find every membership expiring in the next 30 days
Candidate preparation
Step 3 · System
Group them by how long the person has been a member
Candidate preparation
Step 4 · Person
Choose who gets a personal note instead of the standard reminder
First human decision; this unattended run pauses
Step 5 · System
Send the reminders
Pending the person's choice and authorization; nothing is sent
Read the split
Prefix- 3 entries, including the refresh trigger
First person step- Index 3
After handoff- 1 send step pending
A separate authorized send job could follow the decision, or a designed workflow could pause and resume. The Python sample only maps the steps.
Modify the code
Change the fourth step's actor from "person" to "system", as though the rule "anyone who has been a member for more than five years gets a personal note" were good enough to encode. Run it again and read the last three lines.
What changes, and why
handoff at: becomes None, and the print on the next line raises:
TypeError: list indices must be integers or slices, not NoneType
The crash is the small problem. Handling None would report five system-labelled entries and no handoff, despite the described human choice. Those labels alone would not prove the job has inputs or authority. A map that says "no person is involved anywhere" is a claim, and it is worth being suspicious of when the workflow you were describing a minute ago clearly had somebody deciding something.
Encoding that rule is not automatically wrong. Sometimes a decision genuinely is a rule that nobody has written down, and writing it down is an improvement. What makes it wrong is doing it silently, so that the map no longer records that a judgement is being made. If you promote a person step to system, the rule you replaced them with belongs in the map, in writing, where somebody can disagree with it.
Debug the bug
Somebody asked an assistant to "map out our expense approval process so we can automate it". This came back.
EXPENSES = [
{"step": "Expense claim submitted", "actor": "system"},
{"step": "Check the receipt is attached", "actor": "system"},
{"step": "Approve claims under 200, flag the rest", "actor": "system"},
{"step": "Pay the approved claims", "actor": "system"},
]
It runs. Every function in the lesson accepts it. What is wrong with it?
What's actually wrong
There is no handoff, and there should be at least one. The map has quietly made three decisions on the organisation's behalf:
- "Approve claims under 200" is a
personstep wearing asystemlabel. Somewhere there is a policy about who may approve what, and this map has replaced it with a threshold that appears nowhere in the described process. Nobody agreed to that number in this map; it arrived with the draft. - "Pay the approved claims" moves money. A step that moves money with no person in the chain above it is the single most expensive thing a map can get wrong, and here it is two lines below an invented approval rule.
- Checking the receipt is the useful automatable task in the list. Submission is the trigger, not a checking task. A receipt check can produce a list somebody can act on, provided the system can read the needed fields accurately.
The corrected map:
EXPENSES = [
{"step": "Expense claim submitted", "actor": "system"},
{"step": "Check the receipt is attached and the total matches it", "actor": "system"},
{"step": "Approve the claim under the written approval policy", "actor": "person"},
{"step": "Pay the approved claims", "actor": "system"},
]
The handoff is now at position 2: two entries precede it (submission trigger and receipt check), and payment remains after the decision. The proposed automation can check receipts and hand an approver a list. Labelling payment system is conditional on a written payment-authorization policy and an approved decision; otherwise a person or separately authorized process must handle it.
The general form of this defect: an assistant asked to map a process for automation will tend to produce a map in which everything is automatable, because that is what the request implied it should find. The map is the artefact you are meant to argue with, so read every system label as a claim that needs defending, especially on any step that spends, sends, deletes, or approves.
Try it yourself
Two supplied workflow maps arrive as JSON, in WORKFLOW_JSON and NIGHTLY_JSON. Write plan, which splits a map at the handoff and returns three things: the steps before it, the handoff step itself (or None), and the steps after it. The second map has no person step anywhere, so both branches are checked.
Loading this exercise…
Practical challenge (optional)
Optional, and the transfer task for this lesson: map a workflow nobody has mapped for you.
Pick a piece of routine work you actually do — a weekly report, a set of files you tidy up, a recurring message you send — and write it as a map in the same shape, without looking back at the examples. Then answer three questions about your own map in writing:
- Where is the handoff, and what information would a person need at that moment to decide in under a minute?
- Which steps change something that cannot be undone?
- Which single step, if automated, saves the most time for the least risk?
What a good answer looks like
The third answer is almost never the step that annoys you most. It is usually a dull, exact, high-volume step: checking that something is present, matching one list against another, or renaming things consistently.
If your map has no person step anywhere, look again at the steps that produce a judgement — "work out which", "decide whether", "check it looks right". If your map has no step that changes something, it is probably a report, which is the safest possible automation to start with and a genuinely good first project.
Sign in to track your progress on this exercise.
AI collaboration
Checkpoint
- What are the four questions that produce a workflow map?
- What is the handoff, and why is it the interesting line?
- Why must a step after the handoff wait, and what could resume it?
- A generated map contains the step
{"step": "Pay the approved claims", "actor": "system"}and nopersonstep anywhere. What do you do?
Answers
- What starts it, what happens in order, who is responsible for each step, and what comes out.
- The first step a person is responsible for. Earlier steps are automation candidates after their inputs and authority are checked. The line separates preparation, the human decision, and work that must wait for that decision.
- They cannot run until the person has decided and the action is authorized. The completed decision could trigger a separate job, or a designed workflow could pause and resume; this unattended run does neither. Running through the handoff without the decision would silently make it on the person's behalf.
- Reject the map. A step that moves money with no person above it in the chain is the most expensive thing a map can get wrong, and a map with no handoff anywhere is a claim that no judgement exists in the process, which is worth disbelieving until somebody names the written policy it is following.
Sign in to track your progress on this exercise.
Summary and next step
A workflow map is an ordered list of steps, each with an actor, produced by asking what starts it, what happens, who does each step, and what comes out. The handoff is the first person step: automate up to it, make it easy, and do not automate through it. Next, the map becomes a contract — the exact inputs one run needs, the exact outputs it produces, and the conditions under which it refuses to start at all.