What an Unattended Job Needs From a Service

By the end of this lesson you can decide, before a job makes its first request, which timeout contract each call needs, how many requests the run may start, and when it must stop starting new calls. You can implement a check that names whether the request budget or the pre-request deadline refused the next call. This check does not itself stop a call already in flight or bound total job duration.

A service you call from a script you are watching is a different thing from a service you call from a job that runs at 3am.

When you are watching, a slow response is annoying and you press Ctrl+C. Unattended, the same slow response is a process that sits there. If a scheduler allows overlapping runs every fifteen minutes, one hung call can leave the next run starting alongside it. Repeated overlaps can exhaust machine resources.

An absent or unsuitable timeout is one way this happens. Client defaults differ: some require an explicit timeout, while others supply one. Check the actual transport contract instead of assuming a default.

The second failure is quieter. A job retries a failing service politely, backing off, and eventually succeeds after two hundred requests. Nothing broke and nothing was reported. Meanwhile the service's owner is looking at a traffic graph wondering who is doing that, and your run took forty minutes instead of forty seconds.

Timeout, budget, and pre-request deadline address different parts of that risk. They do not by themselves impose a hard total-runtime limit on an in-flight request or the entire process.

Four decisions, made once, at the top of the job.

1. A timeout on every request. Not on the session, not on the retry loop — on the call. Two numbers are worth separating: how long to wait for the connection to be established, and how long to wait for the response once it is. A connect limit and a read-inactivity limit cover different stalls; library-specific timeout settings may still not impose a total response-download deadline.

Nothing in this course makes a real request, so the timeout here is a value carried in the run's settings and passed to whatever transport a learner uses later. Naming it in the settings is the habit; the library-specific spelling is documented by whichever library you end up using, and it is not the same in all of them.

2. A request budget for the whole run. Not a retry count, and not a page limit — a single number covering every request the run makes, whatever it makes them for. Retries spend it. Pagination spends it. A budget is what stops the pathological case where twenty pages each make five total attempts and nobody predicted a hundred requests.

3. A pre-request wall-clock deadline. Stop starting new calls at or after a chosen time, ideally before the next run is scheduled. This gate catches elapsed time that a request counter misses: fifty calls of forty seconds each fit a 50-request budget but take over half an hour. It cannot interrupt a call already in progress or guarantee that the job has finished by the deadline; the transport and scheduler need their own bounds.

4. What "nothing to do" looks like. A service that returns zero records is not broken; it is telling you there is nothing new. A job that treats an empty result as a failure will alert every weekend. A job that treats an error as an empty result will report a quiet, successful, empty run on the day the service is down, which is worse.

Refuse with the reason. Budget and deadline run out for different reasons and need different responses. A spent budget means the run started its allowed number of requests; investigate data volume, pagination, and retries. A passed deadline means the clock reached the configured gate; investigate request duration, waits, and other work. Reporting "stopped" for both leaves the person reading it no better off, so the check returns which one it was.

Injecting the clock. A deadline is a comparison against the current time, and a test that has to wait for real time to pass is a test nobody runs. Pass the clock in as a function — now() — and a test can hand it one that returns chosen instants. The worked code below exercises only the budget and pre-request deadline. The later illustrative fetcher also injects sleep so retry waits can be tested without real delays.

A budget and pre-request deadline, checked before each proposed request. A per-call timeout belongs to the transport contract and is not exercised by this loop:

def new_run(max_requests, deadline_at, timeout=5, max_attempts=3):
    """Illustrative run settings and a counter for requests started."""
    return {"max_requests": max_requests, "deadline_at": deadline_at,
            "timeout": timeout, "max_attempts": max_attempts, "used": 0}


def may_request(run, now):
    """(allowed, reason). The reason names which limit stopped the run."""
    if run["used"] >= run["max_requests"]:
        return False, f"request budget spent after {run['used']} requests"
    if now >= run["deadline_at"]:
        return False, f"deadline passed at {now:.0f}s"
    return True, ""


def spend(run):
    run["used"] += 1
    return run["used"]


clock = iter([0, 10, 20, 30, 40])
run = new_run(max_requests=3, deadline_at=35)
for now in clock:
    allowed, reason = may_request(run, now)
    if not allowed:
        print("stop:", reason)
        break
    print("request", spend(run), "at", now)
print("used:", run["used"])

The budget is checked before the deadline as a reporting precedence when both have run out. That precedence identifies the first reported gate, not the root cause of a slow run. The worked loop never passes its timeout setting to a real client.

spend is separate from may_request so that the check has no side effect. A check that increments a counter cannot be called twice, and the first thing anyone does when debugging is call it twice.

The clock is an iterator here rather than a real clock, so this pre-request gate example is deterministic and finishes instantly.

The figure separates a timeout passed to an illustrative transport from the two checks this worked loop actually runs. It also separates a successful empty list from a client error. It does not show the worked clock’s answer.

Predict every line.

request 1 at 0
request 2 at 10
request 3 at 20
stop: request budget spent after 3 requests
used: 3

Change deadline_at to 25 and run it again.

request 1 at 0
request 2 at 10
request 3 at 20
stop: request budget spent after 3 requests
used: 3

An assistant was asked to "fetch the records, and retry if it fails". It produced this.

import time

def fetch_all(client):
    records = []
    page = 0
    while True:
        try:
            batch = client.get(page)
        except Exception:
            time.sleep(5)
            continue
        if not batch:
            break
        records.extend(batch)
        page += 1
    return records

def fetch_all(client, run, now, sleep):
    records, page, attempts = [], 0, 0
    while True:
        allowed, reason = may_request(run, now())
        if not allowed:
            return records, reason
        spend(run)
        try:
            batch = client.get(page, timeout=run["timeout"])
        except TimeoutError:
            attempts += 1
            if attempts >= run["max_attempts"]:
                return records, f"gave up after {attempts} attempts"
            retry_now = now()
            allowed, reason = may_request(run, retry_now)
            if not allowed:
                return records, reason
            remaining = run["deadline_at"] - retry_now
            if remaining <= 0:
                return records, "deadline passed before retry wait"
            sleep(min(2**attempts, remaining))
            continue
        if not isinstance(batch, list):
            return records, "bad response"
        if len(batch) == 0:
            return records, "complete"
        records.extend(batch)
        page += 1
        attempts = 0

Write may_request, which decides whether a run may make another request. It is exercised against two supplied scenarios: one that runs out of budget and one that runs out of time. The reason must name which limit stopped it.

Loading this exercise…

Optional, and the transfer task for this lesson: work out your own three numbers.

For a job you would actually schedule, write down the per-request timeout, the whole-run request budget, and the deadline, and — this is the part that makes it useful — write one sentence per number saying what you would do if the job started hitting it every day.

Sign in to track your progress on this exercise.

1. Why is a per-request timeout more important for a scheduled job than for a script you are watching?
2. What does a whole-run request budget catch that a retry count does not?
3. What does a deadline catch that a budget cannot?
4. A job reports "0 records, run complete" every day for a week and nobody notices anything wrong. Name two very different explanations.

Sign in to track your progress on this exercise.

Before the first call: a timeout on every request, a request budget covering the whole run, a pre-request deadline before the next planned run, and a decision about what an empty result means. Check the budget and the deadline before each request, keep the check free of side effects, and return which limit stopped the run. Inject the clock and sleep for fast retry tests; separately bound any in-flight call and the job’s total runtime. Next: where the responses come from when there is no network — recorded exchanges, replayed through the same code path a real call would take.