Pagination, Rate Limits, and Retries That Stop

By the end of this lesson you can collect a paged result set to completion when its contract holds, report an incomplete result when your page cap is reached first, and explain why real requests also need timeouts. You can retry a failing request in a way that helps rather than compounds the problem.

For example, a service with page size 100 might return only the first hundred of 240 matching notices and indicate that more pages are available. A client must follow that service's paging contract to get the rest.

Both halves of this lesson are about the same discipline: bound the number of calls to somebody else's service without trusting it to end the loop for you. A paging loop that trusts the server to eventually return a short page, or a retry loop that keeps trying until it succeeds, can overload that service. The page cap bounds call count; a real network client also needs a per-request timeout so one call cannot hang indefinitely.

Nothing here makes a network call. The stand-in fetch_page returns fixture data, which is how every exercise and every capstone test in this course runs.

Paging comes in two common shapes. Offset-based paging takes limit and offset: ask for 100 starting at 0, then 100 starting at 100. Cursor-based paging returns an opaque token to send with the next request, and ends when no token comes back. Offset paging is easier to reason about and can skip or repeat records if the underlying data changes between requests; a well-designed cursor is often more stable under those changes, but its guarantees depend on the service and you usually cannot jump to an arbitrary page.

Use the stop signals defined by the provider contract, and enforce your own page cap in either style:

• For the offset fixture below, a page shorter than the requested limit marks the end when its count agrees with the reported total.
• For an offset service that reports an exact total, stop after collecting that many records, even if its last page is full. A conflicting short page and total need an error, not a claim of completion.
• For a cursor service, stop when its contract says no next cursor means completion; do not apply that rule to an offset response.
• In either style, stop at your own maximum page count and report an incomplete result.

The page cap is the guard that holds when the server keeps advertising more data. Give it a value derived from what you expect, and treat hitting it before completion as an error, not as a normal ending.

Rate limits are the service telling you how often it will answer. They usually arrive as headers such as a remaining-requests count and a reset time, and are enforced with 429. Read the headers when they exist and slow down before you are told to. On a retryable response, honour a valid Retry-After within your caller-owned wait/deadline budget; report when that wait is beyond the budget rather than sleeping without bound.

Retries need three things: a bounded attempt count, a growing delay, and a rule about what is retryable. Growing the delay, often doubling it, is backoff, and it exists because a service returning 503 may be overloaded or temporarily unavailable, so retrying immediately can make recovery harder. Adding a small random amount to each delay, called jitter, stops many clients that failed at the same moment from retrying in unison.

And retry only what Module 7's first lesson and the provider's contract classify as retryable. The Debug repair below demonstrates attempt bounds, basic backoff and status classification; it does not yet parse Retry-After, add jitter or enforce a total wait deadline. A 403 is normally non-retryable unless the provider explicitly documents a transient case; repeating it blindly is usually just more refusals and a slower error message.

ALL_RECORDS = [{"noticeId": f"N-{n}"} for n in range(1, 8)]


def fetch_page(offset, limit):
    """Stand-in for one API call. No network."""
    window = ALL_RECORDS[offset : offset + limit]
    return {"totalRecords": len(ALL_RECORDS), "limit": limit, "offset": offset, "results": window}


def fetch_all(page_size=3, max_pages=10):
    """Return (records, requests, error). Never makes more than max_pages calls."""
    records = []
    requests = 0
    offset = 0

    while requests < max_pages:
        page = fetch_page(offset, page_size)
        requests += 1
        records.extend(page["results"])
        if len(page["results"]) < page_size:
            if len(records) != page["totalRecords"]:
                return records, requests, "short page conflicts with reported total"
            return records, requests, None
        if len(records) >= page["totalRecords"]:
            if len(records) != page["totalRecords"]:
                return records, requests, "page exceeds reported total"
            return records, requests, None
        offset += page_size

    return records, requests, f"stopped at the {max_pages}-page limit"


records, requests, error = fetch_all()
print(len(records), requests, error)

The loop condition is the page cap, not "while there is more". That inversion is the whole design: the normal endings are return statements inside the loop, and falling out of the bottom means something was wrong.

The fixture reports an exact total, so a short final page and the collected count must agree. An exact full final page can end by reaching the total instead. A contradictory short page or an overrun returns an error; this miniature still assumes the server honours the requested limit and does not validate duplicate record IDs.

records.extend(...) adds every item of the page to the list, unlike append, which would add the page itself as a single nested item. Mixing those two up produces a list of lists and a count that is suspiciously equal to the number of requests.

Predict the single printed line.

7 3 None

Call fetch_all(page_size=3, max_pages=2) and predict the printed line.

6 2 stopped at the 2-page limit

An assistant was asked to add retries to a client. It produced this and said it retries transient failures.

import time


def get_with_retry(send, url):
    while True:
        response = send(url)
        if response["status"] == 200:
            return response["body"]
        time.sleep(1)


calls = {"count": 0}


def always_unavailable(url):
    calls["count"] += 1
    return {"status": 503, "body": None}

RETRYABLE = {429, 500, 502, 503, 504}


def get_with_retry(send, url, attempts=4, base_delay=0.5):
    """Return (body, error). Bounded attempts, doubling delay."""
    delay = base_delay
    for attempt in range(attempts):
        response = send(url)
        if response["status"] == 200:
            return response["body"], None
        if response["status"] not in RETRYABLE:
            return None, f"not retryable: status {response['status']}"
        if attempt < attempts - 1:
            time.sleep(delay)
            delay *= 2
    return None, f"still failing after {attempts} attempts"

Walk a fixture-backed paged source to completion, under a hard page cap, and report both totals.

Loading this exercise…

Optional: make fetch_page return a full page every time, regardless of offset, simulating broken paging. Run your loop and confirm the cap stops it with the reported error, not a success count. Deliberately breaking the thing you depend on, and confirming your guard holds, is a test worth writing for anything that talks to a system you do not control.

Sign in to track your progress on this exercise.

1. For this offset fixture, which normal signals end paging? What signal ends a cursor traversal under its contract, and what should the shared page cap report if reached first?
2. Why is the page cap the loop's condition rather than a fallback inside it?
3. What is backoff, and why does a fixed delay make an overloaded service worse?
4. Why should hitting the page cap be reported rather than returned quietly?

Sign in to track your progress on this exercise.

Offset paging here ends on an agreed short page or exact total; cursor clients use their own token rule. Your local page cap reports incomplete results, while per-request timeouts prevent a single remote call from hanging. Retries need bounded attempts, growing delays and a retryable check. Next: reading the contract these rules come from, and assembling them into a client boundary that runs against stored fixtures.