Invented APIs, Stale Advice, and What Never Gets Pasted

By the end of this lesson you can check a generated claim about a library or an API in a way that does not depend on the claim sounding right, and you can state exactly what never goes into a prompt regardless of how small it looks.

The failure modes in this lesson share one property: the output looks correct. A method that does not exist is named the way a real method would be. Advice about a library reflects a version that was current a while ago. A snippet is convenient to paste and happens to include a line you did not read.

None of these is caught by careful reading, because careful reading is what produces the impression that it is fine. They are caught by mechanical checks, and the checks are short.

Invented APIs. A generated call to a method that does not exist is the single most common failure, and it is well-named, plausibly argued, and confidently explained. Two checks:

• Run it. An AttributeError or a TypeError on the signature settles it in seconds.
• Look it up in the library's own current documentation, not in a summary of it.

dir(obj) and help(obj.method) in a Python session answer "does this exist and what does it take" directly, which is faster than reading an explanation of why it should.

Stale assumptions. An assistant's knowledge has a date, and libraries move. Deprecated arguments, renamed functions, changed defaults, and superseded security advice all read exactly like current information. Anything version-sensitive, and API details and security guidance especially, is worth confirming against the current official source before it goes in.

Insecure defaults. Generated code has recurring habits this course has already met: SQL built by concatenation, except Exception: pass, credentials in URLs, while True around a network call, verify=False on a request, secrets logged inside a configuration object. Each of them is the shortest way to write the thing. Check for them by name.

Dependencies. A suggested package is a decision: it needs a maintainer, a licence, a security history, and a reason. A generated import can also name a package that does not exist, which is a supply-chain risk in its own right, because someone may later publish something under that name. Confirm a package exists and is the one you meant before installing it.

Data privacy. These never enter a prompt, at any size, in any tool:

• Credentials, API keys, tokens, passwords, connection strings.
• Personal data about real people.
• Client, employer, or agency data you are not free to publish.
• Controlled or restricted procurement information.
• Internal hostnames, file paths, and infrastructure detail.
• Full tracebacks or configuration files without reading them first.

The working rule: reproduce the shape with invented values. Structure is what a technical question is about, and structure survives replacing every value. Two synthetic records explain a parsing bug exactly as well as two real ones.

When a snippet genuinely must be shared, run a redaction pass over it first, matching on the names of settings rather than on the shape of their values. A credential can look like anything; the settings that hold one are named predictably.

Finally, scope. This course does not require any paid AI account, and the capstone is completable without ever using an assistant. These are tools with failure modes worth knowing, not a dependency of the curriculum.

import json

CANDIDATE = """
data = json.parse(payload)
records = data.get_list("results")
"""

MARKERS = ("key", "token", "secret", "password")


def exists(module, name):
    """Mechanical check: does this attribute actually exist?"""
    return hasattr(module, name)


def redact(text):
    """Mask the value of any credential-shaped setting, keep the names."""
    lines = []
    for line in text.splitlines():
        name, separator, _ = line.partition(":")
        if separator and any(marker in name.lower() for marker in MARKERS):
            lines.append(f"{name}: REDACTED")
        else:
            lines.append(line)
    return "\n".join(lines)


print(exists(json, "parse"), exists(json, "loads"))
print(sorted(n for n in dir(json) if not n.startswith("_"))[:5])
print(redact("state: OR\ndata_source_key: sample-value-not-real\noutput: shortlist.csv"))

CANDIDATE is generated code of the kind that reads perfectly well. json.parse is what the equivalent is called in JavaScript, and a dictionary has no get_list method. Both are plausible and neither exists.

exists is the whole verification technique: hasattr answers the question the code is implicitly claiming. dir(json) lists what is actually there, which turns "is this the right name" into a lookup rather than a recollection.

redact matches on the setting name, so it does not need to recognise what a credential looks like. Keeping the name and removing the value is what leaves the snippet useful.

Predict the three printed lines. The middle one lists the first five public names in the json module, alphabetically.

False True
['JSONDecodeError', 'JSONDecoder', 'JSONEncoder', 'codecs', 'decoder']
state: OR
data_source_key: REDACTED
output: shortlist.csv

Change MARKERS to ("KEY", "TOKEN") in upper case, leaving name.lower() in place. Predict what the redaction does now.

An assistant was asked how to fetch data from an API when the request was failing with a certificate error. It replied with this and an explanation that the setting disables certificate checking so the request can proceed.

import requests

response = requests.get(
    "https://example-procurement.test/search",
    params={"state": "OR", "access_token": "sample-value-not-real"},
    verify=False,
    timeout=None,
)
print(response.json())

import requests

response = requests.get(
    "https://example-procurement.test/search",
    params={"state": "OR"},
    timeout=10,
)
if response.status_code != 200:
    raise RuntimeError(f"search failed with status {response.status_code}")

Write the redaction pass that runs before a configuration snippet is shared. Two settings are credentials and two are not.

Loading this exercise…

Optional: write your own one-page list of what you will never paste, adapted to your actual situation, employer, client, or personal. Then take a real error you have hit recently and produce a shareable version of it: the same structure, every value invented. Compare the two and check the second still asks the same question. Doing this once makes the habit fast, and the habit is what protects you when you are tired and the answer feels close.

Sign in to track your progress on this exercise.

1. What is the fastest way to check whether a generated method call exists?
2. Why is stale advice about a library harder to spot than an invented method?
3. Name four categories of information that never go into a prompt.
4. Why match a redaction on setting names rather than on value shapes?

Sign in to track your progress on this exercise.

Check existence mechanically rather than by plausibility, confirm anything version-sensitive against the current official source, look for the recurring insecure defaults by name, treat a dependency as a decision, and redact by name before sharing anything at all. Module 14 brings every thread together into the finished Opportunity Review Assistant.