Arguments, Configuration, and Logs That Do Not Leak

By the end of this lesson you can give a script options with defaults, take its settings from the right source for each one, and produce log output that explains what a run did without ever recording a credential.

A script whose settings are edited into the source is a script that cannot be run twice differently, cannot be scheduled, and cannot be tested with different inputs. Every change to what it does is a change to what it is.

Logs matter for the opposite reason. A run that produced a surprising shortlist three weeks ago can only be explained if it recorded which settings it used. And a log line is the most common route by which a credential escapes: nobody prints a key on purpose, but a great many programs print a configuration dictionary that happens to contain one.

Three sources of settings, and each has a job:

• Command-line arguments for what changes per run: a date window, a state, an output path.
• Environment variables for anything secret, and for machine-specific paths. Never in source, never in the repository.
• A configuration file for stable, non-secret defaults a team shares.

Order them so the more specific wins: argument, then environment, then file, then a built-in default.

argparse builds a parser:

parser = argparse.ArgumentParser(prog="review")
parser.add_argument("--state", required=True)
parser.add_argument("--minimum", type=int, default=100000)
parser.add_argument("--dry-run", action="store_true")
args = parser.parse_args(argv)

type=int converts and rejects non-numbers with a usable message. action="store_true" makes a flag that is False unless present. required=True fails immediately, with usage text, rather than three functions later.

Pass argv explicitly, as a list, rather than letting parse_args() read the real command line. It costs nothing, and it makes the entire configuration layer testable: a test supplies any combination of options without a shell.

logging replaces print once a script matters. logging.info(...) for what happened, logging.warning(...) for something a person should look at, logging.error(...) for a failure. Levels let one program be quiet in normal use and detailed when something is wrong, without editing it.

The rules for what goes in a log:

• Record the settings that determine what the run did, so the output can be explained later.
• Record counts: read, skipped, written, and why.
• Never record a credential, and never log a whole configuration object or a whole request, because both carry one eventually.
• Redact where the line is built, not with a filter attached somewhere else. A filter is one refactor away from being detached, and nothing fails when it is.

The safe form for a credential is to log its presence, never its value: credential: set or credential: not set. That is enough to diagnose the overwhelming majority of configuration problems. This project has no credential of its own — it reads fixture files — but the pattern is worth building the habit for now, because the run that needs it is never the run you were paying attention to.

import argparse
import logging

logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")


def build_parser():
    parser = argparse.ArgumentParser(prog="review")
    parser.add_argument("--state", required=True)
    parser.add_argument("--minimum", type=int, default=100000)
    parser.add_argument("--dry-run", action="store_true")
    return parser


def run(argv, credential_present=False):
    args = build_parser().parse_args(argv)

    logging.info("state=%s minimum=%s dry_run=%s", args.state, args.minimum, args.dry_run)
    logging.info("credential: %s", "set" if credential_present else "not set")

    if args.dry_run:
        logging.warning("dry run: no output written")
        return 0

    logging.info("read=%d skipped=%d written=%d", 5, 1, 4)
    return 4


print(run(["--state", "OR", "--dry-run"]))

run takes its arguments as a parameter, so the whole thing can be exercised from a test with any combination of options.

The credential report is a boolean and nothing else. No value is ever bound to a name that could be logged by accident, which is the narrowest path Module 7 argued for. Here the flag is simply False: this pipeline reads fixture files and holds no credential at all, so the line reports the state it is actually in.

The log calls use %s placeholders with the values passed separately, rather than building a string with an f-string. That is the logging module's own convention: the formatting is deferred until the message is actually emitted, so a debug line costs nothing when the level is set higher.

--dry-run returns before writing anything. Any automation that changes data deserves one.

Predict every line, including the level prefixes.

INFO state=OR minimum=100000 dry_run=True
INFO credential: not set
WARNING dry run: no output written
0

Call run(["--minimum", "50000"]) without --state, and predict what happens.

review: error: the following arguments are required: --state

An assistant was asked to add logging to a client so failures can be diagnosed. It produced this.

import logging

logging.basicConfig(level=logging.DEBUG)


def build_config():
    return {
        "state": "OR",
        "minimum": 100000,
        "api_key": "your-key-here",
        "endpoint": "https://example-procurement.test/search",
    }


def run():
    config = build_config()
    logging.debug("config: %s", config)
    logging.info("requesting %s?api_key=%s", config["endpoint"], config["api_key"])

def run():
    config = build_config()
    logging.info(
        "state=%s minimum=%s credential=%s",
        config["state"],
        config["minimum"],
        "set" if config.get("api_key") else "not set",
    )

Complete the parser and print a startup summary. One option is supplied, one falls back to its default, and the credential must be reported without being revealed.

Loading this exercise…

Optional: add a --verbose flag that sets the logging level to DEBUG when present and INFO otherwise, then add one logging.debug line reporting a per-record decision. Run it both ways. Then write one sentence on why debug detail belongs behind a flag rather than being on by default, thinking about who can read the logs of an unattended job.

Sign in to track your progress on this exercise.

1. Which setting belongs in an environment variable rather than a command-line argument?
2. Why pass an explicit argument list to parse_args instead of letting it read the command line?
3. Why redact at the point the log line is built rather than with a filter?
4. What is the safe way to log something about a credential?

Sign in to track your progress on this exercise.

Arguments for what changes per run, environment for secrets, defaults for the rest; parse an explicit list so it can be tested; log settings and counts, never containers; and report a credential's presence rather than its value. Next: making the whole pipeline safe to run twice, and reviewing an automation before it does something you cannot undo.