Skip to content

Check the approved model route

This page describes the earlier fourteen-week layout and is being rewritten; where it differs, the syllabus applies.

Every real model call in this course goes through one approved hosted API configuration, with your own key. Week 0 setup describes, once, how to choose an item, set up your key, the budgets and the replay route. This page lists the commands that check the configuration. None of them runs code a model wrote; model text is compared as data.

The previous local inference walkthrough is retired. The commands below use the offline profile for checks that make no model call and the replay route when a recorded model response is required.

What the configuration fixes

approved-models.json is the course's only model configuration. Each approved item fixes its provider, endpoint, model, request settings, context and output limits, per-run caps and review date. Other course files refer to an item only by its ID. The file's SHA-256 is the policy digest that Week 0 setup publishes. An item that is not enabled yet carries no values at all; until your item is enabled, use the replay route.

offline is the zero-model profile. Launcher runs that make no model call, or only FAKE calls, use it.

The resolver checks the whole file before anything reads a key. Unknown IDs, floating model names, extra or duplicate fields, missing or non-positive caps, a changed version or digest and model environment overrides all fail. A key can only go to the exact host the course reviewed for that item, over HTTPS on port 443.

Commands

Run these from a source checkout or the root of an unpacked student ZIP, with uv installed. The included pyproject.toml and uv.lock select a dedicated environment.

uv sync --project infra/hosted --locked
uv run --project infra/hosted --locked python infra/hosted/resolve_profile.py --profile offline --json
uv run --project infra/hosted --locked python infra/hosted/doctor.py
uv run --project infra/hosted --locked python infra/hosted/smoke.py
  • resolve_profile.py prints the profile ID, version and digest that the launcher checks.
  • doctor.py without options summarizes the configuration. It reads no key file and makes no network request.
  • smoke.py without options checks FAKE replies, then the course broker over a mock connection with a synthetic key. Its result always says real_model_verified: false.

Once your item is enabled, add --profile, --version and --digest from Week 0 setup to doctor.py. It then checks your key file without printing it, and --check-key adds one authentication request that sends no prompt. Doctor prints one fixed state with what to do next: key missing, key file unsafe, key invalid, budget exhausted or provider unreachable. Each message links the replay route.

What the broker accepts

Your code in the sandbox can send only prompt text. The broker on your computer adds the fixed model and settings, sends exactly one request without redirects or retries, and forwards only the reply text. It refuses a reply with any field outside the course contract, a content filter stop, unexpected reasoning output or a copy of your key, and it never reads the body of an error reply. Each run has fixed request, token and cost caps; a request that would exceed them is not sent.

A failure is either "not sent", which is safe to retry, or "outcome unknown", which may still be billed. Report only the state and the policy digest when you ask for help; never paste a key, a key file or an environment dump.

Known limits

  • A slow name lookup can outlast a request's time limit. The broker's deadline cannot interrupt your computer's DNS resolver, so a request or --check-key can wait until the lookup finishes or the resolver gives up. The failure is still "not sent" and safe to retry.