Skip to content

From a plausible answer to checked work

This course-authored example uses a deliberately faulty candidate. Follow Week 0 setup before running it.

Learning objectives

Explain the difference between a language model and an agent (LO1). Identify evidence that supports a generated answer (LO5). Describe why tools need a workspace boundary and a stop condition (LO2, LO7).

A language model produces a response from the context it receives. Fluent text can still be wrong. An agent adds a program around the model: observe, request, validate, call an allowed tool, inspect its result and decide whether to continue. The program owns permissions and stop conditions; a model request is never permission to read every file on your computer.

Minimal example

Consider: “Given two integers, return the larger value; equal values are returned unchanged.” Before asking for a candidate, predict the outputs yourself. Use this request with your agent:

Write only a Python function larger_of_two(a, b). Inputs are integers (not booleans).
Return the greater input, or that same value when equal. Negative integers are valid.
No input(), printing, imports, Markdown or explanation.

For the first check, use the course's deliberately faulty candidate.py:

def larger_of_two(a, b):
    return a

Choose an input where the candidate and requirement disagree. Save these checks as check_generated.py; the expected values come from the requirement, not generated text.

from candidate import larger_of_two

assert larger_of_two(3, 7) == 7, "second input is larger"
assert larger_of_two(9, 4) == 9, "first input is larger"
assert larger_of_two(5, 5) == 5, "equal inputs"
assert larger_of_two(-8, -2) == -2, "negative inputs"
print("PASS: four specified cases")

Hands-on exercise

  1. Create week0-guided in your dedicated course directory. Put candidate.py and check_generated.py there. Predict which check fails first.
  2. Inspect both files, then run python3 check_generated.py from that directory. Expected: AssertionError: second input is larger. Keep the failed baseline.
  3. Add an assertion with an input you chose and a value derived from the requirement. Run it again; adding a check does not repair the candidate.
  4. Ask your supplied coding agent for a replacement using the request above. Inspect the proposed function before copying it to candidate.py. If access is unavailable, repair it yourself and label that work as manual; see fallbacks.
  5. Run the same checks again. A correct candidate prints PASS: four specified cases and exits successfully. If a check fails, decide whether the candidate or expected value is wrong. Keep that result as evidence and try a different input independently.

Verify and explain

Keep the candidate, failed baseline, check results and a brief explanation under one evidence ID. State which input exposed the defect and what check you added. Attach one AI-use note identifying actual agent use or manual repair. Four passing examples do not prove correctness for every integer. A model statement that its code works is not a test.