Skip to content

The internals of coding agents

Week 1 · Agentic Software Engineering

Generate a proposal.
Verify the effect.

LO1: explain the loop · LO9: keep evidence bounded

A coding agent is a loop

flowchart LR
  M[Model proposes] --> C[Controller checks]
  C --> T[Tool acts]
  T --> O[Observation]
  O --> M
  C --> S[Stop or continue]

Each arrow carries a different claim.

Start with a contract

ok → passed
fail → failed
Anything else → unknown

Ignore surrounding whitespace and ASCII letter case.

Generate a plausible candidate

def status_label(raw):
    return {"ok": "passed", "fail": "failed"}.get(raw.lower(), "unknown")

What does it return for " OK "?

Predict before reading the check.

Verify the missing case

The contract requires " OK " → passed.

The candidate returns unknown.

Add .strip() before .lower(), then keep the original and unknown cases.

A patch is a proposal until the regression runs.

The fixture has three tools

Name Accepted action Observation
read result.txt file text
write at most 32 characters written
bash verify only verified / check_failed

Other names, paths and arguments are refused.

The five-step controller

  1. Start history with the task.
  2. Send history; check the reply and call IDs.
  3. Append the reply; no calls means final.
  4. Answer every call with its own tool message.
  5. Exhausted budget means step_limit.

The public starter leaves run() for you to complete.

Generate a successful path

c1 read result.txt       → 5\n
c2 write result.txt 4\n → written
c3 bash verify           → verified
final                    → no more calls

Left: generated actions.
Right: observed results.

Verify the controller

mkdir week01-loop
cp lectures/week-02/fixture/*.py week01-loop/
PYTHONPATH="$PWD/scripts" python3 week01-loop/verify_loop.py

Before run() is complete, the check explains what is missing. Afterwards, read the named checks and retain the output.

A write is not a check

Candidate writes 5\n and receives written.

The fixed check returns check_failed.

Which claim is supported?

Stop reasons are evidence fields

Field Means Does not mean
final no calls in a reply code is correct
step_limit turn budget ended task is complete
check_failed current check failed model is useless

A later write also makes an earlier check stale.

Practise reading the supplied request

python3 lectures/week-02/session/show_session.py \
  lectures/week-02/session/sample-session.jsonl --request 4

Predict first: request count, first tool, failure recovery and stop evidence.

Mark observed and inferred

Observed: a request contains a call, a result or an instruction at a cited index.

Inferred: an explanation of why the agent acted or what it might do next.

The supplied record is hand-authored. Your studio capture supports claims about your agent.

Prompt text is not always enforcement

A system instruction can ask for a plan.

A schema can require an argument.

A tool can reject a path.

Name the component that enforces each rule—or say it only asks.

Count requests, not calls

One response can contain several calls.

Each call gets a tool result under its own ID, in response order.

The viewer's Request N blocks are the unit to count.

Studio: force a failure, then verify

  1. Copy a synthetic status task to a draft directory.
  2. Predict before generation.
  3. Capture the failing check and the repair.
  4. Run the check on the current candidate.
  5. Label each claim observed or inferred.

Keep the original failure and revise one explanation.

Evidence-bounded decisions

Accept only the revision and cases the check covered.

Otherwise narrow the claim, re-check the current revision or stop.

A1 is due this week.