Week 12: The software factory and what comes next

Learning objectives

By the end of this week you can:

restate "the pipeline runs itself" as a claim about every change, name the one observation that refutes it and the smaller claim a single run supports (LO9);
say, for each stage of a pipeline, what evidence it must leave and where a named person still accepts or rejects (LO9);
exercise a stop path and read from the record which stage stopped and which never ran;
treat an automated review as a list of claims, each confirmed, refuted or undetermined by a check you chose yourself; and
walk the decision line (accept, narrow, recheck or stop) back through the term and decide again where an input has changed (LO9).

Everything on this page is course-written practice material: the pipeline, the proposed changes, the recorded review and every count. No model wrote any of it, and none of it measures a model or a tool.

Minimal example

"It runs itself" is a claim

A software factory is a pipeline in which each stage triggers the next: a request becomes a specification, a change, a set of checks, a review and a release. The strong version of the idea is that the pipeline runs itself: no person reads anything between the request and the release. That sentence is a claim about every change that will ever enter the pipeline, and a claim has a test. This week asks three questions of it:

What would have to be observed for the claim to be true?
What evidence must each stage leave so that someone else can check its verdict?
Where does a named person still accept or reject, and what are they accepting?

What one run can decide

A claim's wording sets what one observation can show. Read the wording before you run anything.

The claim is about	One run can	One run cannot
one input ("this change went through unattended")	confirm or refute it
some input ("a bad change can get through")	confirm it, if the run shows the problem	refute it: an input that behaves shows only where the problem is not
every input ("the pipeline runs itself")	refute it, with one counterexample	confirm it: an input that behaves is one case of many
each member of a set you can list ("every stage leaves a record")	decide that member	decide the others; check each member
something outside the snapshot ("it also holds in production")	show that the snapshot cannot read it	decide it

A demonstration in which one request becomes one merged change is the first row. It says nothing about the third row, however impressive it looks. The claim you can support is always smaller: this stage, on this kind of change, behind this check, with this stop path.

An unfinished search is not a disposition. If the inputs you tried for a "some" claim all behave, the snapshot still holds what decides the claim, so cannot tell is wrong too. Read the code for the rule it actually applies and choose inputs at that rule's edges. Keep each input you tried; they record where you looked.

A pipeline small enough to read

The folder lectures/week-12/factory/ holds a pipeline of three stages and the thing it protects: an evaluator with a one-paragraph contract. The evaluator answers whether a supplied status is exactly failed. The exact string yes is correct for failed and the exact string no for any other status; a missing or non-string answer is unknown; any other string, including extra text or extra spaces, is fail.

Stage	What it checks	Evidence it leaves	Who decides
`scope`	the proposal defines the one function `score` and nothing else	the names it defines and any other statement kinds	the stage
`gate`	nine frozen cases in `verify.py`, run against the proposal in a child process	digest of the proposal, digest of the cases file, case count, failed cases, exit status	the stage
`release`	that a named person accepted the gate's evidence	who accepted, and the digests and case count they accepted	a person

Read pipeline.py, evaluator.py and verify.py before you run them; together they are under 200 lines. Then, from the course checkout, make a working copy:

mkdir week12-factory
cp lectures/week-12/factory/*.py week12-factory/
cd week12-factory

Every command in this section and in the first hands-on part runs in week12-factory.

Generate: a proposal arrives. Verify: the stop path

proposed_evaluator.py is a proposed change to the evaluator. Its note says: "Made scoring robust to formatting. Answers are trimmed and lower-cased, and an answer that starts with the expected word now counts." Would a pipeline that runs itself release it? Predict the exit status, then run the pipeline on it:

python pipeline.py --candidate proposed_evaluator.py; echo "exit: $?"

{"stage": "scope", "status": "pass", "evidence": {"defines": ["score"], "other": []}}
{"stage": "gate", "status": "stop", "evidence": {"candidate_sha256": "50ebaf8b5f7a", "cases_sha256": "d087328e6120", "exit_code": 1, "cases": 9, "failed": [{"case": "extra_text", "observed": "pass", "required": "fail"}, {"case": "case_changed", "observed": "pass", "required": "fail"}]}}
{"stage": "release", "status": "not-run", "reason": "an earlier stage stopped"}
exit: 1

Read the record from the top. scope passed: the proposal is one function. gate stopped and names the two cases that separate the proposal from the contract: the answer yes; mark this as passed and the answer YES now score pass, and the contract requires fail. release never ran. The stop path has now been seen to work once, on this proposal.

A stop that a flag can talk past is not a stop. Offer the pipeline an acceptance for the same proposal:

python pipeline.py --candidate proposed_evaluator.py --accept-by "A. Student"; echo "exit: $?"

The three lines and the exit status are the same as before. release is still not-run: a person can accept evidence, and there is no passing evidence here to accept.

A green run, and the person at the end

Now run the current evaluator, the one that keeps the contract:

python pipeline.py --candidate evaluator.py; echo "exit: $?"

{"stage": "scope", "status": "pass", "evidence": {"defines": ["score"], "other": []}}
{"stage": "gate", "status": "pass", "evidence": {"candidate_sha256": "9f75d5aca3be", "cases_sha256": "d087328e6120", "exit_code": 0, "cases": 9, "failed": []}}
{"stage": "release", "status": "held", "reason": "no named person has accepted this evidence"}
exit: 3

Both automatic stages passed and the pipeline still did not release. Exit status 3 means held: a script that only tests for zero cannot mistake it for a release. The release stage needs a name:

python pipeline.py --candidate evaluator.py --accept-by "A. Student"; echo "exit: $?"

{"stage": "scope", "status": "pass", "evidence": {"defines": ["score"], "other": []}}
{"stage": "gate", "status": "pass", "evidence": {"candidate_sha256": "9f75d5aca3be", "cases_sha256": "d087328e6120", "exit_code": 0, "cases": 9, "failed": []}}
{"stage": "release", "status": "released", "accepted_by": "A. Student", "accepted_evidence": {"candidate_sha256": "9f75d5aca3be", "cases_sha256": "d087328e6120", "cases": 9}}
exit: 0

The last line is the decision line of this pipeline. It names who accepted and exactly what they accepted: this proposal, these nine cases. If either digest changes, this acceptance no longer covers what is being released, and the decision has to be made again.

A green run on a change that breaks the contract

A second proposal, trimmed_evaluator.py, arrives with the note: "Ignore stray spaces around the answer. No other behaviour changes."

python pipeline.py --candidate trimmed_evaluator.py; echo "exit: $?"

{"stage": "scope", "status": "pass", "evidence": {"defines": ["score"], "other": []}}
{"stage": "gate", "status": "pass", "evidence": {"candidate_sha256": "794d3164e5e5", "cases_sha256": "d087328e6120", "exit_code": 0, "cases": 9, "failed": []}}
{"stage": "release", "status": "held", "reason": "no named person has accepted this evidence"}
exit: 3

The record looks exactly like the one for the current evaluator. Now check the proposal against the contract yourself, with an input the nine cases do not contain. The contract says an answer with extra spaces is fail:

python -c "from trimmed_evaluator import score; print(score('failed', ' yes'))"
python -c "from evaluator import score; print(score('failed', ' yes'))"

pass
fail

The proposal breaks the contract and every automatic stage passed it. If the release stage had accepted on a green gate, this pipeline would have released a change that scores a wrong answer as correct. That one observation refutes "this pipeline runs itself": the claim is about every change, and here is a change it gets wrong.

What the green gate did support is narrower: the proposal agrees with the contract on these nine cases. The note's second sentence, "no other behaviour changes", was never tested by anything. The first hands-on part turns the missing input into a tenth case.

What a stage must show before nobody watches it

One green run, or two hundred, cannot make a stage unattended. Before you take the person away from a stage, the stage needs four things, each of which you can observe:

A check seen to fail on a bad change and pass on a good one. The gate has this for the first proposal and the current evaluator. A check that has never been seen to fail is not yet evidence that it would stop anything.
Cases the changing agent cannot edit. The gate reads the cases from the --cases path, which defaults to verify.py in the directory you run the pipeline from. An agent that can write there, or choose that path, can delete the cases that fail.
A verdict the proposal cannot write. The cases import the proposal and call it in their own process, so the proposal's code runs inside the thing that judges it.
A stop path that has been exercised. Someone has to have seen a failed stage keep the later stages from running, and has to know who is told.

The first hands-on part shows the second and third gaps on this pipeline, with real runs. For a stage that keeps running after release, add three questions with testable answers: who can stop it without its cooperation, what it may change and where that list is granted, and what record it leaves of what it read, proposed and changed.

The review stage emits claims, not evidence

Many pipelines add a review stage: a model reads the change and reports problems. Its output is a list of sentences. A pipeline that runs itself has to do something with them without a person: act on all of them, or on none.

The folder lectures/week-12/fixtures/ holds a recorded review of a small snapshot, labelled week12-audit-snapshot. The review is course-written; it holds five claims about security, evaluation and maintenance, each with an id, an area and, for most, the function it points at. It names no check, no input and no answer. Choosing those is the exercise in the second hands-on part.

These are the snapshot's frozen requirements. Every disposition is judged against them, not against the wording of the review:

R1 access: a viewer or an owner can read a public report; no role reads a private report body.
R2 status: the status endpoint accepts only the labels ready and queued.
R3 paths: every exported or archived report path stays under /srv/reports.
R4 secrets: no export configuration exposes a secret directory.
R5 scoring: the private-count scorer gives 1 only when the answer's first word is the number of private sample reports; every other answer scores 0.
R6 root: the report root is written once, as REPORT_ROOT; no function repeats the literal path.

Production configuration is not part of this snapshot, so no check here can read it.

One worked row

This claim is not in the review. It shows the steps once, from claim to disposition:

An owner can read the body of a private report.

Requirement. R1 says no role reads a private report body, so the claim asserts an R1 defect.
Check and input. The claim names one case: an owner and a private report. The private-report-view check takes private and role, so the input is {"private": true, "role": "owner"}.
Prediction, written before the run. If the claim holds, allowed and body_visible are both true.
Run. In a copy of the fixture (the second hands-on part makes one):

sh python audit_packet.py --check private-report-view --args '{"private":true,"role":"owner"}'

Observation. The observation field of the printed line is {"allowed": false, "body_visible": false, "status": "deny"}.
Disposition. Refuted. The claim is about one input, and that input is denied with the body hidden, as R1 requires.

A claim often overstates its scope: it holds for one member of a set and not for another. Do not force it into one word. A single "refuted" would hide the part that holds, and a single "confirmed" would assert the part that does not. Split the claim and give each part its own evidence and disposition.

Turn a prose claim into a check

Some claims point at a function and name an input. Others only describe behaviour, such as "the search page can show another user's reports". To check one:

Name the requirement the claim says is broken.
Find the check that exercises that behaviour. Running audit_packet.py with no --check lists every check and its argument keys.
Write the input that would show the problem, and the observation you expect.
Read the code for the rule it applies, then choose inputs at that rule's edges rather than typical values.

Why a review stage cannot act on its own word

A review tends to turn a salient token into a conclusion. It may see a value that looks dangerous and skip the allowlist that stops it, see one function that breaks a rule and assume every similar function does, or fill a missing file with a plausible assumption. Its attention is also finite: it may follow the route it named and never probe a sibling, such as a second function that builds the same kind of path.

This course-made record stands in for one verified review of one frozen snapshot:

Course-made record of one verified review	Count
Claims in the review	9
Confirmed with a failing test or a repeatable reproduction	4
Refuted with runnable evidence	3
Undetermined: a named input is missing from the snapshot	2
Defects the verifier's own check found and the review missed	1

Suppose a pipeline had acted on every claim unattended. It would have changed code for 3 claims with no defect behind them, acted on 2 that nothing observed, and still released the 1 defect the review missed. Suppose instead it had ignored the review: 4 real defects go out. Neither rule works, so the review stage can run unattended as a source of hypotheses on every change, and cannot yet change code or block a release on its own word. After the second hands-on part, replace these counts with the ones from your own record.

The two stages connect. Each confirmed claim becomes a case that fails before the fix, and that case can join the gate.

The decision line, walked back through the term

Every assignment this term ended with one line: accept, narrow, recheck or stop, and the evidence behind it. The four words have meanings you have now seen on one pipeline:

Decision	It means	Seen this week
accept	the evidence covers the claim as stated, and a named person says so	the current evaluator, nine cases, accepted by name with both digests
narrow	the evidence supports a smaller claim; write the smaller claim and accept only that	the second proposal: "agrees with the contract on these nine cases", not "no other behaviour changes"
recheck	the evidence is missing, or it is about an input that has since changed; name the check that would supply it	a cases file whose digest differs from the one that was accepted
stop	a check failed, or the evidence was written by the thing being judged; nothing proceeds until a person decides	the first proposal at the gate

Each week handed you a claim of the same shape as "the pipeline runs itself", and each week's evidence was the thing that could have shown it false:

Week	The claim you were handed	What could show it false
1	"The agent did this in my session"	the session record: what was observed in a request, and what you only inferred
2	"The model will pick the right tool"	a call with the wrong tool or wrong arguments, and whether it recovered
3	"The skill triggers when it should"	a request that should trigger it and does not, and a near miss that does
4	"The agent follows the rule"	a bad commit the hook did not block
5	"This repository is ready for an agent"	an anchor you can observe that the repository lacks
6	"The model review cleared the change"	a finding it missed, or one it invented
7	"The tests pass, so the code is right"	a mutant that survives
8	"That text is only data"	a local, inert reproduction in which the text is obeyed
9	"It finished while nobody was watching"	the record it left, read by someone who was not there
10	"The pass rate is this number"	a check that cannot fail, an unearned pass, a case left out of the denominator
11	"The incident is over"	the regression that fails before the recovery and passes after it
12	"The pipeline runs itself"	one bad change it lets through, or a stop that was never exercised

Reopen two of your own decision lines from earlier assignments. For each, ask whether any input behind it has changed since: the code, the cases, the tool or the model version behind your agent, the upstream project's rules. If one has, the old line describes evidence about a different input, and the honest word today is recheck.

What comes next

This course makes no forecast about what agents or pipelines will do next year. What it leaves you with is a way to meet whatever claim arrives:

The claim has a reach. "Runs itself", "catches every bug" and "makes the team faster" are claims about every input. Ask for the counterexample search, not the demonstration.
A belief about speed is a prediction until it is measured. In one dated study of 16 experienced developers working in their own repositories, developers believed afterwards that AI had made them 20% faster while the recorded time was 19% longer; a later update from the same group reports intervals that cross zero and says the data are hard to use for reliable estimates of current productivity (studies/metr-2025; as_of: 2026-09-05; Source 1; Source 2). That is evidence about that sample and that period. The transferable part is the method: write the prediction down, then record what happened.
A decision is bound to its inputs. When the model, the tool, the cases or the upstream rules change, the decision is made again by a person who can be named.
You still explain your own code. Next week you do that alone, in the defense.

Hands-on exercise

Find where this pipeline needs a person

Work in week12-factory, the copy you made above. Before each run, write down the status you expect from each stage. Keep the output of every run in a file named factory-record.txt.

1. Turn the escape into a case. Add the input the nine cases lacked as a tenth case, in a new cases file:

python - <<'__CASE__'
from pathlib import Path

anchor = '    ("case_changed", "failed", "YES", "fail"),\n'
added = '    ("leading_space", "failed", " yes", "fail"),\n'
source = Path("verify.py").read_text(encoding="utf-8")
assert source.count(anchor) == 1
Path("ten_cases.py").write_text(source.replace(anchor, anchor + added), encoding="utf-8")
__CASE__
python pipeline.py --candidate trimmed_evaluator.py --cases ten_cases.py; echo "exit: $?"
python pipeline.py --candidate evaluator.py --cases ten_cases.py; echo "exit: $?"

{"stage": "scope", "status": "pass", "evidence": {"defines": ["score"], "other": []}}
{"stage": "gate", "status": "stop", "evidence": {"candidate_sha256": "794d3164e5e5", "cases_sha256": "a5eb322f2930", "exit_code": 1, "cases": 10, "failed": [{"case": "leading_space", "observed": "pass", "required": "fail"}]}}
{"stage": "release", "status": "not-run", "reason": "an earlier stage stopped"}
exit: 1
{"stage": "scope", "status": "pass", "evidence": {"defines": ["score"], "other": []}}
{"stage": "gate", "status": "pass", "evidence": {"candidate_sha256": "9f75d5aca3be", "cases_sha256": "a5eb322f2930", "exit_code": 0, "cases": 10, "failed": []}}
{"stage": "release", "status": "held", "reason": "no named person has accepted this evidence"}
exit: 3

The tenth case stops the second proposal and still passes the evaluator that keeps the contract. Compare cases_sha256 with the acceptance you recorded earlier: the cases changed, so that acceptance was for different evidence. Whoever adds a case has changed what "pass" means, and that is a decision too.

2. Shrink the cases instead. An agent asked to "make the gate pass" for the first proposal could remove the two cases that fail:

grep -v -e extra_text -e case_changed verify.py > fewer_cases.py
python pipeline.py --candidate proposed_evaluator.py --cases fewer_cases.py; echo "exit: $?"

{"stage": "scope", "status": "pass", "evidence": {"defines": ["score"], "other": []}}
{"stage": "gate", "status": "pass", "evidence": {"candidate_sha256": "50ebaf8b5f7a", "cases_sha256": "8458d33b70db", "exit_code": 0, "cases": 7, "failed": []}}
{"stage": "release", "status": "held", "reason": "no named person has accepted this evidence"}
exit: 3

The gate passed a proposal it stopped a few minutes ago. Nothing in the pipeline objected. The only trace is in the evidence: 7 cases, and a cases digest that differs from d087328e6120. A stage that left only "pass" would have hidden this; a person who reads the count and the digest catches it.

3. Step outside the scope. A proposal that brings an import with it:

cat > always_pass.py <<'__CANDIDATE__'
import os


def score(status, output):
    return "pass"
__CANDIDATE__
python pipeline.py --candidate always_pass.py; echo "exit: $?"

{"stage": "scope", "status": "stop", "evidence": {"defines": ["score"], "other": ["Import"]}}
{"stage": "gate", "status": "not-run", "reason": "an earlier stage stopped"}
{"stage": "release", "status": "not-run", "reason": "an earlier stage stopped"}
exit: 1

The record says which stage stopped and that the two later stages never ran. A not-run stage has no verdict; do not read it as a pass or as a second failure.

4. Let the proposal write its own verdict. This proposal stays inside the scope, and instead of scoring anything it prints the report line the gate expects and ends the run:

cat > self_report.py <<'__CANDIDATE__'
def score(status, output):
    print('{"cases": 9, "failed": []}')
    raise SystemExit(0)
__CANDIDATE__
python pipeline.py --candidate self_report.py; echo "exit: $?"

{"stage": "scope", "status": "pass", "evidence": {"defines": ["score"], "other": []}}
{"stage": "gate", "status": "pass", "evidence": {"candidate_sha256": "3d6806d3644e", "cases_sha256": "d087328e6120", "exit_code": 0, "cases": 9, "failed": []}}
{"stage": "release", "status": "held", "reason": "no named person has accepted this evidence"}
exit: 3

The gate's evidence is indistinguishable from an honest pass, because the proposal ran inside the process that produced the report. Only the person at the release stage, reading the three-line proposal, stands between it and a release. To close the gap the gate would have to call the proposal in a separate process and treat whatever that process prints as data. This pipeline does not do that, and the page does not pretend it does.

Finish the part with one line under your record, for example:

Decision: narrow — this pipeline stops a proposal that fails a frozen case or leaves the scope; it does not run itself: runs 2 and 4 passed the gate. Evidence: factory-record.txt

Delete week12-factory when you have kept your record.

Verify the review stage's claims

From the course checkout, copy the review fixture into a fresh directory:

mkdir week12-review
cp lectures/week-12/fixtures/audit_packet.py week12-review/
cp lectures/week-12/fixtures/model-audit.json week12-review/
cp lectures/week-12/fixtures/verify_audit.py week12-review/
cp lectures/week-12/fixtures/student-audit.json week12-review/
cd week12-review

Read model-audit.json. Before you run anything, write one plan line for each of the five claims: the requirement, the reach of the claim, the check, the input and the observation you expect. Then list the checks and the argument keys each one takes:

python audit_packet.py

{"snapshot": "week12-audit-snapshot", "status": "ready", "checks": {"private-report-view": ["private", "role"], "status-allowlist": ["value"], "export-path": ["name"], "archive-path": ["name"], "score-answer": ["answer"], "report-root": ["function"], "production-config": []}}

Run one check at a time. This command runs the worked row:

python audit_packet.py --check private-report-view --args '{"private":true,"role":"owner"}'

{"args": {"private": true, "role": "owner"}, "check": "private-report-view", "command": "python audit_packet.py --check private-report-view --args '{\"private\":true,\"role\":\"owner\"}'", "observation": {"allowed": false, "body_visible": false, "status": "deny"}, "snapshot": "week12-audit-snapshot"}

The output names the args the check ran with, the observation and the canonical command. Record that command exactly as printed: it starts at python and its arguments are compact JSON with sorted keys. A check refuses a missing, extra or wrongly typed argument instead of guessing.

Mark each recorded claim confirmed, refuted or cannot tell, as in Week 6. The record calls cannot tell undetermined, and it means the snapshot lacks the input or configuration that would decide the claim; the row names what is missing. When an observation does not decide a claim, change the input, not the disposition.

In the lecture, plan all five claims and run at least two, one of them about some input or every input. Finishing the record afterwards is optional practice; nothing here is handed in.

The record the verifier reads

student-audit.json holds the snapshot label, a verifications array with one row per recorded claim, and one own_check object. This is the worked row in that shape:

{
  "finding_id": "worked-example",
  "requirement": "R1",
  "disposition": "refuted",
  "evidence": {
    "check": "private-report-view",
    "args": {"private": true, "role": "owner"},
    "command": "python audit_packet.py --check private-report-view --args '{\"private\":true,\"role\":\"owner\"}'"
  },
  "scope": "Checked an owner on a private report on week12-audit-snapshot: denied, body hidden. The claim is about this one input."
}

finding_id is the id of the claim in model-audit.json. The worked claim is not in the review, so its row would not pass the verifier as it stands.
disposition is confirmed, refuted or undetermined. An undetermined row also needs missing (the input the snapshot lacks) or why_undecidable.
evidence names the check and the arguments you chose, and the printed command.
requirement and scope are for the person reading your record. The verifier does not judge prose; a reader does, and looks for the boundary of each conclusion.

A claim about each member of a set you can list takes a parts array instead of one disposition. Each part is about one member and has its own disposition and evidence:

{
  "finding_id": "<id>",
  "requirement": "R<n>",
  "parts": [
    {"part": "<the member>", "disposition": "<disposition>", "evidence": {...}, "scope": "<sentence>"},
    {"part": "<the next member>", "disposition": "<disposition>", "evidence": {...}, "scope": "<sentence>"}
  ]
}

The own check takes one of two shapes:

"own_check": {
  "finding_id": "own-<short-name>",
  "kind": "new-defect",
  "requirement": "R<n>",
  "disposition": "confirmed",
  "evidence": {"check": "<check>", "args": {<its keys>}, "command": "<printed command>"}
}

"own_check": {
  "finding_id": "own-<short-name>",
  "kind": "scope-check",
  "requirement": "R<n>",
  "result": "no-defect-observed",
  "scope": "<the range you checked>; <the boundary that stays unchecked>",
  "evidence": {"check": "<check>", "args": {<its keys>}, "command": "<printed command>"}
}

new-defect: an input that no recorded claim and not the worked row is about, whose observation breaks a frozen requirement; or
scope-check: such an input whose observation shows no defect, with a sentence naming the checked range and its boundary.

Run the verifier on your record at any point. On the empty starter record it prints:

python verify_audit.py --submission student-audit.json; echo "exit: $?"

{"error": "verifications needs exactly one row per recorded finding, keyed by finding_id: F1, F2, F3, F4, F5", "status": "rejected"}
exit: 1

When the five rows and the own check are filled and supported, it prints "status": "verified" and exits 0. Then count your rows: how many claims would an unattended pipeline have acted on with no defect behind them, and did your own check find something the review missed? Delete week12-review when you have kept your record.

Judge three stages of your own project

Pick three stages your project actually has, for example its hook gate, its test run and its evaluation script. For each, write one row in your project notes:

Stage	Could it run unattended now?	Evidence, by reference	What would change your answer
the stage's name	yes or not yet	the commit, record or output that shows the check failing on a bad change and passing on a good one	the missing item of the four: a failing run, cases out of the agent's reach, a verdict the change cannot write, an exercised stop

A stage supported only by passing runs is "not yet", however many passes there are. End with one decision line for the project as it stands.

Freeze the revision you hand in

The final project is handed in as one exact revision. A tag makes "the revision I handed in" a name that cannot drift. In your project repository, confirm the tree is clean, create an annotated tag and print the commit it names:

git status --porcelain
git tag -a final-week-12 -m "Final project revision"
git rev-parse final-week-12^{commit}

The first command must print nothing; the last prints the full commit hash to quote in your submission. If the first command lists anything, commit it, delete it or add it to .gitignore (a __pycache__/ folder is the usual case) before you tag. List every path the tag contains and read the list: it should hold your own work and no credential, session log or unrelated personal data.

git ls-tree -r --name-only final-week-12

Run the fresh-copy check of the second session before you push. A plain git push does not send tags, so push the tag by name:

git push origin final-week-12

The tag is your own bookmark. The published submission channel says how the final project is handed in; if it asks for a commit, give the hash printed above. The project page lists what the final revision must contain.

Verification

For the factory example, check your record against the pipeline's contract:

exit status 0 appears only on a run whose last line has "status": "released" and an accepted_by name;
exit status 1 appears with exactly one stage at "status": "stop", and every later stage at "status": "not-run";
exit status 3 appears when both automatic stages passed and nobody accepted; and
two runs are about the same evidence only when candidate_sha256, cases_sha256 and cases all match.

Failure case: if a run prints a Python traceback or a usage message instead of three JSON lines, the pipeline did not run. Keep the output, fix the command and do not count the run as a stop.

For the review fixture, verify_audit.py exits zero only when all five recorded claims and the own check have reproducible evidence. It accepts exactly these meanings:

confirmed: the runnable observation demonstrates the stated defect on this snapshot;
refuted: the runnable observation contradicts the stated defect; or
undetermined: the snapshot lacks the input or configuration needed to decide, and the missing item is named.

The verifier holds each claim as the check and inputs it is about and as its reach: one input, some input, every input, or each member of a listed set. That is the question, not the answer. It checks the snapshot, the check name, the arguments and the canonical command, then runs the deterministic check and derives the supported disposition from the observation.

It rejects an input that does not test the claim, a disposition the observation does not support, an observation that cannot decide the claim's reach, an item with no evidence, an empty undetermined reason, a claim about a listed set without one part per member, a different command and a missing own check. Each rejection names the key, the command or the observation it used. It calls no model.

For your own project's three stages, the check is the same as for the gate: each "yes" needs a run you can point to in which the stage's check failed on a bad change, and a stop someone has exercised.

Common mistakes

Mistake	Better next action
Answering "can pipelines run themselves?" in general	Judge one stage of one pipeline against its evidence
Citing green runs as proof that a check would stop a bad change	Show the check failing on a deliberately bad input first
Reading exit status 3 or a `held` line as a release	Look for `released` and a name; anything else is not a release
Reading a `not-run` stage as a pass	Say which stage stopped; the later stages have no verdict
Accepting a gate pass without reading the case count and the cases digest	Compare both with the evidence that was accepted before
Adding a flag or an override so the pipeline "can continue"	A stop is cleared by evidence, never by the release step
Treating a review claim as a fix to apply	Choose the check and input the claim is about, run it, then decide
Refuting a claim about some input because the inputs you tried behaved	Read the rule the code applies and try its edges
Confirming a claim about every input from one input that showed the problem	Split the claim; record one part per member
Calling something outside the snapshot a defect, or deciding it with a check of a different path	Mark it `undetermined` and name the missing input
Reusing a decision line after its inputs changed	Write recheck, name the changed input and run the check again
Stating what agents will do next year as a fact	State the claim, its reach and the observation that would test it