Capstone milestone rubric
This page describes the earlier fourteen-week layout and is being rewritten; where it differs, the syllabus applies.
This rubric is the marking contract for the individual project. The five milestones total
20 points: M0 2, M1 6, M2 6, M3 3 and M4 3. Each student submits one ci-dashboard
work record and links each claim to an evidence ID. One evidence ID supports one criterion;
repeating a citation never earns extra points. Optional features and extra integrations are
not conditions for full marks.
The complete feature is not part of the project: you build it in your C2 open-source contribution, marked by the open-source contributions rubric.
Purpose
This document defines the evidence and point boundaries used by the teaching team for the five project milestones.
Records
The TA records the evidence IDs, revision and criterion score on the score sheet. The student keeps the work record with the submitted project.
Review
A second calibrated reviewer checks the evidence mapping, arithmetic and boundary rules before the project score is released. Appeals follow the assessment and appeals process.
How to use the tables
Award the points in a row only when the listed evidence is present, attributable to the submitted revision and verified. Scores are whole or half points, and each row lists every score it allows: full credit, the partial-credit values in its partial column, or zero. A row whose partial column names no value has no partial credit; score it at full credit or zero. Award zero when the required behaviour or evidence is missing. A score is not a prediction of effort and never depends on code volume, commit count or whether an assistant produced text. The teaching team records the rubric version and the evidence IDs on its score sheet.
M0 — discovery and specification (2 points)
| Criterion | Points | Full credit | Partial credit | Zero |
|---|---|---|---|---|
| Context and discovery | 1 | Two discovery activities identify a bounded need and constraints, separate observations from assumptions and freeze the discovery record; the context workshop traces a call path, compares two fixed-budget context packets, checks one permission boundary, and records the standard skill trigger test and tested MCP tool server. | 0.5: the discovery record is usable but one context-workshop evidence class is missing, or the context evidence is complete but one discovery activity, observation/assumption separation or freeze decision is missing. | No usable discovery or context evidence. |
| Executable specification | 0.5 | The agreed small change has executable acceptance criteria, including normal, boundary and failure behaviour. | No partial credit: criteria in which one important case cannot be executed or checked score 0. | 0: no executable criteria. |
| Traceability and scope | 0.5 | The work record connects discovery, criteria and the agreed change; the scope is small enough to finish. | No partial credit: one unclear connection or scope decision scores 0. | 0: the change and evidence cannot be identified. |
M1 — reliable change (6 points)
| Criterion | Points | Full credit | Partial credit | Zero |
|---|---|---|---|---|
| Behaviour-preserving change | 1 | The reviewed change implements the M0 criteria, preserves existing callers and explains a bounded diff. | 0.5: the change is bounded and partly verified, but one caller, criterion or explanation is incomplete. | The change is not reviewable or breaks a required existing behaviour. |
| Agent workflow design | 0.5 | The M1 agent-readiness scorecard records start and end observations and one measured blocker removed. Deleting or skipping tests, or turning off a gate, is not a removed blocker. This row takes only the scorecard evidence: both scorecards and the before-and-after observation of the removed blocker. It is not the eight-dimension assessment rubric's dimension of the same name. | No partial credit: a scorecard missing attribution or a measured blocker scores 0. | 0: no scorecard evidence, or the only "removed blocker" deletes or skips tests or turns off a gate. |
AGENTS.md and hook gate |
0.5 | At the submitted revision, the repository's AGENTS.md lists run and test commands that run as written through the launcher and at least three invariants that must not change, each with the test or check that protects it; one hook gate stops a deliberately bad change before a commit, with its failing and passing output linked. This row takes the AGENTS.md file, its command outputs and the hook's failing and passing output. When the removed blocker on the scorecard is the AGENTS.md or the hook, both rows can be earned: the scorecard row cites the scorecard's observation and this row cites the file and hook outputs. |
No partial credit: a context file whose commands fail as written, or a hook with no failing case shown, scores 0. | 0: no AGENTS.md or no hook gate at the submitted revision, or the hook is turned off or bypassed. |
| Tests and evaluation of the change | 1.5 | Regression checks, a surviving-mutant challenge with one property, and a reproducible fuzz input cover the acceptance criteria; failures identify the fault and the green run is reproducible. | 0.5: feature checks run but mutant or fuzz evidence is missing. 1: all check classes run but one failure explanation or reproducibility link is missing. | 0: checks are absent, falsified or unrelated to observable behaviour. |
| Security repair | 1.5 | The supplied local attack is reproduced, the repair closes it, positive and negative regressions preserve allowed behaviour, and the instructor-side seed check reports CLOSED for every seed in the student's assigned variant. Any OPEN seed in that variant leaves this row below full credit. | 0.5: the repair or one regression is present but the seed result is missing or OPEN. 1: the repair, regressions and CLOSED seed result are present but one boundary or explanation is incomplete. | 0: the issue is ignored, the payload is used outside the local mock, or evidence is unsafe. |
| Review and verification record | 1 | An independent review identifies a real risk or confirms the bounded change; the work record states what was checked, why it could fail and the observed result. | 0.5: a review or verification record exists but omits the risk or expected result, or the record is causal but not reproducible. | 0: no credible review or verification evidence. |
M2 — evaluation and operation (6 points)
| Criterion | Points | Full credit | Partial credit | Zero |
|---|---|---|---|---|
| Evaluator | 2 | The evaluator declares request, environment, stop and score, has a fixed oracle and reproducible output, calibrates a judge on 30 human-labelled cases, runs a local regression gate, and includes error analysis of its limits. | 0.5: a draft contract exists; 1: the evaluator runs but oracle or calibration is incomplete; 1.5: it is reproducible and calibrated but one required limit or local regression boundary is missing. | The evaluator is absent, non-reproducible or uses an undisclosed moving target. |
| Preview and incident recovery | 2 | A local preview uses synthetic data, and the incident drill records detection, bounded recovery and a post-recovery check. | 0.5: only preview or recovery evidence is present. 1.5: both are present but one step or post-recovery check is missing. | 0: no usable preview or recovery evidence. |
| Freeze, evidence and explanation | 2 | The week-10 freeze and week-11 incident change are identified; the work record links revision, checks, observations and a short explanation. | 0.5: the revision is identified but checks or observations are missing. 1.5: revision and checks are linked but one observation or explanation is missing. | 0: the evidence cannot be tied to a revision. |
M3 — model audit verification (3 points)
The teaching team runs the model audit on the student's frozen revision. The number or severity of model findings does not determine the score; the rubric measures the quality of the student's verification evidence.
| Criterion | Points | Full credit | Partial credit | Zero |
|---|---|---|---|---|
| Runnable verification evidence | 1 | For the teaching team's model audit of the student's frozen revision, each finding is addressed with a runnable reproduction, counterexample or bounded unresolved result. | 0.5: the evidence is useful but one step, input or revision is missing. | 0: the verification cannot be rerun. |
| Evidence supports the conclusion | 1 | The verification evidence supports the stated conclusion across the review's evaluation, security and maintenance checks; the student distinguishes a demonstrated result from an untested suspicion. | 0.5: one conclusion or boundary is unsupported. | 0: the student's conclusions are unsupported by the evidence or merely restate the model output. |
| Self-check and uncertainty | 1 | The student adds one meaningful check of their own and records a no-finding or bounded limitation without presenting an unverified claim as confirmed. | No partial credit: an unclear check or boundary scores 0. | 0: no self-check or uncertainty boundary is recorded. |
M4 — repair and delivery (3 points)
| Criterion | Points | Full credit | Partial credit | Zero |
|---|---|---|---|---|
| Review response | 1.5 | The student responds to each confirmed model-audit finding, links repair commits and reruns the relevant checks; disagreements are reasoned. | 0.5: one confirmed finding has a response but its repair or rerun is missing. 1: most responses and reruns are complete but one is incomplete. | 0: the review is ignored or the response cannot be verified. |
| User-informed iteration | 0.5 | Three usability sessions scheduled across weeks 11–13 (or an approved equivalent) produce de-identified observations and a justified product change. | No partial credit: incomplete sessions or an incomplete resulting change, even with the gap recorded, score 0. | 0: no admissible user evidence or the evidence contains restricted material. |
| Final product and demo | 0.5 | The final product, deployment evidence and demo show the agreed change on the submitted revision with synthetic data. | No partial credit: a demo with one product or deployment check incomplete scores 0. | 0: the final product or demo is not usable or is from another revision. |
| Delivery record | 0.5 | The final work record lists the revision, checks, unresolved limitations and one AI-use note when applicable. | No partial credit: a record with one required entry unclear scores 0. | 0: no auditable delivery record. |
Evidence and boundaries
M0 and M4 live evidence must reference the teacher-checked consent and purpose record; students submit only de-identified material and the confirmation reference. A withdrawal removes the evidence and uses the approved equivalent route without lowering the score ceiling. A classroom rehearsal is not a participant session. Oral-defense marks, code lines, commit counts, rankings, awards and external-judge opinions never supply milestone points.
The teaching team records the project mark after reviewing the submitted evidence. Keep an unresolved or contradictory item pending; do not turn missing future work into zero by assumption.