Skip to content

Week 2 studio: specifications and model-facing tools

Duration: 75 minutes, including a revision and a final check.

Purpose: turn your project proposal into failure-capable acceptance cases and make one small tool harder for a model to misuse. Work individually on a disposable copy of your project. This is formative studio work; the A2 and project proposal pages define the submissions.

Materials: your proposal, the notes, a text editor, Python 3.12, and a terminal. The retained fixture and session are optional reading.

1. Write cases before code (25 minutes)

Create a scratch directory and a proposal record:

mkdir -p week02-studio
cat > week02-studio/acceptance.md <<'EOF'
# Acceptance cases

## Case 1 — happy path
Given <precondition>
When <request>
Then <observable result>

## Case 2 — forbidden or malformed input
Given <precondition>
When <request>
Then <observable refusal and unchanged state>

## Case 3 — boundary value
Given <precondition>
When <request>
Then <observable result>
EOF

Replace every placeholder with your proposal's core scenario. Add a fourth or fifth case only when it exposes a distinct risk. For each case name the input, the observable result, and the state that must not change on failure. Predict one case that should fail before you run anything.

2. Make the cases executable (20 minutes)

Add a tiny checker beside the record. It must exit nonzero when a case is wrong:

cat > week02-studio/check_cases.py <<'PY'
def export_result(path: str) -> str:
    if not path.startswith("out/"):
        return "refused: read-only root"
    return "exported"


expected = {"happy": "exported", "forbidden": "refused: read-only root"}
actual = {
    "happy": export_result("out/summary.csv"),
    "forbidden": export_result("reports/summary.csv"),
}
missing = [name for name, value in actual.items() if value != expected[name]]
if missing:
    print("failed:", ", ".join(missing))
    raise SystemExit(1)
print("passed:", ", ".join(actual))
PY
python week02-studio/check_cases.py

Replace export_result with the smallest behavior from your project or a synthetic function. Keep the first failing output by changing the forbidden path temporarily, then restore it and make the smallest change that makes the checker pass. If your project cannot run locally, use a pure Python function in the scratch directory and state that boundary in the record.

3. Tighten one model-facing schema (20 minutes)

Choose one small tool your project uses. Copy its current name, description, schema and error text into week02-studio/tool-before.json. Add a closed schema, a required field, or an allowed path/enum. Make the error name the bad field and a repair. Save the revised form as tool-after.json.

If you do not have a tool ready, start with this deliberately loose example:

cat > week02-studio/tool-before.json <<'EOF'
{
  "name": "export_report",
  "description": "Write a CSV report.",
  "inputSchema": {
    "type": "object",
    "properties": {
      "path": {"type": "string"},
      "rows": {"type": "array", "items": {"type": "object"}}
    }
  }
}
EOF
cat > week02-studio/tool-after.json <<'EOF'
{
  "name": "export_report",
  "description": "Write a CSV below out/; never writes reports/.",
  "inputSchema": {
    "type": "object",
    "additionalProperties": false,
    "required": ["path", "rows"],
    "properties": {
      "path": {"type": "string", "pattern": "^out/[A-Za-z0-9_.-]+\\.csv$"},
      "rows": {"type": "array", "items": {"type": "object"}}
    }
  }
}
EOF

Run this direct schema smoke test (it uses only the standard library):

python - <<'PY'
import json
from pathlib import Path
before = json.loads(Path("week02-studio/tool-before.json").read_text())
after = json.loads(Path("week02-studio/tool-after.json").read_text())
assert after != before, "revise at least one model-facing field"
schema = after.get("inputSchema", after)
assert schema.get("type") == "object"
assert schema.get("additionalProperties") is False
assert schema.get("required"), "name a required input"
print("schema smoke test passed")
PY

Run three calls through a tiny stub so the schema and handler boundary are both exercised:

cat > week02-studio/check_tool.py <<'PY'
import json
import re
from pathlib import Path

schema = json.loads(Path("week02-studio/tool-after.json").read_text())["inputSchema"]
assert schema["type"] == "object" and schema["additionalProperties"] is False
assert set(schema["required"]) == {"path", "rows"}

def call_tool(args):
    unknown = set(args) - set(schema["properties"])
    if unknown:
        return "refused: unknown field " + sorted(unknown)[0], False
    if any(field not in args for field in schema["required"]):
        return "refused: missing required field", False
    if not re.fullmatch(r"out/[A-Za-z0-9_.-]+\.csv", args["path"]):
        return "refused: path; use out/<name>.csv", False
    return "exported", True

for args, expected, reached in [
    ({"path": "out/summary.csv", "rows": [{"id": "1"}]}, "exported", True),
    ({"path": "reports/summary.csv", "rows": []}, "refused: path; use out/<name>.csv", False),
    ({"path": "out/summary.csv", "rows": [], "extra": 1}, "refused: unknown field extra", False),
]:
    result, handler_called = call_tool(args)
    assert result == expected and handler_called is reached
    print(result)
assert not Path("out/summary.csv").exists(), "stub must not write files"
PY
python week02-studio/check_tool.py

The valid call reaches the stub handler; the two invalid calls are rejected first, with a field-specific repair and no file effect. Replace the example with your tool and keep the same three observations. The invalid calls must be rejected before the handler changes state, and their messages must tell the model what to fix.

4. Handoff and verify (10 minutes)

Write a six-line handoff containing the goal, constraint source, current files, last failed check, remaining unknown and next action. Compare each line with the evidence you collected. Then rerun the happy and failure cases after the schema edit. Finish with one decision line: Decision: accept | shrink | recheck | stop — evidence: <commands or paths>.

A partner is not required. At minute 45, pause for a staff check-in if your case cannot fail or your tool has no observable refusal. More time or an approved access accommodation should preserve the same cases and evidence.

Studio record

Keep acceptance.md, the before/after schema, the smoke-test output, the first failing run, the corrected run and the decision line in the scratch directory until feedback. Do not put credentials in the proposal, prompt, screenshots or logs. Use the Week 0 safety practices for any coding agent you choose; the studio commands themselves use no model or network.