Skip to content

Agent skills and the CLI

Week 3 · LO2

Today

Run a workflow by hand → encode it → test its trigger boundary → keep CLI evidence.

A skill is a procedure

  • SKILL.md front matter: name and description
  • Body: Steps, Judgement, Verification
  • Selection is a claim to test

Start by hand

git status --short
git diff --check
uv run --locked pytest -q

Save the revision and output before writing the skill.

The description is the boundary

Specific: “run release checks and report evidence when a change is ready for review.”

Vague: “help with code.”

Which one stays quiet for “implement a new feature”?

The body is the procedure

## Steps
1. Read the boundary.
2. Run the check.
## Judgement
- Stop on unrelated edits.
## Verification
Run `git diff --check` and the focused test.

A procedure does not grant permission.

Three positives, one near miss

Request Expected
Trace a request and cite files trigger
Report handlers before edits trigger
Map an unfamiliar module read-only trigger
Implement the change and add tests quiet

Predict before opening the trigger record.

Read the kit examples

research-codebase · facts before edits

plan-change · bounded plan before implementation

review-diff · reproduce findings in an existing diff

Read descriptions first; bodies are loaded after selection.

What the checker checks

skill_check.py checks:

  • front matter and directory name
  • required sections and links
  • 4–12 prompt records
  • mismatch revision details

It does not run a model or observe hidden selection.

Run it directly

# From the course checkout; PROJECT_ROOT is your project.
uv run --locked python lectures/week-03/fixture/skill_check.py \
  "$PROJECT_ROOT/A3-work/skills/my-workflow/SKILL.md" \
  "$PROJECT_ROOT/A3-work/trigger-test.yaml"

Expected: Structure checked.

Prediction versus observation

  • should_trigger: your intended boundary
  • predicted: what you expected before the run
  • observed: what the CLI actually showed

A mismatch is evidence. Keep it and revise the description deliberately.

CLI evidence

Keep the exact request, revision, selection line (or “not shown”), command, exit status, stdout/stderr, and verification result.

A final summary without its command is a claim, not evidence.

Safe command line

Use a dedicated directory. Keep employer and cloud credentials out. Do not paste keys into prompts, screenshots, logs, or commits. For generated commands, inspect before running.

Studio handoff

A3 contains:

  1. hand-run commands and output
  2. SKILL.md
  3. four trigger records
  4. CLI evidence summary
  5. checker output and one decision line

Verify before accepting

git rev-parse HEAD
git diff --check
git diff --name-only

Then compare every observed value with the saved transcript.

Common failure modes

  • writing before the hand run
  • a description broad enough to match everything
  • treating quiet as failure
  • inventing an observation the CLI did not show
  • reporting checker output as model behavior

Exit question

What is the nearest request that should stay quiet for your skill?

Write it down before you submit A3.