Week 3: Agent skills and the CLI

Learning objectives

By the end of this lesson you can:

run a repeated command-line workflow by hand and describe the inputs, decisions, and checks it actually needs;
package that workflow in a small SKILL.md, keeping the description specific enough to select and the body short enough to load;
write three requests that should trigger the skill and one near miss that should stay quiet, then compare your predictions with an observed run; and
preserve command output, the exact request, the repository revision, and the final verification result as evidence.

The skill is a workflow description, not a permission grant. A skill can suggest a command; the normal shell, repository policy, and your review still decide whether it runs. The examples on this page are course-authored records, not model observations.

Minimal example

Do the workflow once by hand

Suppose a project has a repeated release-check routine. Before writing a skill, do one run without an agent:

git status --short
git diff --check
uv run --locked pytest -q

The first command tells you whether unrelated edits are present, the second catches whitespace errors, and the last command is the project's check. Record the revision with git rev-parse --short HEAD and keep the output. If the check fails, write the failure and stop; do not hide it behind a skill.

Encode the procedure

A skill is a directory containing a SKILL.md. Its YAML front matter names the skill and describes when it applies. The body explains steps, judgement, and verification. This small example is deliberately narrower than “help with this repository”:

---
name: release-check
description: Run the repository's release checks and report evidence when a change is ready for review.
---
## Steps
1. Read the requested boundary and run `git status --short`.
2. Run `git diff --check`, then the repository's test command.
## Judgement
- If unrelated files are modified, stop and report them before testing.
- If a check fails, keep its output and do not call the change ready.
## Verification
Run `git diff --check` and `uv run --locked pytest -q`; include the revision and output.

The description is the selection surface. The format uses progressive disclosure: metadata is available before activation, while the body is loaded after selection and bundled resources only when needed (tools/agent-skills; as_of: 2026-10-07; Source 1). A long explanation in the description makes unrelated requests look like matches; a vague description makes the skill invisible. Compare the hand run with the skill line by line: which step is new, which evidence is preserved, and which decision would be impossible from the output alone?

Test selection, including a near miss

The course kit contains three complete examples: research-codebase, plan-change and review-diff. Read only each SKILL.md description first. For research-codebase, predict this table before looking at its trigger record:

Request	Prediction	Reason
“Trace this request and cite the files and lines you observed.”	trigger	It asks for a read-only path record.
“Tell me what the handlers and tests do before we discuss edits.”	trigger	It asks for current facts before a change.
“Build a read-only call-path record for this unfamiliar module.”	trigger	It names the same bounded research job in new words.
“Implement the change and add a regression test.”	stay quiet	It asks for implementation, the near miss.

Now open trigger-test.yaml and compare predicted with observed. The checker verifies the record's shape and that it contains at least three positive prompts and one negative prompt; it cannot observe a model's hidden selection. Keep a mismatch instead of rewriting history, describe the description change under revision, and recheck only the mismatched prompts.

Run the same check for the other two kit examples. Their trigger records are synthetic course examples, so label them as such in your notes. A passing structure check is not evidence that a particular agent selected the skill.

Evidence from a CLI run

A command-line session is evidence only when another reader can identify what happened. Save these fields for the run that uses your skill:

Field	Example evidence
Request	the exact prompt, copied before the run
Revision	`git rev-parse HEAD`
Selection	the CLI's skill-selection line, or “not shown”
Calls and output	the command, exit status, and relevant stdout/stderr
Verification	the exact check command and its result
Boundary	files changed, or `git diff --name-only` if none

Keep the transcript in a scratch directory outside the repository's product files, then copy only the short evidence summary needed for A3. Never put a key in a prompt, screenshot, transcript, or repository. Follow the course safety practices when you run an agent.

Hands-on exercise

Use your own project or the smallest repository you are currently working on. Choose a workflow you have already performed at least twice: for example, a focused test-and-review check, a release-note check, or a read-only status report. Do not choose a destructive workflow. Work in your own project repository's A3-work/ directory and keep the repository revision fixed while you compare predictions.

Hand run (15 minutes). Write the workflow as shell commands. Run it once by hand, save the output, and mark each line as input, decision, action, or check.
Write (20 minutes). Create A3-work/skills/<your-name>/SKILL.md with the same three sections as the example. The description must say what the skill does and when it applies. Keep the body below 500 lines.
Predict (10 minutes). Add three requests that should trigger and one near miss to A3-work/trigger-test.yaml. Fill predicted before you run the agent. Use a neutral observer_detail such as the CLI name and date; do not invent a classmate or teacher.
Run and observe (15 minutes). Use your provided coding-agent CLI once on the fixed revision. Preserve the exact request and the CLI output. Keep observed as true or false, based only on the selection evidence the CLI actually reports. If selection is not shown, write selection: not shown in A3-work/agent-run.txt and explain the basis for each boolean in observer_detail; do not put that phrase in the YAML boolean field.
Check (10 minutes). Run the structure checker, compare prediction with observation, and record one correction or a justified “no change.” The decision line at the end of A3 must say accept, narrow, recheck, or stop, followed by the evidence ID that supports it.

Verification

Run the checker from the course checkout, with paths that point to the files in your project. Run the Git commands from your project root. These commands are offline and use only the repository tools:

PROJECT_ROOT=/path/to/your/project
mkdir -p "$PROJECT_ROOT/A3-work/skills/release-check"
# Write A3-work/skills/release-check/SKILL.md and A3-work/trigger-test.yaml now.
uv run --locked python lectures/week-03/fixture/skill_check.py \
  "$PROJECT_ROOT/A3-work/skills/release-check/SKILL.md" \
  "$PROJECT_ROOT/A3-work/trigger-test.yaml"

git -C "$PROJECT_ROOT" rev-parse HEAD
git -C "$PROJECT_ROOT" diff --check
git -C "$PROJECT_ROOT" diff --name-only

The checker must print Structure checked. and no structural problems. Then inspect the trigger table yourself: there are at least three should_trigger: true prompts and one near miss, and observed records what the CLI showed. A successful checker run does not prove that the agent selected the skill. A failed checker run is useful evidence: fix the smallest structural problem, rerun the same command, and keep both outcomes in your A3 record.

Common mistakes

Starting with a description instead of a hand run. Re-run the workflow manually and record the command and output before editing SKILL.md.
Making the description a slogan. Replace “helps with code” with a task and a when clause; test the near miss again.
Calling the near miss a failure. A skill staying quiet for an implementation request is the expected result. Keep the prediction and observation side by side.
Treating skill_check.py as an agent. It reads files only. It checks structure and consistency, not selection; do not report its output as a model run.
Saving only a final summary. Keep the request, revision, command, exit status, and verification result so another person can reproduce the claim.
Putting credentials in evidence. Stop, remove the secret, revoke it with the provider, and tell the teaching team. Follow the safety practices in the course tools and model access guidance before continuing.