Skip to content

Week 3: Agent skills and the CLI studio

Learning objectives

This studio turns one repeated workflow into a tested skill. You will run it by hand, write a SKILL.md, predict three matches and one near miss, run the supplied checker, and preserve the evidence from one command-line agent session. The work supports LO2 and is the main preparation for A3.

Minimal example

Use the release-check example in the lecture notes. The hand run is the baseline. The skill is a proposal. The trigger table records a prediction and an observation; the checker reports structure, not whether an agent really selected it.

Hands-on exercise

Build and test one skill

Time: 70 minutes. Format: individual work with a ten-minute peer check. Materials: your project repository, the course checkout, and the read-only checker in lectures/week-03/fixture/skill_check.py. No special runner or paid account is required for the file check.

1. Choose and run a workflow (15 minutes)

Choose a non-destructive workflow you have already done twice. Write its shell commands and run them once by hand. Run steps 1 to 3 from the root of your own project repository: git rev-parse needs a Git repository, and every studio file goes in an A3-work/ directory there. The commands below are one such workflow:

mkdir -p A3-work/hand-run
printf '%s\n' "revision: $(git rev-parse --short HEAD)" | tee A3-work/hand-run/run.txt
git status --short | tee -a A3-work/hand-run/run.txt
git diff --check 2>&1 | tee -a A3-work/hand-run/run.txt

Add the project's focused test command and its output. If a command fails, preserve the failure and explain whether the skill should stop or continue. Do not run model-generated unknown commands in a directory containing employer, cloud, or other real credentials.

2. Write the skill (20 minutes)

Still in your project root, create the directory and file, then edit the file in your normal editor:

mkdir -p A3-work/skills/my-workflow
# Write A3-work/skills/my-workflow/SKILL.md from the notes' three-section example.

The front matter name must match my-workflow (or your chosen hyphenated directory). The description must say what the workflow does and include when. In Steps, name the commands. In Judgement, state at least one stop or branch. In Verification, name the exact check command and the evidence it leaves.

3. Predict four requests (10 minutes)

Copy this starter into A3-work/trigger-test.yaml and replace the prompts with your own:

skill: my-workflow
observer: agent
observer_detail: CLI run on the fixed revision of the student's project; each boolean is tied to the saved selection evidence.
prompts:
  - id: matching-request
    prompt: Run my repeated workflow and report the check output.
    should_trigger: true
    predicted: true
    observed: false
  - id: matching-words
    prompt: Inspect the same workflow before we review the change.
    should_trigger: true
    predicted: true
    observed: false
  - id: matching-paraphrase
    prompt: Perform the repository check I have used twice and preserve evidence.
    should_trigger: true
    predicted: true
    observed: false
  - id: near-miss
    prompt: Implement a new feature and add its tests.
    should_trigger: false
    predicted: false
    observed: false

The observed values are placeholders only. Fill them after the agent run. If any observation differs from should_trigger, add a revision section naming the description change and recheck exactly those IDs. Do not rewrite the original prediction.

4. Run the CLI and preserve evidence (15 minutes)

Run the coding-agent command supplied for your seat once on the fixed revision of your project. Keep the exact request and output in A3-work/agent-run.txt; do not include secrets. If the CLI does not print skill selection, record selection: not shown there. Keep each YAML observed value boolean and explain its evidence in observer_detail. Then change to the course checkout, where the checker lives, and run it with paths that point into your project:

PROJECT_ROOT=/path/to/your/project
uv run --locked python lectures/week-03/fixture/skill_check.py \
  "$PROJECT_ROOT/A3-work/skills/my-workflow/SKILL.md" \
  "$PROJECT_ROOT/A3-work/trigger-test.yaml"

The command reads YAML and Markdown only. It does not execute your skill or claim that the agent selected it.

5. Compare and decide (10 minutes)

Make a four-row prediction-versus-observation table. Cite the hand-run file, the CLI run, the checker output, and the revision. Write one sentence at the end: accept, narrow, recheck, or stop, plus the evidence ID. A peer checks that the near miss is actually near and that no observation was invented.

Verification

Your studio record is complete when the following commands run from the course checkout (the git -C commands read your project wherever you run them):

PROJECT_ROOT=/path/to/your/project
uv run --locked python lectures/week-03/fixture/skill_check.py \
  "$PROJECT_ROOT/A3-work/skills/my-workflow/SKILL.md" \
  "$PROJECT_ROOT/A3-work/trigger-test.yaml"
git -C "$PROJECT_ROOT" rev-parse HEAD
git -C "$PROJECT_ROOT" diff --check

Expected evidence is Structure checked., the fixed revision, a clean diff check (or the failure you recorded), and the four-row table. A structure failure is a repair-and-rerun case; it is not a reason to change observed to make the table look better.

Common mistakes

  • Writing the skill before doing the workflow by hand.
  • Using four prompts that all contain the skill name; at least three positive prompts must test the description in natural language.
  • Calling a near miss an error rather than evidence about the description boundary.
  • Filling observed from expectation when the CLI did not show selection.
  • Running an agent outside the course safety practices or leaving a key in the transcript.

A3 handoff

Submit the skill, trigger test, hand-run evidence, CLI evidence summary, checker output, prediction table, and one final decision line. A3 is due this week.