Skip to content

Tests that notice a fault

Answer these formative prompts on your own after the studio, in your existing work record. They are not an additional submission. Each answer should name the input, run or diff line that would let another reader check it.

Use the public answer guidance after your first attempt.

The prompts use the level parser from the notes: exactly one of "0" to "9" returns the corresponding integer, and every other string raises ValueError.

  1. A test contains result = parser("3"), then assert result == result, assert parser and assert result or True. For each assertion, say why it cannot fail on a wrong integer result, and give one replacement that can.
  2. A suite checks only parser("3") == 3 and that parser("x") raises ValueError. Of the mutants all_zero (every accepted string returns 0), nine_to_zero (only "9" returns 0) and accept_trailing ("12" is accepted as 1), which survive, and why?
  3. Write one test for each survivor in prompt 2. What two runs show that a new test has closed its gap?
  4. A report says "mutation result 3 of 3". Name three things you need to know before that sentence tells you anything.
  5. One mutant changes only the text of the ValueError message, and every test still passes. Is that a gap in the tests? What do you write in the report?
  6. The property "for every n from 0 to 9, parser(str(n)) == n" passes on a parser that returns 1 for "12" and raises IndexError for "". Why does it pass, and what property would fail?
  7. A generated input "1836" is accepted by the faulty parser. Shrinking gives "836", "36", "06", "00" and stops. Why does it stop at "00", and what does "00" tell you that "1836" did not?
  8. The course fixture's shrinker deletes one character or replaces one with 0, and keeps a change when the same failure category remains. Name two things a property-testing library does that this helper does not, and one rule both follow.
  9. An agent reports: "Fixed the failing check; two inputs were outside the supported range, so I removed them. All checks pass." The diff changes for text in ("", "12", "x"): to for text in ("x",): and touches no other file. List three checks you run, in order, and what each shows.
  10. Give two forms of test weakening other than deleting inputs, and say when changing a test's expected result is the right change.

Transfer to your project

Take the survivor you killed in the studio. State the one-line fault, the test you added, and the two run results. Then name one fault in the same function that you did not plant, and say whether your current tests would notice it and how you know.

Spaced retrieval

R1. A hook is meant to block a bad commit. What two runs show that the hook is a gate, and why is a passing commit alone not enough?

R2. A model review of your change reports no findings. What does that support, and which check from this week would you add before accepting the change?