Test Engineering

一条测试只有在「存在某个合理的错误实现会被它抓住」时才值得写——覆盖率明确不是目标。

针对「每个函数一条正常路径测试」的默认做法——覆盖率上去了，抓 bug 的能力没变。写每条测试之前，模型必须先说出它能抓住的那个错误实现；说不出就不要写。它还要求先读既有测试并沿用其结构，并通过「亲眼看它失败」来证明测试确实接在了代码上。附加章节涵盖测试先行地修 bug，以及诊断不稳定测试——找出非确定性的来源，而不是加重试或加 sleep。

# Test Engineering

**Use when:** you want tests that catch bugs, not tests that raise a coverage number.
**Pairs with:** [debugging.md](debugging.md), [refactoring.md](refactoring.md)

Ask an agent for tests and you get one test per function, all of them exercising the happy path with a clean input. Coverage goes up. Bug detection does not move.

---

## The prompt

```text
Write tests that would catch a real bug. Coverage is not the goal and I
do not want to hear about it.

THE STANDARD
A test earns its place only if there is a plausible wrong version of the
code that it would catch. Before writing each test, name that wrong
version in your head. If you cannot, do not write the test.

BEFORE WRITING
- Read the code under test. What can actually go wrong here? List it.
- Read the existing tests. Match their structure, naming, and helpers
  exactly. A test file that looks different is a test file nobody reads.
- What is already covered? Do not duplicate it.

WHAT TO COVER, IN THIS ORDER
1. The contract. What the function promises, for a normal input.
2. Boundaries. Zero, one, many, maximum, and the value on each side of
   every comparison in the code. Off by one lives here.
3. Empty and absent. Empty string, empty list, empty object, null,
   undefined, missing key, missing file.
4. Wrong shapes. Wrong type, malformed input, unexpected extra data.
5. Failure paths. Every call that can throw or return an error: make it
   fail and assert what the caller does. This is the most skipped and
   most valuable category.
6. State and order. Called twice. Called out of order. Called after a
   failure. Interrupted halfway.
7. Concurrency, if the code can be entered twice at once.
8. The regression. If this is a bug fix, a test that fails on the old
   code. Show it failing.

TEST QUALITY RULES
- One reason to fail per test. If a test can fail for two reasons, split
  it.
- The name says the condition and the expected result. Not "test login".
  "returns 401 when the token has expired".
- Assert the actual value, not just that something is truthy or that no
  error was thrown.
- No logic in tests. No loops, no conditionals, no computed expected
  values. A test with a bug is worse than no test.
- Test behaviour through the public surface, not private internals. A
  test coupled to internals blocks every future refactor.
- Deterministic. No real clock, no real network, no random without a
  fixed seed, no dependence on test order.
- Fixtures should be the smallest input that shows the case. A 200 line
  JSON fixture hides which field mattered.

MOCKING
- Mock what you do not own and what is slow or non deterministic.
- Do not mock the thing under test.
- A test that mocks everything asserts only that you wrote the code you
  wrote. Prefer a real object where it is cheap.

PROVE THE TESTS WORK
This is required, not optional.
For each important test: break the source code deliberately, in a small
realistic way, run the test, and show it failing. Then restore the code
and show it passing.
A test you never watched fail is not known to work.

REPORT
- The list of behaviours you now cover.
- What you deliberately did not test and why.
- Which tests you verified by breaking the code, with the output.
- What remains untested and would matter if it broke.
```

---

## The mutation question

The single line that most improves generated tests:

```text
For each test, name the specific wrong implementation it would catch. If
a test would still pass against a broken version of the code, delete it.
```

This is mutation testing done in the model's head. It removes the "it returned something and did not throw" tests that make up most generated suites.

---

## For an untested codebase

```text
Do not try to cover this codebase. Pick the highest value tests first.
Rank the untested code by: how bad is it if this is wrong, times how
likely is it to be wrong or to change. Give me the ranked list with your
reasoning, then write tests for the top three only.
```

---

## For a bug fix

```text
Write the regression test before the fix.
1. Write a test that reproduces the bug. Run it. Show it failing, with
   the failure message, and confirm the message describes the real bug
   and not a setup mistake.
2. Fix the code.
3. Run it. Show it passing.
4. Run the whole suite. Show it passing.
Test first is not optional here. A test written after a fix usually tests
the fix, not the bug.
```

---

## For flaky tests

```text
Do not add a retry and do not add a sleep. Both hide the problem.
Find the source of non determinism: time, ordering, shared state between
tests, real network, unawaited async work, randomness, or a resource not
cleaned up. Prove which one by making the test fail on demand. Then remove
the non determinism at its source.
```

---

## Why it works

The standard at the top ("name the wrong version this catches") is the whole prompt in one line. Everything else supports it.

The "prove the tests work" step is the part agents will try to skip. Watching a test fail is the only way to know it is wired to anything. A surprising number of generated tests pass against completely broken code because of a bad mock or a missing await.