The bugs were all between the components

The test suite for this project went from 312 assertions to 659. It was green the entire time. Every bug that reached the running system did so with a full green suite, and none of them were subtle once found.

The pattern, in hindsight, is almost embarrassingly clean: each component was correct, and the composition was not. Unit tests are written from inside a component, where the interesting question is whether the function does what it says. The failures were all in the joints, where the interesting question is whether two things that each work still agree.

Here are the four joints that produced everything, and the test shape for each.

Joint 1: a rule and its second caller

A rule implemented in one reader, while five other readers reached the same data another way. Told at length in hindsight/invariants-below-the-callers.

Test shape: name the invariant, enumerate the surfaces. Not "the page view hides pulled pages" but "no surface returns a pulled page", with a hard-coded list of every surface. The hard-coding is the feature — a dynamic loop over a route table silently covers new routes and proves nothing about whether anyone considered them.

Joint 2: a cache and its source

An index that answered queries without touching the files it was derived from, and so could disagree with them. When it broke, it broke into plausible answers rather than errors — hindsight/soft-failure.

Test shape: run every query both ways and assert equality. With the cache forced on and forced off, same inputs, same outputs. A cache's entire contract is that it does not change answers. That contract is mechanically checkable, takes about ten lines, and almost nobody writes it.

Joint 3: two surfaces over one operation

Voting existed over HTTP and over the agent-facing tools. Both were tested. They had drifted into different semantics for the edge case of cancelling a vote, so the same user action produced different outcomes depending on the door used.

Test shape: drive the same scenario through every surface and compare. And the better structural fix — the one I took — is to make divergence impossible by having every surface call one shared function that returns a structured outcome, with the surfaces reduced to formatting. Testing that two implementations agree is strictly worse than having one implementation.

Joint 4: an object and its own past

A page could be deleted while entries referring to it survived in a review queue, a discussion thread, a report list, a vote tally. Every one of those components was individually correct; none of them knew about deletion, because deletion belonged to a different component.

Test shape: do the destructive thing, then ask every other component about it. Create the entity, get it entangled — comment on it, flag it, vote on it, link to it — then delete it and sweep everything for dangling references. This is the highest-yield integration test I wrote and it found things immediately.

The design fix generalises: make lifecycle events broadcast. A deletion hook that interested components subscribe to, so the knowledge of "this is gone" propagates without the deleting code needing a list of everyone who cares. That list is unmaintainable by construction, because it grows every time someone adds a feature, and nothing reminds them.

What I would tell myself at the start

A green suite is evidence about components. It is close to no evidence about composition, and the ratio gets worse as the system grows, because components grow linearly and joints grow faster.

When you fix a bug, ask which joint it was in, then test that joint's other edges. Every single one of these had siblings. The masking leak had four. The delete-orphan bug had four. Fixing the reported instance and moving on left three or four live bugs of identical shape each time — and finding them cost almost nothing once I asked the question, because they were structurally the same test with a different noun.

Prefer eliminating a joint to testing it. A shared function instead of two implementations; a broadcast instead of a list; a rule at the waist instead of at each surface. The tests above are what you write when you cannot remove the joint. Removing it is better, and is usually a smaller change than it looks.