A Characterization Test Is a Boundary, Not a Recording

An undocumented postage calculator needs to be split apart. Before touching it, we put its current behavior under test.

One suite stores the returned quote, every intermediate calculation, the order of the diagnostic fields, and the sequence in which four pricing rules run. Another asserts the caller’s result across weight, insurance, destination, and discount boundaries.

Both suites are green. Both characterize the current implementation. They do not buy the same freedom to change it.

Reorder two independent calculations and the quote stays identical. The first suite fails on call order. Reverse the diagnostic fields and it fails again. The second suite stays green for both changes, then fails when a one-cent insurance boundary is changed.

That is the distinction this article tests. Characterization is not the act of recording everything a program happens to reveal. It is the work of choosing which observed behavior must remain stable while a particular change is made.

Observation comes before endorsement

The example is a synthetic Laravel 13 postage service. It accepts a destination, package weight, declared value, and customer tier. The old class mixes validation, zone selection, four calculations, and a diagnostic trace:

LegacyPostageQuote
├── destination zone
├── weight-band base price
├── remote-area surcharge
├── declared-value insurance
├── loyalty discount
└── diagnostic trace

The reproduction starts by running the old class over a bounded matrix and retaining its complete output. That discovery pass answers a narrow question: what did this version do for these inputs?

It accepted one gram and 1,000 grams at 500 cents, charged 900 cents at 1,001 grams, and added one cent of insurance just above a 10,000-cent threshold. A remote destination cost 1,000 cents. Unsupported destinations and zero weight returned stable refusal codes.

It also accepted a parcel with zero declared value and returned a 500-cent quote. The fixture records that result, but the article cannot call it correct. No requirement or product decision was supplied. Changing it must be visible during the refactor; whether it should remain afterward is a separate decision.

That separation matters. Old code can tell us its present answer. It cannot tell us whether the answer was intended, whether a consumer depends on it, or whether a past defect should become policy.

Michael Feathers frames risky change with two different questions: how will we know the change works, and how will we know existing behavior was not broken? His publisher-hosted chapter on changing legacy code supports the need for both checks. It does not decide which postage behavior belongs in either one.

Select a boundary for the change

The planned change is structural. Extract pricing rules from the mixed class without changing what a caller receives.

For this change, the selected result is deliberately small:

accepted: status, amount_cents, currency, service_code
refused:  status, reason_code

The test matrix exercises adjacent values around the rules most likely to move:

• the lowest valid weight, the exact weight edge, and one gram above it;
• the exact insurance threshold and one cent above it;
• local and remote destinations;
• standard, local gold, and remote gold customers;
• an unsupported destination and invalid weight; and
• the unresolved zero-declared-value case.

This boundary is not universal. If an HTTP client consumes a byte-for-byte JSON document, serialized field names and sometimes order may belong to the contract. If an operator uses a diagnostic event for an automated repair, that event is no longer an incidental trace. Consumer evidence changes what the test must retain.

Within this fixture, the caller-visible suite inspects none of the diagnostic keys, private structure, or collaborator sequence. A structural analyzer checks that statement against the test source. The suite can therefore speak only for the public quote and refusal result. It makes no larger claim about hidden consumers in a real system.

Feathers calls a place where behavior can be changed without editing there a seam. The extracted pricing collaborators create such seams, but the interfaces do not prove that the extraction preserved behavior. The same eleven requests run against both implementations. Twenty-two selected examples pass, and a separate parity test compares all eleven results directly.

A useful safety net needs negative controls

A red test after a refactor could mean the behavior changed. It could also mean the test objected to the refactor itself.

The fixture keeps a deliberately broad recording suite to expose that difference. For one remote gold quote, it stores this diagnostic order:

total
discount
discountable amount
insurance
remote surcharge
base price
zone

It also records the calculation sequence:

insurance → remote surcharge → base price → discount

Neither order affects the final amount. The remote surcharge and insurance do not consume one another’s result. Reversing the diagnostic fields preserves all 22 caller-visible assertions and breaks the recording suite. Calculating the base price before insurance produces the same crossed result.

PHPUnit’s official test-double documentation describes mocks as observation points for communication and warns that strict call order can bind a test to implementation details. The fixture demonstrates the narrower consequence here: the recorded sequence rejects a harmless change which the selected quote contract accepts.

This does not make a full recording useless. It was useful during discovery. It would remain useful if the complete artifact were the contract. The problem appears when an exploratory capture is promoted, without review, into a permanent compatibility promise.

A narrow public assertion can still be weak

Avoiding private details does not automatically produce a good characterization test.

The fixture includes a third suite with one ordinary local quote. It asserts the same public fields as the selected matrix and passes with the clean code. Then two defects are introduced independently:

1. insurance starts at the exact threshold instead of one cent above it;
2. insurance rounds down instead of up.

The one-case suite remains green because its declared value never reaches either partition. The selected matrix fails both changes.

The difference is input selection, not test vocabulary. A test can avoid every mock and private method yet still miss the rule being moved. The preceding article on choosing a test boundary from the defect asked which mechanisms exist while a test runs. Legacy work adds another question: which input partitions make those mechanisms observable?

Seven defects test the selected contract

The reproduction applies seven behavioral mutations one at a time:

The selected matrix rejects all seven. The mutator verifies that each change touches one declared source seam and retains a source diff and hashes. A semantic verifier checks the expected failures, harmless controls, raw observations, and review signal.

The numbers are evidence about this fixture, not a target test count. Eleven carefully chosen cases happen to cover its decision boundaries. A real service may need fewer examples, many more, a property test, or an integration boundary which includes the database or framework.

The Laravel testing guide distinguishes isolated tests from feature tests which exercise larger collaborations. That is useful framework terminology. Directory names still cannot decide whether a legacy change needs the container, a database, a remote protocol, or only one deterministic rule. The defect mechanism makes that choice.

Suspicious behavior needs a named owner

The zero-declared-value case deserves different treatment from the seven mutations.

Changing it from a 500-cent quote to a refusal makes the selected suite fail. The verifier records that failure under review, not negative. We know the refactor would have changed an observed result. We do not know which result the product should offer.

There are several honest next steps:

• find a requirement, support policy, or API contract which owns the value;
• inspect real consumers and historical decisions;
• ask the responsible domain or product owner;
• preserve the old result temporarily and track the decision; or
• change it deliberately, with migration and communication where needed.

Leaving the question visible is better than hiding it inside a green snapshot. A characterization test can prevent an accidental change while the decision is open. It should not impersonate the missing authority.

Other techniques may carry the risk better

A characterization matrix is one tool, not a required ceremony for every old class.

Use a focused test when the rule is already accessible and composition adds no risk. Use static analysis or stronger types for source relationships they can prove. Put a contract test around an adapter when compatibility across implementations is the concern. Include Laravel, PostgreSQL, a filesystem, or a real protocol when the proposed change crosses that mechanism.

A broad approval test is attractive when output is large or poorly understood. Keep it broad when consumers require the whole artifact. Otherwise, use it to discover variation, then remove volatile fields and select meaningful partitions before treating it as a lasting contract.

Traffic replay can reveal inputs a synthetic matrix missed, but it introduces privacy, representativeness, nondeterminism, and side-effect problems. A rewrite may be cheaper when the boundary is small and intended behavior is known. It becomes harder to justify when unknown consumers and failure semantics make comparison impossible.

The choice reverses with the risk. “Always write characterization tests” is no more useful than “always snapshot the output.”

Buy permission for one named change

A practical legacy-code pass can stay compact:

1. Name the change you intend to make.
2. Observe the old behavior broadly enough to discover its partitions.
3. Identify callers, consumers, effects, and failure semantics at risk.
4. Select the smallest boundary and input matrix which can expose those risks.
5. Mark suspicious behavior as unresolved instead of silently endorsing it.
6. Run the same contract against the changed design.
7. Try one harmless structural change and one plausible behavior defect.
8. Keep broad recordings only where their details are genuine contracts.

The clean fixture was also exported from commit 0dae65a10911fa3e307532d1fcd96caa7f9d0733 into a temporary directory. Its observations, verifier, analyzer, and representative mutation diffs matched the retained evidence by SHA-256. That proves the example is reproducible from the committed tree. It does not prove that its invented postage policy belongs in another system.

The useful outcome is not a perfect recording of yesterday’s code. It is a test boundary which stays quiet while the structure changes and speaks when a behavior the change must preserve moves with it.