Choose the Test Boundary from the Defect

Four test boundaries protect the same reservation flow. Then four defects arrive:

• a quantity of zero becomes valid;
• a database migration loses a uniqueness constraint;
• an inventory provider renames available to stock; and
• browser JavaScript submits an empty string instead of the entered quantity.

Calling all four tests “feature tests” would tell us almost nothing. Putting them into a pyramid would tell us how many tests we hope to own, but not which one can catch each defect.

A test can only expose a failure mechanism that exists inside its execution boundary. That gives us a more useful selection rule: name the defect, identify the mechanism required to produce it, then choose the narrowest test that contains that mechanism.

Draw what exists while the test runs

The word boundary is literal here. It is the set of real code, processes, storage engines, generated assets, and network hops present during execution. The reservation fixture has four:

Unit
PHPUnit ──▶ allocation policy

Integration
PHPUnit ──▶ Laravel database client ──TCP──▶ PostgreSQL

Contract
PHPUnit ──▶ HTTP inventory client ──HTTP──▶ provider process

End to end
Chromium ──▶ built JavaScript ──HTTP──▶ Laravel ──TCP──▶ PostgreSQL

These are observation boundaries, not importance levels. A unit test can be the strongest evidence for a pure rule. A browser test can be weak evidence for that same rule while being the only one of these four capable of observing a defect in the generated JavaScript.

Laravel’s default directory names do not settle the question. Its Unit tests do not boot the application. A Feature test may exercise several objects or an internal HTTP request. Those are useful defaults, but the directory name still does not tell us whether PostgreSQL, a separate provider, or a browser participated.

Zero is a policy defect, not a framework defect

The reservation policy rejects quantities below one:

final class AllocationPolicy
{
    public function assertReservable(int $quantity): void
    {
        if ($quantity < 1) {
            throw new InvalidArgumentException(
                'Reservation quantity must be positive.',
            );
        }
    }
}

The relevant defect changes the rule so zero is accepted. Nothing about that failure requires Laravel, HTTP, or a database. The focused test passes 0 and expects the exception. With the defect enabled, it fails because no exception is thrown.

The clean command completed in 296 milliseconds in the retained run. The seeded version failed in 291 milliseconds. Those durations include creation of a transient Docker container, so they are not a useful PHP microbenchmark. What matters is more direct: the test contains the values and policy capable of producing the defect, and no unrelated mechanism is needed.

Booting the framework would not make this assertion more truthful. It would only add setup and possible failure causes without adding observation power for the rule under test.

An in-memory repository cannot prove a PostgreSQL constraint exists

Now keep the PHP behavior correct and remove this from the migration:

$table->unique(['order_reference', 'sku']);

The intended invariant is that one order cannot reserve the same SKU twice. An in-memory repository can implement that rule perfectly. Its unit test can add a reservation twice and observe a domain exception. The test remains green even while the deployed schema permits both rows.

That green test is not misleading. It proves the in-memory implementation does what it claims. It cannot prove that a PostgreSQL constraint exists because no PostgreSQL server, migration, or index exists inside its boundary.

The integration test runs the real Laravel migration against PostgreSQL 17.6, inserts the same (order_reference, sku) pair twice, and expects the second insert to fail. PostgreSQL documents that a unique constraint is enforced by a unique index. With the clean migration, the test passed in 361 milliseconds. With the constraint removed, it failed in 334 milliseconds. During that same defective run, the in-memory negative control stayed green in 316 milliseconds.

The comparison tells us why the wider boundary earns its cost. The additional machinery is justified by the defect’s location in the schema and storage engine, not by an inherent superiority of integration tests.

A stub agrees with itself when the provider changes

The reservation service asks an inventory provider how many units are available. Its PHP port is small:

interface InventoryAvailability
{
    public function available(string $sku): int;
}

A consumer unit test supplies a hand-written stub that returns 5. The test proves the consumer can reserve 2 when its collaborator supplies that value. It remains green if the separately deployed provider starts returning this:

{"stock": 5}

The stub still returns an integer from available(). It has no way to discover that the provider’s serialized field changed.

The contract test crosses an actual HTTP connection to a separate provider process and passes the response through the production HTTP adapter. The clean provider returns {"available": 5}, and the test passed in 346 milliseconds. After the field changed to stock, the same test failed in 488 milliseconds with Provider response must contain an integer available field. The stubbed consumer test stayed green in 336 milliseconds.

This is the same substitutability problem from the other side. A test double is useful only within the contract it models. A shared executable check makes the behavioral contract visible when either side changes. It does not prove the provider is reliable in production or that every semantic compatibility rule has been captured.

Laravel’s HTTP helper never runs the browser code

The final defect appears in the submitted payload. The page contains a numeric quantity field, but changed JavaScript ignores it:

const quantity = form.dataset.browserDefect === '1'
  ? ''
  : Number(field.value)

Laravel’s internal request test posts the integer directly:

$response = $this->postJson('/reservations', ['quantity' => 2]);

$response
    ->assertCreated()
    ->assertExactJson(['quantity' => 2]);

That test still proves routing, validation, persistence, and the response for the request it makes. Laravel’s HTTP test documentation is precise about the omitted mechanism: its request methods do not issue a real network request; they simulate the request internally. The test never loads the built asset, finds the input, runs JavaScript, or serializes the browser’s value.

The Playwright test does. Chromium loads the served page and generated public/app.js, fills the field, clicks Reserve, sends the real HTTP request, and waits for Reserved 2. The clean journey passed in 2,170 milliseconds. With the JavaScript defect enabled, the internal Laravel request still passed in 396 milliseconds. The browser journey failed in 7,073 milliseconds after the page displayed Reservation failed and the expected text did not appear.

About five seconds of that failure is the assertion timeout. The result says nothing general about browser-to-contract-test speed. It establishes the narrower point: this browser boundary observed a defect the internal request boundary excluded.

The negative controls carry half the argument

A red test shows that something was detected. The green tests beside it show why the chosen boundary was necessary:

Without the negative controls, it would be easy to claim only that broader tests catch more. That is too crude. The policy test is deliberately narrow and catches its defect directly. The broader tests earn their place only when the risk is created by composition: a migration applied to PostgreSQL, two processes agreeing on JSON, or a built asset executing in a browser.

The complete reproduction pins PHP 8.4.12, Laravel 13.20.0, PHPUnit 12.5.31, PostgreSQL 17.6, Node 24.17.0, and Playwright 1.61.1. It retains the command, exit code, duration, and output for every clean case, seeded defect, and negative control. Composer reported no known advisories and npm reported zero vulnerabilities at the time of the run.

It remains one synthetic reservation flow on one machine. The measurements establish what this fixture did, not the runtime, reliability, or ideal distribution of tests in another system.

A pyramid can budget cost, but it cannot locate a defect

The testing pyramid remains a useful warning against putting every assertion through the most expensive path. A portfolio with many fast focused tests and fewer complete journeys can provide quick feedback and manageable diagnosis. That is a cost heuristic, not a defect-selection algorithm.

Quotas fail when the desired shape becomes more important than observation. A repository can own hundreds of fast tests which all replace the provider with the same stale stub. It can also own a browser-heavy suite that slowly repeats policy cases without testing the database constraint operators depend on.

The “narrowest relevant boundary” rule has reversal conditions. A wider test may be cheaper when the behavior is framework wiring and isolating it would require rebuilding the framework with doubles. A complete journey may itself be the contract for a critical checkout across owned components. In those cases, the wider path directly exercises the mechanism under test rather than serving as insurance added after a supposedly real test.

Even then, focused tests can preserve diagnostic precision for policy and adapters. The decision is not one boundary forever. It is one boundary for one named risk.

Let the failure mechanism choose the test

Before choosing Unit, Feature, Integration, Contract, or E2E, write down the wrong behavior the suite must reject. Then ask:

1. Which mechanism could produce it?
2. Which real code, process, storage engine, asset, or network hop must exist for that mechanism to run?
3. What is the narrowest test boundary containing those things?
4. Which plausible narrower test should remain green, and why?
5. What cost or uncertainty would make a wider boundary the better choice?

Coverage can show that the selected code ran, but it cannot answer these questions for us. As the coverage experiment showed, execution evidence and defect sensitivity are different claims.

Choose the boundary from the defect. Then use suite shape, runtime, and maintenance cost to decide how often and how broadly to repeat that evidence.