A Characterization Test Is Not a Product Decision

Fri, 01 Mar 2019 00:00:00 +0000

A label setting behaved differently in test and production. Some customers had come to depend on the difference even though the product already let them choose the label size they wanted.

A characterization test could record both defaults. It could tell us which callers would notice if we made them equal. It could not answer the important question: should the difference survive?

Keeping it would protect customers who had never configured the setting. Removing it would give every environment one rule and make the customer’s choice explicit. The same observation supported arguments for either response. Only one response matched the product we wanted to operate.

Product-Decisions on Brian Faust

A Characterization Test Is Not a Product Decision