AI Should Shorten the Evidence Loop

The candidate answer looks sensible: parse a large JSON identifier with a reviver and convert it to BigInt.

The input is 9007199254740993. The conventional reviver receives 9007199254740992. It then returns 9007199254740992n, preserving the wrong integer with perfect precision.

Nothing in that answer sounds reckless. It names a real API, uses an appropriate JavaScript type, and fits in a code review comment. The defect only becomes visible when someone turns its claim into an executable question: which value reaches the callback?

That is the useful division of labour for AI-assisted research. Let the assistant accelerate candidate explanations, source discovery, competing hypotheses, and test design. Keep the system model, evidence threshold, and decision with the engineer.

Fluency is a candidate generator

An answer is not evidence because it is detailed, cited, or syntactically correct. Those properties make it easier to inspect. They do not make the conclusion true.

The AI-01 fixture retains two synthetic artifacts for one question:

How should a JSON API carry decimal identifiers which may exceed JavaScript’s safe integer range without changing identity?

The answer-first artifact recommends a reviver and BigInt. It is deliberately plausible, not deliberately foolish. The evidence-loop artifact instead breaks the question into assumptions, claims, primary-source candidates, executable cases, counterexamples, unresolved questions, alternatives, and a final human-owned decision.

Neither artifact came from a named model. This is not a model benchmark. It is a protocol fixture for the difference between receiving an answer and building a route by which an answer can be rejected.

The first executable case establishes the boundary:

ordinary JSON.parse
input:    {"id":9007199254740993}
expected: 9007199254740993
observed: 9007199254740992

The conventional reviver cannot recover digits it never receives:

two-argument reviver
input source: 9007199254740993
callback:     9007199254740992
result:       9007199254740992n

RFC 8259 permits JSON implementations to set limits on number range and precision. It identifies integers in the binary64 exact-integer range as interoperable. The ECMAScript definition of Number.MAX_SAFE_INTEGER supplies the matching JavaScript boundary. The reproduction then demonstrates the consequence in the pinned runtime instead of asking either source to prove more than it says.

The candidate has now done useful work: it supplied a falsifiable claim. It has not supplied the decision.

Separate what the loop knows

A flat list of links and test output is still easy to overread. The fixture classifies every material claim as one of five things:

That distinction prevents evidence from quietly gaining authority as it moves through the article.

For example, Node 24.12.0 is an observation boundary, not shorthand for every JavaScript client. The synthetic contract assumes some clients do not expose source text during parsing. It also assumes identifiers are opaque, unsigned, non-zero decimal tokens with no meaningful leading zeros, and that consumers do not perform arithmetic on them.

Those are not facts discovered by the experiment. They are inputs to the decision. In a real API, the client inventory and schema history must supply them.

Three questions therefore remain deliberately open:

• Which runtimes and generated SDKs must remain compatible?
• Does an existing consumer require a numeric schema or perform arithmetic?
• Would a lossless JSON parser meet the real performance and maintenance constraints?

A confidence score would only hide the shape of that uncertainty. Three unresolved compatibility questions do not become small because ten other claims have evidence.

This is the same separation used in Coverage Tells You What Ran. A measurement can report which code executed without proving that the important behaviour was asserted. Here, a source or command can support one claim without authorizing the architectural choice around it.

Critique should be allowed to improve the claim

The first counterexample appears to support a broad rule: revivers are too late to recover a large integer.

That rule is also wrong.

Current JSON.parse semantics may provide a third context argument for primitive values. Its source property exposes the original source text. In the pinned Node runtime, this case succeeds:

const parsed = JSON.parse(
  '{"id":9007199254740993}',
  (key, value, context) => key === 'id' ? BigInt(context.source) : value,
);

// parsed.id === 9007199254740993n

The ECMAScript JSON.parse specification describes the source text supplied while internalizing primitive properties. MDN’s JSON.parse reference also distinguishes the parsed value from context.source. The fixture verifies that behaviour rather than stopping at the convenient first rebuttal.

The revised claim is narrower: a conventional two-argument reviver sees the already converted Number; a source-context reviver can recover the digits in a runtime which supports it.

That correction matters twice. Technically, it keeps a credible alternative open. Editorially, it shows whether the research loop can change its own conclusion. A workflow which uses critique only to decorate the initial answer is answer-first work with more files.

The fixture records four critique findings. It rejects the conventional reviver claim, revises the overbroad reviver rebuttal, rejects the idea that BigInt solves the wire representation, and rejects decimal strings without a validation policy. Each finding points to the source or executable case which earned the change.

Memory precision does not define the wire contract

BigInt represents the large integer exactly in memory. Ordinary JSON.stringify then throws a TypeError. MDN documents that default serialization behaviour and the possible policies, such as a replacer or custom toJSON method.

This is not an argument against BigInt. It is a reminder that three different questions are in play:

• What syntax crosses the wire?
• What representation does the parser produce?
• What type does the application use internally?

One type does not need to answer all three. A client can receive a validated decimal string and convert it to BigInt internally. Another contract can retain numeric JSON syntax and require a lossless parser. The wire decision must account for all declared clients; an in-memory type only has to serve one application boundary.

For the synthetic contract, the selected wire representation is a JSON string matching ^[1-9][0-9]*$. The fixture round-trips "9007199254740993" exactly and tests eleven grammar cases. It accepts 1 and the large identifier. It rejects zero, leading zeros, signs, fractional and exponent forms, empty input, whitespace coercion, and non-decimal text.

The regular expression is not a universal identifier standard. It follows from the declared synthetic domain. If zero or meaningful leading zeros exist in the real identifier space, the grammar must change before the code does.

The attractive alternatives stay on the table

Strings win this bounded decision because the declared clients do not all offer source-text parsing, identifiers need no arithmetic, and the contract requires exact identity. That does not make every other option defective.

A permanent Number.MAX_SAFE_INTEGER bound is simpler when the issuing domain can enforce it. A source-context reviver preserves numeric wire syntax for compatible JavaScript runtimes. A lossless JSON parser can preserve arbitrary numbers across other clients, at the cost of a client capability whose performance and maintenance have not been evaluated here.

Any existing numeric contract adds another cost. Changing its schema to a string may break validation, generated clients, stored fixtures, and consumer code. The fixture therefore requires an explicit versioned migration instead of treating the locally safer representation as a silent patch.

The choice should be reopened if every required client adopts a verified lossless representation, if the issuing domain guarantees the safe-integer bound, or if compatibility requires numeric JSON syntax. Those conditions turn “it depends” into questions which can actually reverse the decision.

This resembles the role of reversal conditions in A Decision Record Needs a Way Back. The point is not to make a decision permanent. It is to preserve why it is reasonable until named evidence changes.

The engineer owns the last mile

The fixture’s synthetic contract owner selects canonical decimal strings. The record cites the exact cases and sources, keeps the rejected and deferred alternatives, lists consequences, and leaves the three compatibility questions unresolved. It also states that neither candidate artifact authorized the decision.

That final sentence is more than governance language. Someone must understand why the observed rounding matters, why context.source changes one claim but not the client assumption, why BigInt is not a wire format, and which new evidence would reverse the choice. If nobody can explain that chain without the assistant, the research loop produced a deliverable without producing an owner.

The reproduction is synthetic. It does not describe Brian’s API, clients, measurements, production history, or preference. It retains seventeen cases, ten categorized claims, four critique findings, thirty rejected semantic mutations, six source responses, twenty-two source hashes, a zero-finding dependency audit, and a clean export of commit f9159547fe70d47d9724dc666d4305c2b6ca0ec3.

AI earns its place in technical research when it reduces the time between a plausible sentence and the evidence which can defeat it. The engineer still chooses the question, checks the boundary, owns the uncertainty, and signs the decision.