Every row arrived. The decision still changed.

Every source record was present in the target.

The row count matched. The identifiers and values matched. Types, null checks, references, hashes and the selected aggregates were all green.

Then a downstream consumer read the migrated state and reached a different decision.

Nothing had disappeared. The meaning had moved.

The constructed failure was small. One account-like entity had three state-setting events. The third event was recorded last but took effect before the second. Effective time says when a state applied; recorded time says when the system learnt about it. The original consumer reconstructed current state using effective time. A flawed target projection used recording order instead.

The late event was new evidence about an earlier state, not the newest state. Sorting it last changed which state the consumer treated as current.

Both representations contained the same three events. At an as_of boundary of 20 January 2026 and a known_at boundary of 25 January, the effective-time consumer selected a score of 85 and returned HOLD. The recorded-time projection selected 55 and returned REVIEW. Here, as_of names the state date being reconstructed, while known_at limits the records available to the consumer. The fictional routing policy sends scores below 40 to ALLOW, scores below 70 to REVIEW, and everything else to HOLD.

This is the class of migration failure I find most interesting. The rows can be right while the answer built from them is wrong.

Recently, I have been working at a related boundary: reconciling loan-management records into a canonical dataset, then deriving recurring monthly reports from that dataset. The example in this essay is entirely synthetic and did not occur in that work. The experience did sharpen the question for me. Once a dataset becomes canonical, confidence depends on more than whether each field was copied correctly. It depends on whether the reports, measures and decisions downstream still mean what their users believe they mean.

My proposal is to test that boundary from the consumer backwards. Name the behaviour that must survive, record the semantics it depends on, then replay the same histories through both representations and compare the outcomes.

I call the pattern consumer-backward differential replay.

The checks passed for a reason

Row-level and structural validation are not weak practices. They answer necessary questions.

Google Cloud’s Spanner migration guidance recommends checking schema mappings, table counts, sampled rows, column aggregates and row-level hashes. It then recommends custom checks for more specific validation. AWS Database Migration Service can compare each source row with its corresponding target row and report mismatches.

Those checks protect a valuable boundary: did the intended data arrive accurately?

In the synthetic example, the answer is yes. The validation suite is green because it measures transport fidelity, and transport fidelity is intact. The failure appears only when a consumer interprets the records as a history and reconstructs the latest effective state.

This distinction becomes important whenever a migration also changes representation. Several source tables may become one canonical entity. A mutable row may become an event history. Codes may be normalised. Corrections may arrive after the period they correct. Currency, precision, null treatment or identity rules may move from one layer to another.

At that point, comparing physical rows is necessary but no longer sufficient. The source and target may not even have corresponding rows to compare. What can still correspond is their externally visible behaviour.

That idea has formal precedent. Wang and colleagues formalised equivalence for database-driven applications whose programs operate over different schemas. Their question was whether corresponding update histories lead to the same observable query result. Their Mediator prototype verified 20 of 21 benchmarks within a restricted intermediate language.

That is stronger and more formal than the engineering gate I propose here. It also clarifies the right unit of comparison: not necessarily storage shape, but observable behaviour under a declared model.

Three layers of migration confidence

I would separate migration confidence into three layers.

The layers accumulate. Decision preservation does not replace row validation. A consumer can happen to return the same answer while records are missing. Nor does one passing report prove that every interpretation of the target is correct.

The practical mistake is to let evidence from one layer silently make a claim about the next.

“Every row matched” is strong evidence about record fidelity.

It is not yet evidence that a monthly measure, eligibility classification or operational alert preserved its meaning.

Start from the other end

Migration plans usually move forward from producers. Inventory the source, define mappings, transform the data, load the target, then validate what arrived.

Consumer-backward replay adds a pass in the opposite direction.

Start with a consequential consumer and one precise behaviour. It could be a monthly total, a state classification, an alert threshold or a set of records selected for action. Then ask what must be true for that result to retain its meaning.

For the synthetic consumer, the dependency chain is short:

decision
  <- current state at the evaluation boundary
  <- ordering by effective time
  <- three state-setting events
  <- stable entity identity

In a real system the list may also include relationship cardinality, correction precedence, units, rounding, null treatment, timezone boundaries and tolerated numerical differences. The point is not to build a universal catalogue. It is to make the assumptions of one important consumer inspectable.

I would record them in a small, versioned coverage manifest:

consumer: synthetic_account_action_v1
outputs: [ALLOW, REVIEW, HOLD]

sample_boundary:
  as_of: '2026-01-20'
  known_at: '2026-01-25'

semantics:
  identity: account_id
  event_order: [effective_at, recorded_at, event_id]
  current_state: last_eligible_event
  threshold_policy: synthetic_policy_v1

comparison:
  route: exact

This manifest is not a data dictionary for the entire organisation. It is a finite statement of what one consumer assumes and what the cutover gate will compare.

The direction matters. Starting from all producer columns asks, “What could these fields affect?” Starting from the consumer asks, “Which meanings can change this outcome?” The second question creates a smaller and more accountable test surface.

There is adjacent work moving in this direction. PrismaDV, a 2026 CIKM demo paper, synthesises task-aware data tests by analysing both tabular data and downstream task code. It maintains a data-to-code assumption graph that links inferred constraints back to the code that consumes them. Its system and evidence are different from this proposal, but the shared lesson is useful: downstream use contains validation information that data profiling alone cannot recover.

Replay behaviour, not query text

Once the manifest is frozen, run the same input histories through both representations.

The old and new systems do not need identical tables or SQL. They need adapters that expose the same consumer boundary. Each adapter receives the same history and evaluation point, then returns a comparable output and the small amount of lineage needed to explain it.

same history
   -> source adapter -> source state -> source decision
   -> target adapter -> target state -> target decision

compare(route)
retain(selected state, lineage) for explanation

If the decisions differ, the verifier should retain the failure as a counterexample. Better still, it should minimise the history until removing any remaining event makes the disagreement disappear.

That reduction changes the debugging experience. Instead of “report totals disagree somewhere in the migrated dataset”, the engineer receives one entity, one consumer, one evaluation boundary and the smallest sequence that changes the result.

The executable example for this essay does exactly that. All nine structural checks pass, including canonical rows, hashes, references and selected aggregates. The manifest defines four as_of boundaries and four known_at boundaries, producing 16 scenarios. The flawed projection changes the route in four of them.

The reducer finds a subset-minimal witness containing only event-002 and event-003. At the 20 January as_of and 25 January known_at boundary, effective-time order is event-003, then event-002, producing score 85 and HOLD. Recorded-time order reverses them, leaving score 55 and REVIEW. No one-event history can produce an ordering disagreement.

After the target projection is corrected, it returns PROMOTE for all 16 scenarios under the same manifest. A negative-control manifest omits event_order; the harness evaluates zero scenarios and returns UNCOVERED rather than guessing.

This is differential testing at a domain boundary. It is not a claim to have invented observational equivalence, data contracts or replay. The proposal is how these pieces are combined: compile a finite gate backwards from named consumers, compare their outcomes over identical histories, minimise disagreements, and make uncovered semantics an explicit result.

Ambiguity should survive the test

Some differences are defects. Others are intentional product changes. Some cannot be classified because nobody has approved what the behaviour should be.

A binary pass or fail hides those cases. I would let the gate return three results:

• PROMOTE: every declared comparison passed within the frozen manifest.
• SEMANTIC_DIVERGENCE: a reproducible consumer outcome changed without an approved exception.
• UNCOVERED: the case depends on a semantic rule the manifest does not own.

UNCOVERED is not a softer pass. It says the evidence is insufficient to promote automatically. The next action is to obtain an accountable product decision, extend the manifest, and rerun the same case.

This is especially important when a migration is supposed to change behaviour. Equivalence is not always the goal. A new fee policy, correction rule or reporting definition may deliberately produce a different result. That divergence should be reviewed as a versioned product change with an owner and effective boundary, not normalised away as migration noise.

A recent preprint on metadata reconstruction, Rosetta, offers a useful adjacent principle. Its language model proposes column semantics, while deterministic evidence classes and a mechanically enforced gate control when the system commits or abstains. It does not evaluate migration equivalence. The relevant lesson is narrower: when evidence does not support a conclusion, abstention should be a system state rather than a sentence the model is merely asked to produce.

Where AI belongs

None of this requires a language model.

AI can still reduce some of the manual work. An agent can inspect downstream code and report definitions, propose candidate dependencies, generate histories around date boundaries, suggest likely semantic mutants, and explain a minimal counterexample.

It should not be the only oracle deciding whether its own inferred semantics are correct.

The durable gate should compare deterministic outputs against an owner-approved manifest. A model can propose that effective time governs current state. A domain owner decides whether that statement is true for this consumer. The replay harness then makes the decision executable.

This separation matters more as agents make transformations easier to generate. As mappings and repairs become cheaper to produce, establishing that the new representation preserves the intended behaviour does not become automatic.

The useful AI-native pattern is therefore not “let the model validate the migration it wrote”. It is “use the model to widen the search, then promote through evidence it cannot silently redefine”.

What a green gate should let us say

The synthetic artifact proves a deliberately small claim. For 16 bounded scenarios, one named consumer and one frozen manifest, the corrected target produces the same decision as the source. It also proves that the flawed temporal projection can pass all nine selected structural checks while changing the route in four scenarios.

It does not prove that row checks are unimportant. It does not verify arbitrary migrations. It does not discover every consumer, settle undocumented product meaning, or show that the same method has caught a production incident.

The statement I would want at cutover is narrower:

Within coverage manifest version X, the old and new representations produced equivalent outcomes for the named consumers over the tested histories and boundaries.

That sentence carries its own limit. It names the consumers, version and evidence surface. When a consumer changes, the manifest changes with it. When an incident reveals a missing history, that history becomes a regression case. When a product owner approves a behavioural change, the expected divergence becomes part of the next version.

There will always be unknown consumers and unimagined failures. Consumer-backward replay does not remove that uncertainty. It stops a green row count from concealing it.

A migration is not complete when the target contains the data.

It is complete, within an explicit boundary, when the people and systems using that data can still rely on the meaning of the answers it produces.