Blog
Converted is not done
By Ted Ghose, Co-founder and CTO · Published · 10 minutes read
The short answer
To prove a migrated warehouse produces the same results, reconcile every converted object against the original on an identical snapshot of inputs, checking in order: schema, row counts, column aggregates, deterministically ordered sample rows, and the business's own checks. Each object gets pass, fail or review, with the evidence exportable for audit. Testing a sample of reports is not proof.
There is a moment in most warehouse migrations when the status turns green and the project is nowhere near finished.
The code has been converted. It compiles. It runs on the new platform without errors. The dashboard reads 100% migrated. Then the finance team runs month end on both platforms side by side, and revenue for one region is off by 0.3%. Nobody can say which of four thousand objects caused it, or whether it is a bug, a rounding change or an error in the old system that everyone had stopped noticing.
The project is now in testing. It will stay there for months.
Why does validation take longer than conversion?
Without agreed answers before testing starts, testing does not converge. It just continues.
- Is a difference in the fourth decimal place a bug or a change in floating-point behavior?
- Did the old procedure ever return the right number, or only the familiar one?
- A query with no ORDER BY now returns rows in a different order. Is that a failure?
- The old platform trimmed trailing spaces in comparisons and the new one does not. Whose behavior is correct?
- Who decides, and what evidence will they accept?
What is data reconciliation in a migration?
Data reconciliation is the process of proving that converted logic produces the same outputs as the original logic when both run on the same inputs. It is different from data validation (checking the data landed) and from functional testing (checking the system does what users expect). Reconciliation answers one narrow, auditable question: does the new version match the old one?
Done properly, it is done per object, not per report. A report can match by coincidence while three procedures underneath it disagree and cancel out.
What should a reconciliation report check, and in what order?
Column aggregates means more than totals: sum, minimum, maximum, distinct count and null count for every column. Sampled rows must be ordered deterministically on both sides, or the comparison measures sort order rather than logic. Business checks are supplied as queries by the people who know what the number should be, typically finance or risk, and run on both platforms.
| Order | Check | What it catches | Example failure |
|---|---|---|---|
| 0 | Identical inputs | Differences caused by different data, not different logic | Source snapshot taken Monday, target Tuesday |
| 1 | Schema | Missing columns, wrong types, nullability, keys | DECIMAL(18,4) became DOUBLE |
| 2 | Row counts per output | Dropped or duplicated rows | A join that now matches twice |
| 3 | Column aggregates | Value drift hidden inside correct row counts | Sum of premium differs by 0.3% |
| 4 | Sampled rows, deterministically ordered | Row-level differences aggregates can mask | Two rows swapped values, totals unchanged |
| 5 | Business-logic checks | Rules only the business knows | Gross minus net does not equal fees |
What results should a reconciliation give?
The review category is what stops testing from running forever. It turns "is this close enough?" from an argument into a decision with an owner and a date.
Here is what one object's entry might look like (illustrative):
- Pass: every check matches.
- Fail: at least one check does not.
- Review: every check passes with a documented, explainable difference inside an agreed tolerance, such as floating-point precision or time zone handling. A named person accepts or rejects it in writing.
| Object | Type | Inputs | Schema | Rows | Aggregates | Sample | Business checks | Result |
|---|---|---|---|---|---|---|---|---|
| usp_calc_monthly_premium | Stored procedure | Snapshot 2026-09-30, hash 7c1e… | Pass | Pass (1,204,331) | Review: premium_amt sum differs at 1e-10 relative | Pass (10,000 rows) | Pass (4 of 4) | Review, accepted by finance lead 2026-10-14 |
What can reconciliation not prove?
Three things, and a good report says so on its face.
It proves the new logic matches the old logic on the tested inputs. It does not prove the old logic was correct. If the original miscalculated, the migration faithfully reproduces the miscalculation, which is usually what you want during a migration and something to fix afterward.
It proves equivalence on the inputs used. Choose them to exercise the branches that matter: month ends, year ends, nulls, edge cases.
It cannot make a non-deterministic source deterministic. A query whose output order was never defined will be flagged by design, not passed.
How does proof change the commercial conversation?
When every object produces a reconciliation result as a by-product of the method, two things become possible that a services engagement struggles to offer.
A threshold in the contract. You can agree up front what share of in-scope objects must pass, by default 95%, which inputs will be used, and what happens if the threshold is missed. At Novuz the First Domain carries exactly that: miss the agreed threshold after a short cure period and the fee is waived or credited.
A shorter parallel run. If objects are reconciled individually and grouped by domain, a domain whose objects all pass can cut over on its own. The parallel run lasts as long as the flagged objects take, not as long as the estate.
The question to ask your auditor this week
Ask your audit or risk lead what evidence they will need to accept that the new platform produces the same numbers as the old one. Then ask your delivery team what evidence they plan to produce.
If the two answers do not match, the project is not late yet. It will be.
Key terms
- Data reconciliation
- proving that converted logic produces the same outputs as the original on identical inputs, object by object.
- Reconciliation threshold
- the agreed share of in-scope objects that must pass for a domain or estate to be accepted.
- Review result
- a reconciliation outcome where all checks pass with a documented difference inside an agreed tolerance, accepted or rejected by a named person.
- Parallel run
- the period when old and new platforms run side by side until the new one is accepted.
FAQ
Frequently asked questions
How do you validate a data warehouse migration?
Reconcile every converted object against the original on an identical input snapshot: schema, row counts, column aggregates, ordered sample rows, then business checks. Record pass, fail or review per object with exportable evidence.
Is testing a sample of reports enough?
No. A report can match while the objects underneath disagree and offset each other. Per-object reconciliation finds the differences that report-level testing hides.
What tolerance should we use for floating-point differences?
Agree it in writing before testing, per data type, typically as a relative tolerance, and treat differences within it as review results that a named person accepts.
Can reconciliation prove our old system was correct?
No. It proves the new system matches the old one on the tested inputs. Errors in the original are reproduced, and can be fixed deliberately after cutover.
How long should a parallel run last?
As long as the flagged objects take to resolve, if reconciliation is per object and cutover is per domain. Estate-wide parallel runs are a symptom of report-level testing.
Sources
- Novuz Methodology: reconciliation checks, result definitions and a sample redacted report (novuz.com/methodology)
About the author
Ted Ghose is co-founder and CTO of Novuz Inc., the deterministic data warehouse modernization company based in Danville, California. He built the Novuz conversion engine and was previously a Cisco principal engineer.
Next step
The Novuz estate assessment includes the coverage statement and the reconciliation approach for your estate, in three business days, for $3,000, credited against what follows. A sample redacted reconciliation report is on the Methodology page.
Last reviewed: