Novuz

Guide

Data warehouse migration validation before cutover

Data warehouse migration validation means proving, before cutover, that the new platform gives the same results as the old one. Do it in two moves. First find the dead code, so nobody validates logic nothing uses. Then reconcile every remaining object against the source on the same inputs, and keep the evidence. This guide sets out both, the checks in order, and how Novuz produces the proof.

Definition

What is data warehouse migration validation?

It is source-to-target validation of the logic as well as the data. A table that copied correctly proves the copy. It does not prove that the converted stored procedure, package or job will write the right rows tomorrow.

So validation has to run the converted logic. Each object runs on the same inputs as the original, and the two outputs are compared. The result is recorded per object: pass, fail or review.

Dead code first

How do you detect dead code before a migration?

Dead code is logic nothing uses. Finding it early shrinks the scope, the price and the testing. Follow these steps.

  1. Build the dependency graph. Map every procedure, view, table, package and job to what calls it and what it calls.
  2. Include the callers outside the database. Schedulers, reports, applications and scripts all count.
  3. Check the run history. An object no job has run and no query has read for a full business cycle is a candidate.
  4. Record the evidence for each candidate: no callers, no reads, last run date.
  5. Ask each domain owner to confirm. Year-end and audit jobs run rarely and are easy to misjudge.
  6. Drop confirmed dead code from the migration scope. Leave it in the source until the platform is retired.

The checks

What checks prove a migrated object matches the source?

Run them in this order. Each one is cheaper than the next and catches a different kind of error.

CheckWhat it comparesWhat it catchesWhat it misses
SchemaColumns, types, nullabilityDropped or retyped columnsWrong values
Row countsRows per table and per partitionMissing or duplicated rowsWrong values in the right number of rows
Column aggregatesSums, minimums, maximums, distinct countsMost value errorsErrors that cancel each other out
Row comparisonSampled or hashed rows, in a fixed orderCompensating errorsRows outside the sample, if sampled
Business checksThe totals your finance or operations team already trustsErrors that matter to the businessAnything nobody thought to check

The method

How do you validate a migration before cutover, step by step?

  1. Agree the rule first. Write down which objects, which inputs, which checks and what share must pass before cutover.
  2. Pin both platforms to one input snapshot. Comparing against moving data proves nothing.
  3. Agree the tolerances. Decide how floating-point differences, time zones and trailing spaces are treated.
  4. Run each converted object and its original on the same inputs.
  5. Compare the outputs with the checks above, and record pass, fail or review for each object.
  6. For each failure, record the cause and the fix, then rerun until it passes or is accepted in writing.
  7. Cut over one domain at a time, when its objects meet the written threshold.

Evidence

What reconciliation evidence should you keep for audit?

  • The object, its version and the rule or person that converted it.
  • The input snapshot the comparison ran on.
  • The checks that ran, and the result of each.
  • Every miss, with its cause and its fix.
  • Who approved any change a rule did not make.

With Novuz

How does Novuz validate a migration?

Novuz lists dead code, with the evidence, in its estate assessment. It then converts by fixed rules and reconciles every converted object against the original on the same inputs: schema, row counts and column-level values. Each object gets pass, fail or review, and the reports export as audit evidence.

On delivered estates, reconciliation is typically above 95% on first pass, and the report names the rest. The First Domain charter states a reconciliation threshold, 95% of in-scope objects by default. Novuz can also reconcile code that another tool converted.

FAQ

Frequently asked questions

How do I prove a migrated data warehouse matches the original?

Run every converted object and its original on the same input snapshot, then compare schema, row counts, column aggregates and sampled or hashed rows. Record pass, fail or review per object, with the cause of each miss. Testing a sample of reports is not proof. Novuz produces this reconciliation report for every object it converts.

How do I find dead code before a data warehouse migration?

Build a dependency graph across code, schedulers and outside callers, check run and read history over a full business cycle, record the evidence for each unused object and have the domain owner confirm. Then drop it from the migration scope. Novuz lists dead code with the evidence in its estate assessment.

What is source-to-target validation?

It is comparing what the source system produces with what the target produces from the same inputs. For a warehouse migration it should cover the logic, not only copied tables: each converted procedure, package and job is run and its output compared with the original.

When should migration testing start?

Before conversion, by agreeing the pass rule: which objects, which inputs, which checks and what share must pass. Then validation runs as each object is converted, not as one phase at the end.

How long should the parallel run last?

As long as it takes to prove the domain being cut over, usually one or two period closes for finance data. Reconciling each object before cutover is what lets the parallel run end on a date.

How we measure: Methodology

Written by Ted Ghose, Co-founder and CTO, Novuz

Last reviewed:

Find out what your logic debt is costing you.

Every object counted, lineage, dead code and a fixed price in three business days. $3,000, credited.

Take the logic debt scorecard

Not sure which door? The scorecard takes ten minutes and tells you.

Scorecard