Blog

Data migration is solved. Logic migration is not.

By Debjani Dhar, Co-founder and CEO · Published · 9 minutes read

The short answer

Copying data between warehouse platforms is a solved problem: storage is cheap and the tools are mature. What stalls migrations is the logic, the thousands of stored procedures, ETL packages, scripts and jobs that encode the business rules. We call it logic debt. It sets the cost of a migration, and it can be counted, converted by rule and proven object by object.

Ask a room of data leaders what the hard part of a warehouse migration is, and someone will say the data. Ten years ago they were right.

In 2014, moving fifty terabytes meant shipping disks, negotiating bandwidth and running copy jobs for weeks. Today every cloud platform ships tools that land a petabyte, most of them free, and the bill for storing it is a rounding error next to the engineering. If your migration plan is organized around terabytes, it is organized around the part that stopped being expensive.

The part that is still expensive is the logic. And most migration plans do not count it until it is too late.

What is a data warehouse, really?

A data warehouse that has run for fifteen years is a body of code with some tables attached. It is thousands of stored procedures, views, functions, macros, ETL packages, scripts and scheduled jobs. Together they encode how the business calculates revenue, recognizes a claim, books a trade, accrues a liability or bills a customer.

Most of that code was written by people who have since left. Much of it is undocumented. Two people in the building understand the billing domain, and one of them is retiring in March. The code is trusted, not because anyone has proven it right, but because the numbers it produces are the numbers the business has reported for a decade.

That is the asset a migration has to move. Not the data. The meaning of the data.

What is logic debt?

Logic debt is the share of technical debt that lives in data logic: the accumulated stored procedures, transformations and jobs whose behavior the business depends on but nobody can fully explain. It is the reason a migration that is on schedule for data is behind schedule for everything else.

It does not appear on a storage invoice or a license renewal. It appears when someone tries to move it. You can see the shape of it in almost every stalled program:

WeekWhat the status report saysWhat is actually happening
3Data landed on the new platformThe easy part is done
860% of objects convertedThe 60% that was straightforward
1695% converted, testing under wayMonth-end numbers differ; nobody knows which object
30Parallel run extendedThe two people who understand billing are in every meeting
40Revised plan approvedThe original estimate was for the data

Why is logic so much harder to move than data?

Logic has three properties that data does not, and each one defeats a different shortcut.

It is specific to the platform. A cursor in T-SQL, a QUALIFY clause or a volatile table in Teradata, a C# Script Task inside an SSIS package, a HiveQL query that relies on implicit casts: none of these has a clean twin on Spark, Fabric or Snowflake. Each one has to be rewritten, and the rewrite has to preserve behavior that was never written down, such as how NULLs sort, how a date rounds, or what happens when a join finds two matches.

It is trusted because it has always run. Nobody can prove the old procedure is correct in the abstract. So the new version does not have to be correct. It has to match. That is a much more specific test, and it needs evidence, not opinion.

It is invisible until it breaks. A table either landed or it did not. A converted procedure can run without a single error and still return a different number at month end. The failure surfaces weeks later, in a meeting, as a variance nobody can trace.

Put those three together and you get the pattern every data leader recognizes: data done early, logic done late, proof never quite done at all.

What are the options for converting warehouse logic?

Generative AI made conversion fast and made the proof problem worse. When code can be produced in seconds, the bottleneck moves to the question nobody can answer in seconds: does it do exactly what the old code did?

MethodWhere it is strongWhere it stops
Platform-native convertersFree; cover common SQL patterns wellStop at complex procedures, ETL packages and orchestration; produce no per-object proof
LLM-based convertersFast on greenfield rewrites; broad language coverageOutput can change between runs; cost grows with every rerun; nobody can prove the result matches
Systems integrator hand rewriteComplete; deep domain knowledgeSlow, priced by the hour, inconsistent across engineers; the estimate moves because the estimate is the work
Deterministic conversion with per-object proofRepeatable, auditable, fixed priceSupports a defined set of sources; very small estates may be cheaper to finish by hand

What does solving logic migration look like?

Data migration became a solved problem when it gained three properties: it became repeatable, it became verifiable and it became predictable in cost. Logic migration needs the same three.

Repeatable means deterministic. Fixed, versioned rules turn the same source into the same target every time. At Novuz, about 95% of objects convert that way, with no model calls at all. Because the output is the same on every run, you can diff it, review it and trace any change to the rule that made it.

Verifiable means a reconciliation report for every object. Schema, row counts, column-level values and the business's own checks, compared against the original on identical inputs. Pass, fail or review, per object, exportable as audit evidence.

Predictable means pricing the thing that sets the cost. A migration's cost is set by how many objects an estate has and how complex they are. Count them, band each one by its constructs, remove what nothing calls, and a fixed price stops being a gamble and becomes arithmetic.

The small share that rules cannot settle, typically around 5%, goes to a resolver that proposes a fix on a model you choose, for one of your engineers to approve. That is where a model belongs in a migration: narrow, supervised and still reconciled afterward.

Why does this matter more now than five years ago?

Two reasons, both specific to the next two years.

The first is the AI program. Almost every enterprise is putting assistants and agents on top of its data. Those systems do not calculate revenue. They read the number the warehouse produced. If that warehouse is being rewritten and nobody can prove the rewrite matches, the AI is confidently reporting numbers nobody has verified.

The second is the calendar. Legacy platforms have dates on them: support ends, renewals arrive, hardware ages out. The migration that has been deferred three times is running out of room to be deferred a fourth, and the estimate is still the thing holding it up.

Three questions to put to any migration plan

If those are hard to answer today, that is the logic debt talking.

  1. How many objects does the estate have? Not terabytes. Procedures, views, functions, packages and jobs, and how many of them anything still calls.
  2. How will each object be proven to match? Not "we will test". Which report, on which inputs, signed by whom.
  3. What is the price, fixed, before work starts? If the answer is a range that depends on hours, the risk is still yours.

Key terms

Logic debt
the share of technical debt carried in data logic, the procedures, packages, scripts and jobs a business depends on but cannot fully explain.
Object (component)
a unit of convertible logic, such as a stored procedure, view, function, ETL package, script or job definition, counted once.
Deterministic conversion
conversion by fixed, versioned rules, so the same input always produces the same output.
Reconciliation report
per-object evidence that converted logic produces the same results as the original on identical inputs.

FAQ

Frequently asked questions

Moving the data itself, the tables and files, is largely solved: the tooling is mature, often free, and storage is inexpensive. Moving the logic that transforms that data is not, which is why migrations that land the data on time still overrun.

Converting and proving the business logic: stored procedures, ETL packages, scripts and scheduled jobs. Conversion is typically about a third of the effort in a hand-run migration; proving the result matches is most of the rest.

Inventory every object, assign each a complexity band from the constructs it contains, map its dependencies, and flag what nothing calls. The object count and complexity mix, not the data volume, predict the cost.

A language model can draft conversions quickly, but the output can differ between runs and cannot be proven to match without independent reconciliation. Models are most useful for the small share of objects rules cannot convert, with an engineer approving every change.

It depends on the logic, not the data. On delivered estates of 1,000 to 1,600 objects, Novuz has typically gone from kickoff to live in four to eight weeks; an assessment of your own estate gives a fixed duration.

Sources

  • Novuz Methodology: definitions of object, automation rate and reconciliation accuracy (novuz.com/methodology)

About the author

Debjani (Deb) Dhar is co-founder and CEO of Novuz Inc., the deterministic data warehouse modernization company based in Danville, California. She previously led $10M+ modernization programs as an Accenture delivery executive.

Next step

The Novuz estate assessment answers all three questions in three business days, for $3,000, credited in full against what you do next: the inventory, a complexity band for every object, the lineage, the dead code, and one fixed price for the whole estate. Find out what your logic debt is costing you.

Last reviewed:

Find out what your logic debt is costing you.

Every object counted, lineage, dead code and a fixed price in three business days. $3,000, credited.

Take the logic debt scorecard

Not sure which door? The scorecard takes ten minutes and tells you.

Scorecard