Lumy Labs
← All insights

Data

Why your data warehouse keeps drifting from reality

The dashboard was right when it was built. Nothing broke. It is wrong now because the business changed underneath a definition nobody owns.

Lumy Labs8 min read
A wall of clocks, each showing a different time

The failure mode is familiar. Finance and operations bring two numbers for the same thing to the same meeting. Both are produced by pipelines that ran green. Neither team is wrong, exactly.

Warehouse drift is rarely a pipeline problem. It is a definition problem that a pipeline faithfully executed for eighteen months after the definition stopped being true.

Definitions decay silently

An active customer meant something specific when the metric was written. Then a free tier launched, then a trial changed length, then a region started counting a dormant account differently because a regulator asked it to.

None of those changes touched the pipeline, so nothing failed. The number kept being produced, kept being trusted, and kept diverging from what anyone in the room actually meant by the word.

Ownership is the missing column

Most warehouses record where a field came from and when it last loaded. Very few record who is accountable for what it means. Lineage tells you the path. It does not tell you who to ask when the path is fine and the answer is still wrong.

A definition with a named owner gets revisited when the business changes, because someone is on the hook for it. A definition without one is maintained by whoever last touched the SQL, which is usually the person with the least context.

  • A named owner, not a team alias
  • The business question the metric answers, in one sentence
  • The decisions that depend on it
  • The date it was last reviewed against reality

Tests that assert freshness, not truth

Warehouse test suites are usually strong on mechanics and silent on meaning. They check row counts, nulls, referential integrity and load times. All of that can pass while the number is nonsense.

The tests that catch drift are the ones that encode a business invariant: that this total reconciles with the source system to within a tolerance, that this ratio has never historically left a range, that these two independently derived figures agree. Those fail loudly when the world changes, which is exactly what you want.

One number, one place

Drift accelerates when the same metric is computed in three places: once in the warehouse, once in a dashboard filter, once in a spreadsheet that a director maintains. Each copy is a chance to disagree, and each one is maintained on a different schedule.

The fix is unglamorous. Compute it once, expose it as the only sanctioned source, and make the spreadsheet read from it rather than recompute it. Most of the value in a semantic layer is not the technology. It is the social agreement that there is one definition and it lives somewhere specific.

Let's talk

Living with this problem?

If this one landed close to home, tell us where you are stuck and we will tell you honestly whether we can help.

Or email start@lumylabs.co

What happens next

  1. A real replyWithin 48 hours
  2. Before specificsNDA first
  3. Not a sales pitchA real proposal