Introduction: Picking Up the Quantum Thread
In Part 1 of this two-part series, I confessed that this whole journey was sparked by a strange mix of life experiences: day 271 of learning Chinese (thanks to a challenge from my son) and binge-watching MIT quantum mechanics lectures. Somewhere between struggling with tones in Mandarin and watching qubits collapse under measurement, I wondered: What if enterprise data could behave like quantum information?
In Part 1, I explored quantum principles as metaphors: superposition for metadata ambiguity, entanglement for lineage, and the no-cloning theorem for golden records. These analogies gave us a new vocabulary for data management challenges, but they left an open question:
Could quantum computing itself solve our most stubborn problem — creating one physical instance of data that supports many valid representations?
This idea of stopping the insanity of data duplication has been the holy grail of data management for decades. And many of us are aware that there have been several notable attempts, including SAP HANA and Hewlett-Packard’s Neo database, more than a decade ago.
Spoiler: No. But the path forward doesn’t require exotic physics. It’s right here in the math we already use, matrices, and semantic models.
Why Qubits Aren’t Enough
Quantum mechanics gives us tantalizing imagery. A qubit can be in a superposition of 0 and 1, and multiple qubits can explore many possible states in parallel. It’s tempting to think: “Aha! One qubit = many enterprise views. Maybe we can extend this idea and apply it to data management and end up with no more ETL!”
But then, reality intervenes:
- Measurement collapse means you only ever get one outcome from a qubit.
- The no-cloning theorem prevents copying the state to peek at it from multiple perspectives.
- Qubits are phenomenal for processing complexity (search, optimization, simulation), but terrible for persistently representing many states at once.
So, if we’re searching for “one data instance, many representations,” the direct translation of a quantum won’t save us.
Matrices: The Real Superpower
Fortunately, we don’t need qubits. Matrices already give us what we want.
- Enterprise data is matrix-shaped: Rows of records, columns of attributes.
- Transformations are matrix operations: Joins, splits, and aggregations are all linear algebra under the hood.
- Different views are projections: One core matrix multiplied by different transformation matrices yields many valid perspectives.
Example:
Dataview = Datacore × TransformationMatrixData_{view} = Data_{core} \times TransformationMatrix
- Finance applies a summation matrix for quarterly rollups.
- Marketing uses a segmentation matrix to identify customer clusters.
- Operations applies a hierarchy matrix for product categories.
The core data doesn’t change. Infinite valid views can be generated without endless duplication.
The astute reader will already know that applying the calculation is often the easy part. What is more difficult is arranging messy data into a matrix in the first place.
To a great extent, this is why database views exist. They provide a means to shape the data so that it can be processed temporarily. The problem is that calculations and rules for measuring the business can easily become buried and intertwined in their data manipulation logic.
Semantic Models: Business-Friendly Matrices
Stretching the quantum mechanics analogy to its limits, we might think of the business metric calculations as the equivalent of eigenvectors. They are the key to understanding, but they can’t be trusted if they don’t adhere to standards and are not well understood.
That’s where the semantic model comes in; it’s the enterprise-friendly wrapper around the matrix algebra. In other words, semantic models can serve as the standard for metric calculations and be consumed by technologies such as database views to ensure that results conform to that standard and usage.
A semantic model:
- Harmonizes entities (Customer, Product, Location).
- Aligns attributes (Territory = East/West vs Region).
- Unifies transactions (Invoice, Order, Payment).
- Defines metrics once and exposes them everywhere.
Below are the transformation rules, effectively matrices cataloged and governed. That is a somewhat abstract way to think about it, but for the business user, they’re simply clean, reusable definitions.
Are We Just Shifting the Pain?
I hear some of you thinking this sounds like traditional ETL. We used to concentrate on resolving complexity up front by baking rules into warehouse and data mart schemas. The semantic approach defers complexity, applying rules at runtime.
And yes, complexity never disappears. But the trade-off is flexibility:
- ETL = Rules embedded in the star schema design and ETL, resolved early, one version of the truth, rigid but fast.
- Semantic = Rules abstracted, resolved late, flexible, but computationally heavier.
Yes, we are shifting the pain and workload, but for very valid reasons. We need to manage the semantics and metric calculations the business uses as a core asset and abstract them from the mechanics of specific technical implementations.
The State of Semantic Models Today
So, how mature is this vision?
- Yesterday: OLAP cubes provided governed semantics, but were rigidly tied to ETL. We all remember the days when we thought cubes were getting us out of the long data warehouse update cycles, only to end up in long cube update cycles and a confusing proliferation of cubes.
- Today: Modern semantic layers (dbt Semantic Layer, AtScale, Fabric Metrics, LookML) decouple semantics from BI tools, integrate with catalogs, and expose metrics as APIs. We also see all the major database vendors aggressively adding semantic-driven processing on top of existing view capabilities and extending toward a handshake with LLMs.
- Tomorrow: Semantic models will be the front door to enterprise data, powering not just BI, but also AI copilots, applications, and governance platforms. And I don’t mean just from a business consumption perspective. AI agents will use policies and autonomous processes to maintain the definition and use of semantic models, as well as to automate their use.
Still, maturity is uneven, and there will be fits and starts as enterprises climb the ladder from tool-specific definitions to centralized semantic layers to enterprise semantic governance.
The Payoff for Data Leaders
The real win for data leaders isn’t physics envy or matrix worship; it’s the mindset shift:
- Stop proliferating physical copies for every new reporting need.
- Maintain a single authoritative data core.
- Encode transformations as logical, reusable rules in a semantic model.
- Deliver multiple valid representations without breaking consistency.
Conclusion: Quantum Lessons, Practical Tools
Occasionally, I find it helpful to understand what is happening in other domains and apply these insights to our world of data management. Quantum mechanisms and information theory have been my proxy for that this time. It’s been a fun thought experiment, but I think I’ve reached the logic limits of its usefulness.
Here’s where our two-part journey leaves us:
- Part 1 taught us humility from quantum information theory — data is ambiguous, entangled, imperfect, and always contextual.
- Part 2 showed that while qubits won’t solve enterprise data representation problems, matrices and semantic models offer a path forward.
I hope you enjoyed the diversion. Next time, I will return to a less abstract topic.
