The Context Gap: Why Data Products Anchor AI Success
Urgency of bounded context and the architectural quantum of data that enables it.
About Our Contributing Expert
Srinivasa Mathkur | Tech Leader, Data Engineering & Solutions
Srinivasa Mathkur is a technology leader specialising in enterprise data engineering, data platforms, and AI-ready architectures. As a Principal Data Engineer at The Modern Data Company, he helps organisations build governed, scalable data foundations that accelerate AI adoption and business outcomes.
With over two decades of experience spanning financial services, retail, healthcare, and enterprise technology, Srinivasa has led large-scale initiatives in cloud modernisation, data governance, streaming, data products, and platform engineering.
A passionate advocate for automation-first engineering and modern data practices, he frequently writes about data products, enterprise AI, and the architectural foundations required to make AI trustworthy, efficient, and production-ready. We’re thrilled to feature his insights on Modern Data 101.
We actively collaborate with data experts to bring the best resources to a 20,000+ strong community of data leaders and practitioners. If you have something to share, reach out!
🫴🏻 Share your ideas and work: community@moderndata101.com
*Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality & relevance. We keep it information-first and do not support any promotions, paid or otherwise!
Let’s Dive In
The moment when a model answers you, a model has a set of weights trained on a static snapshot of text, plus whatever you put in the prompt. Nothing else. No access to your systems or any sense of what changed yesterday, and no idea which of your fields are trustworthy and which are stale.
That fact explains most enterprise AI failure. The model isn’t weak. “Knowing a lot in general” and “knowing enough about this specific situation, right now” are different properties, and only the second one is what a real business decision needs.
Differentiating between Data, Information, and Context
Data is Not Information
A number in a column (net_revenue = 48213.55) is data and cannot be used as information without added context. Information requires a frame: what period, under what accounting rule, before or after returns, in which currency, from which system.
Two analysts can look at the same number and reach opposite conclusions if only one knows the frame. This is why data dictionaries and metadata catalogs have existed for decades.
Interestingly, an analyst can go ask someone while a model can’t, but it supplies a frame regardless, possibly the wrong one, with total fluency.
Information Is Not Context
Say the model does know the frame. Is that enough? Not necessarily. Context sits above information: what’s normal right now, what’s changed recently, what this specific data point is unreliable about.
Two ER physicians can read the same borderline lab result with the same training. One has noticed a local cluster of atypical presentations the standard risk score misses, and knows the hospital’s assay is less precise near this threshold. She admits the patient. The other follows protocol and discharges him. Both used correct information. Only one used context.
This is the gap driving AI hallucination in production. Estimates put the direct cost to businesses in the tens of billions annually.
Industry estimates put the global cost of AI hallucinations at $67.4 billion in 2024. A Stanford HAI and RegLab study, peer-reviewed in the Journal of Empirical Legal Studies, found that legal AI tools built for a single domain still hallucinate between 17% and 34% of the time on challenging queries. Documented court cases involving AI-generated errors grew from around 10 in 2023 to 37 in 2024, and 73 from the first five months of 2025.
~ Source: Why AI gets it wrong and how data products fix that
Purpose-built legal AI tools still produce unsupported answers on a meaningful share of hard queries, even against peer-reviewed benchmarks. Medical AI models, even after mitigation prompting, still get a real share of answers wrong. None of that is a capability gap, but rather a context gap.
More Information Makes It Worse
The instinctive fix of giving the model more information doesn’t scale, for two reasons.
1. Cost
Without a pre-built semantic layer, an agent reconstructs understanding at query time: exploratory queries, schema guessing, relationship inference, on every call. Every hop adds tokens, latency, and a new failure point.
Organisations that pre-define semantic context instead of discovering it live have cut input token consumption by up to 90%.
At production scale, that’s the line between an economically viable AI program and one that isn’t.
2. Accuracy
Researchers found accuracy can drop significantly simply because the relevant fact sat in the middle of a long context, buried in irrelevant material. Flooding a model with unfiltered data doesn’t average out to better-informed. Instead, it drowns the signal in noise the model can’t tell from signal. It still answers fluently, just incorrectly.
“Add more data” is not a bigger version of “add the right context.” It’s a different, worse strategy.
Bounded Context: A Solution to Information Overwhelm
The resolving concept comes from domain-driven software design: bounded context. It’s a scope within which a term has one agreed meaning.
“Revenue” in billing and “revenue” in sales aren’t the same construct; treating them as interchangeable is where errors show up. Inside its boundary, though, the term is unambiguous.
Applied to AI: the fix for hallucination is a curated, governed, semantically consistent slice of data, scoped tightly to the problem at hand and no wider.
The ER physician isn’t more informed than a colleague who’s read every textbook. She’s differently informed, narrower but precisely calibrated to this hospital, this population, and this month. That precision is the entire source of her value. The target is to achieve bounded information instead of exhaustive information.
Bounded Context Boundary: What Is It Made Of?
A bounded slice needs structure to hold up under use.
Semantics: Not just that a column is called
net_revenue, but the rule behind it: which adjustments apply, how returns are handled, what calendar it runs on. Operates at the field and metric level.Governance: Who’s accountable, who can access it, what regulatory constraints apply. In a regulated industry, this is the line between a defensible decision and a liability.
Data quality: A field that’s 40% null or drifts in freshness can’t carry the same weight as one that passes rigorous checks. Quality metadata tells an agent how much to lean on a given field.
Provenance and lineage: Provenance is origin. Where data came from, who collected it, under what conditions. While lineage is transformation: which jobs touched it, through which systems, when. Together they make an answer auditable.
How provenance and lineage from origin to agentic actions are necessary for AI Explainability | Adapted from concept shared by the Author, curated by Modern Data 101 Domain taxonomy: Where semantics defines terms, taxonomy defines relationships. How a SKU rolls into a product line, a product line into a category. Without it, a model pattern-matches within one entity instead of generalising across them.
Operational metadata: Descriptions, tags, usage signals, certification flags, SLA indicators. What makes the rest findable and consumable.
None of these six pillars is new. Metadata catalogs and MDM platforms have chased pieces of this for decades.
What’s different is packaging all six into a single, versioned unit with a defined interface, built to be consumed directly by an agent as well as browsed by an analyst.
That packaged unit is a data product: the buildable answer to the abstract idea of bounded context.
The Impact of a Weak Semantic Infrastructure
Retail
A demand-forecasting agent in retail has to distinguish shipment volume from sellout volume. These are constructs that look interchangeable but aren’t. Miss it, and the forecast is numerically clean and directionally wrong.
Healthcare
A clinical agent needs to know which fields carry consent restrictions, which codes have institution-specific overlays, which values were manually entered versus system-generated. Skip that, and the output looks certified but isn’t.
Finance
A risk agent in financial services needs verified provenance on counterparty data and a clear mapping between internal and external classification schemes. Without it, the output is fluent and wrong.
Manufacturing
A maintenance agent watching equipment health needs calibration history and a sense of how shift patterns move the anomaly baseline. A raw sensor feed answers none of that.
In every case, the model isn’t underpowered but ungrounded.
The Economics
Once the slice is built properly, the expensive work (freshness, null rates, schema drift, meaning) happens once, at build time, not per query, per agent, forever. Every agent drawing on a shared data product skips re-deriving what a prior agent already worked out.
A well-scoped data product is reusable, too: the same governed slice can feed a forecasting agent, a BI dashboard, and a compliance check at once.
Build cost amortises across every consumer, and every refinement made for one use case sharpens the context available to the rest: a fundamentally different cost curve than each agent rebuilding its own understanding from scratch.
The Boundary Has to Be Validated by Business
Having a boundary isn’t the same as having the correct one. A data product encoding the wrong semantics doesn’t reduce hallucination, but industrialises it, wrapped in the false confidence of certified metadata.
Data products have to be built right-to-left: start from the business problem, work backwards through the data’s reality, and validate the semantic model before it ships to an agent, not after.
The boundary isn’t static, either. What made the ER physician valuable wasn’t generic experience, but the current experience, tied to the hospital’s specific quirks. A data product accurate six months ago, built on a source system that’s since changed schema underneath it, isn’t just stale, but dangerous, because it still answers with the same confidence it had when it was right.
One place the analogy breaks in the data product’s favour:
When the physician retires, her contextual understanding leaves with her. Institutional knowledge has always been fragile: carried in people, lost in turnover.
A data product doesn’t retire. Each governance update and quality certification layers into the product itself, not into someone’s memory.
Done well, the context compounds instead of resetting every time someone leaves.
Back to the Original Question
What does a model have when it answers you? A static training snapshot and whatever’s in the prompt. This is the case for changing what’s in that second part deliberately, rather than just making it bigger.
Before an organisation argues over which model to deploy or which orchestration layer to buy, there are prior questions:
Do we have data products that make those downstream choices work at all?
Have we built the bounded contexts our agents need to be competent, not merely confident?
Models will keep getting better at reasoning over whatever they’re given. Whether what they’re given is trustworthy, scoped, and current, that’s still a data architecture problem.
MD101 Support ☎️
If you have any queries about the piece, feel free to connect with the author(s). Or connect with the MD101 team directly at community@moderndata101.com 🧡
Author Connect 💬
Got questions? Find Srinivasa on LinkedIn or drop a comment below. 💬
From the Modern Data 101 Team ❤
Just Launched: State of Data Products 2026 Q2
State of Data Products is a quarterly read for senior data and AI decision-makers that reports the industry reality of enterprise data and AI. This edition covers ontology as AI infrastructure, the agentic AI deployment gap, observability blind spots, and why lean AI is the economic reality check the industry needed.
 | A guest post by
|
Great piece 🙌
This piece answers the how at mechanism level: what context is materially made of, and why the economics favour resolving meaning once at build time instead of on every call.
The why sits one layer up. Data products are essential because AI operates at operational speed, and infrastructure built for retrospective analysis cannot carry real-time decisions.
🤔 And the next question starts exactly where the piece ends. Bounded context works for one agent consuming one product. Operations run on networks of specialised agents collaborating across boundaries. Coherence and accountability at those boundaries is the genuinely hard part.
Two essays from January 2025 arguing the same thesis from the operational side 👇
Entering AI Era: Are Data Products Essential?
https://bbroum.substack.com/p/entering-ai-era-are-data-products
Entering AI Era: From Data Products to Networked AI Agents
https://bbroum.substack.com/p/entering-ai-era-from-data-products