Using Data Contracts as a Value Assessment Framework for Data or AI Initiatives
On directly measuring the success of contracts to assess operational and business impact.
Before we talk about data contracts, or value frameworks, or any of the machinery around them, we need to ask a much ancient question: what is a contract, fundamentally?
If we forget the tacticals like the YAML schemas, the tooling, the metadata registries, and the platform debates, a contract, at its irreducible core, is a promise with consequences. Nothing more. It says: I will give you this, in this form, by this time. And if I don’t, something happens.
The consequence is what makes it a contract and not merely documentation. Documentation is a description of intent, while the data contract is the enforceable commitment.
Once you hold this distinction clearly, everything that follows changes.
What Does It Mean to Measure the Success of a Data Contract
This seems obvious until you actually try to do it. Most would try to reach for data quality metrics: completeness, freshness, null rates, or schema drift. These are properties of data in isolation and NOT measurements of a contract’s success.
Measuring the success of a contract requires knowing two things that quality metrics alone cannot tell you:
who made the promise,
and to whom.
A null rate of 3% is meaningless unless you know whether the consuming system can tolerate 3% nulls, and whether the producing team ever committed to preventing them. The same metric is either a violation or an acceptable operating condition, depending entirely on what was promised.
This is the first decomposition that must happen before anything else.
We have to separate the data itself from the agreement about the data.
Several data quality frameworks collapse these together, which is why they so rarely translate into organisational action. A quality score tells you something is wrong. A contract breach tells you who is responsible and what it costs.
Decomposing the Data Contract into Independently Solvable Parts
Descartes famously argued that the path to clarity is to divide each problem into as many parts as possible, solving each one independently before reassembling. A data contract, treated as a single object, is unwieldy. Broken apart, each component becomes surprisingly tractable.
The Promise
The Promise is what the producer commits to deliver. It includes the schema, the semantic meaning of the fields, the expected ranges and distributions, the frequency of delivery, and the permitted failure rate. Good contract design forces specificity, which is the first act of genuine engineering discipline.
The Parties
The Parties are the producer and the consumer, each with independent obligations. The producer must honour the terms. The consumer must declare what they actually need, rather than accepting whatever they receive and complaining downstream.
A lot of data pipeline failures, as we all have experienced over the decades, begin not at the point of production but at the point of undeclared dependency: a consumer who assumed something about the data that was never promised. Forcing both parties to be explicit is how we prevent silent drift from compounding into catastrophes months later.
The Terms
The Terms are the observable, measurable properties that define what a “kept promise” looks like. These are the SLAs: delivery by this time, schema stable across this period, volume within this range, field X never null when field Y is populated. The terms must be independently verifiable, which means they must be defined before the data flows, not reverse-engineered from it after.
The Consequence
The Consequence is what makes the contract real.
In legal contracts, consequence is litigation.
In product contracts, it is the rejection of a feature that fails acceptance criteria.
In data contracts, consequence must be the automatic triggering of an incident, a notification to the owning team, and a measurable impact signal.
Without consequence, the contract is decorative. With a defined consequence that both parties are educated on, the contract becomes a forcing function for accountability.
Each of these four components (promise, parties, terms, consequence) can be designed and governed independently. They fail for different reasons, require different ownership, and are fixed by different interventions.
The Product Management Lens: Data Contracts as Acceptance Criteria
Product managers have long understood something that data teams are only beginning to internalise: a feature is not done until it passes acceptance criteria.
The acceptance criteria are written before development begins, agreed upon by both the team building the feature and the stakeholder receiving it, and they serve a precise function: defining the boundary between promise kept and promise broken.
Data contracts are, in this sense, the acceptance criteria for data products.
A data product is not done until the data contract(s) it ships with is/are honoured. The data contract is what the team commits to, and the measurement of contract fidelity over time is the measurement of product health. This point of view is important because it shifts the conversation from engineering metrics to product accountability.
When a data contract is treated as acceptance criteria, two things follow naturally.
First, success becomes binary and auditable: either the contract was honoured in a given period, or it was not.
Second, failure becomes attributable. Every broken contract maps back to a team, a system, and a decision. This creates the feedback loops that make improvement possible, which is the core mechanism of any product development system that works.
Product managers also understand the concept of the user story: a commitment made on behalf of a specific consumer, for a specific use case, under specific conditions. Data contracts are user stories operationalised.
They encode not just what the data is, but what the consuming system needs from it and why. A data contract without a declared consuming use case is an orphan. It is a promise made to no one, which means its success or failure is unobservable.
Data Contracts as Measurable Interfaces
When two services communicate over an API, the API contract (the shape of the request, the shape of the response, the error codes, the rate limits) is what makes each service independently deployable, independently scalable, and independently owned. The contract is what allows teams to work in parallel without colliding.
This is Postel’s Law applied to organisations. As Herb Bowie explains it, "a general social code for software: be strict in your own behavior, but tolerant of harmless quirks in others."
be precise about what you emit,
be tolerant within the bounds you have agreed to absorb.
Violate that boundary, and the downstream service fails in ways that are difficult to diagnose, because the failure is systemic rather than local.
Data contracts do the same thing for data systems that API contracts do for microservices. They create stable interfaces between independently owned data producers and consumers. This enables two things that are otherwise nearly impossible at scale.
1. Independent Deployment
A team that owns a producing data system can change its internal implementation (its storage layer, its transformation logic, its ingestion frequency) without breaking downstream systems, provided the contract is honoured.
Without a contract, any internal change is a potential breaking change for every consumer, which means teams either move slowly out of fear or move fast and break things unnoticed.
2. Blast Radius Containment
A well-designed interface limits the blast radius of impact. When a service goes down, the impact is bounded by the contract: only the systems that depended on that specific contract are affected, and the nature of the dependency is explicit and known.
The same holds for data systems. A broken data contract should trigger a known, bounded set of downstream alerts, not a diffuse wave of unexplained anomalies in dashboards, models, and reports across the organisation.
This is why data contracts are not just a data quality mechanism. They are a systems resilience mechanism. And their success is measurable in exactly the same terms as API reliability: uptime, error rate, latency against SLA, and mean time to detection and recovery.
Where Business Goals Become Data Contracts
Before the value question can be answered, there is a prior question: what is a data product, fundamentally? Not in the platform sense, but in the product sense, as the word “product” is understood in product management.
A product exists to solve a specific problem for a specific user segment. It has a defined goal, a defined consumer, and a measurable definition of success. A product is intentionally scoped.
It is designed to fulfil a specific purpose and is evaluated against that purpose.
A data product, then, is not a well-labelled dataset. It is a data asset built to serve a declared business goal for a declared consuming system or decision. The moment a data product loses its declared goal, it reverts to infrastructure: expensive to maintain, impossible to measure, and difficult to justify.
This distinction matters because the product is an aspect where the data contract becomes significantly resourceful. A contract attached to a data product that serves a specific business goal is a direct, traceable commitment between data behaviour and business outcome. That is the unit of measurement we have been looking for.
The Model Is the Contract in Data Products
In a data product built with right-to-left engineering, what we have called a model-first data product, the model is not a representation of the contract. The model is the contract. The idea is that all the entities that the data product hosts are being operationalised to meet certain goals. Reverse engineered from the business requirements.
The model of these entities, relationships, dimensions, and measures forms the perfect base for implementing and measuring alignment to business along the path of least friction.
A model doesn’t just describe a system, it defines the boundaries within which the system is permitted to operate.
When a data model is written as a SQL query with embedded tests, integrity constraints, quality thresholds, and semantic definitions, it is no longer just a technical artefact. It is a declarative, enforceable statement of what the data must be.
It says: these fields must be populated, this join must resolve, this distribution must fall within these bounds, this entity must never appear in two states simultaneously. Every clause is a term in the contract. The model defines what “kept promise” looks like in machine-executable form.
This has three consequences that are not obvious until you carry them together.
The contract is now testable at every execution
Unlike a document-based contract that must be manually audited, a model-as-contract is verified automatically every time it runs. The test suite embedded in the model is the continuous audit of whether the contract is being honoured. This transforms contract measurement from a periodic review into a real-time signal.
The contract is defensive by design
Defensive programming (the discipline of anticipating failures and building guards against them at the point of code) means the model does not trust upstream data. It validates it. A model-first data product does not hope the input conforms to expectations. It enforces those expectations as a precondition of its own execution.
The model encodes semantics
This is the most important aspect for the value realisation. What separates a schema from a contract is meaning. A schema tells you the shape of the data, while the model tells you what that shape means in the domain it represents. It defines that “customer” means an entity with at least one completed transaction. It defines that “revenue” means recognised revenue after refunds. These semantic commitments are what make a data product trustworthy to the business stakeholders who depend on it.
The Data Contract as a Dual-Axis Measurement Instrument
Data contracts are the centre of a genuine value-assessment framework if architected well: a single contract’s success generates two independent, simultaneously useful signals. One operational and one business-facing.
The Operational Axis
The operational axis reads contract success as a measure of platform and engineering health. On this axis, the questions are:
Is the contract being honoured consistently?
How quickly are breaches detected?
How much of the platform’s contract surface is instrumented versus dark?
What is the mean time to recovery when a contract fails?
These are the signals that engineering teams, data platform owners, and SREs need to allocate resources, prioritise incidents, and demonstrate that infrastructure investment is reducing systemic risk.
They are also the signals that reveal which parts of the platform are load-bearing and which are vestigial. A contract that never breaches and has no consumers is a different kind of signal than one that breaches frequently and is consumed by five downstream systems.
The Business Axis
The business axis reads the same contract success as a measure of decision reliability. On this axis, the questions are:
Which business decisions depend on this contract?
What is the lag between a contract breach and a degraded business output?
Has any business outcome produced during a contract degradation period been audited and corrected?
The business axis requires the same fidelity signal to be interpreted through the lens of declared business dependencies. A contract that fails 2% of the time in a pipeline that feeds a risk model used in real-time lending decisions has a very different business-axis reading than one that feeds a monthly executive report. Failure rates may be identical, but the weight of consequences would differ.
Most data teams report only on the operational axis. This is why they struggle to justify investment in contract infrastructure to business stakeholders. The operational story is: “our pipelines are 98% reliable.” The business story is: “two percent of the time, your model is making decisions on data that violated its own stated constraints.”
What a Set of Contracts Reveals That a Single Contract Cannot
A single contract tells you whether one promise was kept. A portfolio of contracts, measured together over time, tells you something far more valuable: the structural health of an entire data and AI programme, with enough resolution to make initiative-level decisions.
This is where cumulative contract success becomes a strategic instrument rather than an operational one. When you track fidelity across all contracts in a domain (say, the contracts that feed a customer intelligence programme), patterns emerge that are invisible at the individual contract level.
The Contract-Driven Initiative Assessment
These patterns compose into a decision framework that is genuinely usable at the level of data and AI programme management. The central question is:
“What does the contract health of this initiative’s data dependencies tell us about whether it is worth continuing to invest in?”
For any data or AI initiative, you can characterise its contract portfolio along three dimensions:
fidelity trend (improving, stable, or deteriorating over the last quarter),
consumption pattern (growing, stable, or declining),
and business-axis coverage (what percentage of the initiative’s contracts have a declared business dependency and an active outcome trace).
Reference Pattern A
An initiative with deteriorating fidelity, growing consumption, and low business-axis coverage is in a dangerous state: demand is rising, reliability is falling, and no one has formally mapped what the business consequences of failure look like. This is not an initiative to accelerate, but one to stabilise first, or the acceleration will produce the incident that triggers the post-mortem.
Reference Pattern B
An initiative with stable fidelity, declining consumption, and high business-axis coverage tells a different story: the infrastructure is sound, the business use is waning, and the business-axis coverage means the team has the evidence to explain why. This is an initiative to have a direct conversation about and formally assess for sunsetting, with the contract insights as the basis for the conversation.
Reference Pattern C
An initiative with improving fidelity, growing consumption, and expanding business-axis coverage is the signal you are looking for when allocating investment. The infrastructure is getting more reliable, more of the business is depending on it, and the team has been doing the work of declaring and tracing business value. These are the initiatives worth accelerating, and data contract insights are the evidence that justifies the investment in language that both engineering leadership and business stakeholders can read.
The power of this framework replaces the ambiguity that usually surrounds these decisions with observable, continuously updated evidence. Data contract health does not make the decision, but it definitely makes the decision legible.
Data Contracts as Precursors in AI Applications
Analytics systems, when they receive bad data, produce wrong reports. But a human analyst often catches the anomaly. Something looks off, the number doesn’t match last quarter, or the trend breaks in an implausible way.
Humans have intuition about the domain they operate in, and that intuition is a partial safeguard against non-mechanical data failures.
AI systems have no such safeguard. It is not feasible for different domains in different organisations to train AI around all the quirks and specificities. Two marketing domains in two different orgs in the same industry operating the same product can be vastly different when it comes to the quirks, which is where human intuition fills the gaps.
A model does not pause and say “this doesn’t look right.” It reasons over whatever it receives, produces a confident output, and the downstream system acts on it.
This is why data contract health is the definition of AI system reliability.
Every AI initiative that fails to instrument contract fidelity upstream of its model is accepting an unknown, unmanaged risk that grows until it manifests as an unexplained model degradation or a business incident that no one can trace.
Conversely, teams that treat data contracts as the primary instrumentation layer for their AI systems have something rare: an objective, continuously measured link between their data infrastructure and their AI business outcomes.
They know which contract, if broken, breaks which model. They know which model, if degraded, affects which business metric. They can compute, defend, and improve that chain systematically.
Final Note
The reason most data and AI investments struggle to demonstrate value is that the measurement systems used to evaluate them are disconnected from the commitments that govern them. Nothing is joined or continuously integrated in a way to provide initiative-level insights.
The contract-driven value framework joins them. It starts with the contract as the unit of measurement: what was committed to, by whom, under what terms, and whether that commitment was kept.
It reads that commitment on two axes simultaneously, giving engineering teams the operational signal they need and business stakeholders the decision signal they need, from the same source of truth.
And it aggregates those readings across a portfolio of contracts to reveal patterns that make initiative-level decisions possible without waiting for a quarterly review or a post-mortem. Data contracts are the most genuine accounting system a data or AI organisation can build.
MD101 Connect ☎️
If you have any queries about the piece, feel free to connect with the author(s). Or connect with the MD101 team directly at community@moderndata101.com 🧡
Author Connect 💬
Connect with Animesh on LinkedIn 💬
From MD101 team 🧡
The Modern Data Masterclass: Learn from 5 Masters in Data Products, Agentic Ecosystems, and Data Adoption!
With our latest 10,000 subscribers milestone, we opened up The Modern Data Masterclass for all to tune in and find countless insights from top data experts in the field. We are extremely appreciative of the time and effort they’ve dedicatedly shared with us to make this happen for the data community.
Dive in and also get a chance to nominate your favourite masterclass host!
