Zina Verduzco | Founder & Lead Strategist

Zina Verduzco is an AI transformation strategist and founder of Zidian Partners, where she works at the intersection of automation, data infrastructure, and operational design. As a Google AI and DeepMind Developer Partner, she brings a practical, implementation-first approach to helping organisations move beyond AI experimentation into measurable, real-world impact.

With experience spanning small businesses, nonprofits, and growth-stage companies, Zina has consistently observed that operational inefficiencies don’t disappear with scale, but compound. Her work focuses on removing these choke points through intelligent automation, trusted data systems, and scalable process architecture.

Beyond implementation, she is deeply invested in democratizing AI adoption. Through training programs, workshops, and community initiatives, she helps those traditionally underserved adopt AI in a way that is practical, secure, and aligned with business outcomes. Zina’s background also includes communications and systems design, giving her a multidisciplinary lens on how technology, people, and processes intersect. We’re thrilled to feature her insights on Modern Data 101.

We actively collaborate with data experts to bring the best resources to a 20,000+ strong community of data leaders and practitioners. If you have something to share, reach out!
🫴🏻 Share your ideas and work: community@moderndata101.com

*Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality & relevance. We keep it information-first and do not support any promotions, paid or otherwise!

Let’s Dive In

There is a board meeting happening right now in a company you’ve probably heard of where someone is presenting an AI voice agent demo and calling it a strategy.

The demo is impressive, the press release is already drafted, and nobody in that room has asked a single question about who owns the data the agent is collecting, who governs the compliance stack underneath it, or what happens when a caller in the EU invokes their right to erasure.

This is a strategic deficiency leading a double life as “innovation”, and it is becoming one of the most expensive mistakes in enterprise AI.

The Vanity Trap: When “Having AI” Becomes the Goal

According to Gartner’s February 2025 research, organisations will abandon 60% of AI projects through 2026 due to inadequate data infrastructure and unclear business value. But the stat I find more revealing is this: most enterprises deploying AI voice agents today have no defined success metrics before they flip the switch.

The pressure is real and investors do want to see AI in the roadmap. Boards are asking why the company isn’t moving faster and competitors are putting out press releases. So the enterprise deploys quickly, visibly, and without infrastructure.

What does a vanity deployment look like in practice?
It looks like an AI voice agent that answers calls and books appointments, but whose interaction logs exist in three different vendor systems with three different retention policies.

It looks like a beautifully branded “AI assistant” with no audit trail. It looks like a company that can tell you how many calls the agent handled, but cannot tell you what it said on any of them.

What Real ROI Looks Like for AI Voice Agents

Enterprise AI deployments that are built for performance instead of optics look completely different before a single line of code is written. They start with a baseline.

The metrics that matter for voice AI specifically are:

• cost per conversation (what does it cost to handle one inbound or outbound interaction?),

• containment rate (what percentage of calls are fully resolved without human escalation?),

• conversion lift (for sales or appointment-setting agents, what is the delta between AI-assisted and unassisted outcomes?), and

• time-to-resolution (how much faster does the customer reach an outcome?)

These are pre-deployment baselines that make it possible to evaluate whether the AI is actually doing what you built it to do. Without them, you are not measuring performance, but activity. And activity is not ROI.

The enterprises I work with that do this correctly treat their voice AI the same way they treat any other revenue-generating or cost-reduction infrastructure.

They define success before deployment, instrument from day one, and hold the system accountable to measurable outcomes. Skipping this step almost always ends up in rebuilding at significant cost, usually after a compliance incident forces the hand.

Who Owns the AI You Built With Someone Else’s Tools

Here is a question that almost no enterprise AI deployment answers cleanly: when something goes wrong, who is responsible?

The modern AI voice stack is a complex web of third-party dependencies. A typical deployment might include

• a voice platform (like Retell AI or Vapi),

• a large language model (from OpenAI, Anthropic, or Google),

• a telephony provider (Twilio or similar),

• a CRM for logging outcomes,

• and potentially a compliance tool layered on top of all of it.

Each of these vendors has its own terms of service, its own data retention policies, and its own interpretation of who owns the outputs.

Nobody governs the full stack. And that gap is where enterprises are most exposed.

Consider what lives inside a voice agent interaction: the caller’s phone number, their name, their stated intent, their expressed hesitations, and potentially their health, financial, or legal situation, depending on the use case.

Call recordings may contain biometric voice data. Conversation logs may contain PII that, depending on jurisdiction, requires specific handling, consent language, and the right to erasure.

Under GDPR alone, regulators can levy fines of up to 4% of global annual revenue or €20 million, whichever is higher. In 2023, Meta was fined €1.2 billion for cross-border data transfer violations. These are operational realities that no demo ROI calculation accounts for.

The AI ownership question is still largely unsettled in courts and regulatory frameworks. But the compliance question is not. You own the obligation to your callers, regardless of whose infrastructure you built on. That responsibility does not transfer to your vendor.

Compliance is Add-On Infrastructure

The reason most deployments skip the compliance layer is sequencing. Compliance must be embedded at the architectural layer instead of being added after deployment when the stack is already live, and the data is already moving.

We approach every voice agent deployment as an infrastructure build. Before we configure a single prompt or integrate a telephony provider, we ask four foundational questions:

1. What data does this agent collect, and where does it go? Every touchpoint in the stack needs a documented data flow before deployment.

2. Who has access to the interaction logs? Access control is a governance decision, not a technical default.

3. What are the client’s compliance obligations by jurisdiction? CCPA, GDPR, HIPAA, and state-level privacy laws require specific consent language and documentation.

4. How will privacy documentation stay current as the regulatory landscape changes?

A privacy policy written at deployment is already outdated. The fourth question is the one that most enterprises have no infrastructure to answer. What enterprises need is not a static privacy policy.

They need continuous compliance tooling: systems that monitor regulatory changes, update disclosures, and maintain alignment between how AI is used and how it is documented.

The Question Every Enterprise Should Be Asking

Before your next AI voice deployment, ask your team this: if there’s a complete audit of every data point this agent collects, who has access to it, how long it is retained, and what our compliance posture is across every jurisdiction they operate in?

Could we answer that question today?

If the honest answer is no, you have not deployed an AI strategy, but only a demo.

The gap between a vanity deployment and a performance deployment is not as wide as it seems.

It requires the right sequencing, the right questions asked before a single line of code is written, and a governance layer treated as infrastructure from day one.

A concrete example: In California, failing to disclose an agent’s non-human status when sincerely asked is a statutory violation under AB 302. We don’t wait to be asked. Every voice agent we deploy opens the same way: you are speaking to an AI, and this call is being recorded.

It is not the warmest greeting, but consider who is on the other end of that call.

The person deserves to know exactly what they are walking into before they say a single word. Compliance isn’t a warm handshake, but transparency is the foundation of trust.

And trust is the only thing that makes the conversation worth having at all. When encountering unfamiliar processes, one deserves to know what they are dealing with.

And enterprises that do it right will not just have AI on the slide deck and impressive interfaces, they will have AI that works, scales, and protects them.

MD101 Support ☎️

If you have any queries about the piece, feel free to connect with the author(s). O connect with the MD101 team directly at community@moderndata101.com 🧡

Author Connect 💬

Connect with Zina on LinkedIn 💬 and check out more of her work here!

From MD101 team 🧡

The Data Product Playbook

Here’s your own copy of the Actionable Data Product Playbook. With 4000+ downloads so far and quality feedback, we are thrilled with the response to this 6-week guide we’ve built with industry experts and practitioners. Stay tuned to moderndata101.com for more actionable resources from us!

DOWNLOAD!

Dia Adams | Chief Data & AI Officer, ex-White House, Keynote Speaker & Best-Selling Author

Dia Adams is a Chief Data & AI Officer, former White House Enterprise Data Strategist, and Board Chair of The AI Table, where she helps shape the future of responsible AI and enterprise transformation. With more than two decades of experience in data strategy, AI governance, and digital transformation, she has led high-impact initiatives across federal agencies and global enterprises.

An accomplished author of Winning With AI: A Blueprint for Corporate Leaders, Dia equips executives, including CDOs, CIOs, and CEOs, with practical frameworks to successfully adopt AI while aligning technology with business strategy. Her work spans AI policy, MLOps, data governance, and data literacy, with a strong emphasis on building human-centred, compliant, and scalable AI systems.

Dia’s public sector contributions include advancing data-driven decision-making at the White House and developing AI-powered applications at the U.S. Securities and Exchange Commission. As an ACT-IAC Fellow, keynote speaker, and educator, she continues to influence the next generation of data and AI leaders while helping organisations operationalise AI with clarity, control, and purpose. We’re thrilled to feature her insights on Modern Data 101.

We actively collaborate with data experts to bring the best resources to a 20,000+ strong community of data leaders and practitioners. If you have something to share, reach out!
🫴🏻 Share your ideas and work: community@moderndata101.com

*Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality & relevance. We keep it information-first and do not support any promotions, paid or otherwise!

Let’s Dive In

A revolution is underway, spurred by algorithms. These step-by-step computational processes are rewriting the rules of how companies manage, protect, and leverage their most valuable asset: data.

For decades, data governance, the policies, processes, and technologies that ensure data is accurate, secure, and compliant: has been a staid, if essential, function. Governance was once confined to brittle, rule-based workflows: hand-coded data quality checks, static metadata repositories, and siloed policy enforcement, all managed through a patchwork of legacy tools, custom scripts, and tribal knowledge.

But as artificial intelligence seeps into every corner of the enterprise, it is transforming data governance from a back-office chore into a dynamic, predictive, and sometimes even autonomous discipline.

The Old Guard: Data Governance Before AI

Traditional data governance was built on control. Companies established strict rules about who could access what data, how it should be classified, and how long it could be retained. Compliance officers and data stewards acted as gatekeepers, manually reviewing datasets and enforcing policies.

But this approach struggled to keep pace with the explosion of data. By 2025, the world was generating an estimated 463 exabytes of data each day: more than all the data created from the dawn of civilization until 2010. Manual governance couldn’t scale. Errors slipped through. Regulatory fines mounted.

AI Steps In: From Reactive to Predictive

Enter AI. Machine learning models, natural language processing, and generative AI are now being embedded into the very fabric of data governance. The results are profound. AI can now scan vast datasets, identify sensitive information (like personal identifiers or financial records), and classify it in real time, without human intervention.

Predictive Data Quality

Instead of waiting for errors to be reported, AI models analyze patterns to predict where data quality might degrade. A global retailer uses AI to flag potential inconsistencies in its supply chain data before they cause disruptions, saving millions in lost sales.

Dynamic Compliance

Regulatory requirements are constantly evolving, forcing data teams to juggle a labyrinth of regional, industry-specific, and emerging compliance mandates. AI systems can now monitor changes in laws such as GDPR, the EU AI Act, or sectoral frameworks like HIPAA and CCPA, while dynamically adjusting data handling policies, mapping new rules to existing data flows without manual intervention.

Self-Healing Data Pipelines

When data pipelines break, or more critically, when they degrade in ways that traditional monitoring fails to detect, AI does not simply flag anomalies in logs or dashboards. It traces root causes across intricate system dependencies, correlates failures with recent upstream changes, and in the most advanced deployments, autonomously remediates issues by rerouting workflows, adjusting resource allocations, or rolling back problematic data transformations.

🔖 Related read

The Rise of the Data Fabric

The most significant advancement in modern data governance is the development of AI-powered data fabric architectures. These systems integrate disparate data sources, automate data integration processes, and maintain consistency across the enterprise.

A prominent example is found in the manufacturing sector, where data fabric implementations use AI to resolve inconsistencies among factory floor sensors, ERP systems, and customer databases. Previously, these systems operated in isolation, creating inefficiencies and data silos. AI now enables seamless integration, providing a unified and reliable data foundation for decision-making.

This architecture also supports the adoption of data mesh principles. In this model, individual business domains maintain ownership of their data, while AI ensures interoperability and centralized governance. The result is a system that combines domain-specific autonomy with enterprise-wide consistency and control.

Key Challenges in AI-Driven Governance

Despite its advantages, AI-driven governance introduces several challenges that organizations must address.

Transparency and Explainability

AI models can be inherently opaque, making it difficult to explain governance decisions to regulators or stakeholders. When an algorithm flags data or enforces a policy, the lack of transparency can create accountability issues. Organizations require AI systems that are not only effective but also interpretable, ensuring that decisions can be justified and audited.

Skills and Expertise

The shift to AI-driven governance demands a new blend of technical and ethical skills. Data stewards must now possess AI literacy, while organizations need to incorporate ethicists and compliance experts who can assess algorithmic fairness and mitigate bias. The rapid evolution of these requirements has created a skills gap that many companies are still working to close.

Regulatory Dynamics

As AI assumes a greater role in governance, regulatory frameworks are evolving to keep pace. Initiatives such as the EU AI Act introduce stringent requirements for automated decision-making, adding complexity to compliance efforts. Organizations must now treat compliance as an ongoing, adaptive process rather than a static checklist.

Organizational Adoption

There is often resistance to the increased reliance on AI for governance. Concerns about the replacement of human judgment persist, even though the objective is to augment, not replace, human oversight. Successful implementation requires clear communication about the role of AI and the continued importance of human expertise in governance processes.

🔖 Related read

Balancing AI and Human Oversight

It would be most effective to achieve a balance between AI automation and human oversight. AI excels at handling repetitive, data-intensive tasks, while human professionals contribute critical context, ethical judgment, and creative problem-solving.

Looking ahead, several trends are expected to shape the future of AI-driven governance:

• Autonomous Governance: AI systems will not only enforce existing policies but also identify emerging risks and propose adaptive governance measures.

• Ethical AI Frameworks: Organizations will embed mechanisms to ensure that AI-driven governance remains fair, transparent, and accountable.

• Industry Standards: As AI governance matures, standardized frameworks (similar to ISO quality management standards) will likely emerge, providing benchmarks and best practices for implementation.

The New Era of Data Governance

The integration of AI into data governance represents a significant change in how organizations manage and use their data. In this new environment, governance is centered upon intelligence and adaptability, requiring organizations to continuously modernize their strategies, tools, and skill sets to keep pace with technological and regulatory advancements.

MD101 Support ☎️

If you have any queries about the piece, feel free to connect with the author(s). O connect with the MD101 team directly at community@moderndata101.com 🧡

Author Connect 💬

Connect with Dia on LinkedIn 💬

More from Dia Adams

Winning with AI

Lean more

The AI Table

Learn more

From MD101 team 🧡

🌎 Global Modern Data Report 2026

The Modern Data Report 2026 is a first-principles examination of why AI adoptions are getting blocked inside otherwise data-rich enterprises. Grounded in direct signals from practitioners and leaders, it exposes the structural gaps between data availability and decision activation.

With hundreds of datapoints from 540+ data leaders and experts from across 64 countries, this report reframes AI readiness away from models and tooling, and toward the conditions required and/or desired for reliable action. Already downloaded 1000+ times, this report is enabling key insights for several industry practitioners this year. Join the discussions!

Get your copy

Alejandro Aboy | Senior Data & AI Engineer, Writer

Alejandro is a Senior Data & AI Engineer and the voice behind The Pipe & The Line, a technical newsletter where pipelines meet product thinking. With a career that began in marketing and web analytics, he transitioned into modern data engineering, bringing with him a sharp instinct for business context, experimentation, and stakeholder alignment.

At Workpath, Alejandro serves as the main owner of the Data Platform & AI stack. He builds AI agents with RAG architectures (Pgvector on Aurora), designs MCP-based tool ecosystems, and implements production-grade AI observability. His work spans dbt modelling, orchestration with Airflow, Slack-to-Tableau automation, and secure backend design for agentic systems.

Alejandro is deeply engaged in the open data community. He has collaborated with OpenMetadata on agentic data modelling and metadata-first architectures, co-authoring end-to-end implementations that connect lineage, governance, and AI tooling through MCP. His writing is frequently featured by Modern Data 101 and other leading data publications. We’re thrilled to feature his unique insights on Modern Data 101!

We actively collaborate with data experts to bring the best resources to a 20,000+ strong community of data leaders and practitioners. If you have something to share, reach out!
🫴🏻 Share your ideas and work: community@moderndata101.com

*Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality & relevance. We keep it information-first and do not support any promotions, paid or otherwise.

Let’s Dive In

In July 2025, I wrote a guest post framing data professionals as orchestrators of the agentic data stack, with all the new possibilities that implied.

It’s been a while in terms of AI technology progress, but we still hear that AI will replace data engineers, automate modeling, and democratize analytics. But I see we ended up in a different place: a lot of new shiny productivity tools that create a silent workload around them.

6 months ago, my stack was dbt with some Cursor rules. Now I have 7 skills & 5 hooks with Claude Code, Postgres, Miro, and Atlassian MCPs, and I still put the same level of attention.

And that scenario exposed how messed up we are on so many levels, but since AI can’t assume accurately, now the symptoms look worse than before.

Before AI readiness, human readiness. That’s it. That’s the whole thing.

TL;DR

• AI didn’t automate work. It redistributed it. You’re not coding less. You’re maintaining AI toolchains and validating output.

• The five false promises that led us here: context windows replace schemas, denormalization fixes everything, agents infer structure at runtime, MCP learns your business, and skipping modeling fundamentals.

• Your data stack now serves two customers with incompatible needs. Humans muddle through ambiguity. AI hallucinates through it. That gap is the tax.

• The silver lining: Making data “AI-ready” accidentally made it better for humans, too. The BIG question is how far companies are from understanding their own context, regardless of AI.

• Tools will catch up. Human readiness won’t. The teams that win aren’t the ones with the best tools — they’re the ones that were ready before the tools arrived.

The Five False Promises

1. “Context Windows Are the New Schema.”

“200K tokens means AI can just read everything!”

Your 47-table schema without table/column descriptions will just make non-determinism burn all your tokens or give awful answers with 5 JOINs in the best-case scenario.

This one is still a problem and I like to keep it that way since the solutions around it make more sense.

2. “One Big Table for AI.”

“Denormalize everything! AI loves simple!”

I tried that, it’s better. But denormalization without semantic clarity is the same mess, just centralized.

The same way a human will find this overwhelming, there won’t be any difference with AI.

3. “Just-In-Time Modeling.”

“AI agents will infer structure at runtime!”

You can’t “vibe” your way through data modeling without a discovery workflow or useful query patterns that could serve as a few-shot examples for business use cases.

Tools like cube.dev MCP or dbt semantic layer skills make runtime schema discovery more reasonable, but only when the foundations are documented first.

4. “AI Will Learn Your Business.”

“MCP + metadata catalogs = autonomous design!”

This one is a false promise, but not because of AI, but because of how companies operate.

I connected Claude to OpenMetadata via MCP. It was really good. But data lifecycles and business logic are evolving rapidly without robust workflows, which will make this approach stale since it’s not maintained.

Someone has to put metadata in place, define domains, connect assets through lineage and make sure their owners are well labeled. Data governance is useless for AI if you rely on AI figuring out the blanks.

5. “No Need to Learn Data Modeling.”

“Just describe what you want in English!”

Text2SQL has improved. With MCPs, semantic layers, and metadata catalogs on top, you get decent results.

But not being able to spot shuffling queries, unnecessary partitions, or table JOINs that are already covered in other models will just make modeling with AI the greatest data garbage-in-garbage-out ever.

Get familiar with schemas, don’t rush to physical implementation if conceptual doesn’t feel right. Use AI to think through it, you can just talk to it because now it can draw stuff on boards!

The AI Data Modeling Tax

Your data stack now serves two customers with incompatible needs.

• Humans tolerate ambiguous naming, navigate tribal knowledge, work around incomplete docs, ask questions when confused.

• AI requires explicit naming, has zero tribal knowledge, treats documentation as a contract, and never asks questions. It just hallucinates.

🔖 Related Reads

While AI can handle complexity, it can’t handle our complexity.

Think of the shortcuts: The customer_id that means three different things depending on which Slack thread you read from in 2023. That’s the worst type of debt: silent. But we got used to it.

Then we got AI, so as long as you define things correctly, it will follow them properly (most of the time). All the good things we started adding to improve our data modeling setups made them powerful and a huge debt accumulator.

• Before AI: A new column on a backend table meant adding a new table on a dbt model, then tracking which dashboard was using it and updating charts.

• After AI: Add metadata catalog, semantic layer, AI context docs, MCP configs, validation rules, example queries. Skip any step and AI won’t be as good.

True, but every new change brings new constraints, and with the industry's current pace, it’s becoming hard even to tell it to consider X or Y next time.

We added new cool stuff to automate lots of work, and now we need to automate tools to maintain the tools that automate things for us.

The Silver Lining

Preparing data, no matter if it’s for data analytics workflows, internal BI, or AI, accidentally makes it better for humans, too.

Unambiguous naming, explicit documentation, semantic layers, clear relationships: all the stuff we should have been doing but didn’t, because humans could muddle through.

But it goes deeper. As I wrote in How AI Can Improve Your Data Modeling (And What’s Still Missing), the BIG question is how far companies are from understanding their own context, regardless of AI.

Decision rationale lives in Miro boards, Slack threads, and people’s heads. When you treat your schema as a product with a hybrid consumer on the other side (human and AI), you start capturing the “why” by default.

It’s amazing how many tools were added to the stack, how we are more productive in so many ways. We are still lifting up the carpet and finding issues that should have been solved before this.

AI exposed, in most cases, that we never had a structure at all. We were “vibing” before vibe coding.

But I am happy that one way or another, AI opened a lot of challenges that we wouldn’t have tackled at all if it wasn’t for AI readiness needs.

Final Note

In July 2025, we wrote about data professionals becoming orchestrators of the agentic stack. Eight months later, the prophecy landed, but just not where we expected.

Tools will catch up. Context windows will grow. MCPs will improve. When that happens, the teams that invested in human readiness will be ready. The ones that didn’t will pay the tax if they want to play the AI game.

The real orchestration about managing AI agents relies on managing the readiness that makes agents useful.

MD101 Support ☎️

If you have any queries about the piece, feel free to connect with the author(s). Or connect with the MD101 team directly at community@moderndata101.com 🧡

Author Connect

Connect with Alejandro Aboy on LinkedIn 💬 | Or find more of his work on Substack:

The Pipe & The Line

Hands-on guides, tools, and experiments to sharpen your Data & AI Engineering skills from someone who learned it all in the wild.

By Alejandro Aboy

From MD101 team 🧡

🌎 Global Modern Data Report 2026

The Modern Data Report 2026 is a first-principles examination of why AI adoptions are getting blocked inside otherwise data-rich enterprises. Grounded in direct signals from practitioners and leaders, it exposes the structural gaps between data availability and decision activation.

With hundreds of datapoints from 540+ data leaders and experts from across 64 countries, this report reframes AI readiness away from models and tooling, and toward the conditions required and/or desired for reliable action. Already downloaded 1000+ times, this report is enabling key insights for several industry practitioners this year. Join the discussions!

Get your copy

Camila Barreto Lima | Product Manager, Data-driven Platforms

Camila Barreto Lima is a Product Manager focused on building and scaling data products within global digital ecosystems across fintech and travel tech. She has led the end-to-end delivery of data products used in 55+ countries, delivering measurable outcomes such as increasing product adoption from 70% to 94%, enabling over $10B in transactions, and achieving NPS scores above 90%.

At Conetic, Camila owns the strategy and delivery of data products, defining roadmaps, governance frameworks, and lifecycle best practices. She works at the intersection of business and engineering, translating complex data and technical architectures into scalable, user-centric products that drive decision-making and operational value.

Previously, she has contributed to global platforms at Dell Technologies and other organizations, leading cross-functional teams across B2B and B2C environments. Her work consistently focuses on establishing single sources of truth, improving data accessibility, and embedding product thinking into data initiatives. We’re thrilled to feature her insights on Modern Data 101.

We actively collaborate with data experts to bring the best resources to a 20,000+ strong community of data leaders and practitioners. If you have something to share, reach out!
🫴🏻 Share your ideas and work: community@moderndata101.com

*Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality & relevance. We keep it information-first and do not support any promotions, paid or otherwise!

Let’s Dive In

Across industries, dashboards are built at scale. They sit on top of modern data lakes, cloud architectures, advanced models, and sophisticated BI tools. And yet, many of them fail because the intention is unclear.

You may know what your data is supposed to show.

• But can your users immediately understand it?

• Can they act on it confidently?

• Can they make a decision in under a minute?

Data, by nature, is made of numbers. And numbers alone rarely speak for themselves. They require interpretation, context, and logic. Often, only those who built the model truly understand the metrics, transformations, or calculations behind it. But dashboards are not created only for data specialists.

Different teams interpret the same data in different ways:

• The business team sees strategic direction.

• The commercial team looks for revenue and performance signals.

• The HR team searches for workforce insights.

If your users need to inspect metric logic, DAX formulas, or data transformations just to understand what they’re looking at, chances of adoption are low.

Data Needs Context

Years ago, I learned through research that people absorb information differently. Some think in numbers. Others in text. Others through visual patterns or stories.

That realization changed how I thought about communication entirely. If people process the world through such different lenses, then presenting raw information, no matter how accurate, was never going to be enough. A spreadsheet full of numbers speaks fluently to one mind and says nothing to another. The question was never really "what does the data say?" It was always "how does this person need to receive it?"

Data may be numerical, but understanding requires context. Meaning does not exist in the number itself, but is formed in the narrative around it. And only a few companies are truly conscious of this.

This is where data storytelling becomes essential. It is not about presenting numbers, but guiding that myriad interpretation. It is about ensuring your audience understands what matters and why.

Note: As humans, we trust what we can read, relate to, and make sense of.

Design Is Not Enough

Modern, attractive design improves usability. But design alone cannot fix poor clarity.

You can have beautiful layouts, modern visuals, powerful infrastructure, and advanced storage (like cloud environments or distributed computing platforms). But without storytelling and clarity, all that investment risks becoming expensive decoration.

Behind your dashboard, you may have a structured data lake, cloud storage solutions, or scalable processing engines. That’s excellent. But without intentional narrative design, it may become wasted effort because…

Why Do Users Still Love Excel

A common concern often arises: “Why do users prefer Excel instead of our BI tool?” The answer is simple:

Excel gives them control.

Users can manipulate the data, transform it, and build their own narrative. That autonomy creates confidence. But here’s the risk:

Are users interpreting the data correctly? Not always.

Manual transformations can introduce bias, errors, and inconsistent logic. Yet when users don’t understand a centralized dashboard, they may distrust it even if it’s technically correct.

When data is structured and governed, but not explained, trust declines, and users revert to what feels safer. And trust is everything.

Now, take a look at your dashboards and ask yourself:

• If I had one minute before a critical decision, what would I need to see?

• What decision is this dashboard designed to influence?

• What behavior should change after viewing this?

• What is essential, and what is noise?

• What must be visible at first glance?

• What requires deeper exploration?

If you can’t answer these questions clearly, your dashboard probably can’t either.

Vanity or Value

You’ve probably heard the term “vanity metrics.”

Vanity data does not drive decisions, nor does it influence action. They inflate perception without influencing outcomes. They distract from what truly drives performance.

Data storytelling requires relevance, and it demands that every metric has a purpose. A powerful example of storytelling through data is Spotify Wrapped.

Why does it work?

• It has a clear purpose: reflecting on your year.

• It aligns emotionally with a moment of reflection.

• It shows direct, meaningful consumption data.

• It focuses on what matters to you.

Imagine if it showed:
“Top 1,000 Least Listened Artists.”

Technically interesting? Maybe. Meaningful? No.

Instead, it tells a story about your behavior. It guides you through your own data. It is clear, focused, and intentional. It leads you through the narrative, not the other way around.

I see companies investing in predictive thinking. It would resolve their problems to get insight, and you can invest millions to get the perfect prediction. However, it will still be wasted because the problem is not that you don’t have the right data, but that you’re not presenting it correctly.

Here are a few tips to build Data Storytelling

1. Start with decisions, not data.

Map the core decisions your users make daily.

2. Prioritize ruthlessly.

Structure visuals by hierarchy:

• Overview (strategic signal)

• Critical risks

• Sensitive or attention-driven areas

• Supporting insights

3. Reduce interpretative friction.

Limit conflicting filters on the same visual and avoid ambiguous titles.

4. Replace broad questions with specific ones.

Instead of “How are users engaging?” use: “Is feature X increasing weekly retention?” Specific questions produce actionable answers.

5. Optimize for cognitive clarity.

You don’t need a whole page for one single visual, believe me.

Final Note

Data storytelling is not a luxury but a must-have if you want your product to grow and be meaningful and impactful. A dashboard should be seen for what it is: a communication tool.

If your data does not tell a story, your users will create their own, and that version may not be the one you intended. So the real question is: “Are we telling the right story?”

MD101 Support ☎️

If you have any queries about the piece, feel free to connect with the author(s). O connect with the MD101 team directly at community@moderndata101.com 🧡

Author Connect 💬

Connect with Camila on LinkedIn 🤝🏻

From MD101 team 🧡

The Data Product Playbook

Here’s your own copy of the Actionable Data Product Playbook. With 4000+ downloads so far and quality feedback, we are thrilled with the response to this 6-week guide we’ve built with industry experts and practitioners. Stay tuned to moderndata101.com for more actionable resources from us!

DOWNLOAD!

Andrew Jones | Creator of Data Contracts, Principal Engineer & Author

Andrew Jones is a Principal Engineer and Author, specialising in modern data architecture and platform design. He is widely recognised as the creator of data contracts, an approach introduced in 2021 that redefines how organisations manage data quality, ownership, and governance by treating data as a product with explicit, enforceable interfaces. He is also part of the LinkedIn Top Voices recognition, an invitation-only group of experts across the professional world.

Andrew has spent over a decade building large-scale data platforms, including leading foundational data platform initiatives at GoCardless, where he pioneered the implementation of data contracts to enable reliable, revenue-generating data products. He currently serves as a Principal Data Engineer at Springer Nature.

In 2024, he authored a definitive book on data contracts, consolidating years of practical experience into a widely adopted industry reference. Beyond his technical work, Andrew is an active contributor to the data community, a member of Google Developer Experts, and part of the technical steering committee for Bitol, helping shape open standards for data contracts and data products. We’re honoured to feature his insights on Modern Data 101.

We actively collaborate with data experts to bring the best resources to a 20,000+ strong community of data leaders and practitioners. If you have something to share, reach out!
🫴🏻 Share your ideas and work: community@moderndata101.com

*Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality & relevance. We keep it information-first and do not support any promotions, paid or otherwise!

Let’s Dive In

Traditional data platforms are a mix of disparate datasets with different attributes, different capabilities, and different levels of governance, each created and managed through complex custom workflows.

For data consumers, this results in data that is difficult to use. Every time someone wants to consume a dataset, they need to gather all the context required to do so, from scratch, often by tracking down and asking the right individual in the organisation. This increases the cost of using data, reducing its applicability.

There are no standard naming conventions, and no common keys to join on. If they want to join this data, they will have to create complex pipelines to do so first.

On the other side, data producers find publishing data is expensive and time-consuming. They need to learn how to make this data available and create all the workflows to do so. They then, as data owners, need to learn how to manage this data in accordance with a dozen internal data standards and take responsibility for doing so.

All of this disincentivizes the sharing of potentially valuable data.

What if there is a different way to build a data platform?

One that is made up of consistent, interoperable, governed data products that can be easily and confidently consumed to create valuable applications, either directly or through the use of AI, all built on a Unified Data Infrastructure?

However, to build that, we need to apply more structure and discipline to the data, and in particular, its metadata.

It’s this metadata that gives consistency to the data, easing the use of data by data consumers, human or AI. It also gives the platform the context it needs to automate these management tasks, guaranteeing governance.

We capture this metadata in the data contract.

And so, it’s the data contract that enables the contract-driven data platform.

What is a contract-driven data platform?

A contract-driven data platform moves the focus of data engineering and platform teams away from building point solutions to building a platform of generic capabilities that allow them, and anyone else, to easily and cheaply build interoperable data products without having to think about:

• How do I make this data available?

• How will this data work with existing datasets?

• How do I deploy this data pipeline and related workflows?

• How do I implement data retention?

• How do I manage access?

And so on.

It is a step-change in how we build data platforms that reduces the cognitive load for both data producers and data consumers, enabling the greater availability and applicability of data across the organisation.

As I wrote in my book, Driving Data Quality with Data Contracts, there are three principles to a contract-driven data platform.

• Automation

• Consistency

• Governance

Automation

There are several common tasks that need to be carried out on the data and the resources we use to manage it, no matter what that data is and who owns it. These tasks are great candidates to automate, reducing the effort the data producers need to spend managing the data.

These include taking backups of the data, moving data to different storage engines based on its age, and anonymizing or deleting personally identifiable data as it exceeds its retention period.

Using the metadata defined in the data contract, we can easily build small services that automate these tasks. These services can be deployed alongside each contract and have sensible defaults, so most data producers never need to configure them.

We’ll be looking at some examples of these later in this article.

Consistency

By providing this standard tooling to all data generators through data contracts, we’re promoting greater consistency in how our data is managed, accessed, and consumed.

Every data consumer knows how to discover data that is managed through a contract-driven data platform. They know how to look up the expectations around that data, and how to find its owner. They know how the access controls are set up, and how to ask for the permissions they need.

Similarly, every data producer knows how their data and resources are being managed. They can switch between working on different datasets without losing context or having to learn new tools or a slightly different implementation of the same tools. When there is an incident, they know exactly where to look for their service configuration, to view their observability metrics, and recover data from their backups.

This contract-driven tooling becomes the golden path. It is the only supported tooling and the default choice for data producers across the organization.

Governance

We want to enable data producers across the organization, but we can’t expect everyone producing data to be experts in data management. Instead, we can provide them with the tooling that guides them and take care of as many of these concerns for them as we can.

That allows the data producers to focus on delivering the data to their consumers, without constant engagement and review from central data governance teams, which becomes a bottleneck and slows down development speed across the organization.

The reason why that review is no longer needed is that the governance is embedded in the tooling they are using. As long as data producers are using the tools provided by the contract-driven data platform, the data governance teams can be certain the data is managed in accordance with their governance standards.

We’ll see an example of this shortly.

Implementing the contract-driven data platform

Now that we understand the power of the contract-driven data platform, let’s look at how we would implement it. We will start by defining the data contract, then show how this data contract drives the data platform.

Defining the data contract

The first thing you need when building a contract-driven data platform is, of course, to define the data contract.

This data contract needs to be in a human- and machine-readable format. Being human-readable allows it to act as an artifact that can be discussed by different groups and teams across your organisation, while being machine-readable enables the automation and tooling described earlier.

There are many ways to define a data contract. Many choose a YAML file that follows the structure of the Open Data Contract Standard (ODCS), some define their data contracts in code, some use a custom UI, and some define them in spreadsheets!

The key is to choose a format your users will be happy to write and maintain, which ideally means meeting them where they already are. This reduces their friction to adopting data contracts and provides them with the best user experience.

If those users are software engineers, then it’s best to provide them with a data contract definition they can maintain as code, alongside other code artefacts within their git repository, alongside the code that generates the data. That’s why my first data contract implementation used Jsonnet to define the data contracts, because that’s exactly what our software engineers expected.

However, if your users are not software engineers but instead admins for CRMs or other data systems, then a spreadsheet or custom UI would be a better solution.

The next important consideration is to ensure the data contract is subject to change management. This is critical for providing data products that data consumers can build on with confidence, knowing the structure and semantics of the data will not change in a way that breaks your application without giving you a chance to migrate to a newer version of the data contracts.

In practice, this means your data contract is versioned, and there are checks in place to implement compatibility rules to prevent breaking changes without a new version being created and a migration path followed.

The final part every data contract needs to have is a description of the data - i.e., a schema. A data contract that doesn’t describe data would not be much use.

For the purposes of this example, here’s a minimal data contract defined in YAML following the ODCS:

Driving the data platform from the data contract

Now that we have our data contract, we can start to drive our data platform from it.

Firstly, we can use this data contract to provision a table in the data warehouse by simply taking the data contract, converting it into a format understood by our infrastructure as code platform (Terraform, Pulumi, etc), and having it manage this table.

As the data contract is under change management, then by extension so is the table in the data warehouse. This gives our data consumers a stable interface they can build on with confidence, whether they are building reports, data-driven applications, or leveraging AI.

But with a little bit more context in the data contract, we can add more capabilities to the data platform. For example, we could add quality rules to some of the fields and a channel for alerts:

The data platform can now automatically deploy an observability service to run those rules and route the alerts to the right team.

As a data producer, this is great! I no longer need to set up my own infrastructure to run data quality checks, manage my own orchestrator, and so on. I just describe what I want, and the platform automates it.

Similarly, one of our fields contains the email address of a customer. This is personally identifiable and needs to be managed correctly in line with our data governance policies and local regulations.

As a data owner, I’m responsible for the management of this field, but I’m not an expert in data governance, and I don’t want to have to build a process and automation to do this when I have delivery work to prioritise. Instead, I should be able to categorise it in the data contract:

Then the data platform has all the context it needs to ensure this field is managed correctly, with access granted to those with legitimate need to access the raw data, and data retention policies applied when we should no longer be storing this data.

All of these services are relatively trivial to implement. They simply take the context from the data contract and apply the correct automations and rules to the data it describes.

Because we are building on the metadata, not the data, these tools and any future tools we provide are automatically available to all data associated with a data contract, no matter what domain the data belongs to, its owner, or its specific structure. All the required context is in the metadata, allowing the tools to be generic.

Since I started implementing data contracts over 6 years ago, this is how I’ve been building data platforms, and so far, I’ve yet to find a data platform capability that cannot be implemented in this way, from simple capabilities such as backups to role-based access management.

This is the power of the contract-driven data platform, a step-change in building a data platform that is made up of consistent, interoperable, governed data products.

MD101 Support ☎️

If you have any queries about the piece, feel free to connect with the author(s). Or feel free to connect with the MD101 team directly at community@moderndata101.com 🧡

Author Connect 💬

Connect with Andrew on LinkedIn 🤝🏻

More from Andrew

Learn more

From MD101 team 🧡

🌎 Global Modern Data Report 2026

The Modern Data Report 2026 is a first-principles examination of why AI adoptions are getting blocked inside otherwise data-rich enterprises. Grounded in direct signals from practitioners and leaders, it exposes the structural gaps between data availability and decision activation.

With hundreds of datapoints from 540+ data leaders and experts from across 64 countries, this report reframes AI readiness away from models and tooling, and toward the conditions required and/or desired for reliable action. Already downloaded 1000+ times, this report is enabling key insights for several industry practitioners this year. Join the discussions!

Get your copy

Gaurav Patole | Principal Data Strategy & Governance Advisor, Author of Data Quality ROI

Gaurav Patole is a Principal Data Strategy & Governance Advisor at ThoughtWorks and the author of Data Quality ROI. A seasoned data governance and data quality practitioner, he brings deep, hands-on experience helping global organisations turn trust in data into measurable business value.

Formerly with BCG, Gaurav has led end-to-end data governance and quality programs across industries and geographies, bridging business and technology through his People-Process-Technology (PPT) approach. A frequent speaker and active community contributor, he is known for making data governance practical, people-centric, and ROI-driven. We’re thrilled to feature his unique insights on Modern Data 101!

We actively collaborate with data experts to bring the best resources to a 20,000+ strong community of data leaders and practitioners. If you have something to share, reach out!
🫴🏻 Share your ideas and work: community@moderndata101.com

*Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality & relevance. We keep it information-first and do not support any promotions, paid or otherwise!

Let’s dive in

Many organisations today are moving towards federated or domain-centric operating models, thereby ensuring more accountability and responsibility in the hands of people who are closest to the data.

The approach is supposed to achieve ‘federated’ ways of working. However, in many companies, it has led to ‘fragmented’ ways of working when it comes to data governance or data quality.

Hence, it’s important to diagnose a critical question:

The Data Quality Partnership Matrix

If the promise of federated data governance and quality is to be truly realised, unseen walls between business and data/IT teams must come down.

These two entities are not meant to operate in separate vacuums. Instead, they must work in seamless collaboration to conquer business challenges. This critical interplay is visually represented in what I term the “Data Quality Partnership Matrix,” which underscores the vital balance needed to break free from DQ silos.

No matter how skilled your IT team is, how passionate your data stewards are, or how invested a few business leaders might be, data quality cannot be improved in isolation.

Like physics, it takes a combination of variables to move the needle. In the enterprise world, this means partnership.

• X-axis: Support. This represents the extent to which the central data governance or data quality teams provide education, communication, standards alignment, and tools to the business counterparts to manage and improve data quality.

• Y-axis: Business engagement. This signifies the level of involvement, ownership, understanding, and active participation of business stakeholders in data quality initiatives.

State of Overwhelm and Chaos

Low Business Engagement, Low Support

This is the worst scenario. In this state, business teams are likely struggling with poor data quality that affects their operations and decision-making. However, there’s little to no guidance, training, communication, standardised tools, or alignment on standards from a central data governance or quality function.

Simultaneously, the business may not fully recognise the impact of poor data or feel empowered to address it. Data quality issues go unnoticed or are ignored, leading to poor decision-making, inefficiencies, and possibly significant compliance or regulatory risks.

For example, marketing campaigns often suffer from inaccurate segmentation due to incomplete or outdated customer data. The marketing team blames IT or the data teams for “bad data.” At the same time, IT lacks business context to understand the specific data quality issues, and there’s no formal process for reporting or resolving them.

Eventually, bad data remains in that state for a long time, as there is no ownership involved to resolve the issues.

State of Stagnation

High Business Engagement, Low Support: Non-scalable Data Quality

A common quadrant for most companies today. In this quadrant, business teams are highly aware of and concerned about data quality issues. They might even be proactively trying to fix problems within their domains.

However, the lack of central support in terms of education, communication of best practices, alignment on standards, and provision of practical tools limits their ability to implement scalable and consistent solutions.

As a result, the business is forced to bear a heavy burden in managing DQ on its own, which may lead to burnout, inefficiencies, and localised fixes that don’t address the root causes or integrate well across the organisation.

This is where you’ll see the ‘Shift Left’ phenomenon widely discussed and propagated by business teams, basically because the business teams are tired of temporary solutions to fix data and want to pass on the responsibility to source teams from where the data originates.

For example, a product development team tries to manage product data on its own, meticulously tracks and corrects data inconsistencies in its product specifications using spreadsheets, but these efforts aren’t aligned with how the same data is managed in other systems (e.g., manufacturing, sales) due to lack of central data standards and communication.

They spend excessive time manually cleaning up spreadsheets, which diverts them from their core responsibilities. Product teams, therefore, become overwhelmed and frustrated as they attempt to handle DQ without proper support, resulting in a lack of scalability and potential degradation of data quality over time.

State of Waste

Low Business Engagement, High Support: Wasted Data Potential

Another prevalent situation involves organisations heavily investing in data governance and quality initiatives, often outsourcing these efforts to external IT or consultancy firms, which are expected to manage poor data within day-to-day operations.

In these cases, central data governance or data quality teams often invest heavily in sophisticated tools without assessing core business needs, and many also launch extensive training programs through Learning and Development, as well as align technical processes with source systems.

On paper, the support structure looks strong. But there’s one critical flaw: business teams are not genuinely engaged. They either do not fully understand the importance of the initiatives or fail to see enough value to change their day-to-day behaviour.

As a result, resources are underutilised, and improvements often fall short of their mark. The data quality efforts usually end up solving technical problems without addressing the real business pain points. Over time, this disconnect not only wastes valuable investment but can also reinforce silos due to a lack of shared ownership and accountability.

For example, an organisation rolls out a sophisticated data catalogue and data quality monitoring system. The governance team also creates detailed training modules and conducts workshops. However, business users rarely interact with the platform, resist altering their daily workflows, and often view governance processes as irrelevant overhead.

Without active business participation, the well-supported infrastructure fails to tackle the most pressing data quality issues, ultimately leading to poor adoption and low impact from the DQ programs.

State of Growth

High Business Engagement, High Support: Strategic Data Quality Management

This is the ideal state. Both business and governance teams are fully engaged and supportive of data quality initiatives. Business users actively participate in identifying and communicating quality issues, providing remediation guidance, and improving processes. Meanwhile, centralised governance functions provide the necessary frameworks, tools, training, and coordination to scale these efforts across the organisation.

The result is a strategic, business-aligned data quality management practice that drives real value, thus improving decision-making, operational efficiency, and regulatory compliance. Importantly, this environment fosters true collaboration rather than ownership battles. Data quality becomes everyone’s responsibility, embedded naturally into daily processes rather than seen as an external imposition.

For example, a financial services firm establishes a governance framework that enables business domain leads to work closely with central governance teams. Business units are responsible for defining critical data elements and setting quality thresholds, while the central team provides stewardship platforms, monitoring tools, escalation workflows, and ongoing training.

When issues arise, they are jointly triaged based on their business impact. As a result, data-driven decision-making improves significantly, regulatory audits become easier to manage, and customer trust strengthens over time.

In the end, true data quality excellence requires not just strong tools or passionate teams in isolation, but a partnership where business engagement and all allied teams work in tandem, consistently, transparently, and strategically.

MD101 Support ☎️

If you have any queries about the piece, feel free to connect with the author(s). Or feel free to connect with the MD101 team directly at community@moderndata101.com 🧡

Author Connect 💬

Connect with Gaurav on LinkedIn 🤝🏻

More from Gaurav

Learn more

From MD101 team 🧡

🧡 Join 2000+ on the State of Data Products

Introducing The Catalyst, a special edition of The State of Data Products that compiles insights around the ups and downs in the data and AI arena that fast-tracked change, innovation, and impact.

📘 The Catalyst, first release is our annual wrap-up that compiles the most interesting moments of the year in terms of developments in AI & data products, with expert insights from across the industry.

Some focal topics

• The gap in AI readiness for enterprises

• The shift towards context engineering

• Addressing governance debt with a focus on lineage gaps

• Revisiting fundamentals for better alignment b/w business vision and AI.

Get your copy!

Arun Gamidi | Enterprise Data & Analytics Leader

Arun Gamidi believes most data initiatives fail not because of bad technology, but because they never connect to the decisions that actually matter. A data and analytics leader with deep experience in financial services, Arun has spent his career building and scaling data ecosystems in high-stakes enterprise environments where the cost of unclear decisions is real.

His work is grounded in a decision-first philosophy: data and AI create value only when they clarify the choices leaders must make. His framework, DATA by Design (Direction, Alignment, Trust, and Activation), gives organisations a repeatable path to align data initiatives with business goals, build trust in their information, and ensure insights reach the moment of decision.

Arun currently leads Enterprise Data & Analytics at MGIC. Through that work and his weekly Leadergence newsletter, his focus is on equipping data leaders to operate with confidence, not just access, in an AI-accelerated world. Readers can follow his ongoing thinking at Leadergence on LinkedIn. We're thrilled to feature his insights on Modern Data 101!

We actively collaborate with data experts to bring the best resources to a 20,000+ strong community of data leaders and practitioners. If you have something to share, reach out!
🫴🏻 Share your ideas and work: community@moderndata101.com

*Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality & relevance. We keep it information-first and do not support any promotions, paid or otherwise.

Let’s Dive In

They do not carry the weight of a wrong call or the satisfaction of a right one. That capacity belongs entirely to people. This is not a limitation of current technology, but a structural reality that more advanced systems will not change.

When something goes wrong, no one asks the model what it would do differently. They ask the leader responsible for the project or the model owner running the algorithm. That question has always pointed in the same direction, and it always will.

The Problem: Accountability Drift

No leader decides one morning to stop being accountable. It happens gradually in the space between a recommendation arriving and a decision being made.

Systems are fast, and they are confident. They surface answers before the question has fully formed. That speed creates a pull, and most leaders do not notice they are following it.

The question shifts. It moves from “What am I responsible for deciding?” to “What does the data say?” That may look like diligence, but in practice, it is the beginning of distancing ourselves from the solution and further from the end users.

The problem is not bad intent or negligence, but drift. And drift is harder to catch than negligence because it never announces itself. It accumulates over time and compounds in impact and effect.

Accountability does not follow that drift: it does not reassign itself to the system that produced the insight or the model that ranked the options. It stays exactly where it was- with the person who had the authority to decide. That gap, between where judgment drifted and where accountability remained, is where organisations become fragile.

The Core Tension Between Machine Autonomy and Authority/Ownership

Separation of Concerns

Software has a concept called separation of concerns. The idea is simple: each component should do one thing, and do it well. When components try to do too much, or when their responsibilities bleed into each other, the system becomes fragile. The same principle applies to how organisations use systems to make decisions.

Systems are built for a specific job. They:

• Detect patterns in data that humans cannot process at scale.

• Assign probabilities to outcomes

• Surface trade-offs that would otherwise stay hidden.

These are all precise and valuable functions. It is not the same function as deciding, and it was never designed to be. This is a feature of systems instead of hard flaws. In product design, a well-scoped tool that does its job cleanly is more valuable than an overloaded one that does many things poorly.

The problem is not what systems are built to do, but what organisations ask them to become.

Scope Creep

There is a concept in product thinking called scope creep. It begins when a tool that solves one problem is gradually asked to solve adjacent ones it was not designed for.

Insight and authority are not the same thing. Insight is an input which informs, surfaces, and recommends. Authority is a commitment which decides, owns, and answers for outcomes.

When organisations stop maintaining that distinction, the system is asked to carry weight it was never architected to hold.

Single Point of Failure

In engineering, the concept of the single point of failure is prevalent. It occurs when too much depends on one component that was not built to bear that load. Over-reliance on system recommendations creates a different but related failure mode.

Not a single point of failure, but a single point of diffusion. Responsibility spreads so evenly across dashboards, models, and outputs that it effectively disappears. Everyone referenced the system, but no one owned the call.

This is where the confusion between insight and authority does its real damage. It does not show up immediately in outcomes. It shows up first in behaviour.

And when outcomes eventually disappoint, the post-mortem reveals the same gap every time. The data was there, and the recommendation was clear, but the ownership was not.

Accountability had blurred long before anything failed, because no one had maintained the boundary between what the system was built to do and what only a person can do.

Systems are not accountable. They are not designed to be. Ownership is a human function, and no increase in model capability or self-governance changes that. The tension is not between humans and technology.

The Three Requirements for Leaders

Systems do not close the gap between insight and accountability. Leaders do, and they do it through three specific capacities, each one distinct, each one dependent on the others.

Clarity: Define the Boundary Explicitly

In software systems, when two components interact without a clear contract, each assumes the other is handling something neither actually is. The result is a subtle and soft gap where responsibility falls through. Leadership inside data-driven organisations has the same problem.

Clarity begins with a question leaders rarely ask explicitly. Which decisions can systems inform, and which decisions must people own? Those are not the same category, and treating them as interchangeable is where accountability begins to break down.

The boundary matters because data does not make it visible on its own. A recommendation arrives with confidence along with a probability attached. The output looks complete.

More data does not resolve this, but compounds it, in fact. When information is abundant and confidence intervals are tight, it becomes easier to treat insight as a decision. And that emotion holds until outcomes are tested.

Then the gap becomes visible, and so does the absence of ownership. The boundary has to be defined before the recommendation arrives, not after it is challenged.

Presence: Stay Engaged When Automation Creates Distance

There is a pattern in product development called automation bias. It describes the tendency of users to over-trust automated outputs, especially when the system appears competent, and the feedback loop is slow.

The more reliable the system seems, the more likely a person is to disengage from the decision it is supporting. Leaders are not exempt from this pattern.

As systems scale, the work arrives pre-processed. Inputs are summarised, options are ranked, and outputs feel partially complete before any human speaks. That is the design working as intended. It is also where presence becomes harder to maintain and more necessary than ever.

Presence is not scepticism for its own sake. It is not slowing down every recommendation or second-guessing every model. It is staying connected to the consequences that the system is not built to handle. It sounds like specific questions.

• What assumption is driving this recommendation?

• What has changed in the environment since this model was last validated?

• What outcome are we prepared to own if this is wrong?

Those questions are the mechanism by which leaders remain accountable. When leaders disengage because the system appears confident, accountability does not transfer to the system. It simply goes unclaimed. And unclaimed accountability surfaces in outcomes, long after the moment to exercise judgment has passed.

Discernment: The Moment Leadership Becomes Visible

In product design, the most important interface is the one that handles edge cases, exceptions, and moments where the expected path breaks down. That is where design reveals its actual quality.

Leadership is no different.

Discernment is what happens at the edge of the system’s capability. It is the moment when a recommendation has arrived, the data has been reviewed, and the leader must decide what to do with it. That choice does not belong to the model.

This is not the failure of automation. Automation did its job by processing the inputs, applying the weights, and surfacing an output. What it cannot do is bear the consequence of acting on that output. That responsibility does not transfer, regardless of how capable the system becomes or how confident the recommendation appears.

Discernment is clarity and presence made actionable.

Organisational Maturity Is Measured by Explicitness

That is the through line in nearly every myth worth studying. Icarus did not fall because the wings failed. He fell because no one had clearly established where the boundary was, and he had not internalised why it existed.

It is possible to fly too close to the sun when it comes to AI and machine autonomy. Organisations have already been burned terribly and 95% of pilots failing is a testament to that.

Organisations make the mistake of measuring maturity by capability. By how sophisticated the models are, how fast the dashboards update, and how many decisions the system can support in a given quarter. That is measuring the wings, not the judgment of the person wearing them.

The epics and modern-day case studies alike tell us again and again that the gift and the responsibility are inseparable. One without the other is not a blessing, but a liability.

The question is never whether the tools are powerful enough. At this point in time, technology has never produced more powerful tools, especially in the hands of the masses. The question is whether the humans using them have been explicit about what they are responsible for and if there is a framework to guide that behaviour. Where human judgment must remain decisive and cannot be delegated to a model, regardless of its accuracy.

Maturity is not measured by how advanced the tools are. It is measured by how explicitly leaders answer three questions before the recommendation arrives, before the outcome is tested, and before accountability needs to be assigned.

• Who decides

• Who owns what follows

• And where does human judgment remain the final word

When those answers are clear, the organisation uses its tools better.

The Practical Test

• When accountability is clear: leaders don’t hide behind recommendations; decisions move with ownership

• When accountability is unclear: hesitation, blame, diffusion of responsibility

• This is a leadership design problem, not a technology problem

The Question Worth Carrying

Where does judgment stop being computational and start being mine?

That question is what keeps accountability from drifting, presence from fading, and discernment from being quietly outsourced to a recommendation that arrived with confidence but carries no consequences.

Leaders who carry that question become more effective as systems grow more capable because they remain clear about what the system is doing and what they are doing.

Confidence does not come from certainty, from a tighter model, or a more reliable dashboard. It comes from knowing what you are responsible for and being willing to say so out loud. That willingness, to name the decision as yours, to own what follows, is not a soft skill sitting at the edge of leadership but the centre of it.

MD101 Connect ☎️

If you have any queries about the piece, feel free to connect with the author(s). Or connect with the MD101 team directly at community@moderndata101.com 🧡

Author Connect

Connect with Arun Gamidi on LinkedIn 💬

From MD101 team 🧡

The Data Product Playbook

Here’s your own copy of the Actionable Data Product Playbook. With 3500+ downloads so far and quality feedback, we are thrilled with the response to this 6-week guide we’ve built with industry experts and practitioners. Stay tuned to moderndata101.com for more actionable resources from us!

DOWNLOAD!

Anna Bergevin | Sr. Manager Data & AI Product Management

Anna Bergevin is a data and AI product leader currently serving as Sr. Manager of Data & AI Product Management at Children’s Mercy. She leads platform product strategy focused on enabling secure, compliant, and scalable data systems that power analytics and AI across the enterprise.

Previously, she led global Data & AI platform product initiatives at ResMed, guiding roadmap and governance strategy for a distributed engineering organization supporting business teams in over 140 countries. Her work centers on semantic modeling, metric standardization, metadata-driven governance, and designing data products that balance regulatory rigor with innovation.

Anna writes and speaks about data product management, sociotechnical AI adoption, and the discipline required to build enterprise data systems that scale on When Data Met Product. She believes the future of AI depends less on model sophistication and more on clear definitions, strong governance, and product-led stewardship of data infrastructure. We’re thrilled to feature her unique insights on Modern Data 101!

We actively collaborate with data experts to bring the best resources to a 15,000+ strong community of data leaders and practitioners. If you have something to share, reach out!
🫴🏻 Share your ideas and work: community@moderndata101.com

*Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality & relevance. We keep it information-first and do not support any promotions, paid or otherwise.

Let’s Dive In

Data platform product managers have an impossible job: drive adoption and impact without formal authority over the teams they serve.

To succeed, you need to think like a game theorist. Game theory, the study of strategic decision-making, assumes people act rationally given their incentives and constraints. Behavior that may appear confusing on the surface makes sense if we dig deeper to understand the incentives that drive their behavior.

The problem isn’t that your stakeholders are irrational. It’s that you often don’t understand the games they’re playing.

🔖 Related Read

Software engineers aren’t ignoring your metadata standards because they’re lazy. They’re optimizing for velocity because that’s how they get promoted, and the standards require changes that feel burdensome to adopt.

Analysts aren’t creating warehouse chaos because they don’t care about governance. They’re solving today’s urgent business question because that’s what their VP is waiting for. They aren’t thinking about how their quick solutions create warehouse issues at scale.

The Incentive Map is the invisible layer that powers organizational behavior.
Once you see it, your job as a platform PM gets infinitely easier. You stop fighting human nature and start designing with it. You build products that consider the needs of the diverse set of people engaging with the system, and not just the obvious users at the center.

Building this map? That requires systematic observation across four dimensions: structure, success metrics, systems, and people.

The Four Dimensions of Incentive Mapping

1. Structure

Data platform teams serve the entire company, which means understanding org structure across the business is critical. Organization structure reveals who sets incentives, how information flows, and what leadership believes about how teams should work together. It gives us clues about where collaboration will be easy or difficult.

When I join a new company (or hire someone onto my team), one of my first priorities is understanding the key people who surround my role. Eventually, as individuals, but first and foremost, the teams - who report into whom and how that intersects with my team.

• “Ah, okay she works on DevX, which reports to the VP Eng Platform alongside the data platform. All up through the CIO, interesting.”

• “Alright so this analyst is embedded in finance, but supply chain has their own finance analysts, what’s that dynamic?”

• “He’s an architect on the software side….reports into a product leader….looks like we don’t have a CTO, okay so software eng is seen here as a function under CPO, okay.”

These mappings uncover how the organization thinks about each group. I’ve worked on data teams under both a CTO and a CIO. Under the CTO, all of engineering (software, enterprise applications, data) shared the same org and goals.

Under the CIO, we aligned with enterprise applications while the software engineers were being driven by goals set by the product leadership. That changed the collaboration dynamic significantly.

Organizational structure determines:

• Who sets incentives, and what those incentives reward

• How planning and prioritization happen

• Where silos form and where context needs bridging

• Whether cross-team dependencies are visible or hidden

Org design decisions and changes happen for many reasons: strategy shifts, layoffs, mergers, cost reduction. But regardless of the trigger, leaders are making choices about which teams sit together, who reports to whom, and how work should flow. Those choices reshape incentives and communication patterns, whether that’s the primary goal or a side effect.

2. Success Metrics

Understanding the org structure tells you who sets incentives. Understanding success metrics tells you what those incentives actually reward. What are the called-out KPIs for different teams? Is there clarity or ambiguity? Do some people feel pulled in conflicting directions? Are individual performance measures aligned with team goals, or are they at odds?

Once you understand how the company measures success, you can see why certain tensions exist.

Example: When I didn’t understand the success metrics

Early in my product platform career, I was PM-ing the communications platform supporting email, in-app notifications, and mobile push for all product development teams.

Very quickly, I discovered a tension between Marketing and Product around communications ownership.

• Product felt they should own user engagement comms. Like many product orgs, they were measured by Monthly Active Users, and email was one of the only tools to re-engage dormant users.

• Marketing also wanted to interact with individual users and drive engagement.

At first, I was perplexed. In a B2B SaaS company, why would Marketing care about individual user emails when buyers make the purchasing decisions?

Then I had a breakthrough. I realized marketing was being measured by account retention, which in our B2B edtech product was deeply tied to usage rates. Enterprise buyers purchased licenses, but renewals depended on whether people actually used the product.

Marketing didn’t just want to email buyers - they wanted leverage to drive individual usage. And frankly, it seemed like they didn’t trust Product to do comms as well as they could.

Once I realized this, I had a lot more empathy. Marketing’s success metric was hard to impact through buyer-only lifecycle emails. And they had a point: individual product teams were sending emails that were loose on branding, lacked a comms strategy, and had no coordination across teams. We were just giving them an API and letting them do whatever. I decided to change my strategy with marketing to see if we could collaborate more closely, improve brand alignment, utilize their expertise in comms strategy on critical emails, etc.

The lesson: Conflicting success metrics create predictable tensions.

The reality:
Companies are imperfect at measuring success. As data professionals, we know why: lack of skill at picking the right KPI, office politics driving selection, data quality or availability issues that make measuring what matters difficult.

Even as companies iterate on metrics over time, tensions will probably still exist. But once you understand the metrics, you can strategize and find common ground that lets both teams find success and helps the company reach its overall goals.

3. Systems

While structure and success metrics are slower-changing dimensions, organizations have many cycles that create pressure and shift priorities and incentives for our colleagues.

Cycles like planning, budgets, promotions, and performance reviews - these create unique pressures on individuals, and they aren’t uniform. A few examples to illustrate:

Planning

You learn once again that one of your analysts has created an expensive, monstrous ingest query into Tableau. Why do they keep doing this? Didn’t you just have a great meeting talking about investing in data products, shifting left, and maximizing efficiency for the company?

Well, yeah, you did, but what you didn’t learn is that leadership just pivoted strategy hard in the quarterly planning cycle, and your analyst had to start all over with the dashboard for their group.

The leaders are stressed and want metrics asap so they can see if the pivot is working or not. Your analyst can’t wait to build a curated model on this one; they just had to get it up and running as fast as possible.

Once you understand this, you can meet them where they are. Help them ship fast now, then collaborate on refactoring it into a proper data product next quarter. It’s not resistance to your platform strategy - they just had to deliver under pressure.

Product / Launch Cycle

You’ve been working for months on a new metadata as code project. You are super excited because you’ve been talking to a few teams and think you have a prototype that will work to let them push their metadata to you through a GitHub action, and you’ll validate it from there and push to the rest of the system.

But you approach one team, and they shut you down hard. It’s confusing, you thought they were on board, and other teams have been open to testing.

You ask your manager, and they remind you that the team owns the top priority feature that is launching at a user conference at the end of the quarter. They have zero margin for error. It has to be perfect and demo-able, without any embarrassing bugs. They just cannot take on something “extra” like a governance ask that feels like a distraction they can’t afford right now.

Work with other teams instead, wish them good luck, and circle back next quarter to celebrate their successful launch and bring up metadata as code again. By then, you might even have learned from the early adopters and have a second prototype for them to test.

Leadership Pushing AI Initiatives

Your leaders are bringing up AI across the board, and the pressure is intensifying. Even though you try to showcase progress and point to other work you’re juggling (cost savings, shipping important data pipelines, etc), they just keep coming back to “How are things going on the GenAI pilots?”

The tech is new, and everyone is learning, so progress is slow, and you are irritated - we’ll get there when we get there. Why is he being so intense about this?

What you aren’t seeing is that your leaders have a board meeting and budget cycle coming up. They are worried the board will ask hard questions, and they won’t have clear progress to show on an AI feature that resonates with that audience.

They don’t want a lukewarm update. They want a wow factor. Additionally, with the annual budgeting cycle, they need to justify renewal of that GenAI spending (or maybe an increase) and are worried lukewarm results will make that difficult.

If you understand the pressures from those board/budget cycles, the increasing AI pressure amid steady progress (but without a wow factor) makes sense. Maybe you can achieve something that feels more like a milestone they can run with?

In all these scenarios, the pattern is the same. Behavior that appears irrational, but suddenly comes into focus once you understand the pressure they’re under from these outside cycles. When you understand that, you can pivot your strategy: what you’re doing, the timeline, your priorities, etc and find a path forward.

4. People

The first three dimensions give you a systematic map of incentives: where people sit in the org, what they’re measured on, what cycles they’re navigating. But organizations aren’t just systems. They’re made up of humans with egos, ambitions, histories, and relationships that don’t show up on any org chart.

Some people are more influential than their title suggests. Others have reputations, deserved or not, that shape how their ideas are received. There are grudges from past projects, alliances between teams, and personalities that either smooth collaboration or create friction.

You can’t template this layer. What you can do is stay curious. Pay attention to who holds informal power, which relationships are strained, and where past conflicts still echo.

The Four Stakeholder Archetypes

The four dimensions reviewed above give you the structural foundation of your incentive map - how your organization is designed, what it rewards, the cycles that create pressure, and the people who have influence. But to make the map actionable, you need to understand the players navigating that structure.

While every individual is unique, I have found that certain roles share predictable incentives, constraints, and pain points that shape how they operate. The four stakeholder archetypes below will help you interpret the behavior of your colleagues: what they optimize for, what drains them, and what they commonly misunderstand about each other.

And at the end of your day, understanding your users deeply and empathizing with their pain points is what product is all about.

1. Software Engineers: Optimizing for Velocity

This group includes software engineers, enterprise application engineers, solution architects, and other engineers outside the data team. For these users, the key is understanding that they are incentivized to ship business-prioritized features.

New work is seen as the most valuable, interesting, and engaging–while maintenance and tech debt are viewed as less desirable.

Software engineers value solving complex problems with autonomy, automation over manual processes, and minimizing context switching. They find many meetings burdensome, and processes a distraction; they’ll push back against unexpected pivots with no notice, and face invisible constraints (infosec, dev rules) that make “simple” requests more complex than one might think.

The common misunderstanding

Data platform PMs sometimes don’t realize that software engineers are blind to downstream data usage. They’re building for the application use case and are often not thinking of or even aware of data dependencies on their application data.

They don’t want to break dashboards or ML models, but they have no visibility into those dependencies. If you haven’t built data contracts integrated into their development lifecycle, you can’t really expect them to prevent breakage.

The design implication

Don’t create manual governance processes and expect compliance. Build automated tooling that fits their workflow: data contracts, metadata as code, and programmatic data access. Make the optimal path the easiest path, and you may be surprised by the uptake.

An example: For most businesses, one of the greatest barriers we face in getting value from data is a lack of context on the data. One of the most requested improvements nearly every stakeholder wants is better table and column level descriptions, richer metadata and more context overall.

The challenge data teams face? We don’t own that system, and when we contact the teams that do, we hear things like:

• We have a Confluence page, but it’s out of date

• Todd knows everything, so if you have questions, message Todd

• Honestly, I don’t think it’s written down in any one place (implication: context lives in the code base comments, Jira tickets, and scattered resources).

So how do we overcome this?
In the short term, you can do what I call documentation sprints. You have them send you their messy docs, no matter the format, and you work on cleaning them up and validating them.

You interview Todd and write down his answers. You beg the tech lead to populate a spreadsheet. Even if you are successful, this only solves it for a single point in time; there’s still no plan for keeping it up to date.

But we had a hypothesis that if we made it easier to send us that metadata programmatically, through a standard format, that would be better received. We also thought that we’d make the update as simple as possible: follow the format, and we’ll manage validating it, handling versioning against prior metadata commits, and cascading it downstream to the warehouse and data catalog.

When we did user research with our software engineers, this idea was very sticky. So sticky that we were hit with comments like “Oh my gosh, this would replace our Confluence page, we could use this for our own documentation” and even “I love this, it would be even better if you could add [insert many feature requests].” Suddenly, our software engineers were engaged and eager to use this solution, because it was designed to match their incentives:

• Solves a core problem they have for their application development work: documenting the system in a way that is more structured than a wiki; helping preserve official context to share across the team

• Solves the problem in ways that make sense for their preferred workflow (sending us their metadata as a code file (yaml or json) via GitHub action that manages everything else. Minimizing the burden on them to learn a complex system, while delivering benefits for themselves and the business

Understanding software engineers’ unique incentives and pain points is critical to driving collaboration between source system owners and downstream data consumers.

2. Analysts: Balancing Speed and Precision

Analysts (and data scientists) are the core users of the data platform. They are the final mile that takes all the data work we’ve done and turns it into insights and AI models that have real business impact.

For these users, the key to understanding them is that they are incentivized to solve business problems quickly, but with precision. The business needs their deliverables quickly, and with their focus on precision, they will do whatever wrangling is needed in order to build what the business is asking for and hit a timeline.

They are the rare user who has a foot in both worlds: deeply engaged with the business and its problems, with a strong technical understanding of what it takes to build well with data.

Analysts value the puzzle of finding the right data to bring to the problem and finding the right model, visualization, metric, forecast, etc. They enjoy the process of exploring and uncovering value for the business and then returning to the business to share those assets to guide decisions, power products, and spark critical conversations.

The common misunderstanding

Data platform PMs sometimes don’t realize that data analysts often have no idea what the downstream implications are of their decisions. They don’t realize how the actions of many analysts compound to generate warehouse clutter, out-of-control query costs, and challenges with managing assets over their full lifecycle.

They are deep and narrow, focused on the problems within their remit and not thinking about the health of the ecosystem overall.

💡 Platform PMs think analysts should wait for “perfect golden data products” or that analysts don’t care about data quality.

Wrong on both counts. Analysts need speed, BUT they also care deeply about precision and quality. They may also feel threatened by self-service narratives that position their work as “almost automated.”

Particularly as AI demos show rapid-fire analysis and people declare the death of dashboards. Meanwhile, skilled analysts know that 80% of the work is just getting that dataset ready and knowing enough context to use it well, and we are far from being able to automate that aspect of their work.

The design implication

Plan for speed & scale. Acknowledge that analysts may need to curate just-in-time assets, AND they also need to realize that those curations need to be managed to keep the ecosystem healthy.

Build a platform that supports the curation spectrum: quick access for exploratory work PLUS a pathway to sustainable, scalable solutions (semantic layers, metrics views). Don’t force them to choose between speed and governance. And help them see career evolution, not job elimination.

Note: Data Scientists share many of the same incentives and needs as analysts - requiring speed, precision, and the ability to explore data iteratively. However, they have unique platform requirements around stability, pipeline reliability, and model deployment infrastructure that warrant additional consideration. More here.

3. Leadership: Proving ROI and Managing Risk

Leaders are optimizing for fundamentally different outcomes than individual contributors or middle managers. They’re thinking about margin, growth, and the long-term viability of the business. They answer to boards, investors, and their own bosses. To influence leadership, you need to understand what they’re being measured on and what keeps them up at night.

What are they worried about

Leaders face constant pressure around hard decisions: org design, layoffs, budget allocation, and strategic prioritization. They want to be data-driven, but they don’t understand the complexity.

In particular, right now they have a lot of uncertainty around AI - are they investing in the right areas, are we getting ROI, do we have the right talent for this next phase, when will our industry and market transform, and how?

In short, they need information and confidence, and in particular, they need to know that when it comes to data & AI, the business is moving in the right direction and allocating our scarce resources where needed. We need to provide reassurance that the data team can be:

• An essential driver of innovation and operations, not just a service desk that builds nice-to-have reports

• A strategic partner that helps leadership have the insights they need while finding the right opportunities to leverage data & AI to help the business and our customers

The critical misunderstanding

Data platform PMs often think too narrowly. They think it’s enough to say, “I’m delivering what your team asked for.” What they really need to think about is:

• Am I adding strategic value to leaders in my org?

• Am I making sure their most important initiatives are getting their data needs met - not just responding to requests, but proactively helping identify opportunities

• Do they understand the full scope of what our team delivers for the business so that the timing of their request makes sense in the context of our overall capacity and demand on our team’s resources?

The design implication

Reserve capacity for comms and relationship building. Leadership doesn’t understand your world - you have to build that narrative. You have to connect platform work to business outcomes they care about (margin, growth, competitive advantage).

Communications from the data platform team are critical to the success of the team and your business. Make optimizing those communication channels a priority.

Business Stakeholders: To Each Their Own

Finally, general business stakeholders, though this isn’t really one archetype. The whole point is that each one is different, and you need to spend the time to build the relationship and deeply understand their pain points. What is each stakeholder optimizing for?

• For sales, it’s all about closing deals and optimizing revenue.

• For marketing, it’s about strategic growth levers and making sure we’re growing top of funnel and building brand presence.

• For the product, it’s about making sure we’re shipping what our users need and understanding where we are getting traction and what’s not quite hitting the mark.

• For finance, it’s about a multi-lens view of the business and the overall health, and making sure that those numbers are absolutely precise and can be trusted. And so on and so on for every different department.

You have to do the work to deeply understand your stakeholder’s department level strategy, their OKRs and KPIs, their foundational analytics gaps, and the opportunities AI brings and whether they are tackling them.

The critical misunderstanding

Data Platform PMs limit their impact when they treat business stakeholders as a homogeneous group or just respond to inbound requests. Wrong. Every meeting is a consulting engagement: get up to speed fast, find opportunities, solve together. Hear requests but also help them think bigger about data & AI’s role in their department.

The design implication

When you are staffing your data platform team and have multiple PMs, it may be tempting to align them to different platform features. And maybe some of your PMs can do that (at my last company, we had one PM over the AI platform feature set, for example), but when it comes to data products and engagement with the business, it is beneficial to align PMs to business departments for lasting relationships and deep domain knowledge.

You can’t be impactful without knowing them deeply. This isn’t scalable through templates and ticket queues. It requires dedicated relationship investment.

Influence Without Authority

Remember the opening premise: game theory assumes people act rationally given their incentives and constraints. The problem was never that your stakeholders were difficult or irrational. It was that you couldn’t see the games they were playing.

Once you have your incentive map, everything shifts. You stop being surprised when engineers resist governance or analysts bypass your platform. You understand why. And more importantly, you know how to design solutions that work with their incentives instead of against them.

The incentive map gives you that empathy at scale, helping you see patterns across individuals, anticipate resistance before it happens, and find the alignment points that let everyone succeed.

The map is never complete. It evolves as people change roles, structures shift, and strategies pivot. Your job is to keep updating it, stay curious about what motivates the people around you, and be the bridge-builder who sees all sides. That’s how platform PMs turn organizational complexity from an obstacle into an advantage.

MD101 Connect ☎️

If you have any queries about the piece, feel free to connect with the author(s). Or connect with the MD101 team directly at community@moderndata101.com 🧡

Author Connect

Connect with Anna Bergevin on LinkedIn 💬 | Or dive into her work on Substack:

When Data Met Product

A substack about how product management can help drive better outcomes with data - frameworks, case studies, and lessons learned from a product leader in data & AI platforms.

By Anna Bergevin

From MD101 team 🧡

🧡 Join 2000+ on The State of Data Products

Introducing The Catalyst, a special edition of The State of Data Products that compiles insights around the ups and downs in the data and AI arena that fast-tracked change, innovation, and impact.

📘 The Catalyst, first release is our annual wrap-up that compiles the most interesting moments of the year in terms of developments in AI & data products, with expert insights from across the industry.

Some focal topics

• The gap in AI readiness for enterprises

• The shift towards context engineering

• Addressing governance debt with a focus on lineage gaps

• Revisiting fundamentals for better alignment b/w business vision and AI.

Get your copy!

Vlad Radziuk | Account CTO

Vlad Radziuk is a transformation leader and systems thinker working at Nordcloud, an IBM company. He operates at the intersection of modernisation, competence centres, and AI-driven enterprise foundations with a strong focus on ontologies, knowledge graphs, and structured modelling.

With an interdisciplinary background spanning software engineering, cloud-native architecture, and enterprise data strategy, Vlad has built and led innovation units and competence centres of up to 50 professionals. As Account CTO and former Practice Lead, he combines delivery leadership with long-term architectural vision, shaping AI-powered portfolios, establishing internal knowledge ecosystems, and guiding complex transformation programs.

Beyond corporate leadership, Vlad is an active contributor to the data and knowledge graph community. He writes and researches topics such as the business case for ontologies, cross-domain metadata management, digital sovereignty, and the economics of AI integration. His thinking reflects a systemic lens: technology only creates value when aligned with governance, culture, and long-term strategy. We’re thrilled to feature his unique insights on Modern Data 101!

We actively collaborate with data experts to bring the best resources to a 15,000+ strong community of data leaders and practitioners. If you have something to share, reach out!
🫴🏻 Share your ideas and work: community@moderndata101.com

*Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality & relevance. We keep it information-first and do not support any promotions, paid or otherwise.

Let’s Dive In

What might sound like a bunch of buzzwords actually turned out to be a valuable lesson in systems thinking. I heard this statement during a discussion with a group of enterprise architects about how data mesh could support decentralizing their enterprise.

The challenges they described were familiar: bottlenecks, siloed data, and the all-too-common lament, “We’re sitting on mountains of data but extracting little value from it.”

If you’re a data specialist, these issues likely sound familiar - they’re part of our daily reality.

Then, the conversation took an unexpected turn.
One architect emphasized the need for a holistic approach when transforming organizations toward a platform-based setup with federated governance and decentralized ownership. That’s when he dropped that line.

It was a powerful reminder for me: real-world systems are complex and interdependent. Inspired by this insight, I decided to outline a high-level playbook for developing systematic approaches to decentralized data management methodologies like data mesh.

This playbook will probably not be interesting for everyone, but if you’re looking to understand the broader context and make better decisions (or offer better advice) for your internal or external stakeholders, it’s worth stepping back and examining the bigger picture beyond just the data layer.

Reasoning behind the decentralization of organizations

Decentralization, as the core idea of the data mesh methodology, isn’t an end in itself, but a means to drive better data quality and unlock greater value from it. But the question that occupies me is:

How can organizations maximize benefits from decentralization?

Theory suggests that decentralized organizations move faster, motivate employees more effectively, and foster innovation. My own experience confirms this. When an organization’s culture is focused on delivering customer value, having multiple centers of power can lead to remarkable outcomes.

For example, when I led a competence center for application development (operating outside the company’s traditional hierarchy), I was able to create new narratives, uncover new opportunities, and engage employees in ways that a top-down approach simply wouldn’t allow.

Another example that fascinated me recently
Local leaders in a multinational consulting firm, operating in a smaller market, developed a strategy to sustain growth even during years with a slower macroeconomy. They began serving local startups with attractive pricing, which gave them indirect access to venture capital networks and diversified their pipeline.

Many of us have seen similar success stories, but let’s be honest: often, these are driven by the enthusiasm of individuals rather than institutionalized decentralization. What interests me is how organizations can systematically enable decentralization at scale. What platforms, cultures, and organizational designs are needed to make this work?

Decentralization isn’t just about restructuring. It’s about how an organization operates. If we want to decentralize an enterprise, there must be a clear intention behind it. The shift should be leveraged holistically, not just to improve data value, but to transform the way we work.

In terms of platforms, Netflix can serve as a poster child with their data mesh architecture, knowledge graph, and Unified Data Architecture, which support various domains serving different parts of its platform, including films, series, games, live events, and ads. They regularly post articles about their architecture in their engineering blog (Netflix TechBlog), which I highly recommend reading.

Culture-wise, there is one company that made decentralization its source of competitive advantage and became a role model: a Chinese player in the IoT area called Haier.

They developed a philosophy called RenDanHeYi, after which they gave autonomy to their business units (e.g., a unit producing refrigerators, or other kinds of home appliances), which started to serve as “microenterprises”.

Each of these internal companies got “three rights”:

1. the right to make independent decisions,

2. the right to hire talent and

3. the right to distribute profit

So they can quickly make decisions based on market changes, take responsibility for their own gains and losses, and share value in accordance with the agreement.

Instead of a centralized strategy, Haier’s approach is customer-driven: each team identifies real needs and builds solutions, and coordinates through ecosystems (e.g., “microenterprises” can form alliances to achieve bigger goals, and for higher-level strategy)

Haier still has some centralized institutions that fund high-risk, long-term bets. Although I could not find much about the internal technological setup of Haier, I still brought up this example to illustrate how important it is to think systematically when planning decentralization, and not to just focus on technology in general or data in particular.

Still, this piece is primarily for data professionals, so let’s go back to what we know best and deduce other aspects from our shared vision of the bright data future.

What does it actually mean for data to be FAIR?

FAIR data (data that adheres to the principles of findability, accessibility, interoperability, and reusability) is a cornerstone of modern data management. For organizations aiming to implement FAIR, a data product culture (often supported by methodologies like data mesh) and explicit concept modeling (using ontologies) are essential.

This shift moves the focus from tables to business concepts: entities with clear definitions, data instances, and relationships to other concepts.

The FAIR principles provide a clear North Star for how data should be managed to deliver business value. Let’s consider logistics as an example. FAIR data would align with master data (locations, products, partners) and reference data (units of measure, currency codes).

Each data product or data point would have a persistent identifier (PID) or URI, enabling actions like scanning a pallet’s PID to instantly retrieve its contents, origin, and destination from a shared database.

Data would be fully described using knowledge languages or ontologies. For instance, a controlled vocabulary would standardize status terms like “Received,” “Picked,” or “Shipped” (aligned with the SCOR model), while relationships like “Product → StoredIn → WarehouseBin” would be explicitly defined.

I believe FAIR principles offer a powerful lens for examining the holistic decentralization of organizations.

🔖 Related Read(s)

Let’s imagine the shiny world of FAIR data: how would it look from the user's perspective? Users can find the data they need by typing, for example, concept names, synonyms, or related concepts into the search bar of a data catalog.

What’s even more compelling is that data is aggregated by business concepts, allowing users to navigate from concept to concept, much like on Wikipedia, where you might start with “black tea” and end up reading about Einstein’s biography.

In this case, however, users could search for a product, explore its sold instances, track shipping status, and receive recommendations for similar products. This seamless experience is made possible by combining complete domain ownership of data with federated governance.

But what does this mean when we step outside the data perspective?

1. Users traverse the enterprise network across domains. They don’t just access the knowledge space of different domains (e.g., by navigating a knowledge graph through a graphical interface, clicking on PIDs to move between concepts), but also can also consume data or access software functionality, provided they have the necessary permissions.

2. Domains own their assets, meaning they control business logic, access policies, and boundaries. In essence, they decide who can “visit” and interact with their data, much like setting the rules for guests in their own home.

3. Because domains fully own their business logic, the definitions of business concepts and processes can be traced: from knowledge bases and controlled vocabularies, through data, all the way to software.

4. Since domains provide their assets as products, they are responsible for the full lifecycle: maintenance, updates, monitoring, and beyond.

As you can see, decentralization, driven by data mesh and ontology-based data management, has far-reaching implications for how IT operates. The FAIR criteria don’t just guide data management. They also highlight what’s needed for domains to thrive in a decentralized environment.

Domain as a quantum of decentralized organizations

Decentralization often means empowering domains, but what exactly is a domain? It could be a department, business unit, team, platform team, technical capability, or something else entirely. At its core, a domain is any logical boundary with clear ownership, responsibilities, and alignment to business goals.

In the context of data, a domain refers to an organizational unit that not only produces data but also owns it throughout its entire lifecycle, from creation to consumption. However, ownership doesn’t begin with the data itself. It starts earlier, with business logic and language, a principle well-captured by the domain-driven design (DDD) methodology.

Domains have boundaries, which define their unique knowledge models, language, and rules (something we often call business logic). These boundaries extend beyond abstract concepts and include tangible assets that support domain operations, such as software and data. As a result, domains exist across multiple layers:

• Knowledge layer: The conceptual world of definitions, rules, and language.

• Operational layer: The runtime environment of applications, services, and microservices.

• Analytical layer: The world of data products and insights.

To successfully decentralize an organization, all these layers must be addressed, with each playing a critical role in enabling autonomy, clarity, and alignment.

If we abstract away from data or software and focus instead on business capabilities, something interesting emerges: those capabilities remain consistent, whether viewed through an operational or analytical lens.

The language, definitions, and rules (in other words, the knowledge model) persist across all layers. This alignment means that software and data can be managed in surprisingly similar ways. And, fortunately, we already have the tools to do it.

API-first approach and communication over the web

First of all, in an ideal FAIR world, the way we write software is different from how enterprises did that for decades, as we push business logic towards knowledge models (ontologies) and data, away from software. As Martynas Jusevičius told recently:

“Ora Lassila has claimed that all non-RDF data is legacy data. I would go one step further and claim that all software not built natively on and for RDF is legacy software.” (source).

I find this point interesting, but I personally have yet to check what it actually means and to what contexts it applies. On the contrary, domain-driven design (DDD) is pretty widely adopted nowadays and delivers value by many means, with one of them being modeling the code after business concepts, not technical vocabulary.

Another interesting relationship between software and data is that FAIR ultimately requires data products following similar principles that software engineers seem to have widely adopted: modular development (microservices) and communication over HTTP APIs.

And then it also becomes clear why the architect mentioned at the beginning of the article said, “Data mesh without service mesh does not make sense”. Yes, this phrase was cut from the context. The guy worked at a digital-native company before joining a traditional enterprise, so for him, it is easy to judge.

Not all data products are built like web services, for a number of reasons, and a service mesh would not be ultimately necessary to start implementing a data mesh. But still, the connection between the two concepts does exist and should not be ignored.

Data mesh tools focus on data-specific needs (metadata, lineage, access policies), while service mesh focuses on network-level concerns (encryption, load balancing, observability). One lives in the runtime world of microservices, the other in the organizational world of data domains.

Data mesh could rely on service mesh to provide the underlying networking and security fabric that connects domain-owned data products. Without service mesh, data mesh domains could struggle to enforce access controls, manage traffic, and ensure secure communication. On the contrary, service mesh ensures APIs feeding AI are reliable, secure, and observable.

With data being governed using the data mesh principles, with specialized tools and platforms supporting this, and with service mesh enabling services to talk to each other in a secure, observable way, it is easier to draw the technical boundaries between domains.

With that said, it is also important to mention that technology alone is not sufficient, as teams have to adopt the product mindset when it comes to both data and software. Handling them as a product means not only developing them, but also managing them along the whole lifecycle, including operations, monitoring, and so on.

Communication between the domains

It appears natural that different domains would like to communicate with each other: stages in a supply chain depend on each other, production facilities sometimes need to know the results of the R&D activities, and recommendation engine domain needs to pull historical data for past transactions of a customer.

While different parts of organizations would like to communicate on the architecture level, we would like to prevent tight coupling between domains: scenarios when components are highly dependent on each other, making them less flexible and harder to maintain.

The problem of coupling has many aspects: technological (e.g., programming languages), data format (binary, Parquet, CSV), interaction style (async vs sync, REST vs. GraphQL), and many others. Event-driven architectures seem to be a good way to minimize the coupling.

What is actually an event?

As a quantum of event-driven architectures, it is a signal that the state of the system changed. One necessary property of it is that it is immutable. As a consequence of these two properties, in a decentralized architecture, an event can serve as a contract for integrating different systems.

Then producing and consuming domains can agree upfront on a schema of the events. What attributes would a good schema contain? At the very least, an event needs to contain notification about change.

But the more self-contained an event is, the more successful an integration can be. Thus, we would like to see not only the fact that something changed, but also the nature of the changes (what data was modified) and the context.

In the best case, a domain event should be stateless, immutable, modeled as close to the domain as possible, belong to one of the types from the controlled vocabulary/taxonomy, be scoped to a bounded context, and describe the intent behind the change.

Such an event can ensure interoperability and thus serve as a good “currency” in a decentralized architecture. Furthermore, an event with ID can help with deduplication.

• Producing domains are not only responsible for adding context to their events, but also for versioning and ensuring backward compatibility of it (for some time),

• while consuming domains must take ownership of consumer-related logic.

It is considered a good practice when producers know their consumers and do not give out secrets of their life on a wall at the central square - e.g., they can have a policy that only users with specified roles or keys can read the information in the event payload.

As communication between domains happens over networks, I think it makes sense to touch on some networking aspects, too. If domains are responsible for managing their own software, data assets, and their domain events, it follows naturally that they should also take on more responsibility for managing their network and communication streams.

While it’s unrealistic for domains to manage the entire networking infrastructure, they can realistically own their access policies and monitor their networks using modern observability tools.

Service mesh empowers domains to configure their own ingress/egress rules, retries, timeouts, and load balancing. With observability tools, domains can independently track latency, errors, and traffic - without relying on a central team.

Managing policies (e.g., access or firewall rules) as code enables versioning, testing, and deployment via CI/CD pipelines, ideally in a tool-agnostic way.

Boundaries between domains are enforced at two levels.

Thanks to the network isolation (isolated virtual networks and micro-segmentation) on one side and event-driven architecture on the other, we can decouple workloads and services, enabling independent scaling.

Instead of service A polling service B, service B emits events when changes occur, embodying a “choreography” mindset (vs. “orchestration”) and enabling service-level decentralization. Back on the network layer, tools like PrivateLink and enforced encryption for all inter-domain communication ensure secure connectivity between domains.

The difficult part: human behaviour and business architecture

When decentralizing organizations, you need a systematic approach that will touch different aspects of domain operations. In this text, I touched on the main principles, such as:

• decentralized ownership over software, data, and network assets;

• self-service platform operated by centralized platform teams;

• cross-functional collaboration happening on multiple levels, especially in governance;

• computational governance, policies as code, infrastructure as code, versioned resources and workflows;

• API-first design;

• event-driven architecture.

But as compelling as your approach may sound on paper, reality is far more challenging. You’ll face legacy cultures rooted in the industrial era, organizational inertia, and deep-seated biases.

You’ll need to strike the right balance between centralization and decentralization - one that aligns with your organization’s unique context. And you’ll have to prepare people for new roles, like data stewards, which require both technical and cultural shifts.

To be honest, this is more art than science.

Start with a business problem.

I’ve seen many teams eager to improve how their IT organizations operate, confident that they have management buy-in. But too often, their plans were disconnected from real business problems.

Instead of addressing tangible pain points, they focused on crafting the perfect architecture (on paper). That’s not how transformative changes like decentralization work.

You have to start with a specific, agreed-upon business problem - something concrete that management cares about. For example, instead of deploying an entire data mesh infrastructure upfront, you might begin with a data catalog to help the R&D team gain visibility into the company’s data assets.

This approach achieves two critical things: real buy-in (when your architecture solves a specific, urgent problem, stakeholders see its value immediately) and purpose (real problems provide a clear sense of direction: something abstract architecture diagrams often lack). Start small, solve a real problem, and build momentum from there. That’s how meaningful change happens.

Search for early adopters, build alliances.

In any organization, even the quietest ones, there are always people who are eager to learn and try out new things.

I covered the topic of engaging employees in another article of mine: Every employee is a superhero - when they understand the economics of their jobs | LinkedIn Also don’t be afraid of building alliances across the company to ensure the success of your initiative. Again, the goal is to find people who, for some reasons, are interested in your version of the future.

Decentralization is primarily a business endeavour, not technical one.

I described the case of Haier, which reshaped its whole operational model in a decentralized spirit.

Authority allows the domain to make decisions closer to the outside world, to customers. But they also have to have the right capabilities to support their judgment. One cannot work without the other. And what is most important is that only a culture built on trust, on a positive perception of humans, can allow for a true decentralization, resulting in better decisions.

When I was writing this piece, published a text where he distinguishes between delivery-centric vs. value-centric organizations. While the first ones aim at predictable outcomes and controlled workflows, the latter, which remind me of classical Japanese management practices: aim to grow value over time by continuously improving themselves.

But this learning can only happen close to where the problems emerge. Which, in its turn, requires having true ownership over one's own workflows and assets. Thus, by focusing on the technical side, we risk again hoping for a magic tool (or self-service platform) to solve our problems. But as it usually is, we need to do some homework on the organizational level first.

MD101 Connect ☎️

If you have any queries about the piece, feel free to connect with the author(s). Or connect with the MD101 team directly at community@moderndata101.com 🧡

Author Connect 💬

Connect with Vlad on LinkedIn 💬

From MD101 team 🧡

The Data Product Playbook

Here’s your own copy of the Actionable Data Product Playbook. With 3500+ downloads so far and quality feedback, we are thrilled with the response to this 6-week guide we’ve built with industry experts and practitioners. Stay tuned to moderndata101.com for more actionable resources from us!

DOWNLOAD!

Alejandro Aboy | Senior Data & AI Engineer, Writer

Alejandro is a Senior Data & AI Engineer and the voice behind The Pipe & The Line, a technical newsletter where pipelines meet product thinking. With a career that began in marketing and web analytics, he transitioned into modern data engineering, bringing with him a sharp instinct for business context, experimentation, and stakeholder alignment.

At Workpath, Alejandro serves as the main owner of the Data Platform & AI stack. He builds AI agents with RAG architectures (Pgvector on Aurora), designs MCP-based tool ecosystems, and implements production-grade AI observability. His work spans dbt modeling, orchestration with Airflow, Slack-to-Tableau automation, and secure backend design for agentic systems.

Alejandro is deeply engaged in the open data community. He has collaborated with OpenMetadata on agentic data modeling and metadata-first architectures, co-authoring end-to-end implementations that connect lineage, governance, and AI tooling through MCP. His writing is frequently featured by Modern Data 101 and other leading data publications. We’re thrilled to feature his unique insights on Modern Data 101!

We actively collaborate with data experts to bring the best resources to a 20,000+ strong community of data leaders and practitioners. If you have something to share, reach out!
🫴🏻 Share your ideas and work: community@moderndata101.com

*Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality & relevance. We keep it information-first and do not support any promotions, paid or otherwise.

Let’s Dive In

TL;DR

• An AI agent in production meets every formal definition of a data product. It has users, serves analytical data, needs quality monitoring, evolves through versioned schemas, and breaks silently when you ignore it.

• Observability is not logging. It’s product analytics. Every conversation is a user session. Every tool call is a feature interaction. Every retry is a frustration you can measure.

• Undetected errors are invisible churn. Your agent can score perfectly on generic metrics while hallucinating links, suggesting impossible actions, and losing users you’ll never hear from.

• The feedback loop IS the roadmap. Manual annotation sessions, clustered by patterns, connected to tickets. Not dashboards. Decisions.

• Security is a product contract, not a checklist. 4 defense layers, human-in-the-loop, and scoped permissions. A data product nobody trusts is a data product nobody uses.

• Your agent can be its own onboarding tool. Enable it to explain its own features through conversation. Documentation becomes discoverability. Every “how do I...” question is a product signal.

• The teams winning aren’t the ones with the best prompts. They’re the ones treating agent behavior as a continuous product input.

The Subtle Shift that Went Unnoticed

You shipped your agent. Observability is green. Traces are flowing. Users are chatting. But can you answer these?

1. What are users actually trying to do with it?

2. Which tools should you optimize or kill?

3. What feature requests are hiding in conversations you never read?

Not being able to answer any of this means you have an experiment running in production. Most teams assigned agents to ML engineers or product managers. In my opinion, they are partially right and partially wrong.

The infrastructure underneath: RAG pipelines, semantic layers, observability traces, that’s data engineering territory. And the vision on top screams product thinking.

Your Agent Already Is a Data Product

As we can read in this post, we usually describe data products as assets that stakeholders should be able to discover, access, trust, and interact with in a secure fashion.

You can’t carry that vision without observability, clean architectures, and proactive quality monitoring. Let’s see how your AI Agent can fit conceptually in this definition:

AI Observability Is Product Analytics

Most teams connect to Opik or Langfuse and call it done. But that’s just logging. Product analytics is a different ballgame. It answers: What should I build next? Where should I put my focus?

Every conversation is a user session.

When I started reading conversations manually at Workpath, I found users hacking the agent for use cases we never designed. 50% wanted bulk operations we didn’t support, and others were asking for exports that didn’t exist.

None of this showed up in any dashboard. The tools were not raising errors because everything was failing successfully.

Every tool call is a feature interaction.

We had 50 tools. Users called 5 regularly. Some tools were ghosts because the agent decided to never use them. Others had high usage but huge failure rates.

That’s a product prioritization alert: fix the broken popular ones, kill the unused ones, and merge the ones that always get called together.

Every retry is a measurable frustration.

• When a user rephrases the same question three times, that’s friction instead of a “conversation.”

• When they abandon after “I can’t help with that,” that’s churn.

• When they ask for something five times and give up, that’s a feature request screaming right at your face.

Undetected Errors Are Invisible Churn

My agent had perfect hallucination scores. Then I started reading conversations manually and found:

• Agent hallucinating documentation links after fake-searching the docs

• Agent inferring wrong user information and making incorrect tool calls

• Agent suggesting follow-up actions that literally didn’t exist

Generic metrics (e.g., Hallucination, AnswerRelevance, ContextPrecision) are too abstract to catch this. They tell you something is “good” or “bad” without telling you what to fix.

What works

Binary evaluation criteria specific to your use case. “Did the agent call search_knowledge before citing a URL? Does the URL in the response match the tool output?”

Run it with LLM-as-Judge ten times, and you get the same answer. And if you want a bulletproof validation, ask different colleagues in your company. That’s testable.

Those undetected bugs are signals about what your agent promises and can’t deliver. Each one is either a ticket to fix something or a scope clarification for what the agent should and shouldn’t do.

A data product that fails silently is worse than one that fails loudly. At least loud failures get tickets.

Security as a Product Contract

Security isn’t a checklist you bolt on before launch. For a data product, trustworthiness is a foundational property. If users don’t trust the product, they stop using it.

Your agent accepts natural language from anyone. That’s the largest attack surface in your product. I wrote an entire series on building jailbreak-proof AI agents, and the core idea is defense in depth across 4 layers:

• Input validation is a product boundary definition. You’re deciding what your agent will and won’t accept.

• Guardrails are a scope definition. Topical guardrails keep the agent on-task. PII masking isn’t compliance is built-in trust. Users need to know their data isn’t leaking into responses.

• Output control prevents the agent from fabricating data that looks real. Structured outputs are the difference between an agent that invents a documentation link and one that only cites what it actually found.

• Architecture is where engineering meets product. Least privilege, scoped tools, observability. An agent can’t drop your production table if it doesn’t have permissions.

Then there’s human-in-the-loop. I’ve been evaluating Safety Operations to see if my agents comply with asking users before destructive operations. The agent asking, “Are you sure you want to delete this?” is good UX. It also happens to be good security.

Teaching Your Agent What It Can Do

Discoverability is the first data product property we listed. Traditional data products have catalogs. AI agents have something better: they can explain themselves through conversation.

Users don’t read documentation. Savvy ones ask the agent, “What can you do?” and if the answer is vague, you lose them before they even try the feature they came for.

At Workpath, we enabled Zendesk knowledge base articles as a RAG source. When users ask “how do I create a goal?” or “what can you help me with?”, the agent searches those articles and explains features, workflows, and guidance in conversation. The agent seeks to teach users on demand.

Instead of static docs nobody reads, you have a conversational interface that meets users where they are and explains exactly what they need.

Track those patterns and find discoverability signals you can leverage. If the agent can’t explain a feature clearly, either the knowledge base article is bad, or the feature itself is poorly designed.

Every “how do I...” question is a product signal about what’s confusing, what’s undocumented, and what users actually want to use.

The Feedback Loop Becomes the Roadmap

The process I developed after 1000+ manual (yes, 1 by 1) conversation reviews:

• Before each annotation session, prepare. Look for anomalies to pick up outliers, such as latency, too many tool calls, long-lasting conversations, and high token consumptions. Don’t read “some chats” and call it a day.

• During annotation, collect structured signals. Is this a bug? A missing feature? A scope violation? A prompt gap? Each annotation should map to an action type, not just a thumbs-up or thumbs-down.

• After annotation, materialize the roadmap. Connect patterns to JIRA tickets. Update documentation. Cross-reference against the backlog to validate if the most concerning patterns are already being considered.

I started generating a bunch of GitHub PRs just by reading conversations, taking notes, and applying those annotations as Claude Code plans.

None of these came from user interviews. None came from JIRA. They came from reading what users actually did with the product.

The loop: Conversations → Structured Annotation → Clustered Patterns → Tickets → Ship → Measure → Repeat

Final Words

A robust data product encodes business meaning, enforces quality, traces change, and supports continuous evolution. It earns trust through security. And it teaches users what it can do instead of waiting for them to figure it out.

I haven’t seen a moment like this in my career, where users are telling you what to build next through every single conversation.

The teams winning at AI agents are the ones with the best feedback loops, the strongest trust contracts, and agents that know how to explain themselves.

MD101 Support ☎️

If you have any queries about the piece, feel free to connect with the author(s). Or connect with the MD101 team directly at community@moderndata101.com 🧡

Author Connect

Connect with Alejandro Aboy on LinkedIn 💬 | Or find more of his work on Substack:

The Pipe & The Line

Hands-on guides, tools, and experiments to sharpen your Data & AI Engineering skills from someone who learned it all in the wild.

By Alejandro Aboy

Antonio Sánchez Gómez | Data Management & Governance Specialist

Antonio is a Data Management & Governance Specialist at Roche, where he focuses on building scalable Master Data Management (MDM) and Data Governance operating models. His work centres on establishing clear data ownership, decision rights, and reusable governance structures that transform master data from technical assets into trusted business capabilities.

Antonio designs and implements governance frameworks that create single sources of truth across critical domains such as customer, product, and pricing. He advocates for governance as an operational discipline, embedding stewardship workflows, quality thresholds, and accountability directly into enterprise execution. His perspective positions Data Governance as a revenue enabler that accelerates product launches, strengthens regulatory confidence, and removes friction from core business processes.

With a background spanning data engineering, analytics, and big data across organisations such as the International Olympic Committee and financial services programs at BBVA via Minsait and Deloitte, Antonio combines deep technical expertise with enterprise governance design. We’re thrilled to feature his unique insights on Modern Data 101!

We actively collaborate with data experts to bring the best resources to a 15,000+ strong community of data leaders and practitioners. If you have something to share, reach out!
🫴🏻 Share your ideas and work: community@moderndata101.com

*Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality & relevance. We keep it information-first and do not support any promotions, paid or otherwise.

Let’s Dive In

Data Ownership is one of the most frequently defined roles in Data Governance, and one of the least effectively exercised.

Despite mature frameworks, operating models, and role catalogs, many organizations still struggle to translate “ownership” into real authority, accountability, and business consequence. In most enterprises, Data Owners exist formally but operate with limited decision power.

Such as accepting data-related risk, approving quality thresholds, authorizing usage, or resolving conflicts between value and control. Ownership becomes visible on paper but is absent where governance must operate.

This is not primarily a problem of role definition, but a problem of decision rights.

Governance fails because authority is not explicitly designed into the operating model. When decision rights are unclear, decisions drift toward operational teams, escalation becomes informal or avoided, and accountability dissolves across organizational boundaries. The result is governance that appears complete but is structurally incapable of governing.

This article reframes Data Ownership as a decision-centric role. Ownership is not a collection of activities, but a formally empowered authority embedded in the enterprise decision system. Only when decision rights are explicit, bounded, and institutionally supported does Data Governance function as an operational capability rather than an aspirational framework.

🔖 Related Read

Why Data Ownership Remains Ineffective

The persistent failure of Data Ownership is not due to a lack of documentation, but due to a systemic misunderstanding of what ownership entails in complex organizations.

In many cases, Data Owners are appointed based on hierarchy or domain proximity, without a corresponding transfer of decision authority. Governance programs emphasize responsibilities (reviews, forums, endorsements) while remaining silent on what Data Owners are actually entitled to decide.

Matrixed organizational structures amplify this issue.
Decisions around data quality, access, or structural change cut across domains and systems. When decision rights are not explicit, decisions are either pushed downward, endlessly deferred through consensus, or resolved informally by whoever has operational leverage.

The Data Owner is bypassed not out of negligence, but because their authority was never operationalized.

Until Data Ownership is designed around decision rights rather than activities, it will remain symbolic rather than functional.

From Role Definitions to Decision Authority

Governance operates through decisions, not tasks. Outcomes are shaped by who can approve thresholds, accept risk, authorize usage, or prioritize remediation, not by who attends meetings or produces reports.

Many organizations assign Data Owners extensive accountability without granting authority to enforce standards, block usage, or resolve conflicts. This structural asymmetry (responsibility without decision rights) erodes both the credibility of the role and the effectiveness of governance.

A decision-rights perspective forces a different question:

What decisions is the Data Owner empowered to make, under which conditions, and with what consequences?

Decision rights must be explicit, bounded, and supported by formal escalation paths. Authority that exists only through personal influence collapses under pressure. Moving from role descriptions to decision design is not a semantic refinement; it is a structural redesign from governance as documentation to governance as an executable system.

What Ownership Means in an Enterprise Context

Enterprise ownership combines authority, accountability, and risk acceptance.

Data Ownership is inseparable from risk. Decisions on quality thresholds, usage permissions, access models, or structural change all imply acceptance of residual financial, operational, regulatory, or reputational risk.

Data Owners do not own only data artifacts, but also the business outcomes enabled by data. This is why ownership cannot sit exclusively with technical or operational roles. Authority must rest with those accountable for business performance.

Ownership must also be institutionally recognized. Informal authority collapses under competing priorities. Decision rights must be documented, escalation paths defined, and accountability mechanisms enforced. Without this, ownership defaults into coordination rather than governance.

Core Decision Domains of the Data Owner

In practice, Data Owner decision rights consistently fall into a limited set of domains:

• Data Quality: Defining acceptable quality thresholds, approving exceptions, and accepting residual risk, not monitoring metrics or fixing defects.

• Data Usage: Authorizing or restricting use based on value, sensitivity, contractual obligations, or regulatory exposure.

• Data Access: approving access models and exceptions. Provisioning is operational while authorization is a business decision.

• Structural & Lifecycle Change: approving changes to models, hierarchies, or reference data, balancing short-term delivery against long-term consistency.

• Risk Acceptance: making explicit decisions about tolerated exposure and escalation.

These domains define the practical scope of Data Ownership. Without explicit authority in these areas, ownership cannot function operationally.

Data Owner vs Data Steward: The Decision Boundary

The most common governance failure is not role confusion, but misallocated decision authority.

Data Stewards are frequently asked to approve thresholds, accept deviations, or close issues: decisions that carry business risk they are not accountable for. Conversely, some organizations reduce Data Owners to passive signatories, disengaged from real decisions.

When governance models describe who does what but not who decides what, decisions inevitably migrate toward execution. This is a structural inevitability.

Effective governance requires a clear separation: Data Owners decide and Data Stewards execute, monitor, and escalate.

🔖 Related Read

The Cost of Undefined Decision Rights

Undefined decision rights produce systemic consequences:

• Organizationally: Accountability erodes, escalation becomes performative, and ownership is perceived as optional.

• Operationally: Decisions stall, exceptions are inconsistent, and teams default to risk aversion or workarounds.

• From a business perspective, organizations oscillate between over-control and under-control, slowing delivery or exposing themselves to unmanaged risk.

Most critically, risk ownership becomes invisible.

Designing Decision Rights into the Operating Model

Making decision rights explicit requires intentional design, not more documentation.

Effective models start from recurring decision points (quality thresholds, usage authorization, access exceptions, structural change) and explicitly assign authority, scope, and escalation criteria. Decision rights must be embedded into workflows, not layered on top.

Escalation thresholds must be predefined to prevent operational roles from absorbing risk by default. Authority must be institutionally reinforced through leadership mandate, alignment with risk management, and performance accountability.

Finally, decisions must be visible and traceable. Governance matures when risk acceptance is deliberate rather than accidental.

How Decision Rights Evolve with Maturity

Decision rights evolve as governance matures:

• Early stages: Corrective, visible, and frequent intervention to establish legitimacy.

• Stabilization: Focus shifts to approving standards and tolerances, enabling operational autonomy within boundaries.

• Advanced maturity: Decisions become strategic and anticipatory, enabling new capabilities and managing trade-offs between innovation and control.

Misalignment occurs when mature expectations are applied to immature structures—or when decision models remain overly prescriptive after governance stabilizes.

Signals of Effective Data Ownership

Functional Data Ownership is visible through behavior:

• Clear, timely decisions at escalation points

• Explicit, traceable risk acceptance

• Consistent decision patterns across similar cases

• Confident execution by operational roles

• Reduced decision latency and predictable outcomes

Where these signals are absent, ownership exists structurally but not operationally.

Final Note

Governance works when decisions have owners. Data Governance succeeds or fails on the clarity of decision ownership.

Frameworks provide structure, but they do not govern. Governance becomes real only when authority to decide is explicitly assigned, supported, and exercised.

Until organizations design Data Ownership around decision rights, governance will remain an intention rather than an operational capability. Governance works when decisions have owners.

MD101 Support ☎️

If you have any queries about the piece, feel free to connect with the author(s). Or connect with the MD101 team directly at community@moderndata101.com 🧡

Author Connect

Connect with Antonio Sánchez Gómez on LinkedIn 💬

From MD101 team 🧡

The Modern Data Masterclass: Learn from 5 Masters in Data Products, Agentic Ecosystems, and Data Adoption!

With our latest community milestone, we opened up The Modern Data Masterclass for all to tune in and find countless insights from top data experts in the field. We are extremely appreciative of the time and effort they’ve dedicatedly shared with us to make this happen for the data community.

Master Modern Data, One Session at a Time!

Explore classes

Jessica Talisman

Knowledge Infrastructure Strategist, Founder & Advocate of The Ontology Pipeline Framework

Jessica Talisman is a Semantic Engineer, Information Architect, and knowledge infrastructure strategist dedicated to building information systems. With more than 25 years of experience in enterprise architecture, e-commerce content systems, digital libraries, and knowledge management, Jessica specialises in transforming fragmented information into coherent, machine-readable knowledge systems.

She is the founder of the Ontology Pipeline, a structured framework for building semantic knowledge infrastructures from first principles. The Ontology Pipeline emphasises progressive context-building: moving from controlled vocabularies to taxonomies, thesauri, ontologies, and ultimately fully realised knowledge graphs.

Professionally, Jessica has led semantic architecture initiatives at organisations including Adobe, where she architected an RDF-based knowledge graph to support Adobe’s Digital Experience ecosystem, and Amazon, where she worked in information architecture and taxonomy. She also founded Contextually LLC, providing consulting and coaching services in ontology modelling, NLP integration, and knowledge governance.

An educator and thought leader, Jessica publishes regularly on her Substack newsletter, Intentional Arrangement, where her writing frequently explores the relationship between semantic systems and AI. We’re thrilled to feature her unique insights on Modern Data 101!

We actively collaborate with data experts to bring the best resources to a 15,000+ strong community of data leaders and practitioners. If you have something to share, reach out!
🫴🏻 Share your ideas and work: community@moderndata101.com

*Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality & relevance. We keep it information-first and do not support any promotions, paid or otherwise.

Let’s Dive In

The Opportunity Before Us

Ontologists and semantic engineers face a persistent professional challenge to demonstrate that our work produces tangible value. We build controlled vocabularies, develop metadata standards, structure taxonomies, and design ontologies. Yet too often, these artifacts remain siloed rather than operational infrastructure.

Business stakeholders often struggle to connect our deliverables to measurable outcomes due to a lack of understanding about the mechanics of semantic systems.

Entity resolution changes this equation. When semantic infrastructure directly enables entity reconciliation (determining who is who and who is related to whom across data sources), our work becomes measurably operational.

Match rates improve. False positives decrease. Disambiguation succeeds where string matching fails. The thesaurus we build from entity resolution results becomes part of the critical infrastructure, which makes the use case work.

The emergence of tools like Senzing’s sz-semantics library and specifically, the Senzing thesaurus creates an inflection point for our profession. These tools leverage the Ontology Pipeline: the systematic methodology I have articulated for constructing semantic knowledge management systems to shape logic through executable code.

For taxonomists, ontologists and semantic engineers, this means our frameworks now have concrete tooling, with a methodology and deployable artifacts. While elegantly simple in its shape and code base, the Senzing thesaurus demonstrates how essential our expertise becomes as neurosymbolic AI solutions based upon Semantic Web principles emerge in the public repositories of enterprises.

Why Entity Resolution Needs Us

Entity resolution engines excel at resolving multiple records from different data sources based on evidence, for which Senzing uses entity-centric learning. The system architecture supports the detection of names, addresses, phone numbers, identifiers, and other features that constitute entity profiles.

They handle messy, inconsistent data with remarkable sophistication, accommodating variations in spelling, formatting, and completeness that plague enterprise data integration. What they do not do or what they cannot do without semantic infrastructure is interpret what matches mean within a knowledge domain.

When entity resolution operates without our work, results remain technically useful but semantically isolated. The output captures what matches, but not why it matters. Two records merge because phone numbers align,

• but what does that merged entity represent in the domain?

• What relationships does it participate in?

• What constraints govern its behavior?

Semantic engineering and ontologies can model and architect domains to answer these questions, thereby adding rich context and meaning to the underlying data.

On the way to building out supporting, semantic infrastructure, the Ontology Pipeline becomes essential. The pipeline’s iterative stages (controlled vocabulary, metadata standards, taxonomy, thesaurus, ontology, knowledge graph) prepare data and information for semantic interpretation.

Each stage builds towards maturing semantic systems to support the meaningful classification of resolved entities. Discovered relationships can be mapped to edges, where the ambiguity inherent in real-world data resolves through principled knowledge organization.

Entity resolution produces candidate matches. But as ontologists and semantic engineers know all too well, the semantic infrastructure determines what those matches mean.

The Ontology Pipeline as Professional Framework

The Ontology Pipeline provides the technical methodology and professional framework for scoping, estimating, and delivering semantic engineering projects. When business stakeholders ask what ontology work costs or how long it takes, the pipeline offers concrete answers. Each stage has defined inputs, outputs, and success criteria. Each stage maps to identifiable skills and time investments.

To build resilient systems that fit into the expectations of organizations, the pipeline’s defined stages translate to tangible project deliverables. Controlled vocabulary construction produces deduplicated, defined term lists with clear scope and cardinality.

Metadata standards development delivers schema documentation with element definitions, value constraints, and validation rules, informing semantic engineering products of how data traverses complex data ecosystems.

Taxonomy construction yields hierarchical concept schemes in SKOS format with defined depth and breadth.

The thesaurus extends the taxonomy by adding associative relationships, synonyms, and cross-references that handle domain ambiguity. Ontology development introduces classes, properties, attributes, relations and logical constraints. Knowledge graph deployment represents an architecture that synthesizes all stages into a dynamic and queryable knowledge infrastructure.

Each deliverable can stand on its own, and be easily purposed for specific needs and use cases. Each can be reviewed, validated, and versioned. Each contributes measurably to the semantic system’s capabilities.

When entity resolution connects to this pipeline, stakeholders and the business can understand exactly how ontology work enables data integration outcomes.

The controlled vocabulary ensures consistent class, entity and attribute naming conventions and definitions. Metadata standards define schema constructs and logic while also revealing the mapping logic for entities, classes and attributes. The taxonomy classifies entities, lending inference by way of a hierarchy that shapes parent-child relations.

The thesaurus adds more contextual understanding to the taxonomy by encoding alternative labels, hidden labels, relationships and documentation properties. The ontology defines semantic interpretations while providing logical reasoning. The knowledge graph can include all parts of the Ontology Pipeline, making it all queryable.

This is how we justify investment and build buy-in. Trust is formed when we move beyond abstract arguments and demonstrate how each pipeline stage directly enables entity resolution quality and contextual understanding for machines and humans.

The Thesaurus Stage, The Sweet Spot

Of all the Ontology Pipeline stages, the thesaurus is the low-fidelity powerhouse that can provide the most value to entity resolution. A thesaurus handles ambiguity by forming associative relationships between terms, going beyond the parent-child relationships within a hierarchy.

It captures synonyms, near-synonyms, variant spellings, acronyms, cross-domain equivalences and linked data enrichment. This is what entity resolution needs but cannot generate on its own.

When Senzing determines that records from different sources describe the same person, it performs a matching function. When the semantic thesaurus assigns that resolved entity a URI, classifies it within a taxonomy, and links alternative names as skos:altLabel relationships, it performs a knowledge organization function. The thesaurus transforms raw matches into semantically meaningful entities that participate in broader information ecosystems.

The sz-semantics library implements this transformation directly. Its Thesaurus class takes entity resolution results and a domain taxonomy as inputs, then generates RDF fragments that extend the taxonomy with resolved instance data. Alternative names discovered through matching become thesaurus relationships. Entity types map to taxonomy concepts. The output serializes a SKOS-compliant RDF formatted corpus, and applies these values to entity resolution outcomes.

For semantic engineers, this means our thesaurus expertise directly determines entity resolution quality.

The more precise our taxonomy classification, the more meaningful the resolved entity profiles. The more complete our associative relationships, the more discoverable the connections between entities. The thesaurus can provide the necessary support to imbue data with rich context and meaning.

Human-in-the-Loop: Expertise Becomes Essential

Automated entity resolution and semantic transformation cannot replace human judgment. The Strwythura project, which implements the complete Ontology Pipeline workflow, explicitly includes a human-in-the-loop curation stage.

As noted in The Strwythura project’s GitHub readme, the human-in-the-loop (HITL) is an essential step, whereby

“humans curate definitions for entities extracted from the unstructured content...There is a ‘cascade’ effect, where the quality of entity definitions poses a limiting factor for how well the system can perform relation resolution downstream.”

Senzing formalizes HITL because it is instrumental to the success of semantically supported entity resolution, highlighting the critical importance of semantic quality and expert oversight.

What’s involved with curation exactly?
Entity resolution produces candidate matches with varying confidence levels. The semantic thesaurus assigns these entities to taxonomic categories based on available evidence.

Some assignments will be correct while others will require refinement.

• Which synonyms should be promoted to preferred labels?

• Which alternative names represent genuine aliases versus data entry errors?

• Which taxonomic classifications capture the entity’s primary identity versus incidental attributes?

These are knowledge organization questions that semantic engineers answer through thesaurus construction and enrichment. Our training in classification theory, facet analysis, and vocabulary management prepares us to make exactly these judgments.

Our understanding of domain context enables us to recognize when automated assignments miss nuance. Our familiarity with SKOS and ontology engineering allows us to encode corrections in standard, machine readable, interoperable formats.

HITL curation prioritizes data quality and semantic integrity according to thesaurus and ontology standards. Entity resolution produces the matching framework while the thesaurus supports the capture of rich semantics and context, to ultimately enable reliability and accuracy.

While automated transformation produces candidates and matches, the thesaurus produces validated knowledge. As ontologists and semantic engineers, the Senzing thesaurus was engineered by ontologists with design decisions and judgment that defines our expertise.

NIEM: A Case Study in Pipeline Application

The National Information Exchange Model (NIEM) presents a compelling case study for ontologists applying the pipeline to entity resolution. NIEM already institutionalizes the first two pipeline stages: it provides controlled vocabularies for government information exchange and metadata standards that define how those vocabularies apply to data elements. For semantic engineers working in the public sector, NIEM represents existing infrastructure ready to be extended, with the objective.

Entity resolution challenges are at the core of NIEM use cases. Law enforcement agencies share suspect information across jurisdictions. Emergency management coordinates resources across organizational boundaries.

Immigration systems reconcile traveler identities from multiple sources. Each scenario involves determining who is who when data quality varies, terminology differs, and completeness fluctuates.

Through the Senzing RDF thesaurus, we map SKOS concept schemes to NIEM data components, organizing elements, types and attributes through matching frameworks based upon vocabulary concept definitions and semantic relationships.

We build thesauri that capture the alternative terminologies different agencies use for the same concepts. We develop ontologies that constrain how NIEM-compliant data participates in entity resolution.

The NIEM-Senzing mapping exercise is semantic engineering work. Aligning NIEM element definitions with Senzing feature types (NAME_FULL to nc:PersonName, ADDR_LINE1 to nc:LocationStreet, organizational identifiers to NIEM organization components) constitutes a crosswalk, leveraging thesaurus concepts and properties to support resilient mapping and transformer logic.

This mapping becomes a metadata standard, enabling entity resolution results to flow directly into NIEM-compliant information exchanges.

For ontologists working with the NIEM ecosystem, entity resolution integration represents an opportunity to demonstrate that semantic engineering enables interoperability outcomes.

Federated entity resolution across agencies becomes possible because semantic infrastructure ensures consistent definitions and interpretation. The thesaurus captures variant agency terminologies while the ontology enforces exchange constraints; the knowledge graph provides queryable access. Our work makes it work.

The Value of Ontology Work

Semantic knowledge systems are rapidly emerging as essential for achieving highly performant AI applications and services. Large language models require clean, well-structured, semantically enriched data to provide accurate and reliable results.

Knowledge graphs underpin retrieval-augmented generation systems that ground LLM outputs in verifiable facts. Entity resolution ensures that the entities referenced in AI responses correspond to accurate, real-world subjects.

The Ontology Pipeline positions semantic engineers at the center of this infrastructure. By following the pipeline’s rigorous, iterative process (cleaning, preparation, reconciliation, modeling, testing, enrichment, enforcement), we ensure the data quality that AI systems require.

Entity resolution validates that the knowledge we organize corresponds to reality. The semantic thesaurus ensures that the vocabulary AI systems use is consistent and unambiguous.

Tools like sz-semantics and Strwythura demonstrate that our frameworks support complex ecosystems, with or without AI. The Thesaurus class generates SKOS from entity resolution results. The pipeline workflow produces entity-resolved knowledge graphs ready for GraphRAG integration. Interactive visualization makes our work accessible to stakeholders who need to understand what the semantic system contains.

This architecture demonstrates the value of semantics in supporting complex knowledge infrastructures, and moreover, a professional opportunity. Semantic engineers and ontologists are essential to making semantic architectures such as Senzing’s a reality.

Entity resolution needs semantic engineers because matches without meaning are just data. The Ontology Pipeline provides the methodology; these tools provide the implementation; our expertise provides the quality, context and meaning. Ontologists make the knowledge work.

Getting Started With Designing for Infrastructure

For semantic engineers ready to apply the Ontology Pipeline to entity resolution, the work involves the familiar in addition to new tools, methodologies and applications.

Audit your existing semantic assets

What controlled vocabularies, metadata standards, taxonomies, and ontologies already exist in your organization? These represent completed pipeline stages. Assess their quality, coverage, and the SKOS compliance of controlled vocabularies. Identify gaps that entity resolution integration would expose.

Learn the Senzing Vocabulary and NIEM Vocabularies

The Senzing Entity Specification documents available entities and features: NAME_FIRST, NAME_LAST, ADDR_LINE1, PHONE, and dozens of others. Familiarize yourself with the NIEM standard, paying attention to the object-oriented logic of NIEM. NIEM organizes its standard by types, elements and attributes.

For further guidance, the NIEM Naming and Design Rules (NDR) specifies guidelines for creating data models, namespaces, schemas, and messages, in addition to guidelines for naming conventions, syntax, semantics and RDF crosswalks.

Understanding this vocabulary enables you to design crosswalks between your domain’s metadata elements and entity resolution features. This semantic engineering work is extremely effective at supporting interoperability while preserving rich semantics, entity reconciliation and mapping logic.

Structure Your Domain Thesaurus in SKOS

If your existing taxonomies use spreadsheets or proprietary formats, migrate them to SKOS. After your taxonomy is SKOS compliant, add alternative labels, hidden labels and definitions (if absent) to create your SKOS thesaurus. The SKOS taxonomy or thesaurus produces the TTL file that sz-semantics requires—and positions your taxonomy or thesaurus for broader semantic web interoperability.

Plan for Thesaurus Extension

Entity resolution will discover synonyms, variant spellings and acronyms that your current vocabulary does not include. These newly discovered terms can be integrated into a domain taxonomy or thesaurus and represented as alternative labels.

To support this work, establish governance processes for incorporating these discoveries. Define criteria for promoting matches to canonical semantic definitions. This is where your expertise in vocabulary management will be handy, and essential to operations.

Engage With the Tooling

The sz-semantics and Strwythura repositories are open source and documented. Work through the tutorials. Understand how the thesaurus class transforms entity resolution results. Identify where your semantic assets would integrate into the workflow. Prototype with sample data before proposing production integration.

Build the Business Case

Document how each pipeline stage contributes to entity resolution quality. Estimate time and cost for each deliverable. Define metrics that demonstrate semantic infrastructure value, such as match rate improvements, false positive reductions, and disambiguation successes. Speak the language of measurable outcomes.

The Moment

The Ontology Pipeline offers the framework to support semantic engineering, from documentation practices to infrastructure discipline. Entity resolution provides the use case that demonstrates this transformation concretely.

Controlled vocabularies enable consistent entities, classes relations, properties and attributes. Taxonomies classify resolved entities meaningfully, as concept classes. The thesaurus disambiguates and documents alternative names and documentation properties.

Ontologies encode the logical constraints and semantic interpretations. Ontologists and semantic engineers are critical to making this work happen, to build robust and resilient semantic knowledge infrastructures.

Tools like sz-semantics implement the pipeline stages as executable code. The thesaurus we build becomes the infrastructure that entity resolution depends upon. The SKOS concepts we define become the URIs that identify resolved entities globally. The associative relationships we curate become the semantic edges that make knowledge graphs queryable.

Entity resolution benefits from the semantic infrastructures we build, to transform matches into knowledge.

As new frameworks and tools emerge, there are exciting opportunities to participate in new innovations that support semantic interoperability, integrations between systems and semantic data transformations.

Operationalizing RDF and semantic web standards to negotiate data mapping logic, enable transformations, and support entity resolution tasks is a fantastic opportunity for ontologists and semantic engineers that will increase demands for our skillsets.

For ontologists seeking to demonstrate the practical value of their discipline, entity reconciliation provides a compelling application domain, and sz-semantics provides the tools to realize it.

If you have any queries about the piece, feel free to connect with the author(s). Or feel free to connect with the MD101 team directly at community@moderndata101.com 🧡

Author Connect

Connect with Jessica Talisman on LinkedIn 💬 | Or dive into her work on Substack:

Intentional Arrangement

I am an Information Architect and Semantic Engineer. I have been organizing and building digital knowledge ecosystems since 1997. Intentional Arrangement is the name of my book Intentional Arrangement, to be published in 2026!

By Jessica Talisman

From MD101 team 🧡

🌎 Global Modern Data Report 2026

The Modern Data Report 2026 is a first-principles examination of why AI adoption stalls inside otherwise data-rich enterprises. Grounded in direct signals from practitioners and leaders, it exposes the structural gaps between data availability and decision activation.

With hundreds of datapoints from 500+ data leaders and experts from across 64 countries, this report reframes AI readiness away from models and tooling, and toward the conditions required and/or desired for reliable action.

Get your copy

References

DerwenAI. (2024-2026). Strwythura: Construct an entity-resolved knowledge graph from structured data sources and unstructured content sources. GitHub. https://github.com/DerwenAI/strwythura

Senzing. (2025). sz-semantics: Transform JSON output from Senzing SDK for use with graph technologies, semantics, and downstream LLM integration. GitHub. https://github.com/senzing-garage/sz-semantics

Senzing. (2025). Senzing Entity Specification. https://senzing.com/docs/entity_specification/

Talisman, J. (2025). The Ontology Pipeline: A Semantic Knowledge Management Framework. Intentional Arrangement. https://jessicatalisman.substack.com/p/the-ontology-pipeline

Talisman, J. (2025). Knowledge Graph Lite, Part III: The Mighty Thesaurus, SKOS in Action. Intentional Arrangement.

Tesfaye, L., & Jonas, J. (2025). Expert Analysis: How Do I Get Started with a Semantic Layer? Enterprise Knowledge.

W3C. (2009). SKOS Simple Knowledge Organization System Reference. https://www.w3.org/TR/skos-reference/

ANSI/NISO Z39.19-2005 (R2010). Guidelines for the Construction, Format, and Management of Monolingual Controlled Vocabularies.

ISO 25964-1. Information and documentation — Thesauri and interoperability with other vocabularies.

Vu Trinh | Data Engineer

Vu Trinh is a Data Engineer and educator known for breaking down complex data engineering concepts into simple, structured explanations. Currently working at Amanotes, a music game company, Vu combines hands-on industry experience with a deep passion for learning and teaching.

He is the creator of VuTrinh, a fast-growing newsletter read by 17,000+ data engineers worldwide. Through long-form articles and curated weekly resources, Vu explores everything from Apache Spark internals and lakehouse table formats to BigQuery pricing models and the evolution of data engineering itself. His writing is practical, research-driven, and grounded in first principles.

Based in Ho Chi Minh City, Vu’s mission is simple: cut through shallow content and help engineers truly understand how data systems work under the hood. As he often says, his mom started reading his articles to support her son, and now she can design data architectures and write ETL scripts! We’re thrilled to feature his unique insights on Modern Data 101!

We actively collaborate with data experts to bring the best resources to a 15,000+ strong community of data leaders and practitioners. If you have something to share, reach out!
🫴🏻 Share your ideas and work: community@moderndata101.com

*Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality & relevance. We keep it information-first and do not support any promotions, paid or otherwise.

Let’s Dive In

Apache Spark now cares about the ultra-low latency.

When it comes to stream processing, everybody usually brings Spark and Flink to the table. Flink is famous for its true stream processing: every record is processed continuously, nearly as soon as it arrives. Spark Structured Streaming, on the other hand, relies on its batch-processing engine to process a continuous stream in microbatches.

The latency is indeed higher than Flink’s; however, Spark has the advantage of its user base: many users already use Spark, and if they want to switch from batch to stream processing, they only need to tweak the code a bit.

Thus, you commonly see advice like this: “Only if you actually need millisecond latency, invest in Flink. Spark is still a good choice for second-tier latency use cases, given its simplicity and familiarity (Flink has a steep learning curve and requires significant resources to manage).

However, in the latest version, Spark introduced a new mode for Spark Structured Streaming that promises to overcome the limitations of the original approach and achieve millisecond latency, similar to Flink. The cofounder of Databricks said that this new mode could offer the latency competitive to Flink:

Source

In this article, we will dive into Spark Structured Streaming’s real-time mode to see how this feature could change the “latency game” in stream processing. We will first briefly review Spark and Spark Structured Streaming to understand their limitations in low-latency scenarios better; only then will we see how the new streaming feature addresses them.

This article only has a brief overview of Spark and Spark Structured Streaming. For the deep dives, you can check these two articles:

• If you’re learning Apache Spark, this article is for you

• Everything you need to know about Spark Structured Streaming

Spark Cluster

Spark is a distributed processing engine. Thus, there must be a cluster of nodes.

Every Spark application is associated with a Spark cluster, which comprises a driver and a set of executors. All of these nodes are JVM processes that consume resources from a physical cluster, which consists of physical machines and is managed by the Cluster Manager process.

When you submit your application, the cluster manager first initiates the driver on one of the physical machines (if you run it in cluster mode). The driver contacts the cluster manager to launch the executor.

Anatomy

Spark has hierarchical processing units:

• Job: A job represents a series of transformations applied to data. It encompasses the entire workflow from start to finish.

• Stage: A stage is a job segment executed without data shuffling. A job is split into different stages when a transformation requires shuffling across partitions.

  • DAG: In Spark, RDD dependencies are used to build a Directed Acyclic Graph (DAG) of stages for a Spark job. The DAG ensures that stages are scheduled in topological order.

• Task: A task is the smallest unit of execution within Spark. Each stage is divided into multiple tasks that process data in parallel across different partitions.

Scheduling

To provide the executor with tasks to run, the driver must first plan execution based on the user-defined logic. Then the driver starts scheduling the execution according to the plan.

This scheduling process involves some components (which also live inside the driver):

• The DAGScheduler for stage-oriented scheduling

• The TaskScheduler for task-oriented scheduling

• The SchedulerBackend interacts with the cluster manager and provides resources to the TaskScheduler.

The DAGScheduler schedules stages according to the DAG’s topological order. Each stage is submitted once all its upstream dependencies are completed. If stages don’t have dependencies on each other, they can be run in parallel.

Remember this: Stages are scheduled in topological order.

The DAGScheduler creates fully independent, unprocessed tasks for that stage. Then, the DAGScheduler sends them to the TaskScheduler. This component is responsible for scheduling those tasks on available executors. It requests resources from the SchedulerBackend to schedule tasks.

The Micro-batch Structured Streaming

Overview

After the revision on how Spark works, it is straightforward to understand how Spark Structured Streaming operates.

Its core design principle is to treat a continuous stream as a bounded dataset. When we start a streaming application in Spark Structured Streaming, we create a long-running Spark application (a Spark cluster).

The Driver process remains active at all times, managing the entire streaming query lifecycle. Each stream will have a trigger. This trigger determines when Spark checks for new data. When the trigger fires, the Spark engine does the following:

• It queries the source (e.g., asks Kafka “what are the latest offsets?”).

• It identifies the new data that has arrived since the last batch (e.g., Kafka offsets 1001 to 5000).

• This chunk of new data is a micro-batch. Internally, Spark treats it as a static DataFrame.

• And for every micro-batch, things will happen pretty much like how batch processing works in Spark: the driver, parse the logic, create/optimize plan, and schedule the stages in order.

The Delay

Although the current Spark Structured Streaming approach can leverage Spark’s robust batch engine for stream processing, it has some disadvantages, especially regarding the low latency performance:

• First, the stages are scheduled in order, which means the stage can only be scheduled when all its upstream stages are completed.

• For every batch, the driver needs to coordinate the communication between the nodes, parse, plan, schedule the stages/tasks, and commit the output. These overheads are included in each record-processing latency.

That’s why some online sources suggest that if you prefer super-low latency, you should go with Flink rather than Spark.

The Real-time Mode

However, the Spark 4.1 release (Dec 16, 2025) released a very cool feature: Structured Streaming Real-Time Mode (RTM).

Databricks introduced this feature in 2025 August and later contributed it to the community version. (Source)

The RTM targets millisecond latency for use cases, while the original micro-batch is suitable for seconds-latency use cases. RTM achieves this through these clever changes (not exhaustive, but I believe these are the crucial changes)

Consuming Fresh Data

In micro batch mode, the source reading process occurs only when a trigger is present. In real-time mode, the engine keeps reading fresh, available data from the source in longer-running batches (default: 5 minutes).

Stages Scheduled Concurrently

Unlike in (micro)batch processing, where stages are scheduled sequentially, RTM schedules all stages concurrently, eliminating wait time for upstream stages before scheduling a new one.

Longer batches and concurrent scheduling of stages mean lower coordination and planning overhead.

Streaming Shuffle

If you work with Spark, you must know about data shuffle. There are two transformations in Spark: narrow and wide dependencies.

The first is where each partition in the child RDD has a limited number of dependencies on partitions in the parent RDD. These partitions may depend on a single parent (e.g., the map operator) or on a specific subset of parent partitions known beforehand (e.g., with coalesce).

This means that operations like map and filter do not require data shuffling. RDD operations with narrow dependencies are pipelined into a single set of tasks per stage (e.g., map and filter can be baked into a single stage)

The latter requires data to be partitioned in a specific way, where a single partition of a parent RDD contributes to multiple partitions of the child RDD. This typically occurs with operations such as groupByKey, reduceByKey, or join, which involve data shuffling. Consequently, wide dependencies result in stage boundaries in Spark’s execution plan.

Thus, when a shuffle occurs, data must be exchanged between executors.

When thinking about Spark, people often assume that every operation must run in memory. That’s not 100% true. Although Spark tries to keep data processing in memory and spills only when needed, data shuffling occurs on the executor’s local disk.

The “map” tasks will write shuffle data to the executor’s local disk. They first write data to a memory buffer. When the buffer is full, they spill data to temporary disk files and merge them into the final shuffle files.

The “reduce” tasks communicate with the shuffle service, pull data from the target executor’s local disk into the memory buffer, and then start the processing.

However, exchanging data like this might not meet the RTM latency requirement.

Thus, RTM allows data to be passed between stages as soon as it is produced using an in-memory streaming shuffle.

Note: I only know that streaming shuffle allows for faster data exchange between stages in memory. I can’t find the details of the streaming shuffle implementation anywhere at the moment. Hope more resources are released in the future.

Checkpointing

In micro-batch mode, checkpointing is done straightforwardly. During the planning phase, the driver already knows which data ranges (the batch) the cluster must process, and the start and end of that batch are written to the checkpoint location before processing.

However, in real-time mode, things become more challenging because the driver doesn’t know the batch’s end boundary beforehand.

Thus, the strategy must change a bit.

Checkpointing occurs only at the end of batch processing, rather than at the beginning.

If the batches have a wider data range, RTM checkpointing occurs less frequently, reducing overhead but increasing the read workload because the engine must read a broader range of data from the source when failures occur.

If the batch is smaller, checkpointing occurs more often, which harms the low-latency. When failure happens, the recovery process might be faster as it reads a narrower range of data from the source.

My Thoughts

The key thing about this feature is that the user only needs to make a minor change to the micro-batch code.

The user needs to change the trigger to RealTimeTrigger. Source

This convenience allows users to seamlessly switch to the new mode if the latency performance of the original Spark Structured Streaming does not satisfy them anymore

For me, this is a game-changer.

In the past, the only way to achieve that lower-latency performance was to use Flink. However, managing, operating, and developing Flink requires significant effort (as much as you use Spark).

Now with RTM, users can consider this new mode instead of Flink. If users can leverage RTM, they don’t need more resources to manage Flink and still stay in the Spark ecosystem, which is a big win for any organization that has been investing in Spark (e.g., engineers who are only familiar with Spark)

Although the real adoption of RTM is still being observed, and Flink won’t be replaced anytime soon (its true stream processing capabilities, robust watermarking, and state management still make it a strong stream processing engine), RTM will change how companies choose a stream processing engine.

Final Note

I opened this article out of curiosity about Spark Structured Streaming’s real-time mode, which promises to deliver competitive low-latency performance compared to Flink. Then we have a brief review of how Spark works as a cluster, its processing units, and the scheduling process. Next, we revisit Spark Structured Streaming, which processes continuous data in small batches, each handled in the same way as Spark batch processing.

From there, we understand the factors that impact the latencies of the micro-batch processing approach. Finally, we see how the real-time mode solves these problems by consuming fresh data, scheduling stages concurrently, using a streaming shuffle, and employing a different checkpointing strategy.

Thank you for reading this far. See you in my next articles.

MD101 Support ☎️

If you have any queries about the piece, feel free to connect with the author(s). Or feel free to connect with the MD101 team directly at community@moderndata101.com 🧡

Author Connect

Connect with Vu Trinh on LinkedIn 💬 | Or dive into his work on Substack:

VuTrinh.

My mom reads my articles to support her son. Now, she can design a data architecture and write ETL scripts.

By Vu Trinh

From MD101 team 🧡

🌎 Global Modern Data Report 2026

The Modern Data Report 2026 is a first-principles examination of why AI adoption stalls inside otherwise data-rich enterprises. Grounded in direct signals from practitioners and leaders, it exposes the structural gaps between data availability and decision activation.

With hundreds of datapoints from 500+ data leaders and experts from across 64 countries, this report reframes AI readiness away from models and tooling, and toward the conditions required and/or desired for reliable action.

Get your copy

References

[1] Databricks, Real-Time Mode Technical Deep Dive: How We Built Sub-300 Millisecond Streaming Into Apache Spark™ (2025)

[2] Databricks, Real-time mode in Structured Streaming Documentation

John Wernfeldt | Managing Director: Data, Analytics & AI

John Wernfeldt works with CDOs and senior data leaders who are under pressure to “be AI-ready” while still struggling with trust, ownership, and decision clarity in their data. His focus is practical data governance and foundations, the kind that reduce rework, end metric debates, and give executives confidence to act.

Based in Stockholm, John is Managing Director at Northridge Analytics, where he helps organisations turn data and analytics into tangible business value, and President of DAMA Sweden. His background spans data strategy, governance, architecture, and analytics leadership roles at Gartner, Capgemini Invent, and KPMG.

Known for cutting through hype, John emphasises clear decision rights, fit-for-purpose quality, and governance embedded into delivery. His work helps organisations move from fragmented, fragile data to trusted foundations that scale with analytics and AI. We’re thrilled to feature his unique insights on Modern Data 101!

We actively collaborate with data experts to bring the best resources to a 15,000+ strong community of data leaders and practitioners. If you have something to share, reach out!
🫴🏻 Share your ideas and work: community@moderndata101.com

*Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality & relevance. We keep it information-first and do not support any promotions, paid or otherwise.

Let’s Dive In

I’ve never seen a “data quality issue” that wasn’t, at its core, an ownership problem. I’ve seen this play out more times than I can count.

A number looks off, someone flags it, everyone agrees it’s “important.” And then the questions begin. Sales uses it in forecasts, finance uses it in reporting, marketing uses it in KPIs, analytics builds dashboards on top, and IT moves the data and gets asked to fix it.

When a metric breaks, the same explanations always show up

When the number is questioned, the conversation goes in predictable directions.

A recent survey on the state of data and AI in enterprises conclusively suggests how the same metric gets corrupted as it changes hands. Because no one in particular owns the health of the metric. Many of us touch the same metric, and we end up seeing different versions.

93% of respondents say they encounter conflicting metrics. Nearly half do not fully trust their own data, and 68% explicitly state that it is not trustworthy enough for AI

- Findings from the Modern Data Report 2026

There could be many reasons behind this experience. For example, the definition is unclear, the source system changed, there was manual input, and the data quality dropped. All of these things are often true, but they’re also mostly irrelevant.

Because the only question that actually matters never gets answered:

When that person doesn’t exist, the discussion goes nowhere.

The cycle that follows is always the same

When ownership is missing, organisations fall into a loop that looks like this:

The metric breaks, and a hero steps in to fix it. People argue about whose fault it is, and a temporary workaround is put in place. Then everyone moves on. Only until next month.

This is why so many teams believe they have reporting problems, BI problems, or tooling problems. What they truly have is a governance gap.

Why “data quality” becomes the wrong label

Calling this a data quality issue is convenient. It allows the organisation to:

• Treat the problem as technical

• Push responsibility toward IT or analytics

• Focus on fixing symptoms instead of decisions

Data quality becomes a proxy for unresolved ownership. But quality cannot exist in a vacuum. It only improves when someone is accountable for the outcome. Until then, every fix is temporary.

The problem is decision authority, not data. At its core, a metric is not a technical artifact. It’s a decision artifact. It exists so someone can make a decision, explain it, and stand behind it.

If no one has the authority to:

• Define the metric

• Approve changes

• Reject misuse

• Be accountable when it’s wrong

Then the metric will always be fragile, no matter how clean the pipeline looks.

A simple model to fix decision authority for data

This situation doesn’t require a new tool, a maturity program, or a framework rollout. It requires making ownership explicit and enforceable.

Here’s a proven practical model to improve or establish decision authority.

1. Name a single accountable owner

Not a team, a forum, or “the business”. One named person who is accountable for:

• The meaning of the metric

• The calculation logic

• When it is allowed to change

This person may delegate work, but cannot delegate accountability.

2. Separate ownership from contribution

Many functions will contribute to a metric. That’s fine. But contribution is not ownership. We need to make it explicit:

• Who provides inputs

• Who builds pipelines

• Who validates data

• Who approves changes

Only one person owns the outcome.

3. Define change rules upfront

Most metric chaos comes from silent change. Every metric needs clear answers to:

• What counts as a breaking change

• Who approves changes

• How changes are communicated

• When historical values are recalculated

If this isn’t defined, change will happen anyway. Just informally.

4. Tie data quality rules to ownership

Data quality rules should never exist in isolation. For each critical rule:

• Name an owner

• Define acceptable thresholds

• Agree on consequences when it breaks

Quality improves when breaking the rule creates friction instead of tickets.

5. Enforce, don’t escalate

If every issue escalates to a steering group, ownership is fake. Owners need:

• The mandate to say no

• The ability to block changes

• Support from leadership when decisions upset someone

Without enforcement, governance has no authority.

The Sovereign Data Ownership Canvas

What changes when this model is enforced

When ownership is explicit and enforced, metrics stop drifting, quality issues surface earlier, blame games decrease, fixes stick, and trust increases. Data is almost never perfect, but these changes are kicked off because accountability is clear.

The uncomfortable part

Most organisations avoid implementing such a model because it’s uncomfortable. It forces decisions instead of alignment, accountability instead of consensus, and clear trade-offs instead of vague agreements.

But avoiding discomfort is exactly what creates recurring data quality issues.

Final Note

Adapted from concepts shared by the author | Curated by Modern Data 101

If a metric matters, someone must be accountable for it. Until that happens, data quality initiatives won’t stick, BI tools won’t help, and AI will amplify the problem instead of solving it.

Until we have fixed the root of the problem and assigned an owner, nothing downstream really matters.

I’m collecting practical tools like this in my Substack, starting with a Metric Specification Template I actually use. If that’s useful, it lives here: link to metric specification template

MD101 Support ☎️

If you have any queries about the piece, feel free to connect with the author(s). Or feel free to connect with the MD101 team directly at community@moderndata101.com 🧡

Author Connect

Connect with John Wernfeldt on LinkedIn 💬 | Or dive into his work on Substack:

Data Governance Field Library

Practical tools, templates, and resources for data governance practitioners and leaders

By John Wernfeldt

From MD101 team 🧡

🌎 Global Modern Data Report 2026

The Modern Data Report 2026 is a first-principles examination of why AI adoption stalls inside otherwise data-rich enterprises. Grounded in direct signals from practitioners and leaders, it exposes the structural gaps between data availability and decision activation.

With hundreds of datapoints from 500+ data leaders and experts from across 64 countries, this report reframes AI readiness away from models and tooling, and toward the conditions required and/or desired for reliable action.

Get your copy

Olawale Oyeneye | Senior Data Governance Advisor

Olawale Oyeneye is an accomplished Data Governance leader with extensive experience designing and implementing enterprise-wide data governance frameworks across complex, regulated organisations. As Senior Data Governance Adviser at The Pensions Regulator, the UK regulator of work-based pension schemes, he plays a pivotal role in strengthening governance controls across cloud platforms, applications, and systems.

A Certified Data Management Professional (CDMP) and ISO 27001:2022 ISMS-certified practitioner, Olawale is widely recognised for his expertise and has successfully delivered governance programmes spanning multiple business units and regions, defining critical data elements, building business glossaries, establishing data ownership, and championing the adoption of governance tooling. We’re thrilled to feature his unique insights on Modern Data 101!

We actively collaborate with data experts to bring the best resources to a 15,000+ strong community of data leaders and practitioners. If you have something to share, reach out!
🫴🏻 Share your ideas and work: community@moderndata101.com

*Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality & relevance. We keep it information-first and do not support any promotions, paid or otherwise.

Let’s dive in

Most organisations believe they have a data governance problem. But what they actually have is a practice problem. Governance, in its formal sense, is rarely missing.

Policies, frameworks, maturity models, all exist. What is missing is their presence in the ordinary moments where data is created, interpreted, and acted upon.

It would shape which metric is trusted in a meeting, which dataset is reused instead of recreated, and which issue is escalated before it becomes visible in an audit.

Instead, governance tends to live outside the work itself: captured in documents and decks, consulted only after something has already gone wrong.

But governance cannot be defined by its existence on paper. It can only be defined by its effect on behaviour.

Anything that does not change how people act, decide, or take responsibility in their daily work does not govern anything at all. It may inform, it may advise, but it does not govern.

This reframes how governance artefacts should be understood. Policies and frameworks are not truths to be adopted by default. They are just hypotheses. Each one makes an implicit claim: if this is followed, outcomes will improve.

That claim is only validated when the policy survives contact with real workflows, real incentives, and real time pressure. Until then, governance is not something an organisation has, but something it assumes.

Reduce Governance to Irreducible Components

Governance fails when it is treated as something large, centralised, and abstract. When it becomes an artefact to be owned by a function, a compliance exercise to be satisfied periodically, or a retrospective control applied after damage has already been done, it loses contact with reality. In those forms, governance is always too far away, too slow, and too detached from the moments that actually matter.

To understand why, the problem has to be reduced to its smallest truths. Governance does not begin with committees or tools.

• It begins with behaviour: what people actually do with data when no one is watching.

• It continues with language: whether people can explain what the data means, where it comes from, and when it should or should not be trusted.

• It depends on authority: who is allowed to fix issues, make decisions, and say no.

• And it is sustained by feedback: what gets measured, noticed, and reinforced over time.

Strip away everything else, and these are the irreducible components. Change behaviour without changing language, and confusion follows. Define roles without authority, and governance stalls. Measure artefacts instead of actions, and adoption becomes performative.

Governance is not a system that can be installed or rolled out. Systems are static. Governance is dynamic: a pattern of repeated, local decisions, made under pressure, with imperfect information. When those decisions converge toward trust, governance exists. When they don’t, no framework, however elegant, can compensate.

🔖 Related Read

Rebuild Governance in the Correct Order

From simple, observable actions to complex outcomes.

Start with Behaviour over Policy

Governance begins at the moment data is created, touched, reused, or questioned. If governance is not present there, it is already too late. The aim is not to ask people to remember rules, but to shape environments where the right behaviour is the default.

When tagging, validation, and checks happen naturally in the flow of work, governance stops feeling like an extra task. It becomes muscle memory.

Translate Policy into Human Language

Most governance policies crumble because they are written to satisfy auditors instead of guiding users. If the people using data cannot explain what a policy means for their work in a short conversation, the policy has no operational value.

Governance only becomes real when it is expressed in everyday language. Language that connects abstract principles to concrete actions. Understanding must come before compliance, and comprehension must come before trust. Without that sequence, policies remain technically correct and practically irrelevant.

Empower the Roles Closest to the Data

Frameworks do not fix issues. People do. Governance collapses when authority is centralised far away from the data, or when responsibility is assigned without the power to act.

Trust only scales when the people closest to the data are trusted to make decisions, resolve issues, and escalate when needed. Governance moves from theory to practice when decision rights move downward, while visibility and accountability remain upward.

🔖 Related Read

Measure Behaviour Over Artefacts

What gets measured defines what gets taken seriously. Counting how many assets are catalogued or how many policies are signed measures effort instead of impact. Real governance shows up in behaviour: which datasets are reused, how quickly issues are resolved, and whether people recommend governed data to others.

Voluntary trust is the strongest signal of success. When people choose governed data without being told to, governance is no longer enforced since it is already working.

Leadership as Living Proof

Governance becomes culture only when it is modelled. Culture is not shaped by what leaders approve in principle, but by what they default to in practice. When leaders ask for data quality in reviews, use governed datasets in decisions, and reward teams for doing the right thing with data, governance gains legitimacy. When they don’t, no amount of policy can compensate.

🔖 Related Read

Review Until Nothing Is Missing

Most conversations about governance end with structures like new councils, new checkpoints, or new artefacts. That instinct is understandable, but it misses the point.

Governance succeeds when it no longer needs to announce itself, when it no longer requires a meeting, a reminder, or a mandate.

When governance vanishes as an agenda item and reappears as instinct, something fundamental has shifted. People no longer ask whether they should follow governance, they assume it as part of how work gets done.

Decisions are made with an implicit awareness of quality, lineage, ownership, and impact, without anyone having to invoke the framework by name.

Governance that must be enforced has already failed.

The future of data governance will not be written into longer documents or stricter controls. It will be enacted subtly, repeatedly, and everywhere data moves. The goal is to get to a day when governance becomes reflex rather than a requirement.

MD101 Support ☎️

If you have any queries about the piece, feel free to connect with the author(s). Or feel free to connect with the MD101 team directly at community@moderndata101.com 🧡

Author Connect

Connect with Olawale Oyeneye on LinkedIn 💬

From MD101 team 🧡

🌎 Global Modern Data Report 2026

The Modern Data Report 2026 is a first-principles examination of why AI adoption stalls inside otherwise data-rich enterprises. Grounded in direct signals from practitioners and leaders, it exposes the structural gaps between data availability and decision activation.

With hundreds of datapoints, this report reframes AI readiness away from models and tooling, and toward the conditions required and/or desired for reliable action.

Get your copy

🧡 The Data Product Playbook

Here’s your own copy of the Actionable Data Product Playbook. With 3500+ downloads so far and quality feedback, we are thrilled with the response to this 6-week guide we’ve built with industry experts and practitioners. Stay tuned to moderndata101.com for more actionable resources from us!

DOWNLOAD!

Heather Hedden | Taxonomy Consultant

Heather Hedden is a globally respected taxonomy consultant and educator whose work has shaped how organisations structure, govern, and retrieve knowledge at scale. With more than two decades of hands-on experience, she brings rare depth across taxonomy design, information architecture, metadata, and ontologies, translating conceptual clarity into practical systems that materially improve search, discoverability, and decision-making. As the author of the seminal book The Accidental Taxonomist, now in its third edition, Heather has educated and influenced generations of practitioners, establishing herself as one of the most authoritative voices in the field.

Heather advises enterprises, platforms, and institutions on building sustainable, standards-aligned taxonomy programs, while also training professionals through courses, workshops, and conferences worldwide. Her contributions extend beyond consulting into stewardship of the discipline itself, including active involvement in ISO standards for thesauri and long-standing editorial leadership with BARTOC. Known for her intellectual rigour, generosity in teaching, and unwavering commitment to quality, Heather continues to elevate the practice of knowledge organisation for the modern information economy. We’re thrilled to feature her unique insights on Modern Data 101!

We actively collaborate with data experts to bring the best resources to a 15,000+ strong community of data leaders and practitioners. If you have something to share, reach out!
🫴🏻 Share your ideas and work: community@moderndata101.com

*Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality & relevance. We keep it information-first and do not support any promotions, paid or otherwise.

Let’s Dive In

There is a growing interest in “semantics” among data experts, such as in developing semantic data models and semantic layers. Information professionals, meanwhile, have always been involved with semantics (whether or not they called it that) in developing classification systems, indexes, taxonomies, tagging systems, and metadata schemas.

Semantics is about meaning, rather than just strings of text. The same text string can have different meanings in different contexts, and the same concept can be represented by different text strings for different users, use cases, and systems.

Thus, control, governance, policies, and standards are needed around semantics, especially when it comes to data and information sharing and interoperability across systems, repositories, and users.

Semantic Web Standards

The ultimate information-sharing platform is the World Wide Web, whose core standards have been developed primarily by the World Wide Web Consortium (W3C) since its founding in 1994.

This has included standards for basic hypertext technologies, including HTML (HTML5 in its final version), CSS, XML, and Internationalized Resource Identifiers (IRI). As web standards became adopted both behind the firewall within enterprises and for web-bases (SaaS), Semantic Web standards became important for the technology and data administration of organizations, and not just the World Wide Web.

The extension of the web technologies to support greater semantics in the hypertext links is known as the Semantic Web, or Web 3.0, which is also relevant to organizations. To this end, the W3C developed additional recommendations, with the foundational standard or data model being RDF (Resource Description Framework).

The W3C defines RDF as a “standard model for data interchange on the Web.” The principles of RDF are that the web of data is based on interlinked statements of subject-predicate-object “triples” and all resources have URIs (or IRIs).

In addition to the web and web applications, there is now a class of graph database management systems based on RDF, which are commonly known as “triple store” or RDF graph databases.

Other standards based on RDF are, in order of publication, from 2004 through 2017: RDFS, GRDDL, SKOS, POWDER, PROV, RIF, SAWSDL, RDB2RDF, OWL, SPARQL, RDFa, JSON-LD, and SHACL.

This may seem like an alphabet soup, but some of these standards are more prevalent and important to understand than others. RDF, RDFa (embedded structured data in HTML), SPARQL (the standard query language for RDF), and JSON-LD (JSON-based serialization for Linked Data, used by developers) are widely used in systems today.

OWL, RDFS, SKOS, and SHACL are especially used by semantic experts, knowledge graph experts, knowledge and data engineers, ontologists, and taxonomists. The other standards have more limited adoption and use.

Leveraging the Combined Power of Standards

Controlled vocabularies modeled in SKOS can be integrated with ontologies modeled in RFDS and described more fully in the OWL language, and all can be expressed in a computer-readable form of JSON-LD (among other serializations) and then queried using SPARQL.

Knowledge Organization Systems

The Semantic Web standard SKOS stands for Simple Knowledge Organization System. The concept of a “knowledge organization system” comes from the field of library and information science.

According to the International Society for Knowledge Organization (ISKO)’s Knowledge Organization Encyclopedia, knowledge organization systems are

…functional items designed for organizing knowledge and information, and making their management and retrieval easier…they are basically made of terms/concepts. | (Source)

Broadly defined, knowledge organization systems include term lists with definitions and/or other data (including dictionaries, glossaries, gazetteers, and terminologies) and structured arrangements of terms or concepts (including taxonomies, subject heading schemes, classification schemes, thesauri, and ontologies).

Although ontologies are considered knowledge organization systems, they are better known as knowledge representation systems. SKOS supports these various kinds of knowledge organization systems with the exception of ontologies, since there are other standards from the W3C for ontologies to support their greater complexity.

SKOS does not support all knowledge organization systems, and we usually refer to what is supported by SKOS as “vocabularies.”

Most software today for creating and managing knowledge organization systems, especially ontologies, thesauri, and taxonomies, is based on semantic web standards.

SKOS (Simple Knowledge Organization System)

SKOS was developed by the W3C starting in 2006 and published in 2009, with the objective of supporting the publishing, sharing, and linking of knowledge organization systems on the Web and thus contributing to the Semantic Web.

SKOS was adopted relatively quickly by both open source and commercial vendors of software for managing thesauri and taxonomies.

Of course, most software is also Web-based (SaaS), so following Web protocols, such as using URIs, is practical even for managing proprietary data.

Although SKOS is often referred to as a “standard,” more precisely, it is a common data model recommended by the W3C for making knowledge organization systems both human-readable and machine-readable, in addition to being interoperable.

An example of machine-readable presentation for the concept with the preferred label of Animals in English, the alternative label of Creatures in English, and the preferred label of Animaux in French would be:

ex:animals rdf:type skos:Concept;
skos:prefLabel “animals”@en;
skos:prefLabel “animaux”@fr;
skos:altLabel “creatures”@en.

The designations before the colons are the namespaces, which for our taxonomy example taxonomy is ex. The namespace of skos is used to prefix all SKOS elements.

An example of expressing a hierarchical relationship between two concepts as a subject-predicate-object statement is:

ex:animals skos:narrower ex:mammals.

SKOS takes a concept-approach to modeling vocabularies, and the concepts have various types of properties, including both types of relationships to other concepts and attribute data, such as various types of labels and notes.

If you are familiar with ontologies, you will recognize the features of classes, properties, and attributes. SKOS actually is an upper-level ontology, which is then used for modeling domain-specific vocabularies. However, if you are not familiar with ontologies, understanding that the SKOS model is an ontology is not necessary, though, to use SKOS.

It’s also not necessary to be an expert on SKOS when creating and managing taxonomies or thesauri, now that nearly all taxonomy/thesaurus management software is based on SKOS, and what is presented in the user interface is more user-friendly, such as “Preferred Label” instead of prefLabel.

SKOS Elements

The following table presents all of the elements of SKOS, grouped by type:

Concept Scheme

The “concept scheme” is the highest level of organization for SKOS vocabularies. A concept scheme is a collection of concepts. Concepts are arranged in hierarchical relationships of broader/narrower to each other. The relationship between the highest level of a taxonomy hierarchy and the concept scheme is instead that of “top concept”.

Collection

There is an additional, optional way to organize concepts, and that is by designating a “collection” to which selected concepts belong based on some other designated criteria that is not simply based on hierarchy. Use of collections allows you to create one master taxonomy, in which subsets of it are used in different implementations.

Concepts & Labels

Concepts are things or ideas that have labels. Each concept has one designated “preferred label” (per natural language, if it is in a multilingual vocabulary) and any number of “alternative labels,” to designate synonyms or variants.

A “hidden label” is an optional kind of alternative label that is designated not for display in certain circumstances or user interfaces. “Notation” is a field designated for use in classification schemes that have a numeric or alphanumeric code corresponding to the concept’s classification.

Notes

Concepts may have various notes, such as a “scope note” or a “change note”, definitions, and even examples. A scope note is a standard field in traditional thesauri, and it is used to indicate any restrictions on the meaning and use of a concept for the vocabulary’s context. Notes are used as needed, and their inclusion is optional. The use of definitions tends to be determined by an organization’s editorial policy.

Semantic Relationships

The relationships available between concepts are (1) hierarchical, involving the inverse pair of “broader” and “narrower,” and (2) associative, involving the reciprocal relationship of “related.”

The related concept relationship is standard in thesauri and optional in taxonomies, although most taxonomies make little, if any, use of “related.” The hierarchical relationships of broader and narrower are inherently transitive through multiple levels of hierarchy, thus supporting the inferencing of hierarchy.

There also exists the inverse pair of “broader transitive” and “narrower transitive,” which in practice are not used directly in the construction of a taxonomy, but rather are derived via reasoning from the direct relationships and thus may be used in statements of SPARQL queries to query the taxonomy.

Mapping Relations

Finally, SKOS includes a set of mapping relations, which did not exist previously in thesauri. These relationships are for use across different concept schemes, or different taxonomies or vocabularies, as they may be called.

Use of SKOS mapping relations allows you to link, map, or connect taxonomies together on a concept-by-concept level. These different taxonomies may be on your local server, or they may be external but linked on the web with URLs.

Controlled Vocabularies

Most knowledge organization systems managed in SKOS are controlled vocabularies. Controlled vocabularies are knowledge organization systems that have a particular focus on managing concepts or terms and their labels and that are used especially for tagging or indexing content to support findability and retrieval.

Users select a concept or category from a controlled vocabulary to retrieve the content that has been tagged with it. Controlled vocabularies exclude certain knowledge organization system types, such as dictionaries or glossaries, where the focus is on defining words.

Controlled vocabularies also exclude classification systems, which focus on mutually exclusive class hierarchies, whose labels are of lesser importance, since often a system of alphanumeric codes is used.

The following are common types of controlled vocabularies, starting with the most semantically rich:

Thesauri

An information thesaurus (in contrast to a type of dictionary of synonyms) is a structured, semantically controlled vocabulary comprising terms and both hierarchical (broader-narrower) and associative (related).

Notes, especially scope notes, tend to be used in thesauri. Another basic feature of thesauri is the use of synonyms or variants to point to the preferred term. Traditional, pre-SKOS thesauri have been based on “terms” rather than concepts and labels, but a thesaurus built on the SKOS model, with concepts and labels, functions just the same for users.

Among controlled vocabularies, thesauri are distinguished by having formal standards that provide recommendations for style, format, and construction of thesauri. These include:

• The international standard ISO 25964. Thesauri and interoperability with other vocabularies

• The American standard ANSI/NISO Z39.19 Guidelines for the Construction, Format, and Management of Monolingual Controlled Vocabularies

Taxonomies

Taxonomies are more informal and varied than thesauri. They are characterized by a hierarchical or category-based organization of terms/concepts. Unlike most thesauri, taxonomies have a limited number of hierarchies to which all concepts belong.

The top-level grouping of taxonomy concepts may be by “facet” or aspect type (e.g., topic, location, organization, person type), which may be implemented in a faceted search/browse user interface. Facets may or may not have further hierarchy within them. When designing a faceted taxonomy, each facet usually corresponds to a SKOS concept scheme.

The name “taxonomy” may suggest classification, but a taxonomy is not a classification system that has mutually exclusive classes. The hierarchy of a taxonomy does not imply a classification or hierarchy of things but rather provides a logical way to organize and find the concepts that are used in tagging and retrieval.

The ISO and ANSI/NISO thesaurus standards are also relevant to taxonomies to the extent that they are applicable. Taxonomies are widely used in business and industry to support information access, both for internal/enterprise and customer/public-facing use.

Name Authorities

Name authorities are lists of named entities or proper nouns, such as names of people, companies, organizations, brand-name products, laws, publications, etc. The lists are not hierarchical, unlike a taxonomy, but alternative labels or variants (synonyms) are typical, and are supported by SKOS.

The designation of “name authority” comes from library science, and the best-known example is the Library of Congress Name Authorities. With SKOS, each named authority would be a concept scheme.

Concepts in a name authority typically have additional descriptive attributes, such as birthdate for people and headquarters location for companies. SKOS does not support the creation of such additional customized attributes, although RDFS does, so combining SKOS with RDFS (for ontologies) fully supports the details of named entities.

Term Lists

Term lists are any controlled lists of terms, typically to pick from values in various metadata properties. Sometimes the designation of “controlled vocabulary” is used synonymously with this simplest kind of vocabulary.

In SKOS, each term list is a Concept Scheme. As with name authorities, there is no hierarchy in simple term lists, so all terms are top concepts of their concept scheme. Term lists also lack alternative labels. Although it may seem as if SKOS is not needed for simple term lists, in practice, term lists often exist alongside other metadata properties, each as a SKOS concept scheme.

For example, controlled vocabularies for tagging and retrieving documents may have facets managed as concept schemes for each of the following: topics (hierarchical taxonomy or thesaurus), named people (authority file), publication language (term list), document type (term list), and source (term list).

Final Note

Controlled vocabularies, such as taxonomies and thesauri, existed before SKOS as a standard was published, but the vocabularies were created and managed in application silos in different applications, and they could not easily be shared, exported, and imported.

With vocabularies based on the SKOS model, they can easily be shared, linked, and reused. This has led to a greater alignment of vocabularies and the information to which they are tagged, with the resulting benefits of improved enterprise-wide information intelligence and knowledge sharing. This, in turn, has further promoted the benefits and uses of controlled vocabularies, which has led to even wider adoption and use.

MD101 Support ☎️

If you have any queries about the piece, feel free to connect with the author(s). Or feel free to connect with the MD101 team directly at community@moderndata101.com 🧡

Author Connect

Connect with Heather Hedden on LinkedIn 💬

Or dive into her work on these platforms:

• Almost 200+ on the Accidental Taxonomist Blog

• The Accidental Taxonomist, 3rd Edition

• Hedden Information Management

From MD101 team 🧡

🧡 The Data Product Playbook

Here’s your own copy of the Actionable Data Product Playbook. With 3500+ downloads so far and quality feedback, we are thrilled with the response to this 6-week guide we’ve built with industry experts and practitioners. Stay tuned to moderndata101.com for more actionable resources from us!

DOWNLOAD!

Winfried Etzel | Data Governance & Data Management Specialist & Podcast host at MetaDAMA

Winfried is a senior data governance and data management practitioner, educator, and community builder with deep experience across energy, consulting, and public-sector organizations. He currently serves as Senior Specialist in Data Management at Equinor. Formerly, he served as Manager of Data Governance & Architecture at Aker BP and did extensive consulting work with Bouvet ASA.

As a prominent contributor to the data community, he is the host of the MetaDAMA podcast and has been recognised as one of the Nordic 100 in Data, Analytics & AI and nominated multiple times as Data & AI Influencer of the Year.

Winfried’s work is grounded in a strongly socio-technical view of data governance. Especially as organizations embed automation, AI, and increasingly autonomous systems into their core operations. He complements this foundation with deep professional certification, including CDMP Master, Microsoft Azure AI Fundamentals, and Generative AI credentials. We’re thrilled to feature his unique insights on Modern Data 101!

We actively collaborate with data experts to bring the best resources to a 15,000+ strong community of data leaders and practitioners. If you have something to share, reach out!
🫴🏻 Share your ideas and work: community@moderndata101.com

*Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality & relevance. We keep it information-first and do not support any promotions, paid or otherwise.

Let’s dive in

Rethinking Data Governance in Times of AI

Data governance is often experienced as a burden: slow, bureaucratic, and disconnected from how organizations actually work. Yet, at the same time, it is increasingly recognized as indispensable, especially as organizations embed automation, AI, and autonomous agents into their core operations.

This tension stems from a fundamental mismatch between how data governance is conceptualized and the reality of modern socio-technical systems.

Traditional Old School data governance is rooted in an essence-based worldview:

…the assumption that data has inherent meaning and value that can be defined, standardized, and controlled. A worldview that was rational in centralized/aggregated, slow-moving systems.

In distributed, AI-enabled environments, this assumption no longer holds.

Yes, we still need to respect regulatory realities, where data does carry non-negotiable obligations, where safety-critical data does have inherent risk implications, or where some meaning is constrained and not freely enacted. But a realization that is slowly unfolding is that context and purpose are limiting these non-negotiables.

Meaning is not stable, but is enacted with intentionality. Yet, New School data governance has not solved the inherent issues of data governance either.

With a focus on automation and design patterns, we finally found the right methodologies to scale data governance efforts. Yet we are still not doing “the right thing the right way,” because…

…scale without intentionality automates existing assumptions.

This is not because we got it “wrong”, but because data governance has evolved, from compliance-focus to value-focus. Now it needs to realize contextual obsolescence and move towards becoming a motor for AI advancement.

Drawing on the socio-technical foundations of data governance, the challenges introduced by autonomous AI systems, the existential reframing of value and meaning, and the operational shift toward embedded and computational data governance, we are required to propose a different lens:

Data governance is not about controlling data or AI.

It is about intentionally structuring socio-technical systems so that meaning, direction, and accountability can be sustained over time. Don’t think of this as a philosophical indulgence. This existential reframing is a practical necessity for governing AI at scale.

Data Governance as a Socio-Technical Foundation

At its core, data governance is not a technical discipline, but largely a social system.

Data governance exists to ensure that collective action remains purposeful, accountable, and legitimate within complex socio-technical systems.

Its role is to help organizations

• negotiate requirements and meaning,

• set direction for data, and

• maintain accountability in relation to data and data-driven decisions.

Simply put, a socio-technical system is any system where humans and machines work together to achieve goals. And this kind of system, with all its interactions and connectedness between humans and machines, has become the reality in most organizations.

These are not different kinds of governance, they are different contexts in which the same governance logic must operate. The purpose remains constant while only the methods evolve.

This framing is critical, because it exposes the core problem facing data governance today: it has often been treated as an operational or compliance function, rather than as a foundational element of how organizations coordinate action and responsibility.

The Limits of Essence-Based Data Governance

Both Old School and New School data governance are built on an implicit belief: that data has an essence. This belief manifests in familiar practices:

• defining canonical data models,

• enforcing enterprise-wide definitions,

• measuring quality against fixed rules,

• assuming alignment once standards are met.

Such practices aim to stabilize meaning by fixing it in advance. They presume that if data is correct, meaning will follow, and that interpretation can therefore be ignored. While there is value in these practices, they treat data and standards for data as constant, while in reality, they are temporary agreements, not eternal truths.

This approach is still viable in centralized, slow-moving systems, bounded context, or facing regulatory deterministic rules. But it breaks down in distributed data landscapes, where data is created, interpreted, and reused across domains with different goals, constraints, and assumptions.

Here, meaning must be negotiated constantly. This brings us to the introduction of a shift in the way we think: from essence to existence. Data cannot reliably carry meaning within itself. Meaning arises only through use, context, and purpose.

Once this is accepted, the role of data governance changes fundamentally.

AI as an Accelerator of Data Governance Failure

Generative AI systems are probabilistic, adaptive, and increasingly autonomous. They consume and produce data like other systems, and the quality of their outputs can have immediate and profound consequences for people, organizations, and society.

Crucially, AI systems act without understanding intent or taking accountability for interpretation. They operate on statistical proxies of meaning and optimize patterns.

Data governance fails when we project understanding onto systems that operate on unknown or misaligned interpretations of intention. Hallucinations, bias, drift, and opacity are the most well-known failure modes of AI.

But they are not merely technical defects. They are symptoms of data governance that failed to:

• negotiate context,

• provide clear direction,

• or maintain accountability as systems evolved.

AI compresses decision cycles and shifts action away from human presence. This renders data governance models that rely on retrospective control or human-in-the-loop approvals structurally inadequate.

What this reframing makes explicit is that the challenge of AI governance is an acceleration of existing challenges. The same socio-technical questions data governance has always faced now surface faster, at greater scale, and with less room for retrospective correction.

Embedded, computational data governance is the only way the existing ambition survives.

From Control to Accountability

A common reaction to AI-driven incidents is to demand more control: additional approvals, stricter rules, heavier oversight. But control does not create meaning and rather constrains behavior.

The deeper issue is accountability. Existential thinking reminds us that accountability cannot be delegated away. Organizations remain accountable for the systems they design, deploy, and scale. Even when those systems act autonomously.

Data governance exists to make that accountability explicit and actionable. This is why there is a distinction between human-in-the-loop and human-designed feedback loops. The former attempts to preserve control by inserting humans into execution. The latter:

The Stable Core: Data Negotiation, Data Direction, Data Audit

Despite technological change, the core roles of data governance remain remarkably stable. These core roles that data governance has to play in an organization are a defining part of what data governance is.

The definition of data governance to tackle what is to come is different from what you may have encountered before. It is aligned with corporate governance, yet defined for data:

Data governance is a human-based system by which data assets in a socio-technical system are directed, overseen, and by which the organization is held accountable for achieving its defined purpose.

Based on this definition, data governance consistently fulfills three functions:

1. Data Negotiation
   Governance aligns business objectives, regulatory constraints, ethical considerations, and contextual realities. In AI systems, negotiation governs meaning: what data represents, how outputs should be interpreted, and where boundaries lie.

2. Data Direction
   Governance provides strategic direction. It answers not only whether systems perform correctly, but whether they should exist at all. Data Direction ensures that automation serves organizational purpose rather than undermining it.

3. Data Audit
   Data governance maintains accountability. In autonomous systems, audit shifts from periodic review to continuous assurance, real-time signals, observability, and feedback loops that allow responsibility to be exercised before harm occurs.

These roles are existential stabilizers in complex socio-technical systems. They operate on different levels, from strategy to tactics to assurance. This is not about thinking of data governance as a way to monitor and enforce, but as a discipline that designs for accountability.

Governance Is One

A key implication of this framing is that governance cannot be fragmented in purpose, even when execution is context-specific. Corporate governance, IT governance, data governance, and AI governance share the same purpose: directing collective action and ensuring accountability.

Treating them as separate disciplines creates gaps precisely where AI systems operate across boundaries. Data governance must be understood as part of organizational governance, not as a technical add-on, while remaining context-sensitive in execution. AI makes this integration unavoidable. This is the very reason why the wording of the data governance definition above is aligned with the syntax of the ISO definition of corporate governance (ISO 37000:2021).

From Human-Led to Human-Directed Governance

Old School data governance depends on human compliance, documentation, and incentives. This approach does not scale once decisions are automated and systems begin to act autonomously.

New School data governance approaches improve speed and decentralization, but still rely on implicit assumptions about meaning and accountability that break down once decisions are executed autonomously by systems.

Embedded data governance responds by making data governance computable. Enforcement, monitoring, and escalation are delegated to systems. Not as a replacement for human judgment, but as an extension of it.

Intent, constraints, and accountability are encoded in contracts, policies, and control mechanisms that platforms and AI agents can execute consistently at scale. This is the automation of existing data governance patterns, redesigned for environments where behavior emerges at runtime rather than being fully predefined.

As systems become agentic, data governance must observe and steer behavior over time. Humans negotiate meaning, set direction, and remain accountable for outcomes. Machines enforce constraints, monitor behavior, trigger escalation, and provide feedback loops through which data governance can learn and adapt.

MD101 Support ☎️

If you have any queries about the piece, feel free to connect with the author(s). Or feel free to connect with the MD101 team directly at community@moderndata101.com 🧡

Author Connect

Data Governance in the Wild

Distributed data landscapes, the advancement of AI, new requirements, they all require a shift in data governance. But not by loosing the core of data governance out of sight: negotiating data requirements, setting directing, and assuring accountability.

By Winfried Adalbert Etzel

MD101 Support ☎️

If you have any queries about the piece, feel free to connect with the author(s). Or connect with the MD101 team directly at community@moderndata101.com 🧡

From MD101 team 🧡

🧡 Join 1500+ Data Leaders Subscribed to The State of Data Products

Introducing The Catalyst, a special edition of The State of Data Products that compiles insights around the ups and downs in the data and AI arena that fast-tracked change, innovation, and impact.

📘 The Catalyst, first release is our annual wrap-up that compiles the most interesting moments of 2025 in terms of developments in AI & data products, with expert insights from across the industry!

Some focal topics

• The gap in AI readiness for enterprises

• The shift towards context engineering

• Addressing governance debt with a focus on lineage gaps

• Revisiting fundamentals for better alignment b/w business vision and AI.

Get your copy!

Alongside stellar strategies and impactful data governance wins, Tiankai’s way of expression, delivery, and evangelism strikes a chord that most of us in the data space never expected: to be reminded of that childlike learning bug.

Below are the top ten quality hits and behind the scenes from the man himself. Get your records on!

Top 5 Hits by @KaiFeng & Exclusive Behind the Scenes!

Data - The Musical 🎵

This parody medley of Broadway musicals blends a humorous self-introduction with a playful, musical take on the daily madness of data work. It drops quirky takes (“You’ll be back, soon you’ll see, coz your user story’s not complete”), fun biographical deets, and great storytelling to poke fun at the hyper-structured world of data people. The lighthearted tone sets up the message:

The humour works because it exaggerates the seriousness of data life while showing how human and relatable it really is.

The song highlights the hard realities of data, like endless lines to review, missing context, lost documentation, and the eternal search for KPIs that live “somewhere.” It makes fun of relying on Excel, the fact that not everyone knows SQL, and the universal pain of waiting for business input.

Yet beneath the jokes is a sincere emphasis that data literacy, collaboration, and joint ownership are the only path to true “data democracy,” and it can be unforgiving. Blending comedy with truth, the song celebrates the data professionals, acknowledges their frustrations, and encourages everyone to learn, participate, and enjoy the journey.

Tiankai Unveils the Behind-the-Scenes! 🎥

It all started with me being a huge fan of Broadway musicals, because I was always fascinated by the emotions and the contextual storytelling that these musicals could convey in their respective songs. One day, I finally had this genius spark of how these songs would sound like if they were about data, and in a few days, I had this whole collection of parodies written down, so I just had to record it and share with the world.

Governors of Data 🎤

Governors of Data, an original by Tiankai, is one of the biggest hits in data musicals, with thousands of plays, conveying key aspects and principles in the art of governance.

The song highlights how Governance acts as the bridge between business and technology that ensures clarity and trust across the organisation. Rather than being “data police,” governance teams function as sherpas. They’re guiding, enabling, and supporting teams so they can use data with confidence.

The original also highlights the pillars of governance: people, processes, and tools. It covers foundational practices like business rules, data models, metadata, master data, and catalogs, along with the need for automation and thoughtful solution design. Ultimately, governance is encased as a holistic, empowering framework that makes data reliable, discoverable, and ready for decision-making.

Imagine all this in a song!

Tiankai Unveils the Behind-the-Scenes! 🎥

When I had switched my career from being an analytics leader to being a Data Governance leader, my boss at the time joked that she expected me to write a song about Data Governance, because it really could use some rebranding and ice-breaking.

Well, what can I say, I took it seriously and I created this rap song about data governance, which is to this date the most-watched and listened song I ever wrote. It has truly worked magic for me too, because at that time it opened the doors for me to talk to senior leaders who weren’t approachable before, and nowadays my collaborators and clients use this song to break the ice with their stakeholders as well.

Digital Analytics Anthem 🎼

Unlike the general unfortunate practice, Analytics doesn’t end with charts. It is, in fact, more about translating raw information into decisions that make a business smarter. The song emphasises that business and analytics must work together, blending context with data to create insights that visibly move the needle. It frames analytics as a pure craft which is rooted in interpretation, communication, and partnership rather than isolated number-crunching.

It also highlights pride in the work. Opening the laptop isn’t just the start of a job, but part of a mission to bridge gaps and turn data into action. The repeated theme of “translating” hints at the role of analysts as interpreters who connect data to real outcomes that are measurable in business.

Tiankai Unveils the Behind-the-Scenes! 🎥

…and it all began with a lot of pain. Pain about digital analytics getting misunderstood and not getting enough priority, and I channeled all that into a rap song to just express how badass I think digital analytics really is.

Initially, I only shared it with colleagues, but those encouraged me to make it public and the encouragement and feedback I got was the spark that ignited many years of data music, and I still continue to enjoy doing it!

If ‘80s Songs Were About AI 🎶

This song uses humour and references some ‘80s great hits to capture the chaos of today’s AI-obsessed world. It pokes fun at how everyone wants to “do AI” without building the foundational data work needed to support it, and how people rush to prompts, tools, and hype before they’re even ready. The musical twists like “another job bites the dust,” “what’s love got to do with it,” “don’t stop investing,” add levity while highlighting the very real anxiety around rapid AI adoption, job displacement, and the overwhelming number of tools that appear every day.

Under the humour, this creation lands the message that AI may have powerful potential, but it must be built on solid data foundations which create lasting trust. Governed responsibly and supported by ongoing investment. By blending comedy with caution, the song encourages data folks to stay grounded, build wisely, and approach AI with both excitement and responsibility.

Tiankai Unveils the Behind-the-Scenes! 🎥

Something that I realized about the AI hype is that we keep on repeating same behavioral patterns from the past whenever a new technology came by, so I made the choice to cover 80s songs and reinterpret them for the age of AI, to also make a statement how if we can be positively nostalgic about things, can we not also remember the mistakes to avoid them in current times? The lyrics of these parodies are pretty fun, too.

If Disney Songs Were About Data 🎹

This song uses humour and musical parody to highlight the everyday struggles of data professionals, especially overdependence on Excel. The playful references such as “Do you want to build a dashboard?” and “We don’t talk about Excel… no no no,” turn common frustrations into lighthearted comments. From broken APIs to mysterious errors you “look and look” at but can’t figure out. Overall, making the shared experience fun rather than frustrating.

Beneath the jokes, the message is on better data literacy, stronger governance, and shared understanding of tools, from Python to R, being essential for improving data quality. The song encourages teams to move beyond spreadsheets, build real dashboards, adopt modern skills, and collaborate on governance. It ends on a note of both optimism and self-awareness:

Tiankai Unveils the Behind-the-Scenes! 🎥

Well, I’ll admit it, I’m a HUGE Disney fan, and I can probably sing along with most of the Disney songs that are popular - so in my view it was just a matter of time when I would combine it with my passion for data.

The most fun thing about writing this one was that I imagined the original singers / characters of the movies singing my newly interpreted data-related songs, and it made me chuckle quite a bit - and I hope this medley brings joy to everyone young and old as well.

Countless More Genius on Kai’s Channel

5 more special mentions!

🎵 If Broadway songs were about data
🎶 Match for life: A data quality themed love song
🎤 Data Mesh musical (parody medley)
🎹 When you believe (in data governance)
🎼 You & I & AI

The latest brews

🎧 Defining quality (a wicked parody)
🥁 (Unsolicited) jingles for data vendors
👩🏻‍🎤 Halloween songs about data
🦾 AI pressure (“under pressure” parody)
🎤 If ABBA songs were about agile

Tune in to the Kai Feng Collection

Gaurav Patole | Principal Data Strategy & Governance Advisor, Author of Data Quality ROI

Gaurav Patole is a Principal Data Strategy & Governance Advisor at ThoughtWorks and the author of Data Quality ROI. A seasoned data governance and data quality practitioner, he brings deep, hands-on experience helping global organisations turn trust in data into measurable business value. Formerly with BCG, Gaurav has led end-to-end data governance and quality programs across industries and geographies, bridging business and technology through his People-Process-Technology (PPT) approach. A frequent speaker and active community contributor, he is known for making data governance practical, people-centric, and ROI-driven. We’re thrilled to feature his unique insights on Modern Data 101!

We actively collaborate with data experts to bring the best resources to a 15,000+ strong community of data leaders and practitioners. If you have something to share, reach out!
🫴🏻 Share your ideas and work: community@moderndata101.com

*Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality & relevance. We keep it information-first and do not support any promotions, paid or otherwise!

Let’s dive in

When it comes to Data Quality, there is a standard piece of advice: “The business side must own the data.” We are told that every data initiative needs to be tied to a business use case, and the business should own the data issues.

While true, this advice often fails to solve the real problem. Why? Because simply telling a business leader that data quality matters doesn’t automatically make them want to fix it.

It feels uncertain, disruptive, and uncomfortable. Inaction, on the other hand, feels safe. Here is why your organization might be stuck in a cycle of inaction, and how to break free.

The Comfort of “Just How We Do Things”

In many companies, ignoring data issues isn’t a malicious choice. It’s a coping mechanism. When leaders don’t fix known data problems, the organization finds ways to survive around them.

• Workarounds become standard procedure.

• Broken processes get patched with quick fixes instead of repairs.

• Data teams stop raising flags to avoid awkward conversations.

Slowly, a dangerous narrative takes over: “This is just how it works around here.”

This is Cultural Inertia.

It happens when legacy mindsets (“we’ve always done it this way”) and decision fatigue collide. It’s safer to do nothing than to expose flaws in a process or risk being blamed for a problem.

The Latent Signals of Inaction

Inaction doesn’t always look like a dramatic failure. It is usually quiet, slow, and embedded in the daily routine. You might recognize it in scenarios like these:

• The “Cleanup” Delay: A monthly report takes two weeks to publish. Not because the analysis is hard, but because someone spends ten days manually fixing the data.

• The “Shadow” Expert: An executive doesn’t trust the official dashboard. Instead, they ask for a report from “Steve in Finance” because he’s the only one who knows how to filter out the errors.

• The Stalled Launch: A project is delayed because no one can agree on which dataset is the “real” one, and not because the strategy is bad.

• The Excel Trap: Your highest-paid analysts spend 60% of their time acting as janitors, cleaning rows in spreadsheets.

When you live with broken data for too long, it becomes normal. You stop fighting the bad data and start fighting the culture that accepts it.

How to Shift the Mindset: The Business Awakening Curve

To move from inaction to action, you can’t just implement a new tool.

I call this The Business Awakening Curve. It is not a technical maturity model, but a human one. It tracks how a business evolves from detachment to total ownership.

Stage 1: Ignorance

The Mindset: “That’s IT’s problem, not mine.”

Here, the business assumes data quality is a technical issue. If the report is wrong, they blame the dashboard or the system. The cost of bad data is accepted as the “price of doing business.” There is zero accountability.

Stage 2: Awareness

The Mindset: “We know it’s broken, but who is going to fix it?”

Usually triggered by a crisis, a wrong decision or a customer complaint, the business finally sees the impact of bad data. However, they are stuck in a loop of frustration. They ask, “Why isn’t this fixed?” but they aren’t ready to help fix it. This is a dangerous stage where teams often get stuck circling the problem.

Stage 3: Ownership

The Mindset: “This affects my results. We need to fix it.”

This is the turning point. Leaders realize that if Sales targets the wrong customers, or Finance can’t balance the books, it isn’t a “glitch” but a real strategic risk. Finger-pointing stops. Business teams start collaborating with data teams and assigning clear owners to their data.

Stage 4: Action

The Mindset: “We don’t just care about data quality, we design for it.”

This is the North Star. Data quality becomes part of the daily workflow. It is tracked in KPIs and discussed in planning meetings. Teams don’t just react to errors; they proactively design processes to prevent them.

Moving Forward

If your organization is stuck in the Ignorance or Awareness stages, you are vulnerable. Awareness without ownership leads to frustration. Ownership without action leads to cynicism.

Sometimes that means showing them the uncomfortable cost of their inaction.

Remember: Action leads to motivation, not the other way around. Good data doesn’t happen by default. It happens because you decide to act.

MD101 Support ☎️

If you have any queries about the piece, feel free to connect with the author(s). Or feel free to connect with the MD101 team directly at community@moderndata101.com 🧡

Contribute as an Author

Join one of the most diverse and forward-thinking data communities. With 65+ authors spanning data architects, tech founders, strategists, consultants, product managers, SMEs, and industry leaders, our collective voice shapes how the world understands modern data.

Modern Data 101 is read across all 50 U.S. states and reaches readers in 135+ countries, giving every contribution a truly global footprint. We’re excited to learn more about you and explore how your expertise can help push the conversation forward. Let’s get started.

Join Modern Data 101 Authors

We actively collaborate with data experts to bring the best resources to a 15,000+ strong community of data leaders and practitioners. If you have something to share, reach out!
🫴🏻 Share your ideas and work: community@moderndata101.com

*Note: Opinions expressed in contributions are not our own and are only curated by us for broader access and discussion. All submissions are vetted for quality & relevance. We keep it information-first and do not support any promotions, paid or otherwise!