I was literally thinking about this yesterday. There is just one difficulty, it is the fact materialized gold RDF graph is not as compressed as parquet tables, the foundation of open table format. It is a big deal. Currently I was thinking to go with one pipeline generating one ephemerous materialized in-memory RDF graph generated after query. But it means works to interpret the SPARQL query, through one ontology.
I have been thinking about another type of pipeline that works a little bit like this, but has a simple ontology model on each stage. For example, getting orchestra events from a website. I was thinking of getting them first with a simple class called web snippet and another one which is organisation. The HTML is a datatype property in the web snippet and then from that I create my next layer, the event, using again a simple model and leave those left to be harmonised, such as people and venues, as Literals. Then the next layer takes those literals, and makes them into proper and harmonised nodes. WDYT? Should I try to draw this?
Veronika, this framing is sharp. The IRI minting step in Silver is the piece most medallion implementations skip and then spend months paying for when joins break across sources.
One thing we ran into building KyraDB — a causal context engine for enterprise AI agents — on top of knowledge graphs: the Gold layer solves the human analyst problem cleanly, but agents have a different query profile. SPARQL is expressive but the 1-3 hop bounded traversals agents need at sub-millisecond latency want a different access pattern — prefix-keyed adjacency rather than triple-store scans.
Would be curious whether you have seen teams try to serve agents directly from the SPARQL endpoint and where that broke down.
Brilliant piece, Veronika Heimsbakk! Redefining the medallion architecture into intentional, strategic semantic enrichment layers is exactly the paradigm shift the industry needs right now.
If I can inject a practical challenge from my years navigating industrial digital threads: I frequently caution executive teams against chasing a "universal" or "global" web of knowledge. In complex production environments, a single, definitive universe is a costly mirage. What an enterprise actually requires is a targeted, boundary-enforced sectorial ontology that maps strictly to the real constraints of the business.
Furthermore, we cannot afford to build semantics for the sake of abstract data modeling. To drive true operational ROI, data readiness must grow gradually, strictly fueled by the immediate value delivered during use-case rollouts.
In my architecture framework, I ground this semantic journey directly into utility thresholds: Bronze handles localized project-level data, Silver upgrades the asset to a reusable enterprise standard for human teams, and Gold is strictly reserved for automating continuous value streams. This incremental ladder prevents data teams from over-engineering structures they cannot maintain, ensuring our documentation always matches the exact hands—biological or algorithmic—that hold it.
Veronika, I’d love to get your thoughts on how we can best keep data architecture teams laser-focused on immediate business value rather than letting the scope of the graph expand too early!
How would you operationalize governance and performance at scale for this kind of semantic Gold layer, specifically how do you manage ontology evolution, SHACL validation, and IRI minting strategies so that SPARQL queries over billions of triples remain both performant and trustworthy enough for frontline, rather than just analytical exploration?
Well, the core of it all is semantics. On an LPG, you'll loose your semantics, as it is a property graph. RDF triples can be converted to labeled property graph structures on import, but will loose native RDF semantics. No SPARQL, semantic reasoning or inference. There are plenty of semantic solutions on the market, tho. I will publish a vendor overview soon. :-) LPGs is however splendid for traditional graph analysis, just not for semantics.
I was literally thinking about this yesterday. There is just one difficulty, it is the fact materialized gold RDF graph is not as compressed as parquet tables, the foundation of open table format. It is a big deal. Currently I was thinking to go with one pipeline generating one ephemerous materialized in-memory RDF graph generated after query. But it means works to interpret the SPARQL query, through one ontology.
maplib (Python library for RDF) now supports Parquet based serialisation format for RDF graphs. :-)
I have been thinking about another type of pipeline that works a little bit like this, but has a simple ontology model on each stage. For example, getting orchestra events from a website. I was thinking of getting them first with a simple class called web snippet and another one which is organisation. The HTML is a datatype property in the web snippet and then from that I create my next layer, the event, using again a simple model and leave those left to be harmonised, such as people and venues, as Literals. Then the next layer takes those literals, and makes them into proper and harmonised nodes. WDYT? Should I try to draw this?
Veronika, this framing is sharp. The IRI minting step in Silver is the piece most medallion implementations skip and then spend months paying for when joins break across sources.
One thing we ran into building KyraDB — a causal context engine for enterprise AI agents — on top of knowledge graphs: the Gold layer solves the human analyst problem cleanly, but agents have a different query profile. SPARQL is expressive but the 1-3 hop bounded traversals agents need at sub-millisecond latency want a different access pattern — prefix-keyed adjacency rather than triple-store scans.
Would be curious whether you have seen teams try to serve agents directly from the SPARQL endpoint and where that broke down.
Brilliant piece, Veronika Heimsbakk! Redefining the medallion architecture into intentional, strategic semantic enrichment layers is exactly the paradigm shift the industry needs right now.
If I can inject a practical challenge from my years navigating industrial digital threads: I frequently caution executive teams against chasing a "universal" or "global" web of knowledge. In complex production environments, a single, definitive universe is a costly mirage. What an enterprise actually requires is a targeted, boundary-enforced sectorial ontology that maps strictly to the real constraints of the business.
Furthermore, we cannot afford to build semantics for the sake of abstract data modeling. To drive true operational ROI, data readiness must grow gradually, strictly fueled by the immediate value delivered during use-case rollouts.
In my architecture framework, I ground this semantic journey directly into utility thresholds: Bronze handles localized project-level data, Silver upgrades the asset to a reusable enterprise standard for human teams, and Gold is strictly reserved for automating continuous value streams. This incremental ladder prevents data teams from over-engineering structures they cannot maintain, ensuring our documentation always matches the exact hands—biological or algorithmic—that hold it.
For those looking at how to align this targeted, use-case driven path with machine autonomy and C-suite accountability, I’ve shared a parallel blueprint in my recent deep dives on [Agent-Ready Gold: Why Your AI Needs a Map] : https://engineeringtrust.substack.com/p/agent-ready-gold-knowledge-system-binding and [Purpose Fit for Data Products: The Trustworthiness Medals]: https://engineeringtrust.substack.com/p/purpose-fit-for-data-products-the .
Veronika, I’d love to get your thoughts on how we can best keep data architecture teams laser-focused on immediate business value rather than letting the scope of the graph expand too early!
How would you operationalize governance and performance at scale for this kind of semantic Gold layer, specifically how do you manage ontology evolution, SHACL validation, and IRI minting strategies so that SPARQL queries over billions of triples remain both performant and trustworthy enough for frontline, rather than just analytical exploration?
Curious to learn how this approach scale for large volumes - please share insights on this aspect
How much of what you say applies only to RDF/OWL versus LPG?
Well, the core of it all is semantics. On an LPG, you'll loose your semantics, as it is a property graph. RDF triples can be converted to labeled property graph structures on import, but will loose native RDF semantics. No SPARQL, semantic reasoning or inference. There are plenty of semantic solutions on the market, tho. I will publish a vendor overview soon. :-) LPGs is however splendid for traditional graph analysis, just not for semantics.