How someone could write such a lengthy article and not use the word 'mart' anywhere in the convo is beyond me. Let me give you a clue .. there's not one silver there's many. Not one gold, many. That's why people are so confused by medallion - they always miss that one important point. There is no reason for sales to be looking at HR data, is there!!??
I don't see medallion architecture as the culprit in this. This is all about governance, and with tools that track lineage, you can become very efficient with what you do or do not need. Data mesh is a framework built for people and processes; it is not a tool. I see the medallion as the foundation on which you apply data mesh. There are no set rules on where your users have to pull their data. ML/DS pull from silver and even bronze. Data Marts can and do pull from silver. You can also sub-divide your layers, there are no major rules. This is all predicated on how mature an organization is, and anything is better than siloed Excel sheets unless we want to try to start the "edge data architecture.".....
How someone could write such a lengthy article and not use the word 'mart' anywhere in the convo is beyond me. Let me give you a clue .. there's not one silver there's many. Not one gold, many. That's why people are so confused by medallion - they always miss that one important point. There is no reason for sales to be looking at HR data, is there!!??
I don't see medallion architecture as the culprit in this. This is all about governance, and with tools that track lineage, you can become very efficient with what you do or do not need. Data mesh is a framework built for people and processes; it is not a tool. I see the medallion as the foundation on which you apply data mesh. There are no set rules on where your users have to pull their data. ML/DS pull from silver and even bronze. Data Marts can and do pull from silver. You can also sub-divide your layers, there are no major rules. This is all predicated on how mature an organization is, and anything is better than siloed Excel sheets unless we want to try to start the "edge data architecture.".....
You’re not wrong - pulling clean and lean requirements down the medallion architecture sounds like a great idea.
In practice is it ever going to work out that way? Where we always have the luxury to shift left on quality? Maybe some day.
Very elaborate and well explained.