The Data Product Marketplace: A Single Interface for Business
The Data Product Marketplace: A Single Interface for Business
WHATs and WHYs of Marketplaces, Contrasts with Catalogs, and Marketplace Implementation Options
This is a community contribution from Arielle Rolland, a Data & AI Strategy Consultant, specialised in Data Product & Datamesh Enablement. She is also the founder of the Data Product Meetup in Montreal and a member of the Data Product Leadership Community. Her expertise focuses on developing Data Product strategy at scale and executing on the implementation to help organisations collaboratively achieve sustained and tangible returns on data product through simplicity, resiliency and re-usability. We’re thrilled to feature Arielle’s unique insights on Modern Data 101!
We actively collaborate with data experts to bring the best resources to a 9000+ strong community of data practitioners. If you have something to say on Modern Data practices & innovations, feel free to reach out! Note: All submissions are vetted for quality & relevance. We keep it information-first and do not support any promotions, paid or otherwise!
In this article, we’ll define what a data product marketplace is, explore how to get started, and offer guidance on assessing various tools. We’ll also provide insights into what a successful marketplace looks like and how it can benefit organizations.
A Marketplace for Data: Guide 101
Today, there’s a marketplace for almost everything, transforming how we consume as individuals and professionals daily. We have software & App Stores where you can find an app for nearly anything, a best-in-class Amazon marketplace overflowing with virtually any product imaginable, and grocery online store to order from. Yet, amidst all these conveniences, one critical resource remains absent: a dedicated marketplace for data.
And yet, organisations continue to struggle with effectively accessing, understanding and consuming their data.
Fragmented and siloed data make it difficult to locate the right information,
Applications controlled and operated by different lines of business frequently restrict data sharing.
A lack of common definitions and even data dictionaries leads to misuse of data,
Immature metadata practices and poor data lineage documentation complicate the understanding of where and how data is used.
Data becomes more complex to manage, which prevents organizations from leveraging data as a competitive advantage to become truly data-driven.
These challenges partially underscore the need for a data product marketplace— not only as a new tool or piece of technology to add to your enterprise architecture landscape but also as a concept designed to enhance the experience of anyone who wants to search, explore, publish, and consume data without needing to write a single line of code.
What is a Data Marketplace?
A data product marketplace provides a centralised and standardised intuitive shopping experience for data consumers and enables data product owners to share data products within (and sometimes beyond) the organisation.
Just as Amazon allows us to focus on products without concern for their origin or specific vendors, the marketplace should provide a holistic enterprise view of data available, remaining independent of any particular technology, database, or application.
The marketplace can either be internal or external.
The marketplace should enable three core functionalities:
Find/Search (or Discover/Explore)
Users can explore data product details, read the data contract (when available), preview data samples, and submit requests and feedback.
Publish
Data Product Owners can make their data products visible with clear descriptions and define the product performance KPIs, including usage and feedback.
Manage
Product Owners can establish standards and policies, monitor performance, notify issues and incidents and groom their product backlog based on the feedback received.
📝 Related Reads
The Front-End: The Consumer Experience
The data product marketplace serves as a one-stop shop for data consumers. Users can interact with data without requiring technical skills (such as SQL or Python). They can preview datasets, read descriptions, check feedback and ratings, and understand service level agreements (SLAs) or existing DQ issues/incidents.
When users seek more information, they can view the information of data product owners, allowing them to contact them directly. They can also explore business data lineage to see where the data is sourced and how it is transformed. Recommendations for similar data products are also available to ensure relevance and personalization for the user.
Users can sort data products by popularity, relevance, or rating and receive notifications about changes or issues with the datasets they use. Feedback and ratings from other consumers inform choices, enabling data product owners to make necessary improvements.
The Back-End: The Data Product Owner Toolkit
Traditionally, data owners struggle to understand how their data is consumed. The marketplace provides the necessary visibility, allowing them to manage their datasets actively.
They gain visibility into usage, can approve access requests, communicate directly with users about product issues or new releases, and even set a selling/subscription price.
Collecting feedback and ratings from users through this marketplace can now help inform future enhancements, grooming the data product backlog.
Data Marketplace vs. Data Catalog
Some may wonder how a data marketplace differs from a data catalog. While a data catalog can sometimes function like a repository—a dumping ground for assets. It does serve its purpose as a backbone platform for data management, lineage and metadata management, ensuring reliability and quality of the data assets. The marketplace tends to add a layer of commercialization. It acts as a curated registry, a searchable collection of unique data elements.
These two capabilities do not compete; rather, they complement each other. Many current data catalog technologies (such as Collibra, Informatica) are evolving to include marketplace features, and vice versa.
Buy vs. Build: How to Decide?
When considering tools to support this shopping experience, organisations should evaluate whether to buy a solution or build one in-house.
Custom-built Solution
I was recently speaking to a PM peer, and with a strong data architecture on their AWS cloud environment and strong naming convention and bucket structure, they were able to align documentation on Confluence and Excel with their physical data architecture for anyone to consult.
Implementing the data marketplace concept through the disciplined use of Confluence or Google Sheets can also be effective. However, this approach does require good process, ownership and governance and can also make automation and scalability challenging later on.
If you’re leaning towards a more sophisticated custom-built solution, consider the benefits of tailored user interfaces and workflows. Features like data product duplication detection and value tracking can be critical for aligning use cases with various data and tech products. For instance, Delight by Mindful is an excellent example of a tool that encapsulates these functionalities.
If your marketplace is intended for clients, partners, or customers, managing confidentiality, purchase rights, and data-sharing agreements requires centralized control. This aspect is often lacking maturity within the current vendor landscape.
Out-of-box SaaS
Marketplace vendors vary significantly in their offerings. Some are add-ons to existing data catalogs but may provide a less-than-optimal technical experience, while newcomers might offer a great user interface but lack the enterprise-level features needed by larger organizations. The size and structure of your organization will significantly influence your marketplace strategy.
Other key factors include:
User Experience: How customized do you want the experience to be?
Internal vs External Facing: Determine if the marketplace will also serve clients or customers.
Skillset: Do you have the resources and skills to integrate and maintain a marketplace at scale?
Cost: Consider licensing and infrastructure expenses, balancing capital expenditures (CAPEX) and operational expenditures (OPEX).
Integration: Discuss how the tool can connect with your existing ecosystem to automate and scale the population of the marketplace.
Gartner will provide a more detailed MarketScan report, but some notable vendors in this space include Collibra, Informatica, Snowflake, Databricks, Ataccama, Erwin, and Harbr.
📝 Editor’s Note
If you’re looking for Gartner’s Hype Cycle Recommendations, here are all the vendors recognised in the Data Product Category 2024 (Source):
Agile Lab, IBM, Informatica; K2view; The Modern Data Company; Nexla; One Data; Revelate
Conclusion
In today’s fast-paced environment, we all seek convenience and efficiency, especially when it comes to searching for and accessing data. With an intuitive and accessible interface and access to any relevant, reliable and trusted datasets, businesses are better positioned to unlock the true value of their data assets and drive better data-driven decisions. Let’s not wait eight weeks or six months to determine where to source the data.
As data products and data product owners emerge in the industry, the marketplace also becomes an essential tool for those data product owners to manage, monitor, and enhance their data products. They have the right platform to expose, publish and maintain the right types of information to consumers, facilitating cross-organisational collaboration.
The marketplace also serves as an excellent starting point for defining selling prices or subscription models for those data products in the future. By facilitating a chargeback (also known as a subscription) financing strategy based on actual consumption, organisations can fairly distribute the costs of data among those who are truly utilising it. This approach not only promotes accountability but also ensures constant relevance of the data.
MD101 Support ☎️
If you have any queries about the piece, feel free to connect with the author(s). Or connect with the MD101 team directly at community@moderndata101.com 🧡
Author Connect 🖋️
Find me on LinkedIn 🙌🏻
From The MD101 Team
Bonus for Sticking With Us to the End!
🧡 The Data Product Playbook
Here’s your own copy of the Actionable Data Product Playbook. With 700+ downloads so far and quality feedback, we are thrilled with the response to this 6-week guide we’ve built with industry experts and practitioners.
Stay tuned on moderndata101.com for more actionable resources from us!
 | A guest post by
|
Hi Arielle,
Really a great article and very helpful. In this article you have mentioned a tool "Delight by Mindful " Could you share more information about it.
Thanks