Article
Building Data Products on an Open Platform: Stackable + Entropy Data
Dr. Simon Harrer
Co-Founder & CEO, Entropy Data ·
Many companies want a modern data platform without locking themselves into a single vendor. Proprietary cloud offerings are quick to set up, but data, logic, and knowledge end up in a closed system. Our demo shows another way: an open platform built from open source and open standards, self-hosted and fully under your own control.
The platform: Stackable
The Stackable Data Platform is the foundation. It provides proven open source components such as Trino, Airflow, Superset, and OPA on Kubernetes, whether in your own data center or in the cloud. The components are tuned to work together, securely configured, and rolled out via GitOps. Stackable takes care of security and support, without creating vendor lock-in.
The marketplace: Entropy Data
On top of the platform, data products emerge that build on each other. Entropy Data makes them discoverable and usable as a data marketplace. Every data product has a data contract that describes which data it delivers, what the data means, and in which quality. A shared business ontology connects terms such as customer, country, or brand across all data products. Lineage shows how the data flows, and access is requested and approved through the marketplace. Trino checks with every query whether the access is permitted.
Everything is based on open standards: the Open Data Contract Standard (ODCS) for data contracts, the Open Data Product Standard (ODPS) for data products, Apache Ossie for the ontology, and OpenLineage for lineage.
Contract-first, with a coding agent
In the video, we build a "Nation Scorecard": an overview of how each country develops as a market, with revenue, customer value, delivery quality, and local suppliers.
The data contract comes first. It describes what should be built before a single line of code exists. Country and region are not defined anew but taken over from the business ontology. With one click, the data product is created from the data contract.
A coding agent does the implementation. It gets three things: the data contract as the description of what to build; skills that know the conventions of the platform; and the marketplace with the data it can build on. The instruction is a single sentence: "Implement the data product." The agent analyzes the target, searches the marketplace for the existing data products it needs, gets access to their data, sets up the dbt project, builds the table in Trino including the data pipeline, tests the populated table against the data contract with the open source Data Contract CLI, and publishes everything to the marketplace, including lineage. After a few minutes, the new data product is active. This is the Data Product Builder at work.
Free choice on every layer
The demo shows one possible combination, not a fixed one. On the platform layer, Stackable can be extended: Stackable ships operators for further components such as Spark, Druid, or HBase, and because everything runs on Kubernetes, your own services and existing systems find their place as well.
The same applies on the governance layer. Entropy Data builds on ODCS, ODPS, Apache Ossie, and OpenLineage, so further tools can be connected, such as an existing data catalog, your own quality checks, or additional rules in OPA. Data contracts and data products live as open standards in your own Git repository. The decision which tools to use stays with you.
Why it fits together
Open standards are more than a principle here. They give the agent a clear frame: the data contract is machine-readable, the tests are deterministic, and the result can be verified. That is how an instruction turns into a reliable result.
The role of people shifts: away from programming, toward the question of what should be built and what value it creates. The platform makes sure that every new data product stands on open standards and fits seamlessly into the existing data landscape. For more on this way of working, see the talk Building a Data Product Contract-first (with AI).
If you want to dig deeper: the entire demo is openly available on GitHub, including the platform configuration, data products, and skills: github.com/entropy-data/entropydata-stackable-showcase.
Let's talk
Do you want to build an open data platform or develop data products contract-first? Get in touch with us:
- Stackable Data Platform: Sönke Liebau, CPO & Co-Founder of Stackable, on LinkedIn.
- Entropy Data: Dr. Simon Harrer, CEO & Co-Founder of Entropy Data, on LinkedIn.
About the author
Dr. Simon Harrer
LinkedIn of Simon HarrerCo-Founder & CEO, Entropy Data
Simon builds tools that help data, people, and AI work together. He holds a PhD in distributed systems, spent seven years at INNOQ as a software architect and consultant, and co-founded Entropy Data, where he leads the company as CEO.