Building a Data Product Contract-first (with AI)

The Speaker

The talk followed a session on interface specifications and code generators. Simon agreed with its focus on structure and explicit interfaces, and announced a different way to get from specification to code: an AI coding agent. The audience could compare the two.

Simon Harrer is a software engineer at heart who moved into data with the rise of Data Mesh. He translated the Data Mesh book into German ("so I actually had to read the full book"), and publishes free material such as datamesh-architecture.com.

He sits on the Technical Steering Committee for the open standards at the Linux Foundation, the Open Data Contract Standard and the Open Data Product Standard. He co-created and maintains the Data Contract CLI, with about 1.4 million monthly downloads. His day job is Entropy Data, a data product marketplace built on data contracts and semantics: six people in Germany, "but using a lot of tokens".

Three Inputs, One Coding Agent

The code is written by a coding agent, "because that's how you do it now. No one writes code manually anymore." What makes the difference is what you feed it:

• The data contract YAML: what to build. The specification of the interface, linked to a data product definition and to the business ontology.
• Skills: how to build it. Your company's patterns and conventions. How do you build a data product for Snowflake with dbt? What are your guidelines? It lives in a Git repository of skills and templates, so the agent builds with the technologies and rules you control.
• A data marketplace: the data to build on. Data products build on other data products. The agent needs to know which ones exist, what their data means, whether it can stand on the shoulders of one of them, and then it needs access to actually query that data.

With all three in place, the instruction to the agent is a one-liner: implement it. The finished data product goes back into the marketplace, and the next one can build on it.

Live Demo: Contract First

The setting is an online shop. The goal is a data product for slow-selling brands: which brands barely sell, so you can talk to them, run a campaign, or take them out of the assortment.

The contract comes first. Simon creates it in the open-source Data Contract Editor inside Entropy Data. The form writes ODCS YAML in the background. He names the table slow_selling_brands, then adds the fields. brand is not defined from scratch: it is taken from the business ontology, where someone has already defined what a brand is. Then come slow selling products (an integer) and latest sold date (when a product of that brand last sold). Finally, the server: Snowflake, database SALES_PROD, schema slow_selling_brands_v1. Then the conference Wi-Fi dropped. "It's always live, you know. Bear with me."

A contract is only the interface, so the next step is the box around it: a data product with one output port that points to the contract. That is just another small YAML file. "We just said what we want. Like the cruise control in your car: let's go for 100 kilometers per hour."

In the terminal, Simon installs the skills as a Claude Code plugin from a public Git repository. It contains a dataproduct-bootstrap skill that lays out the dbt project and a dataproduct-implement skill that builds the pipeline. Then he connects the agent to the marketplace with the entropy-data CLI and an API key.

The prompt is one line, "Implement the data product slow-selling-brands with its data contract", plus a few demo-specific lines: do not ask questions, approve access requests if you have the rights, follow the Snowflake naming conventions, and publish the test results afterwards.

A run usually takes several minutes, so the agent kept working in the background while the talk moved on. "Fingers crossed. But it always works. I did this demo tens of times."

Input 1, the What: Start With a Conversation

The coding agent is a commodity, and the output can be inspected. The inputs are what matter, and the first one starts with people, not YAML. Get the people who need the data into a room. Print the free Data Product Canvas on A0, put it on the wall, and discuss which data product to build.

"This is still a thing humans should do. This is our place. Not the building: defining what we want, and where the business value is."

The Open Data Contract Standard

After the workshop, the result has to become executable: a YAML structure. Simon uses the Open Data Contract Standard (ODCS), the de facto industry standard for data interfaces at the Linux Foundation. Many companies use it, many vendors have adopted it, and the big ones have announced support. "If you don't know it yet, it will touch your lives at some point in the data world."

A contract sits next to a dataset, for example a set of tables in Snowflake, and states everything that is guaranteed about it:

• Fundamentals: ID, version, and status for lifecycle management
• Schema: tables and columns, with classification and examples
• Data quality: SQL-based or library-based checks the consumers can rely on
• Team: who owns it, and how to reach them (here, a Slack channel)
• Terms of use: what consumers may and may not do with the data, including the purpose
• SLAs: retention, freshness, latency
• Servers: where the data actually lives

"And agents all love these things. They thrive on this."

Excel or Editor, It Ends Up as YAML

There are several ways to write a contract. The ODCS Excel template covers the whole structure and converts to YAML automatically.

The open-source Data Contract Editor, used in the demo, also has a visual mode, so you can do real data modeling in it. Either way you get the YAML file that the agent needs as input.

Anchored in the Business Ontology

In the demo, the brand column was linked to the ontology. That ontology belongs to the business, not to the engineers: the business decides which entities and properties make up how it works.

The e-commerce example has two tables, orders and line_items, and three columns that all mean an order ID: the key in orders, the foreign key in line_items, and PREVIOUS_ORDER_ID, which links a replacement order to the cancelled one. All three point to the same Order ID concept. You define it once and reuse its metadata everywhere.

That also helps the AI. It sees that these fields are the same thing, so it knows it can join the two tables on them, even across two different data contracts. "We don't have to teach it. It's implicit in the metadata."

Contract + Semantics + Data Product

The contract describes an output port, and output ports belong to a box: the pipeline that produces them. One pipeline can serve version one, two, and three of an interface in production at the same time. That building block is the data product. It has input ports that consume other contracts and output ports that offer its own, all linked through the semantics.

Every part uses an open standard: the Open Data Product Standard for the building block, the Open Data Contract Standard for the interface, and Apache Ossie (formerly Open Semantic Interchange, started by Snowflake) for the semantics. RDF works the same way.

Give the AI access to all of that metadata, and it understands how everything fits together well enough to implement it. None of it depends on a tool. It is just standards, YAML, and structure. That is the first input.

Input 2, the How: Skills

Without instructions, the agent builds from what it learned on the web, and every prompt may produce a different implementation. "We want control. We want a say in how our data products are being built."

So the conventions go into a skills repository: skills, templates, and hooks that run at points in the lifecycle, for example a double check at the end of every prompt. This is where engineers, data architects, and even the consultants you hire put their knowledge. "And make yourself obsolete at some point. That's a little bit of a problem, but that's how this is set up." Writing all of that is a lot of work, so you use AI for it too. An open example is the dbt Data Product Builder.

The skills also build in feedback loops with the Data Contract CLI. The agent lints the contract to make sure it follows best practices. Once the pipeline has run, it tests whether the data in Snowflake keeps the contract's guarantees. If not, it iterates. "You are not done yet." The agent can run for a long time and burn a lot of tokens, but it knows when it is done.

The skills follow a standard folder structure, so Claude Code, OpenAI Codex, GitHub Copilot, and other coding agents all understand them. See Data Product Builder for the details.

Compliant by Design

Most companies already write down their policies, governance rules, and code conventions somewhere, often in Confluence. Leave them there. Run an AI routine every day, or on every change, that keeps the skills conformant to those rules. Keep the data product repositories conformant to the skills in the same way. Then every data product conforms to the company policies by transitivity.

If the rules say how to anonymize data or how to handle a top-secret field ("or maybe you're in the military, and the highest level is cosmic top secret"), the skills carry that, and the AI implements it.

You can also check the other way around and ask the AI whether a product conforms to the skills. But enforcing conformance is more powerful, because it needs no human: if it's not conformant, make it conformant.

Input 3, the Data to Build On: A Marketplace

To build a data product, the agent needs a set of capabilities. To stand on the shoulders of giants, it must discover data products, get their data contracts, resolve semantics, and request access. When it is done, it must publish the new data product and contract, the test results, and the lineage, so that others can observe it and build on it in turn.

"You can implement it however you like." Simon showed it with Entropy Data because that is what he built, but the concept is general: provide these capabilities, and an agent can use them.

In the demo, the agent reached the marketplace through a CLI, not MCP. MCP works as well, but coding agents work especially well with command-line tools, and a CLI does not pollute the context. Under the hood it is a thin wrapper around the REST API: search for "customer orders", get a data product, request access.

Live Demo: What the Agent Built

Claude Code finished in under six minutes, without asking a single question. Its summary: 41 brands materialized in Snowflake, all dbt tests (3/3) and data contract tests (11/11) green, the test results published to Entropy Data, and the data product set to active.

In the marketplace, the new data product is live. The agent worked out on its own that it had to build on two existing data products, articles latest and shelf warmers. It requested access to both through the marketplace.

In Snowflake, the table exists and is filled with brands, counts of slow-selling products, and last-sold dates. It does not live in SALES_PROD, though. The agent followed the naming conventions in the skills repository and renamed the database and schema, and it updated the contract to match.

The generated dbt project has input ports that carry their contracts and source descriptions, intermediate models, and an output port with the new contract and the SQL. The SQL builds common table expressions over shelf warmers and articles, joins them on the SKU column, groups by brand name, computes the count and the max, and casts to the contract's types. If the join columns had different names, it would still have worked, because both point to the same concept in the semantics.

The dbt run also sent OpenLineage events, so the data flow shows up as lineage in the marketplace. The product is still private. One click on Publish to Marketplace, and others can build on it.

Live Demo: Changing the Data Product

Creating a data product happens once. Changing it happens all the time, and it works the same way: change the contract first.

Simon adds an impact score column, a number described as a score based on the total number in stock and the list price. A slow-selling brand with one cheap product can be ignored. An important brand with a lot of stock cannot.

Then he resumes the Claude Code session and gives the same command again: implement it. The agent picks up the saved change. The input has changed, so the output has to change. The same loop also applies when you change the skills instead, for example your coding conventions or platform-specific details: everything flows through the agent again.

Iterate by Changing the What

When you change a data product, you usually change only the what. The skills stay the same, the marketplace stays connected, and that is how data products evolve. The human work moves to the contract, and to checking whether the result is right.

Setting up the skills takes some effort up front, but you reuse them for every data product after that.

"As a famous movie character said: this is the way. This is how I think data products should be built, and will be built. But maybe that's just my opinion. You can decide for yourself."