Data Contract Best Practices
What are the best practices for data contracts?
By Dr. Simon Harrer · · 2:58 video · 7 slides
In short
Use one data contract for one output port, with one schema and one read role. Keep the data contract in the Git repository of the data product. Version it like an API, and treat the active data contract as the source of truth.
Watch on YouTube
One to one
One data contract in one major version describes one output port. This output port is one database schema with one or more tables and one read role. You can use a different structure, but it causes problems. With one data contract for each table, you get too many data contracts.
Which data it protects
A data contract protects one database schema with all its tables. The schema section of the data contract lists these tables. The output port, the schema, and the server have the same name. Each name contains the major version, and each environment has its own server.
One read role
Each data contract defines one read role. The platform creates this role automatically, with read access to this schema only. Data consumers request access to this role, so they know what to ask for. A second role, for example for personal data, is possible and causes few problems.
Where data contracts live
Data contracts live in the Git repository of the data product, next to the code. The output port data contract tells what the data product gives. An optional input port data contract tells what the data product uses from another data product. You can add your own quality checks to it.
A new column is a minor version
Version the data contract like an API. Strictly, a new column is a breaking change, because SELECT star then returns one more column. We do not allow SELECT star. Thus a new column is only a minor version.
A breaking change gets v2
A breaking change gets a new major version. Version 2 has its own output port, schema, and read role. Version 1 becomes deprecated, and both versions run in parallel for a grace period. The data consumers change to version 2 at their own speed. Then version 1 stops.
The source of truth
When a data contract is active, it is the source of truth. If the data and the data contract do not agree, the data is incorrect. Thus each change starts in the data contract. Then the code and the data follow.