Data Mesh in practice: what works, what doesn’t, and what nobody tells you

Data Mesh beyond the hype. The 4 principles, a real implementation with Unity Catalog, and why most companies get it wrong.

Data Mesh is probably the most misunderstood concept in modern data engineering. Everyone talks about Data Mesh, few implement it well, and many use it as an excuse for every team to do whatever they want.

In this post I get straight to the point: what it is, what it isn’t, and how to implement it in practice.

What Data Mesh (really) is

Data Mesh is an organizational architecture proposed by Zhamak Dehghani in 2019. It’s not a technology, it’s not a product, it’s not “every team gets its own lakehouse”.

It rests on 4 principles:

1. Domain Ownership

Data is the responsibility of the team that produces it, not of the central data team.

Before vs after: a centralized data team as a bottleneck vs domains owning their Data Products.

What works: domain teams know their data better. Transformations end up more correct.

What doesn’t: if the Sales team doesn’t have a data engineer, they won’t be able to maintain quality pipelines. Data Mesh requires every domain to have technical capacity.

2. Data as a Product

The data a domain publishes must be treated as a product: with documentation, SLAs, guaranteed quality, and an accountable owner.

Listing 1: Definition of a Data Product with SLA, schema, and consumers

# Data Product: daily sales
product:
  name: daily_sales
  domain: sales
  owner: sales-data-team
  sla:
    freshness: "daily by 8:00 AM UTC"
    availability: "99.5%"
  schema:
    table: prod.sales.daily_revenue
    format: Delta
    documentation: "https://wiki/sales/daily-revenue"
  quality:
    - "No nulls in revenue column"
    - "Date range: last 3 years"
    - "Reconciled with ERP daily"
  consumers:
    - finance-team
    - executive-dashboards
    - ml-churn-model

What works: when you treat data as a product, quality goes up. There’s an owner, there are SLAs, there’s documentation.

What doesn’t: if you don’t have Data Contracts and Quality Monitors, “data as a product” is just a pretty name for a table nobody maintains.

3. Self-Serve Data Platform

A platform team provides the tooling so domains can publish their data products without depending on a central team.

Self-Serve Data Platform: the platform provides templates, CI/CD, governance, and compute to the domains.

What works: Databricks Asset Bundles + Unity Catalog is a great combination for this. Each domain has its bundle template, deploys with CI/CD, and governance stays centralized.

What doesn’t: building the platform takes months. If you kick off Data Mesh before the platform is ready, it’s chaos.

4. Federated Computational Governance

Governance is global but execution is local. The platform team defines the rules; each domain implements them.

Listing 2: Federated governance: table properties and PII masking

-- Global governance: rules defined by the platform

-- All Gold tables must have these properties
ALTER TABLE prod.sales.daily_revenue SET TBLPROPERTIES (
  'domain' = 'sales',
  'data_product' = 'daily_revenue',
  'owner' = 'sales-data-team',
  'sla_freshness' = 'daily',
  'pii' = 'false'
);

-- All PII columns must be masked
-- (the platform defines this rule, each domain implements it)
ALTER TABLE prod.sales.customers
ALTER COLUMN email SET MASK platform.security.mask_email;

Implementation with Databricks + Unity Catalog

Catalog structure

Listing 3: Per-domain catalog structure in Unity Catalog

-- One catalog per domain
CREATE CATALOG sales;
CREATE CATALOG marketing;
CREATE CATALOG finance;
CREATE CATALOG platform;  -- for shared functions

-- Standard schemas in every domain
CREATE SCHEMA sales.bronze;
CREATE SCHEMA sales.silver;
CREATE SCHEMA sales.gold;      -- data products get published here

-- Permissions: each domain manages its own catalog
GRANT ALL PRIVILEGES ON CATALOG sales TO `sales-data-team`;
GRANT USE CATALOG ON CATALOG sales TO `data-consumers`;
GRANT SELECT ON SCHEMA sales.gold TO `data-consumers`;

DABs per domain

Listing 4: Databricks Asset Bundle configured for the sales domain

# domains/sales/databricks.yml
bundle:
  name: sales-domain

include:
  - ../../platform/shared-config.yml

resources:
  pipelines:
    sales_pipeline:
      name: "[${var.env}] Sales Data Pipeline"
      target: sales.silver
      serverless: true
      libraries:
        - notebook:
            path: ./notebooks/transform.py

  jobs:
    daily_gold:
      name: "[${var.env}] Sales Gold Refresh"
      tasks:
        - task_key: build_gold
          notebook_task:
            notebook_path: ./notebooks/gold.py
      schedule:
        quartz_cron_expression: "0 0 7 * * ?"
        timezone_id: "America/Montevideo"

Quality Monitors

Listing 5: Quality Monitor with custom metrics on a Data Product

-- Automated monitor on a data product
CREATE OR REPLACE QUALITY MONITOR sales.gold.daily_revenue (
  TIME_SERIES (
    timestamp_col = "date"
  ),
  CUSTOM_METRICS (
    (
      name = "revenue_not_negative",
      definition = "AVG(CASE WHEN total_revenue < 0 THEN 1 ELSE 0 END)",
      type = AGGREGATE
    ),
    (
      name = "row_count",
      definition = "COUNT(*)",
      type = AGGREGATE
    )
  )
);

The most common mistakes

1. “Data Mesh = no central data team”

Wrong. Data Mesh changes the central team’s role: from building pipelines to building the platform. If you eliminate the central team, you have no governance.

2. “Each domain picks its own stack”

Wrong. There is one platform. If Sales uses Spark, Marketing uses dbt, and Finance uses Pandas, you have no interoperability. The platform team defines the stack; each domain uses it.

3. “We’re starting Data Mesh tomorrow”

Wrong. Data Mesh is an organizational transformation, not a technical one. You need: ownership defined, teams with DE capacity, a self-serve platform ready, and management buy-in.

4. “Data Mesh replaces the data warehouse”

Wrong. Each domain’s data products can feed a centralized warehouse for executive reporting. Data Mesh and the warehouse coexist.

When does it make sense?

Situation	Data Mesh?
Company with 3-5 data sources	No, overkill
Data team < 10 people	No, centralized works better
50+ sources, 5+ domains, 20+ data people	Yes
Domain teams without DE skills	Not yet, train them first
You already have Unity Catalog + DABs	A good foundation to start

Links

• Data Mesh (Zhamak Dehghani) — the original reference
• How to Move Beyond a Monolithic Data Lake (Zhamak) — the article that started it all

• Unity Catalog (Databricks) — federated governance in practice

Next week: back to Databricks Tips with Lakeflow Declarative Pipelines.