# Recce AI Blog — Full Content
> All AI blog articles in a single document for LLM consumption.
> Recce is an AI-powered data review agent for dbt teams.
---
# How Do You Turn Ad-Hoc Data Checks into Automated Institutional Knowledge?
> Senior data engineers carry validation knowledge in their heads. Learn how to capture ad-hoc checks as reusable preset validations that run automatically on every dbt PR, turning tribal knowledge into team-wide institutional knowledge.
Date: 2026-03-31
Source: https://blog.reccehq.com/from-ad-hoc-checks-to-automated-institutional-knowledge
Tags: data-quality, workflows, best-practices, dbt
## The Knowledge Problem Behind Data Validation
Every data team has the same hidden vulnerability: critical validation knowledge locked inside the heads of senior engineers. One team lead described the problem this way: "I reviewed three PRs in one day. Each touched core metrics but each developer checked different things. One caught a revenue metric issue, another completely missed it. Not because they weren't good, they just didn't know what they don't know."
This is not a process problem. It is a knowledge problem. The checks a senior engineer runs are not just validation steps. They are artifacts of domain knowledge, learned over time through incidents, bugs, and experience with how the data pipeline behaves.
## Why Do Ad-Hoc Checks Dominate Data Validation?
Data developers validate their work during development using whatever tools are at hand: a profile diff here, a custom query there, a value diff on a specific column. Each of these validations is a check that encodes knowledge about what matters for a given change.
The problem is twofold. First, developers know how to validate what they changed but often do not know how to validate the _impacts_ of what they changed. If you modified the CLV calculation in the `customers` model, you might verify the row counts and schema are stable. But do you know that the marketing team cares specifically about the high/medium/low customer segment distribution? That knowledge lives with the reviewer, not the developer.
Second, the validation results stay siloed. Developers told us their manual process:
1. Prepare two datasets for stakeholders
2. Export results to spreadsheets or screenshots
3. Explain over Slack why the changes are expected
4. Hope that the next developer touching the same model does similar checks
This workflow produces correct validations that immediately vanish. The same checks get reinvented from scratch by different developers on different PRs, with varying levels of thoroughness.
## Why Do Checklists Fail Without Collaboration?
Many teams try to solve the knowledge problem with checklists. The logic is sound: document your validation steps so others can follow them. In practice, checklists fail when there is no collaboration layer.
One developer explained the disconnect: "Why should I add this validation into a checklist? I validate on my local machine. If the stakeholder wants to see the result, I just paste a screenshot into Slack."
This approach felt good enough until the same developer started asking how to crop screenshots better to make results clear. Data engineers were spending time perfecting screenshots instead of validating data.
| Validation Approach | Knowledge Preserved? | Collaboration Possible? | Automated? |
| -------------------------------------- | -------------------------------------- | ------------------------------- | ---------- |
| **Local checks + screenshots** | No (lost when Slack scrolls) | Minimal (async, lossy) | No |
| **Local checklists** | Partially (list exists, results don't) | No (requires tool access) | No |
| **Cloud checklists with shared links** | Yes (checks + results preserved) | Yes (stakeholders click a link) | Partially |
| **Preset checks** | Yes (accumulated at project level) | Yes (run automatically) | Yes |
The breakthrough happens when checklists move from a local solo activity to a shared collaboration surface. When the same teams that ignored checklists locally moved to Recce Cloud, they described checklists as "a really powerful feature" because suddenly a reviewer or stakeholder could click a link and see the full validation context without installing anything.
## What Are Preset Checks and How Do They Capture Knowledge?
Preset checks are the mechanism that transforms individual validation knowledge into team-wide institutional knowledge. The concept is straightforward: when you run a check during PR validation and think "every PR that touches this area should verify this," you mark it as a preset check. That check then runs automatically across every future PR in the project.
For example, during a PR that modified CLV calculations, you might discover that the marketing team relies on a top-k diff of `customer_segments.value_segments` to understand segment distribution. You mark that check as a preset. From that point forward, any PR that impacts CLV-related models will automatically run that check, regardless of who authored the PR or how much experience they have.
This matters most for three scenarios:
- **New hires**: A junior data engineer creates their first PR. The preset checks run automatically, catching issues the engineer would not have known to look for because they haven't yet experienced the production incident that taught the team to add that check.
- **Cross-functional changes**: An engineer working on payments touches a model they don't usually own. Preset checks defined by the model owner ensure the right validations run even when the author lacks domain context.
- **Team turnover**: When a senior engineer leaves, their validation knowledge persists in the preset checks they defined, rather than walking out the door with them.
## How Do Preset Checks Differ from dbt Tests?
Preset checks and dbt tests serve complementary purposes. Understanding when to use which is part of effective [data review best practices](/ai-blog/data-review-best-practices/).
| Aspect | dbt Tests | Preset Checks |
| ---------------------- | ---------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
| **What they validate** | Schema constraints, referential integrity, accepted values | Data impact: how values changed, distribution shifts, metric movements |
| **When they run** | Every build | Every PR (scoped to impacted models) |
| **What they catch** | Hard failures (nulls, duplicates, broken references) | Soft changes that [pass tests but produce wrong data](/ai-blog/why-dbt-data-wrong-when-tests-pass/) |
| **Who defines them** | Analytics engineers in YAML | Anyone who identifies an important validation pattern |
| **Knowledge source** | Schema documentation | Production experience and stakeholder feedback |
dbt tests answer "is the data structurally valid?" Preset checks answer "did the data change in ways the team expects and stakeholders can accept?"
## Building a Knowledge Accumulation System
The vision beyond individual preset checks is a system where validation knowledge accumulates over time at the project level. Each PR becomes an opportunity to capture a new check that strengthens the safety net for future changes.
Recce Cloud stores preset checks at the data project level. As a team lead, this represents an ever-growing safety net: no matter how experienced a team member is, the PRs they create run through the accumulated wisdom of every past validation the team has deemed important.
The next evolution is context-aware automation: rules like "when model X is impacted, run check Y." This connects impact radius analysis with preset checks, ensuring that checks run when they are relevant rather than on every PR. For teams building their [dbt CI pipeline beyond basic tests](/ai-blog/what-should-dbt-ci-check-beyond-tests/), preset checks become the data-aware layer that CI has always been missing.
## FAQ
**Q: What is institutional knowledge in data validation?**
A: Institutional knowledge in data validation refers to the accumulated understanding of which metrics need care, which models are critical, and which validation checks should be performed when specific columns or models are modified. This knowledge typically exists only in the heads of senior data engineers.
**Q: What are preset checks in Recce?**
A: Preset checks are validation checks that have been marked as reusable and are automatically run across every PR in a data project. When a team member validates something and decides every future PR should include that check, they mark it as a preset check, creating an automated safety net.
**Q: Why do checklists fail for local data validation?**
A: Checklists fail locally because there is no collaboration layer. Data engineers validating on their local machine have no reason to document their checks in a checklist when sharing results means taking screenshots and pasting them into Slack. Checklists only become useful when they enable real-time collaboration with reviewers and stakeholders.
**Q: How do you preserve validation knowledge when team members leave?**
A: Preserve validation knowledge by capturing each ad-hoc check as a preset check stored at the project level in a cloud platform. When a senior engineer identifies an important validation, it becomes part of the automated pipeline rather than disappearing when that person changes roles or leaves the team.
---
# What Is Data Renegades? The Podcast for Real Stories Behind Data Tools
> Data Renegades is a podcast where engineers behind tools like Apache Airflow, Django, Datasette, and Apache Flink share unfiltered stories about building the data tools teams use every day.
Date: 2026-03-31
Source: https://blog.reccehq.com/introducing-data-renegades-the-podcast-for-the-real-stories-behind-data-tools
Tags: community, data-engineering, podcast
## Why Do Data Engineers Need a Podcast About Tool Origin Stories?
Data teams use tools like Apache Airflow, Datasette, and dbt every day, but rarely hear the full story of how those tools came to exist. Data Renegades is a podcast that fills this gap — featuring the actual engineers behind foundational data tools, sharing the unfiltered decisions, mistakes, and breakthroughs that shaped the technology.
Most data engineering content focuses on how to use tools. Data Renegades focuses on why tools were built the way they were, and what that means for teams choosing and operating them today.
## Who Are the Data Renegades?
The podcast features creators and core contributors behind some of the most widely adopted open-source projects in the data ecosystem:
| Guest Background | Tools/Projects | Why It Matters to Data Teams |
| ------------------------------- | -------------- | --------------------------------------------------- |
| Workflow orchestration creators | Apache Airflow | Understanding DAG-based scheduling design decisions |
| Web framework pioneers | Django | How web framework patterns influenced data tooling |
| Data exploration builders | Datasette | The philosophy behind lightweight data publishing |
| Stream processing architects | Apache Flink | Real-time vs. batch processing tradeoffs |
These are not marketing interviews. Each episode is a long-form conversation about the real engineering challenges — the dead ends, the compromises, and the moments where a design choice locked in years of consequences.
## What Makes This Different from Other Data Podcasts?
Most data podcasts fall into one of two categories: product demos dressed up as interviews, or high-level discussions that stay safely abstract. Data Renegades sits in the space between — technical enough to be useful, honest enough to be interesting.
The format prioritizes depth over breadth. Rather than covering five tools in thirty minutes, each episode dedicates the full conversation to one project and the person who built it. This means you hear:
- **The origin moment** — what problem triggered the creation of the tool
- **The hard tradeoffs** — what they gave up to ship, and what they'd change
- **The scaling surprises** — what happened when adoption outpaced the original design
- **The maintenance reality** — what it actually takes to keep a widely used tool alive
For data engineers evaluating tools or building their own internal platforms, these stories provide context that documentation never captures.
## How Does This Connect to Data Review and Quality?
Understanding how tools are built changes how you use them. When you know that a tool's data handling was designed for a specific scale or use case, you make better decisions about where it fits in your stack — and where it doesn't.
This is the same principle behind [data review best practices](/ai-blog/data-review-best-practices/): the more context you have about how data flows through your system, the better you can validate that changes don't break things. Tools are not black boxes. They carry the assumptions and constraints of their creators.
Recce's own approach to [AI-assisted data review](/ai-blog/what-is-ai-data-review-agent/) grew out of similar frustrations — the gap between what tools promise and what actually happens in production.
## What Topics Does the Podcast Cover Beyond Individual Tools?
Beyond specific tool histories, Data Renegades explores recurring themes across the data ecosystem:
- **Open source sustainability** — how projects survive after the initial creator moves on
- **Community vs. commercial** — the tension between open-source communities and the companies that fund development
- **Standards and interoperability** — why data tools still struggle to work together seamlessly
- **The accidental architect** — how engineers who built tools for their own team ended up shaping an industry
These themes resonate with anyone who has wondered why the data tooling landscape looks the way it does, and where it might be heading.
## How to Get Started with Data Renegades
New episodes are published through the Recce blog and available on standard podcast platforms. Each episode stands alone — there's no required listening order. If you work in data engineering and want to understand the decisions behind the tools you depend on, start with whichever tool is most relevant to your stack.
The podcast represents Recce's broader commitment to the data engineering community: building tools that help teams ship better data, and creating spaces where practitioners share what they've actually learned — not just what looks good in a conference talk.
## FAQ
**Q: What is the Data Renegades podcast about?**
A: Data Renegades is a podcast featuring the engineers behind widely used data tools — including Apache Airflow, Django, Datasette, and Apache Flink — sharing the unfiltered stories of how those tools were built, the tradeoffs they faced, and the real challenges of creating data infrastructure at scale.
**Q: Who hosts Data Renegades?**
A: Data Renegades is produced by Recce (InfuseAI Inc.), the company behind the AI data review agent. The podcast features long-form conversations with engineers and creators who built the foundational tools that data teams rely on daily.
**Q: Which data tools are featured on the Data Renegades podcast?**
A: Episodes feature the people behind Apache Airflow, Django, Datasette, Apache Flink, and other widely adopted open-source data and web tools. The focus is on the origin stories, design decisions, and unexpected challenges behind these projects.
**Q: Where can I listen to Data Renegades?**
A: Data Renegades episodes are available through the Recce blog and standard podcast platforms. Each episode covers one tool or project in depth, with the creator or a core contributor sharing their first-hand experience building it.
---
# What Are the Most Common Data Problems and How Do You Fix Them?
> Five real-world data problems — from AI agent benchmarking to DuckDB reconciliation to dbt cleanup — tackled live during the Data Valentine Challenge, with practical fixes for each.
Date: 2026-03-31
Source: https://blog.reccehq.com/data-valentine-challenge-wrapped
Tags: data-quality, dbt, best-practices, community
## Why Do the Same Data Problems Keep Showing Up?
Every data team has a version of the same story: pipelines that looked fine in development break in production, models accumulate technical debt faster than features, and nobody is sure when the data last matched reality. The Data Valentine Challenge — a five-day event where companies tackled real data problems live — confirmed that these issues are nearly universal.
Data reconciliation failures, untested AI agents, and sprawling dbt projects are not edge cases. They are the baseline state of most data platforms. The question is not whether your team has these problems, but whether you have a systematic way to find and fix them.
## What Are the Five Most Common Data Problems?
The challenge surfaced five distinct categories, each representing a different failure mode in the data lifecycle:
| Day | Problem | Root Cause | Fix |
| --- | ------------------------------- | ----------------------------------------------------------- | ------------------------------------------------------------------------ |
| 1 | AI agent benchmarking gaps | No standardized evaluation for data-specific AI agents | Define repeatable test suites with known-good datasets |
| 2 | Cross-system data discrepancies | Data drifts between warehouses, files, and APIs | Use DuckDB for lightweight reconciliation without infrastructure |
| 3 | Fragile no-code pipelines | Visual pipeline builders hide complexity and failure modes | Add validation checkpoints and schema contracts at pipeline boundaries |
| 4 | dbt project sprawl | Models accumulate without cleanup or ownership | Audit with DAG lineage, remove orphaned models, enforce naming standards |
| 5 | Missing data versioning | No record of what data looked like at a given point in time | Implement snapshot strategies and change tracking on critical tables |
Each problem was tackled live, with real teams showing their actual workflows — not sanitized demos.
## How Do You Benchmark AI Agents on Data Tasks?
AI agent benchmarking for data tasks is still in its early stages. Unlike traditional software testing where inputs and outputs are deterministic, AI agents produce variable results that need evaluation against business-specific criteria.
The challenge revealed a practical approach:
1. **Define a reference dataset** with known correct answers
2. **Run the agent** against the dataset under controlled conditions
3. **Measure three dimensions**: accuracy (did it get the right answer?), latency (how long did it take?), and cost (what did it spend in tokens or compute?)
4. **Track over time** to detect regressions when models or prompts change
This matters for data review because teams increasingly rely on AI agents to validate data changes — and an agent that gives confident but wrong answers is worse than no agent at all. Understanding [what an AI data review agent actually does](/ai-blog/what-is-ai-data-review-agent/) is the first step toward benchmarking one effectively.
## How Do You Reconcile Data Across Systems with DuckDB?
Data reconciliation — comparing datasets across different systems to identify discrepancies — traditionally required dedicated infrastructure. DuckDB changes this by running as an in-process analytical engine that can query CSV files, Parquet files, and database exports without a server.
A typical reconciliation workflow:
1. Export source data from your warehouse and target system
2. Load both into DuckDB (locally or in CI)
3. Run comparison queries — row counts, column distributions, value-level diffs
4. Flag discrepancies for investigation
This approach complements the [data diff techniques](/ai-blog/what-is-a-data-diff/) that catch issues before they reach production. The key advantage is speed: you can run a reconciliation in seconds without provisioning anything.
## How Do You Clean Up a Sprawling dbt Project?
dbt projects grow organically. Models get added for one-off analyses and never removed. Tests reference columns that no longer exist. Documentation covers models nobody uses. Over time, the project becomes a liability — every change risks breaking something nobody understands.
The cleanup process starts with visibility:
- **Map your DAG lineage** to understand which models feed which downstream consumers. [DAG lineage analysis](/ai-blog/what-is-dbt-dag-lineage/) reveals the actual dependency graph, not just the intended one.
- **Identify orphaned models** — models with no downstream dependencies that exist only because nobody deleted them.
- **Consolidate duplicate logic** — look for CTEs or models that compute the same metric differently in different places.
- **Enforce naming conventions** going forward to prevent the same sprawl from recurring.
The challenge showed that teams who invested a single day in cleanup eliminated 15-30% of their models with zero impact on downstream consumers.
## What Does Data Versioning Actually Look Like in Practice?
Data versioning means tracking what your data looked like at a specific point in time — not just what it looks like now. Without versioning, you cannot answer basic questions like "when did this metric change?" or "what did this table look like before the last deploy?"
Practical data versioning approaches include:
- **dbt snapshots** for slowly changing dimensions in your warehouse
- **Git-based versioning** for seed files and configuration
- **Change data capture (CDC)** for tracking row-level mutations in source systems
- **Environment comparison** — diffing development data against production before merging, as part of your [CI checks beyond tests](/ai-blog/what-should-dbt-ci-check-beyond-tests/)
The challenge confirmed that most teams have no versioning strategy at all. They discover data changed only when a stakeholder reports something wrong — days or weeks after the fact.
## What Did the Data Valentine Challenge Prove?
The biggest takeaway was not any individual fix. It was confirmation that data problems are systematic, not accidental. Every team faces the same categories of failure, and the teams that manage them well share a common trait: they have processes for validating data changes before those changes reach production.
Whether that process involves AI agent benchmarking, DuckDB reconciliation, dbt cleanup, or data versioning, the principle is the same — [validate the data, not just the code](/ai-blog/why-dbt-data-wrong-when-tests-pass/).
## FAQ
**Q: What are the five most common data problems teams face?**
A: Based on the Data Valentine Challenge, the five most common categories are: benchmarking AI agents against real data, reconciling data across systems like DuckDB, building reliable no-code data pipelines, cleaning up sprawling dbt projects, and implementing data versioning to track changes over time. Each represents a different stage of the data lifecycle where teams commonly lose confidence in their data.
**Q: How do you benchmark AI agents on data tasks?**
A: Benchmarking AI agents on data tasks requires defining repeatable evaluation criteria, running agents against known datasets with expected outcomes, and measuring accuracy, latency, and cost. The challenge revealed that most teams lack standardized benchmarks for data-specific AI agent work, making it difficult to compare tools or track improvement over time.
**Q: What is DuckDB data reconciliation?**
A: DuckDB data reconciliation is the process of comparing datasets across systems using DuckDB as a lightweight analytical engine. Because DuckDB runs in-process without a server, teams can quickly diff local files, database exports, or API outputs to identify discrepancies — making it useful for ad-hoc data validation without spinning up infrastructure.
**Q: How do you clean up a messy dbt project?**
A: Cleaning up a messy dbt project starts with auditing unused models using DAG lineage analysis, removing orphaned tests and documentation, consolidating duplicate logic, and establishing naming conventions. Tools that provide column-level lineage and impact analysis help identify which models are actually used downstream and which can be safely removed.
---
# What Framework Catches the Data Errors That Tests Miss?
> A practical framework for catching semantic data failures that pass all tests — covering why data tests miss business logic errors and how to validate data correctness before production.
Date: 2026-03-31
Source: https://blog.reccehq.com/the-production-data-reality-check-a-framework-for-catching-what-tests-miss
Tags: data-quality, best-practices, dbt
## Why Do Data Tests Give a False Sense of Security?
Every data team has experienced this: the CI pipeline is green, all tests pass, the PR gets merged — and then someone reports that the numbers are wrong. The data was structurally valid the entire time. The failure was semantic, not structural.
Semantic data failures are errors where the output conforms to every schema constraint and test assertion but is meaningfully wrong for the business. A revenue metric that doubled because of a JOIN fan-out. A customer count that dropped because a filter was too aggressive. A conversion rate that looks plausible but uses the wrong denominator.
These failures pass tests because tests check structure. They miss meaning. And for data teams, the cost of semantic failures — lost trust, wasted investigation time, bad business decisions — is far higher than the cost of a null value in a column.
## What Types of Data Errors Do Tests Miss?
Understanding the categories of failure is the first step toward building a framework that catches them. The [relationship between passing tests and wrong data](/ai-blog/why-dbt-data-wrong-when-tests-pass/) comes down to a fundamental mismatch between what tests check and what can go wrong:
| Error Category | Example | Why Tests Miss It |
| ---------------- | --------------------------------------------------- | ------------------------------------------------------------- |
| Logic errors | Revenue calculated with gross instead of net column | Both columns are valid numeric, non-null |
| Filter mistakes | WHERE clause excludes valid records | Remaining records still pass uniqueness and type checks |
| JOIN fan-out | One-to-many join produces duplicate rows | Each row is individually valid; row count tests may not exist |
| Upstream drift | Source column's meaning changes | Format and type remain the same |
| Aggregation bugs | SUM applied where COUNT was intended | Result is a valid number |
| Temporal errors | Date filter off by one day | Dates are valid; range is just wrong |
Each of these produces output that is structurally indistinguishable from correct data. The only way to catch them is to check the actual values, not just the constraints.
## What Does a Production Data Validation Framework Look Like?
A practical framework operates in four layers, each catching a different class of error. The layers are ordered from cheapest (most automated, least context needed) to most expensive (requires human judgment):
### Layer 1: Structural Tests
This is what most teams already have — dbt tests, schema checks, null constraints. These are essential but insufficient. They form the base of the pyramid.
**What they catch:** Missing columns, null values, duplicate keys, invalid references.
**What they miss:** Everything that is structurally valid but logically wrong.
### Layer 2: Statistical Validation
Compare column-level statistics between your development environment and production. Profile diffs check whether the distribution of values has changed meaningfully — means, medians, min/max values, null percentages, and cardinality.
**What they catch:** Subtle shifts in data shape that indicate logic changes. A column that used to average 150 now averages 300? Something changed.
**What they miss:** Changes that are statistically plausible but semantically wrong — for example, a 2% shift in a metric that should have been 0%.
### Layer 3: Semantic Validation
This is where [data diffs](/ai-blog/what-is-a-data-diff/) come in. Compare actual data values between environments at the row level for critical models. Check whether specific metrics, counts, and aggregations match expected values.
**What they catch:** The errors that statistical checks miss — specific rows that changed, specific values that shifted, specific models where the output diverged from the baseline.
**What they miss:** Novel business logic that has no production baseline to compare against (new models, new metrics).
### Layer 4: Human Review
Domain experts review high-stakes changes where the cost of being wrong is significant and automated checks cannot fully validate correctness. This is the most expensive layer and should be reserved for changes with the highest impact.
**What they catch:** Business context violations that no automated system can detect — "this metric should never exceed X" or "these two segments should never overlap."
**What they miss:** Nothing, in theory — but human attention is scarce and cannot scale to every PR.
## How Do You Implement This Framework in Practice?
Implementation does not require building everything at once. Start with the highest-leverage additions to your existing pipeline:
### Step 1: Identify Critical Models
Not every model needs all four layers. Map your DAG to identify models that feed customer-facing reports, financial calculations, or ML pipelines. These are your critical models — the ones where being wrong is expensive.
### Step 2: Add Automated Diffs to CI
For critical models, add schema diffs, row count diffs, and profile diffs to your CI pipeline. These run automatically on every PR, adding minutes to CI time but saving hours of incident response. This is the core of [what dbt CI should check beyond tests](/ai-blog/what-should-dbt-ci-check-beyond-tests/).
### Step 3: Scope Reviews with Lineage
Use column-level lineage to determine which downstream models are affected by a change. This prevents two failure modes: reviewing too much (wasting time on unaffected models) and reviewing too little (missing affected models that aren't obvious from the code change).
### Step 4: Establish Review Triggers
Define clear criteria for when human review is required. Examples:
- Changes to models that feed financial reports
- Changes that affect more than N downstream models
- Changes where profile diffs show more than X% shift in key metrics
- Changes to models that have caused incidents in the past
## How Does This Framework Compare to Data Observability?
Data observability tools monitor production data for anomalies after deployment. The validation framework described here operates before deployment — at the PR level. They are complementary, not competing:
| Approach | When | What It Catches | Limitation |
| ------------------------ | --------------- | ---------------------------------- | --------------------------------------- |
| dbt tests | Build time | Structural violations | Misses semantic errors |
| **Validation framework** | **PR time** | **Semantic errors via comparison** | **Requires production baseline** |
| Data observability | Post-deployment | Anomalies in live data | Catches issues after users are affected |
The ideal setup uses all three. But if a team can only add one new layer, PR-level data validation has the highest return — it prevents issues rather than detecting them after the fact.
## What Results Should Teams Expect?
Teams that implement systematic pre-merge data validation consistently report fewer production incidents, faster PR reviews (because reviewers can see what changed in the data, not just the code), and improved trust from stakeholders. The [vaidukt case study](/ai-blog/vaidukt-reduces-data-complaints-70-percent/) demonstrated a 70% reduction in data complaints from a three-person team.
The framework is not about replacing tests. Tests remain essential for structural validation. The framework adds the semantic layer that tests cannot provide — checking that the data is not just valid, but correct.
## FAQ
**Q: Why do data tests pass but data is still wrong?**
A: Data tests validate structural properties — schema conformance, null constraints, uniqueness, and referential integrity. They do not validate semantic correctness: whether calculations produce the right results, whether filters include the right records, or whether business logic matches real-world rules. Semantic errors produce structurally valid output that passes all tests while being meaningfully wrong.
**Q: What is a semantic data failure?**
A: A semantic data failure is a data error where the output is structurally valid (correct types, no nulls, proper relationships) but logically incorrect for the business. Examples include a revenue calculation that uses the wrong column, a filter that silently excludes valid customers, or a join that produces duplicated rows. These failures pass all automated tests because the tests check structure, not meaning.
**Q: How do you validate data correctness before production?**
A: Validate data correctness by comparing development data against production baselines before merging. Use data diffs to check row counts, column distributions, and value-level changes. Apply impact analysis via column-level lineage to scope which models are affected. Automate these checks in CI on critical models, and involve domain experts for high-stakes changes.
**Q: What should a data validation framework include?**
A: A comprehensive data validation framework should include four layers: structural tests (schema, nulls, uniqueness), statistical validation (distribution checks, profile diffs), semantic validation (business logic verification through data diffs against production), and human review (domain expert verification of high-impact changes). Each layer catches a different class of errors.
---
# How Do AI Agents Automate dbt Data Reviews?
> AI agents automate dbt data reviews using multi-agent architecture, MCP-only tool access, and structured prompts. Learn the reliability patterns that make automated PR summaries trustworthy.
Date: 2026-03-31
Source: https://blog.reccehq.com/designing-reliable-ai-agents-for-dbt-data-reviews
Tags: ai, data-review, dbt, mcp
## Why Do dbt Pull Requests Need Automated Data Review?
A dbt pull request shows code changes. It does not show downstream impact, row count shifts, or schema breaks. Data engineers spend hours manually running queries, tracing lineage, and checking row counts to answer the questions that actually matter: How many rows changed? Did the schema break? Which downstream models are affected?
Automated data review closes this gap. Given an active PR, an AI agent produces a structured summary — impact analysis, key changes, risk factors — in seconds as part of the CI pipeline. No manual queries, no lineage tracing by hand. The analysis is waiting when reviewers open the PR.
The challenge is building an agent that produces reliable, trustworthy output. This requires careful architectural decisions, not just prompt engineering.
## How Does Multi-Agent Architecture Improve Reliability?
Instead of building a monolithic agent, Recce uses a multi-agent system where specialized agents handle different parts of the analysis. An orchestrator delegates to two focused subagents:
| Subagent | Responsibility | Tools |
| ------------------ | ------------------------------------ | ----------------------------------------------------- |
| **git-context** | Fetches PR metadata and file changes | Git Host MCP |
| **recce-analysis** | Executes data validation queries | Recce MCP (lineage_diff, schema_diff, row_count_diff) |
Each subagent runs with its own isolated context window, a narrow scope, a small toolset, and a tightly focused prompt. This specialization makes each agent more predictable than a single all-purpose agent trying to handle everything.
The orchestrator receives tagged summaries from each subagent — prefixed with `[GIT-CONTEXT]` or `[RECCE-ANALYSIS]` — making it straightforward to integrate responses into the final output.
This pattern directly addresses a problem teams encounter with the [single-prompt approach to data review](/ai-blog/how-recce-built-an-ai-data-review-agent/): as PR complexity grows, a single agent hits context limits and starts losing information. Delegating deep analysis to specialists effectively multiplies available context capacity.
## What Is MCP-Only Architecture and Why Does It Matter?
A key design decision is restricting agents to MCP-only tool access. File system tools (Bash, Read, Write, Grep) are explicitly disabled, forcing the agent to use only MCP tools for all operations.
This constraint improves reliability because the agent cannot attempt creative workarounds that produce unreliable results. Without file system access, the agent cannot:
- Write ad-hoc scripts that might silently fail
- Read cached or stale data from disk
- Attempt workarounds that bypass the validated tool interface
The agent runs as a TypeScript application using the Claude API with an explicit denylist for non-MCP tools. Counterintuitively, removing capabilities makes the agent more capable at its intended task — producing trustworthy data review summaries.
## How Do You Prevent Hallucinated Lineage in AI-Generated DAGs?
AI models tend to invent edges in DAG diagrams based on semantic inference rather than actual data. Without constraints, a model might infer that `stg_payments` feeds `payments_final` based on naming conventions alone — even when the actual lineage says otherwise.
A two-phase approach solves this:
**Phase 1 — Output raw data:**
```
NODES from lineage_diff:
{"idx": 0, "name": "customers", "change_status": null, "impacted": true}
{"idx": 1, "name": "orders", "change_status": "modified", "impacted": true}
EDGES from lineage_diff:
[[5,0], [4,0], [5,1], [4,1]]
```
**Phase 2 — Generate Mermaid from raw data:**
```
[5,0] means idx 5 → idx 0: stg_orders --> customers
[4,0] means idx 4 → idx 0: stg_payments --> customers
```
By forcing the model to show its raw index mapping before rendering the diagram, hallucinations become visible before they reach the output. The DAG reflects actual [lineage data](/ai-blog/what-is-dbt-dag-lineage/), not semantic guesses.
## What Prompt Engineering Patterns Make AI Reviews Reliable?
Three prompt engineering techniques shape output quality:
**Structured output with required markers.** The prompt specifies sections tagged `[REQUIRED]` — Summary, Key Changes, Impact Analysis — ensuring consistent structure in every output. Without this, the agent produces different formats each time.
**Explicit negative constraints.** Telling the model what _not_ to do matters as much as affirmative instructions. Negative constraints prevent the agent from being overly helpful — attempting workarounds that produce unreliable results.
**Performance-aware instructions.** Constraints prevent expensive operations. For example: never use view models in `row_count_diff` or `profile_diff`, because views trigger expensive upstream queries. Instead, filter with `select:"config.materialized:table"`.
## AI Agent Reliability Patterns: A Summary
| Pattern | Problem It Solves | How It Works |
| ------------------------ | ------------------------------ | ---------------------------------------------- |
| MCP-only tools | Unpredictable agent behavior | Restrict action space to validated operations |
| Subagent delegation | Context window limits | Distribute analysis across isolated contexts |
| Two-phase DAG generation | Hallucinated lineage edges | Force raw data output before diagram rendering |
| Tagged responses | Response integration confusion | Prefix subagent output with identifiers |
| Negative constraints | Overly helpful agent behavior | Explicitly state what the agent must not do |
| Required markers | Inconsistent output format | Tag required sections in the prompt |
These patterns emerged from iterating on real PRs, not from theoretical design. Each addresses a specific failure mode discovered during production use.
## What Does the Output Look Like?
The AI-generated summary for a real PR includes:
- **Concise overview** of what changed and why
- **Key Changes** with before/after values and percentage changes
- **Impact Analysis** with a Mermaid DAG showing affected models and the [impact radius](/ai-blog/what-is-impact-radius/)
- **Validation checklist** with pass/fail status for each check
The review is generated in seconds as part of CI, replacing hours of manual investigation. Data engineers can focus review time on business context and edge cases rather than mechanical validation work.
## Key Takeaways for Building Reliable AI Data Agents
Architecture matters as much as prompts. The combination of multi-agent delegation with Claude Agent SDK, MCP-only tool access for reliability, and structured prompts with explicit constraints produces summaries that data engineers can actually trust. Constraints improve reliability, specialized agents beat general-purpose ones, and forcing the model to show its work eliminates a class of hallucination errors that would otherwise erode reviewer confidence.
## FAQ
**Q: How do AI agents automate dbt data reviews?**
A: AI agents automate dbt data reviews by combining multi-agent architecture with MCP tools. An orchestrator agent delegates PR context extraction and data validation to specialized subagents, each operating with a narrow toolset. The subagents fetch git metadata, run lineage diffs, schema diffs, and row count comparisons against actual warehouse data, then the orchestrator synthesizes findings into a human-readable impact summary.
**Q: Why use multi-agent architecture instead of a single prompt for data reviews?**
A: A single prompt approach hits context window limits and loses information as PR complexity grows. Multi-agent architecture delegates specific tasks to specialized subagents, each with its own isolated context window and focused toolset. A git-context subagent handles PR metadata while a recce-analysis subagent runs data validations. This specialization produces more consistent, reliable output than a single general-purpose agent.
**Q: How do you prevent AI hallucination in DAG lineage diagrams?**
A: A two-phase approach prevents hallucinated DAG edges. In phase one, the agent outputs raw node indices and edge data from the lineage_diff tool. In phase two, it maps those indices to model names and generates the Mermaid diagram. By forcing the agent to show its raw index mapping before rendering, hallucinations become visible before they reach the final output.
**Q: What role does MCP play in AI-powered data reviews?**
A: MCP (Model Context Protocol) provides the tool interface between AI agents and data validation capabilities. By restricting agents to MCP-only tools and explicitly disabling file system access (Bash, Read, Write), the agent cannot attempt creative workarounds that might produce unreliable results. This constraint counterintuitively improves reliability by narrowing the action space to validated operations like lineage_diff, schema_diff, and row_count_diff.
---
# How Did Recce Build an AI Data Review Agent?
> Recce evolved from a single prompt to a multi-agent AI system for dbt data reviews. Learn the architectural iterations, token limit challenges, and engineering decisions behind production-grade AI data review.
Date: 2026-03-31
Source: https://blog.reccehq.com/how-we-build-data-review-agent
Tags: ai, data-review, dbt, engineering
## Why Did a Single Prompt Approach Fail?
The first approach seemed obvious: gather all context of a code change, write one prompt, see the output, iterate. Using Anthropic's API, the team fed PR details, code diffs, lineage changes, row counts, and schema diffs into a single large prompt.
The first PR looked good. The second PR looked good. Then the edge cases arrived.
The summary format varied between runs. The output missed information randomly. The model stopped following prompt instructions carefully. The root cause: cramming everything into one API call pushed against prompt size limits. Information got cut off or the model could not process it all properly.
More critically, the LLM could only work with pre-selected data. Reviewing the output, the team would think "why didn't it check the row count for model X?" — then realize the answer was simple: because that data was not included in the prompt. The model could not go get data on its own.
The single prompt approach works for simple PRs. It breaks down as complexity increases.
## How Did the Agent Architecture Change Everything?
Moving to an agent architecture meant the LLM could call tools during its reasoning. Instead of pre-selecting all context, the agent could explore:
1. Check the lineage diff, notice model X changed
2. Follow the lineage, check downstream impacts
3. Run row counts for model X, see a 50% drop
4. Investigate why — check code diff, schema changes, write custom queries
The agent explores like a human reviewer: discover something, dig deeper, follow the trail.
The first agent architecture wrapped Recce as an MCP tool (giving the agent access to [lineage diffs](/ai-blog/what-is-dbt-dag-lineage/), row counts, query results, and schema changes), used `gh` CLI for PR context, and let the agent decide which tools to call and when.
Results improved immediately. Then dogfooding on more complex internal PRs revealed new failure modes.
## What Token Limits Constrain AI Agent Design?
Three limits shape what an AI agent can realistically do:
| Limit | Value | Impact |
| -------------------- | ----------------------- | ---------------------------------------- |
| Model context window | 200k tokens (Claude) | Total information the agent can hold |
| Single prompt limit | ~90k tokens (practical) | Maximum per API call |
| MCP tool response | 25k tokens per call | Maximum data returned by any single tool |
The MCP tool limit hit first. A lineage diff for a PR with only 5 changed models easily exceeded 25k tokens when returning the full API payload — every node detail, all dependencies, complete diff information. The tool call failed outright.
PR context fetching was similarly expensive. The agent needed 5-10 back-and-forth `gh` CLI calls to gather details, code diffs, comments, and metadata. Each round-trip added latency and burned tokens.
## How Do Subagents Solve the Context Problem?
With a single agent hitting both context window and prompt limits, the team needed to distribute the analysis load. Subagents — specialist agents each with their own 200k context window — provided the solution:
- **Main agent**: Orchestrates the overall job, receives summaries (not full details) from subagents
- **pr-analyzer subagent**: Extracts and interprets PR context, returns a summary
- **recce-analyzer subagent**: Explores data with Recce MCP tools, returns findings
The main agent does not need full exploration details — just the summaries. By delegating deep analysis to specialists, available context capacity effectively triples. This is the same [multi-agent pattern](/ai-blog/how-ai-agents-automate-dbt-data-reviews/) that makes the production system reliable.
## How Did the Team Optimize MCP Tool Responses?
Staying under the 25k MCP tool limit required several optimizations:
**Return only changed and downstream nodes.** In a 200-model dbt project, most models are unchanged upstream dependencies. For a PR with 5 changed models, keep those 5 plus their downstream impacts, filter out the ~150 unchanged upstream models.
**Use dataframes instead of key-value objects.** Key-value entries repeat keys for every record, wasting tokens. Dataframe format reduces duplication dramatically.
**Use numeric indices instead of long node IDs.** Replacing `model.my_project.customer_orders` with compact integers (1, 2, 3) substantially reduced token counts.
**Wrap PR fetching in a single GraphQL call.** Instead of 5-10 `gh` CLI round-trips, one custom MCP tool using a single GitHub GraphQL call fetches complete PR context. This reduced latency, token usage, and the risk of the agent losing track of partially fetched information.
## How Was the Lineage Graph Accuracy Problem Solved?
Generated lineage graphs were sometimes incorrect, showing wrong connections between models. The original response format followed `manifest.json` structure with a `parent_map` — a nested mapping the agent had to reason over carefully to produce correct Mermaid diagrams.
The fix was changing the tool output to match Mermaid's native edge representation. Instead of:
```json
{
"parent_map": {
"node_1": ["node_2", "node_3"]
}
}
```
The response became:
```json
{
"edges": {
"columns": ["from", "to"],
"data": [
["node_2", "node_1"],
["node_3", "node_1"]
]
}
}
```
With explicit edge pairs, the agent no longer needs to infer relationships from nested mappings. The generated diagrams became significantly more stable. This lesson — match the output format to how the consumer will use the data — applies broadly to MCP tool design.
## What Happened When the Agent Read Its Own Previous Output?
An unexpected failure: the agent consumed its own previous PR comment summaries as part of the PR context. When it read all PR comments to understand the discussion, it treated old summaries as new information. This caused two problems:
1. It failed to follow updated prompts, thinking the old summary was the correct format
2. It mistook old analysis as current context
The fix was filtering out agent-generated comments by their signature before feeding PR context to the agent. A simple but non-obvious solution to a problem that only surfaces during real usage.
## Architectural Evolution: A Timeline
| Stage | Architecture | What Broke |
| ----- | ----------------------------------- | -------------------------------------------------- |
| 1 | Single prompt with API calls | Prompt size limits, no autonomous exploration |
| 2 | Agent with Claude CLI + MCP tools | Token limits, unreliable lineage, slow PR fetching |
| 3 | Subagents + optimized MCP responses | Scaling needs, multi-platform support requirements |
| 4 | Claude Agent SDK | Production infrastructure for ongoing iteration |
Each stage was driven by dogfooding on real PRs of increasing complexity. What looks like a simple prompting problem turned out to be an infrastructure and observability challenge that took two months of iteration.
## What Did the Team Learn?
Domain-specific AI agents require deep iteration on token limits, context management, tool design, output formats, and edge cases discovered only through real usage. The gap between a weekend prototype and a production system is wider than most teams expect.
The system now works on GitHub PRs and GitLab MRs, uses six Recce MCP tools for [data validation](/ai-blog/what-is-a-data-diff/), and understands context across PR metadata, Recce analysis, and the data warehouse. But every feature in that list represents a specific failure mode that was discovered, diagnosed, and solved through iteration on real data.
## FAQ
**Q: How did Recce build its AI data review agent?**
A: Recce iterated through four architectural stages: a single prompt with API calls, an agent architecture with Claude CLI, custom MCP tools and subagents for token limits, and finally migration to Claude Agent SDK for production infrastructure. Each stage was driven by real failures discovered through dogfooding on internal PRs of increasing complexity.
**Q: Why did a single prompt approach fail for AI data reviews?**
A: A single prompt approach failed because cramming PR context, code diffs, lineage changes, row counts, and schema diffs into one API call exceeded prompt size limits. Information got cut off or the model could not process it properly. The LLM could only work with pre-selected data and could not explore context independently — it could not go check row counts for a model unless that data was already included.
**Q: What token limits affect AI agent architecture?**
A: Three token limits constrain AI agent design: the model context window (200k tokens for Claude), the single prompt limit (approximately 90k tokens in practice), and the MCP tool response limit (25k tokens per tool call). A lineage diff for just 5 changed models can exceed the 25k MCP limit when returning full API payloads. These limits drove the move to subagent architecture and MCP response optimization.
**Q: How do subagents solve AI agent context limits?**
A: Subagents solve context limits by distributing the analysis across multiple isolated 200k context windows. A main orchestrator agent delegates PR understanding to a pr-analyzer subagent and data exploration to a recce-analyzer subagent. Each specialist works with full context in its domain, then returns a summary to the main agent. This effectively triples available context capacity.
---
# How Does Simplified Automation Drive Data Tool Adoption?
> Complex CI/CD requirements block data teams from adopting validation tools. Learn how sessions architecture and metadata separation eliminated 10+ minutes of setup per validation and unlocked shift-left data validation.
Date: 2026-03-31
Source: https://blog.reccehq.com/simplified-automation-eliminated-adoption-barrier
Tags: workflows, adoption, dbt, CI-CD
## Why CI/CD Complexity Blocks Data Validation Adoption
Data teams that [struggle with tool adoption](/ai-blog/why-data-teams-struggle-with-tool-adoption/) often hit the same technical wall: the automation layer that makes validation useful is too complex to set up. The tool works. The concept is proven. But bridging the gap between "run it once manually" and "automate it for every PR" requires CI/CD expertise that most analytics engineers simply do not have.
The fundamental burden is artifact orchestration: for every validation run, the system needs metadata from two environments (production baseline and development branch), properly configured, and assembled into a format the validation tool can use. This process typically adds 10+ minutes per validation and requires writing custom CI/CD scripts that download artifacts, configure environments, and manage state files.
## What Was the Monolithic State File Problem?
Early data validation tools, including Recce's open-source version, used a monolithic state file that bundled everything together: environment artifacts from both base and PR branches, plus session management data like checks, runs, and runtime information. This created a cascade of problems:
1. **Users had to manually prepare multiple documents** every time they wanted to validate
2. **Production metadata was re-downloaded** for every single validation run, even though it rarely changed
3. **Local and CI validation required different preparation workflows**, doubling the configuration burden
4. **The state file was ephemeral**: when a validation session closed, the file and all its context disappeared
When data engineers tried to automate this, their CI/CD scripts grew into multi-step pipelines:
```yaml
# What teams had to write for every PR:
- name: Get Production Artifacts # Download base metadata
- name: Prepare dbt Base environment # Configure production env
- name: Prepare dbt Current environment # Configure PR branch env
- name: Generate Development Artifacts # Build PR metadata
- name: Upload Recce State File # Package everything together
```
Most analytics engineers either abandoned the effort entirely or simplified to a PR-only workflow where CI handled everything automatically, sacrificing the ability to validate during local development.
## How Does Sessions Architecture Solve This?
The breakthrough came from a simple realization: production deployments already generate the metadata that validation tools need. Every team running `dbt build` in production already creates `manifest.json` and `catalog.json`. Why force every validation run to download, configure, and re-orchestrate those artifacts?
Sessions architecture separates the monolithic state file into two independent pieces:
| Component | What It Contains | How It Is Generated | Update Frequency |
| ------------------- | ------------------------------------------ | ------------------------------ | -------------------------- |
| **Base session** | Production metadata (manifest + catalog) | Existing deployment pipeline | Once per production deploy |
| **Current session** | Development/PR branch metadata | PR creation or local dev | Once per PR or dev session |
| **State file** | Session management (checks, runs, runtime) | Generated after Recce launches | Per validation session |
The base session is generated once by the team's existing CD process and stored in the cloud. Every PR and every local development session references the same base session. When production deploys, the base session updates, and all active validations automatically sync to the latest production metadata.
## What Does the Simplified CI/CD Look Like?
The difference in automation complexity is dramatic:
```yaml
# Production baseline (CD pipeline):
- name: Update production metadata
uses: DataRecce/recce-cloud-cicd-action@v0.1
# PR validation (CI pipeline):
- name: Update PR metadata
uses: DataRecce/recce-cloud-cicd-action@v0.1
```
For local development, no script is needed at all. Since the base session exists in the cloud from the existing deployment process, developers can validate any time during development without environment preparation.
This reduction in complexity has measurable impact:
| Metric | Before (Monolithic) | After (Sessions) |
| ------------------------------------- | ------------------------------------ | ------------------------------------ |
| **CI/CD lines of config** | 30-50+ lines of custom scripts | 4-6 lines using pre-built actions |
| **Time per validation** | 10+ minutes for environment prep | Seconds (metadata already available) |
| **Local dev validation** | Requires manual environment setup | Zero setup (cloud base session) |
| **Infrastructure knowledge required** | Docker, secrets, artifact management | Basic GitHub Actions usage |
| **Base metadata freshness** | Stale (downloaded once per PR) | Always current (synced on deploy) |
## How Does This Enable Shift-Left Data Validation?
Shift-left validation means catching data issues during active development rather than waiting for PR review. It is widely accepted as a best practice in software engineering, but data teams have historically been unable to practice it because the setup cost of running validation locally was too high.
Sessions architecture makes shift-left validation practical because the base session is always available in the cloud. A developer working on a local branch can validate their changes against production metadata at any point during development without preparing environments, downloading artifacts, or writing scripts.
This restores the validation workflow data teams actually want:
- **During development**: Test changes locally against the automated base session in the cloud. Catch issues in seconds while context is fresh.
- **During PR review**: PR session metadata is generated by CI. Reviewers see the [impact radius](/ai-blog/what-is-impact-radius/) and can run targeted diffs immediately.
Teams that previously caught issues only at PR time (when fixes require context-switching back to a completed feature) can now catch those same issues during active development when the fix is a quick edit.
## Value-First Adoption in Practice
Sessions architecture directly enables the value-first adoption path that overcomes [data team adoption barriers](/ai-blog/why-data-teams-struggle-with-tool-adoption/):
1. **Immediate exploration**: Sign up and explore validation workflows with sample data, zero configuration
2. **Upload metadata**: Upload production and development metadata to see your own project's changes
3. **Connect warehouse**: Unlock [data diffing](/ai-blog/what-is-a-data-diff/) and custom queries
4. **Connect GitHub**: Enable PR-based validation with automatic session creation
5. **CI/CD automation**: Two pre-built actions replace dozens of lines of custom scripts
Each step delivers standalone value. A team that completes step 2 already has meaningful insight into their change impacts. The critical difference from the old approach is that no step requires mastering infrastructure concepts unrelated to data validation.
## The Architecture Lesson for Data Tooling
The broader lesson is that user research should drive technical architecture decisions, not the other way around. The monolithic state file made perfect engineering sense as a self-contained artifact. But it created a setup burden that blocked the adoption path users actually followed.
When the architecture was redesigned around how data teams work rather than how the system was originally structured, adoption barriers dissolved. The same validation capabilities that required DevOps expertise now require clicking a link. The tool did not get less powerful. The architecture just stopped asking users to solve problems that were not theirs to solve.
## FAQ
**Q: What is the biggest automation barrier for data validation tools?**
A: The biggest barrier is artifact orchestration: downloading production metadata, configuring dual environments for base and PR branches, and assembling everything into a state file for every validation run. This adds 10 or more minutes per validation and requires CI/CD expertise most analytics engineers do not have.
**Q: What is sessions architecture for data validation?**
A: Sessions architecture separates production metadata (base session) from development metadata (current session) into independent artifacts. The base session is generated once by existing deployment pipelines and reused by all PRs, eliminating redundant environment preparation.
**Q: What is shift-left data validation?**
A: Shift-left data validation means catching data issues during active development rather than waiting for PR review. When production metadata is available in the cloud without manual setup, developers can validate their changes locally at any time instead of waiting for CI/CD to run at PR creation.
**Q: How much time does sessions architecture save per validation?**
A: Sessions architecture saves 10 or more minutes per validation run by eliminating the need to download production artifacts, configure both environments, and orchestrate state files. Developers validate instantly against the cloud-hosted base session instead of preparing it locally every time.
---
# What Is Recce Showing at Coalesce 2025?
> Recce is at Coalesce 2025 in Las Vegas demonstrating Recce Cloud, AI-powered data review, and hosting the Data Renegade Happy Hour for the data engineering community.
Date: 2026-03-31
Source: https://blog.reccehq.com/coalesce-2025
Tags: community, recce-cloud, events
## Why Is Recce at Coalesce 2025?
Coalesce is the largest annual gathering of dbt practitioners — analytics engineers, data engineers, and data platform teams who build and maintain the transformation layer of the modern data stack. For Recce, it is the natural venue to show how AI-powered data review fits into the dbt workflow that these teams already use.
At Coalesce 2025 in Las Vegas, the Recce team is on the conference floor demonstrating Recce Cloud and connecting with data teams who are struggling with the gap between passing dbt tests and actually trusting their data.
## What Is Recce Cloud and What Does It Do?
Recce Cloud is the hosted version of Recce's data review platform. It plugs into your existing dbt project and CI/CD pipeline to automate the data validation steps that most teams do manually — or skip entirely.
Here is what Recce Cloud provides on every pull request:
| Capability | What It Does | Why It Matters |
| -------------------- | ------------------------------------------------------- | ---------------------------------------------------------- |
| Automated data diffs | Compares dev data against production | Catches value-level changes that dbt tests miss |
| Schema diff | Detects column additions, removals, type changes | Prevents breaking changes from reaching production |
| Column-level lineage | Maps exactly which downstream models are affected | Scopes review to the blast radius of a change |
| Profile diff | Compares statistical distributions between environments | Surfaces subtle shifts in data shape |
| AI review summaries | Generates plain-language explanations of data changes | Reduces time to understand what a PR actually does to data |
For teams already using dbt, Recce Cloud adds the data validation layer that sits between `dbt test` and the merge button. Understanding [what an AI data review agent does](/ai-blog/what-is-ai-data-review-agent/) in this context helps clarify why automated data checks are becoming essential.
## What Is the Data Renegade Happy Hour?
The Data Renegade Happy Hour is a recurring community event that Recce hosts at major data engineering conferences. It started as a way to bring people together outside the structured conference format — no slides, no pitches, just conversations between practitioners.
The happy hour returns at Coalesce 2025 after a successful first edition. The format is intentionally casual: data engineers, analytics engineers, and anyone else in the data ecosystem gathering to talk about what actually happens in their day-to-day work. The "renegade" framing ties back to the [Data Renegades podcast](/ai-blog/data-renegades-podcast-data-tools-stories/), which features the engineers behind foundational data tools sharing unfiltered stories.
For attendees looking to connect with the Recce team, the happy hour is the most direct opportunity — away from the conference floor noise.
## What Data Review Challenges Are Teams Bringing to Coalesce?
Based on conversations at previous conferences and in the Recce community, data teams heading to Coalesce 2025 are grappling with recurring themes:
- **CI pipelines that test but don't validate** — dbt tests pass, but nobody checks whether the actual data changed in ways that matter. Teams want to know [what dbt CI should check beyond tests](/ai-blog/what-should-dbt-ci-check-beyond-tests/).
- **No visibility into change impact** — a model change that looks small can cascade through dozens of downstream models. Without [column-level lineage](/ai-blog/what-is-column-level-lineage/), teams cannot scope the blast radius of a PR.
- **Review bottlenecks** — data teams lack the equivalent of code review for data. A software engineer's PR gets automated tests, linting, and peer review. A data engineer's PR gets a green checkmark from `dbt test` and a prayer.
- **Slow incident response** — when bad data reaches production, teams spend hours figuring out what changed and when. Data diffs and lineage analysis can compress that timeline from hours to minutes.
These challenges are not unique to any one team or industry. They are structural gaps in how the data ecosystem handles change management.
## How Does Recce Compare to Other Tools at Coalesce?
Coalesce attracts a range of vendors in the data quality and observability space. Recce's approach is distinct in that it focuses specifically on the PR-level review workflow rather than post-deployment monitoring:
| Approach | When It Acts | What It Catches |
| ----------------------- | --------------- | ----------------------------------------------------------- |
| dbt tests | Build time | Structural violations (nulls, uniqueness, relationships) |
| Data observability | Post-deployment | Anomalies in production metrics and freshness |
| **Recce (data review)** | **PR time** | **Value-level changes between dev and prod before merging** |
This pre-merge positioning means Recce catches issues before they reach production — complementing rather than competing with tools that monitor data after deployment.
## How to Connect with Recce at Coalesce 2025
The Recce team is available throughout the conference for live demos of Recce Cloud. Whether your team is evaluating data review tools, looking to automate CI validation, or just curious about how AI-assisted review works in practice, the conference floor and the Data Renegade Happy Hour are both good starting points.
For teams that cannot attend in person, the Recce Cloud demo is available online, and the conversations happening at Coalesce will inform upcoming content on the Recce blog and the Data Renegades podcast.
## FAQ
**Q: What is Recce showing at Coalesce 2025?**
A: At Coalesce 2025, Recce is demonstrating Recce Cloud and its AI-powered data review capabilities. This includes automated data diffs in CI, column-level lineage for impact analysis, and AI-generated PR review summaries that help data teams validate changes before merging.
**Q: What is the Data Renegade Happy Hour?**
A: The Data Renegade Happy Hour is a recurring community event hosted by Recce at data engineering conferences. It provides an informal space for data practitioners to connect, share experiences, and discuss the real challenges of building and maintaining data platforms — outside the vendor-driven conference floor.
**Q: What is Coalesce and who attends?**
A: Coalesce is the annual conference organized by dbt Labs, bringing together analytics engineers, data engineers, and data platform teams. It features talks on dbt best practices, data modeling, and the broader modern data stack. Coalesce 2025 takes place in Las Vegas.
**Q: What is Recce Cloud?**
A: Recce Cloud is the hosted version of the Recce data review platform. It integrates with dbt projects and CI/CD pipelines to automatically run data diffs, generate impact analysis, and provide AI-assisted review summaries on pull requests — giving data teams the same code review workflow for data changes.
---
# Three Essential Data Team Workflows Using Impact Radius
> Learn three daily workflows where impact radius transforms data validation: root cause discovery, developer validation, and data PR review. See how metadata-first analysis replaces expensive blanket data diffing.
Date: 2026-03-31
Source: https://blog.reccehq.com/building-impact-radius-3-three-essential-workflows-for-data-teams
Tags: workflows, data-modeling, dbt, best-practices
## Why Most Data Teams Validate the Wrong Things
Most data teams jump straight to expensive data comparisons without understanding the scope of their changes first. When a PR modifies a dbt model, the instinct is to run a full diff on every downstream table. This approach is slow, expensive, and ironically less thorough because teams run out of time and skip models they assume are unaffected.
Metadata-first validation flips this approach: view lineage to understand what changed and what is impacted, then focus data diffing only on actually impacted areas. The result is 10x faster validation, lower compute costs, and higher confidence.
Here are three daily workflows where [impact radius](/ai-blog/what-is-impact-radius/) transforms how data teams validate changes.
## Workflow 1: How Do You Trace the Root Cause of a Data Issue?
When a stakeholder reports that a dashboard metric looks wrong, the natural reaction is to start querying tables. A metadata-first approach is faster and more systematic.
**Step 1: Start at the problematic metric.** Click into column-level lineage for the reported metric (for example, `customer_segments.value_segment`). The lineage shows you that this column depends on `customers.customer_lifetime_value`, which is computed upstream.
**Step 2: Trace upstream through transformations.** Follow the lineage to find where `customer_lifetime_value` is calculated. In a Jaffle Shop example, CLV is computed in a CTE within the `customers` model using data from `stg_payments` and `stg_orders`.
**Step 3: Investigate the source data.** Run a targeted custom query on the payments data to understand what is feeding into the calculation. This is where you might discover that returned and pending orders are being included in CLV, inflating the numbers.
The key insight is that you arrived at the root cause using metadata navigation and one targeted query, not by diffing every table in the pipeline. For more on how [column-level lineage](/ai-blog/what-is-column-level-lineage/) enables this workflow, see our dedicated explainer.
## Workflow 2: How Should Developers Validate Before Creating a PR?
After identifying an issue and planning a fix, developers need to confirm their changes work correctly and do not break anything unexpected. This is where impact radius becomes a pre-PR safety net.
### Check what will be impacted before making changes
Before writing any code, use downstream lineage to understand the blast zone. If you are modifying the CLV calculation in the `customers` model, impact radius shows you that `customer_segments.value_segment` and `customer_segments.net_customer_lifetime_value` are downstream dependents.
### Validate each impact path after making changes
Once you make the fix, launch Recce and use Impact Radius to scan the change at the column level. This reveals the specific impact paths:
| Impact Path | What to Check | Validation Method |
| ------------------------------------------------------------------------ | ---------------------------------------- | ------------------------------------ |
| `stg_payments.coupon_amount` -> `customers.net_customer_lifetime_value` | New column values are correct | Custom query comparing before/after |
| `customers.customer_lifetime_value` -> `customer_segments.value_segment` | Segment distribution changed as expected | Top-k diff on value segments |
| `customers.customer_lifetime_value` -> `customer_order_pattern` | No unintended side effects | Profile diff for statistical summary |
### Prepare stakeholder-ready evidence
Before creating the PR, run a [data diff](/ai-blog/what-is-a-data-diff/) on the impacted business metrics. A top-k diff on `customer_segments.value_segment` produces a clear chart showing how the segment distribution changed. This chart goes directly to stakeholders so they can see the impact and adjust their work, such as updating marketing budgets for the new high-value customer threshold.
## Workflow 3: How Can Reviewers Validate a PR They Did Not Author?
PR review is where validation historically breaks down. The reviewer has no context on what was checked during development, no systematic way to see data impacts, and no time to run every possible comparison.
### Map code changes to impacted models
Start by clicking through the lineage in the PR to map changed files to modified models. At a glance, confirm the modified models match what the PR description claims.
### Review the proof provided with the PR
Well-structured PRs should include saved check results. When developers follow [data review best practices](/ai-blog/data-review-best-practices/), they save profile diffs and custom query results as checklist items that reviewers can view instantly or rerun for verification.
### Run targeted additional validations
If the reviewer wants to be thorough, they can run their own checks. For example, query the data to verify that all excluded orders had non-completed status. This takes seconds when you know exactly which model and column to check, rather than hunting through the entire DAG.
### Provide data-driven recommendations
With validation complete, the reviewer can make recommendations grounded in data. For instance: "The max high-value customer CLV dropped from \$10,092 to \$6,852. We should consider lowering the high-value threshold from \$4,000 to \$3,500 to maintain similar segment sizes."
## Metadata-First vs. Data-First Validation Compared
| Aspect | Data-First (Traditional) | Metadata-First (Impact Radius) |
| ----------------------------- | ------------------------------------- | ---------------------------------------- |
| **Starting point** | Run diffs on all downstream tables | View lineage to scope what is impacted |
| **Compute cost** | High (full-table comparisons) | Low (targeted diffs only) |
| **Time to validate** | Hours | Minutes |
| **Coverage confidence** | Low (often skip tables due to time) | High (all impacted paths are mapped) |
| **Stakeholder communication** | Ad-hoc screenshots and Slack messages | Structured charts and quantified changes |
| **Reviewer experience** | Flying blind | Guided by lineage and saved checks |
## Connecting the Workflows Into a Continuous Cycle
These three workflows form a continuous validation cycle. Root cause discovery identifies issues and surfaces fixes. Developer validation confirms the fix works correctly and documents the evidence. PR review verifies the work independently and adds a second perspective.
The common thread across all three is that [impact radius](/ai-blog/what-is-impact-radius/) scopes the work. Instead of checking everything or guessing what to check, teams trace the actual dependency chain and validate precisely what is affected. This is what makes data validation scalable as dbt projects grow from tens of models to hundreds.
## FAQ
**Q: What are the main workflows that use impact radius?**
A: The three main workflows are root cause discovery (tracing a reported issue upstream through column-level lineage), developer validation (checking all impacted models before creating a PR), and data PR review (systematically validating a teammate's changes using lineage and targeted data diffs).
**Q: What does metadata-first validation mean?**
A: Metadata-first validation means analyzing lineage, schema changes, and model dependencies before running any data queries. This approach scopes your validation to only the models and columns actually impacted by a change, avoiding expensive full-table data comparisons.
**Q: How does column-level lineage help with root cause analysis?**
A: Column-level lineage lets you click on a problematic metric and trace it upstream through each transformation to find where incorrect data originates. Instead of querying every model, you follow the dependency chain directly to the source of the issue.
**Q: How does impact radius reduce PR review time?**
A: Impact radius shows reviewers exactly which models and columns are affected by a code change, so they can run targeted diffs on only the impacted areas. This replaces hours of blanket data comparison with minutes of focused, scoped validation.
---
# How Did vaidukt Reduce Data Complaints by 70% with Systematic Validation?
> German energy platform vaidukt reduced customer data complaints by 70% using Recce for systematic PR-level data validation, transforming how a 3-person data team catches errors before production.
Date: 2026-03-31
Source: https://blog.reccehq.com/-vaidukt-press-release
Tags: case-study, data-quality, recce-cloud
## What Data Problem Was vaidukt Facing?
vaidukt is a German energy platform whose data team powers customer-facing reports and operational dashboards. Like many data teams, they relied on dbt tests and manual spot-checks to validate changes before deploying to production. And like many data teams, they discovered the hard way that passing tests does not mean correct data.
Customer complaints about data accuracy were a regular occurrence. The issues were not dramatic failures — they were subtle: a metric calculated slightly differently after a model change, a filter that excluded records it shouldn't have, an upstream schema change that passed all tests but shifted downstream values. Each complaint eroded trust and consumed engineering time to investigate.
For a team of just three data engineers, this was unsustainable. Every hour spent investigating a data complaint was an hour not spent building new capabilities.
## Why Weren't dbt Tests Enough?
This is one of the most common frustrations in data engineering. dbt tests validate structural properties — not-null constraints, uniqueness, accepted values, referential integrity. They confirm that data looks right at a schema level. But they do not confirm that the data is right at a business level.
The gap between structural validity and business correctness is where most data complaints originate. [Understanding why dbt data can be wrong even when tests pass](/ai-blog/why-dbt-data-wrong-when-tests-pass/) is the first step toward closing that gap.
vaidukt's complaints fell into predictable categories:
| Complaint Type | dbt Test Coverage | Root Cause |
| ---------------------------------------------- | -------------------------------------------- | ----------------------------------------- |
| Metric values shifted unexpectedly | Not tested (values were non-null and unique) | Logic change in upstream model |
| Report showed different totals than last month | Not tested (schema unchanged) | Filter condition modified during refactor |
| Customer segment counts changed | Not tested (accepted values still valid) | JOIN condition produced different fan-out |
| Dashboard showed stale data | Not tested (freshness check passed) | Incremental model skipped records |
In every case, the data was structurally valid. The tests did exactly what they were designed to do. The problem was that nobody was checking whether the actual values were correct.
## How Did vaidukt Implement Systematic Validation?
Rather than hiring dedicated data quality engineers or building custom validation scripts, vaidukt integrated Recce into their existing CI/CD pipeline. The approach was straightforward:
1. **Every PR triggers automated data diffs** — Recce compares the development environment's data against production for affected models
2. **Schema diffs catch structural changes** — column additions, removals, and type changes are flagged before review
3. **Profile diffs surface statistical shifts** — if a column's distribution changes meaningfully, the reviewer sees it immediately
4. **Row count diffs flag data volume changes** — unexpected increases or decreases get attention before merging
5. **Impact analysis scopes the review** — column-level lineage shows exactly which downstream models are affected by the change
The key insight was making validation automatic and pre-merge. The team did not need to remember to run checks or build custom scripts for each model. Validation happened on every PR, for every change.
## What Were the Results?
The headline number — a 70% reduction in customer data complaints — captures the business impact. But the operational improvements were equally significant:
- **Faster PR reviews** — reviewers could see exactly what changed in the data, not just the code, reducing review time
- **Fewer production incidents** — catching issues before merge eliminated the investigation-and-fix cycle
- **Higher team confidence** — engineers were less afraid to refactor models because they could verify the data didn't change unexpectedly
- **Better stakeholder trust** — fewer complaints meant stakeholders gradually stopped double-checking reports manually
For a three-person team, the time savings alone justified the investment. The hours previously spent investigating complaints could be redirected to building new models and improving existing ones.
## What Can Other Small Data Teams Learn from vaidukt?
vaidukt's experience reinforces several [data review best practices](/ai-blog/data-review-best-practices/) that apply regardless of team size:
### Automate Validation at the PR Level
Post-deployment monitoring catches problems after they reach users. PR-level validation catches them before merge. For small teams that cannot afford the incident response overhead, pre-merge validation is the higher-leverage investment.
### Focus on Critical Models First
vaidukt did not try to validate every model from day one. They started with the customer-facing models that generated the most complaints, then expanded coverage as the process matured. This mirrors the approach of [prioritizing CI checks on high-impact models](/ai-blog/what-should-dbt-ci-check-beyond-tests/).
### Make Validation the Default, Not the Exception
The most important change was cultural: data validation became an automatic part of every PR, not something engineers did when they remembered. When validation is opt-in, it gets skipped under deadline pressure. When it is automatic, it becomes the team's safety net.
## Is 70% Complaint Reduction Realistic for Other Teams?
The specific number depends on where a team starts. Teams with no pre-merge data validation will likely see dramatic improvements. Teams that already do some manual validation may see more modest gains but save significant engineering time.
The underlying principle is consistent: checking actual data values before merging catches the semantic errors that structural tests cannot. Whether the improvement is 40% or 80%, the direction is always the same — fewer surprises in production, faster reviews, and more trust in the data.
## FAQ
**Q: How did vaidukt reduce data complaints by 70%?**
A: vaidukt, a German energy platform, reduced customer data complaints by 70% by implementing systematic data validation using Recce on every pull request. Instead of relying solely on dbt tests and post-deployment monitoring, their 3-person data team began reviewing actual data diffs before merging, catching semantic errors that structural tests missed.
**Q: What is vaidukt?**
A: vaidukt is a German energy platform that manages data pipelines for energy industry operations. Their data team of three engineers maintains the dbt-based transformation layer that feeds customer-facing reports and operational dashboards.
**Q: Can a small data team implement systematic data validation?**
A: Yes. vaidukt demonstrated that a 3-person data team can implement systematic data validation without dedicated data quality engineers. By integrating Recce into their existing CI/CD pipeline, they automated data diffs on every PR, reducing manual review burden while significantly improving data accuracy.
**Q: What is systematic data validation?**
A: Systematic data validation is the practice of automatically comparing data output between development and production environments on every code change, rather than relying on ad-hoc checks or post-deployment monitoring. It catches value-level errors — like incorrect calculations or unexpected row count changes — before they reach production.
---
# What Happens When AI Builds Your dbt Models?
> A firsthand account of letting Claude Code build an analytics warehouse end-to-end with dbt. The interesting part was not the generated code — it was the setup, review, and guardrails that made the output usable.
Date: 2026-03-31
Source: https://blog.reccehq.com/i-let-claude-code-build-my-dbt-models.-the-interesting-part-wasnt-the-code
Tags: ai, dbt, data-engineering
## What Does AI-Assisted dbt Development Actually Look Like?
The promise of AI-generated code is compelling: describe what you want, get working models back. The reality is more nuanced. When you let an AI tool like Claude Code build an analytics warehouse end-to-end — Snowflake tables, S3 ingestion, sources, staging, intermediate, and mart models — the interesting part is not the generated code. It is everything surrounding it.
AI-assisted analytics engineering is not a prompting problem. It is an infrastructure problem. The skills, the MCP configs, the schema conventions, the guardrails — that is the actual work. The generation is the easy part, and it is the part that still needs a human reviewing every decision.
## What Setup Does AI Need Before Generating dbt Models?
Before any code generation, the real work is preparation. This includes:
- **Custom skills** defining naming conventions, primary key patterns, model structure, and dev environment commands
- **MCP integrations** for both dbt (build, test, compile) and validation tools (row count diffs, schema comparison, profiling)
- **Golden scenarios** showing what good output looks like for specific model types
- **Context documents** detailing production table schemas, key fields, and the business intent behind analytics requirements
With these foundations in place, the AI can work with context rather than guessing. Without them, every output requires heavy correction. The setup phase is where the most thought-intensive work happens — and it is collaborative by design. The data team needs to define what they expect before the AI can deliver it.
## What Does the AI Get Right?
Given proper setup, the results are genuinely impressive. The AI follows naming conventions, uses CTEs as expected, organizes folders correctly, and even makes intelligent decisions the developer did not explicitly request — like making certain intermediate tables incremental after inferring from data patterns that the source tables were append-only.
The AI also creates its own verification plan. Row count diffs between dev and production, statistical profiling (min, max, average, distinct counts), distribution histograms. Using Recce MCP, it compares environments and runs dbt tests as part of the workflow. Setting that up manually takes real time, and the AI does it because the tools and skills are available.
## Where Would You Not Trust AI-Generated dbt Models?
"It ran" and "I'd ship this" are fundamentally different standards. Several categories of mistakes emerged during review:
| Issue | Why It Matters | Guardrail |
| ----------------------------------- | ---------------------------------------------------------------- | ------------------------------------------------------------- |
| Inner joins where left joins belong | Silently drops rows on edge cases — bites you six months later | Default to left joins; require documented reasons for inner |
| Ignoring existing models | Rebuilt date logic from scratch when `dim_dates` already existed | Add repo conventions to skills: don't reconstruct what exists |
| Generic descriptions | "Timestamp of creation" is useless for a semantic layer | Require business-contextual descriptions |
| Silent data quality decisions | Filtered out rows with missing `org_id` instead of flagging | Missing foreign keys should be flagged, not silently handled |
| Wrong model routing | Mart models pulled from staging instead of intermediate tables | Enforce the medallion layer contract in skills |
The most concerning pattern is silent data quality decisions. An `org_id` should never be missing. If it is, that is potentially a production bug. The AI made a judgment call that should have been flagged, not handled. That kind of decision should be documented in the YAML as a conscious choice — not an AI deciding on its own that missing data is unimportant.
This is exactly why [data review best practices](/ai-blog/data-review-best-practices/) matter more, not less, in AI-assisted workflows. Every decision the AI makes needs the same scrutiny as a human-written model.
## How Does the Iteration Loop Change with AI?
The real value is not the one-shot generation. It is the iteration and documentation loop that follows:
- Every bad join becomes a rule in the skills file
- Every ignored existing model becomes a convention
- Every silent data quality decision becomes an explicit guardrail
- The AI can update those skills itself based on session discussions
The skills get better every time. The next run on a different set of production tables will be tighter because of everything caught on the current one. New team members do not need deep contextual knowledge to get quality output — the accumulated guardrails do the teaching.
This feedback loop means AI-assisted development gets more reliable with each project. But it requires someone reviewing every decision in early iterations. The [pull request review process](/ai-blog/how-to-write-a-good-dbt-pull-request/) remains essential — perhaps more essential, because at least when a human makes a bad join, they know they made it.
## How Does Data Validation Fit Into AI-Generated Workflows?
Manual verification has to happen regardless of how the code was written. Running queries in notebooks, comparing to known data, checking distributions. But AI-assisted workflows can integrate validation directly into the generation loop.
With Recce MCP available as a tool, the AI runs row count diffs, profile diffs, and schema comparisons as part of its build process — not as an afterthought. This catches the [kinds of issues that dbt tests miss](/ai-blog/why-dbt-data-wrong-when-tests-pass/): row count drops, distribution shifts, schema changes that technically pass all constraints.
The combination of generation tools (dbt MCP) and validation tools (Recce MCP) creates a tighter feedback loop than either provides alone. The AI builds a model, validates it against production, catches a row count discrepancy, investigates, and adjusts — all in a single session.
## What Are the Realistic Time Savings?
For a first-time data ingestion project, the time savings are modest. The Snowflake configuration debugging saved an afternoon. The iteration loop was faster than starting from blank files. But the one-shot itself required every bit as much review as hand-written code — maybe more, because the mistakes are less predictable.
The compounding value comes from the skills and guardrails that accumulate across projects. The first project is heavy on review. The second is lighter. The tenth runs with guardrails that encode months of decisions. AI-assisted analytics engineering is an investment in infrastructure, not a shortcut on any single project.
The decisions that matter are still yours. The AI handles the mechanical work. The judgment — which joins, which filters, which data quality rules — remains human.
## FAQ
**Q: Can AI build dbt models end-to-end?**
A: Yes, AI tools like Claude Code can generate dbt sources, staging, intermediate, and mart models that compile and run. However, the generated code requires careful human review. Common issues include wrong join types (inner instead of left), ignoring existing models in the repo, generic descriptions insufficient for semantic layers, and silent data quality decisions like filtering out rows with missing foreign keys without flagging them.
**Q: What setup is required before letting AI generate dbt models?**
A: Effective AI-assisted dbt development requires significant upfront investment: custom skills defining naming conventions, primary key patterns, model structure, and dev environment commands. MCP integrations for both dbt and data validation tools. Golden scenarios showing what good output looks like. The setup phase is where the most thought-intensive work happens — the generation itself is the easy part.
**Q: What mistakes does AI make when writing dbt models?**
A: Common AI mistakes in dbt model generation include using inner joins where left joins are safer, ignoring existing models and rebuilding logic from scratch, writing generic column descriptions instead of business-contextual ones, silently filtering out rows with missing values instead of flagging potential data quality issues, and routing mart models through staging instead of intermediate tables the AI itself created.
**Q: How does data validation help when AI generates dbt models?**
A: Data validation tools like Recce MCP provide automated cross-environment comparison during AI-assisted development. The AI agent can run row count diffs, statistical profiling, and distribution histograms as part of its workflow, catching issues that dbt tests alone would miss. This transforms validation from a manual post-hoc step into an integrated part of the AI generation loop.
---
# What Is Guided Data Review for dbt Pull Requests?
> Guided data review uses context engineering to tell dbt PR reviewers what changed, why it matters, and what to validate. Learn how it solves the "where do I start?" problem in data PR reviews.
Date: 2026-03-31
Source: https://blog.reccehq.com/guided-data-review
Tags: data-review, dbt, AI, workflows
## Why Data PR Reviews Are Uniquely Difficult
Every data engineer knows the feeling: you open a pull request, see a lineage diff with 5+ impacted models, and freeze. Data PR review is fundamentally harder than code review because a single upstream change can cascade through dozens of downstream models, metrics, and dashboards. The reviewer has to answer not just "is this code correct?" but "what did this change do to the actual data?"
Without guidance, reviewers default to one of two bad patterns. They either check everything (wasting hours running unnecessary diffs) or check almost nothing (hoping dbt tests will catch problems). Neither approach scales, and both leave teams exposed to data quality regressions that [pass tests but still produce wrong results](/ai-blog/why-dbt-data-wrong-when-tests-pass/).
## What Is Guided Data Review?
Guided data review is an approach that provides reviewers with an intelligent, context-aware summary of what changed in a data PR, why it matters, and what specific validations they should perform before merging. Rather than leaving reviewers to figure out where to start, guided review meets them where they already work: in the pull request itself.
The concept emerged from a key observation: data teams do not start their review workflow inside a validation tool. They start in the PR. Even teams using open-source Recce had hacked together ways to post a Recce Summary to every PR, showing which checks ran and their results. The problem was that this summary only showed results after someone had already done the work. What reviewers needed was guidance _before_ they started.
## How Does Context Engineering Power Guided Review?
The difference between a useful AI review summary and a generic one comes down to context engineering versus simple prompt engineering.
| Approach | Input | Output Quality |
| ----------------------- | ----------------------------------------------------------------------------------- | ------------------------------------------------- |
| **Prompt engineering** | PR description, commit messages | Generic summaries that restate the PR description |
| **Context engineering** | dbt artifacts, metadata dependencies, tool access for data checks, domain knowledge | Specific, data-driven, actionable guidance |
With context engineering, an AI agent receives:
- **dbt artifacts**: The metadata and dependency graph for both base and PR branches
- **Tool access**: The ability to run Recce checks such as value diffs, profile diffs, and custom queries
- **Analysis capabilities**: Impact radius calculation and downstream impact tracing
- **Domain knowledge**: Understanding of dbt, data warehousing, and analytics engineering patterns
This enables the agent to actually perform checks rather than just suggest them. It can tell a reviewer: "The `customer_lifetime_value` column decreased by 15% because returned orders are now excluded. Here are the downstream models affected." That is actionable. A prompt-engineered summary saying "this PR modifies CLV calculations" is not.
## Who Benefits from Guided Data Review?
Different team members need different things from a review summary, and this is one of the core challenges guided review addresses.
| Reviewer Persona | What They Need |
| ----------------------------------- | ------------------------------------------------------------------------- |
| **Developer reviewing own work** | Confirmation that intended changes look correct, flag anything unexpected |
| **Teammate reviewing a PR** | Quick assessment of merge safety, specific checks to run |
| **Team lead managing multiple PRs** | High-level risk assessment, which PRs need deeper investigation |
| **Early-stage team** | What new insights does this PR unlock? |
| **Mature production team** | What could break? What anomalies need investigation? |
Guided review adapts to these personas by providing layered information: a quick yes/no merge recommendation with supporting evidence, plus detailed check results for those who want to dig deeper. This approach connects directly to broader [data review best practices](/ai-blog/data-review-best-practices/) that emphasize scoping validation to what actually matters.
## How Does Guided Review Fit into the dbt Workflow?
Guided data review integrates at the PR stage of the standard dbt development cycle:
1. **Develop**: Engineer modifies dbt models locally
2. **Create PR**: Code changes trigger metadata generation
3. **Guided review appears**: An AI-assisted summary posts as a PR comment, showing what changed in the data, the [impact radius](/ai-blog/what-is-impact-radius/) of changes, and recommended validations
4. **Validate**: Reviewers follow the guidance to run targeted checks
5. **Merge**: Team merges with confidence that impacts are understood
The key innovation is step 3. Instead of a blank canvas where the reviewer must figure out what to check, they receive a structured starting point grounded in actual data analysis.
## What Makes Guided Review Different from Checklists and Templates?
Teams have tried several approaches to structure data PR reviews: in-app bubble guides, rule-based suggestion engines, user preference settings, and template-based checklists. These approaches share a common weakness: they are rigid and cannot adapt to the unique context of each PR.
An in-app guide tells you the same three steps regardless of whether your PR touches one staging model or restructures an entire mart layer. A checklist cannot know that _this specific change_ reduces customer lifetime value by 15% and downstream dashboards will show different segment distributions.
Guided review powered by context engineering adapts to each PR because it actually analyzes the change, runs checks against real data, and synthesizes findings based on what the specific change impacts.
## Getting Started with Guided Data Review
Recce Cloud ships guided data review as an AI-assisted PR summary. When a PR is created against a dbt project connected to Recce, the agent analyzes the change and posts a comment summarizing what changed, what the data impact looks like, and what the reviewer should validate.
For teams already doing [dbt CI checks beyond basic tests](/ai-blog/what-should-dbt-ci-check-beyond-tests/), guided review is the natural next step: it turns your CI metadata into actionable review intelligence instead of raw check output that reviewers must interpret on their own.
## FAQ
**Q: What is guided data review for dbt?**
A: Guided data review is an approach that uses context engineering and AI to automatically analyze dbt pull requests, telling reviewers what data changed, why it matters, and what they should validate before merging.
**Q: Why do data PR reviews need guidance?**
A: Data PR reviews are uniquely difficult because a single upstream code change can impact 5 or more downstream models. Reviewers see a lineage diff but have no systematic way to decide where to start validating, leading to missed issues or wasted time on irrelevant checks.
**Q: What is the difference between prompt engineering and context engineering for data review?**
A: Prompt engineering gives an LLM only PR descriptions and commit messages. Context engineering provides dbt artifacts, metadata dependencies, tool access to run actual data checks like profile diffs and value diffs, and domain knowledge about data warehousing, producing far more actionable review summaries.
**Q: How does Recce implement guided data review?**
A: Recce posts an AI-assisted summary as a PR comment that describes what changed in the data, why it matters, and what the reviewer should check. The agent uses context engineering to access dbt artifacts, run Recce checks, calculate impact radius, and trace downstream impacts.
---
# Why Do Data Teams Struggle with Tool Adoption?
> Data teams often love a tool in demos but abandon it weeks later. Learn why setup complexity, wrong adoption sequences, and cognitive load create adoption barriers, and how value-first design solves them.
Date: 2026-03-31
Source: https://blog.reccehq.com/we-built-something-data-teams-wanted-but-couldnt-setup
Tags: workflows, adoption, dbt, best-practices
## The Adoption Gap in Data Tooling
A familiar pattern plays out across data teams: an analytics engineer sees a tool demo, says "this is exactly what we need," and then quietly abandons it two weeks later. The tool worked perfectly in the demo. The concept was right. The problem it solved was real. But somewhere between "this is brilliant" and daily reality, adoption collapsed.
Tool adoption failure in data teams is rarely about the tool's core capabilities. It is almost always about the gap between what a tool requires for setup and what data practitioners are willing or able to invest before seeing value.
## Why Does Setup Complexity Kill Data Tool Adoption?
The root cause is a mismatch between the skills data practitioners have and the skills tools demand for setup. Analytics engineers are experts in SQL, dbt, and data modeling. Most data validation tools require them to also become proficient in:
- Docker containers and Dockerfiles
- CI/CD pipeline configuration (GitHub Actions, GitLab CI)
- Secrets management and credential configuration
- Environment orchestration and artifact management
- Port forwarding and networking concepts
Each of these is a reasonable prerequisite from an engineering perspective. But stacked together, they represent an enormous cognitive load that has nothing to do with the actual task of validating data changes. Data teams just want to know what changed in their data models. None of that goal requires understanding Docker.
## What Happens When Teams Hit the Complexity Wall?
When adoption friction is high, teams follow a predictable degradation pattern:
1. **Install locally**, run the tool a few times during development
2. **Hit the automation wall** when trying to integrate into CI/CD
3. **Abandon local usage** because running manually every time is too much overhead
4. **Default to PR-only usage** where CI automation handles everything
5. **Celebrate partial success** while missing the full validation potential
This pattern means teams end up solving roughly 30% of their validation problems while the tool they adopted could address 80%. Issues that could be caught locally in seconds during development slip through to PR review, where they require production-scale data and formal documentation to investigate.
The cruel irony is that teams work harder than necessary while missing opportunities to catch issues earlier, all because setup complexity blocks them from using the full capabilities they are already paying for.
## Why Do Teams Prioritize PR Review Over Local Validation?
It might seem irrational for teams to invest setup effort only in PR-time validation while abandoning the local development workflow. But the reasoning is sound when you consider their constraints.
| Validation Stage | Value | Setup Burden | Typical Outcome |
| --------------------- | ----------------------------------------- | ------------------------------ | ---------------------------------- |
| **Local development** | Catch issues early, fast iteration | High (manual environment prep) | Abandoned due to friction |
| **PR review** | Systematic validation, team collaboration | Lower (CI handles automation) | Adopted because CI automates setup |
PR review is where the validation gap hurts most. During development, engineers have workarounds: spot checks, row count comparisons, quick queries. But during PR review, reviewers have no systematic way to see what changed in the data. They are flying blind on whether changes are safe to merge.
Teams focus their limited setup energy on solving their biggest pain point first. That rational prioritization means they never get around to optimizing the development-time validation that would actually save them the most time. Understanding [what dbt CI should check beyond tests](/ai-blog/what-should-dbt-ci-check-beyond-tests/) becomes critical for these teams.
## What Does a Value-First Adoption Path Look Like?
The alternative to "setup everything first" is value-first adoption, where teams experience meaningful utility at each step before being asked to invest in the next level of integration.
A well-designed value-first path for data validation looks like this:
1. **Immediate exploration** (zero setup): Explore validation workflows with sample data to understand the tool's capabilities
2. **Metadata upload** (minimal setup): Upload production and development metadata to see actual changes and [impact radius](/ai-blog/what-is-impact-radius/) in your own project
3. **Warehouse connection** (moderate setup): Connect your data warehouse to unlock data-level diffing and custom queries
4. **Git integration** (moderate setup): Connect your GitHub or GitLab repo for PR-based validation workflows
5. **CI/CD automation** (advanced setup): Automate metadata uploads and trigger validation checks on every PR
Each step delivers standalone value. A team that stops at step 2 still gets meaningful insight into what their changes impact. A team at step 3 can run targeted [data diffs](/ai-blog/what-is-a-data-diff/) without any CI/CD knowledge.
## Common Adoption Anti-Patterns in Data Tools
Beyond setup complexity, several anti-patterns contribute to adoption failure:
| Anti-Pattern | Example | Why It Fails |
| ----------------------------- | ------------------------------------------------- | ---------------------------------------------------------------- |
| **Prerequisite overload** | Require Docker + CI/CD + secrets before first use | Users never reach the value |
| **Wrong adoption sequence** | Assume local-first, then automate | Users need PR validation most urgently |
| **Shifting complexity** | Replace local setup with cloud container setup | Different complexity is still complexity |
| **Documentation-as-solution** | Point struggling users to docs | Setup friction is a product problem, not a documentation problem |
| **Power-user default** | Design for the 5% who can write CI pipelines | Alienate the 95% who cannot |
## How Recce Approaches Value-First Adoption
Recce Cloud was redesigned around the principle that data teams should never need to become DevOps engineers to validate their data. The approach eliminates the traditional setup barriers by separating production metadata (which already exists from deployment pipelines) from development metadata (which is the only piece that needs generating per PR).
This means teams can launch Recce with just two metadata files. No Docker. No local environment setup. No CI/CD expertise required upfront. Teams see their actual data changes instantly, then decide if they want to integrate deeper into their [dbt pull request workflow](/ai-blog/how-to-write-a-good-dbt-pull-request/).
The lesson extends beyond any single tool: data teams adopt tools that respect their expertise and deliver value before demanding infrastructure work. Tools that ask analytics engineers to become DevOps engineers will always struggle with adoption, no matter how good their core capabilities are.
## FAQ
**Q: Why do data teams abandon tools they initially love?**
A: Data teams abandon tools primarily due to setup complexity. Analytics engineers are asked to become experts in Docker, CI/CD pipelines, and DevOps infrastructure just to use a data validation tool. The cognitive load of learning infrastructure blocks them from reaching the value they saw in a demo.
**Q: What is a value-first adoption strategy for data tools?**
A: A value-first adoption strategy delivers immediate utility before requiring complex setup. Instead of asking users to configure CI/CD, containers, and credentials upfront, it lets them experience core value first, then incrementally add integrations as they see the benefit of each step.
**Q: Why do data teams default to PR-only validation?**
A: Data teams default to PR-only validation because CI/CD automation handles everything automatically at that stage with no manual setup on their part. Even though catching issues during local development is faster and cheaper, the setup burden for local validation drives teams to validate only during code review.
**Q: How can data tools reduce adoption barriers?**
A: Data tools can reduce adoption barriers by eliminating infrastructure prerequisites, providing immediate value with minimal configuration such as metadata-only uploads, and designing adoption paths that let teams go deeper only when they choose to, not because the tool requires it.
---
# Why Do dbt MCP Workflows Need a Separate Data Validation Layer?
> dbt MCP handles building and testing models but cannot compare branch output against production. Learn why MCP-based dbt workflows need a dedicated validation layer for cross-environment data diffs.
Date: 2026-03-31
Source: https://blog.reccehq.com/the-validation-gap-in-dbt-mcp-workflows
Tags: data-validation, dbt, mcp
## What Is the Validation Gap in dbt MCP Workflows?
Forty percent of records disappeared because a JOIN condition silently filtered them out. `dbt build` passed. Tests passed. The SQL compiled. Everything a build tool could verify checked out — and none of it caught the problem.
This is the gap between "it builds" and "it's correct," and every data team running dbt Model Context Protocol (MCP) hits it eventually. Not because dbt MCP is flawed. Because validation is a fundamentally different job than building.
dbt MCP is excellent at what it does: ten tools covering the core development loop — `build`, `run`, `test`, `compile`, `parse`, `show`, lineage tracing, metadata inspection. That is a complete development toolkit. But it talks to one environment at a time. Comparing what a branch produces against what is already in production requires stepping outside the tool's design.
## How Does dbt MCP Differ from a Data Validation Tool?
The distinction shows up in everyday development tasks. When an AI agent has both dbt MCP and Recce MCP available, the task determines which server it reaches for.
| Task | Tool | Why |
| ------------------------------------- | ----------------------------- | ---------------------------------------- |
| Preview model output | dbt MCP `show` | Single-environment exploration |
| Trace DAG dependencies | dbt MCP `get_lineage_dev` | Static lineage within one environment |
| Scaffold schema YAML | dbt MCP `generate_model_yaml` | Build-time concern |
| Compare row counts against production | Recce MCP `row_count_diff` | Cross-environment comparison |
| Detect schema changes from a refactor | Recce MCP `schema_diff` | Compares current vs. production columns |
| Show impact radius of a change | Recce MCP `lineage_diff` | Change-annotated DAG, not static lineage |
| Run a reusable validation suite | Recce MCP `run_check` | Preset checks defined in YAML |
The handoff happens when the task shifts from building to validating. dbt MCP's `get_lineage_dev` shows the static DAG — the same graph regardless of what changed. Recce's [lineage diff](/ai-blog/what-is-dbt-dag-lineage/) annotates each node with a change status (added, removed, modified) and flags downstream models as impacted. One is a map. The other is a map with the change marked on it.
## What Is the DIY Cost of Cross-Environment Validation?
Any analytics engineer can write a comparison query. The problem is not difficulty. The problem is that the queries are slightly different every time, live in different places (PR comments, Slack threads, personal notebooks), and no one inherits them when a new team member joins.
**Row count diff — the manual way:**
Two queries. Two target switches. Manual comparison. For two models, this is manageable. For every modified model in a PR, it becomes a chore nobody does consistently.
**Profile diff — worse:**
The aggregation SQL grows per column, per model. Min, max, average, median, distinct count, null proportion — each requires its own expression, repeated for every environment.
**Query diff with primary key matching — the worst:**
Run the same query on both schemas, JOIN on primary keys, compare every column for differences, surface additions, removals, and changes. The JOIN logic varies by model. Most teams skip this entirely.
| Factor | DIY with dbt show | Recce MCP |
| -------------------- | -------------------------------------------- | -------------------------------------------- |
| Queries to write | 2 per comparison (base + current) | 1 call |
| Target switching | Manual or schema hardcoding | Automatic |
| Output format | Raw rows, manually compared | Structured diff with base and current values |
| Primary key matching | Custom JOIN logic per model | Built-in parameter |
| Repeatability | Save queries somewhere, remember to run them | Preset checks in YAML, run on every PR |
| Coverage | Whatever someone remembers to check | Systematic for defined checks |
One forgotten check is one silent regression. The [data diff](/ai-blog/what-is-a-data-diff/) that catches a problem is only useful if someone actually runs it.
## How Does Recce MCP Work as a Validation Layer?
Recce MCP exists to do one thing dbt MCP cannot: compare two environments at once. It reads two sets of dbt artifacts — `target/manifest.json` and `target/catalog.json` for the current branch, and `target-base/` equivalents for production. Every tool operates across both, returning structured diffs.
Eight tools cover the validation surface:
**Environment comparison (the core):**
- `row_count_diff` — Row counts for base vs. current, side by side
- `profile_diff` — Statistical profiles for every column, both environments, one call
- `query_diff` — Execute SQL on both environments and compare row by row with primary key matching
- `schema_diff` — Column-level changes across all modified models
**Change-aware lineage:**
- `lineage_diff` — The DAG annotated with change status and [impact radius](/ai-blog/what-is-impact-radius/)
**Preset checks:**
- `list_checks` and `run_check` — Execute validation suites defined in YAML, run on every PR
**Flexible querying:**
- `query` — Execute SQL with Jinja support against either environment
Where dbt MCP asks "does this model build correctly?", Recce MCP asks "does this model produce the right data compared to what is already in production?"
## How Do Both Servers Run Together?
Both servers run side by side in a single `.mcp.json` configuration:
```json
{
"mcpServers": {
"dbt": {
"type": "stdio",
"command": "uvx",
"args": ["dbt-mcp"]
},
"recce": {
"type": "stdio",
"command": "recce",
"args": ["mcp-server"]
}
}
}
```
No integration work beyond the config. Both servers are available to any MCP-compatible AI agent. The agent decides which server to use based on the task at hand.
One prerequisite: Recce MCP reads production artifacts from a `target-base/` directory. In CI, a workflow step typically builds main and stores its artifacts there. Recce Cloud handles artifact management automatically for teams using CI integration.
## When Should Data Teams Add a Validation Layer?
The validation gap becomes harder to ignore as teams adopt AI-assisted dbt development. dbt tests check correctness against rules: not-null constraints, unique keys, accepted values. But as covered in [why dbt data can be wrong when tests pass](/ai-blog/why-dbt-data-wrong-when-tests-pass/), those rules do not catch row count drops, distribution shifts, or silent schema changes.
Cross-environment comparison catches what rule-based tests miss. The development loop has always had two halves — build and validate. dbt MCP handles the build. A validation layer like Recce MCP handles the second half. Both servers, one config, and the gap between "it builds" and "it's correct" closes.
## FAQ
**Q: What is the validation gap in dbt MCP workflows?**
A: The validation gap is the difference between "it builds" and "it is correct." dbt MCP connects to one environment at a time and verifies that models compile, build, and pass tests. It cannot compare branch output against production data. This means a JOIN that silently drops 40% of records will pass dbt build and dbt test but ship incorrect data.
**Q: Can dbt MCP compare data between dev and production environments?**
A: No. dbt MCP talks to one environment at a time — either a dev schema or a prod schema, not both simultaneously. Comparing what a branch produces against what is already in production requires stepping outside dbt MCP and using a tool designed for cross-environment comparison, such as Recce MCP.
**Q: What is the difference between dbt MCP and Recce MCP?**
A: dbt MCP is a build engine that handles model development: compile, run, test, preview, and inspect lineage within a single environment. Recce MCP is a validation layer that compares two environments, returning structured diffs for row counts, schemas, statistical profiles, and query results. dbt MCP answers "does this model build?" while Recce MCP answers "does this model produce the right data compared to production?"
**Q: How do dbt MCP and Recce MCP work together in an AI agent workflow?**
A: Both servers run side by side in a single .mcp.json configuration file, available to any MCP-compatible AI agent. The agent uses dbt MCP for discovery and development tasks (previewing output, tracing DAG dependencies, scaffolding YAML) and switches to Recce MCP when the task shifts to validation (row count diffs, schema diffs, profile comparisons, impact radius analysis).
---
# How to Use Claude Code for dbt Analytics Engineering
> A practical account of using Claude Code to build an end-to-end dbt analytics warehouse, from Snowflake ingestion to mart models, and why the setup infrastructure matters more than the prompt.
Date: 2026-02-27
Source: https://blog.reccehq.com/i-let-claude-code-build-my-dbt-models.-the-interesting-part-wasnt-the-code
Tags: ai, dbt, build-in-public
## What Does It Take to Use Claude Code for dbt?
Using an LLM to generate dbt models is not a prompting problem — it is an infrastructure problem. Before Claude Code can produce useful analytics engineering output, it needs custom skills: explicit rules for naming conventions, primary key patterns, model structure, dev environment commands, and what "good" data models look like in your project.
This goes beyond writing a system prompt. A practical setup includes:
- **Naming conventions and folder structure** so generated models match existing project patterns
- **MCP integrations** — dbt MCP for project context and Recce MCP for data validation (row count diffs, schema comparison, profiling)
- **Golden scenarios** — examples of expected model output that Claude can reference as ground truth
The setup phase is where the actual engineering happens. The generation step is the easy part.
## How Well Does Claude Code Handle End-to-End dbt Generation?
In a real-world test building an analytics warehouse from Snowflake ingestion through mart models, Claude Code handled the full pipeline: creating Snowflake tables, loading data from S3, and building sources, staging, intermediate, and mart layers following a dimensional model organized into medallion architecture layers.
The results were structurally sound. Claude Code:
- Followed project naming conventions and used CTEs consistently
- Organized files into folders matching the existing project structure
- Inferred that certain intermediate tables should be incremental based on append-only data patterns — without being told
- Created its own verification plan using [data diffs](/ai-blog/what-is-a-data-diff/) — row count comparisons, profiling (min, max, average, distinct counts), and distribution histograms via the Recce MCP
The code compiled and ran. But "it ran" and "I'd ship this" are different things.
## Where Does AI-Generated dbt Code Need Human Review?
Every AI-generated dbt model needs review. Common issues that appeared in practice:
| Issue | Risk | What Should Happen |
| -------------------------------------------------------------- | ------------------------------------------------------- | ----------------------------------------------------------------------- |
| Inner joins used where left joins are appropriate | Silently drops rows on edge cases — breaks months later | Default to left joins; require explicit justification for inner joins |
| Generic column descriptions | Inadequate for semantic layer consumption | Descriptions must reflect business context, not just technical metadata |
| Mart models referencing staging instead of intermediate tables | Bypasses incremental logic the AI itself created | Enforce layer dependencies in skills |
| Missing edge cases in row-number logic | Incorrect deduplication for multi-connection entities | Document known limitations rather than shipping silently |
These are not unusual mistakes — they mirror what a distracted human engineer might produce. The difference is that when you write a bad join yourself, you at least know you wrote it. With AI-generated code, every assumption needs explicit verification.
## What Role Does MCP Play in AI-Assisted dbt Development?
MCP (Model Context Protocol) is what separates "AI writing SQL" from "AI doing analytics engineering." Without MCP, Claude Code generates models blind — it can only reason about what the SQL should do. With MCP integrations, it can validate what the SQL actually does.
Two MCP servers make the difference:
- **dbt MCP** — provides project metadata, compilation context, and model dependencies
- **Recce MCP** — enables [data review](/ai-blog/data-review-best-practices/) capabilities: row count diffs between dev and prod, schema comparison, column profiling, and distribution histograms
When Claude Code has access to Recce MCP, it builds verification into its own workflow. Instead of generating models and hoping they work, it runs dbt tests, compares row counts across environments, and profiles column distributions — the same checks a human [data reviewer](/ai-blog/what-is-ai-data-review-agent/) would perform.
This matters because incorporating MCPs as part of customized skills and workflows produces better outcomes than prompting alone.
## Why Is the Iteration Loop the Real Value?
The one-shot generation is impressive to watch but provides modest time savings on its own. The compounding value comes from the iteration and documentation loop:
- Every bad join becomes a rule in the skills file
- Every ignored existing model becomes an enforced convention
- Every silent data quality decision becomes a guardrail
- Every edge case becomes a documented known limitation
Each run tightens the skills. The next batch of production tables produces fewer issues because the previous batch's mistakes are now encoded as constraints. Over time, teammates who lack deep contextual knowledge of the data can produce the same quality output because the skills carry that institutional knowledge.
This is the same principle behind [CI checks for dbt](/ai-blog/what-should-dbt-ci-check-beyond-tests/) — systematic encoding of quality expectations that compound over time.
## What Did AI Actually Save vs. What Still Requires Manual Work?
| Task | AI Contribution | Human Still Required |
| ------------------------------------------- | --------------------------------------------------------------------- | ------------------------------------------------------------ |
| Snowflake configuration debugging | Diagnosed and fixed issues — saved an afternoon | Verify the fix is correct |
| Model scaffolding (sources, staging, marts) | Generated complete layer structure | Review every join, every reference, every description |
| Verification plan | Built row count diffs, profiling, histograms automatically | Interpret results, query in notebooks, compare to known data |
| Iteration speed | "Build, review, catch problems, discuss" loop faster than blank files | Every decision still needs human judgment |
| Documentation | Skills file captures conventions automatically | Business context must come from domain knowledge |
The honest assessment: manual verification still has to happen regardless. The time savings were modest for the initial data ingestion. The agent might introduce mistakes you wouldn't have made yourself. But the iteration loop — where each mistake becomes a permanent rule — creates compounding returns that grow with every subsequent run.
## Summary
AI-assisted analytics engineering with Claude Code is an infrastructure problem, not a prompting problem. The real work is building custom skills, configuring MCP integrations (dbt MCP for project context, Recce MCP for data validation), and creating golden scenarios. Claude Code can handle end-to-end dbt generation — Snowflake tables through mart models — but the output requires the same review scrutiny as any engineer's code. The compounding value comes from the iteration loop: every mistake becomes a rule, every convention becomes a constraint, and the skills get better with each run. The generation is the easy part. The decisions that matter are still yours.
## FAQ
**Q: Can Claude Code build dbt models end-to-end?**
A: Yes. Claude Code can create Snowflake tables, load data from S3, and build dbt sources, staging, intermediate, and mart models in a single run. It follows naming conventions, uses CTEs, and can infer patterns like incremental materialization from append-only data. However, the output requires human review — issues like incorrect join types, weak descriptions, and inconsistent model references appear regularly.
**Q: What setup does Claude Code need for dbt projects?**
A: Effective dbt generation with Claude Code requires custom skills (naming conventions, primary key patterns, model structure), MCP integrations (dbt MCP for project context, Recce MCP for data validation), and golden scenarios showing expected model output. This setup infrastructure is where the real engineering work happens — not the prompt itself.
**Q: What is MCP and how does it help Claude Code with dbt?**
A: MCP (Model Context Protocol) gives Claude Code access to external tools. The dbt MCP provides project metadata and compilation. The Recce MCP enables row count diffs, schema comparison, and profiling between dev and prod environments. Together, they let Claude Code validate its own output rather than generating code blind.
**Q: What mistakes does Claude Code make when building dbt models?**
A: Common issues include using inner joins where left joins are safer (silently dropping edge-case rows), generating generic column descriptions instead of business-contextual ones, referencing staging models instead of intermediate tables, and missing edge cases in row-number logic. Each mistake becomes a rule in the skills file for the next run.
---
# How to Generate a Time Spine in dbt
> A time spine is a table with one row per time period used for filling gaps in event data. Learn how to generate a time spine in dbt using date_spine, generate_series, and MetricFlow conventions.
Date: 2026-02-23
Source: https://blog.reccehq.com/generating-a-time-spine-in-dbt
Tags: dbt, how-to, data-modeling
## What Is a Time Spine and Why Do You Need One?
A time spine (also called a date spine) is a utility table that contains one row for every time period in a defined range. Most commonly, it has one row per day, but it can be hourly, weekly, or monthly depending on the grain you need.
The purpose is simple: event-based data has gaps. If no orders were placed on January 15th, your orders table has no row for that date. When you aggregate by day, January 15th disappears from the results. A chart built on that query shows a misleading jump from January 14th to January 16th instead of a zero on the 15th.
A time spine fixes this. By LEFT JOINing from the spine to your event data, every period gets a row — periods with events get actual values, periods without events get zero or null. This is essential for accurate time series reporting, dashboard visualizations, and metric calculations.
## What Goes Wrong Without a Time Spine?
Consider a simple query that counts daily orders:
```sql
SELECT order_date, COUNT(*) AS order_count
FROM orders
GROUP BY order_date
ORDER BY order_date
```
This returns rows only for dates that have orders. The gaps cause three problems:
1. **Charts mislead** — line charts connect adjacent points, so a gap between Tuesday and Thursday looks like Wednesday had the same trend, when in reality there were zero orders.
2. **Aggregations break** — a rolling 7-day average that skips days produces incorrect results because it averages over fewer data points than expected.
3. **Metrics layer errors** — MetricFlow (dbt's semantic layer) requires a time spine to correctly calculate cumulative and derived metrics. Without one, metric queries fail or return incorrect values.
## How Do You Generate a Time Spine with dbt-utils?
The most portable approach uses the date_spine macro from dbt-utils. Create a model in your project:
```sql
-- models/utilities/time_spine.sql
{{ config(materialized='table') }}
WITH spine AS (
{{ dbt_utils.date_spine(
datepart="day",
start_date="cast('2020-01-01' as date)",
end_date="cast('2026-12-31' as date)"
) }}
)
SELECT
date_day
FROM spine
```
This generates one row per day from January 1, 2020 through December 31, 2026. The `date_spine` macro works across all dbt-supported warehouses because it uses cross-joins or recursive logic internally rather than database-specific functions.
## Can You Use generate_series Instead?
For databases that support it, the native `generate_series` function is more performant:
```sql
-- Postgres / DuckDB
SELECT generate_series(
'2020-01-01'::date,
'2026-12-31'::date,
'1 day'::interval
)::date AS date_day
```
generate_series is a SQL function that produces a set of values from a start to an end at a specified interval. It runs natively in the database engine, making it faster than the macro approach. The tradeoff is portability — this syntax does not work on Snowflake, BigQuery, or Databricks.
## What About Databases Without Native Series Generation?
For databases that lack both `generate_series` and efficient cross-join support, a recursive CTE approach works:
```sql
WITH RECURSIVE date_series AS (
SELECT CAST('2020-01-01' AS DATE) AS date_day
UNION ALL
SELECT DATEADD(day, 1, date_day)
FROM date_series
WHERE date_day < '2026-12-31'
)
SELECT date_day FROM date_series
```
This starts with a seed date and adds one day at a time until it reaches the end date. It is the least performant approach for large ranges but works on virtually every SQL database.
## Which Approach Works Best for Each Warehouse?
| Warehouse | Recommended Method | Notes |
| -------------- | ----------------------------------------------- | ----------------------------------------------------------------------------- |
| **Snowflake** | `dbt_utils.date_spine` | No native `generate_series`; macro uses `GENERATOR` table function internally |
| **BigQuery** | `GENERATE_DATE_ARRAY` or `dbt_utils.date_spine` | BigQuery has `GENERATE_DATE_ARRAY` which is native and performant |
| **Postgres** | `generate_series` | Native function, most performant option |
| **DuckDB** | `generate_series` | Same syntax as Postgres |
| **Redshift** | `generate_series` or `dbt_utils.date_spine` | Redshift supports `generate_series` but with some limitations on large ranges |
| **Databricks** | `SEQUENCE` or `dbt_utils.date_spine` | `SEQUENCE` function generates arrays that can be exploded into rows |
If your project runs on a single warehouse, use the native approach for that platform. If you need portability across warehouses, `dbt_utils.date_spine` is the safest choice.
## What Does MetricFlow Require for the Time Spine?
dbt's semantic layer (powered by MetricFlow) requires a time spine table to calculate cumulative metrics, derived metrics, and fill-values. The required format is specific:
```sql
-- models/utilities/metricflow_time_spine.sql
{{ config(materialized='table') }}
WITH spine AS (
{{ dbt_utils.date_spine(
datepart="day",
start_date="cast('2020-01-01' as date)",
end_date="cast('2026-12-31' as date)"
) }}
)
SELECT
date_day,
DATE_TRUNC('week', date_day) AS date_week,
DATE_TRUNC('month', date_day) AS date_month,
DATE_TRUNC('quarter', date_day) AS date_quarter,
DATE_TRUNC('year', date_day) AS date_year
FROM spine
```
MetricFlow looks for a model registered as a time spine in your `dbt_project.yml` or semantic manifest. The model must have `date_day` as the primary time column. Adding truncated columns for other grains allows MetricFlow to aggregate metrics at weekly, monthly, or quarterly levels without additional models.
## Where Should the Time Spine Live in Your Project?
Most teams place the time spine in a `utilities` or `staging` directory:
```
models/
staging/
...
utilities/
time_spine.sql
intermediate/
...
marts/
...
```
The time spine is not a staging model (it does not clean raw data) and not a mart (it is not business-facing). It is infrastructure — a utility that other models reference. Placing it in a dedicated `utilities` folder makes its purpose clear in your [DAG](/ai-blog/what-is-dbt-dag-lineage/).
**Grain selection**: daily is the most common grain because it balances storage and utility. Hourly spines are large but necessary for real-time dashboards. Weekly or monthly spines are rare as standalone models — it is simpler to truncate a daily spine.
**Range management**: hardcoded start and end dates are simple but require manual updates. Dynamic ranges using `MIN(date)` from source data and `CURRENT_DATE` are more maintainable but add a dependency on source freshness.
## How Do You Validate Time Spine Changes?
Modifying a time spine — changing the date range, switching from daily to hourly grain, or adjusting the generation method — affects every model that joins to it. This makes time spine changes high-impact despite their apparent simplicity.
Before merging a time spine change, validate:
- **Row count** — does the new spine have the expected number of rows? A daily spine from 2020-01-01 to 2026-12-31 should have exactly 2,557 rows.
- **Date range** — does `MIN(date_day)` and `MAX(date_day)` match your intended range?
- **No duplicates** — is `date_day` unique? A duplicate date in the spine will double-count events when joined.
- **Downstream impact** — run [data diffs](/ai-blog/what-is-a-data-diff/) on models that reference the time spine to confirm they still produce correct results. Tools like Recce can compare the output of downstream models between your PR branch and production, surfacing differences caused by the spine change.
These checks are especially important because time spine errors are silent — they do not cause query failures, they cause subtly wrong numbers.
## Summary
A time spine is a utility table with one row per time period, used to fill gaps in event-based data and enable accurate metrics. The most portable generation method is the `dbt_utils.date_spine` macro; databases like Postgres and DuckDB can use native `generate_series` for better performance; recursive CTEs work as a universal fallback. MetricFlow requires a specific time spine format with `date_day` as the primary column. Place the time spine in a `utilities` directory, default to daily grain, and validate changes carefully — especially row counts, date ranges, and downstream data diffs — because time spine errors produce silently wrong results rather than failures.
## FAQ
**Q: What is a time spine in dbt?**
A: A time spine (also called a date spine) is a utility table that contains one row for every time period in a defined range — typically one row per day, but can be hourly, weekly, or monthly. It is used to fill gaps in event-based data so that periods with no events still appear as rows with zero or null values, which is essential for accurate time series reporting and metric calculations.
**Q: How do you generate a time spine in dbt?**
A: The most common approach is using the dbt-utils date_spine macro, which generates a series of dates between a start and end date at a specified interval. For MetricFlow compatibility, dbt recommends a specific time spine model format with date_day as the primary column. You can also use SQL generate_series (Postgres, DuckDB) or recursive CTEs for databases without native series generation.
**Q: Why is a time spine important for metrics?**
A: Without a time spine, metrics queries return no row for periods with zero events, causing gaps in charts and misleading aggregations. A time spine ensures every period has a row, so a LEFT JOIN from the spine to your event data produces zero-filled results. MetricFlow (dbt's metrics layer) requires a time spine table to correctly calculate cumulative and derived metrics.
**Q: What is the difference between date_spine and generate_series?**
A: date_spine is a dbt-utils macro that works across all dbt-supported databases by using a recursive approach or cross-joins internally. generate_series is a native SQL function available in some databases (Postgres, DuckDB, Redshift) that is more performant but less portable. Both produce the same result — a table with one row per time interval.
---
# What Is an AI Data Review Agent?
> An AI data review agent automates dbt PR review by analyzing code changes, running data validations, and generating impact summaries. Learn how multi-agent architecture produces trustworthy reviews.
Date: 2026-02-22
Source: https://blog.reccehq.com/designing-reliable-ai-agents-for-dbt-data-reviews
Tags: ai, data-review, dbt
## What Problem Does an AI Data Review Agent Solve?
Pull request reviews in dbt projects have a fundamental gap: reviewing the SQL tells you what logic changed, but reveals nothing about how the actual data was affected. Engineers spend significant time manually running queries, checking row counts, tracing lineage, and interpreting results before they can say whether a change is safe to merge.
An AI data review agent automates this mechanical work. It analyzes the PR's code changes, runs data validations against actual warehouse output, and generates a human-readable impact summary — all before a human reviewer opens the PR.
Unlike traditional CI checks that report raw numbers (row count: 1,042,387), an AI agent interprets the results: "Row count increased 3.2% due to the new filter including previously excluded records from the APAC region. This aligns with the stated intent of the PR."
## How Does Multi-Agent Architecture Work for Data Review?
A single monolithic agent trying to handle git context, data validation, and analysis synthesis tends to produce inconsistent results. Multi-agent architecture solves this by delegating specific tasks to specialized subagents, each with a narrow scope and focused toolset.
A typical data review agent system uses an orchestrator pattern:
| Agent | Responsibility | Tools Available |
| ------------------------ | --------------------------------------------------------------------------------- | ------------------------ |
| PR Analysis Orchestrator | Coordinates the review workflow, delegates to subagents, synthesizes final report | Task delegation only |
| Git Context Agent | Extracts PR metadata, changed files, commit messages, and modified model names | Git and GitHub API tools |
| Recce Analysis Agent | Runs data validations — lineage diff, schema diff, row count diff, profile diff | MCP tools (Recce server) |
| Synthesis Agent | Combines raw data from other agents into a structured, human-readable summary | Text generation only |
Each subagent runs in an isolated context. The git context agent cannot access warehouse data. The Recce analysis agent cannot modify files. This tool constraint principle — restricting each agent to only the tools it needs — dramatically improves reliability.
## What Role Does MCP Play in AI Data Review?
MCP (Model Context Protocol) provides a standardized interface for AI agents to invoke external tools. In data review, an MCP server exposes Recce's validation capabilities as callable tools:
- `lineage_diff` — compare DAG structure between environments
- `schema_diff` — detect column additions, removals, and type changes
- `row_count_diff` — compare row counts across modified models
- `profile_diff` — compare column-level statistics
- `query_diff` — run arbitrary SQL comparisons
This matters because it lets agents work with real data rather than guessing at impact from code alone. An agent that can only read SQL might guess that a filter change reduces row counts. An agent with MCP access can confirm the row count dropped 12.4% and report which downstream models are affected.
## What Design Principles Make AI Agents Reliable?
Building a reliable AI data review agent requires more than connecting an LLM to tools. Several design principles distinguish agents that produce trustworthy output from those that hallucinate or miss critical issues.
**Specialized agents over general-purpose.** Narrow scope produces consistent output. An agent that only extracts git context will do that well every time. An agent that tries to extract context, run validations, and write analysis in one pass will cut corners under token pressure.
**Show your work.** Require agents to output raw data before generating diagrams or summaries. If a lineage diagram is generated from raw edge data that the orchestrator can verify, hallucinated edges are caught. If the diagram is generated directly, there's no ground truth to check against.
**Negative constraints.** Explicit "do NOT" instructions are surprisingly effective. Telling an agent "do NOT infer column relationships from naming patterns — only report relationships confirmed by lineage_diff output" prevents a common hallucination mode.
**Required output structure.** Mark critical sections of the output format with `[REQUIRED]` markers. Agents are more likely to include sections that are explicitly labeled as non-optional than sections that are merely listed in a template.
## What Makes a Good AI Data Review?
Not all AI-generated reviews are equally useful. Here is a framework for evaluating whether an AI data review agent is producing trustworthy output:
| Quality Criterion | Good Review | Poor Review |
| ------------------------------------ | ------------------------------------------------------------------------------------------------- | -------------------------------------------------- |
| Grounded in data | Cites specific numbers from actual diffs (e.g., "row count increased from 50,412 to 51,823") | Makes vague claims ("row counts may have changed") |
| Scoped to the change | Focuses on models modified in the PR and their direct downstream dependencies | Reports on the entire DAG regardless of relevance |
| Distinguishes intent from regression | Identifies which changes align with the PR description and which are unexpected | Treats all differences as equally noteworthy |
| Actionable next steps | Suggests specific checks a reviewer should run ("verify the APAC region filter in dim_customers") | Ends with generic advice ("review carefully") |
| Transparent about limitations | States what it could not check ("no primary key available for value diff on this model") | Silently skips validations without noting the gap |
| Reproducible | Another run with the same inputs produces the same conclusions | Output varies significantly between runs |
### AI-Assisted vs. Fully Manual Review
The practical impact of an AI data review agent becomes clear when comparing workflows:
| Aspect | Fully Manual Review | AI-Assisted Review |
| --------------------- | ----------------------------------------------------- | ------------------------------------------------------------------------ |
| **Trigger** | Reviewer opens the PR and starts from scratch | Agent runs automatically on PR open |
| **Context gathering** | Read code diff, manually trace downstream models | Agent extracts PR metadata, changed files, and impact scope |
| **Data validation** | Open a SQL editor, write and run comparison queries | Agent runs lineage diffs, schema checks, and profile comparisons via MCP |
| **Interpretation** | Reviewer interprets raw query results | Agent generates a structured summary with cited numbers |
| **Human focus** | Everything — from mechanical checks to judgment calls | Business context, edge cases, and final approval |
| **Typical time** | 30–90 minutes per complex PR | 5–15 minutes of human attention per complex PR |
The agent does not remove the human from the loop. It removes the mechanical work that precedes human judgment. The reviewer still decides whether the change is correct — they just start from an informed position rather than a blank screen.
## How Does This Connect to Data Review Best Practices?
An AI data review agent implements many [data review best practices](/ai-blog/data-review-best-practices/) automatically: scoping the review to the impact radius, running structural checks before drilling into values, and documenting what was checked. The agent's output becomes the first draft of the review checklist that teams refine and approve.
The key insight is that AI agents work best when they have access to real validation data — not just code. Combining [data diffs](/ai-blog/what-is-a-data-diff/) with AI interpretation bridges the gap between raw numbers and actionable review.
## Summary
An AI data review agent automates the mechanical work of dbt PR review: extracting context, running data validations via MCP, and generating structured impact summaries. Multi-agent architecture with specialized subagents produces more reliable output than monolithic approaches. Design principles like tool constraints, required output structure, and negative constraints improve consistency. The result is not a replacement for human review but an informed starting point — reducing review time from hours to minutes while keeping human judgment in the loop for business context and edge cases.
## FAQ
**Q: What is an AI data review agent?**
A: An AI data review agent is an automated system that analyzes dbt pull requests by examining code changes, running data validations (lineage diffs, schema diffs, row count comparisons), and generating impact summaries. Unlike traditional CI checks that report raw numbers, an AI agent interprets the results and produces human-readable analysis of what changed, what might be impacted, and what to look for during review.
**Q: How does a multi-agent architecture improve data review?**
A: A multi-agent architecture delegates specific review tasks to specialized subagents rather than using a single monolithic agent. Separate agents handle git context extraction, data validation execution, and analysis synthesis. Each subagent runs in an isolated context with a narrow toolset, producing more consistent output than a general-purpose agent handling all tasks.
**Q: What is MCP in the context of data review?**
A: MCP (Model Context Protocol) provides a standardized way for AI agents to access external tools and data sources. In data review, MCP servers expose Recce's validation capabilities (lineage_diff, schema_diff, row_count_diff) as tools that AI agents can invoke. This enables agents to run actual data validations rather than guessing at impact from code alone.
**Q: Can AI replace human data reviewers?**
A: AI data review agents augment human reviewers rather than replace them. They automate the mechanical work — fetching PR context, running standard diffs, checking row counts, generating summaries — so human reviewers can focus on business context, edge cases, and judgment calls. The agent handles the first 80% of review effort; humans handle the nuanced 20%.
---
# What Is the dbt DAG? A Guide to Lineage and Dependencies
> The dbt DAG is a directed acyclic graph that maps dependencies between your data models. Learn how to read the DAG, use lineage for impact analysis, and understand the difference between static and diff-aware lineage views.
Date: 2026-02-21
Tags: concepts, dbt, data-lineage
## What Is a Directed Acyclic Graph?
A directed acyclic graph (DAG) is a structure made up of nodes and edges, where each edge has a direction and no path leads back to its starting node. In plain terms: things depend on other things, and those dependencies never form a loop.
In the context of dbt, each node is a model, source, seed, snapshot, or exposure. Each edge is a dependency created by a `ref()` or `source()` call in your SQL. The "directed" part means the relationship has a direction — model B depends on model A, not the other way around. The "acyclic" part means circular dependencies are impossible: if B depends on A, then A cannot also depend on B (directly or through any chain of intermediate models).
## How Does dbt Use the DAG?
Every time you write `{{ ref('some_model') }}` in a dbt model, you create an edge in the DAG. dbt parses all models, resolves all `ref()` calls, and constructs the complete graph before running anything. This graph determines **build order** — dbt executes models in topological order so that every model runs only after its upstream dependencies are complete.
This is why dbt can parallelize builds: models that do not depend on each other can run simultaneously, while dependent models wait for their parents to finish.
## How Do You Read the dbt DAG?
A typical dbt DAG flows left to right through several layers:
| Layer | Description | Examples |
| ---------------- | --------------------------------------- | ------------------------------ |
| **Sources** | Raw data ingested from external systems | `source('stripe', 'payments')` |
| **Staging** | Cleaned, renamed, lightly transformed | `stg_stripe__payments` |
| **Intermediate** | Business logic, joins, aggregations | `int_orders__joined` |
| **Marts** | Final business-facing models | `fct_orders`, `dim_customers` |
| **Exposures** | Downstream consumers (dashboards, ML) | `exposure: revenue_dashboard` |
Reading the DAG from left to right tells you the story of your data: where it comes from, how it is transformed, and where it ends up. Reading right to left tells you the lineage of any specific model — which upstream models contributed to it.
## Why Does DAG Complexity Grow Over Time?
A new dbt project with ten models has a DAG you can understand at a glance. An enterprise project with 500 models has a DAG that looks like a dense web of interconnections. This growth is natural — as a business adds use cases, the DAG accumulates models, cross-references, and shared intermediate logic.
The problem is not complexity itself but the review burden it creates. When you modify a model in a 500-model DAG, understanding which downstream models are affected requires tracing paths through a graph that no human can hold in working memory. This is where tooling becomes essential.
## What Is the Difference Between Static and Diff-Aware Lineage?
Static lineage shows the current state of your DAG — all models and their dependencies as they exist right now. This is what you see in `dbt docs generate` and the dbt Cloud IDE. It answers the question: "What does my project look like?"
Lineage diff compares the DAG between two states — typically your PR branch and production — and highlights what changed. It answers a different and more actionable question: "What did my changes affect?"
| Aspect | Static Lineage (dbt docs) | Lineage Diff |
| ------------ | ----------------------------- | -------------------------------- |
| **Shows** | Current state of all models | Difference between two states |
| **Purpose** | Exploration and documentation | Impact analysis and PR review |
| **Scope** | Entire DAG | Only changed and affected models |
| **Use case** | Understanding the project | Reviewing a specific change |
Static lineage is valuable for onboarding and documentation. But for day-to-day PR review, it forces you to mentally filter hundreds of unchanged models to find the ones that matter. A lineage diff does that filtering for you.
## What Is the Modified+ View?
The modified+ view shows the modified models in your PR plus all their downstream dependents. This represents the potential [impact radius](/ai-blog/what-is-impact-radius/) of your changes — every model that could be affected by what you changed.
Consider an example: you modify `int_orders__joined`. The modified+ view shows that model plus the five mart models and two exposures downstream of it. Instead of scanning the entire DAG, you now have a focused list of seven models to validate.
This scoping is the default starting point for data impact analysis. You examine each model in the modified+ set — checking schema diffs, row counts, and profile diffs — to confirm the change behaved as expected and did not introduce unintended side effects.
## How Does Lineage Diff Help with PR Review?
When reviewing a dbt pull request, the first question is always: "What is the blast radius?" A lineage diff answers this immediately by showing:
- **Which models were directly modified** — the ones the author changed
- **Which models are downstream** — the ones that could be indirectly affected
- **Which models were added or removed** — structural changes to the DAG itself
This information scopes the review. Instead of reading every SQL file in the diff, the reviewer focuses on the modified models and their downstream dependents. For each model in the impact radius, the reviewer checks whether the data changed as expected using [data diffs](/ai-blog/what-is-a-data-diff/) — schema comparisons, row count checks, and value-level validation.
Without a lineage diff, reviewers either check too little (only the directly modified models, missing downstream breakage) or too much (the entire project, wasting time on unaffected models).
## How Does Column-Level Lineage Add Granularity?
Model-level lineage tells you that model B depends on model A. But if you changed only one column in model A, you may not need to review all of model B — only the columns that depend on the one you changed.
[Column-level lineage](/ai-blog/what-is-column-level-lineage/) provides this precision. It traces individual columns through transformations, showing exactly which downstream columns are derived from your changed column. On large DAGs, this can reduce the review scope from dozens of models to a handful of specific columns.
Column-level lineage is the next level of granularity beyond the model-level DAG. It does not replace model-level lineage — it refines it.
## DAG Complexity and the Case for Tooling
The relationship between DAG size and review effort is not linear — it is combinatorial. A 50-model DAG might have a few dozen dependency paths. A 500-model DAG can have thousands. Manual impact analysis at that scale is slow, error-prone, and inconsistent between team members.
This is why diff-aware lineage tools exist. dbt Cloud provides lineage visualization in its IDE. Recce provides lineage diff with integrated data validation — comparing the DAG between your PR branch and production, highlighting modified models, and letting you run data diffs directly from the lineage view. The goal is the same: reduce the cognitive load of understanding how a change propagates through a complex graph.
## Summary
The dbt DAG is a directed acyclic graph that maps every dependency in your project. It determines build order, enables parallelism, and — most importantly — defines how changes propagate. Reading the DAG tells you the story of your data from sources through marts to exposures. As projects grow, static lineage (dbt docs) becomes insufficient for PR review; lineage diffs that compare two states of the DAG are essential for scoping impact analysis. The modified+ view focuses review on the models most likely affected by a change. For even finer granularity, column-level lineage traces individual columns through transformations. The bigger your DAG, the more you need tooling that makes its complexity manageable.
## FAQ
**Q: What is the dbt DAG?**
A: The dbt DAG (directed acyclic graph) is a visual representation of all models in your dbt project and the dependencies between them. Each node represents a model (or source, seed, snapshot), and each edge represents a ref() dependency. The DAG defines the order in which dbt builds models — a model only runs after all its upstream dependencies have completed. The DAG is "acyclic" because circular dependencies are not allowed.
**Q: What is the difference between dbt docs lineage and a lineage diff?**
A: The dbt docs lineage shows the current state of your DAG — all models and their dependencies as they exist right now. A lineage diff compares the DAG between two states (typically your PR branch and production) and highlights what changed: which models were modified, added, or removed. Lineage diffs are essential for impact analysis because they show not just the current structure, but how your changes affect it.
**Q: How do you use the dbt DAG for impact analysis?**
A: Start from the modified models and trace downstream through the DAG to identify all dependent models — this is the impact radius. For each model in the impact radius, check whether its schema, row count, or data distributions changed. Focus your review on the "modified+" view — the modified models plus all their downstream dependents — rather than reviewing the entire DAG.
**Q: What is the modified+ view in lineage?**
A: The modified+ view shows the modified models in your PR plus all their downstream dependents. This represents the potential impact radius of your changes — every model that could be affected. It is the default scoping view for data impact analysis because it focuses review effort on the models most likely to have changed.
---
# Recce vs Datafold: Which Data Validation Tool?
> A comparison of Recce and Datafold for dbt data validation. Covers validation philosophy, CI/CD integration, pricing, and when to choose each tool.
Date: 2026-02-20
Source: https://blog.reccehq.com/recce-vs-datafold
Tags: tools, comparison, dbt
## Why Compare Recce and Datafold?
Both Recce and Datafold help data teams validate dbt model changes before merging to production. They solve the same core problem — SQL changes alone don't reveal how the actual data was affected — but take fundamentally different approaches to getting there. Understanding where they diverge helps you pick the tool that fits your team's workflow.
## What Is Each Tool's Validation Philosophy?
The biggest difference between Recce and Datafold is not features but philosophy.
**Recce: validate what matters.** Recce treats data diffing as one tool among several, not the default starting point. You begin with lineage and metadata — understanding what changed and what's downstream — then drill into targeted diffs where the signal warrants it. Data validation is selective and human-in-the-loop.
**Datafold: automate everything.** Datafold runs cross-environment diffs across all modified models on every PR by default. The goal is comprehensive coverage — catch every difference, then let reviewers triage. Its Slim Diff feature reduces volume but selects at the model level, not by business relevance.
This philosophical split shapes every downstream decision: what runs in CI, what gets reported, and how much compute you burn.
## How Do the Features Compare?
| Capability | Recce | Datafold |
| ------------------------ | -------------------------------------------------- | ---------------------------------------- |
| Lineage Diff | Yes — visual DAG comparison between environments | Limited — model-level dependency view |
| Breaking Change Analysis | Yes — detects schema and contract-breaking changes | No dedicated feature |
| Column-Level Lineage | Yes — traces column transformations across models | Yes — column-level tracking |
| Schema Diff | Yes | Yes |
| Row Count Diff | Yes | Yes |
| Profile Diff | Yes — column-level statistics comparison | No direct equivalent |
| Value Diff | Yes — per-column match percentage with primary key | Yes — row-level data diff |
| Top-K Diff | Yes — categorical distribution comparison | No direct equivalent |
| Histogram Diff | Yes — overlaid distribution visualization | No direct equivalent |
| Query Diff | Yes — arbitrary SQL comparison | No direct equivalent |
| CI Integration | Opt-in, scoped via recce.yml | Auto-diff all changed models by default |
| Open Source | Yes — free CLI, public pricing for Cloud | No — original data-diff tool sunset |
| Pricing | Public pricing, free tier available | Commercial, pricing behind sales process |
| Self-Serve Setup | Yes — install and configure independently | Requires sales engagement |
## How Does CI/CD Integration Differ?
Recce's CI is opt-in and scoped. You define which checks to automate in your `recce.yml` configuration file, choosing from schema diffs, row count checks, profile comparisons, or custom queries. Only the checks you've validated manually first get promoted to CI. This means your automated checks reflect real review experience, not a generic "diff everything" rule.
Datafold auto-diffs all changed models on every PR by default. Slim Diff reduces the volume by selecting only models that were directly modified, but the selection is at the model level — it doesn't distinguish between a cosmetic column rename and a revenue-critical calculation change. Every diff gets the same treatment.
For teams working on large DAGs, this distinction matters. A single upstream change can propagate through the entire dependency chain, touching models that the author never intended to affect. Recce lets you focus CI on the models where being wrong is expensive. Datafold reports on everything and leaves triage to the reviewer.
## Why Do Teams Switch From Datafold?
Common reasons teams evaluate alternatives to Datafold:
- **Setup friction** — Datafold requires a sales process and onboarding. Teams wanting to evaluate quickly find the barrier high.
- **Noisy results** — auto-diffing every model on every PR generates alert fatigue. Reviewers learn to skim or ignore the reports.
- **Limited control** — you can't easily scope what gets diffed based on business context or risk level.
- **Compute costs** — comprehensive diffing triggers heavy warehouse queries. On large datasets, auto-diff budgets add up fast.
- **Pricing opacity** — without public pricing, teams can't plan costs or compare options independently.
These aren't flaws in Datafold's design — they're tradeoffs of a coverage-first philosophy. Teams that prefer targeted, context-driven validation often find Recce a better fit.
That said, Datafold has legitimate strengths. Its automated cross-environment diffing requires minimal configuration — once connected, every PR gets coverage without any per-model setup. For large-scale migrations (warehouse moves, dbt version upgrades), exhaustive row-level comparison across hundreds of models is exactly what you need. And teams with dedicated data quality engineers who can triage alerts effectively may prefer the comprehensive approach over manual drill-down.
## How Should You Decide Between Them?
Use this decision framework based on your team's priorities:
| Criterion | Choose Recce | Choose Datafold |
| ---------------------- | --------------------------------------------------------------- | ------------------------------------------------------ |
| Validation approach | You want to validate selectively based on business context | You want comprehensive automated coverage |
| Team size | Small to mid-size teams that value signal over volume | Larger teams with dedicated data quality roles |
| DAG complexity | Large DAGs where diffing everything is impractical or expensive | Manageable DAGs where full coverage is feasible |
| Budget sensitivity | Need public pricing and predictable costs | Budget is flexible and sales engagement is acceptable |
| CI philosophy | Prefer opt-in checks that you curate over time | Prefer out-of-the-box automated diffing |
| Migration use case | Day-to-day PR validation and iterative development | Large-scale migrations requiring exhaustive comparison |
| Open-source preference | Want an open-source foundation with optional cloud | Commercial-only is acceptable |
| Review workflow | Drill-down: lineage first, then targeted diffs with checklist | Top-down: see all diffs, then triage and dismiss |
Neither tool is universally better. The choice depends on whether your team's bottleneck is coverage (you miss things because nothing checks them) or noise (you miss things because everything is flagged).
## How Do They Fit Into the Broader dbt Ecosystem?
Both tools complement dbt's built-in testing. dbt tests validate structure and constraints; [data diffs](/ai-blog/what-is-a-data-diff/) validate actual output against a known-good baseline. The question is how much automation and scope you want around that diffing.
Other tools in the ecosystem include `dbt-audit-helper` for lightweight relation comparison, SQLMesh with built-in table diff, and custom CI scripts. Recce and Datafold sit at the more capable end of this spectrum — the difference is in how they wield that capability.
For teams building a structured review process, combining Recce's selective diffing with [CI checks beyond dbt tests](/ai-blog/what-should-dbt-ci-check-beyond-tests/) provides a practical middle ground: automate what you've validated, investigate everything else with context.
## Summary
Recce and Datafold solve the same problem — validating data changes before they reach production — with opposite philosophies. Recce is selective and human-in-the-loop, starting with lineage and drilling into targeted diffs. Datafold is comprehensive and automated, diffing all changed models by default. Choose Recce when signal-to-noise ratio and cost control matter most. Choose Datafold when exhaustive coverage and large-scale migration support are the priority. Both are stronger than no data validation at all.
## FAQ
**Q: What is the difference between Recce and Datafold?**
A: Recce and Datafold take different philosophical approaches to data validation. Recce uses selective, human-in-the-loop validation — you start with lineage and metadata, identify what matters, then drill into targeted diffs. Datafold uses comprehensive automated diffing, running diffs across all modified models on every PR by default. Recce prioritizes signal-to-noise ratio; Datafold prioritizes coverage.
**Q: Is Recce open source?**
A: Yes. Recce was born as an open-source project and maintains a free CLI for local use. A Cloud plan is available for team collaboration and GitHub integration. Datafold's original open-source data-diff tool has been sunset; all core features now require a commercial license behind a sales process.
**Q: Which tool is better for large dbt projects?**
A: For large DAGs, Recce's selective approach reduces noise and compute costs by diffing only the models that matter. Datafold's comprehensive approach provides broader coverage but can generate alert fatigue and high compute costs when every PR triggers diffs across hundreds of modified models. The best choice depends on whether your team prefers targeted validation with business context or full automated coverage.
**Q: How does each tool handle CI/CD integration?**
A: Recce's CI is opt-in and scoped — you decide which checks to automate in your recce.yml configuration. Datafold auto-diffs all changed models on every PR by default with its Slim Diff feature, which reduces volume but selects at the model level rather than business relevance. Recce focuses on automating the checks you've validated manually first.
---
# Data Review Best Practices for Modern Data Teams
> A structured guide to implementing data review processes that catch data quality issues before they reach production. Covers impact analysis, automated checks, and CI/CD integration for dbt projects.
Date: 2026-02-19
Source: https://blog.reccehq.com/data-review-best-practices
Tags: data-quality, best-practices, dbt
## What Is Data Review?
Data review is the practice of systematically validating data model changes before merging them into production. Unlike code review, which examines logic, data review examines the actual output — the rows, columns, and values that downstream consumers depend on.
Modern data teams working with dbt (data build tool) face a core challenge: a single model change can affect every downstream dependency in the DAG — from intermediate models to dashboards and ML features. Data review provides visibility into this blast radius before changes ship.
## Why Data Review Matters
Traditional data quality approaches rely on post-deployment monitoring — catching issues after they've already affected production dashboards and reports. Data review shifts this left:
- **Pre-merge validation**: Compare branch output against production baseline
- **Impact analysis**: Understand which downstream models are affected
- **Automated diffing**: Detect schema changes, row count shifts, and value distribution changes
- **Human judgment**: Flag changes that are technically valid but semantically wrong
## Core Components of a Data Review Process
### 1. Impact Analysis
Before reviewing data, understand the scope of change. Impact analysis maps which models are modified and traces their downstream dependencies. This tells reviewers where to focus attention.
Key metrics for impact analysis:
| Metric | What It Measures | Why It Matters |
| ------------------ | ----------------------- | -------------------------- |
| Modified models | Direct code changes | Primary review targets |
| Downstream models | Transitive dependencies | Blast radius of the change |
| Affected exposures | Dashboards, ML features | Business impact visibility |
| Row count delta | Production vs. branch | Data volume changes |
### 2. Automated Checks
Automate the repetitive parts of data review:
- **Schema diff**: Detect added, removed, or renamed columns
- **Row count comparison**: Flag unexpected increases or decreases
- **Value distribution**: Compare histograms of key columns
- **Primary key validation**: Ensure uniqueness constraints hold
### 3. PR-Level Reporting
Integrate data review results into your pull request workflow. A data review summary posted as a PR comment gives reviewers context without switching tools.
## Implementing Data Review with Recce
Recce automates data review for dbt projects. The typical workflow:
1. Developer opens a PR with model changes
2. CI runs `dbt build` on the PR branch
3. Recce compares branch output against the production baseline
4. Recce posts a diff report as a PR comment
5. Reviewers approve or request changes based on data impact
### Integration with dbt CI/CD
Recce plugs into existing dbt CI pipelines. After `dbt build` completes, Recce runs its comparison checks and reports results. No changes to your dbt project structure are required.
## Best Practices
1. **Review data, not just code**: A syntactically correct model can produce wrong results. Always check the output.
2. **Scope reviews to impact radius**: Don't review every model — focus on modified models and their direct downstream dependencies.
3. **Automate the baseline**: Use CI to maintain a production baseline that Recce compares against automatically.
4. **Set blocking thresholds**: Define what constitutes a blocking data change (e.g., >10% row count change) and enforce it in CI.
5. **Document expected changes**: When a PR intentionally changes data output, annotate the expected changes in the PR description.
## FAQ
**Q: What is data review?**
A: Data review is the practice of systematically validating data model changes before merging them into production. It combines automated checks (schema diff, row count comparison, value distribution analysis) with human review to catch data quality regressions early in the development cycle.
**Q: How does data review differ from data testing?**
A: Data testing validates that data meets predefined rules (not-null constraints, accepted values, relationship tests). Data review goes further by comparing the actual output of changed models between your development branch and production, surfacing unexpected differences that tests alone would miss.
**Q: What tools support automated data review?**
A: Recce is a purpose-built data review tool that integrates with dbt projects. It provides impact analysis, automated diff checks, and PR-level reporting. Other approaches include custom CI scripts, Great Expectations for data validation, and dbt tests for schema-level checks.
**Q: How do you integrate data review into CI/CD?**
A: Configure your CI pipeline to run Recce after dbt build completes. Recce compares the PR branch output against the production baseline, generates a data diff report, and posts results as a PR comment. Teams can set blocking rules so PRs with unexpected data changes require explicit approval.
---
# What Is Impact Radius in Data Modeling?
> Impact radius measures how far a data model change propagates through your DAG. Learn how to calculate, visualize, and use impact radius to scope data reviews and reduce production risk.
Date: 2026-02-19
Source: https://docs.reccehq.com/4-downstream-impacts/impact-radius/
Tags: concepts, data-modeling, dbt
## Defining Impact Radius
Impact radius measures how far a data model change propagates through your data pipeline. When you modify a model in a directed acyclic graph (DAG), the impact radius is the complete set of downstream nodes that depend — directly or transitively — on that model.
Understanding impact radius is critical for data review: it tells you exactly where to look when validating a change.
## How Impact Radius Works
Consider a simplified dbt DAG:
```
raw_orders → stg_orders → fct_orders → mart_revenue
→ mart_customer_ltv
→ fct_order_items → mart_product_performance
```
If you modify `stg_orders`, the impact radius includes:
- `fct_orders` (direct dependent)
- `fct_order_items` (direct dependent)
- `mart_revenue` (transitive via fct_orders)
- `mart_customer_ltv` (transitive via fct_orders)
- `mart_product_performance` (transitive via fct_order_items)
The impact radius is 5 models. The modification touches 1 model but affects 5.
## Calculating Impact Radius
Impact radius calculation is a graph traversal problem. Starting from each modified node, perform a breadth-first or depth-first traversal of all downstream edges.
| Input | Output |
| -------------------------------- | ------------------------------------ |
| Set of modified models | All transitively dependent models |
| Modified model + depth limit | Dependents within N hops |
| Modified model + exposure filter | Only affected dashboards/ML features |
### Depth-Bounded Impact Radius
For large DAGs, full transitive impact radius can be overwhelming. Depth-bounded analysis limits traversal to N hops downstream:
- **Depth 1**: Direct dependents only — the models that SELECT FROM the modified model
- **Depth 2**: Dependents of dependents — one additional layer of propagation
- **Depth N**: Full transitive closure when N equals the DAG diameter
## Why Impact Radius Matters for Data Review
### Scoping Reviews
Without impact radius, reviewers must manually trace dependencies or review everything. Impact radius automatically identifies the relevant subset of models to check.
### Risk Assessment
Larger impact radius means higher risk. A change to a foundational staging model that affects 30 downstream models requires more scrutiny than a change to a leaf mart model with no dependents.
### CI/CD Gating
Teams can set CI rules based on impact radius:
- Impact radius < 5: Auto-approve with standard checks
- Impact radius 5–15: Require data review approval
- Impact radius > 15: Require senior review + stakeholder notification
## Reducing Impact Radius
Design patterns that limit propagation:
1. **Interface layers**: Insert stable interface models between raw/staging and mart layers. These absorb schema changes.
2. **Model modularity**: Break large models into focused components. A change to one component doesn't propagate through unrelated paths.
3. **Schema contracts**: Use dbt contracts to enforce column-level stability. Downstream models depend on the contract, not the implementation.
4. **Incremental isolation**: Design incremental models so that logic changes affect only the incremental window, not the full table.
## Impact Radius in Recce
Recce calculates impact radius automatically from your dbt project manifest. When a PR modifies models, Recce:
1. Parses the project DAG from `manifest.json`
2. Identifies all modified models from the git diff
3. Traverses downstream edges to compute full impact radius
4. Filters to relevant exposures and metrics
5. Reports the impact radius in the PR comment with visual lineage diff
This gives reviewers immediate visibility into the scope of every change without manual DAG tracing.
## FAQ
**Q: What is impact radius in data modeling?**
A: Impact radius is the set of downstream models, exposures, and data products affected by a change to a specific data model. It is calculated by tracing the directed acyclic graph (DAG) from the modified model to all its transitive dependents.
**Q: How is impact radius different from blast radius?**
A: The terms are often used interchangeably. Blast radius typically refers to the worst-case scope of failure, while impact radius more precisely measures the actual downstream propagation path of a specific change. In practice, both describe the same DAG traversal.
**Q: How do you reduce impact radius?**
A: Reduce impact radius by modularizing your DAG (breaking large models into smaller, focused ones), using interface layers between raw and mart models, and designing models with stable schemas that absorb upstream changes without propagating them downstream.
**Q: Can impact radius be calculated automatically?**
A: Yes. Tools like Recce automatically calculate impact radius from your dbt project DAG. Given a set of modified models, Recce traces all downstream paths and reports the full impact radius, including affected exposures and metrics.
---
# What Should a dbt CI Pipeline Check Beyond Tests?
> dbt tests check structure, not data impact. Learn what additional checks — schema diffs, row counts, profile diffs, and automated preset checks — your CI pipeline should run to catch issues before merging.
Date: 2026-02-18
Tags: dbt, ci-cd, best-practices
## What Gap Do dbt Tests Leave in CI?
Most dbt CI pipelines follow the same pattern: run `dbt build`, execute tests, and merge if everything is green. The problem is that dbt tests validate constraints — not-null, uniqueness, accepted values, referential integrity — but they do not validate data output.
A model can pass every test and still produce [wrong results](/ai-blog/why-dbt-data-wrong-when-tests-pass/). An incorrect filter silently drops records. A bad JOIN fans out rows. A calculation uses the wrong column. The tests pass because the output is structurally valid. The data is just wrong.
This is the gap: dbt tests check structure, but your CI pipeline needs to check data impact.
## What Additional Checks Should CI Include?
Beyond dbt tests, a robust CI pipeline should compare PR branch output against the production baseline using four types of data validation checks:
| Check Type | What It Detects | When It Matters |
| -------------- | ----------------------------------------- | ---------------------------------------------------- |
| Schema diff | Column additions, removals, type changes | Every PR — schema changes break downstream consumers |
| Row count diff | Data loss, duplication, unexpected growth | Every PR — row count shifts signal logic errors |
| Profile diff | Distribution changes in column values | PRs touching business logic or filters |
| Value diff | Specific value-level differences | PRs affecting critical models |
### Schema Diff
A schema diff compares the column structure of a model between your PR branch and production. It catches added columns, removed columns, renamed columns, and type changes. Schema changes are especially important because they can silently break downstream models, dashboards, and reverse ETL pipelines that depend on specific column names or types.
### Row Count Diff
A row count diff compares the number of rows in a model between environments. A significant drop often indicates a filter bug or failed JOIN. A significant increase may indicate duplication or an overly permissive filter. Either way, it is a signal that something changed beyond what the code alone would suggest.
### Profile Diff
A profile diff compares statistical summaries of columns — min, max, mean, median, null percentage, distinct count — between environments. It catches distribution shifts that row counts miss. For example, if the average order value drops by 30% because a filter excluded high-value transactions, the row count might barely change, but the profile diff will flag it immediately.
### Value Diff
A value diff compares actual row-level data between environments. This is the most granular and most expensive check. Reserve it for critical models where being wrong has direct business impact — revenue tables, customer-facing metrics, models feeding ML pipelines.
## What Are Critical Models and How Do You Identify Them?
Not every model in your dbt project needs the same level of CI validation. Critical models are the ones where being wrong has significant business consequences.
Critical models typically share one or more of these characteristics:
- **Customer-facing** — powers dashboards, reports, or APIs that stakeholders see directly
- **Revenue-related** — feeds billing, financial reporting, or pricing logic
- **ML pipeline input** — serves as a feature table for machine learning models
- **High fan-out** — has many downstream dependencies, amplifying the impact of errors
Identify them through domain knowledge. Ask your team: "Which models trigger a stakeholder call when something goes wrong?" Those are your critical models.
## How Do You Automate Data Checks in CI?
Automation turns data checks from a manual practice into a repeatable process. The key concept is preset checks — a configuration file that defines which models to check and what types of diffs to run.
A typical preset configuration (in a file like `recce.yml`) specifies:
- **Target models** — which models to run checks on (usually critical models)
- **Check types** — schema diff, row count, profile diff, or value diff for each target
- **Thresholds** — what constitutes a blocking difference (e.g., row count change > 5%)
### A Practical CI Workflow
Here is the end-to-end workflow for a dbt CI pipeline with data checks:
1. **`dbt build`** — build and test the PR branch against a CI environment
2. **Run preset checks** — execute the configured data checks, comparing PR output against the production baseline
3. **Compare to baseline** — evaluate results against defined thresholds
4. **Post summary to PR** — format the results as a [structured PR comment](/ai-blog/how-to-write-a-good-dbt-pull-request/) so reviewers have immediate context
5. **Block on mismatches** — if critical model checks detect unexpected differences, mark the CI check as failed
This workflow integrates naturally with GitHub Actions or any CI provider. The data checks run after `dbt build` completes and report results back to the pull request.
## What Does "All Signal, No Noise" Mean for CI Checks?
A common failure mode for data checks is alert fatigue. If every PR generates a wall of "check passed" messages, reviewers stop reading them. The all signal, no noise philosophy means: only report when something is different.
In practice this means:
- **Do not report passing checks** — if the schema matches, the row count is stable, and the profile is unchanged, there is nothing to report
- **Report only mismatches** — surface the specific models and columns where differences were detected
- **Provide context with the signal** — show the actual values (e.g., "row count changed from 142,387 to 138,201") so reviewers can assess severity immediately
- **Distinguish blocking from informational** — some checks should block the merge; others are advisory
This approach keeps the CI output actionable. When a reviewer sees a data check result, it means something actually changed and needs attention.
## How Do CI Checks and PR Review Work Together?
CI checks and human PR review serve complementary roles:
| | CI Checks | Human Review |
| -------------- | --------------------------------------------------------- | --------------------------------------------- |
| **Speed** | Runs in minutes on every PR | Requires scheduling and availability |
| **Scope** | Predefined checks on critical models | Can explore any model or question |
| **Strength** | Catches obvious regressions consistently | Evaluates intent, context, and business logic |
| **Limitation** | Cannot judge whether a change is correct for the business | Cannot run on every PR at full depth |
CI catches the obvious and repetitive — schema breaks, row count drops, distribution shifts. Human review handles the nuanced — "Is this metric change expected given the business context?" Together, they form a layered defense against data errors.
## How Does This Prevent "Tests Pass but Data Is Wrong"?
The [tests-pass-data-wrong scenario](/ai-blog/why-dbt-data-wrong-when-tests-pass/) happens when a change is structurally valid but semantically incorrect. dbt tests confirm the structure is fine. CI data checks close the gap by comparing actual output:
- A buggy filter that drops records? **Row count diff catches it.**
- A calculation error that shifts averages? **Profile diff catches it.**
- A column rename that breaks downstream? **Schema diff catches it.**
- A JOIN change that subtly inflates metrics? **Value diff on the critical model catches it.**
Without these checks, the only defense is a human reviewer manually running queries — which happens inconsistently at best. Automated CI checks make this validation happen on every single PR.
## Summary
dbt tests validate structural constraints, but your CI pipeline needs to go further. Add data-level checks — comparing structure, volume, distributions, and values between environments — to catch issues that tests miss. Focus validation on critical models — the ones where being wrong triggers stakeholder calls. Automate checks using preset configurations that define what to check and what thresholds to enforce. Follow the "all signal, no noise" philosophy: only report differences, not passing checks. CI handles the obvious regressions; human review handles the nuanced judgment calls. Together, they close the gap between "tests pass" and "data is correct."
## FAQ
**Q: What should a dbt CI pipeline check beyond tests?**
A: Beyond dbt tests, your CI pipeline should check for schema changes (column additions, removals, type changes), row count changes (data loss or unexpected growth), profile diffs (distribution shifts in critical columns), and value-level diffs on business-critical models. These checks compare your PR branch output against production, catching data-level issues that structural tests miss.
**Q: How do you automate data checks in dbt CI?**
A: Define preset checks in a configuration file (like recce.yml) specifying which models to check and what types of diffs to run. Configure your CI workflow to execute these checks after dbt build, compare results against the production baseline, and post a summary to the PR. Checks that show unexpected differences block the merge until reviewed.
**Q: What are critical models and how do you identify them?**
A: Critical models are the models in your dbt project where being wrong has significant business consequences. They typically include customer-facing tables, revenue metrics, models feeding ML pipelines, and tables with many downstream dependencies. Identify them through domain knowledge — ask which models trigger stakeholder calls when something goes wrong.
**Q: Should data checks in CI block PR merges?**
A: Data checks should block merges when they detect unexpected differences on critical models. The philosophy is "all signal, no noise" — only flag actual mismatches rather than reporting when everything matches. This prevents alert fatigue while ensuring that meaningful data changes are reviewed before merging. Not every check needs to be a hard block; some can be informational.
---
# How to Write a Good dbt Pull Request
> A structured guide to writing dbt pull requests that include data validation, not just code changes. Covers PR templates, data impact documentation, and review workflows.
Date: 2026-02-17
Tags: best-practices, dbt, workflow
## Why Are Data PRs Different from Code PRs?
In a typical software project, a pull request tells a clear story: here is the code that changed, here is what it does, here are the tests that prove it works. Reviewers can read the diff and reason about correctness.
dbt pull requests are fundamentally different. The code — SQL or Jinja — is visible, but its output is not. A one-line change to a `WHERE` clause can silently shift revenue numbers, customer counts, or ML feature values across every model downstream of the change. You can read the SQL and understand the intent; you cannot read it and know whether the data is correct.
This is the core problem: code is visible, but data is a black box. A good dbt PR must open that box.
## What Should a Good dbt Pull Request Include?
A dbt PR comment template standardizes the information that every pull request should contain. Without a template, PRs tend toward vague descriptions like "updated customer model" — leaving reviewers to guess at the scope and impact.
A structured template should include these sections:
### Type of Change
Classify the change so reviewers know what to expect:
| Type | Description | Review Focus |
| --------------- | --------------------------------------- | ------------------------------------------------ |
| New model | Adds a new model to the project | Schema design, naming conventions, test coverage |
| Bugfix | Corrects incorrect logic | Data diff against production, downstream impact |
| Refactor | Restructures without changing output | Confirm output is identical to production |
| Breaking change | Intentionally changes output | Full data validation, stakeholder notification |
| Source change | Updates source definitions or freshness | Upstream dependency review |
### Description and Motivation
Explain _why_ the change exists, not just _what_ it does. A good description answers: What business problem does this solve? What triggered this change? What alternatives were considered?
### Related Issues
Link to the issue tracker. This creates traceability between business requests and data changes.
### Lineage Diff
Show which models are directly modified and which downstream models are impacted. A lineage diff visualizes the blast radius of your change — the set of models, exposures, and dashboards that could be affected.
### Data Validation Results
This is the section most dbt PRs lack entirely. Include the results of [data diffs](/ai-blog/what-is-a-data-diff/) on impacted models:
- Did columns change? (schema comparison)
- Did the volume of data change? (row count comparison)
- Did the statistical distribution of key columns shift? (profile comparison)
- For critical models, how do specific values compare between dev and prod? (value-level comparison)
### dbt Test Results
Confirm that all dbt tests pass on the PR branch. This is the baseline — necessary but not sufficient.
### Impact Considerations
Note any downstream consumers that should be aware: dashboards, reverse ETL pipelines, ML features, or other teams' models.
### Reviewer Checklist
Provide a checklist of items for the reviewer to verify, such as: naming conventions followed, tests added for new models, data validation reviewed, breaking changes communicated.
## How Do You Perform a Data Impact Assessment?
A data impact assessment compares the actual data output of your PR branch against the production baseline. The goal is to answer: "Did the data change the way I expected, and only the way I expected?"
The process follows a funnel — start broad, narrow down:
1. **Run lineage analysis** — identify all models in the impact radius of your change
2. **Check schema diffs** — confirm no unintended column changes
3. **Compare row counts** — catch data loss or unexpected duplication
4. **Run profile diffs** — check that distributions on key columns look reasonable
5. **Drill into value diffs** — on critical models, compare actual values between dev and prod
This layered approach is efficient. Most models will pass the first two checks and need no further investigation. You only invest deep review time where the data shows something unexpected.
## What Are the Benefits of Structured PR Templates?
Structured templates deliver three distinct benefits:
**Define your own work.** Writing a structured PR forces the author to think through the impact of their change. You cannot fill in a "data validation results" section without actually running the validation. The template makes thoroughness the default.
**Help your reviewers.** Reviewers should not have to reverse-engineer the purpose and impact of a change from a code diff alone. A structured PR gives them the context they need to review efficiently and ask the right questions.
**Create a historical record.** Six months from now, when someone asks "why did the revenue model change in February?", the PR is the source of truth. A well-documented PR with data validation evidence is far more useful than a one-line commit message.
## How Do Teams Use Structured PR Review in Practice?
Teams across industries have adopted structured data PR review. Municipal government data teams use PR templates to document changes to public-facing datasets, where data errors can erode public trust. The structured format ensures that every change to a critical model includes validation evidence and a clear explanation of intent.
In the dbt community, the Jaffle Shop demo project demonstrates how even a small project benefits from documenting data impact alongside code changes. The pattern scales: what works for a demo project works for a 500-model production project.
The common thread is that teams who adopt structured PR review catch more issues before production and spend less time debugging after.
## How Do Teams Automate PR Validation?
Manual validation works, but it depends on the author remembering to run diffs, format results, and paste them into the PR comment. Consistency drops when the process is entirely manual.
Tools like Recce automate this by generating PR-ready validation checklists that export directly to GitHub comments. After analyzing the PR branch against the production baseline, the tool runs structural and statistical checks on impacted models and formats the results as a checklist. Every PR gets the same level of validation — not just the ones where the author was thorough.
For teams looking to extend this into CI, [preset checks in your CI pipeline](/ai-blog/what-should-dbt-ci-check-beyond-tests/) can run these validations on every pull request automatically, closing the gap between manual best practice and repeatable process.
## Summary
A good dbt pull request goes beyond code changes to include data validation evidence. Use a structured template with sections for change type, description, lineage diff, data validation results, test results, and a reviewer checklist. Perform data impact assessments by comparing dev output against production — checking structure, volume, distributions, and values at increasing levels of granularity. Structured templates define your work, help reviewers, and create a historical record. Tools like Recce automate the generation of PR-ready validation checklists, making thorough data review the default rather than the exception.
## FAQ
**Q: What should a good dbt pull request include?**
A: A good dbt PR goes beyond code changes to include data validation evidence. It should contain: a description of the change and its motivation, the type of change (new model, bugfix, refactor, breaking change), a lineage diff showing impacted models, data validation results (profile diffs, value diffs, schema checks) on affected models, dbt test results, and a checklist of items for the reviewer to verify.
**Q: How do you document data impact in a dbt PR?**
A: Document data impact by running data diffs (schema, row count, profile, value) on impacted models and including the results in your PR comment. Use a structured template with sections for lineage diff, validation results, and impact considerations. Tools like Recce generate PR-ready checklists that export directly to GitHub comments.
**Q: Why is code review not enough for dbt PRs?**
A: Code review shows what SQL changed but not how the data changed. A one-line filter change can cascade through dozens of downstream models, shifting metrics in ways that are invisible from the code alone. Data validation — comparing actual output between environments — is the only way to confirm the change did what you intended.
**Q: What is a dbt PR comment template?**
A: A dbt PR comment template is a structured markdown boilerplate that defines sections for PR authors to fill in when opening a pull request. Good templates include sections for change type, description, related issues, lineage diff, data validation results, dbt test results, and a reviewer checklist. Templates standardize what information reviewers need and prevent ambiguous or superficial PR comments.
---
# Why Is My dbt Data Wrong Even When Tests Pass?
> dbt tests validate structure, not meaning. Learn why data can pass all tests and still be wrong, and what practices catch the semantic errors that automated testing misses.
Date: 2026-02-16
Tags: data-quality, dbt, best-practices
## The False Sense of Security
Your dbt CI pipeline is green. All tests pass — not-null, unique, accepted_values, relationships. You merge the PR. Two days later, a stakeholder messages: "The revenue numbers look wrong."
This scenario is more common than most data teams admit. Data correctness — whether the data is right for the business — is fundamentally different from data quality, which measures structural integrity like completeness, format, and uniqueness. Data can be high quality and still be wrong.
## What dbt Tests Actually Check
dbt's built-in tests validate structural properties:
| Test Type | What It Checks | What It Misses |
| ----------------- | ------------------------------------ | -------------------------------------------------- |
| `not_null` | Column has no null values | Whether the non-null values are correct |
| `unique` | No duplicate values in a column | Whether the values themselves are right |
| `accepted_values` | Values fall within a defined set | Whether the distribution across values makes sense |
| `relationships` | Foreign keys reference valid parents | Whether the join logic produces correct results |
Custom tests and packages like `dbt-expectations` extend this to statistical checks (e.g., column means within bounds), but they still validate against predefined rules. They cannot catch a bug you didn't anticipate.
## Why Semantic Errors Slip Through
Semantic errors are logical mistakes that produce structurally valid but meaningfully wrong data. They pass all tests because the output looks fine at a structural level.
Common examples:
- **Incorrect filter logic** — a `WHERE` clause that silently excludes valid records. Row counts and uniqueness are fine, but the data is incomplete.
- **Wrong JOIN condition** — a join that fans out rows or drops records. The output has the right columns and no nulls, but metrics are inflated or deflated.
- **Calculation bugs** — business logic that uses the wrong column, wrong aggregation, or wrong date range. The result is a number, just not the right number.
- **Upstream changes** — a column's meaning changes in a source system. Your tests still pass because the format hasn't changed, but the semantics have.
In each case, the data "passes the test but fails the business."
## A Real-World Example
A data team pushed a change to a core model that fed a reverse ETL pipeline powering marketing automation. The model was fully tested — schema checks, null checks, uniqueness constraints. But the bug was a logical one: an incorrect filter that subtly changed which records were included in a calculation.
The corrupted data reached the experimentation platform and wasn't discovered for almost a week. The aftermath:
- Pipeline halted, financial data updates paused
- Downstream consumers had to delete bad data and re-ingest
- Model fix took a full day; platform cleanup took several more
- The experimentation team was blocked the entire time
Nothing looked obviously broken at a glance. It only became apparent when someone calculated metrics and noticed patterns that didn't make sense. The monetary cost was significant, but the loss of trust was immeasurable.
## How to Catch What Tests Miss
The gap between testing and correctness requires a different approach: comparing actual data output against a known-good baseline.
### Cross-Reference Against Production
Before merging, compare your development environment's data against production. If historical metrics changed when they shouldn't have, something is wrong. This is the historical context test — trusted production data serves as your benchmark for correctness.
### Use Data Diffs at the Right Granularity
[Data diffs](/ai-blog/what-is-a-data-diff/) compare datasets between environments. Start with cheap structural checks (has the schema changed? did row counts shift?), move to statistical checks (are column distributions still reasonable?), and drill into row-level comparisons only where the signal warrants deeper investigation.
For the example above, a profile diff would have shown the CLV distribution shifting. A value diff would have quantified that 99% of rows changed in the affected column while all other columns matched 100%.
### Automate Checks on Critical Models
Every project has models where being wrong is expensive — customer-facing tables, revenue metrics, models that feed ML pipelines. Identify these critical models and automate [data checks in your CI pipeline](/ai-blog/what-should-dbt-ci-check-beyond-tests/) so they run on every PR, not just when someone remembers.
### Involve Domain Experts Strategically
Not every change needs human review. Focus human attention on changes where the cost of being wrong is high and detection time is slow. For a marketing-critical model, ask: "Would a stakeholder notice if these numbers shifted by 5%?" If the answer is "not until next month's report," that model deserves a human-in-the-loop review.
## Building a Data Correctness Workflow
A practical workflow combines automated checks with targeted human review:
1. **dbt tests** — catch structural issues (they're still essential)
2. **Automated data diffs in CI** — schema, row count, and profile checks on critical models
3. **Manual exploration on high-risk changes** — use lineage to scope impact, run targeted diffs, check distributions
4. **PR documentation** — record what you checked, what you found, and why the change is safe to merge
5. **Domain review on high-stakes changes** — get a second pair of eyes when the business impact is significant
This layered approach follows the [data review best practices](/ai-blog/data-review-best-practices/) that catch issues at the right level. Tests are the foundation, but they're not the whole building.
## Summary
dbt tests validate that data meets structural constraints — not that it's correct for the business. Semantic errors (wrong calculations, incorrect filters, unexpected upstream changes) pass all tests while producing wrong results. To catch these issues, compare development data against production baselines using data diffs, automate checks on critical models in CI, and involve domain experts on high-stakes changes. The goal is not to replace testing but to complement it with data-level validation that catches what tests inherently cannot.
## FAQ
**Q: Why is my dbt data wrong when all tests pass?**
A: dbt tests validate structural properties — not-null constraints, uniqueness, accepted values, referential integrity. They do not validate whether the data is semantically correct for your business. A logical bug in a calculation, an incorrect JOIN condition, or a filter that silently excludes valid records will pass all structural tests while producing wrong results. These semantic errors require comparing actual data output against a known-good baseline.
**Q: What is the difference between data quality and data correctness?**
A: Data quality measures structural integrity and completeness — are columns the right type, are there nulls where there should not be, are values within expected ranges. Data correctness measures whether the data is right for the business — do the numbers make sense, do metrics match reality, would a domain expert agree the output is accurate. Data can be high quality (structurally sound) but fundamentally incorrect (wrong business logic).
**Q: How do you catch silent data errors before production?**
A: Compare development data against your production baseline before merging. Use data diffs to check whether metrics changed as expected. Cross-reference new calculations against known-good historical data. Involve domain experts in reviewing high-stakes changes. Automate impact checks on critical models in CI. The key is validating actual data output, not just structural constraints.
**Q: What should I check beyond dbt tests in my CI pipeline?**
A: Beyond dbt tests, your CI pipeline should include schema diff (detect unexpected column changes), row count diff (catch data loss or duplication), profile diff (spot statistical distribution shifts), and targeted value diffs on critical models. Tools like Recce can automate these checks via preset configurations that run on every PR, complementing dbt tests with data-level validation.
---
# What Is Column-Level Lineage and Why Does It Matter?
> Column-level lineage tracks how individual columns flow through your data pipeline. Learn how CLL works, its three core use cases, and how it compares across dbt ecosystem tools.
Date: 2026-02-15
Tags: concepts, data-lineage, dbt
## What Is Column-Level Lineage?
Column-level lineage (CLL) tracks how individual columns flow through transformations across your data pipeline. While model-level lineage shows that model B depends on model A, column-level lineage shows that `B.total_revenue` is derived from `A.price * A.quantity`.
This granularity matters because not every column in a model is affected by every change. When you modify a calculation in an upstream model, CLL tells you exactly which downstream columns are impacted — and which are safe to ignore.
## Three Core Use Cases for Column-Level Lineage
### Source Exploration
During development, CLL helps you understand how a column is derived. When you encounter a column like `customer_lifetime_value` in a downstream mart, CLL traces it back through intermediate models to the original source columns, showing each transformation along the way.
### Impact Analysis
When modifying column logic, CLL lets you assess potential impact across the entire [DAG](/ai-blog/what-is-dbt-dag-lineage/). Instead of manually checking every downstream model after changing a column definition, you trace the column forward to see exactly which models and columns depend on it. This scopes your [data review](/ai-blog/data-review-best-practices/) to the affected columns rather than entire models.
### Root Cause Analysis
When a downstream metric looks wrong, CLL helps identify the possible source of the error. Trace the problematic column backward through the lineage to find where a transformation may have introduced the issue.
## How Column-Level Lineage Works
CLL is typically constructed by parsing SQL and analyzing how columns are referenced, transformed, and projected through each model. The process involves:
1. **Parsing SQL into an abstract syntax tree (AST)** — tools like SQLGlot parse each model's SQL into a tree structure that can be traversed programmatically.
2. **Traversing scopes** — each CTE, subquery, and root query is a scope with locally available columns. The parser walks through each scope to resolve column references.
3. **Classifying transformations** — for each output column, the parser determines how it relates to its input columns.
## Column Transformation Types
Understanding how a column was transformed is as important as knowing which upstream columns it depends on.
| Transformation | Description | Example |
| -------------- | ------------------------------------------------------- | ---------------------------------------- |
| Pass-through | Column selected as-is, no modification | `SELECT user_id FROM ref('users')` |
| Renamed | Single upstream column with an alias change | `SELECT user_id AS id FROM ref('users')` |
| Derived | Result of an expression, calculation, or aggregation | `SELECT price * quantity AS total` |
| Source | Not based on any upstream column (literal or function) | `SELECT CURRENT_TIMESTAMP AS created_at` |
| Unknown | Parsing failed or logic involves unsupported constructs | Ambiguous references in complex JOINs |
Derived columns usually deserve the closest attention during review because they introduce business logic. A pass-through column is unlikely to be the source of a data issue, but a derived column that aggregates or transforms data is where bugs hide.
## The WHERE Clause Caveat
An important limitation of most CLL implementations: columns used in `WHERE` clauses, `JOIN` conditions, or `GROUP BY` expressions typically do not appear in the column-level lineage graph. These clauses filter or group data but don't directly produce output columns.
For example, if a model filters on `order_status = 'completed'`, the `order_status` column won't appear in the CLL for any output column — even though changing its values would affect the model's output. This is a model-to-column relationship rather than a column-to-column dependency.
This means CLL is powerful for tracing data flow but should be complemented with model-level lineage and [data diffs](/ai-blog/what-is-a-data-diff/) for full coverage.
## Comparing CLL Tools in the dbt Ecosystem
Each CLL implementation differs in meaningful ways:
| Tool | CLL Display | Key Characteristics |
| ----------------------- | ------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Recce** | Integrated on lineage DAG diff | CLL shown directly on the main lineage view; click columns to trace dependencies; shows transformation types (pass-through, renamed, derived); open source |
| **dbt Cloud Explorer** | Separate column-level view | Enterprise only; each column is a distinct node; requires navigating in/out of model detail pages |
| **Power User (VSCode)** | Integrated in editor lineage | Open source; CLL shown within the model lineage panel; stays in your editor; requires beta UX toggle |
| **SQLMesh** | Native feature | Open source; works with dbt projects; shows CTEs as nodes (can cause info overload); upstream-only; each click refreshes lineage |
The best choice depends on your workflow. If you want CLL as part of a broader [data validation workflow](/ai-blog/data-review-best-practices/), tools that integrate CLL with diff capabilities (like Recce) reduce context-switching. If you primarily need CLL for exploration during development, editor-integrated tools work well.
## Column-Level Lineage and Impact Radius
CLL directly improves how you calculate [impact radius](/ai-blog/what-is-impact-radius/). Without CLL, a change to any column in a model means you must consider all downstream models as potentially impacted. With CLL, you can narrow the impact radius to only the downstream models and columns that actually depend on your changed column.
This precision matters on large DAGs where a single model might have hundreds of downstream dependents. CLL turns a potentially overwhelming review into a targeted one.
## Summary
Column-level lineage tracks how individual columns flow and transform through your data pipeline. Its three core use cases — source exploration, impact analysis, and root cause analysis — make it essential for efficient data review on complex DAGs. Most CLL implementations work by parsing SQL into an AST and classifying each column's transformation type. While CLL has limitations (notably the WHERE clause caveat), it significantly reduces the effort needed to validate data model changes when combined with model-level lineage and data diffs.
## FAQ
**Q: What is column-level lineage?**
A: Column-level lineage (CLL) is a granular form of data lineage that tracks how individual columns flow through transformations across your data pipeline. While model-level lineage shows relationships between tables, CLL shows exactly which upstream columns feed into each downstream column and how they are transformed — whether passed through unchanged, renamed, or derived through calculations.
**Q: What is the difference between model-level and column-level lineage?**
A: Model-level lineage shows the relationships between tables or models in your DAG — which models depend on which. Column-level lineage goes deeper, showing the specific columns that are passed, renamed, or transformed between models. Model-level lineage tells you that model B depends on model A. Column-level lineage tells you that B.total_revenue is derived from A.price multiplied by A.quantity.
**Q: How does column-level lineage help with impact analysis?**
A: When you modify a column in an upstream model, column-level lineage traces exactly which downstream columns are affected. Instead of checking every downstream model manually, you can see precisely which columns in which models depend on your change. This scopes your review to the affected columns rather than entire models, significantly reducing validation effort on large DAGs.
**Q: Which tools support column-level lineage for dbt?**
A: Several tools support CLL for dbt projects: Recce provides CLL integrated directly on the lineage DAG diff with transformation type classification. dbt Cloud Explorer (Enterprise) shows CLL as separate column-level nodes. Power User for dbt (VSCode) adds CLL to the editor. SQLMesh includes native CLL that also works with dbt projects. Each implementation differs in how it displays and navigates column dependencies.
---
# What Is a Data Diff and When Should You Use One?
> A data diff compares datasets across two environments to surface what changed. Learn the types of data diffs, when each is useful, and how to avoid the hidden costs of diff-everything approaches.
Date: 2026-02-14
Tags: concepts, data-validation, dbt
## What Is a Data Diff?
Data diff is the practice of comparing two versions of a dataset to identify what changed between them. In dbt workflows, this typically means comparing the output of a model in your development environment against the same model in production. The goal is to understand the actual data impact of your code changes before merging to production.
Unlike a code diff, which shows you what lines of SQL changed, a data diff shows you what happened to the rows, columns, and values that downstream consumers depend on. This distinction matters because a one-line code change can ripple through your entire DAG — affecting models, dashboards, and metrics in ways that are invisible from the SQL alone.
## Types of Data Diffs
Not all diffs operate at the same granularity. The right type depends on what you're trying to validate.
| Diff Type | What It Compares | Cost | Best For |
| -------------- | -------------------------------------------------- | ----------- | ------------------------------------- |
| Schema Diff | Column names, types, ordering | Very low | Catching structural breaking changes |
| Row Count Diff | Total row counts per model | Very low | Detecting data loss or duplication |
| Profile Diff | Column-level statistics (min, max, avg, null rate) | Low | Spotting distribution shifts |
| Histogram Diff | Value distributions overlaid on shared axes | Medium | Visualizing how distributions shifted |
| Top-K Diff | Most frequent categorical values | Medium | Comparing category distributions |
| Value Diff | Per-column match percentage using a primary key | Medium-high | Quantifying exact change scope |
| Query Diff | Row-by-row comparison of arbitrary queries | High | Fine-grained spot-checks |
The key insight is that these types form a natural funnel. Start cheap and broad, then drill down where the signal warrants it.
## When Should You Use a Data Diff?
Data diffs serve two distinct validation modes:
**Impact analysis** — when you expect data to change and want to verify the change is correct. For example, fixing a `customer_lifetime_value` calculation to only include completed orders. You expect CLV to decrease. A value diff confirms that CLV changed while other columns stayed the same.
**Regression testing** — when you expect data to remain unchanged. For example, refactoring a model's SQL without changing its logic. A profile diff or row count diff can quickly confirm nothing shifted.
For both modes, the drill-down approach works best:
1. **Start with lineage** — identify which models were impacted and scope your review to the [impact radius](/ai-blog/what-is-impact-radius/)
2. **Check structure** — schema diff and row count catch the obvious issues
3. **Check distributions** — profile diff, histogram overlay, and top-k reveal statistical shifts
4. **Spot-check values** — value diff and query diff confirm specific rows when needed
## When a Data Diff Is Not Enough
A data diff shows you _what_ changed, but not _why_ or _what to do next_. Not all differences are problems. Without context, diffing generates false alerts that demand attention but not action.
The hidden costs of a "diff everything" approach include:
- **Compute cost** — diffing two full tables triggers heavy queries on large datasets. Auto-diffing every model on every PR drains warehouse budgets.
- **Noise** — a small upstream change cascades through the DAG, creating downstream diffs that mostly don't matter. Teams learn to ignore the alerts.
- **Configuration burden** — accurate row-level diffs require primary keys or unique identifiers, which aren't always available or documented.
Better alternatives often exist for the first pass. Data profiling (null rates, distributions), group-based aggregation (counts and sums by dimension), and [column-level lineage](/ai-blog/what-is-column-level-lineage/) can tell you where to focus before you start diffing.
## How Data Diffs Fit Into dbt PR Review
The most effective teams use data diffs as part of a structured [data review process](/ai-blog/data-review-best-practices/). The workflow looks like:
1. **Explore** — use lineage diff to scope the blast radius
2. **Validate** — run targeted diffs on the models that matter
3. **Document** — add diff results to a checklist with notes explaining what you checked and why
4. **Share** — export the checklist to your PR comment for reviewers
This approach treats diffing as a tool in a larger toolkit, not as the goal itself. The goal is understanding — confirming that your change did what you intended and nothing else.
## Data Diff Tools in the dbt Ecosystem
Several tools support data diffing for dbt projects:
- **Recce** — open-source toolkit with lineage diff, profile diff, value diff, top-k diff, histogram overlay, query diff, and a checklist workflow for PR review. Supports selective, human-in-the-loop validation.
- **dbt-audit-helper** — dbt package for comparing relations with `compare_relations` and `compare_column_values` macros. Lightweight but manual.
- **Datafold** — commercial platform with automated cross-environment diffing on every PR. Full coverage but can generate noise on large DAGs.
- **SQLMesh** — dbt alternative with built-in table diff capabilities.
The right choice depends on your team's workflow. If you want targeted validation with business context, tools like Recce that support a drill-down approach work well. If you need comprehensive automated coverage, full-table diff tools may fit better.
## Summary
A data diff compares datasets between environments to surface what changed. Use structural diffs (schema, row count) for quick sanity checks, statistical diffs (profile, histogram, top-k) for distribution insight, and row-level diffs (value, query) for fine-grained confirmation. Scope your diffs to the models that matter rather than diffing everything, and combine diffs with lineage analysis and business context for effective data review.
## FAQ
**Q: What is a data diff?**
A: A data diff compares two versions of a dataset — typically between development and production environments — to identify what changed. It can operate at multiple granularities: structural (schema and row counts), statistical (column profiles and distributions), or row-level (individual value comparisons). Data diffs are a core tool for validating dbt model changes before merging to production.
**Q: When should you use a data diff?**
A: Use a data diff when you need to validate the impact of a data model change before merging. Start with structural diffs (schema, row count) for a quick sanity check, move to statistical diffs (profile, histogram, top-k) for distribution-level insight, and use row-level diffs (value diff, query diff) only when you need fine-grained confirmation. Avoid diffing everything — scope your diffs to the models that matter.
**Q: What is the difference between a value diff and a profile diff?**
A: A profile diff compares statistical summaries of columns (min, max, average, null rate) between two environments. A value diff compares the actual values in each column row-by-row using a primary key, reporting the percentage of matching values. Profile diffs are cheaper to compute and good for spotting distribution shifts. Value diffs are more precise but require a primary key and more compute resources.
**Q: How is a data diff different from a dbt test?**
A: A dbt test validates that data meets predefined rules (not-null, unique, accepted values). A data diff compares actual output between two environments to surface unexpected differences. Tests check structure and constraints; diffs check whether the data actually changed in the way you intended. Both are complementary — tests catch known failure modes, diffs catch unexpected side effects.