How Did vaidukt Reduce Data Complaints by 70% with Systematic Validation?
What Data Problem Was vaidukt Facing?
vaidukt is a German energy platform whose data team powers customer-facing reports and operational dashboards. Like many data teams, they relied on dbt tests and manual spot-checks to validate changes before deploying to production. And like many data teams, they discovered the hard way that passing tests does not mean correct data.
Customer complaints about data accuracy were a regular occurrence. The issues were not dramatic failures — they were subtle: a metric calculated slightly differently after a model change, a filter that excluded records it shouldn’t have, an upstream schema change that passed all tests but shifted downstream values. Each complaint eroded trust and consumed engineering time to investigate.
For a team of just three data engineers, this was unsustainable. Every hour spent investigating a data complaint was an hour not spent building new capabilities.
Why Weren’t dbt Tests Enough?
This is one of the most common frustrations in data engineering. dbt tests validate structural properties — not-null constraints, uniqueness, accepted values, referential integrity. They confirm that data looks right at a schema level. But they do not confirm that the data is right at a business level.
The gap between structural validity and business correctness is where most data complaints originate. Understanding why dbt data can be wrong even when tests pass is the first step toward closing that gap.
vaidukt’s complaints fell into predictable categories:
| Complaint Type | dbt Test Coverage | Root Cause |
|---|---|---|
| Metric values shifted unexpectedly | Not tested (values were non-null and unique) | Logic change in upstream model |
| Report showed different totals than last month | Not tested (schema unchanged) | Filter condition modified during refactor |
| Customer segment counts changed | Not tested (accepted values still valid) | JOIN condition produced different fan-out |
| Dashboard showed stale data | Not tested (freshness check passed) | Incremental model skipped records |
In every case, the data was structurally valid. The tests did exactly what they were designed to do. The problem was that nobody was checking whether the actual values were correct.
How Did vaidukt Implement Systematic Validation?
Rather than hiring dedicated data quality engineers or building custom validation scripts, vaidukt integrated Recce into their existing CI/CD pipeline. The approach was straightforward:
- Every PR triggers automated data diffs — Recce compares the development environment’s data against production for affected models
- Schema diffs catch structural changes — column additions, removals, and type changes are flagged before review
- Profile diffs surface statistical shifts — if a column’s distribution changes meaningfully, the reviewer sees it immediately
- Row count diffs flag data volume changes — unexpected increases or decreases get attention before merging
- Impact analysis scopes the review — column-level lineage shows exactly which downstream models are affected by the change
The key insight was making validation automatic and pre-merge. The team did not need to remember to run checks or build custom scripts for each model. Validation happened on every PR, for every change.
What Were the Results?
The headline number — a 70% reduction in customer data complaints — captures the business impact. But the operational improvements were equally significant:
- Faster PR reviews — reviewers could see exactly what changed in the data, not just the code, reducing review time
- Fewer production incidents — catching issues before merge eliminated the investigation-and-fix cycle
- Higher team confidence — engineers were less afraid to refactor models because they could verify the data didn’t change unexpectedly
- Better stakeholder trust — fewer complaints meant stakeholders gradually stopped double-checking reports manually
For a three-person team, the time savings alone justified the investment. The hours previously spent investigating complaints could be redirected to building new models and improving existing ones.
What Can Other Small Data Teams Learn from vaidukt?
vaidukt’s experience reinforces several data review best practices that apply regardless of team size:
Automate Validation at the PR Level
Post-deployment monitoring catches problems after they reach users. PR-level validation catches them before merge. For small teams that cannot afford the incident response overhead, pre-merge validation is the higher-leverage investment.
Focus on Critical Models First
vaidukt did not try to validate every model from day one. They started with the customer-facing models that generated the most complaints, then expanded coverage as the process matured. This mirrors the approach of prioritizing CI checks on high-impact models.
Make Validation the Default, Not the Exception
The most important change was cultural: data validation became an automatic part of every PR, not something engineers did when they remembered. When validation is opt-in, it gets skipped under deadline pressure. When it is automatic, it becomes the team’s safety net.
Is 70% Complaint Reduction Realistic for Other Teams?
The specific number depends on where a team starts. Teams with no pre-merge data validation will likely see dramatic improvements. Teams that already do some manual validation may see more modest gains but save significant engineering time.
The underlying principle is consistent: checking actual data values before merging catches the semantic errors that structural tests cannot. Whether the improvement is 40% or 80%, the direction is always the same — fewer surprises in production, faster reviews, and more trust in the data.
Frequently Asked Questions
- How did vaidukt reduce data complaints by 70%?
- vaidukt, a German energy platform, reduced customer data complaints by 70% by implementing systematic data validation using Recce on every pull request. Instead of relying solely on dbt tests and post-deployment monitoring, their 3-person data team began reviewing actual data diffs before merging, catching semantic errors that structural tests missed.
- What is vaidukt?
- vaidukt is a German energy platform that manages data pipelines for energy industry operations. Their data team of three engineers maintains the dbt-based transformation layer that feeds customer-facing reports and operational dashboards.
- Can a small data team implement systematic data validation?
- Yes. vaidukt demonstrated that a 3-person data team can implement systematic data validation without dedicated data quality engineers. By integrating Recce into their existing CI/CD pipeline, they automated data diffs on every PR, reducing manual review burden while significantly improving data accuracy.
- What is systematic data validation?
- Systematic data validation is the practice of automatically comparing data output between development and production environments on every code change, rather than relying on ad-hoc checks or post-deployment monitoring. It catches value-level errors — like incorrect calculations or unexpected row count changes — before they reach production.