The Challenges of Driving a Data-Driven Culture

The Frustrating Paradox

Here is something nobody tells you about building data infrastructure: you can do everything right and still have most of your organization not using it.

By early 2026, the data environment I manage for my team had 203 production tables, 162 published views, and six fully operational medallion pipelines running on automated schedules. The data was there. The quality was improving. The architecture was sound.

And yet: only about an eighth of our organization qualified as “Intermediate or above” on an internal measure of data-tool adoption.

That number is the most humbling metric I’ve worked with in my career. Not because it means the infrastructure failed — but because it reveals that infrastructure was never the real problem.

---

What Is a Data-Maturity Measure?

Before I get into the lessons, some context.

We track an internal measure of how much each person actually uses the data tooling — not whether they have access, but whether they query, build, and explore with any depth. “Intermediate or above” means someone regularly uses multiple data tools meaningfully.

At the start of 2026, only about an eighth of the organization cleared that bar. The goal for the year was to lift it substantially.

That gap is not a technical problem. You can’t solve it by building more pipelines or publishing more dashboards. I know because we tried that approach, implicitly, and it didn’t move the number.

---

The Infrastructure Fallacy

The easiest assumption to make as a data engineer is that people will use what you build if you build it well enough.

They won’t.

This is what I’d call the infrastructure fallacy: the belief that access equals adoption. That publishing a clean, well-documented dataset is equivalent to having that dataset used. That putting a dashboard on Confluence is equivalent to having someone act on its insights.

The fallacy is seductive because the technical side of data work is legible. You can measure whether a pipeline runs. You can see whether a table exists. You can verify whether a view is queryable. All of those things feel like progress, and they are — but they’re progress toward a necessary condition, not a sufficient one.

The sufficient condition is something much harder to engineer: a team that reaches for data before reaching for intuition.

---

Three Real Blockers

After spending months running interventions — leader meetings, hands-on sessions, individual coaching — three patterns emerged as the real obstacles. None of them are technical.

1. The Invisible Pipeline

A table with no description and no consumers is worse than no table at all. At least an absent table prompts someone to ask for it. A table that exists but can’t be understood creates false confidence: “We have data on this.” The reality: you have rows in a database that nobody can interpret.

We found this the hard way when we catalogued our governance audit results. 145 tables were flagged as low-quality — meaning their metadata was too generic to be useful. “device data” as a table description tells you approximately nothing about what the table contains, who owns it, or why it exists.

The fix sounds simple: write better descriptions. The underlying problem is a cultural one — data engineers (myself included) are trained to build systems, not to document them for non-engineers. The mental model we use to understand a table is not the mental model our consumers need.

2. The Language Gap

Leaders in operations, IT, and security speak in headcount, cost reduction, and incident count. Data engineers speak in schemas, SLA percentages, and cardinality.

In my first round of leader meetings, I showed up with a slide deck explaining our Data Mesh architecture, medallion pipeline design, and governance framework metrics. I got polite nodding. Zero follow-up action.

In the second round, I showed up with a different framing: “You have 17 people on your team who are one tool away from being classified as Intermediate. Here’s who they are and what they need to do.” That conversation lasted 45 minutes and led to a scheduled hands-on session the following week.

The data didn’t change. The frame changed. Leaders don’t need to understand Bronze/Silver/Gold architecture. They need to understand what their team’s data maturity score means for their operational goals — and they need someone who can translate between the two.

3. The Permission Problem

There’s a third blocker that’s harder to articulate: people need a problem they feel, not one you diagnose for them.

I could show a team that their Looker Studio dashboard has 89 viewers and 33 editors. That statistic is real and meaningful to me. To the team’s leader, it means nothing — because they didn’t feel the problem that the statistic represents. They didn’t experience the frustration of having data they couldn’t modify. They didn’t ask the question that the statistic answers.

The most effective interventions I ran were the ones where I started with a problem the team already felt: “Your team spends 2 hours every Monday morning extracting data for the weekly report. Here’s how to build a dashboard that does it automatically.” That’s a problem they can feel. The answer happens to require data skills.

---

What Actually Moved the Needle

After eight months of working on data culture — which is a strange thing to say, because you can’t really work on culture, you can only change conditions — a few things demonstrably worked.

flowchart TB
  A["Broadcast: 'come learn the tools'"] --> A1["Shows up: people who<br/>already know the tools"]
  B["Cohort: 'your team, one step<br/>from fluent — here's who'"] --> B1["Shows up: the people<br/>who actually need it"]

Cohort-based interventions over broadcast announcements. A general “come learn BigQuery” session gets attendance from the people who already know how to use BigQuery. A session specifically designed for “the 17 people on a specific leader’s team who are closest to Intermediate” gets attendance from the people who need to be there.

Hands-on sessions over presentations. Every hour spent explaining data tools is worth ten minutes of having someone actually use them. The goal of any session is to get someone to their first successful query, their first chart, their first saved filter. The theory doesn’t matter until they’ve had the experience.

Consumption builds familiarity. Every dashboard we published and shared was a low-friction on-ramp: people who started by just viewing data gradually started filtering it, then editing it, then building their own. The infrastructure and the culture program fed each other — the more useful, findable data we shipped, the more people interacted with it, and interaction is the first step toward fluency.

The “edit, don’t just view” message. When we looked at dashboard usage, the vast majority of people only ever viewed — a small fraction ever edited. That gap between consuming data and interacting with it is the real target. The simplest intervention: tell people explicitly that editing — adding a filter, renaming a field, building a simple chart from an existing source — is something they’re allowed and able to do. Most assume dashboards are read-only objects created by data teams.

---

The Alignment Feedback

At some point in this process, I received feedback that I “needed more alignment” on the cultural dimension of my work.

I’ll be honest: my first reaction was defensive. I was running leader meetings, building data products, shipping pipelines, and managing governance metrics. How was I misaligned?

But the feedback was valid, and understanding why took several weeks of reflection.

The misalignment wasn’t about effort — it was about communication. I was doing deep infrastructure work and reporting on outputs (tables built, pipelines running) while the organization needed to hear about outcomes (what teams could do now that they couldn’t do before, why that mattered for their goals).

Architectural excellence without communication is infrastructure without users. That’s what I had. A technically correct system that hadn’t yet made its case to the people who needed to use it.

The lesson isn’t that you should communicate instead of building. It’s that communication is part of building. Every data product needs a narrative — not a technical brief, but a story about what problem it solves and for whom. Writing that narrative is not a soft skill peripheral to data engineering. It is, increasingly, the core competency that separates infrastructure from impact.

---

Closing: Eventual Consistency

Data culture doesn’t update atomically. You can’t deploy it, push it to production, and see it take effect in a single transaction. It operates more like distributed systems: eventual consistency, with temporary divergence between what you’ve built and what the organization actually uses.

The work is to keep shipping, keep communicating, and trust that the gap closes over time. Some nodes update faster than others. Some require more interventions, more replication, more patience. The architecture eventually propagates.

By March 2026 the number had climbed by roughly a third over the quarter. Slow progress. Real progress. The target is still a good way off.

The pipelines are running. The dashboards are live. The sessions are scheduled. The conversations are happening.

That’s all you can do: build the conditions, show up consistently, and let adoption follow.