<!-- Source: automation.html. Author: Gonzalo Rodriguez Pardo. -->

04 · What I do

# Automation

Data strategy · 5 min read

A data team spending all of its hours keeping things alive is a team running entirely on operating expense. Pipelines restarted by hand, numbers reconciled before every meeting, requests answered one at a time, checks performed by someone remembering to perform them. It works, in the sense that the lights stay on. It also means the team is breathing, not growing, and that every additional demand has to be paid for with additional people.

In short

- Work repeated to keep systems alive is operating expense, and it scales only by adding people.

- Automation is capital expense: it costs delivery up front and returns on every run afterwards, building value on top of value already achieved.

- Spend it on scalable process automation, assurance through tests and contracts, and monitoring, funded by a protected share of every sprint.

## Where the hours actually go

The honest first step is an inventory, because manual work rarely appears on any plan. Nobody writes a ticket for the twenty minutes spent every morning checking that last night's load finished, or the half day before board week spent reconciling two numbers that should agree, or the recurring extract someone produces by hand because it was urgent once and became a habit.

Counted honestly, this is usually a startling share of the week. And it has a particular quality: it produces nothing that will still exist tomorrow. The work has to happen again, at the same cost, next week and the week after. That is the definition of operating expense, and the reason a team can be fully occupied and still not moving.

## Operating expense scales with headcount, and nothing else

The trouble with a system held together by attention is that its capacity is bounded by the number of people paying it. To handle twice the pipelines, twice the reports, twice the requests, you need close to twice the hours. In practice you need more than that, because coordination costs rise with team size and each new person needs someone experienced to explain how the manual parts work.

So hiring becomes the only available growth lever, and it is a lever that never resets. Every person added to keep the current system running is a permanent commitment against future budget, and none of that spend accumulates into anything. Stop paying it and the capability disappears the same week.

## Capital expense compounds

Automation is the other kind of spending. It costs engineering time up front and produces an asset: an ingestion framework that absorbs the next twenty sources, a test suite that makes every future change safe, a monitor that watches something nobody now has to remember. The expense is paid once; the benefit arrives every day afterwards.

It also builds on itself. The second pipeline is cheaper than the first because the framework exists. The tenth is cheaper still. Where operating expense buys the same day repeatedly, capital expense buys value on top of value already achieved, and the gap between the two approaches widens the longer the system lives.

> Operating expense buys you today. Capital expense buys you every day after it.

## The trade is real, and it is short term pain

None of this is free, and pretending otherwise is how automation programmes lose their funding halfway through. Building the framework means the quarter it is built in delivers fewer reports. Writing the tests means the feature ships later. Instrumenting the pipeline means someone is not answering an ad hoc question that a stakeholder genuinely wanted answered.

That is the deal: giving up a short term benefit to buy a long term one. It only survives if it is made explicitly, with the people whose requests will be slower for a while, and framed in the terms they care about. Not that the team wants to build tooling, but that in six months the same team will absorb three times the demand without asking for more headcount.

## What the investment actually buys

Three things, in roughly this order. First, scalable process automation: orchestration that handles retries and dependencies without a human in the loop, ingestion that is configured rather than coded per source, transformation that runs the same way in every environment, and self-service paths so that routine requests never reach a person at all.

Second, assurance. Tests on the data as well as the code, contracts at the boundaries where other teams supply input, continuous integration that will not let a broken change through, and pipelines reproducible enough that rerunning yesterday produces yesterday's numbers. Assurance is what converts a fast system into a trusted one, and trust is the thing that stops people building private spreadsheets.

Third, monitoring. Freshness, volume and quality checks that alert the owner before the consumer notices, lineage that answers what broke and what it touches, and cost visibility per workload so that spending stays connected to value. Monitoring is what allows a small team to stop watching, which is the entire point.

## Funding it without stopping the world

The practical mechanism is a standing share of capacity rather than a project that competes with delivery and loses. A fixed fraction of every sprint, protected, spent on whatever is currently consuming the most manual hours. Measure it: hours of toil before, hours after. It is the only way the investment stays visible to people outside the team.

Sequence by repetition, not by irritation. The task that runs daily and takes ten minutes is worth more automated than the quarterly ordeal everyone complains about. And prefer eliminating work to accelerating it. A report nobody reads should be deleted rather than automated; a reconciliation that exists because two systems disagree is better fixed at the source than scripted forever.

## A team that stays small on purpose

The goal is not a smaller team for its own sake. It is a team whose size is decided by the ambition of what it is building rather than by the volume of what it is maintaining. Those are very different organisations, even at identical headcount: one spends its week on yesterday, the other on next year.

Which is why this is a leadership decision before it is an engineering one. Left alone, the manual path always wins in the short run, because it is faster this week and the cost lands later and elsewhere. Somebody has to decide, deliberately and repeatedly, to buy the asset instead of renting the outcome.

Written by Gonzalo Rodriguez Pardo, Chief Data Officer
