Engineering

The hidden cost of building agent infrastructure yourself

Every agent framework hands you routing, retries, and state as libraries to wire together yourself — but the real cost shows up months later, in production.

Marcus Webb

Teams almost never budget for agent infrastructure the way they budget for the agent itself. The agent gets a sprint, a demo, and a launch date. The infrastructure underneath it — routing, retries, observability, cost tracking — gets built reactively, one incident at a time, usually after something has already gone wrong in production.


The cost that doesn't show up in the sprint plan

If you asked most engineering teams how long it took to build their agent, they'd tell you a number measured in weeks. If you asked how long it's taken to keep it running reliably since, the honest answer is usually much larger — and much harder to pin down, because that work rarely gets tracked as a discrete project. It shows up as a Slack message here, a hotfix there, an afternoon lost to a mysterious cost spike, spread out over months instead of concentrated in a single line item.

This is the hidden cost: not the time spent building the happy path, but the ongoing time spent handling everything that happens when the happy path doesn't hold.


Why it's hard to see this cost while it's happening

Because this work is distributed across many small interruptions rather than one visible project, it rarely gets flagged in retros or planning cycles the way a failed sprint would. Each individual incident feels like a one-off — "that was a weird bug, fixed now" — rather than part of a pattern. It's only when you add every one of those interruptions up over a year that the real size of the cost becomes visible.


Three examples we hear constantly


The retry loop that didn't terminate

A team's agent hit a tool error, retried, hit it again, retried again — and kept retrying for six hours before anyone noticed, generating thousands of dollars in model calls for a task that should have failed fast after three attempts.


The silent model degradation

A provider's model quality dipped during a rolling update. Nothing crashed. Nothing errored. The agent just started giving worse answers, and nobody knew until a customer complained — because there was no baseline to compare against and no alert configured to catch drift.


The Friday deploy that couldn't be undone

A config change went out before a long weekend. It caused a spike in failed approvals. Rolling back meant reconstructing the previous config from memory and a few old Slack messages, because there was no versioning system tracking what had actually changed.


Why this keeps happening

None of these are failures of engineering skill. They're the predictable result of not having infrastructure designed for these specific failure modes — because building that infrastructure wasn't the assigned task. The assigned task was "ship the agent," and infrastructure for handling failure gets built only after failure has already happened once.

This creates a strange incentive structure: teams effectively wait for an incident to justify building the fix for that incident, rather than having the fix in place before it's needed. It's reactive by design, not because anyone chose it that way, but because there was never a natural point in the process to prioritize it proactively.


The planning problem underneath it

Nobody ever puts "build retry-loop termination logic" on a roadmap before a retry loop has failed to terminate. There's no natural trigger to prioritize infrastructure for a failure mode you haven't experienced yet — which means, structurally, this kind of work almost always happens after the fact, no matter how skilled or well-intentioned the team is.


What it actually costs, added up

Take a rough estimate: a mid-sized engineering team spends 15 to 20 percent of its time across a year on exactly this kind of reactive infrastructure work — investigating cost spikes, building ad hoc monitoring, handling silent failures, reconstructing what changed after an incident. That's not a small tax. Over a year, it's the equivalent of losing an entire engineer's worth of output to work that produces no new customer-facing value — it just keeps the existing system from breaking in new ways.


The fix isn't better planning

It's tempting to conclude the answer is "plan better — budget time for infrastructure up front." In practice, this rarely works, because the specific failure modes are hard to predict before you've hit them. You don't know you need retry-loop termination logic until a retry loop doesn't terminate. You don't know you need drift detection until a model silently degrades.

The actual fix is not needing to build that plumbing in the first place — using infrastructure where these failure modes have already been designed for, by people who've seen them across many different teams' agents, not just your own.

Teal gradient background

TRUSTED BY 200+ AI TEAMS

Ship your agents to production.

Book a 30-minute demo, or start in the docs. No infrastructure required.

Teal gradient background

TRUSTED BY 200+ AI TEAMS

Ship your agents to production.

Book a 30-minute demo, or start in the docs. No infrastructure required.

Teal gradient background

TRUSTED BY 200+ AI TEAMS

Ship your agents to production.

Book a 30-minute demo, or start in the docs. No infrastructure required.

Create a free website with Framer, the website builder loved by startups, designers and agencies.