Every agent framework starts with the same promise: here are the building blocks — routing, memory, tool-calling, retries — now assemble your agent. And for a prototype, that promise holds. You wire the parts together, it works on your machine, and the demo is impressive.

Then you try to put it in production. In production, the questions change: what happens when a model times out mid-run, who approves a risky action, why did the agent do that last Tuesday, what did it cost and which team pays. None of these are agent-logic questions — they're infrastructure questions, and frameworks leave them to you.

The gap between a demo and a product

A demo has to work once, in front of an audience that wants it to succeed. Production has to work every time, in front of users who don't care about your architecture and will simply stop using the product the moment it breaks. That difference in bar is enormous, and it's exactly the part frameworks don't address.

Frameworks optimize for time-to-first-demo. They give you the shortest path from "I have an idea for an agent" to "look, it works." That's a real and valuable thing to optimize for — but it quietly stops being the constraint the moment you ship. After launch, the constraint becomes: does it keep working, can you tell when it doesn't, and can you fix it fast when it breaks.

Why this distinction gets missed early on

Most teams don't discover this gap until they're already past it. The demo succeeds, leadership is excited, a launch date gets set — and the infrastructure questions that production requires haven't been asked yet, because nothing in the demo-building process forced them to the surface. They only appear once real traffic starts arriving, at which point they're urgent instead of planned.

What actually breaks in production

Three categories of failure show up almost immediately once real traffic hits an agent.

Model-level failures

Providers have outages, elevated latency, and rate limits. A framework doesn't know your fallback model, because you haven't told it there needs to be one — that's a decision you have to design and implement yourself, usually after the first outage teaches you it was necessary. Until that first outage, most teams don't even realize this is a gap, because a framework's happy-path documentation rarely dwells on what happens when the provider itself is the thing that fails.

Silent errors

A tool call returns malformed data. A retry loop doesn't actually terminate. An agent takes an action it shouldn't have. Without full tracing on every run, these failures are invisible until a customer reports one — and even then, reconstructing what happened requires log-diving that a framework doesn't set up for you by default. The absence of tracing isn't a missing feature so much as a missing category of concern; frameworks are built around what the agent does, not around what happens when you need to inspect what it did after the fact.

Cost drift

Nobody notices agent spend creeping upward until the invoice arrives. Frameworks have no opinion on budgets, because cost control isn't an agent-logic concern — it's an operations concern, and operations concerns are exactly what get skipped when you're focused on shipping the demo. A framework can't warn you about a cost spike it was never designed to track in the first place.

Why this isn't a framework problem to fix

It's tempting to think the solution is "a better framework" — one with retries and observability built in. But this misses something structural: a framework is a library you import into your own service, which means you're still the one responsible for running that service reliably, deploying it, monitoring it, and paying for the infrastructure underneath it. Adding more features to the library doesn't remove that responsibility; it just gives you more code to maintain.

A runtime is a different shape of thing entirely. Instead of code you import and operate yourself, it's a layer that already runs — deployment, retries, observability, approvals, and cost control included, running as a managed system rather than a library you're on the hook for keeping alive.

The operational responsibility doesn't disappear, it just moves

This is the part that's easy to miss when comparing a "feature-rich framework" against a runtime on paper. Even a framework with retries and logging built in still needs someone to run the service that uses it, patch it, scale it, and be paged when it goes down. A runtime takes that operational responsibility off your plate entirely — not by giving you better tools to do the work yourself, but by doing the work for you.

The actual dividing line

The useful question when evaluating any agent-building tool isn't "does it have routing" or "does it have memory" — most frameworks do, at this point. The useful question is: when this breaks in production at 2am, whose problem is it, and how do they find out?

If the answer is "yours, and you'll find out from a customer complaint," you have a framework. If the answer is "the platform's, and you'll get an alert before the customer notices," you have a runtime.

"The framework gets you to the demo. The runtime gets you to production."

We built Sublayr because we kept hitting this exact wall — building agents that worked beautifully in a demo, then spending months building the infrastructure to make them survive contact with real users. That infrastructure is not differentiated work for any individual team. It's the same problem, solved the same way, for every team building agents. So we built it once, as a runtime, instead of asking every team to rebuild it themselves.

Elena Vasquez is co-founder and CEO of Sublayr. She spent a decade running platform engineering before starting Sublayr.