My content pipeline spent $2.07 on model calls over the last thirty days. My coding agent spent more than that before lunch on Tuesday. Both numbers are correct, and the gap between them is basically the whole subject of this post.
For about two years I didn’t think seriously about which model handled which job. There was no point. Whatever I was running, something cheaper and better would land in four months and quietly fix whatever I’d been annoyed about. Optimising around a model felt like optimising around a specific CPU stepping. Wasteful, and obsolete by the time you finished.
That stopped being true this year, and Drew Breunig put a name on it that I keep coming back to.
The free lunch, and the essay it’s named after
Breunig’s piece is called Fable and the End of the Free Lunch, and the analogy is Herb Sutter’s 2005 essay The Free Lunch Is Over. Sutter’s argument was that while Moore’s Law held, you didn’t need to optimise your code, because a faster CPU was coming. When single threaded performance stalled in the mid 2000s, suddenly everyone had to care about parallelism, memory locality, and where work actually ran.
Breunig’s claim is that inference just hit the same wall, for different reasons. Not because models stopped improving. Because the best one got expensive enough that you stop reaching for it reflexively. His line is the one that stuck with me: “So we started to think about what work went where.”
I want to push back on the framing slightly, because I think it’s less “the free lunch ended” and more “the menu got longer”. Cheap models didn’t get worse. They got dramatically better while the top of the range got more expensive. That’s a wider spread, not a ceiling. But the practical consequence Breunig describes is right either way: for the first time, model choice is a real engineering decision instead of a default.
What the billing data actually shows
The Ramp AI Index is the most interesting number I’ve seen on this, because it’s built from actual corporate card spend rather than survey responses or vendor announcements.
Their July figures: Fable 5 accounted for roughly 6% of the tokens businesses bought from Anthropic, but about 11.4% of the dollars. That’s the price gap showing up directly in the data, at roughly $10 per million tokens, around twice GPT-5.6 Sol. And Opus 5, which shipped later in July at a lower price, had already overtaken Fable 5 in enterprise spending by the time the index came out.
The honest caveat: Ramp sees Ramp customers. That’s tens of thousands of mostly US companies, skewed toward startups and mid market, and it’s not the whole market. Anyone building a thesis on one dataset should hold it loosely. But the shape matches what I hear from every team I talk to, which is that somebody looked at the invoice and asked whether the flagship model was doing flagship work.
Usually it wasn’t. Usually it was renaming variables.
One model to decide, another to type
The split I landed on is boring and it works: use the expensive model to think, the cheap model to execute.
In practice that means I’ll have a real conversation with a frontier model about a design, argue with it, let it poke holes in what I want to do, and then write a tight brief. The brief goes to a much cheaper model that does the actual editing. Breunig describes the same workflow, and points out that GLM 5.2 runs at roughly a ninth of Fable’s cost and about a fifth of Opus 5’s.
Is a ninth of the price a ninth of the quality? For rote work, no, not close. Renaming things, writing tests against a spec you already wrote, converting a schema, filling in a repetitive component. That work barely cares which model does it as long as the context is good.
The catch is that “as long as the context is good” is carrying an enormous amount of weight in that sentence. A cheap model with a precise brief beats an expensive model with a vague one, most of the time. A cheap model with a vague brief produces confident garbage that costs you an hour to unpick, which wipes out the savings for the whole month. The savings are real, but they’re contingent on you doing more work upfront, and that trade isn’t free.
What routing looks like in code
The naive version, which is what I had for a long time:
def summarise(article: str) -> str:
return client.messages.create(
model=EXPENSIVE_MODEL,
max_tokens=1024,
messages=[{"role": "user", "content": f"Summarise this:\n\n{article}"}],
).content[0].text
Every call, same model, no thought required. Which was the point, back when thinking about it was a waste of time.
Here’s roughly what replaced it:
TIERS = {
"cheap": "glm-5.2",
"standard": "claude-opus-5",
"hard": "claude-fable-5",
}
ROTE = {"summarise", "classify", "extract", "rename", "reformat"}
def pick_model(task: str, input_tokens: int, attempt: int) -> str:
if attempt >= 2:
return TIERS["hard"]
if task in ROTE and input_tokens < 32_000:
return TIERS["cheap"]
if input_tokens > 120_000:
return TIERS["hard"]
return TIERS["standard"]
Three things about this that took me embarrassingly long to work out.
The attempt parameter matters more than the task type. Escalating on retry means a cheap model gets first crack at everything and you only pay for the expensive one when the cheap one demonstrably failed. That single rule saved me more than the task classification did.
Input size belongs in the routing decision. Long context is where cheap models fall apart in ways that are hard to detect, because the output still looks fine. It’s just quietly wrong about something on page four.
And you need the escalation to be observable. If you can’t see how often you’re falling through to the expensive tier, you have no idea whether your router is saving money or just adding a failed cheap call to the front of every expensive one. I log the tier on every call and check the ratio weekly. When it drifts above about 20%, my briefs have gotten sloppy.
The pushback, taken seriously
The obvious counter is that inference prices always fall, so this is temporary and in a year we’ll all go back to sending everything to the biggest model.
Breunig’s answer, which I find convincing, is that the same efficiency gains lift the cheap models too. If everything drops by the same factor, the ratio holds, and the ratio is what drives the decision. Meanwhile harnesses keep improving, which makes it easier to feed a weaker model enough context to do well.
There’s a second reason I think the split sticks, and it has nothing to do with price. Frontier model access is getting more conditional: access controls, data retention requirements, degradation under load. Once a company has done the work to route some traffic to a model it can run or replace, that capability doesn’t get thrown away just because the flagship got cheaper. Optionality is worth something on its own.
I’m not certain about any of this. Ask me in six months. But building the router costs a day and the downside is a day.
The cost nobody puts in the spreadsheet
Every cost comparison I see counts tokens. Almost none of them count evaluation.
Routing means you now have at least two models producing output, and you need some way of knowing whether the cheap one is doing an acceptable job. That’s an eval harness, and it costs real money to run and real time to maintain. Same for retries, which are pure overhead when the cheap tier misses.
For a small pipeline like mine this is fine, because the volume is low and the failure mode is a bad blog draft rather than a bad migration. For anything customer facing, budget for it honestly, or you’ll build a router that saves 40% on tokens and spends it back on engineering time. I wrote about picking model size for a specific job in when a small model is the right call, and the same rule applies here: measure on your workload, not on a benchmark.
What I’d do this week
Pull your last thirty days of model spend and group it by task, not by model. Most billing dashboards won’t do this for you, which is itself informative. You probably need a tag on each call.
Then find your single highest volume task and try it on a model that costs a fifth as much, with a better brief than you’d normally write. Run both outputs side by side on twenty real examples. Not a benchmark. Your actual inputs, including the ugly ones.
If the cheap model holds up, you’ve found your first routing rule. If it doesn’t, you’ve learned that your expensive spend is justified, which is worth knowing too and takes an afternoon to establish.
Most of the automation work I build for clients now has a router in it somewhere, and it’s usually the least clever part of the system doing the most for the invoice. There’s more on how I approach that kind of build on my site. The lunch wasn’t free before either. We just weren’t the ones paying for it.
