Should You Switch to Sonnet 5? We Ran It Against Opus and Read the Invoice

A cost-per-outcome look at Claude Sonnet 5 vs Opus 4.8 on the same real task.

Anthropic shipped Sonnet 5, and the first thing most builders noticed was not the benchmarks. It was the price. $2 per million input tokens and $10 per million output tokens on the introductory rate, for a model Anthropic positions as approaching Opus. If you are running a previous-generation Sonnet in production, the instinct is immediate: swap the new one in and pocket the upgrade.

Before you change a line of code, it is worth testing that instinct. We did, and the result was not the clean upgrade we expected. It changed how we think about switching models at all.

Our rule for any new frontier model is simple. We do not trust benchmarks. We do not trust launch-day threads. We run the model against our own real workloads, and then we look at the bill.

The setup

The app we build, Associum, is an AI associate for knowledge work. Under the hood it is a model-agnostic harness: rather than being wired to one provider, it routes each task to whichever model fits, across Anthropic and GPT-class models. That gave us a clean way to run this comparison.

Associum exposes two execution modes. Smart Mode handles the bulk of day-to-day work and runs on Sonnet-class models. Expert Mode is reserved for harder reasoning and runs on Opus-class and GPT-class models.

So the experiment picked itself. Put Sonnet 5 in Smart Mode. Keep Opus in Expert Mode. Give both the exact same brief, a full financial projection, and compare the results.

You can read both runs in full and judge the quality for yourself:

● Opus, Expert Mode: https://staging.associum.ai/s/chat/6b32b206-841c-46ee-aa60-ff220ba05945

● Sonnet 5, Smart Mode: https://staging.associum.ai/s/chat/21568194-f7dc-465a-a87c-e5f0f0484f0f

I expected Sonnet 5 to be a clean, cheaper upgrade for Smart Mode. That is not what happened.

The result nobody on my team predicted

Sonnet 5 in Smart Mode did not just hold its own against Opus in Expert Mode. It beat it.

We took both outputs and handed them, blind, to three independent LLM judges. Every one of them scored Sonnet 5's output higher. Not by a landslide, but without a single exception:

Different judges, same verdict every time. Sonnet 5 was the better piece of work, by somewhere in the range of ten to seventeen percent.

Let me be clear about how strange that is. A model I had slotted into the cheap, high-volume tier was outperforming the tier I reserve for the hardest problems, on default settings. When you push Sonnet 5, it does not behave like a modest successor to the previous Sonnet. It behaves like something that finishes work the older models would have left half done.

Then I looked at the invoice.

The part that changed my mind

For the exact same task, Sonnet 5 burned $4.30 in tokens. Opus burned $1.90. That is 2.3x the cost for the same job.

That is the part worth sitting with, because the per-token price points the other way. Sonnet 5 lists at $2/$10. Opus 4.8 lists at $5/$25. On paper, Sonnet is far cheaper per token. And it still came out more than twice as expensive per completed task.

Three things drove that.

It generated far more output. The quality came from the model doing more thinking and producing more, not from being efficient.

It made far more tool calls before landing on an answer: 48 versus 15 for Opus, more than three times as many.

And it uses a new tokenizer. The same text now maps to meaningfully more tokens than before, somewhere between the same and about a third more depending on the content. So even at a lower headline rate, you are billed for more tokens to say the same thing.

Put those together and a lower price per token turned into a much higher price per outcome. Here is the whole experiment on one line:

Better output. More than double the cost to produce it.

Benchmarks do not pay your cloud bill

This is the lesson the switching decision really turns on, and I think a lot of teams are about to learn it the hard way.

When we compare models we fixate on the wrong numbers. Cost per million tokens. Benchmark rankings. Elo scores. None of those is the number that shows up on your invoice.

The only metric that matters in production is cost per successful outcome. How much does it cost to solve one real business problem, end to end, including every retry and every tool call along the way?

By that measure, a model that scores ten percent higher while costing more than twice as much is not automatically an upgrade. For some tasks it will be worth it. For most high-volume work, it is a worse production choice wearing a better benchmark.

The trap underneath the switch

Here is the part that should make any team pause before standardizing on one provider.

A single pricing and tokenizer change did this. Not a worse model. A better one, launched with attractive introductory pricing, that quietly shifted the token economics underneath us. We did not change our prompts. We did not change our workload. The cost profile moved because the vendor's model, tokenizer, and behavior moved.

Now imagine you have built your entire product on one provider. You have said, proudly, “we are all in on Vendor X.” That feels like conviction right up until Vendor X changes the economics. One tokenizer update, one routing policy, one pricing decision, and your inference cost can double overnight. You do not control their roadmap. You do not control their pricing. You inherit both.

The teams happily standardizing on a single AI vendor today are not wrong to like the model. They are underestimating how little of their own cost structure they actually control. That is a fragile place to build.

So, should you switch?

For our workloads, today, the answer is no. We are not moving Sonnet 5 into Smart Mode or Expert Mode yet.

It is an excellent model. On the right task it clearly outperforms Opus. But “better output” is not what you ship on. You ship on cost per outcome, and at more than twice the cost for a modest quality gain, Sonnet 5 has not yet earned a default slot in either tier. Making it the default now would be us making the very mistake this whole exercise warns against: adopting a model because it is impressive rather than because it is the right tool for a specific job. When we can pin down exactly where its quality justifies its cost, it goes in. Not before.

That patience is only possible because the harness is model-agnostic. Every model has strengths, weaknesses, and its own cost curve, and as this test showed, that curve can move the moment a vendor ships an update. Our job is not to be loyal to a vendor. It is to pick the model that delivers the best combination of intelligence, reliability, latency, and cost for each task, and to switch the moment that calculation changes.

The real advantage

The cloud taught us to avoid locking our infrastructure into a single provider. AI deserves the same discipline.

The biggest edge over the next few years will not be access to the smartest model. Everyone will have that. The edge will be the freedom to move to the smartest model only when it actually makes economic sense, and to move off it just as fast when it stops.

The cheapest model is not always the best. The smartest model is not always the best. The best model is the one that returns the most business value per dollar. That is the number the switching decision should turn on, and it is the one thing a single-vendor bet quietly takes away from you.