Same Token Price, 3.5x the Bill: Sol vs Sonnet 5.5

Abstract

Anthropic released Claude Sonnet 5.5 on September 28, and OpenAI released GPT 6.1 Sol on September 29. The two models carry identical API prices on OpenRouter: 2 dollars per million input tokens and 10 dollars per million output tokens. A hands-on evaluation ran both models through the same workbench and the same reseller channel across three real tasks: 3D escape-room development, a current-policy investigation, and a Nobel chemistry explainer. GPT 6.1 Sol spent 38.5 dollars. Claude Sonnet 5.5 spent 10.9 dollars. The gap is about 3.5 times, despite identical unit pricing. This article reconstructs the test, explains why the cost and time advantage reversed three times across the tasks, and draws conclusions about what actually decides the value of a model in production.

1. The Test Design

The two labs chose a similar product direction: compress near-flagship capability into a faster, cheaper layer built for agent work and professional tasks. Sonnet 5.5 emphasizes finishing clearly bounded daily work with fewer steps. GPT 6.1 Sol targets complex coding, computer operation, and professional work.

The evaluation used Claude Code CLI as the test platform and called both models through the OpenRouter reseller. Both models could search the web, read and write files, run commands, and drive a browser.

Three tasks were chosen deliberately. Each stressed a different capability. The escape-room task tested long-horizon coding with a hard self-verification requirement. The policy task tested time-sensitive evidence discipline. The chemistry task tested the ability to separate factual sourcing from pedagogical explanation.

2. Task One: 3D Escape-Room Development

The requirement was to build a first-person 3D escape room with Three.js: three rooms and six interactive objects. The constraints were strict:

• Objects must form a chained dependency, not six parallel puzzles. Object A unlocks B, B and C combine to open D, D opens the third room, and so on. The chain must be drawn explicitly.
• Every object click must give clear feedback. If an object is not usable yet, the prompt must say why, not respond silently.
• A current-goal hint must keep the player from getting stuck.
• The full flow must be completable within five minutes.
• All files and intermediate files must stay in the current folder.

After building, the model had to walk the completion flow from start to finish and write down every step and its feedback. If it could not finish, it had to state where it got stuck.

GPT 6.1 Sol produced a game called AFTERLIGHT with a complete retro duty-room look: green wainscoting, a checkered floor, three-dimensional signage, an inventory bar, a sound switch, and a persistent goal hint. The six objects formed a two-branch dependency chain: fuse A restores power to panel B, access card C and the power state jointly unlock terminal D, which opens the third room, where key E opens exit F.

Sonnet 5.5 built a gray cell and electrical-room style with fewer decorations, but the puzzle chain stayed complete. The player takes a screwdriver from under the mattress, opens a vent, and gets the janitor key. The key opens both the cell door and a desk drawer. The drawer's access card unlocks the tool cabinet and yields a battery. The access card plus the battery opens the exit electronic door. The light portal in the third room only triggers the settlement, and it was not counted among the six interactive objects.

The evaluators noted a contrast in feel. Sol's version read like a checklist journal, with every step marked, and the access card design looked physically suspended. Sonnet's version had suspense, with realistic key texture and keyhole details.

Timing: GPT 6.1 Sol finished in 33 minutes 37 seconds. Claude Sonnet 5.5 took 74 minutes 32 seconds, roughly 2.2 times longer.

3. Task Two: Current-Policy Investigation

This task tested time-sensitive facts and evidence discipline. Executive orders, international joint statements, export rules, and research-safety policy were mixed together, making it easy to dress political wording as a technical breakthrough or to inflate a search summary into an established fact.

The prompt required a report for Chinese technical practitioners, based on United States Executive Order 14434, the Kyoto Vision, and related primary sources:

1. Explain what the "AI renamed to SI" change did and did not alter at the legal, technical, and industry levels.
2. Build a claim, original-evidence, credibility, and possible-misreading table.
3. Analyze three actual impacts on Chinese AI companies, open-source communities, and research cooperation.
4. List five plausible but inaccurate Chinese headlines and correct each one.
5. Every time-sensitive fact must carry a source. Policy wording must not be treated as a technical breakthrough.

GPT 6.1 Sol first wrote its verification method, then ran a deep-research workflow, and delivered a report of 14 files: the formal report, a source and verification record, a method note, five official original documents, four auxiliary extracted texts, and a SHA-256 checksum list. The five originals totaled about 600,000 bytes. Downloads passed hash verification. The five core conclusions were each handed to three independent verification passes, leaving 15 judgments, followed by a full review and revision of the report.

Sonnet 5.5 finished in 5 minutes 11 seconds, delivering a 172-line report of about 23,000 bytes using 32 tool calls. The structure was complete: the same 12-row verification table, three China impacts, five misleading headlines, and four dates worth tracking. It also caught a difference between the White House text and the Chinese side on whether the term change was agreed, an angle the evaluators called highly valuable.

The time contrast was extreme: Sonnet took 5 minutes 11 seconds, while Sol took 54 minutes 50 seconds, about 10.6 times longer.

The evaluators' reading: Sonnet drew a news map quickly, while Sol worked to patch boundaries and ensure data truthfulness. For fast, high-timeliness news, Sonnet gets a first version sooner. For legal boundaries, China-related policy, and auditable citations, Sol's evidence package came closer to a reviewable draft.

One concrete output difference stood out. Sonnet's report noted that Executive Order 14434 section 3(a) defines SI as the same AI under 15 U.S.C. 9403(1), with no capability threshold, and that it binds only the US federal executive branch. Section 3(b) requires a legislative proposal for a federal SI definition by about November 28, but that is only a recommendation. The Kyoto Vision was a one-page non-binding statement referencing 17 congressional committees, without defining SI or mentioning China, open source, or export controls. The White House outcome list said the two leaders agreed to use the term super intelligence, while the Xinhua list only mentioned a China-US AI dialogue, and the foreign ministry said it respected the US formulation.

4. Task Three: Nobel Chemistry Explainer

The task required the model to fetch the official 2026 Nobel Prize in Chemistry announcement and popular-science material on its own, then deliver:

1. An explanation of chirality, enantiomers, nonlinear effects, and asymmetric autocatalysis accessible to ordinary readers.
2. A clear account of which step Kagan and Soai each solved, without conflating them.
3. A single-file HTML interactive demo showing how a tiny initial bias is amplified in autocatalysis.
4. An explicit statement that the demo is a teaching model, not an exact kinetic reproduction of the Soai reaction.
5. A five-page presentation outline, with all numbers, years, and drug-related wording verified.

The task forced the model to act as science editor, front-end developer, and fact-checker at once. The hardest part was keeping three layers separated: award facts from official material, chemistry explained for laypeople, and an interactive page that only shows mechanistic intuition without pretending to reproduce Soai kinetics.

GPT 6.1 Sol delivered 42 files: seven documents, one interactive page, a five-page outline, and three verification scripts. Of 19 fetch records, 13 source files were archived successfully with size and hash verification. Core sources covered the Nobel announcement, popular-science material, scientific background, and FDA documents. Sol distinguished R/S absolute configuration, optical rotation direction, and D/L naming. It split the timeline into five nodes: Frank 1953, Kagan 1986, Soai 1990, 1995, and 2003, avoiding the conflation of theory, nonlinear transfer, autocatalysis, and small-bias amplification into one discovery.

Sonnet 5.5 delivered five files: a plain-language explanation, an interactive page, a five-page outline, a fact-check table, and a README. Its interactive page offered three modes: one batch, multi-batch relay, and random side selection. The user could adjust initial ee, mutual inhibition strength, and catalytic selectivity. Setting inhibition to zero made the amplification disappear. The random mode ran 60 times at once, letting the reader see how small fluctuations decide the final direction.

Timing: Sol took 65 minutes 41 seconds, and Sonnet took 12 minutes 54 seconds, a gap of about 5.1 times.

Both models correctly separated Kagan, Soai, and the teaching model. The advantages fell on different sides. Sol emphasized conceptual boundaries, historical nodes, and drug-case corrections, and leaned on Nobel and FDA primary sources. Sonnet leaned on fact-checking and scientific background documents, using partial web-summary data as material. Sonnet's interactive design and numerical self-check were richer. Sol's source archiving, FDA original correction, and real-browser verification formed a more complete publication loop.

5. Why Time and Cost Reversed Three Times

The two models were positioned in a similar band from the start. OpenAI defines GPT 6.1 Sol as providing near-Astra complex-work performance at lower cost, with a context window of 1.05 million tokens and a maximum output of 128,000 tokens. Anthropic positions Claude Sonnet 5.5 as faster and cheaper than Opus 5.5, aimed at daily tasks, bug fixes, and document, slide, table, and design work.

On the Artificial Analysis Intelligence Index, version 4.3.2, which integrates ten evaluations, both models sit in the leading band. The index data is not yet public.

The time difference traced to different action paths. In the escape room, Sol finished implementation and real completion faster, while Sonnet reworked repeatedly across Playwright versions, browser dependencies, and low-frame-rate environments, with automated walkthroughs stretching the round. In the policy and chemistry tasks, the pattern reversed: Sonnet produced retrieval and a report framework quickly, while Sol spent more time on source archiving, hash verification, multi-pass review, and full-text audit.

Wall-clock time for an agent depends on model strategy, tool availability, environment failures, and acceptance depth. It cannot be compressed into a fixed speed ranking.

The pricing was identical: 2 dollars per million input tokens, 10 dollars per million output tokens, and 2.5 dollars per million cache-write tokens on OpenRouter. The billing panels showed GPT 6.1 Sol at 38.5 dollars and Claude Sonnet 5.5 at 10.9 dollars, roughly 3.53 times. By share of total, Sol was 78 percent and Sonnet 22 percent.

Same unit price, a 3.5-times bill gap. The cause was the action path. Sol launched multi-pass verification in the policy task, archived many originals and ran a real browser in the chemistry task, and repeatedly walked the full flow in the escape room. More tool calls, longer context, and more repair rounds all enter the bill. Sonnet more often chose the shortest deliverable path, especially in the policy task.

6. The Real Cost Variable Is Not the Token Price

The takeaway is sharp: token unit price is one cost item, not the cost of a task. Two models with identical per-token pricing can differ by 3.5 times on the final bill because of how many calls, how much context, and how many retries each task needs.

For teams that run agents in loops, the metric that matters is cost per completed task, not cost per token. A model with a higher per-request spend can be cheaper per task if it needs fewer repair rounds. A model with a low per-token price can be expensive per task if it burns calls on rework.

This is why cost observability has to sit at the task level. When multiple models and providers are involved, the accounting needs a single place that records every call, the model that served it, the tokens spent, and the task it belonged to. An API gateway such as 4sapi provides that layer: one stable interface across providers, with per-model routing, retry handling, and centralized token accounting. A team can then compare cost per task across model variants without stitching together logs from several billing consoles.

The same mechanism that makes the bill attributable also makes the model choice reversible. If one model proves too expensive per completed task, the routing can switch to another without touching application code.

7. Where Mid-Tier Models Now Sit in the Industry

The test reflects a visible shift: frontier models are moving from content generators into task-execution systems. They search materials, read originals, write programs, drive browsers, fix errors, and assemble deliverables. The yardstick expands accordingly. Answer quality still matters, but whether the process can be audited, whether failures are visible, and whether cost is controllable now decide whether a model can enter real work.

In industry layering, Sol and Sonnet represent the mid-tier models of their two companies, and they are becoming the default working layer of enterprise AI systems. Flagship models explore capability ceilings, small models handle high-frequency simple tasks, and mid-tier models absorb the largest number of complex production tasks. What enterprises actually buy is stable, verifiable, cost-predictable task success. Token unit price is only one line in that cost.

8. Two Different Engineering Philosophies

GPT 6.1 Sol demonstrated the potential to take on research-engineering and quality-assurance work. It delivered raw materials, hashes, test scripts, browser logs, and failure records together, and added a review layer for high-risk conclusions. This raises the ceiling for autonomous execution of complex tasks, pushing the model from producing content toward verifying content. The cost is a longer action chain and higher resource use: 38.5 dollars total, close to an hour on the policy task, and multi-pass review cannot natively add independent evidence.

Claude Sonnet 5.5 showed another route with equal importance: packing stronger reasoning and production capability into a shorter, cheaper execution path. It assembled a complete report and a deliverable science demo quickly, and proactively stated boundaries such as missing originals and unverified browser behavior. Its 10.9-dollar total, about 28 percent of Sol's, lands in a cost range usable for daily work. The limitation is real: short paths sometimes stop at a usable prototype, and visual acceptance and peripheral sources still need human handling.

The two directions are different. Sol shows how far a model can verify itself. Sonnet 5.5 lowers the time and price threshold for high-quality agents to enter high-frequency tasks. In plain terms, Sonnet 5.5 fits as a high-performance daily workhorse, with better composite score, coding tasks, and generation speed. GPT 6.1 Sol fits as a depth agent for a single task, good at polishing details. In future work, different models can take different roles in the same workflow: a fast model explores and drafts, then a careful model checks the key facts and the execution path.

9. Competition Is Now About the Whole System

Looked at more deeply, AI competition is expanding from single-point model capability to whole-system capability. The model affects the reasoning ceiling. The tools and runtime decide which actions it can perform. The verification mechanism affects the trustworthiness of results. The stopping strategy affects time and the bill.

Across the three tasks, the speed and quality advantages kept reversing. No single benchmark can cover these variables. The real performance of an agent comes from the combined effect of the model, the tool chain, the task design, and the acceptance standard.

This is a structural point for anyone building on top of frontier models. The model is a component, not the whole system. The routing layer, the tool environment, the verification loop, and the stop conditions together determine what the user actually receives and what it costs. Engineering that layer is where a large share of production value now sits.

10. Conclusion

The headline number is the gap: identical token prices, yet GPT 6.1 Sol billed 38.5 dollars and Claude Sonnet 5.5 billed 10.9 dollars across the same three tasks, a difference of about 3.5 times. The cause was not model quality in a simple ranking sense. It was the action path each model took, and the resulting number of calls, context size, and repair rounds.

Time reversed three times. Sol won the escape room but took ten times longer on the policy task. Sonnet won the two research tasks by a wide margin but spent more than twice as long on the game. There is no clean winning line, and there is no fixed speed ranking. Wall-clock time depends on model strategy, tool availability, environment failures, and acceptance depth.

The engineering lesson is consistent. Token unit price is not task cost. Cost per completed task is the metric that matters, and it needs task-level observability. The model sets the capability ceiling, but the tool chain, the verification approach, and the stopping strategy decide what is delivered and what is paid. For teams running several models and providers, a gateway that centralizes routing, retries, and accounting is what turns a 3.5-times bill gap from a surprise into an attributable, controllable number.

AI is moving past the stage of giving a good answer. It is beginning to own the continuous process from understanding a problem to delivering a result. The models that win in production will be judged not by a single benchmark but by whether the whole system can deliver stable, verifiable, cost-predictable task success.