International access: https://4sapi.com
Domestic access: https://4sapi.org
Sequential autoregressive generation has long been the dominant paradigm for large language models. From GPT series models to Claude variants, token-by-token prediction defines how most mainstream LLMs produce text. This design delivers reliable reasoning quality, yet it creates a hard ceiling on generation throughput. Diffusion-based language models emerged years ago as an alternative path, celebrated primarily for raw speed. The newly released Mercury 3 from Inception Labs marks a critical turning point for this technical route. It proves diffusion LLMs can scale dramatically, retain their speed advantage, and match or beat autoregressive competitors on complex reasoning and software engineering benchmarks.
Announced on October 19 by the lab founded by Stanford professor Stefano Ermon, Mercury 3 has released benchmark results that challenge assumptions around model scaling and generation speed. Its peak output throughput reaches 1068 tokens per second on NVIDIA hardware. In direct comparison, GPT-6 Luna and Claude Haiku 5.5 hit roughly 130 tokens/s and 240 tokens/s respectively. This means Mercury 3 can generate text at nearly 8 times the speed of GPT-6 Luna in optimal conditions. The most striking detail is how this performance is achieved: its total parameter count expands more than 40× relative to the original Mercury release. Earlier diffusion models were often dismissed as fast but shallow, suitable only for simple chat responses. Mercury 3 breaks that stereotype by combining massive scaling, strong reasoning capabilities, and low per-token pricing. During its launch window, input tokens cost $0.10 per million and output tokens cost $0.35 per million; after the introductory phase, regular pricing shifts to $0.20 per million input tokens and $0.70 per million output tokens. Its context window also expands substantially, growing from 260,000 tokens on Mercury 2.5 up to 1,000,000 tokens for Mercury 3.
Benchmark data released by Inception Labs demonstrates that the model’s speed gains do not come at the cost of reasoning performance. On GPQA Diamond, a benchmark built to test graduate-level scientific knowledge and multi-step inference, Mercury 3 scores 89.23. By comparison, GPT-6 Luna scores 86.36 and Claude Haiku 5.5 scores 84.51. The performance gap becomes even wider on practical engineering tasks. On SWE-Bench Atlas, a benchmark focused on real software development challenges, Mercury 3 achieves 51.08 points. The competing autoregressive models only reach 33.33 and 32.26 respectively. It also posts a score of 76.84 on IFBench, which evaluates instruction-following fidelity. These results carry important meaning. Inception Labs is no longer presenting a niche speed demo. Mercury 3 shows that diffusion language architectures can handle knowledge QA, long, complex instructions, and production-grade software workflows, rather than only generating short, simple text quickly.
To understand why this result matters, it helps to contrast the core generation mechanics between autoregressive LLMs and diffusion LLMs. Autoregressive models such as GPT and Claude generate text sequentially. They predict one single next token at each forward pass. Even with highly optimized GPU parallelism, this serial dependency cannot be eliminated. Every new token must wait for the tokens before it to be finalized. Diffusion language models use a fundamentally different process. They populate many positions in the output sequence at once and iteratively refine the full text across multiple rounds. A useful analogy compares two writing workflows. The autoregressive approach writes sentence by sentence, finishing one section before moving onward. The diffusion approach drafts an entire block of content simultaneously, then revises many positions together in subsequent passes. Each forward pass updates a batch of tokens in parallel, cutting down the total number of sequential steps required. Even with multiple refinement iterations, the aggregate generation latency can undercut sequential token prediction.
This parallel-first design introduces a difficult engineering tradeoff. Image diffusion works by gradually removing noise from pixel grids. Text, however, consists of discrete tokens. The model must decide which positions are stable enough to lock in and which need further revision. Larger parallel batches accelerate generation, but insufficient refinement cycles can break syntax, logical reasoning, and code structure. The central technical challenge for diffusion LLMs is balancing parallel update breadth against final output quality. The original Mercury series began exploring this transformer-based parallel prediction pipeline in its initial technical reports. Mercury Coder Mini, released in 2025, already hit a measured speed of 1109 tokens per second in third-party tests. At that stage, speed remained its defining selling point, while complex reasoning was not its primary strength. Inception Labs rolled out a long sequence of iterative upgrades across the Mercury family: expanded training datasets, refined denoising network structures, revised training objectives, optimized inference algorithms, low-level kernel tuning, and overhauled serving infrastructure.
Mercury 3 represents the payoff of this multi-year roadmap. Its aggregate capability on standard benchmarks is approximately 3× higher than Mercury 1 Mini, while generation throughput stays firmly in the thousand-token-per-second range despite the 40× scale expansion. This validates a core hypothesis for the diffusion LLM field: scaling model parameters and training compute does not automatically erase the parallel generation advantage. For years, critics argued diffusion LLMs would lose their speed edge once pushed to the complexity frontier. Mercury 3 provides concrete empirical evidence against that argument.
The strongest use case for this class of model emerges within AI agent systems. In agent workflows, raw peak generation speed is only one part of the equation. Latency compounds across repeated model calls. A web-navigation agent, for example, must open pages, scan content, judge relevance, follow links, extract structured fields, and backtrack when information is incomplete. Every observation-action cycle triggers a fresh LLM call. After dozens of sequential rounds, small delays on each invocation stack up into long end-to-end waiting times for end users. Mercury 3 has been validated in two production agent deployments that highlight this advantage.
TinyFish, a developer of browser agent tooling, integrated Mercury 3 into its live web-agent pipeline. The team ran tests on the Mind2Web benchmark set covering 50 real-world websites. Mercury 3 completed 25 out of the 50 tasks, for a 50% success rate. That rate exceeds the result from GPT-6.1 Sol tested under identical conditions. The total token volume and action steps required were comparable, but Mercury 3’s per-token operating cost was substantially lower. For browser agents that run frequent, repeated observation-action loops, lower cost combined with faster response creates an economically viable foundation for high-frequency automation.
Octen AI, which builds retrieval tools for smart eyewear hardware, shared another production measurement. In its Sense API pipeline, retrieval overhead only consumed roughly 80 milliseconds; model generation was the dominant bottleneck. After switching to Mercury 3, the median generation latency dropped to 0.9 seconds, with full end-to-end response latency landing at 2.2 seconds. For wearable voice interfaces, this is a meaningful user-experience threshold. A five-second wait feels like submitting a background job. A two-second response feels like natural conversational interaction.
Tri Dao, the original author of FlashAttention, commented publicly on this release. He articulated a split-agent architecture pattern: a high-capacity “planner” or “orchestrator” model defines goals and breaks down hard problems, while a fast, cost-effective “executor” model handles repeated retrieval, page parsing, field extraction, and routine tool calls. Diffusion LLMs like Mercury 3 fit naturally into this executor role. This framework reshapes how teams evaluate model selection. Developers no longer need to assign their most expensive top-tier model to every single step in an agent loop. Task decomposition creates a division of labor: one model owns strategic planning, and another handles high-volume, iterative operational work. This separation cuts overall inference expense while keeping task completion speed high. When an agent runs 20 or 30 consecutive tool calls, small latency savings on each call compound to drastically reduce total wall-clock time.
API gateway infrastructure simplifies the process of swapping models like Mercury 3 into existing agent stacks without rewriting core workflow logic. 4sapi offers unified routing for heterogeneous LLM endpoints, letting developers route planning and execution tasks to separate model backends while maintaining consistent request schemas and observability. This kind of abstraction becomes especially valuable for teams experimenting with mixed-model agent designs.
The release of Mercury 3 does not mark the end of autoregressive LLMs. Sequential generation still holds advantages for certain long-form reasoning and formal mathematical proofs. What it establishes is a new competitive axis for the LLM ecosystem. Diffusion models are no longer limited to speed-only niche use cases. They can scale, improve reasoning, and retain parallel throughput gains. Future agent systems will increasingly be built as heterogeneous ensembles, matching model architecture to subtask requirements rather than relying on a single universal model for all steps. As benchmark results and production deployment data continue to accumulate, diffusion LLMs will become a standard option for latency-sensitive, cost-constrained agent workloads.
