Introduction
During the 2026 Yunqi Conference, Alibaba Cloud officially confirmed that Qwen-4, built upon a brand-new native architecture, has entered formal training. The company also released a multi-generation evolution roadmap for the Qwen foundation model family. According to the disclosed plan, subsequent iterations including Qwen-4.5 and Qwen-5 will scale toward 5 trillion to 10 trillion total parameters. The core objective of this product sequence is to solve long-duration, multi-step complex tasks and advance technical capabilities toward Artificial Superintelligence (ASI).
The newly designed underlying architecture has completed validation experiments using Qwen3.8-Flash-Next as the prototype. This pre-verification phase proves the architecture can strike a balanced tradeoff between raw model performance and the total expense of training and inference. The optimized framework substantially cuts capital and operational costs associated with scaling up ultra-large foundation models. In parallel, the research team has deployed preliminary Recursive Self-Improvement (RSI) experimental workflows. The RSI mechanism enables the model to autonomously complete multiple rounds of training iterations, accumulating experience to provide technical foundations for future base model upgrades.
Alibaba’s flagship production model at the present stage is Qwen3.8-Max, which contains 2.4 trillion parameters. It is critical to clarify that the planned 10-trillion-parameter specification falls under a Mixture-of-Experts (MoE) design. For MoE models, the stated parameter quantity represents the aggregate total parameters across all expert modules, rather than parameters activated during every inference pass. Such ultra-large MoE architectures impose stringent requirements on computing clusters, AI chips, and high-quality training datasets. To match the expanding model scale, Alibaba is simultaneously advancing the development of its Hanguang AI chip series and building large-scale computing infrastructure.
The published roadmap marks a strategic turning point for domestic large model development in China. Local foundation model developers are no longer limited to catching up with capabilities of existing closed-source overseas models. Instead, they begin pushing toward higher theoretical intelligence ceilings. Even so, deploying ultra-large parameter models in real-world production environments still faces tangible engineering obstacles. Key challenges include high inference costs, runtime stability, value alignment and safety control. Qwen will continue its open-source strategy throughout this multi-year development cycle, balancing cutting-edge technical research with industrial implementation. This milestone signals that domestic foundation models have entered a new era combining massive parameter scaling and automated self-evolution, creating direct competition with leading global model vendors.
1. Technical Validation of The New Generation Architecture
Before launching full-scale training for Qwen-4, Alibaba’s research group carried out pre-verification work on the novel architecture with Qwen3.8-Flash-Next. This prototype stage serves as a low-risk sandbox, to test core design hypotheses without committing the full computing budget reserved for Qwen-4.
Traditional dense large language models encounter a series of bottlenecks when scaling beyond trillions of parameters. The most prominent limitation is the non-linear growth of training and inference costs. Dense models activate nearly all parameters during each token generation step, so every expansion of parameter count directly increases compute consumption. The MoE paradigm adopted by Qwen’s new architecture addresses this pain point by activating only a subset of expert parameters for each input token. This design separates total parameter capacity from real-time compute overhead, allowing models to hold far more knowledge while keeping inference resources manageable.
The Qwen3.8-Flash-Next validation experiments collected measurable data on memory bandwidth utilization, training convergence speed and per-token inference latency. The results demonstrate that the revised architecture retains strong reasoning performance while reducing the overall cost of scaling. This is the core prerequisite for moving forward to Qwen-4, Qwen-4.5 and eventually Qwen-5 at the 5–10 trillion parameter tier.
Another innovative module validated in this phase is Recursive Self-Improvement, abbreviated as RSI. RSI builds a closed-loop workflow for model self-refinement. After the model completes a task, it can analyze its own output, identify logical errors or factual gaps, generate improved training samples, and initiate further fine-tuning iterations without heavy manual intervention. In previous large model development cycles, human engineers spent massive effort on data curation, filtering bad samples and writing evaluation suites. RSI shifts part of this workload onto the model itself.
It is important to set realistic expectations for RSI at this stage. The current implementation remains experimental. The model cannot independently fix all fundamental flaws or bypass safety alignment constraints. What it can do is generate supplementary training data and iterate over specific task domains, accelerating the optimization cycle for long-chain reasoning and agent workflows. For long-period complex tasks such as multi-step scientific simulation, end-to-end software development and cross-document legal analysis, this self-improvement mechanism is expected to deliver the most obvious capability gains.
2. Parameter Scale: MoE Total Parameters versus Activated Compute
Qwen3.8-Max, Alibaba’s flagship model before Qwen-4, reaches 2.4 trillion total parameters under MoE architecture. The roadmap targets Qwen-5 at up to 10 trillion total parameters. Many industry observers may misinterpret this number as a dense model parameter count, which creates unrealistic assumptions about inference resource requirements.
MoE models split the neural network into multiple expert sub-networks. A gating module selects only a small fraction of experts for each incoming token. Total parameters refer to the sum of all experts stored on disk and in memory. Activated parameters are the subset actually calculated for a given request. This distinction is fundamental for cost planning. A 10-trillion-parameter MoE model may activate only several hundred billion parameters per token, depending on gating configuration. This is why MoE becomes the mainstream technical route for trillion-parameter frontier models.
Nevertheless, ultra-large MoE systems bring new engineering burdens. The gating network must distribute traffic evenly across experts to avoid load imbalance. Unbalanced expert activation leads to uneven GPU utilization and increased tail latency. The system also requires high-speed interconnect between computing nodes to route tokens and exchange hidden states across experts. These requirements impose strict standards for cluster networking and chip interconnection bandwidth.
This explains Alibaba’s parallel investment in Hanguang AI chips and large-scale computing infrastructure. The chip hardware, cluster networking, storage systems and model architecture must be co-designed. A mismatch between hardware capability and model design will lead to low utilization of expensive GPU resources and drastically increase the total cost of ownership. For frontier model projects at this scale, hardware and software development can no longer progress independently.
3. Target Workload: Long-duration Complex Tasks and ASI-Oriented Research
The primary capability target for Qwen-4 and subsequent releases is solving long-cycle complex tasks. Simple single-turn question answering or short text summarization is already well handled by existing large models. The next technical frontier lies in tasks that require sustained reasoning across hours or even days of continuous interaction.
Representative long-cycle workloads include full-stack software engineering agent tasks, multi-stage scientific research workflows, end-to-end engineering design, long legal contract review and multi-step mathematical theorem derivation. These tasks require the model to maintain consistent state over extremely long context windows, track intermediate results, backtrack when errors appear, and adjust planning strategies dynamically. Current models often suffer from state drift, logical degradation and accumulated mistakes when task duration extends.
Alibaba frames this series of model iterations as incremental steps toward Artificial Superintelligence. It is necessary to clarify that the roadmap does not promise immediate ASI arrival with Qwen-5. Instead, it treats the model sequence as a research platform to investigate core abilities required for advanced intelligence: persistent state management, self-critique, iterative plan revision, and recursive improvement.
This research orientation also reshapes the model evaluation methodology. Traditional benchmark tests with short prompts and single answers become insufficient. Evaluation must adopt agent-based benchmarks, where models complete multi-step missions and are scored by final task success rate rather than isolated question accuracy. The shift from static benchmark scoring to end-to-end task success evaluation is one of the core changes accompanying this generation of foundation model research.
4. Challenges Facing Ultra-Large Foundation Model Deployment
Despite promising architectural validation, ultra-large MoE foundation models still face several well-documented barriers for production deployment.
The first challenge remains inference cost. Even with MoE’s activated parameter optimization, serving trillion-parameter models consumes substantial GPU memory and compute. For high-volume business services, per-request expense can quickly exceed acceptable thresholds. Enterprises must adopt hybrid deployment strategies: route simple lightweight tasks to smaller, cheaper models, and reserve ultra-large foundation models only for high-value complex agent jobs.
The second issue is runtime stability. Long agent chains amplify small model defects. Minor hallucinations or logical mistakes in early steps propagate through the whole workflow and eventually cause task failure. Stability control requires layered evaluation checkpoints, self-verification logic and fallback mechanisms, all of which add system complexity.
Third, safety and value alignment grow harder at larger scales. More capable models exhibit stronger ability to find workarounds of alignment constraints. Researchers need continuous red-teaming, multi-layer safety guardrails, and automated monitoring pipelines to mitigate emerging risks. Alignment work cannot be completed once during training; it must be maintained and updated through every model iteration.
Data quality constitutes another bottleneck. Training trillion-parameter models demands enormous volumes of high-quality, diverse, low-contamination data. Low-quality or duplicated data leads to diminishing returns. Sourcing, cleaning, deduplication and annotating datasets at this scale represents a major operational project.
5. Open-Source Strategy and Market Competition Landscape
Alibaba confirms Qwen will preserve its open-source strategy throughout the Qwen-4, Qwen-4.5 and Qwen-5 development cycle. This is a key differentiator compared with many competing frontier models that remain fully closed-source.
The open-source route allows external developers, research institutions and enterprise teams to access model weights, carry out secondary development, and build domain-tuned variants. It accelerates ecosystem growth, attracts community contributors and expands real-world testing coverage. Thousands of teams can deploy, fine-tune and audit the model, which helps discover safety issues and capability limitations that internal testing might miss.
From an industry competition perspective, this roadmap shows Chinese foundation model developers moving from catch-up mode to frontier exploration. Previously, most domestic model releases focused on matching benchmark scores of GPT, Claude and Gemini. Now Alibaba sets sights on long-duration agent capability and recursive self-improvement, competing on new technical dimensions.
Global model vendors are also investing heavily in long-context agent optimization and self-training loops. The next phase of competition will not be defined only by parameter count or static benchmark numbers. Winners will be judged by total inference cost, reliability on long complex workflows, safety control, ecosystem maturity and enterprise deployment friendliness.
For engineering teams integrating multiple foundation models, unified traffic management becomes essential. 4sapi, an API gateway, simplifies cross-model request routing and workload governance for mixed LLM deployments.
6. Enterprise Architecture Considerations for Upcoming Qwen Generations
For enterprise teams planning to adopt future Qwen series models, several architectural choices deserve advance preparation.
First, implement workload classification and model routing. Simple tasks like text classification, short extraction and basic chat can run on smaller Qwen variants. Reserve Qwen-4 and later ultra-large models for complex agent planning, deep technical analysis and long document reasoning. This tiered approach controls inference spending without sacrificing capability where it matters.
Second, design checkpointing and state persistence for agent workflows. Long-cycle tasks need periodic state saving. When the model makes errors, the system can roll back to earlier checkpoints instead of restarting the whole task from scratch. This reduces token consumption and improves task completion rates.
Third, build custom evaluation suites based on business scenarios. Public general benchmarks cannot reflect performance on proprietary enterprise workflows. Teams should construct internal test sets to measure task success rate, hallucination frequency and end-to-end latency for their actual use cases.
Fourth, prepare observability and cost tracking. Ultra-large MoE models have variable token consumption depending on task complexity. Fine-grained monitoring can identify expensive request patterns, detect abnormal token usage and optimize routing rules.
7. Conclusion
Alibaba’s announcement at the 2026 Yunqi Conference marks a landmark milestone for the Qwen family. Qwen-4 has started training on a redesigned architecture, with a clear roadmap extending to Qwen-5 targeting 5 to 10 trillion total MoE parameters. The new architecture has been validated via Qwen3.8-Flash-Next, delivering better balance between model performance and compute cost, while early RSI experiments unlock semi-automated model iteration.
The 10-trillion-parameter target is built on MoE design, meaning only a subset of parameters activates per inference request. This design choice demands co-development of AI chips and large-scale computing clusters, which Alibaba is actively pursuing through Hanguang chip development and infrastructure buildout. The core capability objective focuses on long-cycle complex agent tasks, as a step toward ASI research.
The roadmap demonstrates that domestic foundation model research has entered a new era of self-evolution and large-scale model scaling. While major engineering hurdles around inference cost, stability and safety alignment remain, the continued open-source policy will foster broad community adoption. As competition shifts from benchmark chasing toward practical end-to-end task performance, multi-model API orchestration will become a standard component of enterprise AI stacks.
International access: https://4sapi.com
Domestic access: https://4sapi.cn
