DeepSeek-V4.1-Flash: KV Cache Compression Explained

Introduction

DeepSeek has unveiled DeepSeek-V4.1-Flash, a multimodal Mixture-of-Experts (MoE) foundation model with 552 billion main parameters and support for a 1 million-token maximum context window. Built upon the Causal Encoder-Decoder (CED) architecture, the model stack contains a 20-layer causal encoder followed by a 20-layer decoder, forming a 40-layer Transformer structure. The core innovation of the CED design lies in the global KV cache for the decoder: it is projected from the final hidden state of the encoder, instead of being generated independently at each decoder layer. This architectural change drastically reduces the memory footprint of long-sequence inference.

The model incorporates a novel SWA (Sliding Window Bounded Replay) mechanism. Under SWA, the model only reconstructs the SWA KV cache from the most recent n_win tokens, avoiding the requirement to offload the full KV cache to SSD. This optimization cuts the KV cache memory consumption down to roughly 1/8 of the original DeepSeek-V4-Flash implementation.

For further compression, DeepSeek-V4.1-Flash leverages Compressed Sparse Reuse (CSR2) alongside a Hierarchical Reindex Indexer. CSR2 assigns three distinct static modes to each attention head: Full, Reuse and Sparse. This limits heavy full-mode key-value retrieval to the first pass only, and subsequent passes rely on the hierarchical sparse indexer, which reduces the computational overhead of deep-layer sparse retrieval. With this design, every 16 attention channels share one E4M3 scaling factor. The resulting global KV cache memory footprint per token drops to 890 bytes, approximately one-fourteenth of DeepSeek-V4-Flash (E2M1 format).

Additional architectural components include Single-Pass mHC, optimized Mega-mHC residual mixing, the 1960-parameter Engram memory record, DSpark speculative decoding and semi-autoregressive sampling with adaptive scheduling validation. This MoE model uses 1 shared expert and 384 routed experts, activating only 6 routed experts for every token.

The multimodal vision encoder, DeepSeek-VIT, is trained from scratch. It adopts 2D-RoPE and 3×3 pixel mixed downsampling, paired with a two-layer MLP projector to convert image input into embeddings, which are fused with text embeddings from the very beginning of the pre-training phase.

Pre-training and Post-training Pipeline

The pre-training phase of DeepSeek-V4.1-Flash uses a multimodal corpus of 45 trillion tokens. Training is performed with sparse attention at 64K sequence length, scaling to a total of 34 trillion tokens after context extension, reaching the 1 million-token limit.

The post-training workflow follows standard SFT → RL → Online Preference Distillation (OPD) paradigms. No fundamental algorithm changes were introduced; most adjustments are applied to the data pipeline. The team constructs a large-scale synthetic intelligent agent task environment and progressively expands datasets, tasks and trajectory samples. The model supports continuous and controllable reasoning strength, configurable as an integer from 1 to 100.

Benchmark Performance on Agent Evaluation Suites

The model was benchmarked against competing models including Kimi-K3, GLM-5.3, Opus5 and GPT5.6-Sol across four key agent benchmarks: Terminal-Bench 3.0, DeepSWE v1.1, CyberGym and Automation-Bench.

• Terminal-Bench 3.0: DeepSeek-V4.1-Flash scores 30.0, while Kimi-K3 reaches 17.7 and GLM-5.3 scores 28.3
• DeepSWE v1.1: DeepSeek-V4.1-Flash scores 74.2, outperforming Kimi-K3 (67.5), GLM-5.3 (66.9), Opus5 (74.0), GPT5.6-Sol (73.0)
• CyberGym: DeepSeek-V4.1-Flash achieves 88.1, exceeding Kimi-K3 (80.0), GLM-5.3 (84.5), Opus5 (84.5), GPT5.6-Sol (84.5)
• Automation-Bench: DeepSeek-V4.1-Flash scores 54.8, versus Kimi-K3 (46.7), GLM-5.3 (48.8), Opus5 (50.3), GPT5.6-Sol (45.8)

KV Cache Memory Reduction Trend

One of the most tangible engineering gains is the continuous reduction of global KV cache per token across DeepSeek’s model family:

1. DeepSeek-V1 (Nov 2023): 389,120 bytes per token
2. DeepSeek-V3.2 (Dec 2025): 48,068 bytes per token (8.1× smaller than V1)
3. DeepSeek-V4-Flash (Apr 2026): 3,514 bytes per token (13.7× smaller than V3.2)
4. DeepSeek-V4.1-Flash (Sep 2026): 890 bytes per token (3.9× smaller than V4-Flash, 437× smaller than DeepSeek-V1)

This steady compression curve is critical for long-context agent workloads. Large KV cache sizes are the primary bottleneck for serving million-token windows, requiring expensive high-bandwidth GPU memory. The 890-byte per-token KV cache greatly lowers memory requirements, making long-sequence inference more affordable for production deployments.

Comprehensive Benchmark Results

All base models were evaluated within DeepSeek’s internal framework under identical settings. Scores within 0.3 points are treated as equivalent performance. The table below summarizes core benchmarks for DeepSeek-V4.1-Flash-Base, compared against DeepSeek-V4-MoE and DeepSeek-V4-Pro-Base.

General Knowledge, Reasoning & Code Benchmarks

Long-context evaluation on LongBench-V2 (EM, 1-shot) yields 45.2 for DeepSeek-V4.1-Flash-Base. For multimodal benchmarks, the model scores 77.9 on MMMU-Pro, 77.5 on CV-Bench, 95.6 on DocVQA and 86.0 on RefCOCO-avg.

Agent and Tool-Use Benchmarks (Max Reasoning Strength)

DeepSeek-V4.1-Flash was tested alongside Opus-5.0, GPT-5.6 Sol, K3 and GLM-5.3 across specialized agent benchmarks. Key results are shown below:
| Benchmark | DeepSeek-V4.1-Flash | Opus-5.0 | GPT-5.6 Sol |
| ---- | ---- | ---- | ---- |
| GPQA Diamond (Pass@1) | 90.9 | 93.4 | 94.1 |
| HLE Diamond (Pass@1) | 39.1 | 56.3 | 44.5 |
| CyberGym (Pass@1) | 88.1 | — | 84.5 |
| DeepSWE v1.1 | 74.2 | — | 73.0 |
| Terminal-Bench 3.0 (Pass@1) | 30.0 | 43.3 | 34.4 |
| Automation-Bench (Pass@1) | 54.8 | 50.3 | 45.8 |
| Agent:Catastrophy Exam | 31.8 | 28.6 | 26.7 |
| Babyvision (Pass@1) | 89.6 | 94.1 | 78.9 |

These results demonstrate that DeepSeek-V4.1-Flash delivers competitive or superior performance on agent and coding tasks, while retaining drastically reduced memory overhead from its compressed KV cache design.

Recommended Inference Parameters

The official recommended generation configuration for DeepSeek-V4.1-Flash:

• temperature: 1.0
• top_p: 0.95 or 1.0
• context_window: 1M tokens
• max_tokens: ≥ 256K

The model supports adjustable reasoning strength, ranging from 1 to 100. All agent benchmark tests use the maximum reasoning setting, with temperature=1.0, top_p=0.95, context length capped at 1M tokens, and a maximum of 500 steps per agent task. The benchmark environment excludes network access, and each test sample runs 3 times for sampling variance reduction.

Deployment and Engineering Considerations

The release package includes Jinja chat templates, an encoding.py Python reference implementation for multi-turn conversation, and test cases for tool invocation, chain-of-thought, numeric reasoning, intermediate system prompts and multimodal image inputs. A Rust SDK and Python recipe package are also provided for production integration.

For production workloads, developers can use the provided message conversion utilities, which translate raw request payloads into the Conversation format accepted by DeepSeek endpoints. The model exposes separate API entry points for text, image, tool calls and agent workflows.

Teams running multiple model variants in production often use centralized routing and authentication infrastructure. When deploying long-context MoE models such as DeepSeek-V4.1-Flash together with other LLM services, an API gateway can streamline endpoint management and request observability. 4sapi serves as an API gateway to consolidate multi-model traffic, simplifying quota tracking and access control across mixed model deployments.

Implications of KV Cache Compression for Long Context

KV cache is one of the dominant memory costs for transformer inference. For million-token context windows, every token’s stored key and value tensors consume GPU VRAM. Traditional systems either allocate massive high-end GPUs or offload cache to system SSD, which introduces high latency. DeepSeek’s stacked innovations, including CED architecture, SWA bounded replay and CSR2 hierarchical sparse indexing, cut memory consumption without sacrificing core reasoning and agent performance.

This unlocks new practical use cases. Long code repositories, full project documentation, multi-hour transcript analysis and extended agent session history can all fit within available GPU memory. Agent workflows that require continuous state tracking over hundreds of thousands of tokens become economically feasible. Previously, such workloads were limited to small-scale research demos or costly high-end clusters.

The tradeoff worth noting is that sparse KV compression adds small computational overhead during index lookup. However, this overhead is far outweighed by the reduction in memory bandwidth pressure and GPU memory requirements. Benchmark results confirm that agent, coding and reasoning performance remains strong after compression.

Limitations and Practical Guidance

While DeepSeek-V4.1-Flash achieves excellent results on agent benchmarks, it still exhibits weaknesses on certain high-difficulty pure reasoning benchmarks such as HLE Diamond. Developers should select the reasoning strength parameter carefully. Higher reasoning strength improves complex problem-solving, but increases inference latency and token consumption.

For multimodal scenarios, the model’s vision capability is solid, but complex image reasoning must be validated with your own dataset. The MoE architecture activates only a small subset of experts for each token, which helps control compute cost, but expert load balancing should be monitored during high-throughput production serving.

When building agent systems, developers should still follow standard practices: validate tool call outputs, add retry logic for failed actions, and guard against prompt injection. The model’s strong benchmark scores do not eliminate the need for application-layer safety checks.

Conclusion

DeepSeek-V4.1-Flash represents a major milestone in long-context model optimization. Its CED architecture, SWA bounded replay and CSR2 sparse KV cache compression reduce per-token KV memory down to 890 bytes, which is 437× smaller than the original DeepSeek-V1. At the same time, the model maintains strong benchmark performance across coding, agent automation, tool use and multimodal tasks, with support for a 1 million-token context window.

This work proves that aggressive KV cache compression can drastically lower the cost of long-sequence inference, without sacrificing the model’s ability to operate complex agent workflows. For engineering teams building AI agents, code assistants and long-document analysis pipelines, DeepSeek-V4.1-Flash offers a powerful option that balances performance, context length and memory cost.

As more long-context MoE models enter production, unified traffic management will become more important for teams running heterogeneous model stacks. API gateway solutions help manage authentication, logging and routing across different model endpoints.

International access: https://4sapi.com
Domestic access: https://4sapi.cn