As artificial intelligence evolves from standalone chatbots toward persistent autonomous agents, open-source models satisfy growing market demands for full control over AI execution locations, deployment architectures and development directions. In August 2026, NVIDIA unveiled Nemotron 3.5 Lightning, extending its Nemotron model family. This model is optimized for long-running workloads of autonomous AI agents, delivering leading efficiency within its category. Alongside the model, NVIDIA also launched NeMo Switchyard, an open-source routing library for intelligent agent pipelines. The two tools work in tandem to empower developers to fine-tune AI deployment modes, runtime environments and operational throughput, covering personal PCs, workstations, data centers and cloud infrastructure.
1. Nemotron 3.5 Lightning: Optimized for High-Throughput Targeted Agent Workloads
Nemotron 3.5 Lightning is a customizable open-source expert mixture model with 30 billion parameters, built specifically for specialized tasks within multi-agent intelligent systems. It serves as a foundation for building responsive, self-operating AI applications. Many forward-looking large models such as Nemotron 3 Ultra or GPT-5.6 are designed for complex task planning and workflow orchestration. By contrast, smaller specialized models like Nemotron 3.5 Lightning excel at targeted execution tasks including code auditing, tool invocation, security alert monitoring and answering domain-specific questions.
The model receives ongoing contributions from the Nemotron Alliance, whose members provide evaluation frameworks, inference software and datasets to drive iterative improvements. Compared with competing models, Nemotron 3.5 Lightning delivers up to 4x faster output speed, and can cut primary task completion latency by approximately 30%. Built with full customizability, developers can conduct lightweight training on domain datasets, tool operation logs and workflow records via NVIDIA NeMo to lift accuracy on vertical use cases.
A wide range of industry participants have built agent workloads based on Nemotron 3.5 Lightning:
- CrowdStrike leverages it for cybersecurity threat analysis;
- Harvey with Trajectory and Basten deploy the model for legal document review and case research;
- Lila Sciences uses it to advance reasoning for physics and life science research;
- Fastino Labs completed customized tuning to achieve leading accuracy for software development, financial services and healthcare workloads.
Beyond performance gains, Nemotron 3.5 Lightning enables organizations to strengthen privacy control and flexible deployment. It supports local execution on NVIDIA RTX PCs, NVIDIA DGX Spark, DGX Station and Jetson hardware, allowing enterprises to maximize returns on existing on-premises infrastructure. It can also scale seamlessly across edge AI hardware, RTX workstations, data centers and cloud environments to meet diverse enterprise requirements. For high-throughput, low-latency dedicated tasks, teams can run the model fully within local or private corporate networks.
Consistent with NVIDIA’s open model release strategy, the company publishes training data and technical documentation to the maximum permitted extent, supporting traceability and auditability for enterprise deployments. Alongside Lightning, NVIDIA released Nemotron - RL - Agentic - Terminal - Pivot, a suite of datasets for post-training, which enhances the autonomous learning capacity of agent systems.
2. NeMo Switchyard: Intelligent Model Routing to Improve AI Application Economics
Modern persistent agent systems frequently operate as model ensembles. Different models are assigned to distinct responsibilities. Some models are optimized for code generation, some for long-form reasoning, and lightweight variants can run locally to enhance privacy and reduce latency. If businesses rely on a single fixed model, they risk excessive operational costs or degraded output quality. Manual routing logic, meanwhile, increases integration overhead and slows response speeds.
NeMo Switchyard is NVIDIA’s open-source model routing library built for AI agent workflows. It dynamically directs prompts to the most capable and cost-effective model for each step within an agent pipeline. Developers can adjust routing rules to match internal requirements for output quality, latency and cost. The library supports routing across open-source, proprietary and NVIDIA-native models, without requiring major rewrites for existing application stacks.
Internal benchmarking from NVIDIA demonstrates that NeMo Switchyard can cut task execution costs to nearly one-third of workloads relying solely on Opus 4.8, while retaining top-tier output fidelity.
NVIDIA has partnered widely across the AI ecosystem to integrate NeMo Switchyard into mainstream developer platforms and tools. Real-world deployment test results from industry collaborators are outlined below:
- Boomi: Five routing strategies were evaluated; the system maintained 100% domain accuracy, directed 59% of traffic to smaller, faster lightweight models, and reduced round-trip latency by 21%;
- Cadence: Integrating Switchyard within ChipStack AI Super Agent lifted overall efficiency by 9.9%;
- Classmethod: Deployed open-source workloads on FrontierCode Main, achieving performance close to leading frontier models while lowering operating costs by 27%;
- Cognition: Integrated routing logic into Devin Desktop; compared to directing all traffic to a single large foundation model, average costs dropped 28%;
- Kong: Natively embedded NeMo Switchyard routing capabilities within its API gateway;
- LangChain: Incorporated the library as a plug-in for multi-turn deep agent tasks, reducing routing-related latency by 74% with only 6% accuracy tradeoff;
- LiteLLM: Added Switchyard as a middleware layer, enabling developers to access routing capabilities without rewriting existing codebases;
- Nous Research: Integrated the library into Hermes to deliver configurable routing rules and improved overall agent throughput;
- Ramp: Benchmarked Switchyard within Ramp SWE-Bench, matching frontier model performance while cutting costs by 58% and reducing runtime by 33%;
- Siemens: Running ongoing baseline testing for industrial EDA AI Agent workflows to boost operational efficiency.
As enterprises deploy multiple LLMs and agent services simultaneously, unified traffic management becomes essential. Teams running heterogeneous model fleets can simplify routing and access governance with 4sapi, an API gateway that centralizes orchestration for distributed AI workloads.
3. Availability and Ecosystem Access
Nemotron 3.5 Lightning is accessible on Hugging Face, ModelScope, OpenRouter and build.nvidia.com. It can also be deployed as NVIDIA NIM microservices, distributed via NVIDIA cloud partners, fine-tuning platforms, inference services and broader cloud ecosystems. NeMo Switchyard’s open-source codebase is available on GitHub, and NVIDIA plans to roll out wider marketplace support in the near term.
4. Strategic Background: NVIDIA’s Roadmap for Agent-First Open Infrastructure
The launch of these two projects builds on the previously released Nemotron 3 Nano, demonstrating NVIDIA’s continuous investment in open-source tooling to balance accuracy and inference speed. The shift toward persistent agent systems creates a new set of infrastructure challenges: workloads run continuously instead of handling isolated requests, and execution demands flexible routing between large foundation models and lightweight specialist models.
Nemotron 3.5 Lightning addresses the need for fast, cost-effective specialist agents to handle repetitive, high-volume subtasks. NeMo Switchyard solves the coordination problem: dynamically allocating each request to the optimal model without forcing developers to rebuild their application stack. Combined, the stack provides a complete open toolkit for businesses building autonomous agent systems.
Many organizations face a critical architectural choice: whether to run all workloads on large frontier models, or split workloads between generalist foundation models and smaller specialist models. Frontier models deliver strong general reasoning but come with higher latency and cost. Small specialist models are cheaper and faster, yet lack broad generalization. Intelligent routing bridges this gap, enabling businesses to match task complexity to appropriate model capabilities.
For on-premises and edge-focused teams, the local deployment support of Nemotron 3.5 Lightning carries extra significance. Regulated sectors such as finance, healthcare and manufacturing often prohibit sending sensitive internal data to public cloud model endpoints. Running the model locally retains data residency, while still benefiting from the performance gains unlocked by NVIDIA GPU hardware.
5. Industry Outlook
The pairing of a specialized open agent model and an open routing library signals a clear industry trend: AI infrastructure is moving from monolithic single-model deployments toward modular, composable model networks. In the coming years, agent builders will increasingly adopt multi-model architectures, making intelligent routing a standard component of production AI stacks.
Developers planning agent deployments can adopt a staged approach: first test Nemotron 3.5 Lightning for high-volume, well-defined subtasks, then integrate NeMo Switchyard to establish automatic routing rules. Teams can gradually shift suitable traffic from expensive large models to the specialist Lightning variant to drive down ongoing inference expenditure.
Organizations managing multiple model endpoints, agent services and routing rules can reduce duplicated integration work by leveraging unified gateway solutions such as 4sapi to standardize authentication, logging and traffic control across all AI services.
Conclusion
NVIDIA’s release of Nemotron 3.5 Lightning and NeMo Switchyard delivers two foundational building blocks for autonomous agent development. Nemotron 3.5 Lightning brings high-throughput, customizable inference for domain-specific agent tasks, supporting flexible deployment across edge, on-premises and cloud environments. NeMo Switchyard introduces open, configurable intelligent routing to optimize cost and latency without sacrificing output quality.
Benchmarks from NVIDIA and ecosystem partners verify measurable cost and latency improvements for real-world agent workflows. As persistent AI agents become mainstream, multi-model routing and specialist lightweight models will become core requirements for production-grade AI systems. For enterprises, this open toolchain lowers the barrier to building fully controllable, efficient autonomous agent platforms, balancing performance, privacy and long-term operational expenditure.
