Nvidia on Tuesday unveiled Nemotron 3.5 Lightning, a 30-billion-parameter model designed to run up to four times faster than previous versions, alongside NeMo Switchyard, an open-source routing library that can slash AI task costs by roughly two-thirds. The releases represent Nvidia's push to make smaller, customizable models the workhorses of enterprise AI systems, with frontier models reserved for planning while Lightning handles execution. The company also released a full dataset for training coding agents through agentic reinforcement learning.
Nemotron 3.5 Lightning is a mixture-of-experts model developed with contributions from the Nemotron coalition, offering reasoning capabilities that approach those of the much larger Nemotron 3 Super model, according to Nvidia. Both Nvidia models trail Google's similarly sized Gemma 4 31B on Artificial Analysis' Intelligence Index. In internal benchmarks, a Switchyard-routed system mixing several open models with Anthropic's Opus 4.8 maintained frontier-level accuracy while reducing task-completion costs to approximately one-third the expense of running Opus alone. Early partner results showed comparable savings: LangChain achieved 74% lower costs across 145 multi-turn Deep Agents tasks by directing only 7% of calls to a frontier model, though with a 6% accuracy tradeoff, while Ramp matched frontier model performance on its internal SWE-Bench while cutting costs by 58% and runtime by 33%.
"General agentic benchmarks are just the starting point, but in production, what matters is accuracy on your task — and that's where post-training makes the biggest difference," Nvidia's Kari Briski said during a press briefing. Working with partners including CrowdStrike and CodeRabbit, Nvidia found that a fine-tuned open model like 3.5 Lightning could perform as well as, and sometimes better than, larger proprietary models when handling specialized tasks it was post-trained on. The company argues that speed matters less than the ability to modify and optimize the model for specific workflows, which Nvidia hopes will differentiate Lightning from competitors.
The strategic bet centers on what Nvidia Senior Director Joey Conway describes as systems of models becoming the future of AI. At the core of these systems sits a router that determines which model handles which task, according to the company. The routing mechanics are straightforward, but the decision-making is complicated. NeMo Switchyard, written in Rust but accessed through APIs, allows developers to define their own pool of models and set routing criteria and policies, tuning algorithms for quality, latency, and cost depending on what each workflow requires. Nvidia argues its NeMo ecosystem provides all the tools needed to simplify post-training work, even as enterprises grow increasingly cost-conscious about running frontier models. If agents do evolve into ensembles of multiple models, the router becomes the layer where cost and quality decisions actually happen, and Nvidia clearly hopes its open, easily customized models will be the ones those routers select for high-volume work.
Nemotron 3.5 Lightning is now available on Hugging Face, ModelScope, OpenRouter, and build.nvidia.com as an Nvidia NIM microservice, while NeMo Switchyard is available on GitHub with additional partner integrations planned. Existing routers and AI gateways including Kong, OpenRouter, and LiteLLM are already building Switchyard into their offerings. The router layer may ultimately determine whether Nvidia's smaller models can capture the execution tasks that make up the bulk of enterprise AI workloads, or whether the economics of fine-tuning still favor keeping everything with a single frontier provider.