NVIDIA's Nemotron 3.5 Lightning: 30B total, 3B active, and the question of whether you can actually run it

What NVIDIA actually shipped

On 11 August 2026, NVIDIA released Nemotron 3.5 Lightning: a mixture-of-experts model with 30 billion total parameters and roughly three billion activated per token, a one-million-token context window, and a hybrid architecture interleaving Mamba-2 state-space layers with MoE layers and a smaller number of conventional attention layers. It is distilled from Nemotron 3 Ultra, the 550-billion-parameter open-weight flagship we covered in July, and aimed at high-volume agentic work rather than the top of a leaderboard.

The headline facts, before the marketing settles on top of them:

• Licence: OpenMDW-1.1, permissive, free for commercial use. You download the weights from Hugging Face or NVIDIA's build platform without requesting permission and without paying a licence fee.
• Openness beyond weights: NVIDIA also published training data and training recipes, subject to their own licensing constraints. Most releases labelled open-weight ship the weights and nothing else.
• Checkpoints: BF16 instruct and base variants, plus NVFP4 and NVFP4-DFlash quantised builds, with a community GGUF conversion from Unsloth for llama.cpp-class runtimes.
• Companion release: NeMo Switchyard, a model router, announced alongside it — which tells you exactly how NVIDIA expects Lightning to be used.

Everything else here is downstream of one architectural decision, and of the most misunderstood number in the release.

What 30B total and 3B active really means

A mixture-of-experts model splits its feed-forward capacity into many specialised sub-networks, and a small router picks a handful for each token. So a 30B/3B-active model holds thirty billion parameters and uses about three billion on any given token.

The trap is assuming "3B active" means "3B-sized deployment". It does not. Every expert must be resident where the GPU can reach it at low latency, because the router's choice changes token by token. In practice the whole model sits in memory.

Memory looks like 30B, compute looks like 3B

Work the arithmetic from the parameter count. At BF16 — two bytes per parameter — thirty billion parameters is roughly 60GB of weights, before context state, activations or framework overhead. That is an 80GB-class card, or a two-card split on 48GB hardware. At the NVFP4 precision NVIDIA published, four bits per parameter, the same weights come down to about a quarter of that, which is what makes a single 24GB to 48GB card plausible; the community GGUF build extends that logic further down. These are first-order estimates, not measurements: use them to pick which class of card to hire, then measure on the actual checkpoint.

The compute side is where you get paid. With about three billion parameters participating per token, the arithmetic per forward pass is a fraction of what a dense 30B demands. You are effectively renting 30B of memory to buy 3B of latency.

Why the Mamba-2 hybrid matters at a million tokens

The second architectural decision is the interleaving of Mamba-2 state-space layers with attention. Self-attention compares every token against every other token, so cost grows quadratically with sequence length — manageable at eight thousand tokens, punishing at a million. State-space layers instead carry a fixed-size recurrent state through the sequence, so cost scales closer to linearly and that state does not grow the way an attention cache does.

A hybrid keeps a minority of true attention layers, because attention is genuinely better at precise, arbitrary recall, while handing the bulk of the sequence work to layers that do not blow up as context grows. That is the mechanism behind a million-token window on a model this small — the same design lineage as Nemotron 3 Ultra, scaled down.

Being honest about the limits: there are no independent measurements of long-context recall, memory growth or latency curves at extreme lengths for this checkpoint, and anyone quoting precise figures at a million tokens without a methodology is guessing. The architecture gives you a direction of travel, not a number. Long inputs should degrade more gracefully here than on a pure-attention model of comparable size; whether they degrade gracefully enough for your documents is an evaluation you have to run yourself.

The genuinely notable part is the data, not the model

Strip away the benchmark table and the interesting thing here is the licence and what accompanies it. OpenMDW-1.1 is permissive: no revenue thresholds, no monthly-active-user cliffs, no commercial agreement waiting above a certain size. Set that beside the revenue-linked terms Alibaba attached to the Qwen3.8 flagship and the difference in diligence burden is stark.

The licence is only half of it. NVIDIA published training data and training recipes alongside the weights, subject to the constraints on that material — and open weights and open data are not the same commitment at all:

• Legitimate fine-tuning. Knowing the distribution the base model saw is the difference between deliberately extending a model and hoping your domain corpus does not fight it. Recipes give you the hyperparameters and curricula that worked, so the first run is not a shot in the dark.
• Provenance auditing. You can inspect what went in. For any organisation whose customers ask "what was this trained on?" — now most regulated buyers — the honest answer stops being a shrug.
• Regulatory documentation. The EU AI Act's transparency obligations for general-purpose models push towards documented training-data summaries. Published data and recipes turn that into an exercise in citation rather than reconstruction, and UK teams meet the same question through their EU customers.
• No vendor lock-in. Permissive weights you host yourself cannot be deprecated out from under you, repriced mid-contract, or rate-limited during someone else's traffic spike.

None of this makes Lightning a better model. It makes it a more defensible one, and for a certain kind of buyer that is worth more than a few benchmark points.

The benchmarks, and what they do not say

Every figure below is vendor-reported or third-party-reported at launch. We have not independently verified any of them, and you should not treat them as settled until someone outside the release cycle reproduces them.

Benchmark	Reported result	How to read it
SWE-bench Verified	51.56	Competent, not frontier-class. The most important number here.
GPQA Diamond	75.44	Strong for the parameter class
MMLU Pro	81.94	Broad knowledge holds up well
PinchBench	85.37	Some coverage rounds this to 86%; it is the same result
AA-Omniscience (non-hallucination)	69.9% vs 50.3% next-best-in-class	The most commercially interesting claim, if it survives scrutiny
Throughput	~1,200 tokens/sec at p50	Artificial Analysis measurement; serving stack and hardware dependent

Three things worth saying plainly about that table.

First, 51.56 on SWE-bench Verified is not frontier-class, and nobody should pretend otherwise. This is an efficiency-tier model. It will not replace whatever you currently point at hard multi-file refactors. If your evaluation of Lightning starts from "can it do what our flagship does", the answer is no and the evaluation was framed wrongly.

Second, the throughput comparison doing the rounds — one outlet frames the roughly 1,200 tokens/sec p50 figure as about twenty-nine times Gemma 4 26B's 41.45 tokens/sec — is real arithmetic on published numbers that still needs care. Cross-vendor throughput depends heavily on serving stack, batching, quantisation and hardware. A 3B-active MoE outpacing a dense 26B is architecturally unsurprising; the multiple belongs to the measurement setup as much as to the models.

Third, NVIDIA's claim that Lightning completed ten thousand tasks roughly 30% faster than Qwen3.6-35B at similar accuracy is the shape of claim buyers should care about — throughput at constant quality — but it is a vendor claim on a vendor-chosen task set. Artificial Analysis characterised the model as performing similarly to gpt-oss-120b at around a quarter of the total parameters. That is the value proposition: parameter efficiency, not capability.

Where Lightning fits — and where it does not

Lightning is built for the long tail of your traffic: the high-volume, individually-cheap, collectively-expensive work a frontier model handles beautifully and wastefully. Here is how the common workloads sort out.

Workload	Fit	Reasoning
Document classification and routing at volume	Strong	Exactly the profile the active-parameter economics reward
Structured extraction from long documents	Strong	1M context plus hybrid scaling suits large inputs
Agentic tool-calling in a constrained schema	Good, with guardrails	Distilled for agentic work; still needs retry and repair logic
Summarising long transcripts or case files	Good	Reported non-hallucination figures are encouraging if they hold
Multi-file code refactors and production PRs	Poor	51.56 on SWE-bench Verified is not where you want this
Hard multi-step reasoning and novel research	Poor	3B active parameters is a real ceiling on depth
On-device or laptop-class deployment	Conditional	Needs the GGUF or NVFP4 build and honest expectations

The agentic row deserves an asterisk. Smaller models fail at tool-calling in characteristic, fixable ways — malformed arguments, invented parameters, silent schema drift — and the fix is engineering, not model choice. Our guide to making small models reliable at tool-calling covers the retry, repair and verify loop that turns a 90%-correct model into a 99%-correct pipeline. Budget for it before assuming the saving is free.

Measure Lightning against the season's other 30B-class open release too: Meta's Muse Glimmer put a 30B agent model on laptops under Apache 2.0, and the two occupy adjacent ground with different trade-offs. Run both on the same evaluation set rather than trusting either vendor's framing.

Getting a first load running

The repositories are public, and the instruct checkpoint is the one to evaluate. A first serve looks roughly like this — treat the flags as a starting point and reconcile them against the model card.

# Recent vLLM — hybrid Mamba/attention support is version-sensitive
pip install --upgrade "vllm"

# Serve the BF16 instruct checkpoint
vllm serve nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 \
  --served-model-name nemotron-lightning \
  --max-model-len 131072 \
  --tensor-parallel-size 2

# Call it like any OpenAI-compatible endpoint
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "nemotron-lightning",
       "messages": [{"role": "user",
                     "content": "Extract every invoice number as JSON."}]}'

Set a context length you actually need rather than the full million; allocation scales with what you ask for. If the BF16 build will not fit your card, move to nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4, or the Unsloth GGUF conversion for llama.cpp-class runtimes. A base checkpoint is published too, for tuning from the base rather than the instruct model. For turning a working serve into an economical one, our vLLM throughput and latency playbook covers the levers in order of payoff.

The economics for teams in Bengaluru and Manchester

Here is where the dual-market reality bites, and it bites differently on each side.

For teams in Bengaluru, Chennai or Hyderabad, the IndiaAI Mission's subsidised GPU access has changed the arithmetic of self-hosting. Where empanelled capacity is available at subsidised rates, a 30B-class model that fits on one or two cards becomes an operating question rather than a capital one, and the per-token economics are hard for an API vendor to match at volume. Check current terms through the official channels — the schemes evolve.

For teams in London, Manchester or Edinburgh, the calculation is usually data residency and procurement rather than price. A permissively licensed model inside your own perimeter is the shortest conversation you will have with a data-governance review: no cross-border transfer, no vendor terms to assess, no processor agreement. Add published training data and the customer security questionnaire gets shorter still.

What unites both markets is the hardware reality. Reported lead times for B200 and H200 class accelerators have been running at 36 to 52 weeks, so most teams will rent rather than buy. That is not a defeat: renting lets you re-evaluate the model landscape every quarter instead of every three years. If you are assembling capacity from whatever accelerators you can actually get hold of, our guide to running inference across mixed-vendor GPU fleets is the relevant reading.

Which is precisely why NeMo Switchyard shipped alongside Lightning. A router that sends easy traffic to a cheap model and hard traffic to an expensive one is the architecture this release assumes. NVIDIA is not selling a frontier replacement; it is selling the cheap half of a two-tier stack, plus the routing layer.

The bottom line

Nemotron 3.5 Lightning is not the most capable model released this month and was never trying to be. It is a well-engineered efficiency-tier model that does three things at once: activates a tenth of its parameters per token, reads very long inputs without the usual quadratic penalty, and arrives with a permissive licence, its training data and its recipes.

Judge it on the workload you would actually give it. High-volume classification, extraction, routing or constrained tool-calling: a serious candidate. Hard reasoning or production code generation: keep your flagship. And if your reason for looking is regulatory rather than economic — a customer asking what the model was trained on, an auditor asking for provenance — the open data and open recipes are the release's real contribution, worth more than any line in the benchmark table.

Sources: NVIDIA's announcement and the model cards on Hugging Face and build.nvidia.com; launch coverage from MarkTechPost (11 August) and DataCamp; throughput from Artificial Analysis. All benchmark figures are vendor- or third-party-reported and unverified by AI Tech Connect. More in our open-source coverage.