NVIDIA Nemotron 3 Ultra: 550B Open-Weight MoE You Can Host

What builders need to know

• It is a real open release. NVIDIA published weights, training data and recipes for a 550B-parameter model under the Linux Foundation's permissive OpenMDW-1.1 licence — not a research-only or gated one.
• The architecture is the story. A hybrid Mamba-2 plus Transformer MoE, 550B total with just 55B active per token, giving high throughput at a 1M-token context window.
• Speed, not top-of-the-table intelligence. On the Artificial Analysis Intelligence Index it scores roughly 48 — the best US open-weight model, but Kimi K2.6 still leads open weights at about 54.
• You can host it, but it is a cluster. The full weights are around 1.1TB, so realistically a multi-GPU node with vLLM; the NVFP4 quantised checkpoint is the sane starting point.
• Dual-market relevance. In-region hosting on AWS Mumbai or AWS London, plus IndiaAI GPU-hour subsidies and UK sovereign-compute programmes, changes the maths for regulated data.

For the past year the open-weights conversation has been dominated by Chinese labs. Kimi, DeepSeek, GLM and MiniMax have taken turns at the top of the leaderboards while the strongest US models stayed behind API walls. Announced by Jensen Huang at Computex 2026 on 1 June and shipped with weights around 4 June, Nemotron 3 Ultra is NVIDIA's answer: a genuinely open, permissively-licensed US frontier model that you can download, inspect, fine-tune and run on your own hardware. That last part is what makes it interesting for anyone building in India or the UK, so let us look past the launch slides at what it costs to actually use.

What is actually new here

Nemotron 3 Ultra is a Mixture-of-Experts model: 550 billion total parameters, but only about 55 billion are active for any given token. That sparsity is what lets a model this large serve tokens quickly. The more unusual choice is the backbone. Instead of a pure Transformer, NVIDIA interleaves Mamba-2 state-space layers — which scale sub-quadratically with sequence length — with selective attention layers that preserve precise factual recall. The result, NVIDIA says, is a 1M-token context window at far lower memory cost than an attention-only model of the same size.

The throughput claims are the headline. NVIDIA reports up to roughly five to six times higher inference throughput than comparable open LLMs at on-par accuracy, and pre-release testing served the model at over 300 tokens per second — with some endpoints reporting north of 400. Those figures come from NVIDIA and its launch partners, and independent numbers will vary by hardware and batch settings. NVIDIA frames this as roughly 5x faster inference and around 30% lower cost for long-running agentic tasks. Treat the exact percentages as vendor claims until you see them on your own workload, but the direction is credible: sparsity plus Mamba is a genuinely cheaper way to run long contexts.

This slots alongside the broader NVIDIA open-model suite spanning Nemotron, Cosmos and Isaac GR00T, all of which moved to the same OpenMDW licence at once. The strategic read is obvious: NVIDIA sells the GPUs, so a fast open model that runs best on NVIDIA silicon is good for the razor-and-blades business. That is not a criticism — it is exactly why the release is so unusually generous with weights, data and recipes.

The licence — why OpenMDW-1.1 matters

For builders, the licence is often more load-bearing than the benchmark. Many "open" models over the past two years shipped under research-only or custom-community terms that quietly restricted commercial use, competitor use or fine-tuning. Nemotron 3 Ultra uses OpenMDW-1.1, a permissive Linux Foundation licence purpose-built for AI model artefacts — it covers weights, code, data and documentation together, grants royalty-free rights including commercial use, and carries a patent-termination clause.

In practical terms, that means an Indian SaaS startup or a UK consultancy can fine-tune the model on proprietary data and ship it in a commercial product without negotiating a bespoke agreement. It is materially more permissive than the licences attached to many earlier releases. It is not, however, byte-for-byte identical to MIT or Apache-2.0, so the honest advice is to have your legal function read the actual text — the patent clause in particular — before you build a business on it.

How it compares to the open-weights you already run

The useful question is not "is it the smartest model" — it is "where does it fit against the open-weights builders actually deploy". Nemotron's pitch is throughput and permissive licensing, not the top score. Here is how it sits against the models most teams are weighing up, drawing on Artificial Analysis Intelligence Index figures and published architecture specs.

Index figures are approximate and from Artificial Analysis; leaderboards move week to week and exact ranks vary by index version. Treat the table as a shape, not a verdict.

The read: if you want the highest raw intelligence in an open weight, the Chinese frontier — Kimi's coding line, Z.ai's GLM-5.2, DeepSeek V4 — still leads. If you want a fast, permissively-licensed model from a US vendor, with strong long-context throughput and a licence your compliance team is comfortable with, Nemotron is the new default. We covered this whole cohort in the round-up on the open-weight coding-models wave, and Nemotron is the piece that was missing: a US entrant that competes on economics rather than just benchmarks. For agentic and long-context coding specifically, it is worth benchmarking head-to-head against MiniMax M3's 1M-context open weight on your own harness before committing.

The self-hosting reality — footprint and economics

Here is where the enthusiasm meets the invoice. A 550B model in BF16 is roughly 1.1TB of weights. That does not fit on a single GPU, or even two — you need a multi-GPU node with tensor and pipeline parallelism. Realistically that means an 8x H100 or H200 80GB node, or a comparable cluster, to hold the weights in aggregate memory. The saving grace of MoE is that once loaded, only 55B parameters activate per token, so throughput is high; but every expert still has to live in memory somewhere.

NVIDIA ships an NVFP4 quantised checkpoint alongside the full-precision weights, and that is the pragmatic starting point for almost everyone. Four-bit quantisation roughly quarters the memory footprint, bringing the model within reach of a single well-specified node. Serving is standard: vLLM in production is the well-trodden path, and the mechanics of quantisation, batching and cost are covered in our guide to self-hosted serving economics.

A minimal starting point looks like this:

# Pull the NVFP4 quantised checkpoint from Hugging Face
huggingface-cli download nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 \
    --local-dir ./nemotron-3-ultra-nvfp4

# Serve with vLLM across 8 GPUs (tensor parallel)
vllm serve ./nemotron-3-ultra-nvfp4 \
    --tensor-parallel-size 8 \
    --max-model-len 1000000 \
    --quantization nvfp4

Now the economics. An 8x H100 node on AWS (a p5 instance) runs on the order of tens of dollars per hour on demand — call it several thousand pounds or a few lakh rupees per month if you keep it running. Against the hosted endpoints on OpenRouter or NVIDIA NIM — which meter per token — self-hosting only wins above a clear volume threshold. The rough rule: if you are not saturating a node with steady traffic, the managed endpoint is cheaper and you skip the ops burden entirely. If you are running high-volume long-context agents, own the hardware and the per-token cost collapses.

Dual-market angle — hosting it in India and the UK

Where you run this matters as much as whether you run it. For teams handling regulated or personal data, keeping inference in-region is often non-negotiable. In India, the Digital Personal Data Protection framework and sector rules push sensitive workloads towards local hosting; in the UK, GDPR and data-residency expectations do the same. Both AWS Mumbai (ap-south-1) and AWS London (eu-west-2) offer H100 and H200 capacity, so an in-region Nemotron deployment is feasible on either side.

The subsidy picture helps. India's IndiaAI mission has been standing up subsidised GPU-hour capacity aimed exactly at this kind of workload — a national open-weight model running on national compute is close to the stated policy goal. In the UK, the sovereign-compute and AI Research Resource programmes give startups and researchers routes to GPU time that do not depend solely on the hyperscalers. A permissively-licensed model is what makes these programmes usable: you cannot put a research-only model into a commercial product built on subsidised public compute, but you can put an OpenMDW-1.1 one there.

Adoption has been fast on the vendor side — NVIDIA and its partners report integration within roughly 48 hours by names including Perplexity, Nous Research, OpenCode and others across the coalition. That ecosystem momentum matters for builders: it means tooling, quantised checkpoints and serving recipes appear quickly, which lowers the cost of adopting the model yourself. If you are deciding between a fast-moving open weight and a niche one, the size of the surrounding ecosystem is a real, if unglamorous, tiebreaker.

If your ambitions are smaller than a 550B cluster — an on-device assistant, an edge deployment, a laptop-class agent — Nemotron 3 Ultra is the wrong tool, and our guides on picking and quantising a small on-device model are the better read. Match the model to the footprint you can actually afford to run.

So — should you adopt it?

For most builders in India and the UK, the answer is a qualified yes, with a staged approach. Prototype on the hosted endpoint. If the model clears your evals, and you have the volume and the compliance need, move to a quantised self-host in-region. If you want the absolute top intelligence in an open weight, keep Kimi K2.6 or GLM-5.2 in the mix — but if you want speed, a permissive US-origin licence and a 1M-token context window, Nemotron 3 Ultra is now the strongest open option that a self-hosting team can actually stand behind. The bigger point is structural: a genuinely open, high-throughput US frontier model changes the negotiating position of every builder who was previously choosing between expensive closed APIs and geopolitically awkward open weights. That optionality is worth more than any single benchmark row.