NVIDIA Nemotron 3 Ultra Review: 550B Parameters, Open-Weight, and Faster Than Any Chinese API

NVIDIA Nemotron 3 Ultra is the highest-scoring US-built open-weight model ever released, scoring 48 on the Artificial Analysis Intelligence Index when it shipped on Hugging Face on June 4, 2026. It runs at over 300 tokens per second — three to six times faster than comparable Chinese models available through commercial APIs — and its 550-billion-parameter architecture keeps only 55 billion active per token, making it far more efficient to run than a dense model of the same scale.

Key takeaways

• Nemotron 3 Ultra has 550B total parameters with 55B active per token via a Mixture-of-Experts architecture.
• It scores 48 on the Artificial Analysis Intelligence Index — the highest ever for a US open-weight model.
• Speed exceeds 300 tokens per second, 3 to 6x faster than Chinese model APIs at similar capability levels.
• Supports a 1M token context window and outperforms all open-weight competitors on RULER at that length.
• Available on Hugging Face and AWS SageMaker JumpStart with one-click deployment.

What makes this release significant

For the past year, the open-weight AI conversation has been dominated by Chinese labs — DeepSeek, Kimi, Qwen — releasing models that matched or beat US proprietary products at a fraction of the cost to run. Nemotron 3 Ultra is NVIDIA's clearest statement that a US-built open model can compete seriously, not just on intelligence scores but on inference economics.

NVIDIA unveiled it at Computex in Taipei on June 1, 2026, and Hugging Face access went live June 4. The model is part of the Nemotron 3 family that launched at GTC in March 2026 with Nano and Super variants, but Ultra is in a different tier entirely — both in capability and intended use case.

Architecture: why MoE matters here

The 550B parameter headline number is attention-grabbing but slightly misleading on its own. What matters for deployment economics is that only 55B parameters activate per token inference. That is the Mixture-of-Experts architecture at work — the model routes each token through a subset of its total parameter space rather than running everything simultaneously.

NVIDIA also baked in hybrid Mamba-Transformer layers, which handle long-context inputs more efficiently than standard attention mechanisms. This is why the model sustains performance at a 1 million token context window, ranking first among open-weight models on the RULER benchmark at that length. For agentic workflows that need to read large codebases, long document sets, or extended conversation histories, that context efficiency is meaningful.

NVFP4 quantization provides another speed boost — the SageMaker JumpStart deployment ships with NVFP4 enabled by default and delivers 5x faster inference than Nemotron K2.6 on compatible hardware.

Benchmark results in context

The 48 Intelligence Index score puts Nemotron 3 Ultra above every Chinese open-weight model currently available and close to the lower end of the closed frontier models. On PinchBench Agent Productivity, it scores 91%, matching Kimi K2.6 and ahead of Qwen 3.5.

On SWE-bench and Terminal Bench 2.0, it completed tasks using fewer tokens per turn than comparable models, reducing the cost of agentic tasks by up to 30% according to NVIDIA's technical report. For teams running long-context agents that make many tool calls, that token efficiency compounds quickly.

The 300+ tokens per second figure is what surprises people the most. Most Chinese models available through commercial APIs run at 50 to 100 tokens per second. Running Nemotron 3 Ultra locally or on SageMaker with NVFP4 quantization gets you 3 to 6x that speed at comparable or better quality.

Who should actually use this

Nemotron 3 Ultra is not a consumer product. It is aimed at developers and enterprises who want to run capable models on their own infrastructure — either on-prem H100 or H200 clusters, or on cloud services like SageMaker where one-click deployment now works out of the box.

The primary use cases NVIDIA is targeting are long-running agentic workflows: code review pipelines, document analysis at scale, multi-step reasoning over large datasets. For these tasks, the combination of 1M token context, 300+ token/second throughput, and open-weight flexibility is genuinely compelling.

For individuals or small teams without GPU infrastructure, the practical path to using Nemotron 3 Ultra is through SageMaker JumpStart or hosted inference providers that support Hugging Face models. Self-hosting at this scale requires serious hardware.

How it compares to what I was already using

I have been running smaller Nemotron models through API providers for document processing work, and the jump in capability from the Super variant to Ultra is substantial on longer, more structured tasks. Ultra handles a full-length SaaS product documentation set in a single pass in a way that the Super variant couldn't without chunking.

For anyone currently routing expensive analytical tasks to closed models like Claude Opus 4.8 or GPT-5.5, Nemotron 3 Ultra is worth evaluating as a cost-saving alternative — particularly if you can tolerate a few points of benchmark performance difference in exchange for data privacy, no rate limits, and infrastructure control.

Frequently asked questions

Can I run Nemotron 3 Ultra on a single GPU?

No. At 550B parameters, even with MoE efficiency, you need a multi-GPU setup — at minimum an H100 or H200 cluster with sufficient VRAM. For most users without dedicated GPU infrastructure, SageMaker JumpStart or a hosted inference provider is the practical access path.

How does Nemotron 3 Ultra compare to DeepSeek V4?

Nemotron 3 Ultra scores higher on the Intelligence Index (48 vs DeepSeek V4's 44.1) and runs 3 to 6x faster through commercial APIs. DeepSeek V4 remains extremely cost-competitive for basic tasks, but Ultra leads on complex agentic and long-context work.

Is Nemotron 3 Ultra available for free?

The weights are open on Hugging Face and free to download under NVIDIA's Research License. Running it requires hardware or a paid cloud service. SageMaker JumpStart charges standard AWS compute rates for inference.