NVIDIA AI Releases Nemotron-Labs-Sigma, a Continuous Diffusion Language Model with 14 checkpoints at 3B/8B
Summary
- NVIDIA released Nemotron-Labs-Sigma, a family of 14 open-weight continuous diffusion language models available in 3B and 8B parameter sizes.
- Unlike discrete diffusion models, Sigma denoises text within a learned 16-D smooth latent embedding space using a Ministral-style decoder-only transformer backbone.
- The release includes base, instruction-tuned, few-step distilled, and self-speculative checkpoints designed for commercial use under the NVIDIA Nemotron Open Model License.
- Self-speculative 8B variants achieve an average speedup of 3.46x over standard autoregressive decoding on GB200 hardware using frameworks like SGLang.
- The models require a CUDA GPU in bfloat16 format, with CPU inference explicitly unsupported.
Why it matters
Nemotron-Labs-Sigma shifts language model diffusion from discrete tokens to a continuous latent space, enabling practical few-step distillation and high-speed self-speculative generation on NVIDIA hardware.