NVIDIA AI Releases Nemotron-Labs-Sigma, a Continuous Diffusion Language Model with 14 checkpoints at 3B/8B

Impact 8 MarkTechPost

Summary

  • NVIDIA released Nemotron-Labs-Sigma, a family of 14 open-weight continuous diffusion language models available in 3B and 8B parameter sizes.
  • Unlike discrete diffusion models, Sigma denoises text within a learned 16-D smooth latent embedding space using a Ministral-style decoder-only transformer backbone.
  • The release includes base, instruction-tuned, few-step distilled, and self-speculative checkpoints designed for commercial use under the NVIDIA Nemotron Open Model License.
  • Self-speculative 8B variants achieve an average speedup of 3.46x over standard autoregressive decoding on GB200 hardware using frameworks like SGLang.
  • The models require a CUDA GPU in bfloat16 format, with CPU inference explicitly unsupported.

Why it matters

Nemotron-Labs-Sigma shifts language model diffusion from discrete tokens to a continuous latent space, enabling practical few-step distillation and high-speed self-speculative generation on NVIDIA hardware.

Who should care