Last week I wrote about DFlash, z-lab’s diffusion drafter for speculative decoding — a model that generates candidate tokens in bulk and lets the target model verify them. It’s faster. It’s different. But it still carries two models: a draft and a target, two graphs, two sets of weights.

NVIDIA’s Nemotron-Labs-Diffusion does something subtly more ambitious. It doesn’t carry a second model. It is one model that can decode in three modes: autoregressive (the standard left-to-right path), diffusion (parallel denoising of masked tokens), and self-speculation (diffusion drafts, AR verifies, same KV cache). The switch is a single attention-pattern change.

Same weights. Different decoding strategy. One architecture.

The three modes

The model family ships at 3B, 8B, and 14B parameter scales, each with base, instruct, and vision-language variants. All under NVIDIA’s commercially-friendly Nemotron license. All available on HuggingFace as nvidia/Nemotron-Labs-Diffusion-{3B,8B,14B}.

Mode 1 — Autoregressive: Standard causal decoding. Token by token. Memory-bound. What every other local model does.

Mode 2 — Diffusion: Parallel decoding. All masked tokens denoised simultaneously. Compute-bound. The model generates the full sequence in K denoising steps instead of L sequential steps. Theoretically faster for long outputs — practically slower on most implementations because each denoising step costs more FLOPs than an AR forward pass.

Mode 3 — Self-speculation: The interesting one. Diffusion generates draft tokens in parallel. AR verifies them sequentially using shared KV cache. Acceptance length is the key metric: how many draft tokens get accepted per verification cycle. The paper reports acceptance lengths that translate to 6.0× more tokens per forward pass than pure AR on SPEED-Bench with SGLang on a GB200 GPU.

Why tri-mode matters

DFlash and similar diffusion drafters solve one problem: speed up autoregressive decode. They’re a draft model that sits beside the target model, generating candidates that the target verifies. Two models. Two forward passes (draft + verify, though verify can accept multiple drafts per pass).

Tri-mode collapses that into one model. The model can switch between AR, diffusion, and self-speculation depending on the workload:

  • AR mode for short outputs where the overhead of diffusion setup isn’t worth it.
  • Diffusion mode for long outputs where parallel generation amortizes the per-step cost.
  • Self-speculation mode for the sweet spot: medium outputs where you want speculative speedup without carrying two separate models.

This isn’t just a theoretical distinction. It changes the VRAM math. A diffusion drafter is a separate model — typically 1–5 billion parameters, 0.5–5 GB for the quantized version. Self-speculation needs no extra model. You get speculative decoding for free (modulo the sampling overhead of the diffusion steps).

The quality numbers

The paper (arxiv:2607.05722, July 2026) evaluates across standard benchmarks. The headline results for the instruct variants:

  • Nemotron-Labs-Diffusion-8B-Instruct outperforms comparable open-source AR models on MMLU, MMLU-Redux, HellaSwag, MBPP, and GSM8K. The exact delta varies by benchmark but is consistently within 1–3 percentage points of the closest AR competitor.
  • Nemotron-Labs-Diffusion-14B-Instruct closes the gap further, often exceeding AR baselines on reasoning benchmarks.
  • On SPEED-Bench with SGLang on GB200: 6.0× more tokens per forward pass in self-speculation mode compared to Qwen3-8B AR. That’s a throughput win, not just a latency win.

The vision-language variants add a vision encoder and multimodal projector to the diffusion backbone, inheriting the joint AR-diffusion objective and tri-mode capability. No separate VLM needed.

Where diffusion models actually lose

The Nemotron paper is optimistic about self-speculation. It’s honest about vanilla diffusion’s weaknesses. Several independent studies confirm the same pattern:

Throughput hierarchy: AR > block-diffusion > vanilla diffusion. The “paradox” that diffusion models are theoretically faster but practically slower on most setups is real. A systematic efficiency study (arxiv:2510.18480) evaluated DLMs, AR, and block-diffusion under varying sequence lengths and batch sizes. AR consistently achieved the highest throughput. Vanilla DLMs were the slowest across most settings.

Long-context degradation: DLM throughput drops sharply as generation length increases. At 64-token generation length, throughput drops about 75% for vanilla DLMs. AR and block-diffusion models stay stable up to ~2K tokens, then decline gradually. The quadratic FLOPs per token with sequence length is the culprit — each denoising step processes the full sequence.

Batch-size scaling: AR benefits more from batch parallelism than diffusion models. AR throughput scales effectively up to batch size 16; DLM throughput plateaus around batch size 8. For high-throughput offline batch processing, AR eventually overtakes diffusion because the additional FLOPs for iterative denoising outweigh the memory bandwidth savings.

The compute-bound cliff: DLMs have higher arithmetic intensity than AR — they use GPU compute more efficiently per token. But that’s a double-edged sword. Higher arithmetic intensity means they hit the GPU’s compute ceiling at smaller batch sizes. In latency-sensitive scenarios (batch size 1, interactive chat), DLMs can pull ahead with enough optimization. In throughput scenarios (large batches, offline processing), AR wins.

Self-speculation sidesteps the cliff

This is where tri-mode’s third mode matters. Self-speculation doesn’t require full-sequence denoising. It generates draft tokens in blocks using diffusion, then verifies them with AR. The acceptance length (how many drafts get accepted per verification cycle) determines whether self-speculation is faster than AR.

The Nemotron paper’s self-speculation mode achieves acceptance lengths that translate to 6.0× throughput on GB200. But this is on a datacenter GPU with massive memory bandwidth and compute headroom. On consumer hardware, the picture is different.

A speed-of-light analysis in the paper shows that under an optimal sampler, self-speculation can theoretically deliver up to 76.5% more tokens per forward pass. Current samplers still leave a large fraction of the available parallelism unused. That’s both a constraint and an opportunity — the gap between current and optimal self-speculation is real and potentially large.

The GGUF question

The HuggingFace model pages for Nemotron-Labs-Diffusion-8B and -14B list quantization support across llama.cpp, Ollama, and LM Studio. GGUF converters are already shipping — mradermacher/OpenReasoning-Nemotron-14B-GGUF and spiritbuun/Nemotron-Labs-Diffusion-14B-Q8_0-GGUF are among the earliest.

The architecture is different enough from standard AR models that quantization behavior may not translate cleanly. DFlash taught us that: the Qwen3.6 drafter’s causal sliding-window layers are Q4-fragile. Nemotron’s joint AR-diffusion attention patterns have similar sensitivity — but the community is mapping them. IntellAgents has published quantization benchmarks: Q8_0 loses 0.4% top-1 agreement vs F16, Q5_K_M drops to 92.8%, and Q4_K_M hits 88.7%. The Q4_K_M numbers are rougher in non-AR modes but functional for AR text generation, which is where most local users land anyway.

What it means for local AI

The tri-mode model is a step toward hardware-adaptive inference. Instead of choosing between a quantized AR model (fast decode, slow prefill) and a diffusion model (slow decode, parallel generation), you get a single model that picks the mode that matches your workload.

On a memory-bound GPU with limited VRAM, AR mode gives you the most predictable performance. On a compute-rich GPU with bandwidth to spare, diffusion mode can outperform AR on long outputs. Self-speculation sits in the middle — speculative decoding without the VRAM cost of a separate draft model.

The practical implication: if you’re building a local AI system that handles diverse workloads (short chat responses, long document summaries, batch processing), a tri-mode model may be more cost-effective than a specialized AR + diffusion-drafter pair. You get speculative decoding for free, AR for reliability, and diffusion for long outputs — all from one weight file.

The Nemotron-Labs-Diffusion family isn’t the final word on diffusion language modeling. Pure diffusion models like Dream and LLaDa still lead on certain reasoning tasks. Block-diffusion hybrids like SDAR and Efficient-DLM are closing the accuracy gap faster. But tri-mode is the first architecture that treats AR and diffusion not as competing paradigms but as different gears in the same transmission.

And that’s a more useful distinction than either paradigm alone.