DNeo: Your Parallel Draft Model Needs Better Target Feature Encoding

Impact 7 arXiv cs.AI

Summary

  • Researchers introduce DNeo, a new parallel draft architecture designed to improve speculative-diffusion decoding for large language models.
  • Standard feature encoding in parallel drafters suffers from lossy projections (residual mixing), a lack of memory for past proposals, and uniform feature sampling from target layers.
  • DNeo addresses these limitations using Mixture-of-Features Attention (MoFA) with working memory for rejected proposals, 2-tap causal convolutions, prefix-only cross-attention, and non-uniform target layer selection.
  • The proposed DNeo architecture achieves 1.3x faster generation speed than previous state-of-the-art methods like DFlash-2 without impacting the drafter footprint or generation quality.

Why it matters

DNeo improves inference speed for large language models through architectural enhancements in speculative-diffusion decoding, offering a new state-of-the-art approach without increasing compute footprints or degrading text quality.

Who should care