DNeo: Your Parallel Draft Model Needs Better Target Feature Encoding
Summary
- Researchers introduce DNeo, a new parallel draft architecture designed to improve speculative-diffusion decoding for large language models.
- Standard feature encoding in parallel drafters suffers from lossy projections (residual mixing), a lack of memory for past proposals, and uniform feature sampling from target layers.
- DNeo addresses these limitations using Mixture-of-Features Attention (MoFA) with working memory for rejected proposals, 2-tap causal convolutions, prefix-only cross-attention, and non-uniform target layer selection.
- The proposed DNeo architecture achieves 1.3x faster generation speed than previous state-of-the-art methods like DFlash-2 without impacting the drafter footprint or generation quality.
Why it matters
DNeo improves inference speed for large language models through architectural enhancements in speculative-diffusion decoding, offering a new state-of-the-art approach without increasing compute footprints or degrading text quality.