Multi-Future Drafting: Mixture Block Drafters for Tree-Verified Speculative Decoding
Summary
- Researchers identify that product-ranked draft trees in speculative decoding misallocate probability mass by creating chimeric prefixes when handling multiple plausible future continuations.
- The paper introduces Multi-Future Drafting (MFD), utilizing learned mode embeddings and a mixture DDTree to draft coherent blocks and let target verifiers select futures losslessly.
- Experiments across four Qwen models ranging from 4B to 27B parameters show MFD achieves up to 7.8x speedup at batch size 1.
Why it matters
MFD improves the efficiency of speculative decoding in large language models by eliminating chimeric prefix generation, leading to measurable speedups over existing single-mode drafting methods.