ChainFlow: Few-Step Latent Flow Matching for Speculative Decoding
Summary
- Researchers introduce ChainFlow, a novel speculative decoding paradigm utilizing a flow-matching drafter conditioned on a target model's hidden states.
- ChainFlow requires only one or two flow steps by compressing hidden states via a variational autoencoder and restoring inter-token dependencies with feedback heads.
- When evaluated on Qwen3.5 models within vLLM, ChainFlow outperforms EAGLE-3 and DFlash by 39% and 11% respectively.
- The authors have open-sourced their code and drafter models.
Why it matters
ChainFlow accelerates LLM inference speeds more effectively than existing autoregressive or diffusion drafters, offering measurable performance gains for production serving frameworks like vLLM.