Towards Robust Vision-Language Reasoning: A Multi-Stage Optimization Pipeline with Masked Diffusion Model
Summary
- Researchers propose a multi-stage optimization pipeline utilizing masked diffusion models to improve vision-language reasoning in autonomous driving.
- The method addresses domain gaps by optimizing planning input representations during supervised fine-tuning and applying metric-guided diversity filtering during reinforcement learning.
- At inference time, Dynamic Autonomous Driving Decoding adaptively adjusts decoding modes based on scene complexity.
- The resulting Masked Diffusion Drive (MD-Drive) model improves performance by 29.7% over unoptimized models and outperforms autoregressive VLM baselines.
Why it matters
This research demonstrates that discrete diffusion generation can effectively replace or outperform traditional autoregressive vision-language models in autonomous driving, offering a better balance of performance and computational cost.