D\'ecoupage: Directorial Audio-Visual Control for Long Multi-Shot Video Generation
Summary
- Researchers introduced DECOUPAGE, a screenplay-driven framework for long-form, multi-shot, and multi-character joint audio-video generation.
- The system uncouples modality boundaries, separating visual and acoustic tracks while managing persistent character identities.
- It features dual-schedule temporal cross-attention and a directorial lip-sync mechanism that handles off-screen dialogue and silences.
- DECOUPAGE enables precise control over shot transitions, audio timing, and visibility-aware lip synchronization for 15- to 60-second video generation.
Why it matters
DECOUPAGE addresses limitations in existing audio-visual generative models by independently coordinating visual and acoustic timing, enabling complex cinematic storytelling techniques like J- and L-cuts without audio-visual artifacts.