D\'ecoupage: Directorial Audio-Visual Control for Long Multi-Shot Video Generation

Impact 7 arXiv cs.AI

Summary

  • Researchers introduced DECOUPAGE, a screenplay-driven framework for long-form, multi-shot, and multi-character joint audio-video generation.
  • The system uncouples modality boundaries, separating visual and acoustic tracks while managing persistent character identities.
  • It features dual-schedule temporal cross-attention and a directorial lip-sync mechanism that handles off-screen dialogue and silences.
  • DECOUPAGE enables precise control over shot transitions, audio timing, and visibility-aware lip synchronization for 15- to 60-second video generation.

Why it matters

DECOUPAGE addresses limitations in existing audio-visual generative models by independently coordinating visual and acoustic timing, enabling complex cinematic storytelling techniques like J- and L-cuts without audio-visual artifacts.

Who should care