Risk-Controlled Correction for Frozen Multimodal Models under Missing Modalities

Impact 7 arXiv cs.AI

Summary

  • Deployed multimodal models often face missing modalities after their weights are frozen, making training-time repair impossible.
  • The authors show that correctable error is characterized by the Bayes gap between the model prediction and the Bayes prediction given observed modalities.
  • They introduce GUARD (Gated Utility-Aware Recalibration for Deployment), which leaves deployed predictors unchanged while applying retrieval-based correction.
  • GUARD uses conformal risk control to bound the marginal probability that a correction exceeds a prescribed tolerance loss increase.

Why it matters

This research provides a principled method for safely correcting frozen multimodal models when input modalities go missing during deployment, bounding error increases without requiring model weight updates.

Who should care