Risk-Controlled Correction for Frozen Multimodal Models under Missing Modalities
Summary
- Deployed multimodal models often face missing modalities after their weights are frozen, making training-time repair impossible.
- The authors show that correctable error is characterized by the Bayes gap between the model prediction and the Bayes prediction given observed modalities.
- They introduce GUARD (Gated Utility-Aware Recalibration for Deployment), which leaves deployed predictors unchanged while applying retrieval-based correction.
- GUARD uses conformal risk control to bound the marginal probability that a correction exceeds a prescribed tolerance loss increase.
Why it matters
This research provides a principled method for safely correcting frozen multimodal models when input modalities go missing during deployment, bounding error increases without requiring model weight updates.