A Spectral Framework for Universal Proxy Regularization in Model Merging
Summary
- Researchers propose a Spectral Framework for Universal Proxy Regularization to characterize parameter interference in model merging.
- The framework relates proxy and task distributions via spectral coverage and leakage, proving that better alignment reduces parameter interference.
- The study evaluates three instantiation mechanisms for the proxy regularizer under linear and non-linear fine-tuning regimes.
- Empirical validation on vision-language models like CLIP confirms the theoretical predictions.
Why it matters
This research provides theoretical foundations and practical regularizers for model merging, helping engineers combine multi-task models efficiently without violating data privacy or requiring joint training.