A Spectral Framework for Universal Proxy Regularization in Model Merging

Impact 7 arXiv cs.AI

Summary

  • Researchers propose a Spectral Framework for Universal Proxy Regularization to characterize parameter interference in model merging.
  • The framework relates proxy and task distributions via spectral coverage and leakage, proving that better alignment reduces parameter interference.
  • The study evaluates three instantiation mechanisms for the proxy regularizer under linear and non-linear fine-tuning regimes.
  • Empirical validation on vision-language models like CLIP confirms the theoretical predictions.

Why it matters

This research provides theoretical foundations and practical regularizers for model merging, helping engineers combine multi-task models efficiently without violating data privacy or requiring joint training.

Who should care