This source only provides a headline — there is no body to show. Use “Read the original” below to read it at the source.
A Spectral Framework for Universal Proxy Regularization in Model Merging
arXiv:2611.17728v1 Announce Type: new Abstract: Model merging constructs multi-task models without expensive joint training but often suffers from parameter interference. Some solutions mitigate this by constraining finetuning using data from the other tasks to be merged, violating the core independence and privacy assumption of model merging. Recent work offers preliminary empirical evidence that a universal, task-agnostic proxy distribution can substitute for such data, but leaves open when and why it suffices. To this end, we propose a \textit{Spectral Framework for Universal Proxy Regularization}, which characterizes the relationship between a proxy distribution and the distribution of other tasks to be merged through spectral coverage ($\alpha$) and leakage ($\beta$). Based on the framework, we derive a unified parameter interference bound: smaller $(\alpha, \beta)$ provably yields lower parameter interference through better spectral alignment. To exploit such a proxy, we augment…