Computer Science > Machine Learning
Title:Semi-Supervised Learning with Noisy Proxy Covariates: Generalization Bounds and Distribution Regression
Abstract:In many modern machine learning pipelines, abundant pretrained representations serve as noisy proxy covariates, while task-specific labels remain scarce. We study semi-supervised regression in this setting, and propose a simple two stage estimator that learns kernel eigenfeatures from all proxy covariates and fits a ridge predictor on labeled data. We derive finite sample bounds showing that fast labeled sample rates are recovered when proxy perturbation is controlled and unlabeled proxy covariates are sufficiently abundant. We also show that distribution regression is a direct special case, with analogous guarantees when the finite bag size is large enough. Experiments show consistent gains over supervised and semi-supervised baselines, especially in low label regimes.
| Subjects: | Machine Learning (cs.LG) ; Information Theory (cs.IT); Machine Learning (stat.ML) |
|---|---|
| Cite as: | arXiv:2606.00512 [cs.LG] |
| (or arXiv:2606.00512v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2606.00512 arXiv-issued DOI via DataCite | |
| Journal reference: | Proceedings of the 43rd International Conference on Machine Learning (ICML 2026) |
Submission history
Access Paper:
- HTML (experimental)
Additional Features
- Audio Summary
Current browse context:
References & Citations
- Google Scholar
- Semantic Scholar
BibTeX formatted citation
Bookmark
Bibliographic and Citation Tools
Code, Data and Media Associated with this Article
Demos
Recommenders and Search Tools
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.