Statistics > Methodology
Title:Locally Linear Generalized Single Index Regression
Abstract:In this work we propose and analyze a procedure for estimating a generalized single-index regression model. This generalized single-index regression framework models the conditional mean of the responses $Y$ as a function of the covariates $X$ where the mapping is assumed to have the form $f \circ g$, where $g$ is $\beta$-Hölder with $\beta \geq 2$ and $f$ can be a discontinuous function of bounded variation. We propose a general procedure that operates under minimal assumptions about the distribution of covariates $X$, requiring only boundedness, sufficient smoothness and absolute continuity with respect to Lebesgue. Our procedure jointly estimates the gradient vector $\hat v(x) \approx \nabla g(x)$ and the conditional mean $\mu(x) = f(g(x))$ by solving a localized weighted linear least squares problem. We derive central limit theorems for gradient vector estimates $\hat{v}_x$ and regression estimates $\hat{\mu}(x)$, and we derive rates of convergence for our approach, illustrating that it indeed achieves a stochastic convergence rate matching the minimax mean squared error rate when $\beta = 2$ and $f$ has a locally Lipschitz derivative. Unlike prior approaches our velocity estimation results accommodate link functions in the local bounded-variation class, allowing for nonsmooth and even discontinuous links while retaining consistency and asymptotic normality at almost every inference point. Finally, we instantiate this model in the situation where $g = \Pi$ is a closest point projection onto a compact and regular embedded $C^3$ curve $\gamma$, a setting recently studied in the literature. We also supplement our theoretical results with experiments on real and synthetic data to demonstrate the utility of our procedure.
| Subjects: | Methodology (stat.ME) ; Machine Learning (stat.ML) |
|---|---|
| Cite as: | arXiv:2611.22011 [stat.ME] |
| (or arXiv:2611.22011v1 [stat.ME] for this version) | |
| https://doi.org/10.48550/arXiv.2611.22011 arXiv-issued DOI via DataCite (pending registration) |
Submission history
Access Paper:
- HTML (experimental)
Current browse context:
References & Citations
- Google Scholar
- Semantic Scholar
BibTeX formatted citation
Bookmark
Bibliographic and Citation Tools
Code, Data and Media Associated with this Article
Demos
Recommenders and Search Tools
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.