Computer Science > Machine Learning
Title:Arbitrary-Accuracy Neural Approximation with Optimal Neuron Count and Optimal Bit Complexity
Abstract:We study the minimum number of hidden neurons required for arbitrary-accuracy approximation of continuous functions on $[0,1]^d$ and the associated encoding complexity. For $d\geq 2$, we construct two fixed, explicitly defined activation functions and, for each, a closed-form network with two hidden layers of widths $d$ and $1$ that achieves arbitrary accuracy in the uniform norm. We prove that $d+1$ is the exact minimum total number of hidden neurons among standard feedforward networks with locally integrable activations and affine outputs. We further give a simpler construction using a single elementary activation that combines the floor and exponential functions. This construction requires three hidden layers of widths $d$, $1$, and $2$. If a skip connection is allowed, widths $d$, $1$, and $1$ suffice. For a bounded $\alpha$-Hölder class with $\alpha\in(0,1]$, the networks with the digit-extraction activation and with the elementary activation require $O(\varepsilon^{-d/\alpha}\log(1/\varepsilon))$ bits, matching the metric-entropy lower bound up to a logarithmic factor. The other $(d,1)$ network, which stores differences of neighboring values along a spanning tree, needs only $\Theta(\varepsilon^{-d/\alpha})$ bits and is optimal in both neuron count and bit complexity.
| Subjects: | Machine Learning (cs.LG) ; Neural and Evolutionary Computing (cs.NE); Machine Learning (stat.ML) |
|---|---|
| Cite as: | arXiv:2609.35628 [cs.LG] |
| (or arXiv:2609.35628v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2609.35628 arXiv-issued DOI via DataCite |
Submission history
Access Paper:
Additional Features
- Audio Summary
Current browse context:
References & Citations
- Google Scholar
- Semantic Scholar
BibTeX formatted citation
Bookmark
Bibliographic and Citation Tools
Code, Data and Media Associated with this Article
Demos
Recommenders and Search Tools
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.