Computer Science > Machine Learning
Title:Neural Classifiers as Error-Correcting Codes
Abstract:A unified framework is proposed that characterizes supervised deep classification as the decoding of error-correcting codes in the Shannon sense. Cross-entropy optimization is shown to align softmax classifiers with maximum-likelihood decoding, where logits map to Hamming distances to class-specific codewords. This correspondence explains the emergence of standard divergence objectives (e.g., Kullback-Leibler, Jeffreys) and provides a geometric basis for the stability of decision boundaries. Furthermore, the geometries of Neural Collapse and LeJEPA are identified as optimal constellations for decoding under isotropic noise. Unlike classical Error-Correcting Output Codes, deep networks are shown to learn effective codebooks in the feature space implicitly. This insight is leveraged to derive novel criteria for lossless quantization, improved knowledge distillation, and architectural regularization based on code redundancy. Finally, this perspective is extended to self-supervised encoders, offering a cohesive explanation for the geometry of latent representations in modern neural models.
| Comments: | 40 pages, 6 figures. Submitted version. Published in Information Sciences 766 (2027) 124394 |
|---|---|
| Subjects: | Machine Learning (cs.LG) ; Information Theory (cs.IT) |
| Cite as: | arXiv:2611.18239 [cs.LG] |
| (or arXiv:2611.18239v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.18239 arXiv-issued DOI via DataCite (pending registration) | |
| Journal reference: | Information Sciences 766 (2027) 124394 |
| Related DOI : | https://doi.org/10.1016/j.ins.2026.124394 DOI(s) linking to related resources |
Submission history
Access Paper:
- HTML (experimental)
Additional Features
- Audio Summary
Current browse context:
References & Citations
- Google Scholar
- Semantic Scholar
BibTeX formatted citation
Bookmark
Bibliographic and Citation Tools
Code, Data and Media Associated with this Article
Demos
Recommenders and Search Tools
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.