Computer Science > Artificial Intelligence
Title:The Transformer Paradigm: Dynamic Processing and a New Principle of Inference
Abstract:We argue that the Transformer revolution is not adequately understood as a merely quantitative expansion of deep learning, but as the emergence of a new paradigm based on a different principle of inference. We interpret the Transformer as a system in which token representations are transformed by parameterized transformations, or transforming concepts. During inference, some of these transformations are dynamically constructed from the current input: previously acquired transformations are first adapted and then applied. At the mechanical-structural level, this dynamic processing is realized through output-weight interconnections, whereby outputs of some networks determine the weights of others. The resulting mode of processing, which we call Sequence-level Interactive Dynamic Parallel Processing (SIDPP), provides a framework for explaining strong prompt sensitivity and opens research directions concerning interpretability, control, alternative AI architectures, neuromorphic systems, and possible biological realizations of dynamic processing.
| Comments: | v1: 15 pp., 5 figs., 1 table |
|---|---|
| Subjects: | Artificial Intelligence (cs.AI) ; Hardware Architecture (cs.AR); Computers and Society (cs.CY); Emerging Technologies (cs.ET); Neural and Evolutionary Computing (cs.NE) |
| Cite as: | arXiv:2611.17953 [cs.AI] |
| (or arXiv:2611.17953v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.17953 arXiv-issued DOI via DataCite (pending registration) |
Submission history
Access Paper:
Additional Features
- Audio Summary
Current browse context:
References & Citations
- Google Scholar
- Semantic Scholar
BibTeX formatted citation
Bookmark
Bibliographic and Citation Tools
Code, Data and Media Associated with this Article
Demos
Recommenders and Search Tools
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.