This source only provides a headline — there is no body to show. Use “Read the original” below to read it at the source.
A Layered Simplex Architecture for Large Alphabets
arXiv:2608.19908v3 Announce Type: replace-cross Abstract: A learning architecture induces a prior over its expressible models. Recent work links architectural success to a broad complexity spectrum: model probabilities spanning many orders of magnitude. Understanding how depth shapes these priors and prediction may help explain the success of neural networks and transformers. We study this connection in large-alphabet prediction through the layered simplex architecture (LSA), which multiplies independent uniform simplex vectors coordinate-wise and normalizes. At depth zero, the prior is concentrated on the uniform distribution; at depth one, the Bayesian predictor is Laplace's rule. Each additional layer shifts prior mass toward more concentrated distributions. We derive exact Bayesian marginal likelihood and regret expressions using reusable depth-specific kernels. Mixing over depths up to a cutoff logarithmic in alphabet size yields an explicit predictor with rigorous guarantees. F…