Computer Science > Machine Learning
Title:Encoding, Decodability, and Use: A Five-Layer Framework for Benchmark Measurement Audits with Evidence from UAV Intent Recognition
Abstract:Benchmark accuracy is routinely read as evidence that a model understands the construct the benchmark measures. We propose and instantiate on AeroIntent a five-layer framework that decomposes that reading into observation, representation, decoding, use, and behaviour. The framework separates encoding, decodability, and use as distinct levels. In the instantiation, the observation layer is unreliable on 7 of the 20 Drone programs and floors the direct visual correlate of radial velocity at a 0.26\% scale change. The representation layer exposes two task-designed RGB features with nominal signals. RAFT flow radial bias reaches $r=+0.257$ ($p=0.030$), and bounding-box optical-flow divergence reaches $r=+0.203$ ($p=0.029$). No multi-feature readout gives a fold-mean test correlation above the cluster minimum detectable effect (MDE), and the largest positive global out-of-sample correlation is $r=+0.080$. The decoding layer sets the MDE at $|r|=0.599$ at the multi-feature Bonferroni threshold ($\alpha=0.00714$, power 0.80), and the observed $r=+0.080$ has power 0.00. Two injection experiments establish the MDE's condition from both sides: aligned injections are detected at or above the simulated power, and orthogonal injections are not detected. The use layer finds accuracy consistent with program memory (program-identity 0.937/0.597/0.210) and multi-region static cues. The behaviour layer is consistent with accuracy not requiring a radial-velocity readout (appearance 0.6552 vs bounding-box trajectory 0.4080, cluster-level paired $\Delta=+0.2471$, $p=0.0132$).
| Subjects: | Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI) |
|---|---|
| Cite as: | arXiv:2611.22434 [cs.LG] |
| (or arXiv:2611.22434v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.22434 arXiv-issued DOI via DataCite (pending registration) |
Submission history
Access Paper:
- HTML (experimental)
Additional Features
- Audio Summary
Current browse context:
References & Citations
- Google Scholar
- Semantic Scholar
BibTeX formatted citation
Bookmark
Bibliographic and Citation Tools
Code, Data and Media Associated with this Article
Demos
Recommenders and Search Tools
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.