Newshot

What Balanced Accuracy Cannot Identify in Neural-Signal Decoding under Test-Time Adaptation

Summary

arXiv:2611.17925v1 Announce Type: new Abstract: Adaptation for neural-signal classification is usually transductive: each batch is scored after updating on it, and the gain depends on the adapted model [4]. This paper asks what an aggregate change fails to characterise inside the cohort. Full test-time adaptation (full_ttt) was evaluated in 11 dataset-encoder cells (four each on Siena/EEG and PTB-XL/ECG, three on Sleep-EDF/EEG), with splits and source-model evaluation fixed; the subject was the unit, exact sign tests were Holm-corrected and conditional on the shared stream, and bootstrap intervals accompanied a dependence audit. Two results of different kinds follow. In the binary case balanced accuracy gives a one-class decoder the same value as one reporting both classes at 50%, so the aggregate cannot identify a class that stopped being reported; a class covering a tenth of a subject's samples went unpredicted in 26-47 of 112 subject-cells, against 7-9 for the controls, which is a d…

Original text(via arXiv cs.AI)

Computer Science > Artificial Intelligence

Title:What Balanced Accuracy Cannot Identify in Neural-Signal Decoding under Test-Time Adaptation

Abstract:Adaptation for neural-signal classification is usually transductive: each batch is scored after updating on it, and the gain depends on the adapted model [4]. This paper asks what an aggregate change fails to characterise inside the cohort. Full test-time adaptation (full_ttt) was evaluated in 11 dataset-encoder cells (four each on Siena/EEG and PTB-XL/ECG, three on Sleep-EDF/EEG), with splits and source-model evaluation fixed; the subject was the unit, exact sign tests were Holm-corrected and conditional on the shared stream, and bootstrap intervals accompanied a dependence audit. Two results of different kinds follow. In the binary case balanced accuracy gives a one-class decoder the same value as one reporting both classes at 50%, so the aggregate cannot identify a class that stopped being reported; a class covering a tenth of a subject's samples went unpredicted in 26-47 of 112 subject-cells, against 7-9 for the controls, which is a descriptive count. With cohort, split and update rule fixed, the within-subject class contrast follows the encoder, tested directly rather than inferred from tests against zero: two of six Siena encoder pairs survive the dependence readings at their nominal reference and one at a fold-level reference, while two of ten Sleep-EDF stage pairs pass the run-clustered gate at its nominal reference; aggregate and contrast named different encoders in 10 of 11. The protocol bounds both results, since a stream-interleaving effect reverses sign with state sharing (+3.5 to -2.4 pp). An aggregate score can hide individual harm, as [1] showed, but it cannot distinguish a uniform loss from a deleted class; class-level change should therefore be reported alongside the aggregate and subject-level change alongside it, with both results conditional on the cell, the rule and the stream construction.
Subjects:Artificial Intelligence (cs.AI)
Cite as:arXiv:2611.17925 [cs.AI]
(or arXiv:2611.17925v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2611.17925 arXiv-issued DOI via DataCite (pending registration)

Submission history

Access Paper:

Additional Features

  • Audio Summary

Current browse context:

References & Citations

  • Google Scholar
  • Semantic Scholar

BibTeX formatted citation

Bookmark

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

The text above was extracted automatically from the source page and may miss images, tables or code blocks. The original is authoritative.

Published by arXiv cs.AI
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.
Read the original ↗