Computer Science > Machine Learning
Title:Omni-modal decomposition autoencoders learn full-stack wearable disentangled representations
Abstract:Learning disentangled representations is key for general-purpose multi-modal wearable models, yet existing approaches rarely combine classification, disentanglement, fusion, and generation of heterogeneous multi-modal time series in a single model. We introduce Omni-modal Variational Decomposition Autoencoders (OmniDecVAEs), a framework that learns multi-purpose representations from arbitrarily many modalities with a single shared encoder-decoder. OmniDecVAEs extend DecVAEs by learning time-frequency latent subspaces per modality through a multi-view self-supervised decomposition loss and a component-wise conditional decoder. On the omni-modal human activity recognition (HAR) HARWE dataset (up to thirty channels) and the public UCI HAR and WESAD datasets, OmniDecVAEs learn the most disentangled representations for unseen subjects (DCI disentanglement up to 3.0 percentage points (pp) higher). On HARWE, they improve subject-dependent HAR and identity recognition by 1.1 pp and 5.8 pp, and outperform supervised end-to-end models by +31.6 pp and +22.3 pp on unseen subjects of multi-device datasets. Moreover, their generative capacity improves reconstruction mean squared error over multi-modal VAEs (34-74% improvement) and their synthetic data improve affect recognition on unseen subjects (2 pp). With 4.1M parameters, invariant to the number of modalities (21xfewer than a multi-branch VAE), OmniDecVAEs are scalable, single-model full-stack learners for wearable representations.
| Comments: | 14 pages, 7 figures, 7 tables |
|---|---|
| Subjects: | Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS); Signal Processing (eess.SP); Machine Learning (stat.ML) |
| Cite as: | arXiv:2608.07385 [cs.LG] |
| (or arXiv:2608.07385v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2608.07385 arXiv-issued DOI via DataCite |
Submission history
Access Paper:
- HTML (experimental)
Additional Features
- Audio Summary
Current browse context:
References & Citations
- Google Scholar
- Semantic Scholar
BibTeX formatted citation
Bookmark
Bibliographic and Citation Tools
Code, Data and Media Associated with this Article
Demos
Recommenders and Search Tools
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.