Computer Science > Machine Learning
Title:MarathonTS: a benchmark of long synthetic time series from window assembly and long-series generators
Abstract:Many applications of synthetic time series require a series that is much longer than the windows on which a generative model is trained. Such a series can be assembled from a pool of short generated windows, which requires deciding which windows are taken, in what order, and how their seams are processed, or it can be produced directly by a long-series generator. This study presents MarathonTS, to our knowledge the first open benchmark that compares both families under one protocol. Every method produces a series of the same length on 9 open hourly datasets with 10 seeds. The series is evaluated by its fidelity to the real data (83 metrics in 12 groups, against three real references: train, validation and an out-of-time split), by its utility (models are trained on the synthetic series only and tested on real data, TSTR against TRTR, in 14 classification and regression tasks), by its memorisation of the training data and by its computational cost. The statistical analysis is carried out as a separate stage that reads the published tables and gives the rankings of every method in every slice, the answers to the research questions with tests and uncertainty, and the results of a pre-specified confirmatory family. In the main run, the methods differ in both confirmatory cells; the choice of windows matters more than the processing of their seams, which does not improve the series, and no method trains the downstream models as well as the real series. The top tier of the leaderboard is reproduced on other hardware, whereas the exact places within it are not. For the reproducibility of the study, the code, the data hashes and all results are publicly available (links below).
| Subjects: | Machine Learning (cs.LG) |
|---|---|
| Cite as: | arXiv:2611.18190 [cs.LG] |
| (or arXiv:2611.18190v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.18190 arXiv-issued DOI via DataCite (pending registration) |
Submission history
Access Paper:
- HTML (experimental)
Additional Features
- Audio Summary
Current browse context:
References & Citations
- Google Scholar
- Semantic Scholar
BibTeX formatted citation
Bookmark
Bibliographic and Citation Tools
Code, Data and Media Associated with this Article
Demos
Recommenders and Search Tools
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.