Statistics > Machine Learning
Title:Survey-aware Machine Learning: A Guideline for Valid Population Health Inference based on Scoping Review
Abstract:Machine Learning (ML) models trained on complex health surveys such as the National Health and Nutrition Examination Survey (NHANES) often ignore primary sampling units, stratification variables, and sampling weights. This practice violates the independence assumptions of standard evaluation methods. As a result, estimates become biased, uncertainty is underestimated, and fairness assessments fail to reflect population-level disparities. We propose Survey-aware Machine Learning (SaML), a nine-step guideline that incorporates survey design metadata across the ML lifecycle. Through a scoping review of 16 methodological papers, we summarize existing work on weighted model training, design-based cross-validation, and survey-adjusted performance evaluation. We also identify gaps in hyperparameter tuning and deployment. We provide task-specific guidance that clarifies which steps are required for different analytical objectives. SaML provides a checklist for valid population inference from survey data.
| Comments: | Published at the 7th Conference on Health, Inference, and Learning (CHIL 2026), PMLR 333:198-220, 2026. this https URL |
|---|---|
| Subjects: | Machine Learning (stat.ML) ; Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.08963 [stat.ML] |
| (or arXiv:2605.08963v2 [stat.ML] for this version) | |
| https://doi.org/10.48550/arXiv.2605.08963 arXiv-issued DOI via DataCite |
Submission history
Access Paper:
- HTML (experimental)
Current browse context:
References & Citations
- Google Scholar
- Semantic Scholar
BibTeX formatted citation
Bookmark
Bibliographic and Citation Tools
Code, Data and Media Associated with this Article
Demos
Recommenders and Search Tools
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.