Computer Science > Software Engineering
Important: e-prints posted on arXiv are not peer-reviewed by arXiv; they should not be relied upon without context to guide clinical practice or health-related behavior and should not be reported in news media as established information without consulting multiple experts in the field.
Title:Wrong yet Confident: Automated Detection of Silent Failures in Location-Dependent Software
Abstract:We report an experience paper applying round-trip offset---an automated testing oracle for location-dependent software---at deployment scale across 32,019 GPS delivery transactions, corresponding to 24,501 unique coordinates, from a single USDA COVID-19 food assistance deployment in Bayamón, Puerto Rico. The oracle tests each geocoder's implicit spatial accuracy claim by measuring the deviation between an input coordinate and the coordinate recovered after reverse-then-forward geocoding, without requiring manual labeling of correctness. We evaluated one community-contributed and two commercial geocoders against this operationally validated corpus. The experience surfaces two distinct failure classes, which we report with their respective denominators because they are not comparable quantities. First, 2,441 of the 24,501 unique coordinates (10.0%) are invisible to all three platforms: an acknowledged failure with no resolution, and a property of the corpus rather than of any one geocoder. Second, and platform-specific: 10.0% of the 13,221 coordinates Google resolves to a civic address exhibit round-trip offsets exceeding 2~km---5.4% of the full corpus---returned with no error signal, against 0.5% for the second commercial platform and 0.04% for the community geocoder. Round-trip offset detects all large-offset failures observed in this dataset while standard resolution-rate evaluation detects none. We document an eight-mode fault taxonomy, a structured comparison of detection methods, and concrete lessons for integrating automated geocoder verification into CI/CD pipelines. The key lesson: resolution rate measures availability; round-trip offset measures correctness. In non-standard addressing environments these two metrics select for opposite behaviors, and only the latter catches the failures that matter for autonomous action.
| Comments: | Submitted to ASE 2026 (March 2026) |
|---|---|
| Subjects: | Software Engineering (cs.SE) |
| Cite as: | arXiv:2611.21615 [cs.SE] |
| (or arXiv:2611.21615v1 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2611.21615 arXiv-issued DOI via DataCite |
Submission history
Access Paper:
- HTML (experimental)
Additional Features
- Audio Summary
Current browse context:
References & Citations
- Google Scholar
- Semantic Scholar
BibTeX formatted citation
Bookmark
Bibliographic and Citation Tools
Code, Data and Media Associated with this Article
Demos
Recommenders and Search Tools
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.