Newshot

Wrong yet Confident: Automated Detection of Silent Failures in Location-Dependent Software

Summary

arXiv:2611.21615v1 Announce Type: new Abstract: We report an experience paper applying round-trip offset---an automated testing oracle for location-dependent software---at deployment scale across 32,019 GPS delivery transactions, corresponding to 24,501 unique coordinates, from a single USDA COVID-19 food assistance deployment in Bayam\'on, Puerto Rico. The oracle tests each geocoder's implicit spatial accuracy claim by measuring the deviation between an input coordinate and the coordinate recovered after reverse-then-forward geocoding, without requiring manual labeling of correctness. We evaluated one community-contributed and two commercial geocoders against this operationally validated corpus. The experience surfaces two distinct failure classes, which we report with their respective denominators because they are not comparable quantities. First, 2,441 of the 24,501 unique coordinates (10.0%) are invisible to all three platforms: an acknowledged failure with no resolution, and a pro…

Original text(via arXiv cs.SE)

Computer Science > Software Engineering

Important: e-prints posted on arXiv are not peer-reviewed by arXiv; they should not be relied upon without context to guide clinical practice or health-related behavior and should not be reported in news media as established information without consulting multiple experts in the field.

Title:Wrong yet Confident: Automated Detection of Silent Failures in Location-Dependent Software

Abstract:We report an experience paper applying round-trip offset---an automated testing oracle for location-dependent software---at deployment scale across 32,019 GPS delivery transactions, corresponding to 24,501 unique coordinates, from a single USDA COVID-19 food assistance deployment in Bayamón, Puerto Rico. The oracle tests each geocoder's implicit spatial accuracy claim by measuring the deviation between an input coordinate and the coordinate recovered after reverse-then-forward geocoding, without requiring manual labeling of correctness. We evaluated one community-contributed and two commercial geocoders against this operationally validated corpus. The experience surfaces two distinct failure classes, which we report with their respective denominators because they are not comparable quantities. First, 2,441 of the 24,501 unique coordinates (10.0%) are invisible to all three platforms: an acknowledged failure with no resolution, and a property of the corpus rather than of any one geocoder. Second, and platform-specific: 10.0% of the 13,221 coordinates Google resolves to a civic address exhibit round-trip offsets exceeding 2~km---5.4% of the full corpus---returned with no error signal, against 0.5% for the second commercial platform and 0.04% for the community geocoder. Round-trip offset detects all large-offset failures observed in this dataset while standard resolution-rate evaluation detects none. We document an eight-mode fault taxonomy, a structured comparison of detection methods, and concrete lessons for integrating automated geocoder verification into CI/CD pipelines. The key lesson: resolution rate measures availability; round-trip offset measures correctness. In non-standard addressing environments these two metrics select for opposite behaviors, and only the latter catches the failures that matter for autonomous action.
Comments:Submitted to ASE 2026 (March 2026)
Subjects:Software Engineering (cs.SE)
Cite as:arXiv:2611.21615 [cs.SE]
(or arXiv:2611.21615v1 [cs.SE] for this version)
https://doi.org/10.48550/arXiv.2611.21615 arXiv-issued DOI via DataCite

Submission history

Access Paper:

  • HTML (experimental)

Additional Features

  • Audio Summary

Current browse context:

References & Citations

  • Google Scholar
  • Semantic Scholar

BibTeX formatted citation

Bookmark

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

The text above was extracted automatically from the source page and may miss images, tables or code blocks. The original is authoritative.

Published by arXiv cs.SE
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.
Read the original ↗