This source only provides a headline — there is no body to show. Use “Read the original” below to read it at the source.
Accepted but Wrong: Silent Failures of LLM Code Repairs after API Migrations
arXiv:2611.08269v2 Announce Type: replace Abstract: After an API migration, an LLM-repaired procedure can run cleanly yet return wrong results, and a typical-input release check ships it. We report a preregistered study of 73 tasks on 60 documented changes in CPython, Node.js and SQLite. In the main stages, repairs are choices among documented code fragments, run on the real runtimes against withheld contracts. Over the complete repair space, the all-assertions gate on typical inputs released 0.410 of wrong repairs. One edge input written from the release note cut this to 0.065, and to 0.061 on 15 unseen clusters; three inputs from a blind low-cost commercial writer (GPT-5.6-luna) reached 0.070. Two more frozen stages enlarged the menu to 1491 fragments and added 233 hard distractors that match legacy output on typical inputs. The edge input (0.058) and the GPT-5.6-luna writer (0.058) passed preregistered tests (two local-model writers did not), yet 0.168 and 0.153 of hard wrong progra…