Computer Science > Computer Vision and Pattern Recognition
Title:Deciding Without Pointing: A Case Study of Whether a Decision Model Localizes the Anomalies It Detects
Abstract:Decision models can answer a typed question about an image with a probability per option, without generating text. In our case study of Clef-flash (post-trained Qwen3.5-9B backbone) on industrial inspection benchmarks, we ask whether it localizes the anomalies it detects. We obtained uneven results with zero-shot detection: per-category image AUROC 0.924 on MVTec AD and 0.668 on MVTec AD 2. On the former, the decision head's routing attention produces 0.391 mean per-image pixel AUROC, and hiding the image tokens from it leaves detection unchanged. The multimodal LLM's backbone attention does point at the defect, however it falls below a center prior when we change question. It falls below chance if we remove textual context in the question, partly due to backbone attention leaving the object, while the head's map hardly changes. The backbone's states carry the location when probed with normal images, on the test-split images, however, the decision depends on the defect region. A larger model, Clef, that comes with a post-trained Qwen3.8-27B backbone, gives no useful map in the decision head either. We found that it's possible to recover anomaly location by asking question specifically on an image region, retraining the head or reading the backbone, but none matches dedicated few-shot localizers.
| Comments: | 12 pages, 5 figures, 14 tables. Preprint of the technical part of an MSc thesis (Politecnico di Milano). Code and pre-registrations are held privately until venue submission |
|---|---|
| Subjects: | Computer Vision and Pattern Recognition (cs.CV) ; Machine Learning (cs.LG) |
| ACM classes: | I.2.10; I.4.8; I.5.4 |
| Cite as: | arXiv:2611.21995 [cs.CV] |
| (or arXiv:2611.21995v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.21995 arXiv-issued DOI via DataCite (pending registration) |
Submission history
Access Paper:
Additional Features
- Audio Summary
Current browse context:
References & Citations
- Google Scholar
- Semantic Scholar
BibTeX formatted citation
Bookmark
Bibliographic and Citation Tools
Code, Data and Media Associated with this Article
Demos
Recommenders and Search Tools
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.