Newshot

BenchAudit – Auditing LLM benchmarks for leaks, data reuse, and harness gaps

Original text(via HN AI)

Showing 0 benchmarks

No benchmarks match the current filters

Try clearing one or more Domain, Skill, Tier, or Search filters.

Insights

Cross-study findings connecting benchmark integrity scores with live official model leaderboards, and more

1. Fortified Leaders vs. Exposed Over-Performers

Compares current-generation models across actively updated Tier S held-out evaluations and active Tier C & D benchmarks. Frozen legacy leaderboards missing recent frontier models are excluded to prevent cohort bias.

2. Temporal Half-Life: Age vs. Integrity Score

Each shape is one audited benchmark: left to right shows how many months old it is since its last major update, and bottom to top shows its integrity score (0–100). Circles are private held-out tests that stay trustworthy with age; diamonds are public or easy-to-access datasets that tend to leak and lose value over time. Hover or click any shape to inspect it.

3. Domain & Category Contamination Landscape

Distribution of Tiers S–D and average scores across fields. Click any bar to filter the table above.

4. Access Barrier Effectiveness

Groups benchmarks by how strictly their questions and answers are protected—from locked private servers at the top to open public downloads at the bottom. Each bar shows that group's average integrity score (out of 100), its lowest-to-highest range, and clickable benchmark examples.

The text above was extracted automatically from the source page and may miss images, tables or code blocks. The original is authoritative.

Published by HN AI
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.
Read the original ↗