Leaderboard Hacking: Preference-Based Model Evaluations are Vulnerable to Manipulation
Summary
- Arena-style rankings and preference-based evaluations of language models are vulnerable to manipulation.
- Researchers identified three attack vectors: strategic voting, strategic prompting, and strategic nomination.
- These combined manipulation strategies can boost a model's rank by up to 45 places on public leaderboards.
- The vulnerabilities impact any preference-based evaluation using the Bradley-Terry model with open prompting, voting, or submissions.
Why it matters
These findings show that popular crowd-sourced AI leaderboards can be gamed, meaning model rankings may not accurately reflect true capability. This necessitates more robust evaluation safeguards for AI developers and industry consumers.