CoRA-Bench: Evaluating Evidence Preservation by Context Selectors Under Token Budgets
Summary
- Researchers introduced CoRA-Bench, a benchmark designed to evaluate evidence preservation by context selectors under token budgets.
- The study compares methods like BM25, dense retrieval, reranking, and LLMLingua-2 across datasets including BABILong, LOONG Financial Spotlight, NoLiMa, and LongMemEval.
- Results show that method rankings vary significantly across different evidence types and between evidence recall and downstream reader accuracy.
Why it matters
Understanding how context selection techniques preserve critical evidence under tight token budgets is crucial for optimizing long-context LLM inference efficiency without sacrificing answer quality.