Don't Pick a Winner: Knowledge Conflicts Need a Prior, Not a Rule
Summary
- A new study shows that fixed decision rules for resolving knowledge conflicts between retrieved contexts and language model memory are inherently suboptimal across diverse deployments.
- The authors propose using a decision-theoretic framework requiring independent source evidence and an explicit prior on context reliability rather than fixed heuristic rules.
- Testing on GRACE and ClashEval benchmarks, the proposed method outperforms existing baselines by utilizing separate model notes, calibrated confidence, and exact prior shift.
Why it matters
This research improves how retrieval-augmented language models handle conflicting information by replacing rigid heuristic rules with a statistically grounded prior, leading to more robust and accurate responses.