When Lower Harm Misleads: Auditing Concept Corrections
Summary
- Researchers propose a correction audit evaluation protocol to analyze concept prediction corrections in AI models.
- The study demonstrates that lower measured harm can be misleading because harm also drops when a model responds less to corrections or starts from a worse uncorrected prediction.
- Experiments across breast-ultrasound sources and the CUB dataset reveal that objectives reducing harm can paradoxically sacrifice useful gain and increase negative log-likelihood.
- Lower harm metrics reliably indicate better model corrections only when evaluated alongside useful gain, absolute risk, and the initial baseline prediction.
Why it matters
Evaluating AI models using only harm reduction metrics can be misleading and lead to less helpful systems. This research provides a reproducible protocol to accurately audit concept corrections, improving reliability in critical domains like medical diagnosis.