DECODE: Diagnosing Interpretability Failures in Automated Interpretation Process
Summary
- Automated interpretation of large language models using LLMs themselves creates reliability concerns due to self-reference and lack of ground truth.
- The authors introduce DECODE, a visual analytics system that reintroduces human oversight to diagnose interpretability failures in automated processes.
- DECODE decomposes the diagnostic workflow into three stages to isolate failure sources from structural defects to explanation-level misalignments.
- The system uses inter-coder consensus as a reliability proxy and a three-phase active seeding protocol to recommend high-priority instances for review.
- A case study on 16,384 Sparse Autoencoder features demonstrates that DECODE efficiently identifies and attributes failures at scale.
Why it matters
As automated tools increasingly interpret internal model representations without ground truth, systems like DECODE provide crucial human-in-the-loop validation to ensure interpretability reliability.