Beyond Query Similarity: Safety-Conditioned Cache Routing for LLM Response Reuse
Summary
- Researchers proposed SafeCache, a safety-aware framework for semantic response caching in large language models to prevent unsafe reuse when safety conditions change.
- SafeCache separates candidate retrieval from safety approval, formulating response reuse as a safety-conditioned routing problem.
- The framework matches cached-response conditions against new requests, computes an overall risk score, and uses a contradiction guard to prevent safety conflicts.
- It estimates matching uncertainty and calibrates routing policies to choose among direct reuse, selective verification, or regeneration.
- Experiments demonstrate that SafeCache achieves a superior safety-utility trade-off compared to existing evaluated baselines.
Why it matters
Semantic caching improves LLM inference efficiency, but traditional query similarity can cause unsafe response reuse. SafeCache provides a structured method to balance safety and utility without excessive verification overhead.