Reproducible Evaluation as the Substrate for Safe Prompt Compression in Production Agents
Summary
- Researchers present an offline evaluation harness for production conversational agents that maintains reproducibility despite mutable backend enterprise data.
- The evaluation harness serves as a fitness function for GEPA-based prompt optimization using a correctness gate to prevent reward hacking.
- Applying correctness-gated compression to a production merchant-facing agent reduced the prompt surface by 3.5% to 7.6% without task correctness regression.
Why it matters
This research provides a reliable method for evaluating and optimizing conversational agents operating on live enterprise data, solving key challenges in prompt compression and reward hacking.