Context Has a Price: Adding and Removing Context for LLM Coding Agents under Prefix Caching
Summary
- A study of over 20,000 runs on SWE-bench and Loc-Bench investigates how adding and removing context affects the cost of LLM coding agents under prefix caching.
- Retrieved code reduced search commands by 18-34% but increased costs by 33-42% when cached input cost one-tenth of uncached input, without improving resolution.
- Removing old observations saved 19-36% without cache discounts, but increased costs by 38-116% under a tenth-price cache scheme.
- Shorter prompts do not guarantee lower expenses because coding agents may respond by executing more steps.
Why it matters
Context management strategies for LLM agents must account for both token pricing models and behavioral changes in multi-step agent execution, as shorter contexts do not automatically translate to lower costs.