Lessons from scaling teams of coding agents: reach the frontier at 30% of the cost
Summary
- Teams of coding agents unlock inference-time scaling, outperforming single agents by up to 10 percentage points on ProgramBench at 3-7x higher cost.
- Strategic agent allocation matters more than raw count; delegating only exploration and planning matches full-team performance at half the cost.
- A strong lead model paired with cheaper worker models achieves near-frontier performance at a 70% cost reduction.
- Asynchronous leads reduce wall-clock time compared to blocking leads, though prompt design significantly impacts task efficiency.
- Black-box program benchmarks show planning workers scale effectively by running parallel probes and feeding structured insights back to the lead.
Why it matters
Understanding how to structure multi-agent coding teams allows engineering organizations to optimize inference budgets, balancing high performance with cost and wall-clock execution time.