AppSci-Bench: Can LLMs Build AI Agents and Make Them Better?
Summary
- Researchers introduced AppSci-Bench to evaluate how effectively large language models can build AI agents for businesses.
- The benchmark covers seven realistic domains including slides generation, order-to-cash, compliance review, and customer call insights.
- Testing seven frontier models revealed significant limitations, with the best model, Claude Opus 5.0, achieving a mean-minimum score of 59.1%.
- Models struggle particularly with eliciting task requirements and constructing representative internal evaluations.
Why it matters
AppSci-Bench highlights critical limitations in current frontier LLMs regarding end-to-end agent development, showing that automated agent engineering remains a major challenge for AI practitioners.