RoboBuilder: Agentic Robot Co-Design with Real-world Video Feedback in the Loop

Impact 7 arXiv cs.AI

Summary

  • Researchers introduced RoboBuilder, a benchmark for coding agents to co-design a robot's hardware and program using standard parts, motors, and a custom 3D-printed part.
  • The evaluation involves physical testing on a real course, where agents use video feedback and measurements to refine designs and correct simulators.
  • Across tests with four frontier coding agents, 16,427 simulations and 135 physical robots were built and evaluated.
  • Agents successfully solved nearly all easy obstacles, though they faced challenges at harder levels and in the physical world where 66% of simulated passes held.

Why it matters

RoboBuilder measures progress toward AI agents capable of engineering physical machines, demonstrating how real-world video feedback can improve hardware and software co-design.

Who should care