MemU-Bench: Benchmarking GUI Agents on Long-Horizon User Interactions
Summary
- Researchers introduced MemU-Bench, a mobile benchmark containing 118 vague tasks across 29 long-horizon user trajectories to evaluate GUI agents.
- The benchmark measures performance using Success Rate, Interaction Efficiency, and Success weighted by Path Length to penalize exhaustive random searching.
- Current state-of-the-art models struggle with ambiguous, history-dependent instructions, with Gemini-3.1-Pro achieving an SPL of only 16.4%.
- The authors proposed AppMem, a training-free framework that structures user history into an app-centric memory bank to retrieve missing context.
- AppMem improves GPT-5.5's SPL from 15.7% to 66.9% and provides consistent gains across multiple underlying language models.
Why it matters
MemU-Bench and AppMem address critical limitations in evaluating and improving GUI agents handling ambiguous, long-horizon user interactions, significantly advancing reliable task execution in mobile environments.