What it is
A benchmark of 108 realistic long-horizon computer-use workflows that exposes where frontier GUI agents fail on end-to-end professional tasks. Unlike short desktop benchmarks, it evaluates 108 end-to-end workflows with authentic artifacts, self-hosted stateful services, dynamic updates, simulated users, and fine-grained final-state scoring.
Key results
- Claude Opus 4.8 scores 20.6% completion and 54.8% partial at 500 steps
- GPT-5.5 plateaus near 13% completion while using under 1/5 of Opus tokens
- OSWorld 2.0 tasks take humans 1.6h median, about 48x longer than OSWorld 1.0
Current computer-use agents handle local actions but break on long-horizon hidden-state tracking, replanning, and verification.