Dataset

OSWorkerBench

Contains 100 office desktop tasks with task configs and evaluators, plus 3,989 captioned GUI action steps for 33 demonstrated tasks.

Hugging Face ↗Sep 1, 2026Paper ↗Code ↗

  • multimodal
  • gui-agent
  • desktop-automation
  • demonstrations
Headline benchmark comparison showing UI-Mate-27B beats baselines across multiple CUA benchmarks
Headline benchmark comparison showing UI-Mate-27B beats baselines across multiple CUA benchmarks

What it is

Office-centric GUI-agent benchmark with 100 long-horizon desktop tasks, executable evaluators, and released step-level demonstrations for demo-guided evaluation. 100 long-horizon office desktop tasks across 17 business tracks and 41 applications, plus 33 self-demonstrations with 212 subtasks and 3,989 captioned GUI steps.

Key results

  • 100 test tasks across 17 business tracks and 41 apps
  • 33 self-demonstrations with 212 subtasks and 3,989 steps
  • Apache-2.0; Parquet tables plus mirrored runtime cache assets

Useful for benchmarking long-horizon computer-use agents on executable office workflows with or without demos.

One useful ML email a week.

Worth your time, not all of your time.

We track new ML work across papers, code, and data, then keep only the pieces worth a closer look. Selected, not collected.

Papers.Data.Code

The Weekly ML Signal

Selected, not collected.

One useful ML email a week: the papers, repos and datasets worth your attention.

Worth your time, not all of your time.