What it is
Builds robot policies on a frozen promptable 3D grounding model, turning language/point/box prompts into object-centric visual+metric inputs for action prediction. Instead of relying on VLA/WAM backbones to learn grounding from demos, it uses a frozen 3D grounding model to explicitly localize task objects and feed object-centric visual features plus 3D geometry into the action policy.
Key results
- RoboTwin 2.0: 55.3% avg over 50 tasks vs 52.0% Spatial Forcing
- LIBERO-PRO: 61% avg over 16 perturbations vs 53% for π0.5
- Real YAM visual shift: 17/20 successes vs 4/20 for π0.5
Explicit object-centric 3D grounding improves manipulation robustness when targets move, are re-designated, or scenes shift visually.