Paper

Beyond Data Scaling: Representation-Centric Continued Pre-training for Vision-Language-Action Models

Senqiao Yang, Chengyao Wang, Yuxin Chen, Zixuan Wang, Longxiang Tang, Haokun Gui, Jinhui Ye, Changsheng Lu

A representation-centric VLA pre-training recipe that preserves VLM features and avoids single-head lock-in via multi-head action supervision and partial action-space unification.

arXiv ↗Aug 27, 2026Code ↗

robotics
cv
vision-language-action models
robot learning
transfer learning

Comprehensive pipeline diagram showing data flow, model forwarding, training, and deployment

What it is

Continued pre-training for VLA backbones that preserves VLM priors and learns head-agnostic, cross-embodiment action representations from limited robot data. Instead of scaling trajectories or fitting one policy head, it shapes the VLM backbone for transfer: preserve VLM priors, co-train with multiple continuous action heads, and partially unify action spaces across embodiments.

Key results

RoboCasa-GR1: 49.5% with 20% data vs GR00T-N1.6 full-data 47.6%
RoboTwin 2.0: 92.5% success, beating ABot-M0 and LingBot-VLA
LIBERO-Plus: 82.6% success; backbone-only gains are 7.6-21.4 pts

Shows backbone representation quality is an independent lever for VLA performance and transfer beyond simply collecting more robot data.

Worth your time, not all of your time.

We track new ML work across papers, code, and data, then keep only the pieces worth a closer look. Selected, not collected.

What it is

Key results

One useful ML email a week.

More from Aug 31 – Sep 06

Worth your time, not all of your time.