Papers.Data.Code
Paper

LaLa-World: Scaling Latent World Models with Natural Language Actions

Junha Song, Geonmo Gu, Jeongeun Park, Heejeong Nam, Lucas Maes, Byeongho Heo, Dongyoon Han, Sangdoo Yun

Uses language as the action interface so one latent world model can train across heterogeneous control and visual data, with an LM predicting future latents.

arXiv ↗Oct 21, 2026Code ↗

  • cv
  • llm
  • robotics
  • world models
  • model predictive control
  • imitation learning
Composite with unification, model overview, and comparative benchmark results showing core claims
Composite with unification, model overview, and comparative benchmark results showing core claims

What it is

A latent world model that turns actions and visual changes into language, letting one predictor train across robot, sim, image, and video data for planning and policy init. Instead of separate action-specific LWMs or heavier generative video models, it uses language as the shared action interface. Numerical actions are templated into text, and an LM predicts future latents from mixed control, image, and video data.

Key results

  • 72.1% avg MPC success vs 64.6% JEPA-WMs and 62.3% DINO-WM
  • 50.5% avg on Rope/Granular/RoboCasa/DROID vs 44.8% JEPA-WMs
  • Comparable planning to RAEv2 baseline with 6x less train cost, 13.9x faster

Language can unify heterogeneous action spaces and broader visual data without giving up the efficiency advantages of latent world models.

One useful ML email a week.

More from Oct 26 – Nov 01

The whole week →

Picked Oct 30

Picked Oct 29

  • Dataset
    crtproduction/name-that-llm-open-pool

    Unlike most attribution datasets, it provides 254,561 de-duplicated open passages labeled across 18 LLMs with provenance and leakage flags.

    KaggleOct 28, 2026
    →
  • Repo
    google-deepmind/detectorscope

    Unlike black-box AI text detectors, it exposes named feature sums and token-level evidence while preserving accuracy above 0.7 interpretability on COLING and RAID.

    GitHubOct 24, 2026
    →

Picked Oct 28

  • Dataset
    baidu/MarathonDocBench

    Makes hundred-page parsing measurable with cross-page structure and 8 document-level metrics, exposing failures of page-wise and general VLM parsers.

    Hugging FaceOct 26, 2026
    →

Worth your time, not all of your time.

We track new ML work across papers, code, and data, then keep only the pieces worth a closer look. Selected, not collected.

Papers.Data.Code

The Weekly ML Signal

Selected, not collected.

One useful ML email a week: the papers, repos and datasets worth your attention.

Worth your time, not all of your time.