Paper

OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction

Uses proactive generation as both memory writing and response control: a recent visual window is augmented by generated captions and summaries, with selective timing supervision.

arXiv ↗Oct 1, 2026Code ↗

  • cv
  • multimodal
  • video understanding
  • memory
  • online inference
Labeled architecture diagram of OneStreamer showing the method pipeline and core design
Labeled architecture diagram of OneStreamer showing the method pipeline and core design

What it is

A streaming video LLM that proactively writes reusable caption memory and times responses in one causal generation process. It pairs recent visual context with generated text memory for online QA. Instead of keeping long visual history or only reasoning traces, it writes time-grounded caption memory and answers through the same proactive decoder. It combines a recent visual window with generated local captions and event summaries, plus selective state-token supervision for response timing.

Key results

  • Best on all 8 benchmarks; +25.0% avg relative gain vs Qwen3-VL
  • OVOBench overall 72.1 vs 65.3 for the strongest listed competitor
  • PSTL beats dense supervision while using only 27.5% of state tokens

Shows that generated textual memory can preserve past evidence for streaming QA without hurting real-time perception.

One useful ML email a week.

Worth your time, not all of your time.

We track new ML work across papers, code, and data, then keep only the pieces worth a closer look. Selected, not collected.

Papers.Data.Code

The Weekly ML Signal

Selected, not collected.

One useful ML email a week: the papers, repos and datasets worth your attention.

Worth your time, not all of your time.