What it is
Video conversation model for detailed QA and dialogue over videos, plus a benchmark and 100K instruction dataset for evaluating video-chat systems. Generate multi-turn conversations and answers about uploaded videos using an LLM paired with spatiotemporal video features, with released training data and evaluation scripts.
Key results
- Releases 100K video-instruction pairs for training
- Best zero-shot QA: 64.9 MSVD-QA, 49.3 MSRVTT-QA, 51.4 TGIF-QA
- Adds quantitative benchmarking over correctness, detail, context, time, consistency
Shows that instruction-tuned LVLMs can be extended from images to useful video conversation with measurable gains.