Paper

Toward Skill-Native LLMs: Skill Entropy for Benchmarking and Training Long-Horizon Reasoning

Measures cross-skill switching difficulty, then trains on it with RL by predicting per-step skills and answers instead of answers alone.

arXiv ↗Aug 5, 2026Code ↗

  • nlp
  • llm
  • reasoning
  • reinforcement learning
  • evaluation
Composite figure showing benchmark composition, core concept, task example, and training method
Composite figure showing benchmark composition, core concept, task example, and training method

What it is

Introduces Skill Entropy to measure skill-switching difficulty in long-horizon reasoning, plus a benchmark and RL training signal that improves cross-skill performance. Instead of scoring isolated skills, measures how hard it is to switch between skills and uses that same signal for RL. The model predicts both step answers and skill labels, rewarding correctness plus alignment to the gold skill sequence.

Key results

  • Qwen3-4B-Instruct improves 34.4% -> 68.4% on Skill2-Bench
  • Qwen3-1.7B improves 14.6% -> 40.1% on Skill2-Bench
  • Cross-skill setting cuts accuracy by 4% to 13% vs single-skill evals

Long-horizon reasoning is bottlenecked by switching between skills, and supervising the skill sequence directly can substantially reduce that gap.

One useful ML email a week.

Worth your time, not all of your time.

We track new ML work across papers, code, and data, then keep only the pieces worth a closer look. Selected, not collected.

Papers.Data.Code

The Weekly ML Signal

Selected, not collected.

One useful ML email a week: the papers, repos and datasets worth your attention.

Worth your time, not all of your time.