Papers.Data.Code
Paper

Repo-To-Skill: Distilling GitHub Repositories Into AI4AI Skills

Jianlyu Chen, Yuyang Hu, Hongjin Qian, Jiawei Liu, Wenqing Wei, Xiaolong Chen, Defu Lian, Zhicheng Dou

Adds verified skill graphs distilled from repos or task-found sources as operating context, so agents reuse ML know-how instead of rediscovering it during execution.

arXiv ↗Sep 2, 2026Code ↗

  • llm
  • nlp
  • ai agents
  • knowledge distillation
  • tool use
Four-stage pipeline diagram showing the method's core workflow
Four-stage pipeline diagram showing the method's core workflow

What it is

Builds a skill-powered research agent that distills ML repos and task-specific sources into verified reusable skills, then uses them to improve autonomous ML research. Adds an explicit operational-knowledge layer beyond model+harness: distill repos or task-found sources into verified, reusable skill graphs the agent loads on demand instead of relearning by trial and error.

Key results

  • MLE-bench: 1.531 vs 0.653, a 134.3% gain over no-skills agent
  • PaperBench: 39.59 vs 29.45, a 34.4% gain over no-skills agent
  • PassNet: 77.14 vs 70.63, a 14.0% gain over no-skills agent

Shows that compact, verified operational knowledge is a distinct missing layer for research agents, separate from stronger models or better harnesses.

One useful ML email a week.

More from Aug 31 – Sep 06

The whole week →

Picked Sep 4

Picked Sep 3

Worth your time, not all of your time.

We track new ML work across papers, code, and data, then keep only the pieces worth a closer look. Selected, not collected.

Papers.Data.Code

The Weekly ML Signal

Selected, not collected.

One useful ML email a week: the papers, repos and datasets worth your attention.

Worth your time, not all of your time.