Repo

jeeves

Answers structured decision questions with optional reasoning, then outputs calibrated probabilities via a Jev-compatible API and SDK.

GitHub ↗Sep 29, 2026Paper ↗

  • llm
  • calibration
  • speculative-decoding
  • classification
Model architecture showing AR/Diffusion dual-head design with weight initialization scheme
Model architecture showing AR/Diffusion dual-head design with weight initialization scheme

What it is

Reasoning-enhanced Jev-style decision model that improves classification accuracy and calibration, with Jev-compatible serving and speculative decoding. Serve a Jev-compatible classifier that reasons before answering yes/no, multiple-choice, and rating questions, then returns calibrated probabilities from a single 9B model.

Key results

  • Scores 0.889 test accuracy vs 0.857 for Jev and 0.822 for Kev-9B
  • Reaches 0.935 on JevBench overall and 0.865 on JevBench hard
  • Runs in about 0.3 s without thinking; 3.3 s median with thinking on 1 H100

Shows that adding explicit reasoning to Jev-style decision models can materially improve accuracy and calibration on structured classification.

One useful ML email a week.

Worth your time, not all of your time.

We track new ML work across papers, code, and data, then keep only the pieces worth a closer look. Selected, not collected.

Papers.Data.Code

The Weekly ML Signal

Selected, not collected.

One useful ML email a week: the papers, repos and datasets worth your attention.

Worth your time, not all of your time.