What it is
Local typed-decision engine for frozen LMs that returns direct choices, scores, or booleans with calibrated probabilities, conformal sets, and abstention. Turn a frozen local language model into a typed decision API that answers fixed-schema questions from text or images and returns calibrated probabilities, conformal sets, and abstain flags.
Key results
- Supports 73,728-token context with read-once prefix caching
- 0.8B model runs at 22 ms/decision on typed-decisions benchmark
- DecisionBench NLL 1.420 vs hosted 3.196 for the 4B model
Shows a frozen local LM can act as a typed decision system with calibrated uncertainty, not just a text generator.