Shunyam Typed Decisions
Shunyam Typed Decisions
Shunyam scores every candidate answer in a single pass instead of generating one. It reaches 79.3% on the public typed-decisions benchmark — ahead of TypeSafe Jev (72.7%) and Laya (76.6%) — at 16 ms a decision, with probabilities calibrated well enough to gate automation on. Training it on ten thousand of your own records takes about two hours on a laptop.
Scored end to end through the shipped executable over HTTP. One configuration throughout — no per-dataset thresholds, no prompt tweaks, no temperature fitted anywhere.
| Benchmark | Shunyam | Laya | Jev | Measurement condition |
|---|---|---|---|---|
| typed-decisions (2,000) | 0.793 | 0.766 | 0.727 | competitor figures as reported in the Laya write-up |
| calibration error (ECE) | 0.053 | 0.081* | 0.246 | *Laya after fitting a temperature; Shunyam fits none |
| DAIR Emotion | 0.915 | 0.595 | 0.480 | described labels; the largest margin on the board |
| AG News | 0.896 | 0.950 | 0.910 | described labels; a comparison we trail |
| Banking77 (77 intents) | 0.791 | 0.425 | 0.870 | Jev's strongest; a single-task classifier would beat all three |
Competitor figures are as reported in the Laya write-up; the three systems were not run through one harness. The typed-decisions labels agree with themselves only 64.6% of the time and the benchmark publishes a teacher self-agreement ceiling of 73.5%, so every score above that — ours included — is fitting label noise as much as signal.
Enron spam 0.956 · phishing, never trained on, 0.849 · CLINC150 with out-of-scope rejection 0.758 · a toxicity guardrail that catches three in four harmful requests where answering “no” to everything catches none.
On DecisionBench — 43 tasks, 28 domains, none of them trained on — Shunyam reaches 0.520 supported accuracy against 0.419 for a Laya trained on the same typed-decisions corpus. A 10-point margin on data neither has seen.
Describing each option in a few words rather than naming it is worth 10.9 points on AG News. Pass prose descriptions, not bare enum keys — the model reads what an option means.
A request is a record and one or more questions about it. Nothing is generated, so an answer outside the candidates you supply is impossible by construction.
A set of options, each with a short description. Returns the chosen key and a probability for every option.
An ordered list of levels, lowest first. Returns the most likely level and the expected level. Within one level of the answer 97.8% of the time.
A statement that is true or false of the record. Returns the probability it holds — the strongest of the three at 0.873.
A five-question record takes 82 ms of server time, about 16 ms a decision; a single-question request takes 26 ms. Requests arriving together are scored in one pass, and a lone request is never delayed.
A generative model writes an answer and you hope it stays inside the rules. Shunyam scores the candidates you gave it and returns a distribution over exactly those. Cost grows with the number of options, not with the number of answers considered.
Scores every supplied candidate in one pass. An answer outside the list is impossible by construction, and the probability comes from the same pass — not a second scoring step.
A single self-contained binary with no Python at inference. Runs from the command line, a JSON file, or as an HTTP server — on CPU, Apple Metal or CUDA.
Probabilities are meaningful without post-hoc temperature fitting. Enron spam 3.1% ECE, yes/no statements 2.4% — good enough to gate automation on directly.
One script turns your labelled decisions — resolved tickets, approved invoices, triaged alerts — into a model the executable loads. Reference-score checked on export, so a bad build cannot reach production quietly.
This is a different system from the decision model above, built on the same principle: everything in one file, nothing leaving the machine. A domain corpus is embedded directly into the binary and the model answers strictly from it. We handed one to Claude — an oncology knowledge base in a single 574 MB executable — and asked it to try to break the offline claim.
A model that never phones home cannot leak your queries, your patients' symptoms or your legal questions. The airgap is verified, not promised.
Works during outages, on isolated networks, in places with no connectivity at all. Decisions that do not depend on someone else's infrastructure staying up.
No vendor can deprecate it, reprice it or change its behaviour. The binary you have today answers identically in ten years.
The benchmark numbers above are a general model. The point of the design is that a model for your decisions is cheap to make: ten thousand of your own records, five questions each, is about two hours on a laptop. No datacentre GPU, no cluster time, no queue.
Every figure on this page came from a public test split scored through this endpoint. Point the same benchmark at it and check. One POST carries a record and as many questions as you like, all answered in the same pass — no SDK, no client library.