kev vs needle
Open reproduction of TypeSafe's Jev: a LoRA + readout head on Qwen that answers many typed questions about one document in a single prefill pass, returning calibrated probabilities. — versus — A 45M-parameter tool-calling model shipped as one 14MB binary that runs a full session in ~28MB RAM — grammar-constrained JSON, calibrated confidence, tool retrieval, LoRA fine-tuning.
Same bet from two angles: replace a prompted frontier model with a tiny purpose-built one. needle is 45M params for grammar-constrained tool calls, kev is a LoRA+head on Qwen for typed decisions with probabilities.
| kev | needle | |
|---|---|---|
| Stars | 492 | 12k |
| Forks | 30 | 734 |
| Language | Python | Python |
| License | Apache-2.0 | Apache-2.0 |
| Last activity | today | today |
| Topics | training, local | local, agents, training |
| Curated connections | 8 | 4 |
kev — the curator's take
Reach for kev when the task is decisions, not prose: route this ticket, score this risk, answer 30 yes/no questions about one document — and you want a number you can threshold on rather than text you have to parse. The head is trained with cross-entropy on labelled outcomes, so the probabilities mean something (kev-8b: Brier 0.34, 8% confident errors out of domain), and a block-causal mask keeps questions from seeing each other, verified to 4e-6 against separate requests. It speaks TypeSafe's System One API, so their SDK works against localhost with a base_url change. Not a general chat or agent model - it never decodes, and it only answers the question types you define (noul/choice/score). Don't use it zero-shot on your own domain either: the value is in training the head on your labels, and the three 0.6B/4B/8B checkpoints are explicitly previews that failed the author's own release screen on held-out rule reasoning. Real Jev still wins out of domain (0.86 vs 0.77) - kev's pitch is that it runs on your laptop and you own the weights.
needle — the curator's take
The right tool when the agent is a phone, wearable or robot and the job is picking a function and filling its arguments — not reasoning. Three design choices make it usable rather than a demo: byte-level grammar compiled from your schema, so output is always valid JSON; a calibrated confidence head you threshold to escalate to a big model; and a retrieval head that renders only the top five tools per turn, so a large catalogue still fits the 256-token window. Fine-tuning is LoRA on a frozen base merged into a single `.cact`, so a tuned model keeps the same one-file deployment. NOT a chat model and NOT a reasoner — there is no room for multi-step planning, and writing good tool descriptions is most of the accuracy.