LocalJev

A local, Jev-compatible POST /v1/systemone API written in TypeScript for Bun, backed by DiffusionGemma through an OpenAI-compatible Chat Completions endpoint.

The defaults target:

• inference server: http://127.0.0.1:8000
• model: diffusiongemma-26B-A4B-it-4bit
• LocalJev API: http://127.0.0.1:8080

Why a bridge is needed

Jev uses a typed decision API rather than an OpenAI chat API. OpenJev implements the Jev wire protocol and obtains probabilities with a special one-step DiffusionGemma structured read. Its backend depends on unmerged vLLM request extensions such as diffusion_seed_canvas, diffusion_read_only, and requested token logprobs.

The normal oMLX API does not expose those primitives. LocalJev therefore takes the portable approach:

1. translate state and typed Jev questions into a classification prompt;
2. ask DiffusionGemma for a JSON probability scalar/vector;
3. validate the complete result and retry malformed output;
4. normalize vectors and calculate Jev-compatible choices, expected scores, and entropy-based confidence;
5. return the normal Jev response shape.

This is wire-compatible, but not mathematically equivalent to OpenJev's logit read. The probabilities are generated/self-reported by the model rather than read directly from its logits. Evaluate their calibration on your own workload before relying on them for consequential decisions.

Run with oMLX

Requires Bun 1.2+ and a running oMLX server.

bun install
cp .env.example .env
$EDITOR .env # replace the upstream API-key placeholder
bun run start

Bun loads .env automatically. Alternatively, set the key in your shell before starting the server:

# fish
set -gx LOCALJEV_UPSTREAM_API_KEY 'your-local-omlx-key'

# bash/zsh
export LOCALJEV_UPSTREAM_API_KEY='your-local-omlx-key'

LocalJev listens on http://127.0.0.1:8080. Check that the configured model is available:

curl http://127.0.0.1:8080/ready

Make a decision:

curl http://127.0.0.1:8080/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "jev-latest",
    "state": "Hi, I have been trying to connect Stripe but keep getting a 403 error.",
    "questions": {
      "department": {
        "type": "choice",
        "instructions": "Which team should handle this?",
        "criteria": {
          "billing": "Payment or subscription issues",
          "technical": "Bugs or integration problems",
          "sales": "Pricing or account questions"
        }
      },
      "frustration": {
        "type": "score",
        "instructions": "How frustrated does the customer appear?",
        "criteria": ["Calm", "Frustrated but civil", "Very angry"]
      },
      "urgent": {
        "type": "noul",
        "instructions": "Does this require an immediate response?"
      }
    }
  }'

Use the TypeSafe SDK

The SDK requires an API-key value. LocalJev accepts any value unless LOCALJEV_API_KEY is configured. Set the SDK environment for your shell:

# fish
set -gx TYPESAFE_BASE_URL http://127.0.0.1:8080
set -gx TYPESAFE_API_KEY local

# bash/zsh
export TYPESAFE_BASE_URL=http://127.0.0.1:8080
export TYPESAFE_API_KEY=local

from typesafe_sdk import TypeSafeClient

client = TypeSafeClient()
response = client.system_one(
    "I was charged twice this month.",
    {
        "billing": {
            "type": "noul",
            "instructions": "Is this a billing issue?",
        }
    },
)
print(response.nouls["billing"].noul)

jev-latest and jev-preview are accepted aliases so SDK defaults work unchanged.

Configuration