Playground.
Custom request
POSTHow the form works
A request is a state and one or more questions about it. Every question gets its own answer; the questions do not see each other.
State
What the questions are about: a message, a record, a case. Text sends it as you
type it. JSON sends an object or an array; use it for data with fields, such as a
customer record with the message inside. Questions can point at a field by its name in
backticks, for example `email`.
Questions
Add one with + choice, + score or + noul. Each question has:
- id: the name its answer has in the response. Unique within the request.
- type: the kind of answer, see below. Switching it keeps id and instructions.
- instructions: the question itself. Keep it short and put the rules into the criteria fields below it.
The three types
- noul: yes or no
- The answer is the probability of yes. The two fields say what counts as yes and what counts as no. They are optional, but they are the best place for a rule, for example yes: all of these hold: … and no: at least one of them fails.
- choice: one of several options
- Each row is an option: a key, which the answer returns, and an optional description of when it applies. 1 to 255 options. The answer names the most likely key and gives every key a probability. When options can overlap, say in the instructions: go through the options in order and pick the first that fits.
- score: a level on a scale
- Each row is a level, from the lowest (0) to the highest. 1 to 10 levels. The answer is the expected level: 1.8 means between 1 and 2, closer to 2. Describe each level by what it looks like, not only by a word like low or high.
The arrows move a question, an option or a level, ✕ removes it. In a choice the order only decides which letter the model sees for each option; in a score the order is the scale.
Sending and reading the answer
Run request or Ctrl+Enter. On the right each question gets a line in plain words and how sure the model is: very sure from a confidence of 0.9, fairly sure from 0.7, unsure from 0.4. Technical details shows every probability, the confidence and the raw response.
As code
Below the form, Raw JSON is the request exactly as it is sent. You can edit it there, and the form follows. curl, Python and TypeScript show the same request as code. The API tab explains every field.
The form stays in this browser until you change it or pick an example.
State
Questions 0
Request code
Output
Awaiting runAwaiting run
Blocks.
Input
Block format
One block, a JSON array, or one block per line (JSONL). Short form:
{"id", "state", "question", "type", "options", "gold"}, type is
choice (default), score or noul. A full request with
questions works too. The model comes from the Editor tab.
Batch.
Batch evaluation
File format
JSONL file: one request per line, optionally with "gold": {"question": "option" | true | 2}.
Requests run from your browser against this server; the server logs them.
Metrics
Reliability diagram
Misclassifications
API.
What the API does
You send a piece of text or data, the state, and up to 64 questions about it. For every question you get a probability for each possible answer, not generated text. One request answers all its questions at once, and the same request always gets the same answer, however busy the server is.
The Editor builds requests for you: the curl, Python and TypeScript tabs under the form show the current form as code, ready to copy.
A first request
curl /v1/systemone \
-H 'Content-Type: application/json' \
-d '{
"model": "jev-latest",
"state": "Hi, I was charged twice for my subscription this month. Please refund one of the two payments.",
"questions": {
"refund requested": {"type": "noul", "instructions": "Does the customer ask for money back?"},
"team": {"type": "choice", "instructions": "Which team should handle the message?",
"criteria": {"billing": "Payments, invoices, refunds",
"technical": "Bugs, outages, API problems",
"sales": "Plans, pricing, upgrades"}},
"urgency": {"type": "score", "instructions": "How urgent is the message?",
"criteria": ["Can wait a week", "Within two days", "Today"]}
}
}'
The answer, as this server gave it:
{
"model": "qev-gemma4-e4b-jevmix4-0.1.0",
"answers": {
"refund requested": {"type": "noul", "noul": 0.9659},
"team": {"type": "choice", "choice": "billing",
"probabilities": {"billing": 0.9998, "technical": 0.0, "sales": 0.0002},
"confidence": 0.9996},
"urgency": {"type": "score", "score": 1.502,
"legend": {"0": "Can wait a week", "1": "Within two days", "2": "Today"},
"probabilities": {"0": 0.0763, "1": 0.3454, "2": 0.5783},
"confidence": 0.2529}
},
"usage": {"input_tokens": 194, "output_tokens": 3}
}
The same in Python, with nothing but requests:
import requests
response = requests.post(
"/v1/systemone",
json={
"model": "jev-latest",
"state": "Hi, I was charged twice for my subscription this month.",
"questions": {
"refund requested": {"type": "noul", "instructions": "Does the customer ask for money back?"},
},
},
timeout=60,
)
response.raise_for_status()
for name, answer in response.json()["answers"].items():
print(name, answer)
The API follows Jev's /v1/systemone, so the TypeSafe SDKs work against it with
base_url set to . The Editor's Python and
TypeScript tabs show how.
The request
POST /v1/systemone with a JSON body. This server needs
no API key.
| Field | What it is |
|---|---|
model | "jev-latest" or "qev-latest" for the default,
qev-gemma4-e4b-jevmix4. "qev-gemma4-26b-a4b" for the larger model. The
response names the exact model and version; GET /v1/models lists what this server
holds. |
state | A string, an object or an array: what the questions are about.
Objects and arrays reach the model as JSON, keys in your order. Questions can point at a field
by its name in backticks, such as `email`. |
questions | An object, 1 to 64 entries. The key is the question's id (up to 128 characters) and comes back as the key of its answer. |
questions.<id>.type | noul, choice or
score, see below. |
questions.<id>.instructions | The question, as a string (or an object or array). May be left out when the criteria say everything. |
questions.<id>.criteria | Depends on the type, see below. |
qev | Optional settings, see the last section. |
Unknown fields at the top and in questions are ignored, as Jev does.
The three question types
noul: yes or no
"refund requested": {
"type": "noul",
"instructions": "Does the customer ask for money back?",
"criteria": {"true": "The message asks for a refund, a credit or money back",
"false": "No money is asked for, even if the customer is upset"}
}
criteria is optional and holds true, false or both: what counts
as yes and what as no. The answer is {"type": "noul", "noul": 0.93}, the probability of
yes.
choice: one of several options
"team": {
"type": "choice",
"instructions": "Which team should handle the message?",
"criteria": {"billing": "Payments, invoices, refunds",
"technical": "Bugs, outages, API problems",
"sales": null}
}
criteria maps each option's key to a description of when it applies, or to
null when the key says enough. 1 to 255 options, keys up to 256 characters. The answer
has choice, the most likely key; probabilities for every key, in your
order, summing to 1; and confidence.
score: a level on a scale
"urgency": {
"type": "score",
"instructions": "How urgent is the message?",
"criteria": ["Can wait a week", "Within two days", "Today"]
}
criteria lists the levels from 0 upwards, 1 to 10 of them. The answer has
score, the expected level (the sum of level × probability, so 1.8 lies between 1 and 2),
legend with your levels by number, probabilities per level, and
confidence.
Confidence
A number from 0 to 1. For a choice: how far the top probability rises above an even spread, (top − 1/k) / (1 − 1/k) with k options. For a score: 1 minus how far the probability lies from the most likely level, compared with a spread over the whole scale. 1 means the model puts everything on one answer.
The response
| Field | What it is |
|---|---|
model | The exact model and version that answered. |
answers | One answer per question, under the question's id, in the order of the request. Probabilities are rounded to four places. |
usage.input_tokens | The tokens the model read: the state once, plus every question with its options. |
usage.output_tokens | One per question. |
Headers: x-request-id, the same id as in an error; x-qev-latency-ms, the
server's time for the request; x-qev-cache, hit when the state came from the
cache.
Errors
{"error": {"type": "validation_error",
"message": "questions must have between 1 and 64 entries, got 0",
"param": "questions",
"request_id": "7412c1d0-2fba-4755-a9fd-11fa37153237"}}
param names the field at fault, as a path such as questions.team.criteria.
| Status | Type | When |
|---|---|---|
| 413 | request_too_large | The state is over 3,584 tokens, a question with the state over 4,096, or the body over 4 MB. |
| 422 | validation_error | Something in the request does not fit the rules above. Sending it again does not help. |
| 429 | rate_limit_error | More than this server allows per minute, see
below. Wait the seconds in Retry-After. |
| 503 | not_ready | The model is still loading after a restart. |
| 529 | overloaded | The queue is full or the request took too long.
Try again after Retry-After. |
| 500 | internal_error | A fault on our side. The
request_id helps us find it. |
Limits on this server
| Requests | 120 per minute per address |
| Input tokens | 100,000 per minute per address, counted as usage.input_tokens |
| State | 3,584 tokens |
| State and one question | 4,096 tokens |
| Questions per request | 64 |
| Options per choice | 255 |
| Levels per score | 10 |
| Request body | 4 MB |
Requests up to about 1,000 tokens in all are the fastest: they run as one block. Many questions on the same state cost little extra, so ask them in one request rather than one request each.
Writing questions that get the answer you mean
- Put the rule into the criteria. The instructions ask; the yes/no descriptions, option descriptions and level descriptions decide. A rule the model has to guess from the question is the most common reason for a wrong answer.
- Give data as fields. A record such as
{"days_since_purchase": 45}is read more reliably than the same fact inside a sentence. Keep free text to what a person actually wrote. - Order overlapping options. When more than one option could fit, say in the instructions that the first one that fits wins, and list them in that order.
- Separate what you judge. If tone and severity are different questions, say in each which one it is about: judge only the tone of the message, not how serious the problem is.
- Try variants. The examples in the Editor have buttons that change one thing at a time; do the same with your own case before you rely on it.
Optional settings: qev
"qev": {"cache": false, "temperature": 1.0, "debias": "reverse", "return_logits": true}
| Field | What it does |
|---|---|
cache | false computes the state anew even if the server has
it, for timing. |
temperature | 0.05 to 10. Below 1 the probabilities become sharper, above 1 flatter. The default is 1. |
debias | reverse asks each choice and score a second time with
the options in reverse order, rotate in up to 8 rotated orders, and averages. Costs the
extra tokens. The default is none. |
return_logits | true adds the model's raw scores per option. |
With a qev field in the request, the response gets one too, with the server's timing and
whether the cache was used. Other fields in qev are a 422.
Other endpoints
GET /v1/models | The model names this server accepts and what they map to. |
GET /health | 200 when the model is ready, 503 while it loads. |
GET /status | Machine, load and throughput, as shown at the top of this page. |
Gev.
What this is
Gev answers questions about text and data with probabilities. You give it a state, such as an email, a support ticket, or a policy together with a case, and ask about it: which team should handle it, how urgent it is, whether it is spam. For every possible answer it returns how likely the model finds it. It does not write text: an answer is a number your code can act on, to route the ticket, hold the post or page the engineer.
Two models run side by side on a single Mac mini with an M4 Pro and 48 GB of memory; the
Model menu at the top switches between them. The API is compatible with Jev's
systemone interface; the API tab explains it.
Two models
E4B-JEVMIX4, the default, is Google's Gemma 4 E4B, retrained for decisions. It answers
qev-latest and jev-latest. 26B-A4B is Google's Gemma 4 26B-A4B,
a mixture-of-experts model with 26 billion parameters, of which about 4 billion work on each
token.
| E4B-JEVMIX4 | 26B-A4B | |
|---|---|---|
| Requests per second, 8 at a time | 4.54 | 2.52 |
| One request alone, median | 604 ms | 757 ms |
| Correct decisions, Jev Arena (7,671) | 88.8 % | 88.2 % |
| Memory | about 15 GB | about 17 GB |
The E4B handles nearly twice as many requests and is the better choice for the kind of decision it was trained on: rules with limits and dates, levels on a scale, routing and SOC alerts. The 26B knows more of the world. Take it for content moderation and for questions far from everyday business cases.
The E4B: 4.5 times the throughput of llama.cpp
The same comparison for the E4B, on all 316 example requests of this playground, without the state cache. llama.cpp ran Google's E4B weights, Gev the trained ones; both are the same architecture and do the same amount of work.
| llama.cpp | Gev | |
|---|---|---|
| Requests per second, 8 at a time | 1.00 | 4.54 |
| Requests per second, 16 at a time | 1.02 | 4.54 |
| Answer time at 8 at a time, median | 7.7 s | 1.7 s |
| One request alone, median | 874 ms | 604 ms |
The 26B: 2.7 times the throughput of llama.cpp
The usual way to run the 26B-A4B on a Mac is llama.cpp, which computes everything on the GPU. We measured both on the same Mac, with the same model, on the examples of this playground (570 to 850 tokens per request, eight requests at a time):
| llama.cpp | Gev | |
|---|---|---|
| Requests per second, 8 at a time | 1.10 | 2.94 |
| Tokens per second, 8 at a time | 776 | 2,083 |
| One request alone, mean of 8 examples | 912 ms | 689 ms |
| Quality, sysone benchmark | 0.891 | 0.885 |
| Quality, JevBench | 0.866 | 0.866 |
llama.cpp does not get faster with more at once: with 16 requests in parallel it reached 1.14 per second. The quality is close: both read the same model, Gev computes it on other units with other rounding, and that tips a few answers that sit right between two options, some one way, some the other.
How
An Apple chip has three units that can do this kind of math: the GPU, the Neural Engine and the matrix units in the CPU. llama.cpp uses one of them. Gev runs on Mev, an engine we wrote for these models, and uses all three at the same time. For the 26B:
- The fixed work on the Neural Engine. About half of every layer is the same computation for every input: attention, the shared feed-forward part and the router that picks the experts. The Neural Engine is built for exactly that, but it only runs programs of a fixed shape, so each layer is compiled ahead of time for nine block sizes from 512 to 1,024 tokens.
- The experts on the GPU. For every token the router picks 8 of 128 experts, and which ones depends on the text. That part runs on the GPU, in our own Metal kernels: they sort the tokens by expert and compute each expert once for all the tokens that chose it.
- The most used experts on the CPU. The CPU's matrix units take the 8 experts chosen most often in each layer, at the same time as the GPU computes the rest.
- All three at once. While the GPU works on the experts of one block, the Neural Engine already computes the next layer of another. Up to eight blocks are in flight.
- No copying. On Apple silicon the three units share one memory. Each writes its results where the next one reads them.
- Read the answer, do not write it. A question needs no generated reply, only the probability of each answer at one position. The questions of a request share its state: the state is computed once, and every question branches off it.
The E4B has no experts. Its first 14 layers run on the Neural Engine, the next 10 on the GPU. Its last 18 of 42 layers take their keys and values from earlier layers instead of computing their own, so nothing reads their results at the other positions: there, Mev computes only the positions an answer is read from, a handful instead of hundreds.
The same request gets the same answer
Gev gives the same answer to the same request every time, however many other people use it at that moment. That took work: which unit computed which part, where a request sat in a block, and which other requests shared its GPU call all used to change the last digits of a result, and sometimes the answer. Now each request is computed the same way, whatever the load.
Where it runs
A Mac mini with an Apple M4 Pro: 14 CPU cores, 20 GPU cores, a 16-core Neural Engine, 48 GB of memory, macOS 27. Under full load it draws up to about 120 W, idle 2 to 3 W. The bars at the top of this page show the load, the throughput and how many people are here right now.
Run it yourself
Gev is open source under the Apache License 2.0. The code, the scripts and the instructions are at github.com/hilman2/gev; the models and the ready-built engines are on Hugging Face:
| Repository | What it holds |
|---|---|
| hilman2/gev-e4b | GEV-E4B, the model this page calls E4B-JEVMIX4: the weights, and GGUF files for llama.cpp (Q8_0, Q4_0) |
| hilman2/gev-e4b-mev | GEV-E4B as an engine for Mev, built for the M4 |
| hilman2/gemma-4-26b-a4b-mev | Google's 26B-A4B, unchanged, as an engine for Mev, built for the M4 |
On a Mac with an M4
The setup the numbers on this page come from. You need macOS 27, Xcode 27, uv and jq. Then, in a terminal:
xcodebuild -downloadComponent MetalToolchain git clone --recursive https://github.com/hilman2/gev && cd gev zsh scripts/setup-mac.sh # checks the Mac, two Python environments zsh scripts/build-server.sh # builds the server with Mev zsh scripts/fetch-engine.sh gev-e4b # the E4B engine, about 15 GB zsh scripts/serve.sh # http://127.0.0.1:8000/playground/
The first start compiles the Neural Engine programs once, about four minutes; later starts take
seconds. For the 26B as well, run zsh scripts/fetch-engine.sh gemma-4-26b-a4b (about
24 GB) and set GEV_MODELS="gev-e4b gemma-4-26b-a4b" in gev.env; both
together need 48 GB of memory. zsh scripts/install-service.sh keeps the server running
as a login service.
On a PC with an NVIDIA GPU
Here Gev runs on llama.cpp instead of Mev. You need Docker with the NVIDIA Container Toolkit; tested on an RTX 4080 with 16 GB.
git clone --recursive https://github.com/hilman2/gev && cd gev docker compose up # http://localhost:8000/playground/
On Windows, clone with git clone -c core.longpaths=true --recursive ...: some paths in
llama.cpp are longer than Windows allows by default. The first start builds the image and downloads GEV-E4B (Q8_0, about 8 GB). The image is compiled for
RTX 40xx cards; for a 30xx start it with CUDA_ARCH=86 docker compose up --build, for a
50xx with CUDA_ARCH=120.
Every request on the API tab works unchanged against your own server: replace this page's address with yours.