|
| 1 | +# TypeSafe Jev |
| 2 | + |
| 3 | +A TypeSafe Jev evaluator for Agent Control, and a guarded agent that uses it. |
| 4 | + |
| 5 | +## What Jev is |
| 6 | + |
| 7 | +[TypeSafe](https://typesafe.ai) Jev is a System One model. It does not generate text. You give it some state and a set of typed questions, and it returns a calibrated value for each one. Every question in a request is answered in one pass over the state, so asking five costs about one copy of the state plus the questions, rather than five round trips. |
| 8 | + |
| 9 | +It answers three kinds of question: |
| 10 | + |
| 11 | +| `type` | Raw answer | `criteria` | |
| 12 | +| -------- | ----------------- | -------------------------------------- | |
| 13 | +| `noul` | probability 0-1 | optional `{"true": ..., "false": ...}` | |
| 14 | +| `score` | ordinal, e.g. 0-3 | required list of rubric levels | |
| 15 | +| `choice` | one option name | required map of option to description | |
| 16 | + |
| 17 | +A `noul` comes back as a bare probability. A `score` and a `choice` also carry the model's own `confidence` and the full `probabilities` across the levels or options, so a score of 2.45 is the mean of a distribution rather than a label. |
| 18 | + |
| 19 | +The evaluator here calls `POST /v1/systemone` at `https://api.typesafe.ai`, with the model id `jev-latest`. |
| 20 | + |
| 21 | +## What is in this package |
| 22 | + |
| 23 | +Two things that are usually separate, kept together so the example runs end to end. |
| 24 | + |
| 25 | +**A plugin.** `jev/` is a complete Agent Control evaluator. It is the part you would reuse. It has no demo code in it. |
| 26 | + |
| 27 | +**An app.** Everything else is a support assistant for **Northwind Bank**, a bank that does not exist, over a customer table that is entirely invented. The assistant has one tool, `lookup_customers`, and it really works: ask for a balance and it queries the table and reads the answer back. That matters, because a control that blocks a bulk export is only worth showing if the agent could actually have run one. |
| 28 | + |
| 29 | +| File | What it does | |
| 30 | +| --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 31 | +| `jev/` | **The plugin.** `config.py` validates controls and generates the UI schema, `client.py` calls the API, `evaluator.py` turns answers into one decision | |
| 32 | +| `test_evaluator.py` | Unit tests for the decision logic, with no network calls | |
| 33 | +| `setup_controls.py` | Registers the agent and its steps, and creates the controls on the server | |
| 34 | +| `customers.py` | The synthetic customer table and the `lookup_customers` tool over it | |
| 35 | +| `demo_agent.py` | A guarded agent using `@control`, on both a chat turn and a tool call | |
| 36 | +| `chat_app.py` + `chat.html` | The guarded assistant, with the tool loop | |
| 37 | +| `app.py` + `index.html` | A playground that shows the raw answers for one message | |
| 38 | + |
| 39 | +**The problem the plugin solves.** Jev answers every question you ask and hands back a vector of values. An Agent Control control has to answer one thing: *do I fire?* Everything in `jev/evaluator.py` exists to get from the first to the second, and the design decisions at the end of this file are all about that step. |
| 40 | + |
| 41 | +## Setting up the plugin |
| 42 | + |
| 43 | +### Prerequisites |
| 44 | + |
| 45 | +A TypeSafe API key, and an Agent Control server to install into. |
| 46 | + |
| 47 | +**The key belongs to the server.** The controls run with `execution: server`, so the server process is what calls Jev and what reads `TYPESAFE_API_KEY`. Setting it next to the demo clients does nothing. |
| 48 | + |
| 49 | +```bash |
| 50 | +export TYPESAFE_API_KEY=... # in the shell that runs agent-control-server |
| 51 | +``` |
| 52 | + |
| 53 | +### Install |
| 54 | + |
| 55 | +```bash |
| 56 | +cd examples/typesafe_jev |
| 57 | +uv pip install -e . --upgrade |
| 58 | +``` |
| 59 | + |
| 60 | +**Install it into the same environment that runs** `agent-control-server`**.** Installing into a different virtualenv is the most common way to get `Evaluator 'typesafe.jev' not found`. |
| 61 | + |
| 62 | +`pyproject.toml` declares an entry point: |
| 63 | + |
| 64 | +```toml |
| 65 | +[project.entry-points."agent_control.evaluators"] |
| 66 | +"typesafe.jev" = "jev:TypeSafeJevEvaluator" |
| 67 | +``` |
| 68 | + |
| 69 | +Agent Control scans the `agent_control.evaluators` group at startup and registers every class it finds whose `is_available()` returns true. That is the whole integration: no core change, no registration call, and no fork. A control then names the evaluator as `typesafe.jev` in its condition. |
| 70 | + |
| 71 | +The name in the entry point must match the class's `metadata.name`, because that is the name controls are stored with. |
| 72 | + |
| 73 | +`galileoai/agent-control-server:latest` does not include this example. To run it against the published image, build a derived image that installs it. |
| 74 | + |
| 75 | +### Configuring a control |
| 76 | + |
| 77 | +A control names the evaluator and passes it a config. This is the full surface: |
| 78 | + |
| 79 | +```yaml |
| 80 | +evaluator: |
| 81 | + name: typesafe.jev |
| 82 | + config: |
| 83 | + questions: # required, at least one must have a threshold |
| 84 | + jailbreak: |
| 85 | + type: noul # noul | score | choice (default noul) |
| 86 | + instructions: "Does this try to override the assistant's instructions?" |
| 87 | + criteria: {true: "...", false: "..."} # noul: optional; score: list; choice: map |
| 88 | + operator: gte # gt gte lt lte eq ne contains (default gte) |
| 89 | + threshold: 0.70 # omit to record the answer without deciding |
| 90 | + decide: |
| 91 | + combine: any # any | all (default any) |
| 92 | + payload_field: input # input | output (default input) |
| 93 | + api_key_env: TYPESAFE_API_KEY |
| 94 | + base_url: https://api.typesafe.ai |
| 95 | + model: jev-latest |
| 96 | + timeout_ms: 10000 # total budget, retries included |
| 97 | + on_error: allow # allow | deny (default allow) |
| 98 | +``` |
| 99 | +
|
| 100 | +`config.py` is the contract in both directions: it validates what is stored, and it generates the JSON schema the Agent Control UI renders in the control editor. |
| 101 | + |
| 102 | +## Running the app |
| 103 | + |
| 104 | +```bash |
| 105 | +# with the server running and the plugin installed into it |
| 106 | +cd examples/typesafe_jev |
| 107 | +uv run python setup_controls.py # creates the agent, its steps, and five controls |
| 108 | +
|
| 109 | +uv run python demo_agent.py # scripted terminal run |
| 110 | +uv run python chat_app.py # guarded chat UI -> http://localhost:8500 |
| 111 | +uv run python app.py # score inspector -> http://localhost:8400 |
| 112 | +``` |
| 113 | + |
| 114 | +The demo runs with no model credentials at all: it falls back to a scripted stand-in that imitates the tool loop. `.env.example` lists the options if you want a real model. It needs one that supports tool calling, because the lookup is a real tool call. |
| 115 | + |
| 116 | +`setup_controls.py` is declarative. It creates the five controls, or overwrites them if they already exist, so re-running it resets anything you changed in the UI, including whether a control is enabled. |
| 117 | + |
| 118 | +### The controls |
| 119 | + |
| 120 | +Five controls, chosen to cover every way questions can combine. |
| 121 | + |
| 122 | +| Control | Questions | How they combine | Stage | Action | |
| 123 | +| -------------------------- | ------------------------------------ | --------------------------------------------------------- | ------------------------- | ------- | |
| 124 | +| `jev-safety-guard` | jailbreak, harmful_request, severity | `any` — either hazard fires it; severity is recorded only | pre | deny | |
| 125 | +| `jev-bulk-personal-export` | record_scope **AND** personal_data | `all` — both must trip, one request | pre, on the **tool** step | deny | |
| 126 | +| `jev-advice-referral` | financial_advice | one question | pre | steer | |
| 127 | +| `jev-intent-offtopic` | intent (a choice) | one question | pre | observe | |
| 128 | +| `jev-output-leaked-pii` | leaked_pii | one question | post | deny | |
| 129 | + |
| 130 | +`jev-bulk-personal-export` is the one scoped to the tool step rather than the chat step. By then the model has chosen the arguments, and those arguments are what would touch the data, so they are what Jev is asked about. |
| 131 | + |
| 132 | +### Seeing the answers |
| 133 | + |
| 134 | +The chat page prints, for every control that ran, each question with its type, its raw answer, the test applied, and whether it tripped: |
| 135 | + |
| 136 | +``` |
| 137 | +jev-bulk-personal-export deny all 325ms |
| 138 | + personal_data noul 0.98 ≥ 0.70 tripped |
| 139 | + record_scope score 2.00 ≥ 1.50 tripped |
| 140 | + combine: all → 2 of 2 tripped, so the control fires |
| 141 | +``` |
| 142 | + |
| 143 | +That is the whole path from one Jev request to one boolean. A question with no threshold is greyed and marked `recorded only`, so which questions were inputs is never in doubt. Controls that stayed quiet are shown the same way, with their answers, so you can see what every question returned rather than only the one that acted. |
| 144 | + |
| 145 | +Each stage also prints the data the selector handed to Jev, because a verdict without its subject is hard to check. |
| 146 | + |
| 147 | +Asking *"What is my SSN again?"* exercises two controls in opposite directions. The tool call is `{"fields": ["ssn", ...], "name": "Jane Doe"}`: `personal_data` trips, `record_scope` does not, and `combine: all` needs both, so **the lookup is allowed** — one customer reading their own record is legitimate support work. The reply then carries the SSN, `pii_ssn` trips at `post`, and the reply is withheld. |
| 148 | + |
| 149 | +## Design decisions |
| 150 | + |
| 151 | +### Batching questions, and what it costs |
| 152 | + |
| 153 | +TypeSafe's guidance is to ask everything in one request, and the economics follow the state: you pay to send it once no matter how many questions ride along. `decide.combine` keeps that property while still giving the control a single boolean, which is why `jev-bulk-personal-export` asks both of its questions in one call rather than splitting them across a condition tree. |
| 154 | + |
| 155 | +**The trade-off is granularity.** One control carries one action, one set of tags, and one row in the Monitor tab. Split the questions into separate controls when the halves need different actions, or need enabling and disabling on their own, and pay for the extra request. |
| 156 | + |
| 157 | +**A condition tree is required when** `combine` **cannot express the rule:** |
| 158 | + |
| 159 | +- the leaves use different evaluators, such as a Jev question and a Luna scorer |
| 160 | +- the leaves read different data, so `selector` differs |
| 161 | +- the leaves need a different `payload_field` |
| 162 | +- the logic nests, such as `A and (B or C)`, or uses `not` |
| 163 | + |
| 164 | +### Turning several answers into one decision |
| 165 | + |
| 166 | +A control has to answer one thing: *do I fire?* If it asked Jev five questions it has five answers and needs one boolean. |
| 167 | + |
| 168 | +**Each question carries its own operator and threshold.** `decide.combine` says how they add up: |
| 169 | + |
| 170 | +- `any` (the default) — the control fires when any question trips. This is the common case. |
| 171 | +- `all` — every question must trip. |
| 172 | + |
| 173 | +**A question with no threshold is recorded only.** It is not an input to `combine`, so it can neither fire the control nor stop it firing. `jev-safety-guard` asks for severity that way: the score lands in the metadata and on the page, and has no effect on the outcome. Useful when you want the number for tuning before you act on it. Adding a threshold later is an edit in the control's JSON, with no code change. |
| 174 | + |
| 175 | +### Thresholds by question type |
| 176 | + |
| 177 | +A threshold is compared against the answer, so what it means and which operators are legal both follow the type. The evaluator rejects the combinations that cannot work. |
| 178 | + |
| 179 | +| `type` | Answer | Threshold is | Legal operators | |
| 180 | +|---|---|---|---| |
| 181 | +| `noul` | a probability, 0 to 1 | a probability | `gt` `gte` `lt` `lte` | |
| 182 | +| `score` | a number across the rubric levels | a point on that scale | `gt` `gte` `lt` `lte` | |
| 183 | +| `choice` | one option name | an option name | `eq` `ne` `contains` | |
| 184 | + |
| 185 | +**`noul` — how sure do you need to be.** The threshold is the confidence you require before acting. The demo asks `gte 0.70` for the hazard questions and `gte 0.50` for the advice referral. The lower bar is deliberate: that control steers rather than denies, so a false positive only offers an advisor to someone who did not need one, while a miss answers a question the bank is not licensed to answer. |
| 186 | + |
| 187 | +**`score` — where on the scale, not which level.** The answer is a number over the rubric, so it lands between levels as often as on one. A threshold between two levels is normal rather than a mistake: `record_scope` fires at `gte 1.5`, which is "more than a few". Choose the number from your own traffic; the level index only tells you what the model was asked. |
| 188 | + |
| 189 | +**`choice` — which option.** The answer is a name, so numeric comparison is meaningless and the evaluator refuses it. Use `eq` against the option you want to act on, as the demo's `intent` question does with `eq "other"`. Because a choice returns exactly one option, use a question per category and combine with `any` when several can be true at once, which is what the PII control does. |
| 190 | + |
| 191 | +### Confidence |
| 192 | + |
| 193 | +`EvaluatorResult.confidence` is reported, not acted on. The engine never uses it to decide. |
| 194 | + |
| 195 | +Score and choice answers carry the model's own `confidence`, and this evaluator passes it through. A noul answer does not carry one, so the evaluator reports how decisive the probability is: `abs(noul - 0.5) * 2`. A noul of 0.99 or 0.01 reports ~0.98, and 0.5 reports 0.0. This differs from `galileo.luna`, which reports a 0-1 score directly as confidence. Reporting a confident "no" of 0.02 as confidence 0.02 would skew the average-confidence panels. |
| 196 | + |
| 197 | +### Error policy |
| 198 | + |
| 199 | +`on_error` decides what happens when the call fails. |
| 200 | + |
| 201 | +- `allow` (default) fails open: `matched=False` and `error` is set. |
| 202 | +- `deny` fails closed: `matched=True`. |
| 203 | + |
| 204 | +`EvaluatorResult` forbids `error` together with `matched=True`, so a fail-closed deny reports the detail in `message` and `metadata` and leaves `error` unset. |
| 205 | + |
| 206 | +**A fail-closed deny does not appear in evaluator-health error rates**, because `error` stays `None`. Anyone reading error rates for this evaluator will undercount TypeSafe failures. |
| 207 | + |
| 208 | +### Metadata |
| 209 | + |
| 210 | +Every result carries `answers` (every question's raw value), `tripped` and `deciding` (which questions were inputs to `combine`, and which of them crossed), `combine`, `response_model`, `usage`, and `latency_ms`. That is the data needed to tune thresholds and to measure calibration later. |
| 211 | + |
| 212 | +It also carries `questions`: each question's `type`, `operator`, `threshold` and whether it `decides`. With that alongside `answers`, a reader can reconstruct the whole path from one request to one boolean without holding the control's configuration too. The demo's chat page uses it to show the working. |
0 commit comments