You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit b1a052e
Browse filesBrowse the repository at this point in the historyBrowse files
Laya is a small model that **answers questions about a piece of text with probabilities** instead of writing text.
6
-
You give it some text and a question, and it answers in a fraction of a second:
7
+
**Run [Laya](https://github.com/NandhaKishorM/laya) typed-decision models in pure Java.** No Python, no PyTorch, no native model runtime to install.
8
+
9
+
Laya is a small model that answers questions about a piece of text with **probabilities** instead of generating text. It is fast, deterministic, and cheap to
10
+
run locally, which makes it a good fit for routing, triage, moderation, and scoring in applications.
| choice | "Which team should handle this: billing or technical?" |`billing` (92 %), `technical` (8 %) |
12
16
| score | "How frustrated is the customer? Calm / Frustrated / Very angry" |`1.09` (between "Frustrated" and "Very angry") |
13
17
14
-
This project runs the model inside your JVM and, optionally, exposes it as an HTTP endpoint that speaks
15
-
**OpenAI's Decisions API** (`/v1/decisions`) and the **Jev / System One** protocol (`/v1/systemone`).
18
+
laya-java gives you:
19
+
20
+
-**An in-process library** (`Agent.predict`), with no server and no HTTP.
21
+
-**A Spring Boot starter** that exposes the model over HTTP, compatible with **OpenAI's Decisions API** (`POST /v1/decisions`) and the **Jev / System One protocol** (`POST /v1/systemone`).
22
+
-**A JDK-only client** and a **Gradle plugin** for running a local server in development and tests.
23
+
-**Optional macOS acceleration** through Apple's Accelerate framework.
16
24
17
-
> Independent Java port. Not an official release of Convai Innovations or Laya.
18
-
> Model weights are not in the jars; they are downloaded from Hugging Face on first use (Apache-2.0).
25
+
> laya-java is an independent port. It is not an official release of Convai Innovations or the Laya project. Model weights are not bundled; they are downloaded
26
+
> from Hugging Face on first use under their own Apache-2.0 license.
19
27
20
-
## Choose how to use it
28
+
## Contents
29
+
[Install](#install) · [Use it in code](#use-it-in-code) · [Spring Boot endpoint](#spring-boot-endpoint) · [Standalone server](#standalone-server) · [Client](#client) ·
questions.put("dept", Question.choice("Which team should handle this?", List.of("billing", "technical", "sales")));
78
+
questions.put("urgent", Question.noul("Does this convey urgency?", null));
79
+
questions.put("frustration", Question.score("How frustrated is the customer?", List.of("Calm", "Frustrated", "Very angry")));
80
+
81
+
Map<String, Object> result = agent.predict("Help! My payouts have been failing for 3 days.", questions);
82
+
// result.answers.dept.choice -> "billing"
83
+
// result.answers.urgent.noul -> 0.7419
84
+
// result.answers.frustration.score -> 1.0889
41
85
}
42
86
```
43
87
44
-
### 2. Spring Boot endpoint
88
+
`Agent.load` accepts a Hugging Face repository id or a local directory. Checkpoints are cached in `~/.cache/laya` (override with `LAYA_CACHE`). Questions about the
89
+
same text are evaluated independently; the state can be plain text or a JSON-like map or list (such as a chat history).
90
+
91
+
## Spring Boot endpoint
92
+
93
+
Add the starter and your application serves the model over HTTP.
**About `model`:** Laya is the model. The field exists only because OpenAI's and Jev's request formats have it. Use `english`,
74
-
`multilingual` or `typed-decisions` to pick a checkpoint. Any other value (such as `laya` or `gpt-6-luna`) lets the server choose
75
-
English or multilingual from the language of the text.
113
+
**The `model` field.** Laya is the model. The field exists because the OpenAI and Jev request formats include it. Use `english`, `multilingual`, or `typed-decisions`
114
+
to select a checkpoint; any other value routes automatically by the language of the text.
76
115
77
-
**About the API key:**none is needed. Set `laya.api-key`only if you want to require `Authorization: Bearer <key>`.
116
+
**API key.**None is required. Set `laya.api-key` to require `Authorization: Bearer <key>` on `/v1/*`.
78
117
79
-
Properties: `laya.models.<name>` (Hugging Face repo id or local directory), `laya.default-model`, `laya.api-key`,
80
-
`laya.preload` (load at startup), `laya.endpoint.enabled`, `laya.endpoint.base-path`.
|`laya.models.<name>`| Hugging Face repo id or local directory for a model name |`english`, `multilingual`, `typed-decisions` (the published checkpoints) |
121
+
|`laya.default-model`| Model used when none is matched |`english`|
122
+
|`laya.api-key`| Bearer token required on `/v1/*`| none |
123
+
|`laya.preload`| Load the default model at startup |`false`|
124
+
|`laya.endpoint.enabled`| Expose the endpoints |`true`|
125
+
|`laya.endpoint.base-path`| Prefix for the endpoint paths | empty |
82
126
83
-
### 3. Standalone server
127
+
The server needs roughly 2.5 GB of memory for the English model (`-Xmx3g` is a safe setting).
Without Spring, `laya-api` includes a dependency-free HTTP server (`io.github.apscot.laya.api.LayaHttpServer`) configured through environment variables:
132
+
`LAYA_PORT` (default 8000), `LAYA_HOST`, `LAYA_API_KEY`, `LAYA_MODELS`, `LAYA_PRELOAD`. It also serves `/health`.
133
+
134
+
## Client
135
+
136
+
`laya-client` is a JDK-only client for Laya or any Jev-compatible server.
137
+
138
+
```java
139
+
LayaClient client =newLayaClient("http://localhost:8080", null); // second argument: API key, or null
140
+
Map<String, Object> r = client.systemOne("I was billed twice",
141
+
Map.of("refund", Question.noul("Does the customer want money back?", null)));
142
+
double probability =LayaClient.noul(r, "refund");
87
143
```
88
144
89
-
### 4. Gradle plugin
145
+
## Gradle plugin
90
146
91
147
```kotlin
92
148
plugins { id("io.github.apscot.laya") version "0.1.0" }
`layaServe` / `layaStop` start and stop a local server. With `testIntegration`, it runs around your `test` task and passes
97
-
`laya.base-url` to it.
98
-
99
-
## Spring AI TypeSafe
150
+
laya {
151
+
model.set("aac6fef/laya-mlx")
152
+
port.set(8002)
153
+
testIntegration.set(true) // start a server around `test` and pass -Dlaya.base-url
154
+
}
155
+
```
100
156
101
-
[`spring-ai-community/spring-ai-typesafe`](https://github.com/spring-ai-community/spring-ai-typesafe) talks to Jev's `/v1/systemone`.
102
-
laya-java can stand in for the hosted API:
157
+
Tasks: `layaServe` starts a local server and waits until it is healthy; `layaStop` stops it.
103
158
104
-
```properties
105
-
spring.ai.typesafe.base-url=http://localhost:8080
106
-
spring.ai.typesafe.api-key=anything # that client refuses an empty key; laya-java ignores it unless you set laya.api-key
107
-
```
159
+
## Compatibility
108
160
109
-
Tested with `typesafe-java-sdk:0.1.0` (yes/no, choice, score, rejected key).
161
+
-**OpenAI Decisions API**: `predicate`, `choice` and `score` questions, text input.
162
+
-**Jev / System One**: works with [`spring-ai-community/spring-ai-typesafe`](https://github.com/spring-ai-community/spring-ai-typesafe) as a local replacement for the hosted API:
163
+
```properties
164
+
spring.ai.typesafe.base-url=http://localhost:8080
165
+
spring.ai.typesafe.api-key=anything # that client requires a non-empty key; laya-java ignores it unless laya.api-key is set
166
+
```
167
+
-**Checkpoints**: `aac6fef/laya-mlx` (English), `aac6fef/laya-multilingual-mlx`, `aac6fef/laya-typed-decisions-mlx` (converted from the original `convaiinnovations/laya` models).
110
168
111
-
## Speed and memory
169
+
Numerical results are tested against the Python [`laya-mlx`](https://github.com/mizorewww/laya-mlx) reference (tokenizer ids, scores and final answers) for all three
170
+
checkpoints. These tests establish agreement with the reference on a fixed set of cases, not accuracy on every possible input.
112
171
113
-
Run with `--add-modules=jdk.incubator.vector` (with `JAVA_TOOL_OPTIONS` use the `=` form). Without it the library falls back to a slower scalar
114
-
kernel. Allow about 2.5 GB of memory for the English model.
172
+
## Performance
115
173
116
174
Measured on an Apple M5 (10 cores), JDK 21, English checkpoint, warm, 31-token question:
117
175
118
-
| Backend |median|
176
+
| Backend |Median latency|
119
177
|---|---|
120
-
|`laya-accelerate` (macOS Accelerate via JNA) |~60 ms |
178
+
|`laya-accelerate` (macOS Accelerate) |~60 ms |
121
179
| Portable Java (Vector API) |~186 ms |
122
-
| Python/MLX on a GPU (published, M3 Max) |~13 ms |
180
+
| Python / MLX on a GPU (published figure, M3 Max) |~13 ms |
181
+
182
+
A 512-token input takes about one second with Accelerate. laya-java is slower than MLX on a GPU; its advantage is running anywhere a JVM runs.
183
+
184
+
Tips:
185
+
- Start the JVM with `--add-modules=jdk.incubator.vector`. When using the `JAVA_TOOL_OPTIONS` environment variable the flag must include the `=`. Without it, laya-java falls back to a slower scalar kernel.
186
+
- Load the model once and reuse the `Agent`; the first call includes warm-up.
187
+
- Use `laya.preload=true` in servers.
123
188
124
-
A 512-token input takes about 1 s with Accelerate. This port is slower than MLX on a GPU; its advantage is running anywhere a JVM runs.
189
+
## Reading the answers
125
190
126
-
## How it was checked
191
+
Each answer includes probabilities and two confidence values.
127
192
128
-
Tokenizer ids, scores and final answers match the Python `laya-mlx` (CPU, float32) for the English, multilingual and typed-decisions checkpoints
129
-
on the fixtures in `laya-core/src/test/resources/fixtures` (tolerance 2e-3 on scores). The fixtures come from `tools/gen_fixtures.py`, a
130
-
dev-time-only Python script that is not part of any published artifact. This shows agreement on those cases, not accuracy on every question.
193
+
-**choice**: `choice` is the most likely option; `probabilities` covers every option.
194
+
-**yes/no**: `noul` (or `probability` in the OpenAI format) is P(true).
195
+
-**score**: levels are numbered `0 … n-1` in the order you give them. `score` is the expected level: Σ index × probability. For levels `Calm`, `Frustrated`, `Very angry`
196
+
with probabilities `0.03 / 0.85 / 0.12` the score is `1.09`. Always supply levels from lowest to highest, and use the per-level probabilities when you need more than the average.
197
+
-**`answer_confidence`** is the probability of the top answer. **`confidence`** measures how peaked the distribution is (1 = certain, 0 = evenly spread); the two are
198
+
different scales and should not share a threshold.
131
199
132
-
## Not included yet
200
+
Probabilities are calibrated with temperature scaling taken from the checkpoint, but are not guaranteed to match real-world frequencies. Before using a threshold in
201
+
production, measure it on labelled examples from your own data.
133
202
134
-
Checkpoints using the "parallel" option layout (none are published), image input, `predict_shortlist` / `predict_tournament`,
135
-
`min_confidence` gating, email presets, streaming. Language routing is a simple heuristic, not upstream's full language detector.
136
-
The `/v1/decisions` response follows OpenAI's public docs and third-party examples; I could not read OpenAI's full reference, so
137
-
check field names against the OpenAI SDK if you depend on them.
203
+
## Limitations
204
+
205
+
- Checkpoints that use the "parallel" option layout are not supported (none are published yet).
206
+
- Image input is not supported; `/v1/decisions` returns an error for image parts.
207
+
-`predict_shortlist`, `predict_tournament`, `min_confidence` gating, email presets and streaming from the Python library are not included.
208
+
- Language routing between the English and multilingual checkpoints is a simple heuristic, not the full upstream detector.
209
+
- The `/v1/decisions` response follows OpenAI's public documentation and published examples. If your code depends on exact field names, check them against the OpenAI SDK you use.
210
+
- Like the original model, Laya is weaker on criteria that need world knowledge. Validate it on your own data.
gradle build # tests needing checkpoints are skipped unless LAYA_MODEL_DIR, LAYA_MODEL_DIR_MULTILINGUAL, LAYA_MODEL_DIR_TYPED are set
227
+
git clone https://github.com/apscot/laya-java.git
228
+
cd laya-java
229
+
gradle build
155
230
```
156
231
157
-
Requires JDK 17+. License: Apache-2.0 (see `LICENSE` and `NOTICE`). Releasing: see `RELEASING.md`.
232
+
Tests that need model checkpoints are skipped unless `LAYA_MODEL_DIR`, `LAYA_MODEL_DIR_MULTILINGUAL` and `LAYA_MODEL_DIR_TYPED` point at local checkpoints.
233
+
See [TESTING.md](TESTING.md) for a step-by-step walkthrough and [RELEASING.md](RELEASING.md) for the release process. The Python script in `tools/` generates test
234
+
fixtures from the reference implementation; it is a development tool and is not part of any published artifact.
235
+
236
+
## License and acknowledgements
237
+
238
+
Apache License 2.0; see [LICENSE](LICENSE) and [NOTICE](NOTICE).
239
+
240
+
laya-java builds on [Laya](https://github.com/NandhaKishorM/laya) by Convai Innovations and contributors, the [laya-mlx](https://github.com/mizorewww/laya-mlx) port
241
+
(used as the numerical reference), and the ModernBERT architecture. Thanks to the authors of all three.
0 commit comments