Skip to content

Commit af4527d

Browse files
dfa1claude
andauthored
Split modules into model (core), codec, contract (client) and implementations (client-*); byte[] SPIs (#35)
* Split modules into contract (core) and implementations (client-*), align names core keeps the TypeSafeClient interface, its decorators, TypeSafeException, the model and the JsonCodec SPI. DefaultTypeSafeClient, ApiKey and the HttpTransport SPI move to the new client-http module, so the compiler now enforces that core never sees the HTTP client; TypeSafeClient.builder() becomes DefaultTypeSafeClient.builder(). Every module's directory, artifact and package share one name: client-http-jdk, client-http-okhttp, codec-jackson2, codec-jackson3, client-local, client-mapping, client-testkit (io.github.dfa1.typesafe.<module>, dashes as dots). No client-* module depends on another in production. ADR 0003 records the layout (closes #12). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * core: JsonCodec in io.github.dfa1.typesafe.core, so core holds one package The core artifact held two packages (core, codec); every other module holds exactly the package its name gives. JsonCodec now sits in core directly, as HttpTransport sits in client.http. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * client: TypeSafeClient, TypeSafeException and the decorators out of core core is now the model and JsonCodec only; the client contract (TypeSafeClient, TypeSafeException, Retrying/DeadlineTypeSafeClient, TokenCounter) is the new client module, which every client-* module builds on. Serializing the model (core + a codec) no longer pulls in any client code, and a module named X-Y now always builds on module X. ADR 0003 rewritten to match. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * JsonCodec and HttpTransport speak byte[] JSON goes to and from the wire as UTF-8 bytes: writeValueAsBytes / writeValueAsPrettyBytes / readValue(byte[], ...), and a byte[] body on HttpTransport.post and HttpTransportResponse. No String copy on the success path; bodies become text only for error messages. Jackson2Codec enables COMBINE_UNICODE_SURROGATES_IN_UTF8: its UTF-8 writer otherwise escapes emoji as surrogate pairs, which changed what a local model tokenizes (caught by ClefEngineTest; now pinned by a codec test in both codecs). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * core: JsonCodec is now Codec The SPI is byte[] in and out, so it isn't JSON-specific: a binary codec (#37) can implement it. Builder.jsonCodec(...) becomes codec(...); client-http still needs a JSON codec, since the API speaks JSON. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * client-local: Onnx.json() is now codec(), after the Codec rename Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * Codec: drop writeValueAsPrettyBytes Its one caller was the CLI's --verbose, and indentation is a JSON notion that doesn't belong on a format-neutral Codec. --verbose now prints compact JSON; pipe it into jq to read it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * codec: Codec in its own module, so core is pure data core held one piece of behaviour, the Codec SPI, and the codec-* modules broke the X-Y rule (no codec module to build on). Codec now lives in typesafe-java-codec (package io.github.dfa1.typesafe.codec), with no dependencies; codec-*, client-http and client-local depend on it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
1 parent b1a86bb commit af4527d

128 files changed

Lines changed: 1127 additions & 780 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/local.yml‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
name: Local engines
22

3-
# Agreement with Jev and throughput of typesafe-java-local's engines, as a table in the job summary.
3+
# Agreement with Jev and throughput of typesafe-java-client-local's engines, as a table in the job summary.
44
# Model directories are cached: Laya fp32 (+ fp16 on macOS, for the GPU) and Qwen 4-bit, all onnx-community exports.
55
on:
66
workflow_dispatch:
77
push:
88
branches: [main]
9-
paths: ['local/**', '.github/workflows/local.yml', '.github/scripts/local_summary.py']
9+
paths: ['client-local/**', '.github/workflows/local.yml', '.github/scripts/local_summary.py']
1010
pull_request:
11-
paths: ['local/**', '.github/workflows/local.yml', '.github/scripts/local_summary.py']
11+
paths: ['client-local/**', '.github/workflows/local.yml', '.github/scripts/local_summary.py']
1212

1313
jobs:
1414
numbers:
@@ -48,23 +48,23 @@ jobs:
4848
uvx --from huggingface_hub hf download Cloudflare/clef-flash --revision 17f0b0ad64efb65d273590632833508766b2aae6 tokenizer.json --local-dir $M/clef-flash
4949
5050
- name: Build
51-
run: ./mvnw -B -q -pl local -am test-compile dependency:build-classpath -Dmdep.outputFile=$PWD/cp.txt -Dmdep.includeScope=test
51+
run: ./mvnw -B -q -pl client-local -am test-compile dependency:build-classpath -Dmdep.outputFile=$PWD/cp.txt -Dmdep.includeScope=test
5252

5353
- name: Real-model tests
54-
run: ./mvnw -B -pl local -am test -DexcludedGroups=acceptance -Dengine=laya -Dsurefire.failIfNoSpecifiedTests=false
54+
run: ./mvnw -B -pl client-local -am test -DexcludedGroups=acceptance -Dengine=laya -Dsurefire.failIfNoSpecifiedTests=false
5555

5656
- name: Agreement with Jev
57-
working-directory: local
57+
working-directory: client-local
5858
run: |
5959
CP="$(cat ../cp.txt):target/classes:target/test-classes"
6060
ENGINES="laya qwen"
6161
if [ "$RUNNER_OS" = macOS ]; then ENGINES="$ENGINES laya-fp16-gpu"; fi
6262
for e in $ENGINES; do
63-
java -cp "$CP" io.github.dfa1.typesafe.local.JevComparison $e | tee -a ../jev.txt
63+
java -cp "$CP" io.github.dfa1.typesafe.client.local.JevComparison $e | tee -a ../jev.txt
6464
done
6565
6666
- name: Throughput
67-
working-directory: local
67+
working-directory: client-local
6868
run: |
6969
CP="$(cat ../cp.txt):target/classes:target/test-classes"
7070
ENGINES=LAYA,QWEN

‎CHANGELOG.md‎

Lines changed: 15 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -7,25 +7,27 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
- **Breaking: `JsonCodec` is now `Codec`, and it and `HttpTransport` speak `byte[]`** — `writeValueAsBytes`/`readValue(byte[], ...)` (no pretty-printing variant: the CLI's `--verbose` prints compact JSON), `Builder.codec(...)` (was `jsonCodec(...)`), and a `byte[]` request/response body, so JSON goes to and from the wire as UTF-8 with no `String` copy and a binary codec can implement the same SPI; `Jackson2Codec` now writes emoji and other non-BMP characters unescaped, like `Jackson3Codec`. (#35)
11+
- **Breaking: modules split into model (`core`), serialization (`codec`), contract (`client`) and implementations (`client-*`), names aligned** — `core` keeps the model; `Codec` is the new `codec` module; `TypeSafeClient`, `TypeSafeException` and the decorators move to the new `client`, and `DefaultTypeSafeClient`/`ApiKey`/`HttpTransport` to the new `client-http`; `TypeSafeClient.builder()` becomes `DefaultTypeSafeClient.builder()`, and every module's artifact and package now match its directory (`client-http-jdk`, `client-http-okhttp`, `codec-jackson2`, `codec-jackson3`, `client-local`, `client-mapping`, `client-testkit`); see ADR 0003. (#35)
1012
- **docs: local-model setup as three steps** — dependency, Laya download, code, then a model comparison; the optional Clef 4-bit script runs from its GitHub URL with `uv run`, and load errors link to the how-to.
11-
- **`local`: hardening** — special tokens in caller text stay plain text (no prompt injection via `<|im_end|>`), the tokenizer cache is bounded, and Clef checks its weights' sha256 on first load.
12-
- **`local`: shared request validation, `evaluateAsync` off virtual threads** — every engine rejects no questions or empty criteria with `BadRequest` (Laya used to NPE or answer a null choice), and `evaluateAsync` runs on one platform thread so native inference can't pin the JVM's virtual-thread carriers.
13-
- **`local`: `LocalClefTypeSafeClient.loadOnGpu(...)`** — Clef-flash on ONNX Runtime's WebGPU backend (macOS on Apple Silicon), same answers as the CPU; Laya gets no GPU option, since its WebGPU answers drift between launches. (#19)
13+
- **`client-local`: hardening** — special tokens in caller text stay plain text (no prompt injection via `<|im_end|>`), the tokenizer cache is bounded, and Clef checks its weights' sha256 on first load.
14+
- **`client-local`: shared request validation, `evaluateAsync` off virtual threads** — every engine rejects no questions or empty criteria with `BadRequest` (Laya used to NPE or answer a null choice), and `evaluateAsync` runs on one platform thread so native inference can't pin the JVM's virtual-thread carriers.
15+
- **`client-local`: `LocalClefTypeSafeClient.loadOnGpu(...)`** — Clef-flash on ONNX Runtime's WebGPU backend (macOS on Apple Silicon), same answers as the CPU; Laya gets no GPU option, since its WebGPU answers drift between launches. (#19)
1416
- **docs: run Clef-flash on a Mac with MLX** — a how-to pointing the regular client at mlx-community's local Clef-flash server: about 0.55 s per request on an M5, much closer to Jev than Laya. (#14)
15-
- **`local`: `LocalClefTypeSafeClient`** — Cloudflare's Clef-flash (Qwen3.5-9B + joint schema head) on Ollaya's ONNX graph over the upstream bf16 weights; one forward pass per request; `scripts/clef/quantize_q4.py` converts it to 4-bit weights (7.7 GB, 2–4 s per request on an M5 GPU). (#14)
16-
- **`local`: `LocalLayaTypeSafeClient`, `LocalQwenTypeSafeClient`** — new module, one client class per model, evaluating in-process on ONNX Runtime from a local model directory (Laya fp32 from onnx-community, or Qwen2.5 as a prompted-LLM baseline), with a `Local engines` workflow publishing agreement-with-Jev and throughput tables. (#14)
17+
- **`client-local`: `LocalClefTypeSafeClient`** — Cloudflare's Clef-flash (Qwen3.5-9B + joint schema head) on Ollaya's ONNX graph over the upstream bf16 weights; one forward pass per request; `scripts/clef/quantize_q4.py` converts it to 4-bit weights (7.7 GB, 2–4 s per request on an M5 GPU). (#14)
18+
- **`client-local`: `LocalLayaTypeSafeClient`, `LocalQwenTypeSafeClient`** — new module, one client class per model, evaluating in-process on ONNX Runtime from a local model directory (Laya fp32 from onnx-community, or Qwen2.5 as a prompted-LLM baseline), with a `Local engines` workflow publishing agreement-with-Jev and throughput tables. (#14)
1719
- **build: checkstyle engine 14.3.0** — the plugin's default 9.3 can't parse Java 21 pattern matching for switch; same rules. (#14)
1820
- **Breaking: optional API key, `ApiKey` without a public constructor** — `TypeSafeClient.builder()` takes no argument and `.apiKey(key)` is optional (no key, no `Authorization` header, for local servers); `ApiKey` is a final class built with `of(String)`/`fromFile`/`fromDefaultFile`/`fromEnv`, and no longer exposes its value. (#18)
19-
- **`jackson2`: ignore unknown fields, like `jackson3`** — a field the API adds to a response no longer fails decoding with `ResponseDecoding`; Jackson 2's default rejected it. (#18)
21+
- **`codec-jackson2`: ignore unknown fields, like `codec-jackson3`** — a field the API adds to a response no longer fails decoding with `ResponseDecoding`; Jackson 2's default rejected it. (#18)
2022
- **`core`: `Content.fields(...)` and `EvaluateRequest.Builder` keep their order** — both copied with `Map.copyOf`, which reshuffles keys per JVM run, so the same request serialized with fields/questions in a different order each run; they now keep the caller's iteration order.
21-
- **`core`: `TokenCounter`** — running totals of input/output tokens from `EvaluateResponse#usage()`, added via `.decorateWith(tokens::decorate)`; thread-safe, shareable across clients.
22-
- **`core`: `Builder.decorateWith(...)` naming and docs** — named to read as additive and to pair with the `decorate(...)` factories; `Builder` javadoc and the how-to explain decorator ordering.
23-
- **`core`: `build()` rejects a second `RetryingTypeSafeClient`** — throws `IllegalStateException`, since stacked retries multiply attempts; `decorate(...)` now takes the delegate first on every decorator.
23+
- **`client`: `TokenCounter`** — running totals of input/output tokens from `EvaluateResponse#usage()`, added via `.decorateWith(tokens::decorate)`; thread-safe, shareable across clients.
24+
- **`client-http`: `Builder.decorateWith(...)` naming and docs** — named to read as additive and to pair with the `decorate(...)` factories; `Builder` javadoc and the how-to explain decorator ordering.
25+
- **`client-http`: `build()` rejects a second `RetryingTypeSafeClient`** — throws `IllegalStateException`, since stacked retries multiply attempts; `decorate(...)` now takes the delegate first on every decorator.
2426
- **Breaking: retries are opt-in** — `build()` makes one attempt per call; `Builder.maxRetries`/`initialBackoff` are gone (use `.decorateWith(RetryingTypeSafeClient::decorate)`, 5 retries from 500ms), and `Builder.decorator(...)` is renamed `decorateWith(...)`.
25-
- **`core`: `Builder.decorator(...)`** (renamed `decorateWith(...)` below) — stacks decorators from the builder, last added outermost; `build(Function)` stays for a type-preserving outermost decorator.
26-
- **`core`: `RetryingTypeSafeClient.decorate(...)`/`DeadlineTypeSafeClient.decorate(...)`** — static factories taking the client to wrap first, matching `MappingTypeSafeClient::decorate`; constructors are package-private.
27-
- **`core`: `DeadlineTypeSafeClient` caps a call's total time, retries included** — fails with `TypeSafeException.Timeout` past the deadline; `RetryingTypeSafeClient` stops retrying once its future is done.
28-
- **`core`: retry/backoff extracted into a `RetryingTypeSafeClient` decorator** — `DefaultTypeSafeClient` no longer retries itself, and `InternalServer` gains `retryAfter()`.
27+
- **`client-http`: `Builder.decorator(...)`** (renamed `decorateWith(...)` below) — stacks decorators from the builder, last added outermost; `build(Function)` stays for a type-preserving outermost decorator.
28+
- **`client`: `RetryingTypeSafeClient.decorate(...)`/`DeadlineTypeSafeClient.decorate(...)`** — static factories taking the client to wrap first, matching `MappingTypeSafeClient::decorate`; constructors are package-private.
29+
- **`client`: `DeadlineTypeSafeClient` caps a call's total time, retries included** — fails with `TypeSafeException.Timeout` past the deadline; `RetryingTypeSafeClient` stops retrying once its future is done.
30+
- **`client`: retry/backoff extracted into a `RetryingTypeSafeClient` decorator** — `DefaultTypeSafeClient` no longer retries itself, and `InternalServer` gains `retryAfter()`.
2931

3032
## [0.6.0] - 2026-09-30
3133

0 commit comments

Comments
 (0)