Skip to content

api.md

Latest commit

 

History

History
146 lines (120 loc) · 10.5 KB

api.md

File metadata and controls

146 lines (120 loc) · 10.5 KB
Raw

Public API and stability policy

laya-apple follows Semantic Versioning from 1.0.0. This page defines what "the public API" means. Anything not listed here is internal and may change in any release.

Stable (covered by SemVer)

Python

Import these from the top-level laya_apple package:

Name Contract
Laya.from_pretrained(model_id, device="auto", *, dtype="float16", local_files_only=False, batch_size=16, execution="inline", ane_placement="auto", ane_startup="wait", trace=None, ane_handoff=None) Loads a pinned checkpoint. Invalid arguments raise ValueError. Every other failure raises a LayaAppleError subclass
Laya.from_pretrained("auto", ...) -> LayaRouter (since 1.6) Language routing between laya and laya-multilingual, the decision laya-apple serve --model auto and upstream's Router make. Both checkpoints load with the given arguments. LayaRouter has predict, predict_shortlist, submit, apredict, close, wait_for_ane, info and the context manager, with the same contracts as Laya. The chosen checkpoint is RuntimeInfo.model, why is RuntimeInfo.model_routing, and upstream's routing block (model, repo, reason, detection, workflow) is Result.extra["routing"] and the routing key of to_dict(). The reason text is not stable
Laya.predict(context=None, questions=None, *, state=None) -> Result Blocking and thread-safe
Laya.predict_shortlist(context=None, questions=None, *, state=None, embed_fn, k=20) -> Result (since 1.6) Opt-in, never used by predict. Upstream's predict_shortlist: each choice question with more than k labels is cut to the k labels whose embed_fn vectors are most cosine-similar to the state's, then one predict runs on the reduced questions. Result.extra["shortlist"][qid] (and the shortlist key of to_dict()) holds labels, scores, k, n, passthrough. Probabilities on a shortlisted choice are over the kept labels only. An invalid k or embed_fn raises ValueError, a malformed question InvalidRequestError
shortlist_choice(state, criteria, embed_fn, k=20, *, instructions=None) -> list (since 1.6) The top-k labels of one choice question, in rank order; all labels in their order, without calling embed_fn, when k covers them
embed_fn_from_laya(laya, max_length=512, batch_size=32) (since 1.6) An embed_fn that mean-pools the checkpoint's own MLX encoder. Needs execution="inline" and device gpu or auto; otherwise raises BackendUnavailableError
Laya.tokenizer.cache = None (since 1.6) Turns off the token-id cache, which is on by default and exact by construction; texts are then tokenized on every request, as in 1.5. It applies to that instance in both execution modes. On a LayaRouter, set it on each of router.instances.values()
Laya.submit(...) -> concurrent.futures.Future[Result] Same arguments as predict
await Laya.apredict(...) -> Result Same arguments as predict
Laya.close(), with Laya.from_pretrained(...) as laya: Idempotent. Queued work fails with BackendUnavailableError
Laya.wait_for_ane(timeout=None) -> bool Only meaningful with ane_startup="background"
Laya.info() -> dict The keys below are stable. New keys may be added
Result Fields answers, usage, runtime, model, extra, and to_dict(). to_dict() adds each extra entry as a top-level key (since 1.6), never replacing model, answers, usage or runtime
RuntimeInfo Every field listed in laya_apple/result.py at 1.0.0. New optional fields may be added
RequestTrace, QueueSnapshot The fields and duration properties in laya_apple/trace.py. trace= (workers only) calls the callback once per completed request, before its Future resolves; a callback exception is warned once and never fails the request. New fields may be added
The exception classes in laya_apple.errors, re-exported at top level Their hierarchy: every one derives from LayaAppleError, and the artifact errors from ArtifactError
laya_apple.__version__

Stable info() keys:

  • model, repo, revision;
  • device, execution, ane_placement, dtype;
  • mlx, ane_buckets, ane_load_errors;
  • routing_profile, ane_ready, auto_ane;
  • ane_handoff (since 1.5), present only on instances eligible for adaptive execution: its state, and whether it is enabled and why not. When adaptive execution could not start, the dict has only enabled, disabled and disabled_reason.

Argument values:

Argument Values
device auto, gpu, ane
dtype float16, float32 (MLX only; device="ane" accepts only float16)
execution inline, workers
ane_placement auto, thread, process
ane_startup wait, background
ane_handoff None (default: adaptive ANE execution where eligible and available), False (always the 1.4 path), True (required; guide.md)

Routing reasons

The strings in RuntimeInfo.routing_reason are stable:

  • gpu_requested, ane_requested, validated_short_single_question_path;
  • multiple_questions, sequence_exceeds_ane_auto_range;
  • ane_artifact_unavailable, ane_runtime_unavailable, platform_not_validated;
  • ane_backlog_shorter_on_gpu, gpu_backlog_shorter_on_ane, ane_starting.

New reasons may be added in a minor release. Code that switches on the reason must accept unknown values.

RuntimeInfo.model_routing (since 1.6) says why the language router chose the checkpoint, and is None when the caller named one. Its values are stable:

  • language_english, language_non_latin_script, language_not_english;
  • language_undecided_default, no_text_default (both choose laya, upstream's default).

Which device serves a request is not part of the API. Routing thresholds, auto buckets and service-time estimates are measured data. They can change in any release, with the evidence recorded in CHANGELOG.md. The guarantee is that every decision is recorded in RuntimeInfo, never that a given request lands on a given device.

Command line

The laya-apple subcommands, and their arguments as listed in the CLI reference in docs/guide.md, are stable:

  • predict, info, download;
  • artifacts build|list [--capabilities]|verify|warm|prune|export|import, and fetch (since 1.6);
  • parity, calibrate, benchmark;
  • serve (since 1.3.0, needs the [serve] extra).

Exit codes:

  • 0 means success;
  • 1 means a check failed (artifacts verify, parity);
  • 2 means a laya-apple error or an invalid argument value.

Files and environment

Item Contract
LAYA_APPLE_CACHE, XDG_CACHE_HOME, HF_HUB_OFFLINE Their meaning as documented in docs/guide.md
LAYA_APPLE_PREBUILT_REPO (since 1.6) The Hugging Face repository artifacts fetch reads when --repo is not given
Prebuilt index, format: "laya-apple-prebuilt-index", format_version: 1 (since 1.6) index.json at the root of a prebuilt-artifact repository (laya_apple/prebuilt.py). Entries may gain fields; a reader refuses other format versions
Artifact manifest, format: "laya-apple-artifact", format_version: 1 The shipped JSON Schema is laya_apple/data/manifest.schema.json. Fields may be added without a version change, and readers ignore fields they do not know. Removing a field or changing its meaning requires format_version: 2. A release that reads v2 keeps reading v1 for at least one major version
Export archive (artifacts export) A .tar.gz holding manifest.json and model.mlmodelc/
Local capability profile, format: "laya-apple-profile", format_version: 1 <cache>/profiles/<profile>.json

HTTP API (laya-apple serve, since 1.3.0)

Stable, as documented in docs/serve.md:

  • the endpoints POST /v1/systemone, GET /v1/models, GET /health, GET /healthz;
  • the request fields and the status codes;
  • the top-level response keys model, answers, usage, routing, laya_apple, and the keys of the laya_apple block;
  • the --model values, the default port 8642, and the loopback-only default bind;
  • the environment variables LAYA_APPLE_SERVE_MODEL, LAYA_APPLE_SERVE_HOST, LAYA_APPLE_SERVE_PORT and LAYA_API_KEY.

The wire format tracks upstream laya.serve. When upstream changes it, laya-apple follows in a minor release, with the change recorded in CHANGELOG.md. New response keys may be added at any time. The text of routing.reason and of error detail values is not stable.

The cache layout (the directories under <cache>/artifacts/) is not public. Use the CLI or the manifest, not paths.

Internal (no compatibility promise)

These are not covered by SemVer:

  • Laya.prepare, Laya.route, Laya.backlogs, Laya.queue_snapshots, and the mlx / ane attributes;
  • every submodule other than laya_apple.errors: laya_apple.routing, scheduling, executor, artifacts, lifecycle, profiles, derivation, backends, conversion, parity, schema, workload, benchmark, serve, lang, router other than LayaRouter, and shortlist other than the names above (use the CLI and the HTTP API);
  • the bundled data files, except the manifest schema;
  • the MLX fast-path switches (since 1.6): LAYA_APPLE_MLX_COMPILE, LAYA_APPLE_MLX_LENGTH_BUCKETS and MLXBackend(compile_forward=..., length_buckets=...), and the :compiled suffix of RuntimeInfo.artifact_revision. They are documented in the guide, and may change or be removed in any release;
  • everything under scripts/.

Deprecation policy

  • A stable name or behaviour is deprecated before it is removed. Deprecated functionality keeps working for at least one minor release and is removed only in the next major release.
  • Using a deprecated name or argument emits a DeprecationWarning naming the replacement and the version that removes it.
  • Every deprecation and removal is listed under Deprecated or Removed in CHANGELOG.md.
  • Exception: a correctness or safety defect may be fixed in any release, even when the fix changes behaviour. An example is a path that could silently run on an unvalidated configuration. The CHANGELOG says so explicitly.
  • A pinned model revision changes only in a minor or major release. The old revision's artifacts are then reported by artifacts prune.