Skip to content

v1.6.1.md

Latest commit

 

History

History
147 lines (125 loc) · 9.13 KB

v1.6.1.md

File metadata and controls

147 lines (125 loc) · 9.13 KB
Raw

laya-apple 1.6.1: a safer artifacts fetch, and auto on macOS 27

A patch release for the 1.6 line. Two things change for users:

  • laya-apple artifacts fetch now registers an artifact only after every check has passed.
  • On an Apple M4 Max with macOS 27 and coremltools 9.0, device="auto" uses the ANE once the artifacts are built.

predict, the parity tolerances, the routing rule, the macOS 26.6.2 routing table and the prebuilt-artifact pin are unchanged.

pip install -U 'laya-apple[ane]'

artifacts fetch keeps only what passed

Fixed in #146.

  • An existing bucket is skipped only if it passes load_verified, the runtime's own checks. Before, any manifest.json, even an empty one, counted as registered (#132).
  • The placement probe runs on the staged copy before it moves into place, and again at the registered path, since Core ML compiles per path. If anything fails, the new artifact is dropped and the previous one stays. Before, a failed probe left the artifact registered, and --force had already deleted the old one (#133, #147).
  • --offline (or HF_HUB_OFFLINE=1) now covers the index and archive downloads too. Fetch reads them from the Hugging Face cache, still runs the full import checks, and raises BackendUnavailableError if they are not cached (#143).

For how artifacts prune cleans up after an interrupted import, see CHANGELOG.md.

auto on macOS 27 (M4 Max)

1.6.0 had no routing profile for macOS 27, so auto stayed on MLX there (platform_not_validated). 1.6.1 ships one for Apple M4 Max, macOS 27, coremltools 9.0 (#152).

  • On macOS 27.0, all three models built for every explicit ANE bucket, ran 100% on the Neural Engine with 0 transitions, and passed the FP16 parity gate with 0 hard mismatches (research/macos27-validation/).
  • Two laya-apple calibrate passes picked the same auto buckets as on 26.6.2: 64, 96 and 128 for every model. laya-multilingual L256 stays explicit-only (benchmarks/routing-macos27/).
  • Build the artifacts locally with laya-apple artifacts build MODEL (convert extra). There are no prebuilt ones for macOS 27.

Also in 1.6.1

  • laya-apple artifacts list --format table prints one row per artifact: bucket, status, parity and compute plan. JSON stays the default, unchanged (#4).
  • scripts/hardware_report.py --json PATH writes a compact summary of a run (#7, docs/community-benchmarks.md).
  • laya-apple info adds an environment block: Python, MLX and the effective offline mode (#9).
  • laya-apple serve uses one loopback test for the bind address and the Host header, so --host 127.0.0.2 no longer answers 421. Remote binding still needs --allow-remote and LAYA_API_KEY (#135).
  • SECURITY.md covers the 1.6 line, serve and artifacts fetch (#134). CI runs with a read-only token (#137).
  • CI rejects new tracked files over 1 MiB. Larger raw data goes out as external evidence with a checksummed manifest (#136, docs/evidence.md).
  • The READMEs open with install and a runnable quickstart (#138). The guide has worked score and noul examples (#5).

Research, not shipped

  • Compile-cache reuse on macOS 27.0 (#121, results.md). A cold start of laya-typed-decisions (buckets 64/96/128) took 210.152 s (median of 3) against about 2 s warm. Touching the files kept the compile (5.7 s); re-copying them to the same path lost it (210.9 s); a move was inconclusive (88.0 s). An import compiles twice: about 66 s for the registered load after a 27 s parity gate. No code change.
  • The host-side slow state of the no-GIL ANE thread, screened on macOS 27.0 with 1.4's synchronous path (#89, results.md). The positive control was slow in 5 of 6 windows, the negative control in 0 of 6. Holding the GIL, a fixed warm-up and a 1 ms probe thread each removed the slow state. No single cause is named: holding the GIL also changed the GPU reply wait, so that contrast is wider than the GIL alone, and background load was a possible confound. It says nothing about macOS 26.6.2, where the slow state was first seen, or about 1.5's adaptive execution.

Limitations

  • No prebuilt artifacts for macOS 27. They still cover only Apple M4 Max, macOS 26, coremltools 9.0. On macOS 27, artifacts fetch raises ArtifactMissingError; build locally (docs/support-matrix.md).
  • On macOS 27 only build, parity and routing were measured. The release benchmarks, serve beside a local LLM and adaptive ANE execution were not (docs/compatibility.md).
  • The shipped profile matches every macOS 27.x on an M4 Max with coremltools 9.0, but only 27.0 (26A428) was measured. Third-party work saw an enumerated-shape Core ML package run on the GPU on macOS 27.2. laya-apple ships fixed-shape artifacts, and the compute-plan check and the placement probe still run on every machine, but 27.2 is unmeasured (docs/support-matrix.md).
  • The macOS 27 margin at L96 is thin. For laya and laya-typed-decisions the ANE at 96 tokens beats MLX at 64 tokens by 0.21–0.27 ms. On 26.6.2 the margin was 0.37–0.41 ms, but that table came from the Phase -1 harness and this one from calibrate: different methods, not a measured change (benchmarks/routing-macos27/).
  • A cold start still takes minutes: about 210 s on macOS 27 and 273.5 s in one 26.6.2 run (different OS and run count; not compared). Fetching skips the build, not the compile. ane_startup="background" serves on MLX meanwhile.
  • Core ML's compile cache still grows without eviction: 4.36 GB per 3-bucket cold load on macOS 27.
  • No second machine has checked the prebuilt artifacts yet.
  • Every release measurement comes from one Apple M4 Max, now on two macOS versions.
  • The 1.6.0 limitations not repeated here still apply.

Upgrading

  • No code change needed.
  • M4 Max, macOS 27, coremltools 9.0: auto now uses the ANE once artifacts are built, and with execution="workers" it also runs 1.5's adaptive ANE execution, which is unmeasured on macOS 27. The shipped profile replaces any local laya-apple calibrate profile there. Pass device="gpu" to keep 1.6.0's behaviour.
  • On that profile laya-apple info reports platform_validated_for_auto_ane: true (was false), and Laya.info()["routing_profile"] becomes "shipped". Everywhere, laya-apple info also gains the environment block. artifacts list JSON is unchanged.
  • artifacts fetch may re-download a bucket that 1.6.0 skipped, if the registered copy fails verification.

Release gate

Two full scripts/release_gate.py runs on Apple M4 Max, macOS 27.0, coremltools 9.0. Run 1 failed on a timing-sensitive test, fixed by #156 with a fake clock. Run 2 passed every required step (pytest: 1643 passed, 9 skipped).

Differences from 1.6.0's gate, all explained in benchmarks/release-gate-1.6.1-macos27-notes.md: it ran on macOS 27.0 instead of 26.6.2, with the shipped macOS 27 profile instead of routing.json, and before the version bump. It covered fewer artifacts (10 manifests against 15: L64–L128 for every model, plus laya-multilingual L256). derive_routing --check does not cover the macOS 27 profile; unit tests do. Other load ran on the machine during run 2.