Skip to content

Fix routing correctness and harden updates, tracing, and reports - #87

Merged
adrida merged 1 commit into
mainfrom
fix/routing-reliability
Sep 25, 2026
Merged

adrida merged 1 commit into
mainfrom
fix/routing-reliability

Conversation

@adrida

@adrida adrida commented Sep 25, 2026 •

Copy link
Copy Markdown
Owner

Failed updates could leave traces out of sync with the saved model, residual stages could return the wrong label IDs, and FAISS normalization could make a reloaded policy defer every request. This change preserves recoverable artifacts during updates and corrects those routing failures, with regression coverage for tracing, reports, and serving.

  • Stage refits before replacing artifacts, restore the previous generation if publication fails, and retain a recovery backup if restoration fails. Honor explicit configuration without mutation, reuse saved settings by default, and track fit counts.
  • Map probability columns through classifier labels, choose the largest eligible small-data acceptance set, propagate configured seeds, preserve original embedding scales, and reuse the OOD neighbor index.
  • Await asynchronous watched calls, retain parentage and cancellation/error details, and isolate recording failures from application results. Validate local watcher filenames in Python and JavaScript.
  • Escape report content, retain deferred confidence scores, validate embedding inputs, and report correct trace file line numbers.
  • Serve concurrent requests, tolerate disconnected clients, validate request bodies, and close cleanly on interruption. Correct update examples and document changed behavior.

Routing measurements

Deterministic reproductions against main at 0c918712c09a677feae80eede7069442c4e259a9, using identical dependencies:

Case Before After
Small calibration: coverage 30.56% 91.67%
Small calibration: agreement on accepted rows 100% 90.91%
Small calibration: agreement lower bound 81.11% 80.86%
Label-subset prediction agreement 0% 100%
FAISS fit/reload: held-out coverage 0% 99%
FAISS fit/reload: agreement on handled held-out rows No handled rows 100%

Both small-calibration selections pass the same 80% target and alpha 0.1 lower-bound test. The FAISS case uses 240 training rows and 600 separate held-out rows at a 90% target. These are targeted regression measurements, not general workload guarantees. Commands, seeds, dependencies, and limitations are recorded in benchmarks/README.md and benchmarks/routing_reliability.py.

Validation

  • Python 3.12 with FAISS and Plotly: 151 passed, 3 skipped. Skips are one unavailable optional embedding backend and two existing fixture-dependent cases with no deployable policy. The new input-validation and HTTP regression tests run successfully.
  • JavaScript: 33 tests passed; TypeScript build and package dry run passed.
  • Python wheel and source distribution build passed.
  • All six CI jobs passed: Python 3.12, 3.13, and 3.14 with current dependencies, Python 3.12 with minimum core dependencies, Python 3.12 with FAISS/Plotly, and JavaScript.

Compatibility

  • The server now defaults to 127.0.0.1, sends no wildcard CORS header, limits request bodies to 8 MiB, and uses a 10-second socket timeout. Invalid prediction inputs return HTTP 400. Remote/browser deployments must configure their bind address and proxy explicitly.
  • Local watcher names must start with an ASCII letter or digit and contain only letters, digits, dots, underscores, or hyphens, up to 128 characters.
  • Existing FAISS artifacts fitted with unnormalized inputs need refitting from the original embeddings to recover their scale.
  • Updates require space for staging and backup. Use one writer and reload readers after completion; directory replacement is not a concurrent-reader or power-loss transaction.

@adrida
adrida merged commit 169edf8 into main Sep 25, 2026
6 checks passed
@adrida
adrida deleted the fix/routing-reliability branch September 25, 2026 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant