All notable changes to TRACER are recorded here. This project follows semantic versioning.
- Preserve the previous artifact generation when an update fails validation, fitting, or publication. Honor explicit update configuration without mutating the caller, retain saved configuration by default, and increment the fit count.
- Preserve classifier label IDs in residual stages and select the largest eligible acceptance set during small-data calibration.
- Keep original embedding scales when building FAISS indexes and reuse the OOD nearest-neighbor index across predictions. Existing FAISS artifacts trained with unnormalized inputs need refitting to recover the original scale.
- Apply configured seeds to subsampling, splits, calibration, and model training.
- Record async outputs, errors, cancellation, timing, and parent spans correctly; isolate tracing storage/export failures from application calls.
- Escape trace and artifact text in reports and reject unsafe watcher filenames in Python and JavaScript.
- Preserve deferred confidence scores in reports; validate prediction shapes and finite values; report actual file lines for null teacher labels.
- Handle concurrent HTTP requests and client disconnects, reject malformed or oversized request bodies, and close the server cleanly on interruption.
- Correct update examples to use
new_embeddings=.
- The prediction server binds to
127.0.0.1by default and no longer sends a wildcard CORS header. Explicitly configure a remote bind address when needed; browser deployments should configure CORS at their authenticated proxy. - Watcher filenames must start with a letter or digit and use only ASCII letters, digits, dots, underscores, or hyphens, up to 128 characters.
- Invalid prediction inputs return HTTP 400 rather than 500.
- Require Python 3.12 or newer; drop support for Python 3.9–3.11. CI tests Python 3.12, 3.13, and 3.14, plus the minimum core dependencies on 3.12.
- Allow NumPy 1.26.4 through 2.x so Python 3.14 can install compatible wheels. Require scikit-learn 1.4.2+ and joblib 1.3.2+ in core, and PyTorch 2.4.1+ for the optional embedding extras. Remove the obsolete NumPy downgrade advice.
- Build releases and run the documented Python sidecar on Python 3.14.
- The
tracercommand-line executable and command implementations. Use the Python APIs for fitting, scanning, updating, reports, and serving predictions. - The Tracer app client, authentication, and automatic app telemetry exporters in both Python and JavaScript. Existing app credentials no longer trigger uploads. Local recording, custom sinks, and explicitly configured generic HTTP exports remain available.
- App connection instructions and command-line documentation. All current examples use the library APIs.
build_global()and_build_accepting_stage()no longer crash with aTypeErrorwhen all surrogate candidates fail to train. They now return a structured failure state, consistent with every other non-deployable path.Router.load()now raises a clearValueErrorwhenmanifest.jsonreportsselected_method: null, instead of silently loading a stalepipeline.joblibleft on disk from a previous fit.
load_traces()now reports the correct 1-indexed file line number inValueErrormessages. Previously the count excluded blank lines, so the reported line was lower than the actual position of the bad record.
tracer.watch: a decorator that records LLM calls on the OpenTelemetry GenAI (gen_ai.*) schema, with pluggable local and generic export sinks.@tracer-llm/watch: a zero-dependency JavaScript/TypeScript mirror of the watch decorator (async-context aware), so JS pipelines get the same one-line observability as Python.- Lazy package initialization so
import tracerstays fast and only pulls in the heavy pieces when you actually use them.
- Trace ingestion accepts additional input and label aliases.
- New guide for the watch decorator.
tracer.scan(): a fast, conservative day-one read of a traces file, before any training. It groups traffic by similarity and measures, on a held-out slice it never saw, how much a near-free model can answer at your target agreement, using exact Clopper-Pearson bounds, with an optional per-1k and monthly savings estimate. Ships a self-contained HTML report with an interactive 3D map of the embedding space (hover-to-inspect cells, a Verdict/Label colour toggle, and PCA/UMAP/t-SNE layouts). Exposed astracer.scan().- Distance-based OOD safety gate: at inference the router defers inputs that fall far from the training distribution (kNN distance, global and per-predicted-label thresholds) regardless of surrogate confidence, so off-distribution traffic goes to the teacher instead of getting a confident guess.
FitConfig.skip_candidatesto drop named candidates from the zoo, including tree surrogates when a lighter sweep is desired.- Trace loaders accept common key aliases for both input and label
(
input/query/text/prompt/questionandteacher/teacher_output/label/intent/output/answer). - Bring-your-own embeddings for
tracer.scan(): pass a precomputed array withembeddings=, or select a local sentence-transformers model withmodel=.
- The parity gate now certifies on an exact held-out lower bound instead of an in-sample point estimate, so a policy cannot clear the target by in-sample luck and then break the contract on real traffic. Coverage is now monotonic in the target, and a hybrid select-then-verify procedure recovers coverage at strict targets that a plain held-out split discarded.
- The HTML report is restyled to the light Tracer theme, and the word "audit" is dropped across the report and docs.
- Non-monotonic coverage in the gate (a stricter target could deploy more coverage than a looser one).
- NaN-robustness in acceptor fitting on degenerate surrogates.
Initial public releases: the parity-gated router (fit, update, load_router,
serve), the HTML report and Sankey diagram, and the embedder factories.