Repository navigation
Validate FitConfig parameters and fail fast on invalid ranges - #4
Conversation
FitConfig and EmbeddingConfig previously accepted out-of-range values silently (e.g. target_teacher_agreement=1.5, empty frontier_targets, max_fit_labels=0, batch_size=0). The bad value only surfaced — if at all — as a confusing failure deep inside the fit. Add __post_init__ validation that raises ValueError with a clear, specific message at construction time, plus tests. Non-breaking: all existing valid configs still construct fine. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
adrida
left a comment
There was a problem hiding this comment.
Validated and merging. Every bound matches the field's real default and semantics: target_teacher_agreement in (0, 1], frontier_targets non-empty with each value in (0, 1], min_deploy_coverage in [0, 1], max_fit_labels > 0, explore_rate in [0, 1], retrain_every > 0, min_labels_to_start >= 0, batch_size > 0. Neither dataclass had a pre-existing __post_init__, so there's no interaction with FitConfig.embedding's default_factory. Additive, CI green across 3.9 to 3.13, and it merges cleanly on top of the two PRs that just landed.
Reasoning for the fail-fast is right: teacher-parity is the core guarantee, so the knob that controls it should reject bad input at construction with a clear message rather than skewing a fit silently or erroring far from the cause. Thank you.
One note for the record, tied to your #6: explore_rate, retrain_every, and min_labels_to_start are the three fields that issue asks about. I'm keeping them (they parameterize the online loop, more in my reply there), so validating them here is exactly what I want.
Ships everything since 0.1.1 (0.1.2 was bumped in-tree but never tagged): predict_batch() dim guard (#3), FitConfig/EmbeddingConfig fail-fast validation (#4), update() fail-fast on missing all_traces.jsonl (#5), plus the skip_candidates knob and fit() progress logging. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
What
Adds
__post_init__validation toFitConfigandEmbeddingConfigso invalidvalues are rejected at construction time with a clear message.
Why
Today these accept out-of-range values silently:
FitConfig(target_teacher_agreement=1.5)/-0.2/0.0frontier_targets, or values outside(0, 1]min_deploy_coverageoutside[0, 1],max_fit_labels=0,retrain_every=0EmbeddingConfig(batch_size=0)A bad value either silently skews the fit or surfaces much later as a confusing
error far from its cause. Since teacher-parity is the core guarantee, the knob
that controls it should fail fast and loud.
Changes
FitConfig.__post_init__/EmbeddingConfig.__post_init__raisingValueErrorwith specific messages, e.g.
target_teacher_agreement must be in (0, 1], got 1.5.tests/test_fit.pycovering rejection + that valid configs still build.Compatibility
Non-breaking — every valid config (including all existing tests and the demo)
constructs exactly as before. Full suite green locally (26 passed).