Skip to content

Commit 1ad09d8

Browse files
baggepinnenclaude
andcommitted
test: drop flaky joint-vs-single assertion in LPV multi-dataset test
The final assertion `e1 < e2_only1` was both apples-to-oranges (joint-fit training error on d1 vs single-d1-fit held-out error on d2) and, even after fixing the names to `e2 < e2_only1`, would still be flaky: with σy = 0.005 the noise floor is ~2.5e-5, and both quantities saturate there because the true LPV system is identical across d1 and d2 and a 2-state, 22-parameter model fit to 1200 samples already recovers θ to within noise. The "joint beats single" signal does not exist in this regime — the relative ordering is just BFGS noise. The remaining assertions (shape of x0_mat, per-dataset MSE bounds) still validate that the multi-dataset path runs to completion, gives each dataset its own initial state, and explains both datasets well. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 970f1cf commit 1ad09d8

1 file changed

Lines changed: 0 additions & 12 deletions

File tree

‎test/test_lpv.jl‎

Lines changed: 0 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -131,18 +131,6 @@ end
131131
e2 = mean(abs2, d2.y .- ControlSystemIdentification.predict(sys_joint, d2, λ2; x0 = x0_mat[:, 2]))
132132
@test e1 < 0.05
133133
@test e2 < 0.05
134-
135-
# Joint fit must beat a single-dataset fit on the OTHER experiment, since
136-
# the single fit overfits its own dataset's input/noise realization.
137-
res1 = lpv_pem(d1, λ1, 2; basis,
138-
K0 = 1e-6 .* ones(2, 1),
139-
show_trace = false, store_trace = false,
140-
iterations = 200, time_limit = 60)
141-
sys_only1, x0_only1, _ = res1
142-
# Held-out prediction using sys_only1 starts from zero state, which is fair
143-
# since it never saw d2.
144-
e2_only1 = mean(abs2, d2.y .- ControlSystemIdentification.predict(sys_only1, d2, λ2))
145-
@test e1 < e2_only1
146134
end
147135

148136
@testset "LPV PEM basis-of-length-1 ≈ LTI" begin

0 commit comments

Comments
 (0)