Skip to content

Flaky POLAR_MATRICES forwarddiff test on Windows / Julia 1.12 (knot-proximity in piecewise-linear polar interpolation) #360

Description

@ufechner7

Symptom

The AutoForwardDiff matches AutoFiniteDiff (LOOP, POLAR_MATRICES) testset in test/solver/test_forwarddiff.jl fails on Windows and on Ubuntu with Julia 1.12, but passes on Ubuntu with Julia 1.13:

┌ Info: POLAR_MATRICES jacobian norms
│   norm_fwd = 3.2830924234545438
└   norm_fd = 3.2851809771070055
AutoForwardDiff matches AutoFiniteDiff (LOOP, POLAR_MATRICES): Test Failed at D:\a\VortexStepMethod.jl\VortexStepMethod.jl\test\solver\test_forwarddiff.jl:87
  Expression: relative_error(jac_fd, jac_fwd) < 0.0001
   Evaluated: 0.04771407501932747 < 0.0001

Diagnosis

This is a recurrence of a failure mode that has already been hit and patched twice:

  • 320e7565 — "Make the POLAR_MATRICES forwarddiff check robust on Windows" (widened the tolerance)
  • ae11d48d — "Make polar generation deterministic so the forwarddiff test stops flaking on Windows" (pinned BLAS.set_num_threads(1) during NeuralFoil table generation, since matmuls are only bit-reproducible single-threaded)
  • 5a1d5206 — "Keep the finite-difference reference off the polar's alpha knots" (tightened rtol to 1e-11 and the FD step to 1e-8, and the assertion back down to 1e-4) — this is the version now failing again on Windows/1.12.

Root cause: the POLAR_MATRICES aerodynamic model interpolates cl/cd/cm with Interpolations.jl's linear_interpolation over a 5°-spaced alpha grid (src/panel.jl, build_interps, calculate_cl/cd/cm) — piecewise-linear, i.e. continuous but not differentiable at each grid knot.

For the operating point used in the test (aoa_rad = deg2rad(7.5) on the ram_air_matrix_wing from test/test_data_utils.jl, grid alpha_range=deg2rad.(-5:5:15)), the LOOP solver converges at least one panel's local angle of attack to a value extremely close (empirically within ~3e-4° in the 5a1d5206 analysis) to one of the 5°/10° knots. Whether the converged value lands just above or just below that knot depends on ~1-ulp differences accumulated through sin/cos/BLAS rounding during the fixed-point LOOP solve — and those differ between platforms (Linux vs Windows libm) and Julia versions (1.12 vs 1.13's bundled math libraries). When AutoForwardDiff (dual numbers through the interpolation) and the finite-difference secant land on opposite sides of the kink, they measure two different (both locally valid) slopes of the piecewise-linear surface, producing the ~4.8% spurious mismatch seen above — not a bug in linearize(), AutoForwardDiff, or AutoFiniteDiff.

Each previous fix re-tuned constants (BLAS threading, solver rtol, FD step, assertion tolerance) against whichever platform was failing at the time, rather than removing the structural cause: the test's operating point/grid combination reliably parks a panel right at a non-differentiable point, so the test's outcome is inherently a coin flip along ~1-ulp platform differences.

Suggested direction (not yet implemented — needs Windows/Julia-1.12 to verify)

Move the interpolation grid's knots away from wherever the wing's real per-panel operating alphas cluster (e.g. offset alpha_range/delta_range in the ram_air_matrix_wing call by a non-round amount, or otherwise verify no panel's converged alpha sits near a knot), rather than continuing to tighten rtol/FD-step/tolerance, which has already been tried twice and hasn't held.

I don't have access to Windows or Julia 1.12 CI to verify a fix locally (this session is Linux/Julia 1.12.7 with a stale local dev Manifest, and even a locally-reproduced Julia 1.12 run wouldn't reproduce the Windows libm path), so a fix needs to be validated against actual CI runs on both flaky platforms.


🤖 Generated with Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions