Skip to content

hypot[pedantic] answers 0 for (nan, 0) #2384

Description

@jfalcou

eve::hypot[eve::pedantic](nan, 0.0) answers 0. It should answer nan, which is what
eve::hypot and std::hypot both answer.

auto nan = eve::nan(eve::as<double>());

eve::hypot[eve::pedantic](nan, 0.0)   // 0    <- expected nan
eve::hypot[eve::pedantic](0.0, nan)   // nan
eve::hypot[eve::pedantic](nan, 1.0)   // nan

eve::hypot(nan, 0.0)                  // nan
std::hypot(nan, 0.0)                  // nan

Same on wide<double>, lane by lane.

The two lines are in the Borges algorithm. It opens by sorting the magnitudes:

auto test = ax > ay;
eve::swap_if(test, ax, ay); // now 0 <= ax <= ay

Every comparison against a nan is false, so the swap never happens and a nan stays in
ax. The invariant the comment states does not hold. Further down, the tail case relies
on it:

h = if_else(is_eqz(ay), zero, h);

Reading it as "the larger one is zero, so both are" is only true under that invariant.
With the nan left in ax and ay zero, the line overwrites the nan with zero. Reversing
the arguments puts the nan in ay, is_eqz is false, and the answer comes out right,
which is where the asymmetry comes from.

Downstream, kyosu::abs is hypot[pedantic] over the components, so kyosu::abs(nan + 0i)
is 0, and kyosu::dist and kyosu::reldist follow. A suite comparing through them reads
a nan as equal to any finite value whenever the difference carries a zero imaginary part.

Measured with g++-14 -O0 on x86-64, scalar and wide<double>.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions