Skip to content

Skip zero-weight rows in MicroSeries.quantile - #287

Merged
MaxGhenis merged 1 commit into
mainfrom
fix/quantile-zero-weight-first-row
Apr 17, 2026
Merged

Skip zero-weight rows in MicroSeries.quantile#287
MaxGhenis merged 1 commit into
mainfrom
fix/quantile-zero-weight-first-row

Conversation

@MaxGhenis

Copy link
Copy Markdown
Collaborator

Summary

MicroSeries.quantile could select a zero-weight element because np.searchsorted(cumsum_norm, 0, side='left') always returns 0, and the same class of bug fired at internal plateaus of zero weight. The fix drops zero-weight rows before sorting and computing the CDF; an all-zero-weight series returns NaN.

Reproduction

import microdf as mdf

MicroSeries([10, 20, 30], weights=[0, 1, 1]).quantile(0.0)  # was 10, now 20

Test plan

  • Existing 51 tests still pass
  • New test_quantile_skips_zero_weight_rows covers:
    • Zero-weight first row
    • Internal zero-weight plateau (the zero-weight value must never be selected across q in [0, 1])
    • All zero weights → NaN

@MaxGhenis MaxGhenis left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review. Verified:

  • nonzero = sample_weight > 0 filter applied before the CDF search; if no rows have positive weight, return np.nan (scalar q) or a NaN-filled Series (array q) — documented behaviour.
  • Fix addresses both the q=0 case and internal zero-weight plateaus because filtering happens before argsort/cumsum.
  • Regression test covers q=0 with first-row zero weight, internal plateau (sweeping np.linspace(0,1,21) to assert the zero-weight value never appears), and all-zero-weight → NaN.
  • Does not perturb the correct-weight path (all tests still pass).
  • CI green Py 3.9-3.13.

Previously, when the first (or an internal) sorted element had weight 0,
the inverse-CDF search still picked it. E.g.

  MicroSeries([10, 20, 30], weights=[0, 1, 1]).quantile(0)

returned 10 instead of 20. The fix drops zero-weight rows before
sorting and computing the CDF. An all-zero-weight series returns NaN
(previously raised).
@MaxGhenis
MaxGhenis force-pushed the fix/quantile-zero-weight-first-row branch from 79013c9 to 71ce6e1 Compare April 17, 2026 16:16
@MaxGhenis
MaxGhenis merged commit e277a16 into main Apr 17, 2026
8 checks passed
@MaxGhenis
MaxGhenis deleted the fix/quantile-zero-weight-first-row branch April 17, 2026 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant