[Repo Assist] perf(gcm/stats): vectorise merge_p_values_average from O(K2) to O(K)

[github-actions]

🤖 This PR was created by Repo Assist, an automated AI assistant.

Problem

merge_p_values_average in dowhy/gcm/stats.py used a list comprehension that recomputed np.mean(p_values[:m]) from scratch for each m in range(1, K+1), giving O(K2) total work:

return min(
    1.0, float(np.min([2 * np.mean(p_values[:m]) / (2 - (K * u / m))
                       for m in range(1, K + 1) if (K * u / m) < 2]))
)

For large K (e.g. when merging p-values from many bootstrap resamples, independence tests across many variables, or large graph falsification runs), this quadratic scaling is a bottleneck.

Fix

Replace the loop with a vectorised O(K) implementation using np.cumsum:

m_arr = np.arange(1, K + 1, dtype=float)
mask = (K * u) < (2 * m_arr)          # equivalent filter condition
m_valid = m_arr[mask]
prefix_sums = np.cumsum(p_values)      # O(K) — all prefix sums at once
means = prefix_sums[mask] / m_valid    # all prefix means in one shot
values = 2 * means / (2 - (K * u / m_valid))
return min(1.0, float(np.min(values)))

The cumulative sum provides all prefix sums in a single O(K) pass; the remaining filter, division and minimum are also vectorised NumPy operations.

Correctness

The new implementation is mathematically identical to the original. Verified against all existing test cases manually:

Input	Expected	Old	New
[0]	0.0	0.0	0.0 ✅
[1]	1.0	1.0	1.0 ✅
[0.3]	0.6	0.6	0.6 ✅
[0, 1]	1.0	1.0	1.0 ✅
[0, 0, 1]	0.0	0.0	0.0 ✅
[0, 0.5, 0.5, nan, 1, nan]	≈1.0	1.0	1.0 ✅
[0, 0, 1, 1, 1]	1.0	1.0	1.0 ✅

Test Status

• ✅ black --check passes on modified file
• ✅ flake8 — no new violations introduced (pre-existing violations in other functions remain)
• ⚠️ Full test suite requires poetry install (causallearn and other transitive deps not available in the CI-lite environment); the mathematical equivalence is verified by the manual test above and the existing tests in tests/gcm/test_stats.py

Generated by 🌈 Repo Assist, see workflow run. Learn more.

Generated by 🌈 Repo Assist, see workflow run. Learn more.

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/repo-assist.md@11c9a2c442e519ff2b427bf58679f5a525353f76