shuffle_test (discovery.py:903) builds its null by permuting the rows of X while leaving Z fixed:
X_perm = X[rng.permutation(len(X)), :]
This destroys the X–Z dependence as well as the X–Y dependence, so the null is "X independent of (Y, Z)", not H0: X, Y | Z. When the candidate parent is correlated with the conditioning set — the normal situation in an autocorrelated network time series — this inflates false positives.
A restricted/local permutation scheme (permuting within Z-neighborhoods, as in Runge's CMIknn) is the standard remedy. This is consistent with the false-positive rates I measured in the Gaussian path below:
Related: p_value = np.mean(null_cmi >= observed_cmi) can be exactly 0, and the package does report P_Value = 0.0 to users. The standard estimator (1 + #{null ≥ obs}) / (1 + n_shuffles) should be used.
shuffle_test(discovery.py:903) builds its null by permuting the rows ofXwhile leavingZfixed:This destroys the X–Z dependence as well as the X–Y dependence, so the null is "X independent of (Y, Z)", not
H0: X, Y | Z. When the candidate parent is correlated with the conditioning set — the normal situation in an autocorrelated network time series — this inflates false positives.A restricted/local permutation scheme (permuting within Z-neighborhoods, as in Runge's CMIknn) is the standard remedy. This is consistent with the false-positive rates I measured in the Gaussian path below:
Related:
p_value = np.mean(null_cmi >= observed_cmi)can be exactly 0, and the package does reportP_Value = 0.0to users. The standard estimator(1 + #{null ≥ obs}) / (1 + n_shuffles)should be used.