Repository navigation
BUG: Reject missing values and a single group in the cluster ids - #789
Merged
Merged
Conversation
cov_cluster computes the clustered score covariance of the system, IV and panel models. It accepted two kinds of cluster ids that give a meaningless result: * Missing values. np.unique counts all the missing values as one group, but the group boundaries found from the sorted ids treat each missing value as a group of its own. The loop over the groups then added the scores of the first observation with a missing cluster and left out the scores of all the others, without an error. In an example with 60 observations and 5 missing ids, the covariance was exactly that of the clusters of the 55 observations with an id plus the first of the other five as a cluster of its own, which is neither a covariance that drops the five observations nor one that treats each of them as a cluster. * A single group. The scores sum to zero at the estimates, so the covariance is zero up to rounding. SUR with a single cluster reported standard errors of 2e-16 and PooledOLS 9e-17, while SUR with group_debias=True and the IV models raised a bare ZeroDivisionError. cov_cluster now raises a ValueError for both. The ids cannot be dropped with the rows of the data that have missing values, since the model was estimated with them, so an error is the only result that is not silently wrong. As the check is in the shared function, every model that clusters inherits it. No test relied on either input. The tests call cov_cluster directly, with float and object ids, and the system covariance estimators with one-way and two-way clusters. They fail without the change.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #789 +/- ##
==========================================
+ Coverage 99.58% 99.59% +0.01%
==========================================
Files 128 128
Lines 24188 24473 +285
Branches 1809 1823 +14
==========================================
+ Hits 24087 24374 +287
+ Misses 40 39 -1
+ Partials 61 60 -1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
cov_cluster computes the clustered score covariance of the system, IV and panel models. It accepted two kinds of cluster ids that give a meaningless result:
cov_cluster now raises a ValueError for both. The ids cannot be dropped with the rows of the data that have missing values, since the model was estimated with them, so an error is the only result that is not silently wrong. As the check is in the shared function, every model that clusters inherits it. No test relied on either input.
The tests call cov_cluster directly, with float and object ids, and the system covariance estimators with one-way and two-way clusters. They fail without the change.