Repository navigation
BUG: Return the contribution of the absorbed terms in absorbed_effects - #786
Merged
Merged
Conversation
AbsorbingLSResults.absorbed_effects is documented as the fitted values from only the absorbed terms, but it was the purged dependent variable minus the fitted values. That is the dependent variable after the effects were removed (scaled by the square root of the weights) less the fitted values of the whole model, which has one value for each observation and is no quantity of use to a user. With one categorical variable of 12 categories and 600 observations it had 600 different values, where the contribution of the effects has 12, and it differed from the right values by up to 7.5. fitted_values and resids were correct. absorbed_effects is now fitted_values - exog @ params, in the units of the dependent variable, where exog holds the regressors that were estimated. It is the contribution of the effects, the interactions and the continuous variables that were absorbed, and of any regressor that was dropped because the effects absorbed it. It equals the part of the fitted values that the dummy variables of the effects explain in the regression on the regressors and the dummy variables, with or without weights, with interactions and continuous variables, and with rows dropped because of missing values. If exog contains a constant and the effects include a categorical variable, the constant has the mean of the effects, so they have mean 0 (weighted). The docstring says so.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #786 +/- ##
==========================================
- Coverage 99.60% 99.59% -0.01%
==========================================
Files 127 129 +2
Lines 23832 24569 +737
Branches 1783 1821 +38
==========================================
+ Hits 23737 24470 +733
- Misses 35 39 +4
Partials 60 60
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AbsorbingLSResults.absorbed_effects is documented as the fitted values from only the absorbed terms, but it was the purged dependent variable minus the fitted values. That is the dependent variable after the effects were removed (scaled by the square root of the weights) less the fitted values of the whole model, which has one value for each observation and is no quantity of use to a user. With one categorical variable of 12 categories and 600 observations it had 600 different values, where the contribution of the effects has 12, and it differed from the right values by up to 7.5. fitted_values and resids were correct.
absorbed_effects is now fitted_values - exog @ params, in the units of the dependent variable, where exog holds the regressors that were estimated. It is the contribution of the effects, the interactions and the continuous variables that were absorbed, and of any regressor that was dropped because the effects absorbed it. It equals the part of the fitted values that the dummy variables of the effects explain in the regression on the regressors and the dummy variables, with or without weights, with interactions and continuous variables, and with rows dropped because of missing values. If exog contains a constant and the effects include a categorical variable, the constant has the mean of the effects, so they have mean 0 (weighted). The docstring says so.