Repository navigation
DOC: Describe JIVE, the centered J and the C statistic in the IV math… - #773
Merged
Merged
Conversation
… reference Add to the mathematical description of the IV estimators: - The jackknife IV estimator (JIVE1): the leave-one-out first stage, its closed form in terms of the leverage, the estimator and its covariance. The covariance estimators treat the leave-one-out predictions as fixed instruments. Chao, Swanson, Hausman, Newey and Woutersen (2012) show that the asymptotic variance has a second term of order K / r_n with many instruments, which they omit. In a simulation with 200 observations and 100 weak instruments, a 95% interval that uses the robust covariance covered the true value in about 85% of 2,000 replications. The section also reports the Monte Carlo evidence of Davidson and MacKinnon (2006). - The J statistic and the centered J statistic, and the relation J_c = J / (1 - J / n) that holds for the robust estimator in iterated GMM. - The definition of the C statistic that c_stat computes. Both J statistics use the covariance of the model that treats the tested variables as exogenous, and the second one uses the inverse of the sub-matrix of that covariance, which is not the sub-matrix of its inverse. - That the data are multiplied by the square root of the weights, so that the covariance, the specification tests and the first-stage statistics of a weighted model are those of the unweighted model that is fit to the weighted variables. Which specification tests ignore the covariance estimator that was used to fit the model. - The partial F statistic of the first stage. With the unadjusted covariance it is the Wald statistic divided by the number of excluded instruments, with an F distribution, and it is the classical F statistic only if the model is debiased. Otherwise it is the Wald statistic with a chi-squared distribution. - The plug-in rule of Newey and West (1994) that selects the bandwidth of a kernel estimator when none is given. The section was a TODO. Also fix the typos Andersen, Basman and Haussman in the headings, and add the references. estimators.txt is converted from estimators.lyx. LyX 2.5 and pandoc 3 write about 180 unrelated lines of the file differently from the earlier tools (align* instead of aligned, the order of subscripts and superscripts, the title lines). Only the new and changed sections are taken from a conversion of the new estimators.lyx, with the subscripts and superscripts in the order of the source, so that the diff shows the changes only.
…S and memory notes Corrections: - JIVE has no options that change the estimation, but the introduction said that it has no optional arguments and that all of the covariance estimators of the other estimators can be used. It supports the unadjusted, robust, kernel and one-way clustered covariances, selects the bandwidth of the kernel estimator using the data if none is given, raises a ValueError for two-way clusters, and has kappa None. - The weight type of the GMM estimators is 'clustered' (or 'one-way'), not 'cluster' (#631). Two-way clustering is available in IV2SLS and IVLIML only. Fix the quote of 'newey-west', "four" estimation methods (there are five) and "if" weighting matrices. - Add a paragraph on the default bandwidth of the kernel estimators of IVGMM and IVGMMCUE, which is estimated from the data in 7.1, the DefaultBandwidthWarning and how to restore the 7.0 behavior. This documents the change of the branch that makes it, so merge it with that change or after it. - The four rank tests can be imported from linearmodels.iv. New sections: - Specification Tests. Which tests assume homoskedasticity and ignore cov_type (durbin, wu_hausman, sargan, basmann, anderson_rubin, basmann_f), which are robust to heteroskedasticity (wooldridge_score, wooldridge_overid), which use the covariance estimator of the fit (wooldridge_regression), the GMM tests (j_stat, centered_j_stat, c_stat), and what f.stat of the first-stage diagnostics is: a Wald statistic divided by the number of excluded instruments with an F distribution for the unadjusted covariance, which is the textbook F only if debiased, and the Wald statistic with a chi-squared distribution otherwise (#631). - Absorbing Least Squares. What AbsorbingLS is, why it is in the IV package, and the rule of thumb for when to use it and when to use PanelOLS (#680). - Models with Many Dummy Variables. IV2SLS keeps several dense copies of the regressors, so partial out the effects instead of building the dummy variables, using AbsorbingLS, and how the degrees of freedom of the two approaches differ (#607). - Aligning Inputs, which links to the new page on the rule. Add IVJIVE to the plan page, and a pointer to the AbsorbingLS guidance in the docstring of the class.
The docstrings of the classical tests did not say that they ignore cov_type, so the same value for cov_type="unadjusted" and cov_type="robust" looked like a bug (#631). - sargan, basmann, durbin and wu_hausman assume that the errors are homoskedastic and do not use the covariance estimator that was used to fit the model. They point to wooldridge_overid, wooldridge_regression and wooldridge_score, which are robust. - The f.stat column of FirstStageResults.diagnostics is the Wald statistic divided by the number of excluded instruments, with an F distribution, if the covariance is unadjusted. It is the textbook F statistic only if debiased is True, and n / (n - k) times larger otherwise (110.08 and 88.06 for the Kmenta data of the issue, where n = 20 and k = 4). For the other covariance estimators it is the Wald statistic, with a chi-squared distribution. R reports the Wald statistic divided by the number of excluded instruments (284.55 and 142.27). The numbers were checked against R (lm, lmtest::waldtest and sandwich). The docstring of diagnostics becomes a raw string, since it has a formula.
The restrictions in the formula of wald_test are parsed by formulaic, where ":" and "[" are operators. A name that is not valid Python, such as the parameter C(year)[T.1987]:educ, raises "FormulaSyntaxError: Unknown operator ':'" (#523). It has to be enclosed in back-ticks, and the names are the entries of the index of params. Say so in the docstrings of wald_test of the panel and of the IV results, and add an example to each. The examples run, and the statistic of the panel example equals that of the matrix form of the restriction (chi2(2) = 5.42, with a p-value of 0.0665). The examples that were already in the docstring of the IV results used the name I(exper**2) for the parameter, which is named I(exper ** 2), and raised a KeyError. Use the name of the parameter.
Add a page that explains how the IV, absorbing, system and asset pricing estimators pair the rows of their inputs. It mirrors the section of the panel FAQ, and gives the rule that is new in 7.1: - An input that carries labels (a pandas object with an index other than 0, 1, ..., n-1, or an xarray DataArray with coordinates) must have the same labels, in the same order, as dependent, or a ValueError names the input. Which inputs are compared for each estimator, and what happens with missing values, formulas and clusters. - If dependent does not carry labels, the first of the other inputs that does is the reference, and the inputs after it must have the same labels. - Lists and tuples are accepted wherever a parameter is array_like, and do not carry labels. The introductions of the IV and the asset pricing models, and the system index, link to the page, and the panel FAQ gets the rule about the first input with labels. The front page lists the new specification tests and diagnostics: the rank tests of the first stage, the classical and robust tests of IV models, the Hausman test and the Pesaran CD test.
Add a Specification Tests section to the panel introduction, with an example that runs on the wage panel for each test. - hausman: what it tests, the statistic, and the HausmanTestResults that it returns. The requirements that are checked (the unadjusted covariance for both models, the same observations, dependent variable and weights), which coefficients are compared, that a negative statistic is an InvalidTestStatistic, sigmamore and sigmaless, what to do if the other model has effects that the random effects model does not, and that it is not the Wu-Hausman test of IV models. - pesaran_cd: what it tests and which series it uses, including the residuals of the differenced equation for FirstDifferenceOLS. The models that return an InapplicableTestStatistic and why (BetweenOLS, FamaMacBeth, time effects and other effects that are concentrated in some periods), with the output that says so, and that the statistic is computed from dense N by N matrices, which take about 450 MB when N is 4,000. Add the references of Hausman (1978), Pesaran (2021) and Juodis and Reese (2022). The statement about FirstDifferenceOLS describes the branch that makes the test use the differenced residuals, so merge this with that change or after it.
…cts of the panel models The docstrings of fit of the panel estimators did not mention group_debias, the default of debiased, or how PanelOLS counts the effects in the degrees of freedom (#477, #432). - The clustered options of fit of PooledOLS, PanelOLS, BetweenOLS, FirstDifferenceOLS and RandomEffects list group_debias, and debiased says that its default is True. A paragraph in the Notes says what debiased does. - debiased=True with group_debias=True gives the clustered covariance of Stata's regress ..., vce(cluster ...), that is g / (g - 1) * (n - 1) / (n - k). It agrees to ten digits with sandwich::vcovCL(type = "HC1", cadjust = TRUE) in R, on the wage panel and on simulated data with 2, 10 and 200 clusters, and the tests of the package against Stata use these options. - PanelOLS counts the estimated effects, d, in the degrees of freedom whether or not debiased is True: the covariance is scaled by n / (n - d - k) or by n / (n - d). With debiased=True the standard errors are those of the regression that has dummy variables for the effects. They are not counted when the model has a single effect that is nested in the clusters (auto_df). The panel FAQ gets sections on the small-sample adjustments, matching Stata, counting the effects, and the estimated effects. estimated_effects (#559, #553, #392): - It is the combined effect only, in one column, and it is NaN or 0 for models without effects. - It has no inference, since the effects are not estimated consistently when T is fixed. In a simulation with T = 5 the standard deviation of the error of the effects was 0.441, 0.452 and 0.445 with 200, 2,000 and 20,000 entities (sigma / sqrt(T) = 0.447). - The effects of RandomEffects are all 0 if the estimated variance of the random effect is negative and set to 0, since the model is then pooled OLS. - A recipe that splits the entity and the time effects of a model that has both, exactly (the error is 1e-13, also with 25% of the rows dropped). A short section points from the panel FAQ to the guidance on AbsorbingLS.
The notes below the tables of a summary, for example the entity effects that were included or the instruments, are the extra_txt attribute of the Summary that the results return. The class had no docstring, so nothing said that setting extra_txt to None removes the notes from as_text, as_html, as_latex and as_csv (#560). Add the docstring, which says that the summary property of the results creates a new Summary at every access, so that it has to be kept in a variable before extra_txt is changed, and add an example.
Add partialling and autocovariances, and the first names in the new references.
…e Notes The sentence that points to the guidance on when to use AbsorbingLS and when to use PanelOLS was between two paragraphs that other changes of the class docstring (the absorbed variables and the labels of the inputs) also edit, so the changes would conflict. The start of the Notes is not edited by them.
The rsquared of the results of AbsorbingLS includes the explanatory power of the absorbed effects, as the R-squared of the regression with dummy variables does, and absorbed_rsquared ignores them, as the within R-squared of PanelOLS does. They are 0.9527 and 0.7253 for the Grunfeld data with firm and year effects.
…astic errors Add a simulation to the specification tests of the IV introduction. With 400 observations, a regressor that is exogenous, three valid instruments and errors with the standard deviation exp(0.9 z_1), where z_1 is the first instrument, the tests that assume homoskedastic errors rejected a true null at the 5% level in 14% to 15% of 1,500 samples (durbin 15.4%, wu_hausman 15.2%, sargan 14.0% and basmann 14.0%). The robust tests rejected in 4% to 5% (wooldridge_score 4.8%, wooldridge_regression with the robust covariance 5.1% and wooldridge_overid 4.1%).
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #773 +/- ##
==========================================
+ Coverage 99.58% 99.59% +0.01%
==========================================
Files 128 128
Lines 24188 24435 +247
Branches 1809 1820 +11
==========================================
+ Hits 24087 24336 +249
+ Misses 40 39 -1
+ Partials 61 60 -1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
… reference
Add to the mathematical description of the IV estimators:
closed form in terms of the leverage, the estimator and its covariance.
The covariance estimators treat the leave-one-out predictions as fixed
instruments. Chao, Swanson, Hausman, Newey and Woutersen (2012) show that
the asymptotic variance has a second term of order K / r_n with many
instruments, which they omit. In a simulation with 200 observations and
100 weak instruments, a 95% interval that uses the robust covariance
covered the true value in about 85% of 2,000 replications. The section also
reports the Monte Carlo evidence of Davidson and MacKinnon (2006).
J_c = J / (1 - J / n) that holds for the robust estimator in iterated GMM.
use the covariance of the model that treats the tested variables as
exogenous, and the second one uses the inverse of the sub-matrix of that
covariance, which is not the sub-matrix of its inverse.
covariance, the specification tests and the first-stage statistics of a
weighted model are those of the unweighted model that is fit to the weighted
variables. Which specification tests ignore the covariance estimator that
was used to fit the model.
it is the Wald statistic divided by the number of excluded instruments, with
an F distribution, and it is the classical F statistic only if the model is
debiased. Otherwise it is the Wald statistic with a chi-squared distribution.
kernel estimator when none is given. The section was a TODO.
Also fix the typos Andersen, Basman and Haussman in the headings, and add the
references.
estimators.txt is converted from estimators.lyx. LyX 2.5 and pandoc 3 write
about 180 unrelated lines of the file differently from the earlier tools
(align* instead of aligned, the order of subscripts and superscripts, the
title lines). Only the new and changed sections are taken from a conversion
of the new estimators.lyx, with the subscripts and superscripts in the order of
the source, so that the diff shows the changes only.