Merge pull request #805 from bashtage/lm71-fix-factor-model-inference

BUG: Invert the covariance of the pricing errors on its N - K - nrf d…

Author: bashtage

doc/source/asset-pricing/index.rst

@@ -22,9 +22,10 @@ model and allows the specification to be testes with a J-statistic of the form
 
 .. math::
 
-    J = \hat{\alpha}^{\prime} \hat{\Sigma}_{\alpha} \hat{\alpha} \sim \chi^2_P
+    J = \hat{\alpha}^{\prime} \hat{\Sigma}_{\alpha}^{-1} \hat{\alpha} \sim \chi^2_P
 
-where there are P portfolios.
+where there are P portfolios and :math:`\hat{\Sigma}_{\alpha}` is the covariance
+of the estimated pricing errors.
 
 When some factors are not-traded, for example macroeconomic shocks, then a 2-step
 procedure is required since the average value of the factors does not
@@ -37,9 +38,13 @@ has a different distribution,
 
 .. math::
 
-    J = \hat{\alpha}^{\prime} \hat{\Sigma}_{\alpha} \hat{\alpha} \sim \chi^2_{P-K}
+    J = \hat{\alpha}^{\prime} \hat{\Sigma}_{\alpha}^{+} \hat{\alpha} \sim \chi^2_{P-K-n_{rf}}
 
-where K is the number of factors.
+where K is the number of factors, :math:`n_{rf}` is 1 if the risk-free rate is
+estimated and 0 if it is not, and :math:`\hat{\Sigma}_{\alpha}^{+}` is the
+generalized inverse of the covariance of the pricing errors, which has rank
+:math:`P-K-n_{rf}` since the pricing errors satisfy :math:`K+n_{rf}` linear
+restrictions.
 :class:`~linearmodels.asset_pricing.model.LinearFactorModel` implements this
 two-step estimator.
 :class:`~linearmodels.asset_pricing.model.LinearFactorModelGMM` contains a more

doc/source/asset-pricing/introduction.rst

@@ -49,7 +49,7 @@ prior to estimation.
                           LinearFactorModel Estimation Summary
     ================================================================================
     No. Test Portfolios:                  6   R-squared:                      0.8879
-    No. Factors:                          3   J-statistic:                    39.109
+    No. Factors:                          3   J-statistic:                    30.291
     No. Observations:                   819   P-value                         0.0000
     Date:                  Sun, May 21 2017   Distribution:                  chi2(3)
     Time:                          21:18:56

doc/source/asset-pricing/mathematical-detail.lyx

@@ -231,6 +231,27 @@ where
  The reported risk premia are just the average returns on the factors, and
  the covariance of the risk premia estimates is based on the factor residuals
  after demeaning using either the simple estimator or the kernel estimator.
+ If the covariance is debiased, 
+\begin_inset Formula $df=K+1$
+\end_inset
+
+ is the number of regressors of each regression.
+ The risk premia are sample means, so that their covariance uses 
+\begin_inset Formula $T-1$
+\end_inset
+
+ in place of 
+\begin_inset Formula $T-df$
+\end_inset
+
+, and the covariance between the risk premia and the other parameters uses
+ the geometric mean of the two divisors, 
+\begin_inset Formula $\sqrt{\left(T-df\right)\left(T-1\right)}$
+\end_inset
+
+.
+ The block of the covariance of the parameters for the risk premia is
+ then the square of their reported standard errors.
 
 \end_layout
 
@@ -724,18 +745,19 @@ The J statistic
 \end_layout
 
 \begin_layout Standard
-The J statistic is virtually identical to the 
+The J statistic is similar to the 
 \begin_inset Formula $J$
 \end_inset
 
- statistic from traded factor model except that it has a difference distribution.
+ statistic from traded factor model except that the pricing errors satisfy
+ linear restrictions and that the statistic has a different distribution.
 \end_layout
 
 \begin_layout Standard
 \noindent
 \begin_inset Formula 
 \[
-J=\hat{\alpha}^{\prime}\hat{\Sigma}_{\alpha}^{-1}\hat{\alpha}^{\prime}\sim\chi_{N-K}^{2}
+J=\hat{\alpha}^{\prime}\hat{\Sigma}_{\alpha}^{+}\hat{\alpha}=\left(Q^{\prime}\hat{\alpha}\right)^{\prime}\left(Q^{\prime}\hat{\Sigma}_{\alpha}Q\right)^{-1}\left(Q^{\prime}\hat{\alpha}\right)\sim\chi_{N-K-n_{rf}}^{2}
 \]
 
 \end_inset
@@ -745,32 +767,102 @@ J=\hat{\alpha}^{\prime}\hat{\Sigma}_{\alpha}^{-1}\hat{\alpha}^{\prime}\sim\chi_{
 
 \begin_layout Standard
 \noindent
-The loss of degrees of freedom occurs since 
+where 
+\begin_inset Formula $n_{rf}$
+\end_inset
+
+ is 1 if the risk free rate is estimated and 0 if it is not. The loss of
+ degrees of freedom occurs since 
 \begin_inset Formula $\hat{\alpha}=\hat{\eta}$
 \end_inset
 
  which are regression residuals from a regression of 
 \begin_inset Formula $N$
 \end_inset
 
- average returns on 
-\begin_inset Formula $K$
+ average returns on the 
+\begin_inset Formula $K+n_{rf}$
 \end_inset
 
- factor loadings.
+ columns of the factor loadings. The second stage estimates make 
+\begin_inset Formula $\hat{\alpha}$
+\end_inset
+
+ orthogonal to the 
+\begin_inset Formula $K+n_{rf}$
+\end_inset
+
+ columns of 
+\begin_inset Formula $\Sigma^{-1}\hat{\beta}_{c}$
+\end_inset
+
+, where 
+\begin_inset Formula $\Sigma$
+\end_inset
+
+ is the covariance used in the GLS version or the identity matrix, so that 
+\begin_inset Formula $\hat{\alpha}$
+\end_inset
+
+ lies in a space with 
+\begin_inset Formula $N-K-n_{rf}$
+\end_inset
+
+ dimensions and its covariance has rank 
+\begin_inset Formula $N-K-n_{rf}$
+\end_inset
+
+ in theory. 
+\begin_inset Formula $\hat{\Sigma}_{\alpha}$
+\end_inset
+
+ is the block of 
+\begin_inset Formula $\hat{\Sigma}$
+\end_inset
+
+ that contains the covariance of 
+\begin_inset Formula $\hat{\alpha}$
+\end_inset
+
+. The columns of 
+\begin_inset Formula $Q$
+\end_inset
+
+ are an orthonormal basis of the space that is orthogonal to the columns of
 
+\begin_inset Formula $\Sigma^{-1}\hat{\beta}_{c}$
+\end_inset
+
+, and 
+\begin_inset Formula $\hat{\Sigma}_{\alpha}^{+}=Q\left(Q^{\prime}\hat{\Sigma}_{\alpha}Q\right)^{-1}Q^{\prime}$
+\end_inset
+
+ is the generalized inverse of 
 \begin_inset Formula $\hat{\Sigma}_{\alpha}$
 \end_inset
 
- is the bottom right 
-\begin_inset Formula $K\times K$
+ with rank 
+\begin_inset Formula $N-K-n_{rf}$
 \end_inset
 
- block of 
-\begin_inset Formula $\hat{\Sigma}$
+. The statistic is the Wald statistic of the 
+\begin_inset Formula $N-K-n_{rf}$
 \end_inset
 
-.
+ linear combinations of the pricing errors that are not restricted to be
+ zero. If 
+\begin_inset Formula $\hat{\Sigma}_{\alpha}$
+\end_inset
+
+ were exactly singular in the directions of the columns of 
+\begin_inset Formula $\Sigma^{-1}\hat{\beta}_{c}$
+\end_inset
+
+, as the covariance of the pricing errors is in theory, 
+\begin_inset Formula $\hat{\Sigma}_{\alpha}^{+}$
+\end_inset
+
+ would be its Moore-Penrose pseudo-inverse.
 \end_layout
 
 \begin_layout Subsection
@@ -873,6 +965,71 @@ f_{c,t}\left[1\!\hat{\beta}_{N}\right]
 
 \end_layout
 
+\begin_layout Standard
+The pricing errors, 
+\begin_inset Formula $\hat{\alpha}$
+\end_inset
+
+, are the averages of the first moment condition of each portfolio, 
+\begin_inset Formula $\epsilon_{t}$
+\end_inset
+
+. The moment conditions are not zero at the estimates of an over-identified
+ model, and the covariance of their average accounts for the estimation of
+ the parameters.  For efficient GMM it is
+\end_layout
+
+\begin_layout Standard
+\begin_inset Formula 
+\[
+\hat{\Sigma}_{g}=S-G\left(G^{\prime}S^{-1}G\right)^{-1}G^{\prime}
+\]
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+\noindent
+where 
+\begin_inset Formula $S$
+\end_inset
+
+ and 
+\begin_inset Formula $G$
+\end_inset
+
+ are those of the covariance of the parameters.  Both are divided by 
+\begin_inset Formula $T-df$
+\end_inset
+
+, where 
+\begin_inset Formula $df$
+\end_inset
+
+ is the number of factors if the covariance is debiased and 0 if it is not.
+  The standard errors of the pricing errors are the square roots of the
+ diagonal elements of 
+\begin_inset Formula $\hat{\Sigma}_{g}/\left(T-df\right)$
+\end_inset
+
+ that correspond to 
+\begin_inset Formula $\hat{\alpha}$
+\end_inset
+
+.  This covariance has rank 
+\begin_inset Formula $N-K-n_{rf}$
+\end_inset
+
+, the degrees of freedom of the J statistic, where 
+\begin_inset Formula $n_{rf}$
+\end_inset
+
+ is 1 if the risk free rate is estimated and 0 if it is not.
+\end_layout
+
+
 \begin_layout Standard
 \begin_inset Note Note
 status collapsed

doc/source/asset-pricing/mathematical-detail.txt

@@ -48,6 +48,13 @@ and :math:`bw` is the bandwidth. The reported risk premia are just the
 average returns on the factors, and the covariance of the risk premia
 estimates is based on the factor residuals after demeaning using either
 the simple estimator or the kernel estimator.
+If the covariance is debiased, :math:`df=K+1` is the number of regressors of
+each regression. The risk premia are sample means, so that their covariance
+uses :math:`T-1` in place of :math:`T-df`, and the covariance between the
+risk premia and the other parameters uses the geometric mean of the two
+divisors, :math:`\sqrt{\left(T-df\right)\left(T-1\right)}`. The block of the
+covariance of the parameters for the risk premia is then the square of their
+reported standard errors.
 
 The J statistic
 ~~~~~~~~~~~~~~~
@@ -179,16 +186,34 @@ previous section.
 The J statistic
 ~~~~~~~~~~~~~~~
 
-The J statistic is virtually identical to the :math:`J` statistic from
-traded factor model except that it has a difference distribution.
+The J statistic is similar to the :math:`J` statistic from traded factor
+model except that the pricing errors satisfy linear restrictions and that the
+statistic has a different distribution.
 
-.. math:: J=\hat{\alpha}^{\prime}\hat{\Sigma}_{\alpha}^{-1}\hat{\alpha}^{\prime}\sim\chi_{N-K}^{2}
+.. math:: J=\hat{\alpha}^{\prime}\hat{\Sigma}_{\alpha}^{+}\hat{\alpha}=\left(Q^{\prime}\hat{\alpha}\right)^{\prime}\left(Q^{\prime}\hat{\Sigma}_{\alpha}Q\right)^{-1}\left(Q^{\prime}\hat{\alpha}\right)\sim\chi_{N-K-n_{rf}}^{2}
 
-The loss of degrees of freedom occurs since
+where :math:`n_{rf}` is 1 if the risk free rate is estimated and 0 if it is
+not. The loss of degrees of freedom occurs since
 :math:`\hat{\alpha}=\hat{\eta}` which are regression residuals from a
-regression of :math:`N` average returns on :math:`K` factor loadings.
-:math:`\hat{\Sigma}_{\alpha}` is the bottom right :math:`K\times K`
-block of :math:`\hat{\Sigma}`.
+regression of :math:`N` average returns on the :math:`K+n_{rf}` columns of
+the factor loadings. The second stage estimates make :math:`\hat{\alpha}`
+orthogonal to the :math:`K+n_{rf}` columns of
+:math:`\Sigma^{-1}\hat{\beta}_{c}`, where :math:`\Sigma` is the covariance
+used in the GLS version or the identity matrix, so that
+:math:`\hat{\alpha}` lies in a space with :math:`N-K-n_{rf}` dimensions and
+its covariance has rank :math:`N-K-n_{rf}` in theory.
+:math:`\hat{\Sigma}_{\alpha}` is the block of :math:`\hat{\Sigma}` that
+contains the covariance of :math:`\hat{\alpha}`. The columns of :math:`Q`
+are an orthonormal basis of the space that is orthogonal to the columns of
+:math:`\Sigma^{-1}\hat{\beta}_{c}`, and
+:math:`\hat{\Sigma}_{\alpha}^{+}=Q\left(Q^{\prime}\hat{\Sigma}_{\alpha}Q\right)^{-1}Q^{\prime}`
+is the generalized inverse of :math:`\hat{\Sigma}_{\alpha}` with rank
+:math:`N-K-n_{rf}`. The statistic is the Wald statistic of the
+:math:`N-K-n_{rf}` linear combinations of the pricing errors that are not
+restricted to be zero. If :math:`\hat{\Sigma}_{\alpha}` were exactly singular
+in the directions of the columns of :math:`\Sigma^{-1}\hat{\beta}_{c}`, as the
+covariance of the pricing errors is in theory,
+:math:`\hat{\Sigma}_{\alpha}^{+}` would be its Moore-Penrose pseudo-inverse.
 
 GMM Estimation of Linear Factor Models
 --------------------------------------
@@ -238,3 +263,20 @@ previous section. The parameter covariance estimator uses the form
 appropriate for efficient GMM,
 
 .. math:: \hat{\Sigma}=\left(\hat{G}\hat{^{\prime}S}^{-1}\hat{G}\right)^{-1}.
+
+The pricing errors, :math:`\hat{\alpha}`, are the averages of the first moment
+condition of each portfolio, :math:`\epsilon_{t}`. The moment conditions are
+not zero at the estimates of an over-identified model, and the covariance of
+their average accounts for the estimation of the parameters. For efficient GMM
+it is
+
+.. math:: \hat{\Sigma}_{g}=S-G\left(G^{\prime}S^{-1}G\right)^{-1}G^{\prime}
+
+where :math:`S` and :math:`G` are those of the covariance of the parameters.
+Both are divided by :math:`T-df`, where :math:`df` is the number of factors
+if the covariance is debiased and 0 if it is not. The standard errors of the
+pricing errors are the square roots of the diagonal elements of
+:math:`\hat{\Sigma}_{g}/\left(T-df\right)` that correspond to
+:math:`\hat{\alpha}`. This covariance has rank :math:`N-K-n_{rf}`, the degrees
+of freedom of the J statistic, where :math:`n_{rf}` is 1 if the risk free rate
+is estimated and 0 if it is not.

examples/asset-pricing_examples.ipynb

@@ -267,7 +267,7 @@
    "source": [
     "The risk-free rate can be optionally estimated.  This is useful in case excess returns are not available or if the risk-free rate used to construct the excess returns might be misspecified.\n",
     "\n",
-    "Here the estimated risk-free rate is small and insignificant and has little impact on the model J-statistic."
+    "Here the estimated risk-free rate is small and insignificant, and the model is still rejected."
    ]
   },
   {
@@ -292,7 +292,7 @@
    "source": [
     "The default covariance estimator allows for heteroskedasticity but assumes there is no autocorrelation in the model errors or factor returns.  Kernel-based HAC estimators (e.g. Newey-West) can be used by setting `cov_type=\"kernel\"`.\n",
     "\n",
-    "This reduces the J-statistic indicating there might be some serial correlation although the model is still firmly rejected."
+    "This reduces the J-statistic indicating there might be some serial correlation although the model is still rejected."
    ]
   },
   {