DOC: Show the Hausman and Pesaran CD tests in the panel examples note…

examples/asset-pricing_examples.ipynb

@@ -24,7 +24,7 @@
    "source": [
     "## Import data\n",
     "\n",
-    "The data used comes from Ken French\"s [website](http://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html) and includes 4 factor returns, the excess market, the size factor, the value factor and the momentum factor.  The available test portfolios include the 12 industry portfolios, a subset of the size-value two way sort, and a subset of the size-momentum two way sort. "
+    "The data used comes from Ken French's [website](http://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html) and includes 4 factor returns, the excess market, the size factor, the value factor and the momentum factor.  The available test portfolios include the 12 industry portfolios, a subset of the size-value two way sort, and a subset of the size-momentum two way sort. "
    ]
   },
   {
@@ -93,7 +93,7 @@
    "source": [
     "The factor set is expanded to include both the size and the value factors.\n",
     "\n",
-    "While the extra factors lower the J statistic and increases the $R^2$, the model is still easily rejected."
+    "While the extra factors lower the J statistic and increase the $R^2$, the model is still easily rejected."
    ]
   },
   {
@@ -191,7 +191,7 @@
    "source": [
     "## 2-Step Estimation\n",
     "\n",
-    "When one of more factors are not returns on traded assets, it is necessary to use a two-step procedure (or GMM).  In the 2-step estimator, the first estimates factor loadings and the second uses the factor loadings to estimate the risk premia.  \n",
+    "When one or more factors are not returns on traded assets, it is necessary to use a two-step procedure (or GMM).  In the 2-step estimator, the first estimates factor loadings and the second uses the factor loadings to estimate the risk premia.  \n",
     "\n",
     "Here all four factors are used to attempt to price the size-momentum portfolios."
    ]
@@ -226,7 +226,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "The identification requirements for this model are the the $\\beta$s have unique variation -- this requires some cross-sectional differences in exposures and that the loadings are not excessively correlated.  Since these test portfolios do not attempt to sort on value, it is relatively non-dispersed and also correlated with both the market and size. This might make the inference unreliable.  "
+    "The identification requirements for this model are that the $\\beta$s have unique variation -- this requires some cross-sectional differences in exposures and that the loadings are not excessively correlated.  Since these test portfolios do not attempt to sort on value, it is relatively non-dispersed and also correlated with both the market and size. This might make the inference unreliable.  "
    ]
   },
   {
@@ -265,7 +265,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "The risk-free rate can be optionally estimated.  This is useful in case excess returns are not available of if the risk-free rate used to construct the excess returns might be misspecified.\n",
+    "The risk-free rate can be optionally estimated.  This is useful in case excess returns are not available or if the risk-free rate used to construct the excess returns might be misspecified.\n",
     "\n",
     "Here the estimated risk-free rate is small and insignificant and has little impact on the model J-statistic."
    ]
@@ -316,7 +316,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "The final estimator is the GMM estimator which is similar to estimating the 2-step model in a single step.  In practice the GMM estimator is estimated at least twice, once to get an consistent estimate of the covariance of the moment conditions and the second time to efficiently estimate parameters. \n",
+    "The final estimator is the GMM estimator which is similar to estimating the 2-step model in a single step.  In practice the GMM estimator is estimated at least twice, once to get a consistent estimate of the covariance of the moment conditions and the second time to efficiently estimate parameters. \n",
     "\n",
     "The GMM estimator does not have a closed form and so a non-linear optimizer is required.  The default output prints the progress every 10 iterations.  Here the model is fit twice, which is the standard method to implement efficient GMM."
    ]

examples/asset-pricing_formulas.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "# Using Formulas\n",
     "\n",
-    "The asset pricing model estimators all all formulas.  Since the models have multiple dependent variables (test portfolios) as well as multiple independent variables (factors), the standard [formulaic](https://github.com/matthewwardrop/formulaic/) syntax needs to be modified. There are two methods to use formulas.  The first specified both the test portfolio and the factors.  The second specifies only the factors and the test portfolios are passed using an optional keyword argument. The second syntax exists since in many models the number of test portfolios might be large and interest is usually in modifying the factors.\n",
+    "The asset pricing model estimators all support formulas.  Since the models have multiple dependent variables (test portfolios) as well as multiple independent variables (factors), the standard [formulaic](https://github.com/matthewwardrop/formulaic/) syntax needs to be modified. There are two methods to use formulas.  The first specifies both the test portfolio and the factors.  The second specifies only the factors and the test portfolios are passed using an optional keyword argument. The second syntax exists since in many models the number of test portfolios might be large and interest is usually in modifying the factors.\n",
     "\n",
     "## Available Syntax \n",
     "\n",
@@ -36,7 +36,7 @@
    "source": [
     "## Import data and transform to be excess returns\n",
     "\n",
-    "The data used comes from Ken French\"s [website](http://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html) and includes 4 factor returns, the excess market, the size factor, the value factor and the momentum factor.  The available test portfolios include the 12 industry portfolios, a subset of the size-value two way sort, and a subset of the size-momentum two way sort. "
+    "The data used comes from Ken French's [website](http://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html) and includes 4 factor returns, the excess market, the size factor, the value factor and the momentum factor.  The available test portfolios include the 12 industry portfolios, a subset of the size-value two way sort, and a subset of the size-momentum two way sort. "
    ]
   },
   {

examples/panel_data-formats.ipynb

@@ -9,7 +9,7 @@
     "There are two primary methods to express data:\n",
     "\n",
     "  * MultiIndex DataFrames where the outer index is the entity and the inner is the time index.  This requires using pandas.\n",
-    "  * 3D structures were dimension 0 (outer) is variable, dimension 1 is time index and dimension 2 is the entity index.  It is also possible to use a 2D data structure with dimensions (t, n) which is treated as a 3D data structure having dimensions (1, t, n). These 3D data structures can be pandas, NumPy or xarray."
+    "  * 3D structures where dimension 0 (outer) is variable, dimension 1 is time index and dimension 2 is the entity index.  It is also possible to use a 2D data structure with dimensions (t, n) which is treated as a 3D data structure having dimensions (1, t, n). These 3D data structures can be pandas, NumPy or xarray."
    ]
   },
   {
@@ -59,7 +59,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "The `MultiIndex` `DataFrame` can be used to initialized the model.  When only referencing a single series, the `MultiIndex` `Series` representation can be used."
+    "The `MultiIndex` `DataFrame` can be used to initialize the model.  When only referencing a single series, the `MultiIndex` `Series` representation can be used."
    ]
   },
   {
@@ -79,7 +79,7 @@
    "metadata": {},
    "source": [
     "## NumPy arrays\n",
-    "3D NumPy arrays can be used to hand panel data where the three axes are `0`, items, `1`, time and `2`, entity. NumPy arrays are not usually the best format for data since the results all use generic variable names. \n",
+    "3D NumPy arrays can be used to handle panel data where the three axes are `0`, items, `1`, time and `2`, entity. NumPy arrays are not usually the best format for data since the results all use generic variable names. \n",
     "\n",
     "Pandas dropped support for their Panel in 0.25."
    ]
@@ -99,7 +99,7 @@
     "ntime = mi_data.index.levels[1].shape[0]\n",
     "np_lscrap = np_lscrap.reshape((nentity, ntime)).T\n",
     "np_hrsemp = np_hrsemp.reshape((nentity, ntime)).T\n",
-    "np_hrsemp.shape = (1, ntime, nentity)"
+    "np_hrsemp = np_hrsemp.reshape((1, ntime, nentity))"
    ]
   },
   {
@@ -120,7 +120,7 @@
    "source": [
     "## xarray DataArrays\n",
     "\n",
-    "xarray is a relatively new entrant into the set of packages used for data structures.  The data structures provided by ``xarray`` are relevant in the context of panel models since pandas `Panel` is scheduled for removal in the futures, and so the only 3d data format that will remain viable is an `xarray` `DataArray`. `DataArray`s are similar to pandas `Panel` although `DataArrays` use some difference notation.  In principle it is possible to express the same information in a `DataArray` as one can in a `Panel`"
+    "xarray is a relatively new entrant into the set of packages used for data structures.  The data structures provided by ``xarray`` are relevant in the context of panel models since pandas `Panel` has been removed, and so the only 3d data format that remains viable is an `xarray` `DataArray`. `DataArray`s are similar to pandas `Panel` although `DataArrays` use some different notation.  In principle it is possible to express the same information in a `DataArray` as one can in a `Panel`"
    ]
   },
   {
@@ -212,6 +212,33 @@
     ").fit()\n",
     "print(res)"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Aligning inputs\n",
+    "\n",
+    "The rows of `dependent` and `exog` are paired by position. An input that has labels, such as a pandas `Series` or `DataFrame`, must therefore use the same labels, in the same order, as `dependent`, and a `ValueError` is raised if it does not, since the values would be attached to the wrong observations. NumPy arrays do not have labels and are always paired by position.\n",
+    "\n",
+    "Here the regressor is sorted by year and then firm, while the dependent variable is sorted by firm and then year. `reindex` puts the rows of the regressor in the order of the dependent variable."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "hrsemp = mi_data.hrsemp.sort_index(level=\"year\")\n",
+    "try:\n",
+    "    PanelOLS(mi_data.lscrap, hrsemp, entity_effects=True)\n",
+    "except ValueError as exc:\n",
+    "    print(exc)\n",
+    "\n",
+    "hrsemp = hrsemp.reindex(mi_data.index)\n",
+    "print(PanelOLS(mi_data.lscrap, hrsemp, entity_effects=True).fit().params)"
+   ]
   }
  ],
  "metadata": {