Repository navigation
PLS Notes
Brian Pho edited this page Sep 28, 2021
·
9 revisions
- SIMPLS: An alternative approach to partial least squares regression
- https://stats.stackexchange.com/questions/286178/python-sklearn-plsregression-why-is-t-x0w-scores-scaled-data-weights
- https://stats.stackexchange.com/questions/179733/theory-behind-partial-least-squares-regression
- https://stats.stackexchange.com/questions/269306/understanding-nipals-algorithm-for-pls
- Partial least-squares regression: a tutorial
Papers
- Sklearn PLS Implementation
- Partial least squares with a regularized weight
-
Regularized Partial Least Squares with an Application to NMR Spectroscopy
The deflation approach employed via the NIPALS algorithm is not as direct. While one can incorporate regularization into the loadings, wk (as suggested by Chun and Keleş [4]), this is not as desirable.
- PLSRegression is also known as PLS1 (single targets) and PLS2 (multiple targets).
- PLS1 is a special case of PLS2 with one of the blocks consisting of a single variable.
- PLS1, the case of PLS2 where Y is a single column, is a regularization technique, in the same class as ridge regression and principal component regression.
- At each iteration of the outer loop, PLS1 finds a direction in the space of regression coefficients orthogonal to the previous regression coefficients. Regression here is of the single response variable Y on the predictor variables X.
- PLS1 only runs one iteration of the Power method.
- https://www.wikiwand.com/en/Partial_least_squares_regression#/PLS1
- PLS1 deflates matrix X but deflation of vector y isn't performed (not necessary, deflation yields the same results as not deflating).
- The user-supplied variable is the number of latent factors/components in the regression. If it equals the rank of the matrix X, the algorithm will yield the least squares regression estimates.
- The regularization reduces the influence of correlated variables on the model because the weight is shared between the two predictive variables, so neither alone would have strong weights.
- On the other hand, the weights obtained with regularization are more stable.