Skip to content

Commit 14b001e

Browse files
Mamba413claudehappy-otter
committed
fix: correct IRLS Hessian for weighted logistic regression
Two bugs in _abessGLM::_IRLS_fit and abessLogistic::hessian_core caused the weighted IRLS to give different results from fitting on repeated data, breaking sklearn's check_sample_weight_equivalence. Bug 1 (hessian_core): Pi*(1-Pi)*sw was truncated to [1e-3, 1] as a whole. When a near-saturated sample (Pi*(1-Pi) < 1e-3) had sw > 1, the lower clamp replaced the product with 1e-3, losing the sample weight. This made X_new^T*X_full differ between the weighted (n=9, sw=[3,4,...]) and repeated (n=27, all sw=1) cases at saturation. Fix: truncate Pi*(1-Pi) BEFORE multiplying by sw, so near-saturated rows keep their correct relative weight (1e-3 * sw_i vs 1e-3 * 1). Bug 2 (safe D_bare division): The original D.cwiseQuotient(weights) causes 0/0=NaN when sw_i=0 (even if D_i=0 after the hessian fix). Since X_new[i,:]=0 for zero-weight rows, Z[i] can be any finite value. Fix: use 1.0 as placeholder when sw_i=0 to avoid NaN propagation in the matrix products. Together these fixes ensure X_new^T*Z (gradient) and X_new^T*X_full (Hessian) are numerically equivalent for weighted and repeated-data fits, allowing sklearn's check_sample_weights_equivalence to pass. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>
1 parent d24f340 commit 14b001e

1 file changed

Lines changed: 11 additions & 4 deletions

File tree

src/AlgorithmGLM.h

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -254,8 +254,13 @@ class _abessGLM : public Algorithm<T1, T2, T3, T4> {
254254
T1 y_pred = this->inv_link_function(X_full, beta_full);
255255
T1 Z = y - y_pred;
256256
// D_i = h(eta_i) * sw_i; working response needs D_bare_i = h(eta_i) without sw,
257-
// so that X_new^T * Z = sum_i sw_i * x_i * (y_i - mu_i) (correctly weighted score)
258-
Eigen::VectorXd D_bare = D.cwiseQuotient(weights);
257+
// so that X_new^T * Z = sum_i sw_i * x_i * (y_i - mu_i) (correctly weighted score).
258+
// Use safe division: when sw_i=0, D_i=0 and X_new row will be 0, so D_bare can be
259+
// any finite value (1.0 chosen); avoids 0/0=NaN which would propagate in X_new^T*Z.
260+
Eigen::VectorXd D_bare(n);
261+
for (int i = 0; i < n; i++) {
262+
D_bare(i) = (weights(i) > 0) ? D(i) / weights(i) : 1.0;
263+
}
259264
array_quotient(Z, D_bare, 1);
260265
Z += X_full * beta_full;
261266
for (int i = 0; i < X_full.cols(); i++) {
@@ -316,9 +321,11 @@ class abessLogistic : public _abessGLM<Eigen::VectorXd, Eigen::VectorXd, double,
316321
Eigen::VectorXd hessian_core(T4 &X_full, Eigen::VectorXd &y, Eigen::VectorXd &weights, Eigen::VectorXd &beta_full) {
317322
Eigen::VectorXd Pi = this->inv_link_function(X_full, beta_full);
318323
Eigen::VectorXd one = Eigen::VectorXd::Ones(X_full.rows());
319-
Eigen::VectorXd W = Pi.cwiseProduct(one - Pi).cwiseProduct(weights);
324+
// Truncate Pi*(1-Pi) BEFORE multiplying by weights, so zero-weight rows
325+
// yield D_i=0 rather than being clamped to PiPj_range[0].
326+
Eigen::VectorXd W = Pi.cwiseProduct(one - Pi);
320327
trunc(W, PiPj_range);
321-
return W;
328+
return W.cwiseProduct(weights);
322329
};
323330

324331
Eigen::VectorXd inv_link_function(T4 &X_full, Eigen::VectorXd &beta_full) {

0 commit comments

Comments
 (0)