Repository navigation
Commit 9e4e99b
Migrate WoEEncoder to narwhals, add polars support (#1032)
* Migrate WoEEncoder to narwhals, add polars support
fit() splits by backend: pandas keeps _calculate_woe()'s existing
two-groupby implementation unchanged (it's directly unit-tested for that
exact pandas-Series-with-category-index contract); polars/other narwhals
backends use one group_by() instead of two, deriving the negative-class
count as the complement of the positive-class count per category -
benchmarked competitive with, and often faster than, pandas-native at
50k-100k rows. Zero-count-per-class fill_value handling preserved exactly.
Bug fix: _check_fit_input() previously assumed y was always a pandas
Series (y.nunique()/y.min()/y.max()), breaking on a numpy y (e.g. a plain
list/array-like target, which sklearn's check_X_y machinery converts via
column_or_1d). Wrapped numpy y into a narwhals Series aligned to X's
backend; for pandas specifically, also had to line the wrapped Series up
with X's actual index, since _calculate_woe()'s y.groupby(X[var]) aligns
by index and a mismatched default RangeIndex silently drops every row
instead of raising, leaving encoder_dict_ empty. Fixes
test_encoders_when_x_pandas_y_numpy's WoEEncoder case (was failing on the
unmigrated file, confirmed pre-existing).
Verified: 44/44 own tests, full encoding suite 342 passed/16 failed (was
17 pre-existing on the narwhals-encoding-base baseline - one less here
since this branch's own numpy-y bug is now fixed, rest confirmed
unrelated), flake8 and mypy clean, sphinx -W build clean (only the
pre-existing unrelated linkcode_resolve warning).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Adapt WoEEncoder to narwhals-returning check_X
check_X_y now returns a narwhals frame. In _check_fit_input, bind that to
nw_X and keep the original native X: the nwd.is_pandas_dataframe(X) check,
the native_y.index = X.index alignment and the returned X all need native
input, and fit()'s pandas _calculate_woe fast path and nwd checks are then
unchanged (X stays native so no rehydration is needed). Take the y-series
backend from nw_X.implementation instead of re-wrapping X. In transform(),
bind _check_transform_input_and_state to nw_X, keep native X for
_check_contains_na, and pass nw_X to _encode (which now expects narwhals).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Use shared backend test fixtures and helpers in WoEEncoder tests
Replace the file-local data dicts and assert_df_equal/_none_to_nan helpers
with the shared test structure: make_df and data_enc* fixtures, y built with
make_series on the backend under test, isinstance(X, make_df) plus to_dict()
checks, and pytest.raises/warns(match=re.escape(msg)). Add a test passing the
target as a list and as a numpy array, which take a different code path than
a Series.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Use frame_to_dict after the shared helper rename in #1045
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Use add_target_to_X in WoEEncoder, group init tests, match errors
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Compute WoE with narwhals in _calculate_woe, shared by WoEEncoder and SelectByInformationValue
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Replace zero counts by 0.5 in WoE, remove fill_value, add variables_with_zero_counts_
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>1 parent f83c278 commit 9e4e99b
5 files changed
Lines changed: 434 additions & 460 deletions
File tree
- docs/user_guide/encoding
- feature_engine
- encoding
- selection
- tests/test_encoding/test_woe
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
112 | 112 | | |
113 | 113 | | |
114 | 114 | | |
115 | | - | |
116 | | - | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
117 | 119 | | |
118 | 120 | | |
119 | 121 | | |
| |||
162 | 164 | | |
163 | 165 | | |
164 | 166 | | |
165 | | - | |
166 | | - | |
167 | | - | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
168 | 207 | | |
169 | 208 | | |
170 | 209 | | |
| |||
280 | 319 | | |
281 | 320 | | |
282 | 321 | | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
283 | 357 | | |
284 | 358 | | |
285 | 359 | | |
| |||
0 commit comments