Skip to content

Commit 441ed3e

Browse files
solegalliclaude
andauthored
Migrate DecisionTreeDiscretiser to narwhals, add polars support (#1042)
* Migrate DecisionTreeDiscretiser to narwhals, add polars support DecisionTreeDiscretiser now accepts pandas or polars input via narwhals, extending BaseNumericalTransformer directly (independent of the BaseDiscretiser migration). Never imports pandas; confirmed the module loads with pandas import blocked. Merge (single narwhals codepath, one is_pandas branch only at the final column reassembly) over split (branching at every column-selection call site): benchmarked at 10k/50k/100k rows x 1/2/10 cols, full fit+transform time is dominated by GridSearchCV tree training (10-1500ms) vs plumbing (~0.05-1.4ms per call, <1% of total even where a hand-branched pandas path was ~2x faster on the isolated plumbing microbenchmark). Merge also avoids the one-column-at-a-time write pattern that caused pandas fragmentation warnings in the DecisionTreeFeatures migration. Added optional n_jobs (default None = sequential, unchanged behaviour), parallelizing the per-variable tree fits with joblib threads, mirroring DecisionTreeFeatures. Benchmarked: net loss on small workloads (2 vars, small grid: 0.6-0.8x), real win once there's enough work (2-50 vars with a larger grid: 1.4-2.3x). Verified n_jobs=2 produces identical trees and predictions to n_jobs=None. Bug found and fixed (introduced by the base-transformer narwhals migration, not present pre-migration): check_X used to always copy its pandas input; the narwhals-based check_X no longer does, so the old transform()'s in-place `X[feature] = ...` assignments would have mutated the caller's original dataframe. Rewrote transform() to batch every replacement column and apply them in one non-mutating `.assign()` (pandas) / `.with_columns()` (polars) call instead, which also sidesteps polars' immutability and avoids per-column pandas fragmentation. Reimplemented pandas.cut's binning (bin_number/boundaries outputs) without importing pandas: np.digitize for bin assignment, and a from-scratch port of pandas' internal `_round_frac`/`_infer_precision` label-rounding algorithm (rounds each edge, bumping precision globally if that would collide two edges) so boundary labels are byte-for-byte identical to the old pd.cut output. Verified against pandas.cut directly across 500 randomized threshold/precision/value trials with zero mismatches, in addition to the existing hardcoded-value tests passing unmodified. Tests rewritten to one parametrized test per behavior over make_df in [pd.DataFrame, pl.DataFrame], replacing the pandas-only df_normal_dist/df_discretise fixtures with local data dicts (matching the DecisionTreeFeatures precedent, since those shared fixtures are still pandas-only). Fixed test_non_fitted_error, which was instantiating EqualWidthDiscretiser instead of DecisionTreeDiscretiser (a pre-existing copy-paste bug, confirmed present on main before this migration). tests/test_discretisation full suite: 123 passed (was 108 pre-migration, +15 from parametrization), same 5 pre-existing check_estimator failures (numpy-array input rejected by narwhals check_X, unrelated to this file, confirmed identical on the pre-migration baseline). flake8 and mypy clean. sphinx -W build produces only the pre-existing linkcode_resolve warning (confirmed identical on baseline). Docs: added "With polars" and "Training trees in parallel" sections, verified against real output (network available this session, so the existing fetch_openml house-prices example was re-run and confirmed still accurate). The two `binner_dict_` boundary/bin_number code blocks now display floats as plain numbers as before; current numpy's list repr actually renders them as np.float64(...), a numpy-version-only cosmetic drift present across the whole docs tree and not caused by this migration, left as-is and noted here instead. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * Use shared backend test fixtures and helpers in DecisionTreeDiscretiser tests Replace the file-local _normal_dist_data/_discretise_data/_unique_sorted helpers with the shared test structure: data_normal_dist fixture, y built with make_series on the backend under test, isinstance(X, make_df) plus to_dict() checks, and pytest.raises(match=re.escape(msg)). Add a test passing the target as a list and as a numpy array, which must give the same result as a Series. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Use frame_to_dict after the shared helper rename in #1045 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Group init tests and match errors in DecisionTreeDiscretiser tests, check bin_output type Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Move DecisionTreeDiscretiser helper functions into the class as private methods Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Simplify the n_jobs note in the DecisionTreeDiscretiser user guide Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Use the scikit-learn wording for n_jobs in DecisionTreeDiscretiser Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Call is_pandas_dataframe in the condition instead of storing it Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Shorten comment in DecisionTreeDiscretiser.transform Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Reorder DecisionTreeDiscretiser tests to the test file convention Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
1 parent 9a875cf commit 441ed3e

3 files changed

Lines changed: 416 additions & 169 deletions

File tree

‎docs/user_guide/discretisation/DecisionTreeDiscretiser.rst‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -443,6 +443,88 @@ were sorted:
443443
799 0 9
444444
380 0 9
445445
446+
With polars
447+
-----------
448+
449+
:class:`DecisionTreeDiscretiser()` also accepts polars dataframes as input, and returns a polars
450+
dataframe from `transform()`:
451+
452+
.. code:: python
453+
454+
import polars as pl
455+
456+
X_train_pl = pl.DataFrame(X_train[["LotArea", "GrLivArea"]])
457+
458+
disc = DecisionTreeDiscretiser(
459+
bin_output="prediction",
460+
cv=3,
461+
scoring="neg_mean_squared_error",
462+
regression=True,
463+
)
464+
disc.fit(X_train_pl, y_train)
465+
466+
train_t = disc.transform(X_train_pl)
467+
print(train_t.head())
468+
469+
.. code:: text
470+
471+
shape: (5, 2)
472+
┌───────────────┬───────────────┐
473+
│ LotArea ┆ GrLivArea │
474+
│ --- ┆ --- │
475+
│ f64 ┆ f64 │
476+
╞═══════════════╪═══════════════╡
477+
│ 144174.283688 ┆ 152471.713568 │
478+
│ 144174.283688 ┆ 191760.966667 │
479+
│ 176117.741848 ┆ 97156.25 │
480+
│ 144174.283688 ┆ 202178.409091 │
481+
│ 144174.283688 ┆ 202178.409091 │
482+
└───────────────┴───────────────┘
483+
484+
The predictions match those obtained with the pandas dataframe above.
485+
486+
Training trees in parallel
487+
---------------------------
488+
489+
:class:`DecisionTreeDiscretiser()` fits one decision tree per variable, independently of the
490+
others. When there are many variables to discretise, or a large `param_grid` to search, training
491+
can be parallelized across variables with the `n_jobs` parameter:
492+
493+
.. code:: python
494+
495+
import pandas as pd
496+
from feature_engine.discretisation import DecisionTreeDiscretiser
497+
498+
X = pd.DataFrame({
499+
"Age": [20, 44, 19, 33, 51, 40, 41, 37, 30, 54],
500+
"Height": [164, 150, 178, 158, 188, 190, 168, 174, 176, 171],
501+
"Marks": [1.0, 0.8, 0.6, 0.1, 0.3, 0.4, 0.8, 0.6, 0.5, 0.2],
502+
})
503+
y = [4.1, 5.8, 3.9, 6.2, 4.3, 4.5, 7.2, 4.4, 4.1, 6.7]
504+
505+
dtd = DecisionTreeDiscretiser(n_jobs=2, random_state=0)
506+
dtd.fit(X, y)
507+
508+
print(dtd.transform(X))
509+
510+
.. code:: text
511+
512+
Age Height Marks
513+
0 4.533333 5.366667 4.100000
514+
1 6.000000 5.366667 6.500000
515+
2 4.533333 4.133333 4.133333
516+
3 4.533333 5.366667 6.200000
517+
4 6.000000 4.400000 4.400000
518+
5 4.533333 4.400000 4.400000
519+
6 6.000000 6.950000 6.500000
520+
7 4.533333 4.133333 4.133333
521+
8 4.533333 4.133333 4.133333
522+
9 6.000000 6.950000 6.700000
523+
524+
`n_jobs` is the number of jobs to run in parallel. `fit` is parallelized over the variables,
525+
training one decision tree per variable. `None` means 1 unless in a `joblib.parallel_backend`
526+
context. `-1` means using all processors.
527+
446528
Additional considerations
447529
-------------------------
448530

‎feature_engine/discretisation/decision_tree.py‎

Lines changed: 147 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,11 @@
33

44
from typing import Dict, List, Optional, Union
55

6+
import narwhals as nw
7+
import narwhals.dependencies as nwd
68
import numpy as np
7-
import pandas as pd
9+
from joblib import Parallel, delayed
10+
from narwhals.typing import IntoDataFrame, IntoSeries
811
from sklearn.model_selection import GridSearchCV
912
from sklearn.tree import DecisionTreeClassifier, DecisionTreeRegressor
1013
from sklearn.utils.multiclass import check_classification_targets, type_of_target
@@ -115,6 +118,11 @@ class DecisionTreeDiscretiser(BaseNumericalTransformer):
115118
DecisionTreeClassifier(). For reproducibility it is recommended to set
116119
the random_state to an integer.
117120
121+
n_jobs: int, default=None
122+
The number of jobs to run in parallel. `fit` is parallelized over the variables,
123+
training one decision tree per variable. `None` means 1 unless in a
124+
`joblib.parallel_backend` context. `-1` means using all processors.
125+
118126
Attributes
119127
----------
120128
binner_dict_:
@@ -163,19 +171,38 @@ class DecisionTreeDiscretiser(BaseNumericalTransformer):
163171
>>> dtd = DecisionTreeDiscretiser(random_state=42)
164172
>>> dtd.fit(X, y_reg)
165173
>>> dtd.transform(X)["x"].value_counts()
174+
x
166175
-0.090091 90
167-
0.479454 10
168-
Name: x, dtype: int64
176+
0.479454 10
177+
Name: count, dtype: int64
169178
170179
You can also apply this for classification problems adjusting the scoring metric.
171180
172181
>>> y_clf = pd.Series(np.random.randint(0,2,100))
173182
>>> dtd = DecisionTreeDiscretiser(regression=False, scoring="f1", random_state=42)
174183
>>> dtd.fit(X, y_clf)
175184
>>> dtd.transform(X)["x"].value_counts()
185+
x
176186
0.480769 52
177187
0.687500 48
178-
Name: x, dtype: int64
188+
Name: count, dtype: int64
189+
190+
With polars:
191+
192+
>>> import polars as pl
193+
>>> X = pl.DataFrame({"x": X["x"].to_list()})
194+
>>> dtd = DecisionTreeDiscretiser(random_state=42)
195+
>>> dtd.fit(X, y_reg)
196+
>>> dtd.transform(X)["x"].value_counts()
197+
shape: (2, 2)
198+
┌───────────┬───────┐
199+
│ x ┆ count │
200+
│ --- ┆ --- │
201+
│ f64 ┆ u32 │
202+
╞═══════════╪═══════╡
203+
│ -0.090091 ┆ 90 │
204+
│ 0.479454 ┆ 10 │
205+
└───────────┴───────┘
179206
"""
180207

181208
def __init__(
@@ -189,9 +216,14 @@ def __init__(
189216
param_grid: Optional[Dict[str, Union[str, int, float, List[int]]]] = None,
190217
regression: bool = True,
191218
random_state: Optional[int] = None,
219+
n_jobs: Optional[int] = None,
192220
) -> None:
193221

194-
if bin_output not in ["prediction", "bin_number", "boundaries"]:
222+
if not isinstance(bin_output, str) or bin_output not in [
223+
"prediction",
224+
"bin_number",
225+
"boundaries",
226+
]:
195227
raise ValueError(
196228
"bin_output takes values 'prediction', 'bin_number' or 'boundaries'. "
197229
f"Got {bin_output} instead."
@@ -223,21 +255,22 @@ def __init__(
223255
self.variables = _check_variables_input_value(variables)
224256
self.param_grid = param_grid
225257
self.random_state = random_state
258+
self.n_jobs = n_jobs
226259
self.return_empty = return_empty
227260

228-
def fit(self, X: pd.DataFrame, y: pd.Series):
261+
def fit(self, X: IntoDataFrame, y: IntoSeries):
229262
"""
230263
Fit one decision tree per variable to discretise with cross-validation and
231264
grid-search for hyperparameters.
232265
233266
Parameters
234267
----------
235268
236-
X: pandas dataframe of shape = [n_samples, n_features]
269+
X: dataframe of shape = [n_samples, n_features]
237270
The training dataset. Can be the entire dataframe, not just the
238271
variables to be transformed.
239272
240-
y: pandas series.
273+
y: Series.
241274
Target variable. Required to train the decision tree.
242275
"""
243276
# confirm model type and target variables are compatible.
@@ -259,25 +292,18 @@ def fit(self, X: pd.DataFrame, y: pd.Series):
259292
else:
260293
param_grid = {"max_depth": [1, 2, 3, 4]}
261294

262-
binner_dict_ = {}
263-
scores_dict_ = {}
295+
nw_X = nw.from_native(X, eager_only=True)
296+
X_subs = [nw_X.get_column(var).to_frame().to_native() for var in variables_]
264297

265-
for var in variables_:
298+
fitted = Parallel(n_jobs=self.n_jobs, prefer="threads")(
299+
delayed(self._fit_one_tree)(X_sub, y, param_grid) for X_sub in X_subs
300+
)
266301

267-
if self.regression:
268-
model = DecisionTreeRegressor(random_state=self.random_state)
269-
else:
270-
model = DecisionTreeClassifier(random_state=self.random_state)
271-
272-
tree_model = GridSearchCV(
273-
model, cv=self.cv, scoring=self.scoring, param_grid=param_grid
274-
)
275-
276-
# fit the model to the variable
277-
tree_model.fit(X[var].to_frame(), y)
278-
279-
binner_dict_[var] = tree_model
280-
scores_dict_[var] = tree_model.score(X[var].to_frame(), y)
302+
binner_dict_ = dict(zip(variables_, fitted))
303+
scores_dict_ = {
304+
var: tree_model.score(X_sub, y)
305+
for var, X_sub, tree_model in zip(variables_, X_subs, fitted)
306+
}
281307

282308
if self.bin_output != "prediction":
283309
for var in variables_:
@@ -296,64 +322,82 @@ def fit(self, X: pd.DataFrame, y: pd.Series):
296322

297323
return self
298324

299-
def transform(self, X: pd.DataFrame) -> pd.DataFrame:
325+
def transform(self, X: IntoDataFrame) -> IntoDataFrame:
300326
"""
301327
Replaces original variable values with the predictions of the tree. The
302328
decision tree predictions are finite, aka, discrete.
303329
304330
Parameters
305331
----------
306-
X: pandas dataframe of shape = [n_samples, n_features]
332+
X: dataframe of shape = [n_samples, n_features]
307333
The input samples.
308334
309335
Returns
310336
-------
311-
X_new: pandas dataframe of shape = [n_samples, n_features]
337+
X_new: dataframe of shape = [n_samples, n_features]
312338
The dataframe with transformed variables.
313339
"""
314-
315340
# check input dataframe and if class was fitted
316341
X = self._check_transform_input_and_state(X)
317342

343+
nw_X = nw.from_native(X, eager_only=True)
344+
345+
# add all new columns in one step, instead of one per variable, and leave
346+
# the user's dataframe unchanged
347+
new_columns: Dict[str, np.ndarray] = {}
348+
318349
if self.bin_output == "prediction":
319350
for feature in self.variables_:
320-
if self.regression:
321-
preds = self.binner_dict_[feature].predict(X[feature].to_frame())
322-
if self.precision is None:
323-
X[feature] = preds
324-
else:
325-
X[feature] = np.round(preds, self.precision)
351+
X_sub = nw_X.get_column(feature).to_frame().to_native()
352+
if self.regression is True:
353+
preds = self.binner_dict_[feature].predict(X_sub)
326354
else:
327-
tmp = self.binner_dict_[feature].predict_proba(
328-
X[feature].to_frame()
329-
)
330-
preds = tmp[:, 1]
331-
if self.precision is None:
332-
X[feature] = preds
333-
else:
334-
X[feature] = np.round(preds, self.precision)
355+
preds = self.binner_dict_[feature].predict_proba(X_sub)[:, 1]
356+
if self.precision is not None:
357+
preds = np.round(preds, self.precision)
358+
new_columns[feature] = preds
335359

336360
elif self.bin_output == "boundaries":
361+
# __init__ already guarantees precision is set when bin_output is
362+
# "boundaries"; assert narrows the type for mypy.
363+
assert self.precision is not None
337364
for feature in self.variables_:
338-
X[feature] = pd.cut(
339-
X[feature],
340-
self.binner_dict_[feature],
341-
precision=self.precision,
342-
include_lowest=True,
343-
)
344-
X[self.variables_] = X[self.variables_].astype(str)
365+
thresholds = self.binner_dict_[feature]
366+
labels = self._bin_labels(thresholds, self.precision)
367+
values = nw_X.get_column(feature).to_numpy()
368+
bin_idx = self._bin_index(values, thresholds)
369+
new_columns[feature] = np.array(labels)[bin_idx]
345370

346371
else:
347372
for feature in self.variables_:
348-
X[feature] = pd.cut(
349-
X[feature],
350-
self.binner_dict_[feature],
351-
labels=False,
352-
include_lowest=True,
353-
)
373+
thresholds = self.binner_dict_[feature]
374+
values = nw_X.get_column(feature).to_numpy()
375+
new_columns[feature] = self._bin_index(values, thresholds)
376+
377+
if nwd.is_pandas_dataframe(X) is True:
378+
X = X.assign(**new_columns)
379+
else:
380+
new_series = [
381+
nw.new_series(name, values, backend=nw_X.implementation)
382+
for name, values in new_columns.items()
383+
]
384+
X = nw_X.with_columns(*new_series).to_native()
354385

355386
return X
356387

388+
def _fit_one_tree(self, X_sub: IntoDataFrame, y: IntoSeries, param_grid: Dict):
389+
"""Instantiate and fit one decision tree on one variable."""
390+
if self.regression is True:
391+
model = DecisionTreeRegressor(random_state=self.random_state)
392+
else:
393+
model = DecisionTreeClassifier(random_state=self.random_state)
394+
395+
tree_model = GridSearchCV(
396+
model, cv=self.cv, scoring=self.scoring, param_grid=param_grid
397+
)
398+
tree_model.fit(X_sub, y)
399+
return tree_model
400+
357401
def _more_tags(self):
358402
tags_dict = _return_tags()
359403
tags_dict["variables"] = "numerical"
@@ -363,3 +407,50 @@ def _more_tags(self):
363407
def __sklearn_tags__(self):
364408
tags = super().__sklearn_tags__()
365409
return tags
410+
411+
def _round_bin_edge(self, x: float, precision: int) -> float:
412+
"""Round a bin edge the way pandas.cut historically formatted Interval labels:
413+
-inf/inf/0 pass through unrounded, and numbers with magnitude < 1 get extra
414+
decimals so that `precision` significant digits survive past the leading
415+
zeros (e.g. -0.0942 at precision=3 keeps 4 decimals, not 3, since
416+
round(-0.0942, 3) == -0.094 would only keep 2 significant digits).
417+
"""
418+
if not np.isfinite(x) or x == 0:
419+
return x
420+
frac, whole = np.modf(x)
421+
if whole == 0:
422+
digits = -int(np.floor(np.log10(abs(frac)))) - 1 + precision
423+
else:
424+
digits = precision
425+
return round(x, digits)
426+
427+
def _infer_bin_precision(self, thresholds: List[float], precision: int) -> int:
428+
"""Find the smallest precision >= `precision` at which every rounded
429+
threshold is still distinct, mirroring pandas.cut's behaviour of bumping
430+
precision (for every edge, not just the colliding pair) when the requested
431+
precision would make two adjacent bin edges collide."""
432+
for prec in range(precision, 20):
433+
rounded = [self._round_bin_edge(t, prec) for t in thresholds]
434+
if len(set(rounded)) == len(thresholds):
435+
return prec
436+
return precision
437+
438+
def _format_bin_edge(self, x: float, precision: int) -> str:
439+
if x == -np.inf:
440+
return "-inf"
441+
if x == np.inf:
442+
return "inf"
443+
return str(self._round_bin_edge(x, precision))
444+
445+
def _bin_labels(self, thresholds: List[float], precision: int) -> List[str]:
446+
"""Build the `(left, right]` interval label for every bin delimited by
447+
`thresholds`, which starts with -inf and ends with inf."""
448+
precision = self._infer_bin_precision(thresholds, precision)
449+
edges = [self._format_bin_edge(t, precision) for t in thresholds]
450+
return [f"({edges[i]}, {edges[i + 1]}]" for i in range(len(edges) - 1)]
451+
452+
def _bin_index(self, values: np.ndarray, thresholds: List[float]) -> np.ndarray:
453+
"""Map each value to the 0-indexed bin delimited by `thresholds` (which
454+
starts with -inf and ends with inf), bins being closed on the right.
455+
"""
456+
return np.digitize(values, thresholds[1:-1], right=True)

0 commit comments

Comments
 (0)