Class Imbalance for Credit Risk

From selected and label-mature samples to training signals, ranking, probability, decisions, and portfolios.

English · 简体中文 · Historical theory notes · API reference

This project is rooted in Chinese credit-risk practice, but the modeling questions and theoretical connections are open to practitioners everywhere.

It is not an encyclopedia of class-imbalance algorithms, and it does not treat a 1:1 class ratio as an objective. The primary case is retail-credit application probability-of-default (PD) modeling in China. Before choosing a resampling method or loss function, the project asks who enters the sample, which outcomes are observable, and when labels become mature. It then separates changes to the training signal from evidence about ranking, probability, and business action.

population and label -> training signal -> ranking -> probability -> decision -> portfolio

The central modeling claim

Class imbalance is not an isolated algorithm choice. It sits inside an ordered chain of evidence:

Layer	Question	What it must not be reduced to
Population and label	Who applied, who was approved and booked under the historical policy, whose performance window has matured, and which outcomes remain unobserved?	“Unresampled data equals the true all-applicant distribution.”
Training signal	What do the original sample, resampling, class weights, instance weights, and loss function each change?	“Rare bads imply the training set must become 1:1.”
Ranking	Are higher-risk applicants ordered ahead of lower-risk applicants in the target cohort and across time?	“A high AUC means the PD is accurate.”
Probability	Does the output estimate PD for a stated conditional population, bad definition, and horizon?	“A model score is automatically a usable probability.”
Decision and portfolio	How does a score or probability become an approval, review, pricing, or limit action, and what portfolio follows?	“0.5 is the natural threshold.”

A model can rank applicants well without producing calibrated PDs. A calibrated PD can still lead to a poor policy if exposure, approval constraints, review capacity, or portfolio mix are ignored. Rebalancing may help a learner attend to rare bad outcomes, but it can also change the meaning of the raw score.

From costs to thresholds: why 0.5 is a cost assumption, not a universal default

The original Chinese notes begin with cost-sensitive versus cost-insensitive learning. A modern distinction distilled from that reasoning still organizes this project:

Class imbalance describes how often outcomes occur. Cost sensitivity describes the consequences of decisions.

Cost-sensitive learning distinguishes among the consequences of different classification errors and seeks a decision rule with lower expected cost. An accuracy-oriented, cost-insensitive rule instead counts false positives and false negatives as interchangeable unit errors. A rare positive class may make learning difficult, but rarity alone does not justify a 1:1 training sample or a probability cutoff below 0.5.

1. A cost matrix induces a probability threshold

Let $p(x)=P(Y=1\mid x)$ be the positive-class probability. Write $C_{TP}, C_{FP}, C_{FN}, C_{TN}$ for the costs of the four possible decision/outcome pairs:

Decision	Actual $Y=0$	Actual $Y=1$
Predict $0$	$C_{TN}$	$C_{FN}$
Predict $1$	$C_{FP}$	$C_{TP}$

The conditional costs of predicting positive and negative are:

$$R_1(x)=pC_{TP}+(1-p)C_{FP}, \qquad R_0(x)=pC_{FN}+(1-p)C_{TN}.$$

If each misclassification is no less costly than the corresponding correct decision, and at least one is strictly more costly, predict positive when the cost-derived cutoff $t_c$ is met:

$$p(x)\geq t_c, \qquad t_c= \frac{C_{FP}-C_{TN}} {C_{FP}+C_{FN}-C_{TP}-C_{TN}}.$$

When correct decisions have zero cost, this becomes:

$$t_c=\frac{C_{FP}}{C_{FP}+C_{FN}}.$$

Under this zero-correct-decision-cost special case, 0.5 becomes interpretable rather than automatic:

• $t_c=0.5$ when false positives and false negatives have equal cost;
• $t_c<0.5$ when false negatives cost more, so the rule classifies positive at a lower estimated probability;
• $t_c>0.5$ when false positives cost more, so stronger evidence is required before predicting positive.

Here, positive is only a statistical label; in credit risk or fraud detection, it often denotes an adverse outcome. For example, let 1 mean default or fraud, and set $C_{FN}=5$, $C_{FP}=1$, and $C_{TP}=C_{TN}=0$. Then $t_c=1/(1+5)\approx0.167$. At $p=0.20$, predicting positive has expected cost $0.80\times1=0.80$, while predicting negative has expected cost $0.20\times5=1.00$. The lower-cost classification decision is therefore to predict positive, even though the probability is far below 0.5.

2. The cutoff is a decision rule, not a property of every score

The formula requires a calibrated probability for the target deployment population, label definition, and outcome horizon, together with fixed, instance-independent costs. An arbitrary ranking score—or an uncorrected score from a rebalanced training distribution—is not such a probability.

If costs vary with exposure, loss given default (LGD), transaction size, or the individual case, use instance-level expected loss or a case-specific $t_c(x)$; one global cutoff is no longer sufficient.

Nor can every non-0.5 cutoff be interpreted as a cost ratio. A threshold chosen to satisfy an FPR cap, approval-rate target, review capacity, or portfolio constraint reflects that operating constraint. Its value alone does not reveal the relative costs of false positives and false negatives.

3. Thresholding, sampling, and weighting act at different layers

Lever	What it changes	What it does not establish by itself
Cost-derived probability thresholding	The action taken from a calibrated target-population probability	Better ranking or better calibration
Sampling	The empirical training distribution seen by the learner	The real business cost ratio or target-population probability
Weighting	How strongly observations contribute to the training objective	The same fitted model, probability meaning, or deployment cutoff
Prior correction and calibration	How model outputs are interpreted as probabilities	Which business action is optimal

Why do these levers sometimes appear equivalent? Under a known class-conditional sampling or weighting scheme, and for a learner that recovers the corresponding posterior odds, changing class ratios or class weights can represent a shift in the target decision threshold. This is an equivalence at the level of an idealized decision problem—not a promise that regularization, tree splits, early stopping, or finite model capacity will produce the same fitted model.

Sampling and weighting can also change the probability meaning of raw model outputs. Before applying a cost-derived threshold, they may require prior correction under a known class-sampling mechanism, calibration on untouched, representative target-population validation or out-of-time (OOT) data, or both. Neither step repairs selection bias created by historical approvals.

This mechanism is why the historical theory connected cost, thresholding, sampling, weighting, and class priors. The modern project preserves that chain while making its conditions explicit. Its central contribution is to keep four objects separate: outcome prevalence is a property of a population; rebalancing is a training intervention; probability is an estimand; and a threshold is a decision.

See the full historical derivation and the modern ranking, calibration, decision, and portfolio guide.

The observation boundary in application PD

Let X denote application-time features, A the historical approval decision, B whether an approved applicant actually books or draws the loan, Y the bad outcome over a predefined performance window, and M whether that label has matured. In many historical datasets, observed outcomes mainly come from accounts satisfying B = 1, M = 1.

This has four consequences:

• An unresampled validation, test, or out-of-time cohort often represents mature booked accounts under the historical policy—not the full applicant population.
• For A = 0 rejects, “would this applicant have defaulted if approved?” is a counterfactual outcome. It is not identified from approved-account outcomes alone. Approved but unbooked applicants normally lack repayment performance in the product as well.
• Recent accounts whose performance window has not closed are immature or right-censored; absence of an observed bad event does not make them good.
• Sampling belongs inside the training pipeline. Validation, test, and OOT cohorts retain the sampling frame of the population they are intended to evaluate.

See Population, selection, and label maturity for the full treatment.

Start here

1. English documentation path provides the complete map.
2. Population, selection, and label maturity defines the estimand before any modeling method is chosen.
3. What rebalancing changes compares sampling, weighting, loss shaping, prior correction, calibration, and threshold selection.
4. Ranking, calibration, decision, and portfolio separates four outputs that are often collapsed into one model metric.
5. Experiment protocol gives a leakage-safe, time-aware comparison design.
6. The fraud boundary explains why fraud is a useful contrast case, not a second name for application PD.
7. Historical theory notes adapts Ma Xiao's 2018–2021 reasoning into English while keeping modern qualifications visibly separate.

Optional method experiments live under docs/en/labs/; begin with Focal Loss as an optional lab.

Why the historical analysis remains

The theory map above comes from Ma Xiao's 2018–2021 Chinese homepage. Its value is not a claim to have invented the underlying formulas; it is the way the notes connect decision cost, probability thresholds, class-ratio changes, observation weighting, and class priors into one modeling argument.

The English historical edition preserves the original reading and derivation order, including the experimental observations and paper notes. Modern qualifications are kept in a separate editor's note rather than silently written back into the historical argument.

The complete Chinese edition remains a first-class entry, and the verbatim historical homepage remains available for provenance.

Credit risk and fraud are related, not interchangeable

Application PD and fraud detection both involve rare events and asymmetric errors, but they do not share one population, label process, operating clock, or cost model. Fraud often puts more emphasis on extreme false-positive constraints, manual-review capacity, rapid feedback, selective labels, and adversarial drift. Application PD puts more emphasis on historical approval selection, performance-window maturity, the target conditional population, and portfolio risk-return.

The project therefore uses fraud as a separate contrast case rather than compressing both domains into one fixed-cost binary-classification problem.

Engineering scope

The modern package is intentionally small. It provides auditable contracts for binary metrics and operating points, probability correction, cost-based decisions, rebalancing helpers, and selected losses.

Status	Modules
Stable	metrics, operating_points, probability, decision, rebalancing, losses
Optional dependency	sampling
Experimental	experimental.adacost
Compatibility only	thresholding, weighting

The compatibility modules preserve older imports; new work should use the modules in the first three rows.

src/class_imbalance/   small, testable modern Python APIs
tests/                 unit, contract, integration, and scenario checks
examples/              runnable synthetic examples only
docs/en/               substantive English adaptations
docs/zh-CN/            complete Chinese reading path
docs/reference/        authoritative English API reference
docs/history/          provenance and refactor records
legacy/                historical assets, not the stable API

See the API reference for contracts and interpretation boundaries.

Minimal API example

A ranking-score operating point and a calibrated-probability cost threshold are two different paths:

from class_imbalance.decision import cost_sensitive_threshold, predict_with_threshold
from class_imbalance.operating_points import operating_point_at_fpr_cap
from class_imbalance.probability import prior_probability_correction

# Synthetic convention: 1 = bad, 0 = good.
y_true = [0, 0, 0, 0, 1, 1]
ranking_score = [0.05, 0.10, 0.40, 0.65, 0.70, 0.95]

# Ranking path: choose an achievable empirical point under an FPR cap.
point = operating_point_at_fpr_cap(y_true, ranking_score, fpr_cap=0.25)
print(point)

# Probability path: the source values must already have probabilistic meaning.
# Prior correction does not repair historical approval selection.
source_prob = [0.03, 0.08, 0.20, 0.35, 0.55, 0.85]
target_prob = prior_probability_correction(
    source_prob,
    source_prevalence=0.20,
    target_prevalence=0.08,
)
threshold = cost_sensitive_threshold(
    false_negative_cost=5.0,
    false_positive_cost=1.0,
)
actions = predict_with_threshold(target_prob, threshold)
print(actions)

cost_sensitive_threshold applies only to calibrated probabilities for the target deployment population under fixed, instance-independent misclassification costs. A real credit policy also requires portfolio constraints and OOT validation.

Install and verify

From a fresh clone with Python 3.10 or later:

python -m pip install -e ".[dev]"
python -m pytest -q
python -m build

The package core depends only on NumPy. Sampling and estimator integrations are installed through their declared optional dependency groups. The built wheel should also be installed in an isolated environment for a smoke test, followed by all public examples.

Runnable synthetic scenarios

• Credit-risk workflow compares an unrebalanced model with training-only random oversampling, applies a conditional prior correction, and evaluates a cost-based decision on an untouched later cohort.
• Reject-inference boundary uses an oracle available only in synthetic data to show why accepted, mature outcomes do not identify the all-applicant bad rate.
• Fraud under an FPR cap selects an achievable empirical review threshold without interpolating between ROC points.

With the development extra installed:

python examples/credit_risk_workflow.py
python examples/reject_inference_boundary.py
python examples/fraud_fixed_fpr.py

The examples use deterministic synthetic data. They do not encode customer records, production thresholds, internal approval rules, or real institutional parameters.

Historical, public, and license boundaries

• The author's 2018–2021 theory, code, and experiment records are preserved as historical evidence. Historical image files were separately audited and are excluded from the curated tree unless authorship and redistribution rights can be confirmed. These materials are not the stable API and should not be read as current production guidance.
• Third-party papers, screenshots, datasets, and external code are not covered merely because this project uses an open-source license. See THIRD_PARTY.md and the legacy manifest.
• Modern examples and documentation discuss general methods without disclosing any institution's customer data or production implementation.

The modern code and original project materials for which the author holds the rights are released under the MIT License. Author: Ma Xiao.