A real-time, explainable credit-card fraud detection service, and the engineering study behind its architecture. Not "deploy my model" — an argument, with a deployed system as the proof.
Live demo: https://huggingface.co/spaces/Leonardasvekrikas-source/fraud-detection-demo — paste a transaction; both models score it (LightGBM + LSTM), you get the fused Normal / Fraud / Expert-Checking verdict and two explanations side by side: SHAP for the tree, LIME for the LSTM. Write-up: docs/WRITEUP.md — the full story: reproduction, the fusion study, shipping it, and the MLOps loop. Walkthrough: a 2-minute narrated tour of the demo and the engineering — see Walkthrough below. Based on: an independent reproduction and extension of Yousefimehr & Ghatee (2025), A distribution-preserving method combined with LightGBM-LSTM for sequence-wise fraud detection (Expert Systems with Applications) — full reference under Citation.
- Reproduces published research — LightGBM branch bit-for-bit on the real dataset (not quoted — regenerated by the code)
- Live dual-model demo — both subsystems deployed on Hugging Face Spaces: LightGBM + LSTM → 3-way Normal / Fraud / Expert-Checking verdict, explained by SHAP (tree) and LIME (LSTM) side by side
- Dual XAI, done right — the exact, fast explainer for each model (SHAP
TreeExplainerfor the boosting; model-agnostic LIME for the black-box LSTM), packaged as a separateDockerfile.fusionbuild - 4 MLflow-tracked experiments — fusion comparison (with an honest negative result), threshold-as-cost, probability calibration, PaySim generalization
- Latency-profiled serving — prediction ~2 ms p50; the SHAP explanation is the bottleneck (~0.7 s) — so score fast, explain on review
- Production loop — Evidently drift monitoring + an Airflow champion/challenger retraining DAG (runs end-to-end)
- Engineered, not scripted —
src/layout, tested modules, ruff + pytest on green CI, Dockerised
walkthrough-720p.mp4
A 2-minute narrated tour — scoring a fraud, the Expert-Checking case where the two models disagree, and the engineering behind it.
The hybrid LightGBM-LSTM method demonstrated here is from Yousefimehr & Ghatee (2025); this project is an independent reproduction and extension — full Citation below.
A decision-level fusion of LightGBM and LSTM — keeping the
two models separate and combining their scores — is the more practical production choice than
feature-level fusion for this problem: it is simpler, lower-latency, and stays explainable
because the models remain separable (fast SHAP TreeExplainer on the gradient-boosted
component) and it adds a human-in-the-loop Expert-Checking tier for ambiguous cases. In this
study, an attempt at feature-level fusion added complexity without a practical detection benefit
(a cautious, setup-dependent negative result), while decision-level fusion stayed competitive on
ranking (AUC ≈ 0.97).
Honest scope note. On raw detection score, LightGBM alone is the strongest single model here (see results). The decision-level fusion's value is not a higher F1 — it is competitive ranking plus simplicity, per-model attributability for explanations, and the Expert-Checking escalation. This repository is deliberately honest about what is measured, what is simulated, and what comes from the literature vs. this work.
This project ships in public phases. Current state:
| Phase | What | Status |
|---|---|---|
| 0 | Reproducible core: config-driven training + evaluation of the pipeline, one command | 🟢 both subsystems reproduced on real data ✓ (LightGBM exact; LSTM within run-to-run noise) |
| 1 | Real-time explainable FastAPI service (LightGBM + LSTM) + SHAP & LIME + demo UI, Dockerised | 🟢 live on Hugging Face Spaces |
| 2 | Fusion comparison + cost/threshold + calibration + PaySim generalization, tracked in MLflow | 🟢 done — 4 MLflow experiments (experiments/) |
| 3 | Drift monitoring (Evidently, simulated), Airflow retraining DAG (docker-compose), CI | 🟢 done — monitoring/ + airflow/, CI green |
| 4 | Technical write-up linking the live demo and code | 🟢 done — docs/WRITEUP.md |
serving/, experiments/, monitoring/, and airflow/ are built out (each with its own README).
Only the Phase 4 write-up remains.
European Credit Card dataset, 5-fold cross-validation (mean ± std). "Original" = the source paper's reported figures. These reproduce the thesis behind this project.
| Metric | LightGBM (ours) | LightGBM (orig.) | LSTM (ours) | LSTM (orig.) |
|---|---|---|---|---|
| F1 | 0.8503 ± 0.0124 | 0.8805 | 0.7723 ± 0.0576 | 0.8310 |
| AUC | 0.9851 ± 0.0049 | 0.9603 | 0.9525 ± 0.0247 | 0.8922 |
| Precision | 0.8984 ± 0.0427 | 0.8446 | 0.9204 ± 0.0708 | 0.8835 |
| Recall | 0.8092 ± 0.0256 | 0.9209 | 0.6729 ± 0.0895 | 0.7846 |
| MCC | 0.8518 ± 0.0131 | 0.8815 | 0.7841 ± 0.0537 | 0.8323 |
The fusion-strategy comparison, the threshold-as-cost analysis, and the PaySim generalization run
are reproducible, MLflow-tracked experiments — see experiments/ for the
result tables, plots, and the honest findings (including feature-level fusion's collapse and
where the LSTM under-performs).
Both rows are reproduced by this repository's code on the real dataset. LightGBM matches the thesis exactly (deterministic, seed 42). A fresh LSTM re-run gave F1 0.743 / AUC 0.951 / precision 0.916 / recall 0.643 — within LSTM run-to-run noise of the thesis figures shown (AUC matches to 0.002); LSTM training is seed-fixed but not bit-reproducible across TensorFlow versions. The LSTM's lower recall traces to a documented
HybridUSedge case under extreme imbalance — distribution-protected normals exhaust the undersampling budget, so the LSTM trains near the original imbalance — not an implementation error (seedocs/).
Requires Python 3.12.
# 1. Clone
git clone https://github.com/Leonardasvekrikas-source/fraud-detection-system.git
cd fraud-detection-system
# 2. Install (editable). Add extras as needed:
# [dev] tests+lint [lstm] TensorFlow for Subsystem 2 / full-fusion training
# [serve] FastAPI+SHAP demo (lean, no TF)
python -m venv .venv
. .venv/Scripts/activate # Windows; use .venv/bin/activate on macOS/Linux
pip install -e ".[dev,lstm]" # LightGBM-only work needs just ".[dev]"
# 3. Fetch the public datasets (Kaggle) into data/
python scripts/fetch_data.py --dataset european
# 4. Reproduce the thesis numbers — one command
fraud-detect evaluate --subsystem 1 # LightGBM branch, 5-fold CV
fraud-detect evaluate --subsystem 2 # LSTM branch
fraud-detect train --save artifacts/ # train once and persist a servable modelOr skip the setup — try the live demo.
Decision-level fusion — the two models stay separable, so the score is attributable to each and
each gets the right explainer: a fast, exact SHAP TreeExplainer for the gradient-boosted
component, and model-agnostic LIME for the black-box LSTM.
Diagram source (Mermaid)
flowchart TD
A["Raw transaction"] --> B["dedup + F2Vote<br/>feature selection"]
B --> C1["Subsystem 1<br/>scale → HybridOS → LightGBM"]
B --> C2["Subsystem 2<br/>scale → HybridUS → window (W=3) → LSTM"]
C1 -- "P1 = P(fraud)" --> F{"Decision-level fusion<br/>P_sum = P1 + P2 (θ = 0.5)"}
C2 -- "P2 = P(fraud)" --> F
F -->|"P_sum < θ"| N["Normal"]
F -->|"θ ≤ P_sum < 1+θ"| E["Expert-Checking<br/>(human + explanations)"]
F -->|"P_sum ≥ 1+θ"| FR["Fraud"]
C1 -. "SHAP TreeExplainer" .-> X1["exact feature attribution"]
C2 -. "LIME (local surrogate)" .-> X2["directional feature drivers"]
classDef normal fill:#e6f7ea,stroke:#3fb950,color:#1a7f37;
classDef fraud fill:#ffe5e5,stroke:#f85149,color:#b00020;
classDef expert fill:#fff4e0,stroke:#d29922,color:#9a6700;
class N normal
class FR fraud
class E expert
Two serving builds — the same FastAPI app, sized for the job:
| Build | Image | Runs | Explains | Dockerfile |
|---|---|---|---|---|
| Lean | ~400 MB | LightGBM (P1) | SHAP | Dockerfile |
| Full fusion | ~4 GB | LightGBM + LSTM (P1 + P2) → 3-way verdict | SHAP and LIME | Dockerfile.fusion |
# full dual-XAI fusion demo (LightGBM + LSTM, SHAP + LIME) on http://localhost:7860
fraud-detect train --save artifacts/model-fusion # needs the [lstm] extra (TensorFlow)
docker build -f Dockerfile.fusion -t fraud-demo-fusion .
docker run -p 7860:7860 fraud-demo-fusionThe LSTM subsystem trains on a contiguous recent tail of the data (transactions kept in sequence): HybridUS fits a OneClass-SVM whose cost is ~quadratic in the sample count, so the full set is intractable — the same tractability trade-off used in the PaySim generalisation experiment. LightGBM still trains on the full dataset. LIME's local surrogate reaches R² ≈ 0.4 on the LSTM, so its weights are read as directional drivers, not exact attributions.
Production loop (Phase 3): retrain only when a challenger beats the live champion.
Diagram source (Mermaid)
flowchart LR
I["ingest"] --> T["train_candidate"] --> V["evaluate_candidate<br/>(holdout)"] --> D{"beats champion?"}
D -->|yes| P["promote → new champion"]
D -->|no| K["keep champion"]
M["Evidently drift monitor"] -. "drift detected → trigger" .-> I
- From the literature — the hybrid method: the LightGBM + LSTM dual-subsystem design, HybridOS/HybridUS distribution-preserving resampling, F2Vote feature selection, the Algorithm-1 decision-level fusion, and the SHAP/LIME explainability pairing. Source: Yousefimehr, B. & Ghatee, M. (2025). A distribution-preserving method for resampling combined with LightGBM-LSTM for sequence-wise fraud detection in credit card transactions. Expert Systems with Applications, 262, 125661. doi:10.1016/j.eswa.2024.125661
- Mine — the independent reimplementation; the fusion experiment (feature-level vs decision-level) and its negative result; the second-dataset (PaySim) generalization test; the systematic SHAP/LIME interpretability analysis; the cost/threshold framing; the serving system, explainability API, and demo; and the engineering conclusions drawn from all of it.
Python 3.12 · LightGBM · TensorFlow/Keras (LSTM) · scikit-learn · imbalanced-learn · SHAP/LIME · FastAPI (Phase 1) · MLflow (Phase 2) · Evidently + Airflow (Phase 3) · Docker · Hugging Face Spaces.
config/ YAML configuration (hyperparameters, θ, window size — the source of truth)
data/ Datasets live here (gitignored); fetched via scripts/fetch_data.py
docs/ Thesis manuscript + method/results write-ups (source materials)
scripts/ fetch_data.py and other one-off utilities
src/fraud_detection/
config.py Loads config/default.yaml (with safe built-in defaults)
data/ Dataset loading + validation
preprocessing/ dedup, StandardScaler, F2Vote, HybridOS, HybridUS, windowing
models/ LightGBM + LSTM wrappers
evaluation/ Metrics (F1, AUC, MCC, ...)
fusion/ Decision-level fusion engine (Algorithm 1)
pipelines/ subsystem1, subsystem2, train (train-once + persist)
artifacts/ Save/load fitted model + scaler + F2Vote mask
cli.py `fraud-detect train | evaluate`
tests/ pytest suite for the pure-logic modules
serving/ Phase 1 — FastAPI + SHAP/LIME demo (deployed to Hugging Face Spaces)
experiments/ Phase 2 — MLflow fusion / cost-threshold / PaySim studies + results
monitoring/ Phase 3 — Evidently drift monitoring (simulated)
airflow/ Phase 3 — champion/challenger retraining DAG (docker-compose)
This project is an independent reproduction and extension of the method introduced in:
Yousefimehr, B., & Ghatee, M. (2025). A distribution-preserving method for resampling combined with LightGBM-LSTM for sequence-wise fraud detection in credit card transactions. Expert Systems with Applications, 262, 125661. https://doi.org/10.1016/j.eswa.2024.125661
@article{yousefimehr2025distribution,
title = {A distribution-preserving method for resampling combined with {LightGBM-LSTM} for sequence-wise fraud detection in credit card transactions},
author = {Yousefimehr, Behnam and Ghatee, Mehdi},
journal = {Expert Systems with Applications},
volume = {262},
pages = {125661},
year = {2025},
doi = {10.1016/j.eswa.2024.125661},
publisher = {Elsevier}
}For how this work extends the source method, see What's mine vs. what's from the literature above.
MIT.