"Veritas" is Latin for truth. This project is about finding it in transaction data.
A real-time fraud detection system built from scratch. 284,807 credit card transactions. 492 of them fraud. The model's job is to find those 492 without crying wolf on the other 284,315.
This is not a tutorial project, but a full pipeline: raw data in, fraud score out, live API running. Built because I wanted to understand how these systems actually work, not just how they look in a notebook.
A machine learning pipeline that scores credit card transactions for fraud probability in real time. You send it a transaction. It tells you whether to be worried.
Under the hood: XGBoost trained on PCA-transformed transaction features, SMOTE to handle the severe class imbalance, SHAP to explain every decision, and a FastAPI service that responds in under 100ms.
The dataset is the UCI Credit Card Fraud Detection dataset, one of the most studied fraud datasets in existence. 0.17% of transactions are fraud. That tiny number is where all the interesting problems live
False negatives let fraud through. False positives block real people from their money. Both are failures, and they pull in opposite directions.
I wanted to build something that takes that tension seriously. Not just "maximize accuracy" (useless on imbalanced data) but: what is the right metric, what is the right threshold, and how do you explain the decision to a human analyst who has to act on it?
That is what this project is about.
creditcard.csv
284,807 transactions
492 fraud cases (0.17%)
|
v
+---------------------+
| preprocess.py | Drop Time, standardize Amount,
| | stratified 80/20 split
+--------+------------+
|
v
+---------------------+
| SMOTE | Synthetic oversampling on training set only.
| | Never touches the test set. No leakage.
+--------+------------+
|
v
+---------------------+
| XGBoost | Early stopping on PR-AUC.
| | scale_pos_weight for extra imbalance correction.
+--------+------------+
|
+----+----+
v v
Evaluate FastAPI
SHAP + /predict
metrics endpoint
Trained on the standard UCI Credit Card Fraud dataset (284,807 transactions, 492 fraud cases, 0.17%). SMOTE is applied only to the training set, so these numbers reflect performance on the real, untouched imbalance, not on synthetically balanced test data.
| Metric | Score |
|---|---|
| ROC-AUC | 0.9768 |
| PR-AUC | 0.8604 |
| Optimal threshold | 0.9915 (F1 = 0.8492) |
| Precision (fraud) | 0.94 |
| Recall (fraud) | 0.78 |
Out of 56,962 test transactions (98 of them real fraud), the model caught 76 and missed 22, at the cost of wrongly flagging just 5 legitimate transactions out of 56,864. The PR curve shows why: precision stays near 1.0 out to roughly 60% recall, then degrades in stages rather than falling off a cliff. The model is highly confident on most fraud cases, and only trades away precision to catch the hardest remaining ones.
Why PR-AUC and not ROC-AUC
ROC-AUC looks great on fraud data because getting the negatives right is easy when 99.83% of transactions are legitimate. PR-AUC only cares about how well you find the fraud. It is harder to game and more honest about what the model is actually doing.
Why SMOTE plus scale_pos_weight
SMOTE resamples the training set to 50/50 before training starts. scale_pos_weight is computed from that already balanced set, so in this run it comes out to exactly 1.0. It is not adding a second layer of correction. It is confirming the first one worked. I originally framed this as two layers stacking together. The actual numbers showed SMOTE was doing all the work on its own, and that is a more honest thing to say about it.
Why threshold tuning
The default 0.5 threshold is almost never right for imbalanced data. This pipeline finds the threshold that maximizes F1 on the validation set. For this run it lands at 0.9915, meaning the model only flags a transaction as fraud when it is over 99% confident. That is not caution by accident. At ROC-AUC 0.98, fraud and legit scores are already well separated, so pushing the bar that high still catches 76 of 98 fraud cases while wrongly flagging only 5 out of 56,864 legitimate ones.
Why SHAP
Because "the model said so" is not good enough. Every prediction gets decomposed into per-feature contributions. V14 and V4 are the two features that matter most, and they push in opposite directions: low V14 pulls toward fraud, high V4 does the same. That is what a fraud analyst actually needs to investigate, not just a single confidence score.
| What | How |
|---|---|
| Model | XGBoost 2.0 |
| Imbalance | SMOTE via imbalanced-learn |
| Explainability | SHAP TreeExplainer |
| API | FastAPI + Uvicorn |
| Data | pandas, numpy, scikit-learn |
| Serialization | XGBoost native JSON |
Mac users, install this first (XGBoost needs it and macOS does not ship it by default):
brew install libompClone and install:
git clone https://github.com/yanastaciocubas/veritas-flag.git
cd veritas-flag
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txtGet the data (one command, no account needed):
mkdir -p data && cd data
curl -o creditcard.csv https://storage.googleapis.com/download.tensorflow.org/data/creditcard.csv
cd ..Train and evaluate:
cd src
python train.pyThis saves the model to models/ and generates all four plots above into outputs/plots/.
Run the API:
cd ../api
uvicorn app:app --reload --port 8000Open http://localhost:8000/docs for the interactive Swagger UI. Try /predict with a sample transaction and watch a real fraud score come back.