🎯 Customer Value & Loyalty Analytics using Machine Learning

An end-to-end customer analytics project that analyzes 500K+ retail transactions to predict Customer Lifetime Value, segment customers using RFM + K-Means clustering, predict churn, and validate all findings with statistical hypothesis tests.

---

📌 Project Highlights

What I Did	Result
Predicted Customer Lifetime Value (CLV)	R² = 0.74 using XGBoost Regression
Segmented customers via RFM + K-Means	3 segments, Silhouette Score = 0.57
Built a Churn Classifier	AUC-ROC = 0.86, Recall = 64%
Validated with Statistical Tests	All findings significant (p < 0.001)

---

📂 Project Structure

├── CLV_COMPLETE_Enhanced.ipynb    # Complete notebook with code + outputs
├── README.md                      # Project documentation
└── transactions_enriched.csv      # Dataset (not included — 502MB)

Note: The dataset (transactions_enriched.csv) is not included due to GitHub's 100MB file size limit.

---

🔍 What This Project Covers

Part 1 — Data Loading & Feature Engineering

• Loaded 541,909 transactions across 2,500 customers
• Created a temporal train-target split (120-day cutoff) to avoid data leakage
• Engineered 15+ customer-level features using multi-window temporal aggregation:

Time Window	What It Captures
Last 2 weeks	Very recent intent
Last 1.5 months	Short-term trend
Last 3 months	Medium-term behavior
Last 4 months	Quarterly pattern
Last 6 months	Long-term loyalty

Each window includes: store_visits, total_spend, avg_spend, total_items, avg_items

Part 2 — Exploratory Data Analysis (EDA)

Key findings from EDA:

• Target distribution is heavily right-skewed (skewness = 2.42) — most customers spend little, a few "whale" customers spend $5,000+
• Mean CLV = $633 vs Median = $397 — mean is pulled up by big spenders
• 4.9% of customers churned (zero future spend)
• Correlation analysis showed medium-term spending windows (3-4 months, r = 0.88) are the strongest CLV predictors
• High multicollinearity between time windows (r > 0.95) — one reason simpler models performed well

Part 3 — CLV Prediction (Regression)

Compared 5 models using 5-fold cross-validation:

Model	CV RMSE
Linear Regression	325.07 ✅
Random Forest	349.20
Gradient Boosting	356.72
XGBoost	361.60
LightGBM	368.68

Why did Linear Regression win? With only ~2,500 customers, complex ensemble models overfit. Linear Regression's simplicity acts as regularization. With more data (50K+), XGBoost would likely outperform.

XGBoost results: MAE = $227 | RMSE = $337 | R² = 0.74

SHAP analysis revealed that recent spending windows and visit frequency are the most important features.

Part 4 — RFM Segmentation + K-Means Clustering

Computed Recency, Frequency, and Monetary for each customer, standardized the features, and applied K-Means clustering.

Optimal K = 3 (selected via Elbow Method + Silhouette Score = 0.57)

Segment	Customers	Avg Recency	Avg Frequency	Avg Monetary	Profile
Cluster 1	442 (18%)	4 days	284 trips	$8,713	🏆 Champions
Cluster 0	1,987 (79%)	20 days	75 trips	$2,096	👤 Regulars
Cluster 2	71 (3%)	324 days	23 trips	$574	🚪 Lost

Part 5 — Churn Prediction (Classification)

Churn definition: Customer with $0 spend in next 120 days.

Challenge: Severe class imbalance — only 4.9% churned. A model predicting "not churned" for everyone gets 95% accuracy but catches zero churners.

Solution:

• Used class_weight='balanced' to penalize minority class errors more heavily
• Optimized decision threshold from 0.50 → 0.12 using precision-recall curve

Metric	Before (threshold=0.50)	After (threshold=0.12)
Recall	4% (caught 1/25)	64% (caught 16/25)
F1-Score	0.06	0.39
AUC-ROC	0.86	0.86

Key discovery: 100% of Lost segment churned, while 0% of Champions churned — perfectly validating the RFM segmentation.

Top churn predictors: total_spend4month, store_visits1_5_month, total_spend3month

Part 6 — Statistical Validation

Test	Purpose	Result
Shapiro-Wilk	Normality check	Data is NOT normal (p < 0.001)
Mann-Whitney U	Churned vs Retained spending	Significant — churned spent 3x less ($851 vs $2,682, p < 0.001, Cohen's d = 0.86)
Kruskal-Wallis	Spending across segments	Significant — segments are genuinely different (H = 1074, p < 0.001)
Chi-Square	Churn vs segment dependency	Significant — churn rate depends on segment (χ² = 1416, p < 0.001)

---

💡 Business Recommendations

Segment	Action	Rationale
🏆 Champions (442)	VIP rewards, early access, loyalty programs	0% churn — protect these high-value customers
👤 Regulars (1,987)	Personalized offers, upselling	3% churn — push toward Champion status
🚪 Lost (71)	Win-back campaigns or write-off	100% churn — heavy discounts if ROI positive
⚠️ Early Warning	Monitor 1.5-month visit frequency	Top churn predictor — flag declining customers early

---

🛠️ Tech Stack

• Language: Python 3.12
• Data Manipulation: Pandas, NumPy
• Visualization: Matplotlib, Seaborn
• Machine Learning: Scikit-learn, XGBoost, LightGBM
• Explainability: SHAP
• Statistical Testing: SciPy (Shapiro-Wilk, Mann-Whitney U, Kruskal-Wallis, Chi-Square)

---

🚀 How to Run

1. Clone the repository:

   git clone https://github.com/UtkarshVerma1998/Customer-Value-and-Loyalty-Analytics-using-Machine-Learning..git

2. Install dependencies:

   pip install pandas numpy matplotlib seaborn scikit-learn xgboost lightgbm shap scipy

3. Place the transactions_enriched.csv dataset in the same folder as the notebook.

4. Open CLV_COMPLETE_Enhanced.ipynb in Jupyter Notebook or VS Code and run all cells.

---

👤 Author

Utkarsh Verma

• M.Tech, IIT Kanpur
• LinkedIn
• GitHub

---