Customer Lens

ML suite for customer propensity scoring, segmentation, and CLV estimation.

Architecture

flowchart TB
    subgraph load["Data loading (src/sql_queries.py)"]
        PG[("Postgres<br/>customers / orders / sessions")]
        CSV["data/sample_customers.csv"]
        LD["load_data<br/>picks SQL or CSV from config"]
        PG --> LD
        CSV --> LD
    end

    LD --> FE["build_feature_matrix<br/>median impute + one-hot dummies"]
    LD --> SD["build_survival_data<br/>duration = tenure_days, event = churned"]

    subgraph prop["Propensity (src/propensity.py)"]
        FE --> SPLIT["train_test_split"]
        SPLIT --> GS1["GridSearchCV<br/>LogisticRegression pipeline"]
        SPLIT --> GS2["GridSearchCV<br/>XGBClassifier pipeline"]
        GS1 --> PICK{"higher AUC-ROC?"}
        GS2 --> PICK
        PICK --> PM["best_propensity_model.joblib"]
        PICK --> ROC["outputs/roc_curve.png"]
    end

    subgraph seg["Segmentation (src/segmentation.py)"]
        FE --> SEL["select numeric columns"]
        SEL --> SC["StandardScaler"]
        SC --> KSEL["find_optimal_k<br/>inertia + silhouette over k_range"]
        KSEL --> KM["kmeans.joblib + seg_scaler.joblib"]
        KSEL --> EPNG["outputs/elbow_silhouette.png"]
    end

    subgraph surv["Survival / CLV (src/survival.py)"]
        SD --> KMF["KaplanMeierFitter"]
        SD --> COX["CoxPHFitter"]
        COX --> CLV["predict_median x revenue_per_period<br/>outputs/clv_estimates.csv"]
        KMF --> KPNG["outputs/km_curve.png"]
    end

    PM --> COMB["src/score.py<br/>reload models and score every customer"]
    KM --> COMB
    CLV --> COMB
    COMB --> OUT["outputs/scored_customers.csv<br/>customer_id, propensity_score, segment_id, estimated_clv"]

Loading

Overview

• Propensity Modeling: Logistic Regression + XGBoost with GridSearchCV; selects best by AUC-ROC
• Customer Segmentation: K-means with Elbow/Silhouette cluster selection
• CLV Estimation: Kaplan-Meier + Cox Proportional Hazards survival analysis
• Score Export: Combined (customer_id, propensity_score, segment_id, estimated_clv) CSV

Tech Stack

Python 3.11 · Pandas · scikit-learn · XGBoost · lifelines · SQLAlchemy · Matplotlib

Quickstart

python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# Generate synthetic data
python data/generate_sample.py

# Run each pipeline stage
python -m src.propensity     # trains + saves best model
python -m src.segmentation   # k-means clustering
python -m src.survival       # CLV estimation
python -m src.score          # combine → outputs/scored_customers.csv

# Run tests
pytest tests/ -v

Project Structure

customer_lens/
├── src/
│   ├── sql_queries.py      # SQL / CSV data loading
│   ├── features.py         # Feature engineering, imputation
│   ├── propensity.py       # LR + XGB training, ROC comparison
│   ├── segmentation.py     # K-means + elbow/silhouette selection
│   ├── survival.py         # KM + Cox PH CLV estimation
│   └── score.py            # Score combiner + export
├── configs/config.yaml
├── data/generate_sample.py
├── tests/
└── outputs/                # roc_curve.png, km_curve.png, scored_customers.csv

Tests

pytest tests/ -v

Covers feature engineering (tests/test_features.py), propensity training/selection (tests/test_propensity.py), and segmentation (tests/test_segmentation.py). End-to-end check:

python data/generate_sample.py
python -m src.propensity && python -m src.segmentation && python -m src.survival && python -m src.score
test -s outputs/scored_customers.csv && head outputs/scored_customers.csv

Evaluation

Metrics by stage (computed by the scripts under src/):

Propensity (binary classification) — src/propensity.py runs GridSearchCV, prints and saves:

• AUC-ROC (primary model-selection criterion; LR vs XGBoost)
• Precision, Recall, F1 at the default 0.5 threshold and at the threshold that maximizes F1
• Confusion matrix on the held-out fold
• Calibration curve / Brier score (recommended addition for downstream targeting)
• ROC curve image written to outputs/roc_curve.png

Segmentation (clustering) — src/segmentation.py:

• Silhouette score over k ∈ [3, 8] for cluster-count selection
• Within-cluster sum of squares (Elbow plot) saved under outputs/
• Optionally Davies-Bouldin / Calinski-Harabasz indices

CLV / Survival — src/survival.py:

• Concordance index (C-index) for the Cox PH model on the test split
• Kaplan-Meier curves with 95% CIs (outputs/km_curve.png)
• Mean absolute error of predicted vs realized lifetime on a hold-out cohort (when ground-truth churn dates are available)

CLV is reported as median survival time × $50/period revenue assumption; sensitivity to that constant should be reported alongside.