Add governed recommendation baseline

Author: Ibraheem Olasupo

.github/workflows/ci.yml

@@ -57,3 +57,6 @@ jobs:
 
       - name: Verify deterministic segmentation evidence
         run: make verify-segmentation-evidence
+
+      - name: Verify deterministic recommendation evidence
+        run: make verify-recommendation-evidence

Makefile

@@ -1,4 +1,4 @@
-.PHONY: install format lint type-check test quality project-info generate-sample ingest-sample verify-ingestion-evidence analyse-funnels-sample verify-funnel-evidence analyse-retention-sample verify-retention-evidence train-churn-sample verify-churn-evidence segment-users-sample verify-segmentation-evidence
+.PHONY: install format lint type-check test quality project-info generate-sample ingest-sample verify-ingestion-evidence analyse-funnels-sample verify-funnel-evidence analyse-retention-sample verify-retention-evidence train-churn-sample verify-churn-evidence segment-users-sample verify-segmentation-evidence build-recommendations-sample verify-recommendation-evidence
 
 install:
 	python3 -m pip install --upgrade pip
@@ -60,3 +60,10 @@ segment-users-sample:
 verify-segmentation-evidence:
 	python3 scripts/generate_segmentation_evidence.py
 	git diff --exit-code -- docs/evidence/milestone-7
+
+build-recommendations-sample:
+	python3 scripts/generate_recommendation_evidence.py
+
+verify-recommendation-evidence:
+	python3 scripts/generate_recommendation_evidence.py
+	git diff --exit-code -- docs/evidence/milestone-8

README.md

@@ -25,7 +25,7 @@ The repository is written for product data scientists, product analysts, growth
 
 Planned capabilities include synthetic product event generation, clickstream ingestion, validation, funnel analytics, cohort retention, churn prediction, segmentation, recommendation modelling, controlled A/B testing, customer feedback intelligence, GenAI-assisted product insights, Power BI-ready outputs, and Azure-aligned security, governance, monitoring, and deployment patterns.
 
-Milestones 1, 2, 3, 4, 5, 6, and 7 implement the repository foundation, deterministic synthetic NexaFlow data generation, local event ingestion with data-quality validation, governed funnel analytics, governed retention/cohort analytics, leakage-aware churn prediction, and governed user segmentation. Later recommendation, experimentation, GenAI, dashboard, and Azure deployment work remains planned.
+Milestones 1, 2, 3, 4, 5, 6, 7, and 8 implement the repository foundation, deterministic synthetic NexaFlow data generation, local event ingestion with data-quality validation, governed funnel analytics, governed retention/cohort analytics, leakage-aware churn prediction, governed user segmentation, and governed recommendation baselines. Later experimentation, GenAI, dashboard, and Azure deployment work remains planned.
 
 ## Azure Service Mapping
 
@@ -35,8 +35,9 @@ Milestones 1, 2, 3, 4, 5, 6, and 7 implement the repository foundation, determin
 | Raw, trusted, and quarantine storage | Azure Data Lake Storage Gen2 | Local raw, interim accepted, and quarantine zones implemented |
 | Stream processing | Azure Stream Analytics or Azure Functions | Local deterministic micro-batch simulation implemented |
 | Analytical serving | Azure Synapse Analytics | Local governed funnel outputs implemented |
-| Model training and tracking | Azure Machine Learning | Local deterministic churn training implemented |
+| Model training and tracking | Azure Machine Learning | Local deterministic churn and recommendation baseline training implemented |
 | User segmentation | Azure Machine Learning | Local governed rule-based and KMeans segmentation implemented |
+| Recommendation batch generation | Azure Machine Learning | Local governed recommendation outputs implemented |
 | GenAI insights | Azure AI Foundry and Azure OpenAI | Planned |
 | Dashboards | Power BI | Planned |
 | Observability | Azure Monitor and Application Insights | Planned configuration placeholders |
@@ -105,9 +106,15 @@ Feature groups cover account context, activity level, engagement depth, collabor
 
 Runtime segmentation outputs are written under `outputs/models/segmentation/<segmentation_run_id>/`, which is ignored by Git. Concise reproducible evidence for the committed sample is stored in `docs/evidence/milestone-7/`.
 
+## Governed Recommendation Baseline
+
+Milestone 8 adds deterministic recommendation baselines over trusted Milestone 3 accepted outputs. The workflow creates a governed product-action catalogue, maps clickstream events into implicit-feedback strengths, builds point-in-time user-item interactions, applies plan/persona/company-size eligibility, generates candidates, compares popularity, recent-popularity, segment-aware popularity, and item-item collaborative-filtering baselines, and reports offline ranking, coverage, novelty, diversity, segment, and cold-start metrics.
+
+Runtime recommendation outputs are written under `outputs/models/recommendations/<recommendation_run_id>/`, which is ignored by Git. Concise reproducible evidence for the committed sample is stored in `docs/evidence/milestone-8/`. The workflow is an offline synthetic-data baseline only; it is not an online recommender, causal treatment policy, or experiment readout.
+
 ## Analytics, ML, and GenAI Use Cases
 
-Analytics use cases include active user tracking, journey funnels, feature adoption, retention, churn, resurrection, customer lifetime value, and governed user segmentation. ML use cases now include local churn prediction and segmentation demonstrations; recommendation baselines and experiment uplift interpretation remain planned. GenAI use cases will focus on grounded summarisation, feedback theme extraction, insight narratives, and human-reviewed product action recommendations.
+Analytics use cases include active user tracking, journey funnels, feature adoption, retention, churn, resurrection, customer lifetime value, and governed user segmentation. ML use cases now include local churn prediction, segmentation, and recommendation baseline demonstrations; experiment uplift interpretation remains planned. GenAI use cases will focus on grounded summarisation, feedback theme extraction, insight narratives, and human-reviewed product action recommendations.
 
 ## Repository Structure
 
@@ -151,7 +158,7 @@ Analytics use cases include active user tracking, journey funnels, feature adopt
 | 5. Retention and cohort analysis | Measure product stickiness | Cohort tables and retention views | Windowing tests | Completed |
 | 6. Churn prediction | Identify at-risk users | Leakage-aware features and model training | Reproducibility and evaluation tests | Completed |
 | 7. User segmentation | Explain behavioural groups | Rule-based and KMeans segmentation | Determinism and profile tests | Completed |
-| 8. Recommendation baseline | Suggest items or features | Baseline recommender | Ranking tests | Recommendation outputs |
+| 8. Recommendation baseline | Suggest items or features | Baseline recommender | Ranking tests | Completed |
 | 9. A/B testing analysis | Evaluate product changes | Experiment analysis module | Statistical tests | Experiment readout |
 | 10. GenAI product insight assistant | Summarise grounded insights | Prompting and grounding layer | Mocked GenAI tests | Insight briefs |
 | 11. Power BI-ready outputs | Serve decision-ready datasets | Export tables and semantic docs | Schema tests | Dashboard-ready files |
@@ -187,6 +194,8 @@ make train-churn-sample
 make verify-churn-evidence
 make segment-users-sample
 make verify-segmentation-evidence
+make build-recommendations-sample
+make verify-recommendation-evidence
 ```
 
 Generate a synthetic run directly:
@@ -259,6 +268,18 @@ python3 -m product_growth_intelligence segment-users \
   --fixed-run-time 2026-01-02T00:00:00Z
 ```
 
+Run governed recommendations against trusted ingestion output:
+
+```bash
+python3 -m product_growth_intelligence build-recommendations \
+  --input-dir data/interim/<ingestion_run_id> \
+  --output-root outputs/models/recommendations \
+  --snapshot-time 2025-03-31T23:59:59Z \
+  --lookback-days 56 \
+  --holdout-days 28 \
+  --fixed-run-time 2026-01-02T00:00:00Z
+```
+
 ## Quality and Security Principles
 
 The implementation favours typed Python, deterministic behaviour, small interfaces, no embedded secrets, no generated data in Git, clear metric ownership, local validation by default, and Azure-specific adapters only where they are useful. Future Azure deployments should use managed identity, RBAC, Key Vault, private networking where appropriate, and monitoring that avoids leaking customer data.
@@ -278,7 +299,8 @@ The implementation favours typed Python, deterministic behaviour, small interfac
 | Retention and cohort analytics | Completed | Versioned retention definitions, cohort periods, censoring, lifecycle, evidence |
 | Churn prediction | Completed | Local deterministic model training, evaluation, model card, evidence |
 | User segmentation | Completed | Rule-based segments, KMeans, stability, PCA, profiles, evidence |
-| Recommendations, GenAI | Planned | Milestones 8-12 are not implemented |
+| Recommendation baseline | Completed | Local deterministic candidate generation, ranking, metrics, reasons, evidence |
+| Experimentation, GenAI | Planned | Milestones 9-12 are not implemented |
 | Azure deployment | Optional Azure deployment | No live resources required |
 
 ## Synthetic-Data Disclaimer

docs/architecture/azure-service-mapping.md

@@ -13,12 +13,15 @@ The platform is local-first, but each major component is designed to map cleanly
 | Funnel transformations | Local governed funnel pipeline | Azure Synapse Analytics |
 | Cohort transformations | Local governed retention pipeline | Azure Synapse Analytics |
 | Churn feature preparation | Local point-in-time churn feature builder | Azure Synapse Analytics or Azure ML data preparation |
+| Recommendation interaction preparation | Local point-in-time interaction and candidate builder | Azure Synapse Analytics or Azure ML data preparation |
 | Scheduled analytics | Makefile/CI commands | Azure Data Factory or Synapse pipelines |
 | Model training | Local deterministic churn training scripts | Azure Machine Learning |
 | Model tracking | Local reports, manifests, lineage, and metadata | Azure ML jobs, MLflow, and Azure ML registry |
 | User segmentation | Local rule-based and KMeans segmentation | Azure Machine Learning |
+| Recommendation baselines | Local popularity, segment-aware, and item-item CF baselines | Azure Machine Learning |
 | Batch scoring | Local prediction CSV outputs | Azure ML batch endpoints |
 | Batch segment assignment | Local assignment CSV outputs | Azure ML batch endpoints |
+| Batch recommendation generation | Local recommendation CSV and reason JSONL outputs | Azure ML batch endpoints or Synapse scheduled jobs |
 | Online scoring | Future scope only | Azure ML managed online endpoints |
 | GenAI insights | Mocked or disabled by default | Azure AI Foundry and Azure OpenAI |
 | Dashboards | Local documented outputs | Power BI |
@@ -36,4 +39,4 @@ The platform is local-first, but each major component is designed to map cleanly
 - Infrastructure templates are planned, not active, in Milestone 6.
 - No Azure service is deployed by the current repository.
 
-The churn and segmentation workflows map trusted accepted data to ADLS Gen2 trusted zones, feature preparation to Synapse or Azure ML data preparation, experiment tracking to Azure ML jobs and MLflow, model registry to Azure ML registry, monitoring to Azure Monitor and Azure ML monitoring, secrets to Key Vault, identity to managed identity and Azure RBAC, governance and lineage to Microsoft Purview, and dashboard consumption to Power BI. The local implementation does not install Azure SDKs, create clients, deploy resources, or claim an Azure ML endpoint exists.
+The churn, segmentation, and recommendation workflows map trusted accepted data to ADLS Gen2 trusted zones, feature and interaction preparation to Synapse or Azure ML data preparation, experiment tracking to Azure ML jobs and MLflow, model registry to Azure ML registry, monitoring to Azure Monitor and Azure ML monitoring, secrets to Key Vault, identity to managed identity and Azure RBAC, governance and lineage to Microsoft Purview, and dashboard consumption to Power BI. The local implementation does not install Azure SDKs, create clients, deploy resources, or claim an Azure ML endpoint exists.

docs/architecture/data-flow.md

@@ -1,6 +1,6 @@
 # Data Flow
 
-This document describes the logical data flow. Milestone 3 implements the local raw-to-interim ingestion and validation portion. Milestone 4 implements governed funnel analytics over trusted interim data. Milestone 5 implements governed retention and cohort analytics. Milestone 6 implements local leakage-aware churn prediction. Milestone 7 implements governed user segmentation. GenAI, dashboarding, and Azure deployment remain planned.
+This document describes the logical data flow. Milestone 3 implements the local raw-to-interim ingestion and validation portion. Milestone 4 implements governed funnel analytics over trusted interim data. Milestone 5 implements governed retention and cohort analytics. Milestone 6 implements local leakage-aware churn prediction. Milestone 7 implements governed user segmentation. Milestone 8 implements governed recommendation baselines. Experiment analysis, GenAI, dashboarding, and Azure deployment remain planned.
 
 ```mermaid
 sequenceDiagram
@@ -49,6 +49,8 @@ Milestone 6 writes churn-model outputs under `outputs/models/churn/<model_run_id
 
 Milestone 7 writes segmentation outputs under `outputs/models/segmentation/<segmentation_run_id>/`, including segmentation definition, feature catalogue, snapshots, feature matrix, rule-based assignments, cluster candidate metrics, stability, assignments, profiles, centroids, PCA outputs, segment names, metadata, diagnostics, manifest, lineage, and segment card.
 
+Milestone 8 writes recommendation outputs under `outputs/models/recommendations/<recommendation_run_id>/`, including recommendation definition, item catalogue, interaction mapping, point-in-time user-item interactions, candidate items, model comparison, offline metrics, metrics by K, segment metrics, cold-start metrics, item similarity, recommendations, deterministic reasons, catalogue coverage, metadata, diagnostics, manifest, lineage, and recommendation card.
+
 ```mermaid
 flowchart LR
     A[Trusted accepted data] --> B[Point-in-time snapshots]
@@ -77,3 +79,18 @@ flowchart LR
     H --> J[Segment card manifest lineage]
     I --> J
 ```
+
+```mermaid
+flowchart LR
+    A[Trusted accepted data] --> B[Recommendation snapshot]
+    B --> C[Historical interactions]
+    B --> D[Catalogue eligibility]
+    C --> E[Popularity and similarity baselines]
+    D --> F[Candidate generation]
+    F --> G[Ranked recommendations]
+    E --> G
+    B --> H[Future holdout window]
+    G --> I[Offline ranking metrics]
+    H --> I
+    I --> J[Manifest card lineage]
+```

docs/architecture/metric-framework.md

@@ -1,6 +1,6 @@
 # Metric Framework
 
-This document defines governed metric concepts. Milestone 4 implements operational funnel formulas and denominator rules for descriptive product journeys. Milestone 6 adds governed churn-model evaluation definitions.
+This document defines governed metric concepts. Milestone 4 implements operational funnel formulas and denominator rules for descriptive product journeys. Milestone 6 adds governed churn-model evaluation definitions. Milestone 8 adds offline recommendation-ranking metric definitions.
 
 | Metric | Conceptual definition |
 | --- | --- |
@@ -96,3 +96,21 @@ Churn model selection uses validation metrics only. The default rule ranks by va
 | `SEGMENT_PROFILE_SHARE` | Segment population share | Segment users divided by eligible users | Null when no eligible users exist |
 
 Segmentation candidates are selected by rejecting clusters below the minimum-size rule, prioritising silhouette score, using stability as a tie-breaker, and preferring fewer clusters where quality is materially similar. Segment profiles are descriptive and suppression-aware.
+
+## Recommendation Ranking Metrics
+
+| Metric ID | Business meaning | Definition | Null handling |
+| --- | --- | --- | --- |
+| `REC_PRECISION_AT_K` | Share of top-K recommendations that appear in future holdout interactions | Relevant recommended items in top K divided by recommended items in top K | Zero when no evaluated recommendations exist |
+| `REC_RECALL_AT_K` | Share of future relevant items captured in top K | Relevant recommended items in top K divided by relevant holdout items | Zero when no relevant holdout items exist |
+| `REC_HIT_RATE_AT_K` | Share of evaluated users with at least one relevant top-K recommendation | Users with a hit in top K divided by evaluated users | Zero when no evaluated users exist |
+| `REC_MRR` | Ranking position of first relevant recommendation | Mean reciprocal rank of first hit per evaluated user | Zero when no first hit exists |
+| `REC_MAP_AT_K` | Average precision across the ranked top-K list | Mean average precision at K over evaluated users | Zero when no evaluated users exist |
+| `REC_NDCG_AT_K` | Position-aware ranking quality | Normalised discounted cumulative gain at K against binary holdout relevance | Zero when ideal DCG is zero |
+| `REC_CATALOGUE_COVERAGE_AT_K` | Breadth of catalogue exposure | Distinct items recommended in top K divided by active catalogue items | Zero when catalogue is empty |
+| `REC_USER_COVERAGE` | Share of eligible users receiving recommendations | Users with recommendation rows divided by eligible users with model output | Zero when no eligible users exist |
+| `REC_NOVELTY_AT_K` | Preference for less historically common items | Mean negative log historical item share with smoothing | Zero when no recommendation rows exist |
+| `REC_DIVERSITY_AT_K` | Variety of item categories within recommendation lists | Mean distinct item categories divided by recommended items | Zero when no recommendation rows exist |
+| `REC_FALLBACK_RATE` | Share of recommendations produced by fallback logic | Fallback recommendation rows divided by recommendation rows | Zero when no recommendation rows exist |
+
+Recommendation metrics are descriptive offline metrics over synthetic data. The future holdout window is used only for evaluation and never for candidate generation or scoring. Model selection rejects models below coverage guardrails, prioritises NDCG@5, uses recall@5 as a tie-breaker, and prefers simpler baselines where quality is materially similar.