Skip to content

README.md

Latest commit

 

History

History
141 lines (99 loc) · 4.6 KB

README.md

File metadata and controls

141 lines (99 loc) · 4.6 KB
Raw

Claim Denial Risk Modeling

This repository contains the analysis and implementation artifacts for claim denial risk modeling.

The problem is framed as a pre-submission claim review workflow. The review team can inspect only the top 25% of current claims by risk score, so the work is organized around ranked worklist quality, leakage-safe feature engineering, and clear analyst-facing explanations.

Repository Layout

data/raw/                     Source PDF and CSV files
config/                       Modeling configuration and experiment settings
notebooks/                    EDA, modeling, scoring, and GenAI review notebooks
prompts/                      Versioned LLM prompt templates
src/                          Reusable training, scoring, and explanation code
reports/                      Final stakeholder-facing report

Current Artifacts

  • notebooks/01_eda_decision_analysis.ipynb: step-by-step EDA, visual analysis, and decision records.
  • notebooks/02_modeling_and_scoring.ipynb: model training, evaluation, and current-claim scoring.
  • notebooks/03_genai_explanations.ipynb: prompt design, top-10 explanations, and low-risk sanity check.
  • config/modeling.toml: review capacity, feature schema, risk tiers, model hyperparameters, and MLflow settings.
  • config/prompting.toml: prompt version, provider, output paths, and quality gates.
  • prompts/claim_denial_explanation_v1.md: versioned prompt template.
  • src/modeling_pipeline.py: reusable feature engineering, evaluation, MLflow, and scoring utilities.
  • src/train_model.py: trains candidate models, tracks experiments, and registers the local MLflow champion.
  • src/score_current_claims.py: loads the registered champion model and scores current claims.
  • src/generate_explanations.py: generates and evaluates top-risk claim explanations.
  • predictions_current_claims.csv: generated current-claim denial risk scores.
  • top10_claim_explanations.csv: top-10 highest-risk explanations for analyst review.
  • reports/final_claim_denial_report.pdf: concise stakeholder-facing final report.

Environment Setup

This project uses uv for environment management and requirements.txt for Python dependencies.

Install uv if it is not already available:

pip install uv

Create a project-local virtual environment:

uv venv --python 3.12

Install dependencies:

uv pip install -r requirements.txt

For VS Code or Jupyter, select the notebook kernel from:

.venv/bin/python

If uv cannot write to its default cache location on a restricted machine, run the commands with a project-local cache:

UV_CACHE_DIR=.uv-cache uv venv --python 3.12
UV_CACHE_DIR=.uv-cache uv pip install -r requirements.txt

Review EDA

Open notebooks/01_eda_decision_analysis.ipynb.

Run Modeling And Scoring

Train candidate models, compare them under the top-25% review constraint, and register the selected champion model in the local MLflow registry:

uv run python -m src.train_model

Score current_claims.csv by loading the registered champion model:

uv run python -m src.score_current_claims

This writes:

predictions_current_claims.csv
outputs/model_metrics.json
outputs/model_metadata.json
outputs/mlflow/

predictions_current_claims.csv is sorted from highest to lowest denial_probability.

To run training and scoring in one command:

uv run python src/modeling_pipeline.py

If you have already activated the project environment, replace uv run python with python.

Run GenAI Explanations

Generate the top-10 highest-risk explanations and low-risk sanity evaluation:

uv run python -m src.generate_explanations

By default, the prompt workflow is configured for NVIDIA NIM through its chat completions endpoint. For a live LLM run, copy .env.example to .env and set:

NVIDIA_API_KEY=...
NVIDIA_API_BASE_URL=https://integrate.api.nvidia.com/v1
NVIDIA_MODEL=meta/llama-3.1-8b-instruct

If NVIDIA_API_KEY is not available, the script uses the deterministic grounded_local fallback and records that provider choice in the output artifacts.

This writes:

top10_claim_explanations.csv
prompt_evaluation_report.json
outputs/prompt_runs/

Project Package

The project is ready to run with:

  • reproducible setup instructions,
  • notebooks that show the analysis and decision flow,
  • reusable Python scripts for model training, registry-based scoring, and explanation generation,
  • scored current-claim predictions,
  • top-10 GenAI explanations with prompt-evaluation metadata,
  • final PDF report at reports/final_claim_denial_report.pdf.