Bootstrap the research repository foundation

.editorconfig

@@ -0,0 +1,15 @@
+root = true
+
+[*]
+charset = utf-8
+end_of_line = lf
+insert_final_newline = true
+indent_style = space
+indent_size = 4
+trim_trailing_whitespace = true
+
+[*.{md,yaml,yml}]
+indent_size = 2
+
+[Makefile]
+indent_style = tab

.gitattributes

@@ -0,0 +1,12 @@
+* text=auto eol=lf
+
+*.bat text eol=crlf
+*.cmd text eol=crlf
+*.ps1 text eol=crlf
+*.png binary
+*.jpg binary
+*.jpeg binary
+*.gif binary
+*.pdf binary
+*.parquet binary
+*.safetensors binary

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,22 @@
+## Summary
+
+Describe the change and the research or engineering reason for it.
+
+## Validation
+
+- [ ] `ruff format --check .`
+- [ ] `ruff check .`
+- [ ] `mypy src tests`
+- [ ] `pytest`
+- [ ] `python -m build`
+
+## Research integrity
+
+- [ ] No private prompts, credentials, model weights, or restricted raw data are included.
+- [ ] Dataset and model revisions are pinned or documented where relevant.
+- [ ] Probe, training, and test boundaries remain leakage-safe.
+- [ ] Price-only changes remain outside learned model weights.
+
+## Notes
+
+List configuration, license, compatibility, or follow-up considerations.

.github/dependabot.yml

@@ -0,0 +1,13 @@
+version: 2
+updates:
+  - package-ecosystem: "pip"
+    directory: "/"
+    schedule:
+      interval: "monthly"
+    open-pull-requests-limit: 5
+
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "monthly"
+    open-pull-requests-limit: 5

.github/workflows/ci.yml

@@ -0,0 +1,49 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  quality:
+    name: Python ${{ matrix.python-version }}
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.11", "3.12"]
+
+    steps:
+      - name: Check out repository
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+          cache: pip
+
+      - name: Install package and development tools
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install -e ".[dev]"
+
+      - name: Check formatting
+        run: ruff format --check .
+
+      - name: Lint
+        run: ruff check .
+
+      - name: Type check
+        run: mypy src tests
+
+      - name: Test
+        run: pytest
+
+      - name: Build package
+        run: python -m build

.gitignore

@@ -0,0 +1,56 @@
+# Python
+__pycache__/
+*.py[cod]
+*.pyd
+*.so
+*.egg-info/
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.coverage
+.coverage.*
+htmlcov/
+build/
+dist/
+
+# Environments
+.venv/
+venv/
+.python-version
+
+# Editors and operating systems
+.idea/
+.vscode/
+*.swp
+Thumbs.db
+.DS_Store
+
+# Secrets and local configuration
+.env
+.env.*
+!.env.example
+*.pem
+*.key
+
+# Data: manifests remain tracked; downloaded or transformed data does not
+data/raw/
+data/interim/
+data/processed/
+data/cache/
+*.arrow
+*.parquet
+
+# Model and experiment artifacts
+artifacts/
+checkpoints/
+models/
+runs/
+wandb/
+mlruns/
+*.ckpt
+*.pt
+*.pth
+*.safetensors
+
+# Generated notebook state
+.ipynb_checkpoints/

CONTRIBUTING.md

@@ -0,0 +1,53 @@
+# Contributing
+
+LFM2.5-ModelRouter is currently a pre-alpha research project. Contributions should prioritize reproducibility, leakage prevention, and evidence over feature count.
+
+## Set up the development environment
+
+Use Python 3.11 or 3.12:
+
+```bash
+python -m venv .venv
+python -m pip install --upgrade pip
+python -m pip install -e ".[dev]"
+```
+
+Before submitting a change, run:
+
+```bash
+ruff format --check .
+ruff check .
+mypy src tests
+pytest
+python -m build
+```
+
+## Contribution rules
+
+- Keep dataset-specific behavior inside source adapters.
+- Preserve raw scores and token counts before normalization.
+- Add tests for data validation, price calculations, and split leakage.
+- Pin source revisions in manifests instead of relying on mutable defaults.
+- Keep live prices outside learned model weights.
+- Do not expose hidden counterfactual outcomes to online policies.
+- Store reusable logic in `src/`, not notebooks.
+- Report unsuccessful experiments when they affect research conclusions.
+
+## Data, credentials, and generated files
+
+Never commit:
+
+- API keys or `.env` files.
+- Private prompts or user data.
+- Downloaded benchmark contents unless redistribution is explicitly allowed.
+- Model weights or checkpoints.
+- Local caches and experiment service metadata.
+
+## Pull requests
+
+Keep each pull request focused. Include:
+
+- What changed and why.
+- The checks that were run.
+- Dataset/model revisions affected.
+- Any license, leakage, or reproducibility considerations.

Makefile

@@ -0,0 +1,23 @@
+.PHONY: install format lint typecheck test check build
+
+install:
+	python -m pip install -e ".[dev]"
+
+format:
+	ruff format .
+	ruff check --fix .
+
+lint:
+	ruff format --check .
+	ruff check .
+
+typecheck:
+	mypy src tests
+
+test:
+	pytest
+
+check: lint typecheck test
+
+build:
+	python -m build

SECURITY.md

@@ -0,0 +1,11 @@
+# Security Policy
+
+## Reporting a vulnerability
+
+Do not open a public issue for a suspected vulnerability, exposed credential, or private-data leak. Use GitHub's private vulnerability reporting or Security Advisory workflow for this repository.
+
+Include the affected revision, reproduction steps, impact, and any suggested mitigation. Do not include real user prompts, credentials, or restricted dataset records in the report.
+
+## Supported versions
+
+The project is pre-alpha. Security fixes apply to the latest commit on the default branch unless a release policy is published later.

configs/README.md

@@ -0,0 +1,5 @@
+# Configurations
+
+`base.toml` contains dependency-free repository defaults. Dataset, model, training, simulation, and experiment configurations will be added as their corresponding system parts are implemented.
+
+Configurations must be versioned, explicit, and saved with every reported run. Secrets and machine-specific paths do not belong here.

configs/base.toml

@@ -0,0 +1,8 @@
+[project]
+name = "LFM2.5-ModelRouter"
+seed = 3407
+
+[paths]
+data_dir = "data"
+artifacts_dir = "artifacts"
+reports_dir = "reports"

data/README.md

@@ -0,0 +1,14 @@
+# Data Workspace
+
+Only manifests, schemas, and small legally redistributable fixtures are tracked.
+
+Expected local directories are ignored by Git:
+
+```text
+data/raw/          Unmodified source downloads
+data/interim/      Partially transformed data
+data/processed/    Canonical experiment-ready tables
+data/cache/        Regenerable local caches
+```
+
+Do not place credentials, private prompts, or restricted data in tracked files.

data/manifests/README.md

@@ -0,0 +1,5 @@
+# Dataset Manifests
+
+This directory will hold machine-readable source revisions, file metadata, schema snapshots, checksums, license notes, and split manifests.
+
+No manifest has been created yet because xRouteBench has not completed the required source and license audit.

docs/ARCHITECTURE.md

@@ -0,0 +1,51 @@
+# Architecture
+
+LFM2.5-ModelRouter separates stable capability learning from live decision state and deployment feedback.
+
+```text
+query + candidate profile
+        |
+        v
+offline outcome predictor
+  quality / tokens / latency / failure / uncertainty
+        |
+        v
+runtime optimizer <--- live prices, availability, preferences, constraints
+        |
+        v
+selected action
+        |
+        v
+observed feedback ---> online residual adapter and drift tracking
+```
+
+## Offline predictor
+
+The offline model scores candidate configurations independently. Its primary inputs are:
+
+- An LFM2.5 query representation.
+- Structured candidate metadata.
+- A sparse behavioral probe profile.
+- Optional reasoning mode and output budget.
+
+It predicts quantities that remain meaningful when prices change: success/quality, token usage, latency, failures, and uncertainty.
+
+## Runtime optimizer
+
+The optimizer filters invalid candidates and calculates expected utility from current state. Prices and availability are external inputs, not permanent training labels.
+
+Named user modes are presets over a continuous preference space:
+
+- Intelligence emphasizes expected quality.
+- Balanced trades quality against cost and latency.
+- Cost selects the cheapest action likely to meet a configured success threshold.
+
+## Online adapter
+
+Deployment exposes only the selected model's outcome. The first online design will combine the shared offline prior with discounted per-model linear residuals, uncertainty-aware exploration, and a budget pacer.
+
+## Open candidate set
+
+The registry can add or remove models without resizing a fixed classifier. A new model is initialized from metadata and a small probe set, then receives bounded live exploration.
+
+See [PROJECT_BLUEPRINT.md](PROJECT_BLUEPRINT.md) for implementation order and completion gates.

docs/DATASETS.md

@@ -0,0 +1,31 @@
+# Datasets
+
+## Current scope
+
+xRouteBench is the only dataset planned for the first data milestone. Its current repository revision, files, configurations, splits, schema, missingness, license, and redistribution rules must be audited before full download or adapter implementation.
+
+No raw benchmark data is committed to this repository.
+
+## Planned benchmark roles
+
+| Source | Planned role |
+|---|---|
+| xRouteBench | Primary development data, internal testing, and held-out-model experiments |
+| LLMRouterBench | Later cross-dataset validation on modern math, code, tool use, cost, and latency |
+| RouterArena | External black-box evaluation after model choices are frozen |
+| R2-Bench | Later joint model and output-budget experiments |
+| ParetoBandit protocol | Non-stationary budget and adaptation comparison |
+
+## Required source manifest fields
+
+Every ingested source must record:
+
+- Dataset identifier and immutable revision.
+- Verification date and source URLs.
+- Configurations, splits, row counts, columns, and dtypes.
+- File hashes where practical.
+- License and redistribution notes.
+- Download method and authentication requirements.
+- Known missingness, aliases, duplicates, and evaluation caveats.
+
+The first completed audit will be written to `reports/xroutebench_audit.md` with a machine-readable schema snapshot under `data/manifests/`.