Merge pull request #8 from MaharshPatelX/codex/local-350m-compute-plan

docs: adopt local 350M compute plan

Author: MaharshPatelX

README.md

@@ -4,9 +4,9 @@
 
 Research framework for routing prompts across evolving LLM portfolios with a compact Liquid AI encoder.
 
-> **Research status:** Pre-alpha. Parts 0–2 are merged. Part 3 is implemented and validated locally, and Part 4 now produces five real leakage-safe xRouteBench manifests plus an explicit unsupported-temporal record. Review and merge are still pending. No benchmark results or trained router checkpoint are claimed yet.
+> **Research status:** Pre-alpha. Parts 0–4 are merged. The six canonical tables and five real leakage-safe xRouteBench manifests are reproducible; temporal is explicitly unsupported because the source has no dates. Part 5's baseline suite is next. No benchmark results or trained router checkpoint are claimed yet.
 >
-> **Project progress:** [3 of 17 core parts merged — view the tracker](docs/PROJECT_TRACKER.md)
+> **Project progress:** [5 of 17 core parts merged — view the tracker](docs/PROJECT_TRACKER.md)
 
 ## Research question
 
@@ -38,7 +38,7 @@ Development order:
 8. Online bandit adapter and budget pacing.
 9. External benchmarks, ablations, and research release.
 
-See the [project tracker](docs/PROJECT_TRACKER.md) for current progress, the [project blueprint](docs/PROJECT_BLUEPRINT.md) for the full ordered build plan, the [Part 3 canonical-data guide](docs/PART3_CANONICAL_DATA.md) for the real table build, the [Part 4 split contract](docs/PART4_SPLITS.md) for leakage rules and storage setup, and the [research handoff](docs/RESEARCH_HANDOFF.md) for the literature and dataset review.
+See the [project tracker](docs/PROJECT_TRACKER.md) for current progress, the [project blueprint](docs/PROJECT_BLUEPRINT.md) for the full ordered build plan, the [local compute plan](docs/COMPUTE_PLAN.md) for the 350M-only hardware policy, the [Part 3 canonical-data guide](docs/PART3_CANONICAL_DATA.md) for the real table build, the [Part 4 split contract](docs/PART4_SPLITS.md) for leakage rules and storage setup, and the [research handoff](docs/RESEARCH_HANDOFF.md) for the literature and dataset review.
 
 ## Repository layout
 

docs/COMPUTE_PLAN.md

@@ -0,0 +1,103 @@
+# Local Compute Plan
+
+## Active decision
+
+The current project trains and evaluates **only**
+`LFM2.5-ModelRouter-350M`. Larger LFM variants are not part of the active MVP
+or its compute budget. They may be reconsidered only after the 350M research
+release is complete.
+
+| Resource | Current plan |
+|---|---|
+| Query encoder scale | 350M parameters only |
+| Encoder candidate | `LiquidAI/LFM2.5-Embedding-350M`; pin the exact revision in Part 6 |
+| Primary accelerator | Local AMD Radeon RX 7900 XTX with 24 GB VRAM |
+| Host memory | Existing 16 GB RAM |
+| Cloud GPU | Fallback only; not the default training environment |
+| Default training mode | Frozen encoder with cached query embeddings |
+| Fine-tuning | Partial/full fine-tuning only as a later controlled ablation |
+
+There is no RAM-upgrade requirement before development or training begins.
+The project will measure the real workload before recommending any hardware
+purchase.
+
+## Why this is practical
+
+The current canonical xRouteBench build contains 15,339 eligible queries and
+231,750 query/model outcomes. The 350M encoder processes each unique query
+once, and the resulting embedding is referenced by outcome rows rather than
+copied 18 times.
+
+This design keeps both GPU and system-memory use bounded:
+
+- Read Parquet in batches and select only required columns.
+- Cache one embedding per stable query ID.
+- Store outcome-to-query indices instead of repeated embedding tensors.
+- Begin with zero or one data-loader worker.
+- Use small GPU batches plus gradient accumulation when required.
+- Keep only the current and best checkpoint.
+- Avoid CPU or disk offloading unless a measured run requires it.
+
+## Execution by project stage
+
+| Project stage | Default compute |
+|---|---|
+| Part 5 baselines | Local CPU; GPU optional for small learned baselines |
+| Part 6 embedding cache | Local RX 7900 XTX |
+| Parts 7–11 model training and evaluation | Local RX 7900 XTX |
+| Parts 12–13 replay and online policies | CPU first; local GPU only for neural components |
+| Parts 14–15 external tests and ablations | Local GPU in queued, resumable runs |
+| Part 16 release verification | Local reproduction plus optional rented Nvidia cross-check |
+
+## Hardware acceptance gate
+
+When the RX 7900 XTX is installed, Part 6 must first run a bounded acceptance
+test:
+
+1. Create a separate, pinned Python/ROCm environment.
+2. Verify that PyTorch detects the expected GPU and 24 GB VRAM.
+3. Load the 350M encoder and encode a small fixture.
+4. Benchmark 1,000 real canonical queries.
+5. Run 100 optimizer steps with backward propagation.
+6. Record peak VRAM, peak host RAM, throughput, temperature and failures.
+7. Extrapolate measured runtime before scheduling a full run.
+
+The initial Windows route must follow AMD's current
+[PyTorch compatibility matrix](https://rocm.docs.amd.com/projects/radeon-ryzen/en/latest/docs/compatibility/compatibilityrad/windows/windows_compatibility.html).
+If a required training operation is unsupported, native Linux is the next
+local option.
+
+## Host-memory rule
+
+The existing 16 GB of RAM is accepted for Parts 5–6 and remains the default
+for later work unless measurements show a real problem. An upgrade is
+considered only after reproducible out-of-memory failures or sustained paging
+under the bounded loader design.
+
+## Cloud fallback rule
+
+A rented GPU is allowed only when one of these conditions is recorded:
+
+- A required operation is unsupported by the validated local ROCm stack.
+- A deadline makes a long multi-seed queue impractical on one local GPU.
+- Final release verification needs an independent Nvidia/CUDA run.
+
+Every cloud run must use the same repository command and saved configuration
+as the local run. Checkpoints and aggregate results are copied back before the
+instance and unused storage are deleted.
+
+## Planning estimates
+
+These are budgeting ranges, not claimed benchmark results:
+
+| Work unit | Initial estimate |
+|---|---:|
+| Full embedding-cache build | 5–30 minutes |
+| One frozen-encoder router run | 20–60 minutes |
+| One LoRA experiment | 1–3 GPU-hours |
+| One full 350M fine-tuning experiment | 1–4 GPU-hours |
+| MVP experimentation through Part 10 | 20–50 GPU-hours |
+| Research campaign through Part 15 | 120–300 GPU-hours |
+
+The acceptance benchmark replaces these ranges with measured estimates before
+we commit to the expensive experiment stages.

docs/PART3_CANONICAL_DATA.md

@@ -2,9 +2,10 @@
 
 ## Current status
 
-Part 3 is implemented and validated locally on the
-`codex/leakage-safe-splits` branch. The only remaining project-management gate
-is review and merge.
+Part 3 is complete. Its implementation and real-data validation were merged
+into `main` by
+[PR #7](https://github.com/MaharshPatelX/LFM2.5-ModelRouter/pull/7), with both
+Python 3.11 and 3.12 CI passing.
 
 The builder reads the 41 pinned xRouteBench Parquet files from `data/raw/`,
 normalizes them into six versioned tables, validates the complete result in

docs/PART4_SPLITS.md

@@ -2,10 +2,11 @@
 
 ## Current status
 
-The reusable Part 4 engine is implemented on the
-`codex/leakage-safe-splits` branch. It now consumes the real Part 3 canonical
-tables and generates five leakage-safe xRouteBench manifests, a common
-deduplication manifest and an audit report.
+Part 4 is complete. Its reusable engine and real xRouteBench validation were
+merged into `main` by
+[PR #7](https://github.com/MaharshPatelX/LFM2.5-ModelRouter/pull/7). It consumes
+the Part 3 canonical tables and generates five leakage-safe xRouteBench
+manifests, a common deduplication manifest and an audit report.
 
 The pinned xRouteBench source schema does not contain query timestamps or model
 release dates. The temporal strategy therefore requires trustworthy provenance
@@ -128,6 +129,8 @@ forms 14,688 atomic clusters, including 435 clusters with multiple queries.
 Five real strategies pass their leakage checks. Temporal is the sole
 unsupported strategy because all 15,339 eligible queries lack `observed_at`.
 
-## Remaining completion gate
+## Completion status
 
-- Review and merge the pull request, then update the tracker to Done.
+The reproducibility and leakage gates pass, and the implementation is merged.
+The temporal limitation is a property of xRouteBench's missing source dates,
+not unfinished split-engine work.

docs/PROJECT_BLUEPRINT.md

@@ -4,9 +4,11 @@
 **Project name:** `LFM2.5-ModelRouter` (selected)
 **Primary checkpoint name:** `LFM2.5-ModelRouter-350M`
 **Proposed paper title:** **LFM2.5-ModelRouter: Sparse Cold-Start Routing for Evolving LLM Portfolios**
-**Primary encoder:** `LiquidAI/LFM2.5-Encoder-350M`
+**Primary encoder candidate:** `LiquidAI/LFM2.5-Embedding-350M` (pin the revision in Part 6)
+**Active scale:** 350M only; larger model variants are future work
 **Detailed research reference:** [`RESEARCH_HANDOFF.md`](RESEARCH_HANDOFF.md)
 **Current progress:** [`PROJECT_TRACKER.md`](PROJECT_TRACKER.md)
+**Compute policy:** [`COMPUTE_PLAN.md`](COMPUTE_PLAN.md)
 
 This document is the practical build order for the project. Each numbered part is a real system component or a required research stage. We will follow the parts in order and will not treat a later part as complete until its entry conditions are satisfied.
 

docs/PROJECT_TRACKER.md

@@ -3,8 +3,10 @@
 | Tracker summary | Current value |
 |---|---|
 | Last updated | 2026-08-21 |
-| Current phase | Part 3 implementation is complete locally; review and merge are next |
-| Core progress | 3 of 17 parts complete (Parts 0–16) |
+| Current phase | Part 5 — Baseline Suite is next |
+| Core progress | 5 of 17 parts complete (Parts 0–16) |
+| Active model scope | `LFM2.5-ModelRouter-350M` only |
+| Compute plan | Local RX 7900 XTX 24 GB VRAM with existing 16 GB RAM |
 | Optional work | Part 17 is deferred until the MVP is complete |
 
 This file is the quick status board for the project. The
@@ -29,9 +31,9 @@ part's requirements and completion gate.
 | 0 | Project Contract and Scope | ✅ Done | Scope and research question are documented |
 | 1 | Reproducible Repository Foundation | ✅ Done | [PR #1](https://github.com/MaharshPatelX/LFM2.5-ModelRouter/pull/1) merged; CI passed |
 | 2 | Dataset Source Audit and Ingestion | ✅ Done | [PR #6](https://github.com/MaharshPatelX/LFM2.5-ModelRouter/pull/6) merged; pinned ingestion checks passed |
-| 3 | Canonical Data Layer | 🚧 **In progress** | Six real tables built and validated; review and merge are pending |
-| 4 | Deduplication and Leakage-Safe Splits | 🧩 **Partial** | Five real manifests generated; temporal is explicitly unsupported because source dates do not exist |
-| 5 | Baseline Suite | ⬜ Pending | Complete Part 4 first |
+| 3 | Canonical Data Layer | ✅ Done | [PR #7](https://github.com/MaharshPatelX/LFM2.5-ModelRouter/pull/7) merged; six real tables validated |
+| 4 | Deduplication and Leakage-Safe Splits | ✅ Done | [PR #7](https://github.com/MaharshPatelX/LFM2.5-ModelRouter/pull/7) merged; five real manifests plus an explicit unsupported-temporal artifact |
+| 5 | Baseline Suite | ➡️ **Next** | Implement simple and learned baselines on the Part 4 manifests |
 | 6 | LFM Query Encoder | ⬜ Pending | Complete Part 5 first |
 | 7 | Candidate Model Profile System | ⬜ Pending | Complete Part 6 first |
 | 8 | Multi-Outcome Predictor | ⬜ Pending | Complete Part 7 first |
@@ -45,36 +47,30 @@ part's requirements and completion gate.
 | 16 | Research Release | ⬜ Pending | Complete Part 15 first |
 | 17 | Later Extensions | 💤 Deferred | Consider only after the MVP release |
 
-## Part 3 Local Completion Checklist
+## Merged Data Foundation
 
-- [x] Define the six canonical table schemas.
-- [x] Build the canonical query table from pinned xRouteBench rows.
-- [x] Build the canonical model registry with stable IDs, families, and aliases.
-- [x] Build the outcome table with query/model references and preserved source fields.
-- [x] Build the price-history, probe-profile, and online-route-log schemas.
-- [x] Add understandable validation for required fields, ranges, IDs, and relationships.
-- [x] Add canonical Parquet writers and bounded JSONL debug exports.
-- [x] Add tested price recomputation from token counts and price snapshots.
-- [x] Save generated canonical data beneath ignored `data/processed/` storage.
-- [x] Pass every Part 3 completion gate in the project blueprint locally.
-- [ ] Review, open and merge the pull request before marking Part 3 done.
+- [x] Store real data in portable repository-local ignored directories.
+- [x] Build and validate all six canonical Parquet tables.
+- [x] Preserve stable IDs, raw fields, model aliases and price snapshots.
+- [x] Generate five real leakage-safe split manifests.
+- [x] Record temporal as unsupported instead of inventing source dates.
+- [x] Recreate all 21 generated artifacts byte-for-byte.
+- [x] Merge Parts 3 and 4 in PR #7 with Python 3.11 and 3.12 CI passing.
 
-## Part 4 Prework Already Completed
+## Next Part Checklist — Part 5
 
-- [x] Keep real data in portable, repository-local ignored directories.
-- [x] Support an optional external-storage override without hard-coded paths.
-- [x] Implement exact and normalized prompt hashing.
-- [x] Implement deterministic near-duplicate checks and audit counts.
-- [x] Implement source-ID and synthetic-lineage grouping.
-- [x] Implement model alias and family validation.
-- [x] Isolate complete probe clusters from final test traffic.
-- [x] Implement all six split strategies.
-- [x] Add deterministic manifests and a generated deduplication report.
-- [x] Add unit tests for reproducibility and leakage boundaries.
-- [x] Complete Part 3 canonical Parquet tables and JSONL debug exports locally.
-- [x] Document temporal as unsupported because the pinned source has no dates.
-- [x] Generate and validate real xRouteBench manifests in `data/processed/part4/`.
-- [ ] Merge the Part 4 pull request and mark Part 4 as done here.
+- [ ] Define one shared evaluation contract and candidate pool for every baseline.
+- [ ] Implement random, cheapest and largest/most-expensive policies.
+- [ ] Implement best-single-model and per-domain-best policies using only training data.
+- [ ] Implement best-quality and cheapest-successful oracle upper bounds.
+- [ ] Add logistic-regression, small-MLP and fixed-model-ID learned baselines.
+- [ ] Evaluate every method on the same Part 4 split manifest and price snapshot.
+- [ ] Save a baseline result table, quality-cost plot and oracle-gap report.
+- [ ] Add deterministic tests and a reproducible baseline command.
+
+The [local compute plan](COMPUTE_PLAN.md) fixes the active scope at 350M and
+uses the incoming 24 GB RX 7900 XTX with the existing 16 GB of host RAM. Part 5
+remains CPU-first, so hardware arrival does not block the next implementation.
 
 ## How to Maintain This Tracker
 

docs/REPRODUCIBILITY.md

@@ -7,6 +7,19 @@
 - Heavy ML dependencies will be added only after model and platform compatibility checks.
 - CI runs formatting, linting, static types, tests, and package builds.
 
+## Local compute policy
+
+- The active checkpoint scale is 350M only; larger variants are future work.
+- Part 5 is CPU-first.
+- Parts 6 onward use the local RX 7900 XTX with 24 GB VRAM when acceleration is needed.
+- The existing 16 GB of host RAM is the accepted starting configuration.
+- No memory upgrade is required without measured paging or an out-of-memory failure.
+- Cloud GPUs are a documented fallback, not the default environment.
+
+The bounded smoke test, memory controls and cloud fallback conditions are in
+the [local compute plan](COMPUTE_PLAN.md). The exact ROCm, PyTorch and encoder
+revisions must be pinned only after the Part 6 hardware acceptance test passes.
+
 ## Configuration
 
 Repository defaults live in `configs/base.toml`. Every experiment must save the fully resolved configuration used for the run.