You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Updates the README to surface the apples-to-apples comparison from commit
ec16e0d and bumps the test badge to 57/57 (was 51/51; +6 new tests for
the real-scores run, all green).
- Badge line 6: 51/51 -> 57/57 (matches pytest output)
- New subsection 'REAL AlphaMissense + CADD Top-K=200 (apples-to-apples,
commit ec16e0d)' after the existing CADD Top-K section, with:
* Headline table (0.1% ctDNA, 5 seeds x 20 patients, 5,738-mut panel)
from results/cadd_vs_alphamissense_topk_20patient.json:
Uniform 0.9210 / CADD K=20 0.9345 / CADD K=200 0.9785
AM K=20 0.9135 / AM K=200 0.9775 / AM K=500 0.9260
* 7th Mermaid diagram (3 boxes + base + annotation)
* Bug-fix note: AM match rate 0% -> 97.23% after the ref_aa/alt_aa
key-construction fix (alphamissense_weights.py + alphamissense_panel_run.py)
* Honest framing of the two cohorts: 150-patient GDC bulk-WXS
(CADD K=20 wins, +35pp Sens@99%) vs 20-patient curated
(CADD K=200 wins on AUC, AM K=200 within 1 std) - both positive
- 'Honest negative results preserved' bullets extended with the AM bug
(proxy hid the lift; real scores reveal +0.057) and the AM=CADD
at-K=200 finding.
- Documentation section: pointer to docs/ALPHAMISSENSE_REAL_LLR.md and
refreshes roadmap bullet from '7 of 11' to '8 of 12 (2026-09-17 22:30
batch)' to match the 093341d roadmap update.
All cited numbers verified against:
- results/cadd_vs_alphamissense_topk_20patient.json (headline_at_0_1pct)
- results/alphamissense_real_index_full.json (match-rate metadata)
- pytest collection: 57 tests, all passing
- git log: ec16e0d, 093341d, 10c6425 cited correctly
-**Proxy (constant 0.55) AlphaMissense weighting** hid the real AM signal — the previous run reported every AM-weighted method at AUC ≈ 0.917 (proxy indistinguishable from uniform 0.921). With **REAL** AlphaMissense scores (key-construction bug fixed in commit `ec16e0d`), AM Top-K=200 lifts AUC from 0.9210 → **0.9775** at 0.1% ctDNA (+0.057, not the proxy's 0)
753
+
-**AM ≈ CADD at K=200** (0.9775 vs 0.9785 on the same 20-patient cohort, within 1 std) — once K is large enough, the scoring function barely matters; panel size dominates. See `docs/ALPHAMISSENSE_REAL_LLR.md` for the bug-fix details and per-patient coverage.
### REAL AlphaMissense + CADD Top-K=200 (apples-to-apples, commit `ec16e0d`)
770
+
771
+
The CADD Top-K=20 finding above is the winner on the **150-patient GDC bulk-WXS cohort**. On the **smaller 20-patient TCGA-LUAD curated driver set** (the original CADD validation cohort), an apples-to-apples comparison with the SAME 5,738-mutation panel and SAME 5 seeds shows a slightly different ranking: **CADD Top-K=200 wins narrowly over AM Top-K=200 at 0.1% ctDNA**, with both dramatically beating the uniform baseline. Both findings are positive — the K that maximises AUC is just cohort-dependent (more matches per patient on the curated set → K=200 is fine; fewer matches on bulk-WXS → K=20 is the binding choice).
772
+
773
+
**Apples-to-apples headline (0.1% ctDNA, 5 seeds × 20 patients, panel of 5,738 mutations, full CADD + AM scoring pipelines re-run on identical cohort, results from `results/cadd_vs_alphamissense_topk_20patient.json`):**
| AM Top-K=500 (REAL scores) | 0.9260 | +0.005 | regresses past K=200 — too many loci dilute the signal |
783
+
784
+
**Bug fix that unlocked the AM result** (commit `ec16e0d`, full writeup in `docs/ALPHAMISSENSE_REAL_LLR.md`): the previous AlphaMissense-weighted LLR run (commit `5cd6c3e`) was building AM lookup keys from the *nucleotide*`ref/alt` columns of the MAFs, but AlphaMissense is keyed by *amino-acid*`ref_aa/alt_aa` at protein coordinates (`P01116:12:G:D` = KRAS G12D). The bug silently dropped the per-mutation match rate to ≈ 0% and substituted the per-variant-class proxy (0.55). With the fix, **AM match rate on the 20-patient missense cohort jumped from ≈ 0% → 97.23%** (15,462 / 15,903 missense SNVs hit the 71.7M-row AM TSV via a 69,575-key pickle built by streaming the full TSV once in 47 s). The proxy was indistinguishable from uniform at AUC ≈ 0.917; the real scores reveal the +0.057 lift at K=200.
785
+
786
+
**Honest framing of the two cohorts:** the 150-patient GDC bulk-WXS result (CADD Top-K=20 wins, +35pp whole-cohort Sens@99%) and the 20-patient curated-driver result (CADD Top-K=200 wins on AUC, AM Top-K=200 statistically tied) are *both real and both positive* — they just optimise different operating points. The right K is whichever is ≤ the per-patient median CADD match count; the right scoring function (CADD vs AM) barely matters once K is large enough (the K=200 numbers are within 0.001 of each other).
**Ultra-early assay sweep** (0.1% ctDNA; `--skip-sweep` to disable) — panel detection vs background error rate × depth. This is the assay-design guidance: duplex-UMI consensus (~1e-4) or ~50k× depth each bring sens@95% to 1.000 at 0.1% ctDNA:
768
822
@@ -1006,7 +1060,8 @@ graph LR
1006
1060
-[docs/CADD_PER_SUBGROUP_LLR.md](docs/CADD_PER_SUBGROUP_LLR.md) — per-subgroup CADD Top-K lift (+24 to +50pp)
0 commit comments