Merge pull request #1 from CSOgroup/feature/complete-selectsim-py

Add feature-complete Python port of SelectSim (v0.3.0)

Author: Arvindiyer

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,40 @@
+---
+name: Bug report
+about: Report a bug or unexpected behaviour in SelectSim
+title: "[BUG] "
+labels: bug
+assignees: ''
+---
+
+## Describe the bug
+
+A clear and concise description of what the bug is.
+
+## Minimal reproducible example
+
+```python
+import selectsim as ss
+# paste the smallest code that reproduces the issue
+```
+
+## Expected behaviour
+
+What you expected to happen.
+
+## Actual behaviour
+
+What actually happened. Include the full error message / traceback or
+unexpected output.
+
+## Environment
+
+- selectsim version: <!-- python -c "import selectsim; print(selectsim.__version__)" -->
+- Python version: <!-- python --version -->
+- Installation method: <!-- pip / uv / editable install from source -->
+- OS: <!-- e.g. macOS 14.5, Ubuntu 22.04 -->
+- Optional extras installed: <!-- e.g. selectsim[fast] (jax), selectsim[storage] (zarr) -->
+
+## Additional context
+
+Any other context that may be relevant (e.g. dataset size, number of cores
+used, `n_permut`, `backend`/`store` options passed to `null_model_parallel`).

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,26 @@
+---
+name: Feature request
+about: Suggest a new feature or improvement for SelectSim
+title: "[FEATURE] "
+labels: enhancement
+assignees: ''
+---
+
+## Is your feature request related to a problem?
+
+A clear and concise description of the problem or limitation you are experiencing.
+
+## Proposed solution
+
+Describe the feature or change you would like to see, including any API
+design ideas if relevant (e.g. new function signature, new parameter on
+`selectX()`).
+
+## Alternatives considered
+
+Any alternative approaches you have considered, and why you prefer the
+proposed solution.
+
+## Additional context
+
+Any other context, references, or examples that support the request.

.github/workflows/docs.yml

@@ -0,0 +1,38 @@
+name: docs
+
+on:
+  push:
+    branches: [main, master]
+  workflow_dispatch:
+
+permissions:
+  contents: write
+
+concurrency:
+  group: docs-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  build-and-deploy:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v3
+        with:
+          python-version: "3.12"
+
+      - name: Install dependencies (including docs extra)
+        run: uv sync --all-extras
+
+      - name: Build Sphinx docs
+        run: uv run sphinx-build -b html docs/source docs/build
+
+      - name: Deploy to GitHub Pages
+        uses: peaceiris/actions-gh-pages@v3
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          publish_dir: docs/build
+          publish_branch: gh-pages

.github/workflows/tests.yml

@@ -0,0 +1,32 @@
+name: tests
+
+on:
+  push:
+    branches: [main, master]
+  pull_request:
+    branches: [main, master]
+  workflow_dispatch:
+
+permissions: read-all
+
+jobs:
+  pytest:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.10", "3.11", "3.12"]
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v3
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install dependencies
+        run: uv sync --all-extras
+
+      - name: Run tests
+        run: uv run pytest tests/ -q

.gitignore

@@ -11,6 +11,10 @@
 .idea/*
 .python-version
 .vscode/*
+.venv/
 
 # Sphinx documentation
 docs/build/
+docs/jupyter_execute/
+docs/.jupyter_cache/
+docs/source/generated/

CITATION.cff

@@ -0,0 +1,51 @@
+cff-version: 1.2.0
+message: "If you use this software, please cite it using the metadata below, and cite the accompanying article as the primary reference (see 'preferred-citation')."
+type: software
+title: "SelectSim (Python)"
+abstract: >-
+  A Python implementation of the SelectSim methodology for inferring
+  evolutionary dependencies -- co-mutations and mutual exclusivities --
+  between functional alterations across cancer genomes. Estimates expected
+  co-mutation frequencies from individual gene mutation rates and per-sample
+  tumor mutation burden (TMB), then evaluates significance against a
+  permutation-based null model.
+version: 0.2.0
+date-released: "2026-07-12"
+license: MIT
+repository-code: "https://github.com/CSOgroup/SelectSim_py"
+url: "https://csogroup.github.io/SelectSim/"
+authors:
+  - given-names: Arvind
+    family-names: Iyer
+    email: ayalurarvind@gmail.com
+    orcid: "https://orcid.org/0000-0002-8247-700X"
+  - given-names: Marco
+    family-names: Mina
+    email: marco.mina.85@gmail.com
+  - given-names: Miljan
+    family-names: Petrovic
+    email: miljanpet93@gmail.com
+  - given-names: Giovanni
+    family-names: Ciriello
+    email: giovanni.ciriello@unil.ch
+    orcid: "https://orcid.org/0000-0003-2021-8683"
+preferred-citation:
+  type: article
+  title: "Evolving patterns of co-mutations from tumor initiation to metastatic progression"
+  journal: "Nature Genetics"
+  year: 2026
+  doi: "10.1038/s41588-026-02661-4"
+  url: "https://doi.org/10.1038/s41588-026-02661-4"
+  authors:
+    - given-names: Arvind
+      family-names: Iyer
+    - given-names: Miljan
+      family-names: Petrovic
+    - given-names: Debora
+      family-names: Sesia
+    - given-names: Luca
+      family-names: Nanni
+    - given-names: Marco
+      family-names: Mina
+    - given-names: Giovanni
+      family-names: Ciriello

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 SelectSim authors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

NEWS.md

@@ -0,0 +1,104 @@
+# SelectSim (Python) 0.1.0
+
+Initial feature-complete Python port of the [SelectSim R package](https://github.com/CSOgroup/SelectSim).
+
+* Ported the core algorithm: `selectX()` end-to-end pipeline, `AlterationLandscape`,
+  template-matrix generation (`template.py`), TMB-based weighting (`weights.py`),
+  the permutation null model (`null_model.py`), and all overlap / effect-size /
+  FDR statistics (`stats.py`).
+* Added an optional accelerated backend for `null_model_parallel`
+  (`backend="jax"`, via the `selectsim[fast]` extra) alongside the default
+  NumPy backend, and an optional on-disk Zarr store for large null-model runs
+  (`store="zarr"`, via the `selectsim[storage]` extra) so simulations no
+  longer need to be held entirely in memory.
+* Added `selectsim.gam`: a full port of `R/gam_utils.r`'s MAF-ingestion
+  pipeline (`filter_maf_*`, `stat_maf_*`, `maf_to_gam`, `TCGA_maf_schema`,
+  `GENIE_maf_schema`, `mutation_type`) for building Gene Alteration Matrices
+  (GAMs) directly from raw MAF files.
+* Added `selectsim.plotting`: matplotlib/seaborn ports of the R package's
+  visualization helpers -- `theme_publication`, `obs_exp_scatter`,
+  `overlap_pair_extract`, `ridge_plot_ed`, `ridge_plot_ed_compare`.
+* Added `selectsim.io`: Parquet read/write helpers and Zarr null-model store
+  utilities.
+* Bundled real TCGA LUAD example data (`tests/data/*.parquet`, exported from
+  the R package's `luad_run_data`/`luad_result`/`luad_maf` datasets) and
+  added an R-parity test suite (`TestSelectXWithRData` in
+  `tests/test_selectsim.py`) validating the Python port's output against a
+  real `SelectSim::selectX()` R reference run.
+* 126 tests passing across algorithm, statistics, GAM ingestion, plotting,
+  and I/O modules.
+* Added Sphinx documentation (`docs/`) with a full API reference, an
+  "Overview" quick-start, and two tutorial notebooks (`introduction`,
+  `data_processing`) mirroring the R package's vignettes, plus standard
+  open-source project scaffolding (`LICENSE`, `CITATION.cff`, CI workflows,
+  issue templates).
+* Supports Python 3.10-3.12 (`requires-python = ">=3.10,<3.13"`). The upper
+  bound is intentional, not a typo: numpy/scipy/jax/zarr don't all publish
+  prebuilt wheels for 3.13+ yet, and installing on an unsupported interpreter
+  can silently trigger a slow (or failing) from-source build of numpy.
+  3.10/3.11/3.12 were each verified to resolve prebuilt wheels and pass the
+  full test suite.
+
+# SelectSim (Python) 0.2.0
+
+* Fixed several correctness bugs found in code review: `filter_maf_column`
+  computed a union instead of an intersection when excluding rows matching
+  several substring values (`inclusive=False, fixed=False`); `store="zarr"`
+  fully materialized all permutations in memory before writing them to disk,
+  defeating its own purpose (now streams in bounded batches, ~5x lower peak
+  memory); the jax backend recompiled from scratch on every call instead of
+  reusing JAX's compilation cache; `n_cores` was silently ignored with
+  `backend="jax"` (now warns); `ridge_plot_ed` crashed on an empty null
+  model; and `retrieve_outliers`'s `n_sim` parameter was dead code and its
+  90th-percentile threshold was off-by-one relative to the R original.
+* This round of work (the fixes above, plus the earlier module ports,
+  documentation, and uv migration) was developed with the assistance of
+  Claude Code (Anthropic), directed and verified by Arvind Iyer throughout;
+  see `git log` for detailed commit history.
+
+# SelectSim (Python) 0.3.0
+
+* **Performance**: the default numpy null-model backend
+  (`null_model_parallel`, `store="memory"` or `"zarr"`) no longer loops
+  over genes in Python to pick each row's top-k residual columns;
+  it now uses the same vectorized double-argsort rank trick the (now
+  removed) jax backend used, with no per-gene Python loop. Benchmarked
+  as bit-identical to the old implementation (full R-parity test suite
+  still passes) and consistently faster, especially at higher `n_cores`.
+* **Performance**: `store="zarr"` null-model runs now chunk the on-disk
+  Zarr array to match the internal write-batch size instead of
+  defaulting to one chunk per permutation; writing in batches of 200
+  (the default) previously forced 200 separate compress/write calls per
+  batch. Benchmarked at ~2.5x faster zarr writes in isolation.
+* **Performance**: `estimate_p_val`'s `gene_names: List[str]` parameter
+  is now `gene_index: Dict[str, int]` -- callers looking up many pairs
+  (as `estimate_pairwise_p` does, used when `estimate_pairwise=True`)
+  now build the name-to-index mapping once instead of doing an O(n_genes)
+  list scan per gene per pair.
+* **Removed**: the jax null-model backend (`backend="jax"` on
+  `null_model_parallel`, the `selectsim[fast]` extra). Benchmarked
+  against the numpy backend on real LUAD-sized data on CPU (no GPU):
+  jax was consistently 5-10x *slower*, dominated by per-call JIT
+  dispatch overhead that this workload's array sizes never amortize.
+  If GPU-backed acceleration is needed in the future, it should be
+  re-evaluated against the current (vectorized) numpy backend on
+  representative GPU hardware, not reintroduced by default.
+* **Fixed**: `significance_heatmap` was implemented and tested but never
+  exported from the top-level `selectsim` package (only reachable via
+  `selectsim.plotting.significance_heatmap`); it's now `ss.significance_heatmap`.
+* Added `null_model_parallel`/`selectX` `store="auto"`: estimates the
+  null model's memory footprint from cohort size and `n_permut` and
+  picks `"memory"` or `"zarr"` automatically against a detected (or
+  configured, `memory_budget_gb`) RAM budget, so one call scales safely
+  across machines without the caller needing to know cohort dimensions
+  up front.
+* Added `oncoprint` and `oncoprint_pair`: new (not present in the R
+  package) multi-gene and two-gene co-mutation visualizations comparing
+  the observed alteration pattern to a retained null-model simulation,
+  with a per-sample TMB track.
+* Extended the `introduction` tutorial notebook with `significance_heatmap`,
+  `oncoprint`, `oncoprint_pair`, and `ridge_plot_ed_compare` (previously
+  only `obs_exp_scatter` and `ridge_plot_ed` were demonstrated).
+* This round of work was developed with the assistance of Claude Code
+  (Anthropic), directed and verified by Arvind Iyer throughout; see
+  `git log` for detailed commit history.

README.md

@@ -1,10 +1,60 @@
 # SelectSim Python
 
-The goal of `selectsim` package is to implement the methodology to infer
-inter-dependencies between functional alterations in cancer. SelectSim
-estimates the expected number of mutations in a given gene and a given
-sample from the mutation frequency of the gene, f(g), and the tumor
-mutation burden (TMB) of the sample, $\mu$(t). These values can be
-estimated within specific mutation and tumor subsets, to account for
-heterogeneous tumor types, tissue specificities, and distinct mutational
-processes.
+SelectSim infers evolutionary dependencies — co-mutations and mutual
+exclusivities — between functional alterations across cancer genomes. It
+estimates the expected co-mutation frequency for each gene pair from
+individual mutation frequencies and per-sample tumor mutation burden (TMB),
+then evaluates significance against a simulation-based null model.
+
+![SelectSim Method](docs/source/_static/SelectSim_method.png)
+
+This package accompanies the manuscript:
+
+> Iyer A, Petrovic M, Sesia D, Nanni L, Mina M, Ciriello G (2026). Evolving
+> patterns of co-mutations from tumor initiation to metastatic progression.
+> *Nature Genetics*. DOI: [10.1038/s41588-026-02661-4](https://doi.org/10.1038/s41588-026-02661-4)
+
+## Installation
+
+```bash
+cd SelectSim_py
+uv sync                 # recommended; or: pip install -e .
+```
+
+Optional extras: `uv sync --extra storage` (Zarr on-disk storage), `--extra docs` (build docs locally), `--all-extras`. Supports Python 3.10-3.12.
+
+## Quick start
+
+```python
+import selectsim as ss
+
+# M, sample_class, alteration_class: see the Introduction tutorial for how
+# to build these from a MAF, or load the bundled TCGA LUAD example data.
+result = ss.selectX(
+    M, sample_class, alteration_class,
+    n_cores=1, min_freq=10, n_permut=1000,
+)
+
+# Significant evolutionary dependencies
+significant = result['result'][result['result']['FDR']]
+```
+
+See the [Introduction tutorial](docs/source/tutorials/introduction.ipynb) for a full walkthrough on real TCGA LUAD data, and [`docs/`](docs/) for the full API reference (`selectsim.gam` for MAF ingestion, `selectsim.plotting` for visualization, `selectsim.io` for Parquet/Zarr storage).
+
+## Citation
+
+If you use SelectSim in your research, please cite:
+
+> Iyer A, Petrovic M, Sesia D, Nanni L, Mina M, Ciriello G (2026). Evolving
+> patterns of co-mutations from tumor initiation to metastatic progression.
+> *Nature Genetics*. DOI: [10.1038/s41588-026-02661-4](https://doi.org/10.1038/s41588-026-02661-4)
+
+Citation metadata is also available in [`CITATION.cff`](CITATION.cff).
+
+## License
+
+MIT License. See [LICENSE](LICENSE).
+
+## Contact
+
+For bugs or feature requests, use the [issue tracker](https://github.com/CSOgroup/SelectSim_py/issues). This is the Python implementation of the [SelectSim R package](https://github.com/CSOgroup/SelectSim).