Skip to content

Commit 36d75ce

Browse files
mikemckamcmeroCopilot
authored
Add cellsam segmentation, kronos embeddings, mask smoothing and python cellmeasurement rewrite (#17)
* Add CellSAM segmentation support * Add KRONOS embedding generation with optional merged GeoJSON output * Add optional mask smoothing for complex geometries, with morphological or shapely-based simplification * Replace Groovy cellmeasurement with Python-only bin/cellmeasurement.py * Add structure to support running pytest with CI * Update erosion measurement output to use 5 equal-area bins with new ErosionBin_* column names * Change --erosion-steps to a boolean flag; numeric pixel-depth values are no longer supported * Add dist_threshold parameter for nucleus-to-cell ROI matching * Add downsample_factor parameter to speed up cellmeasurement on large images --------- Co-authored-by: Marek Cmero <7799828+mcmero@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 2a3a214 commit 36d75ce

100 files changed

Lines changed: 9066 additions & 539 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/CONTRIBUTING.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,13 @@ nf-test test --profile debug,test_mesmer,docker --verbose
3030
nf-test test --profile debug,test_cellpose,docker --verbose
3131
```
3232

33+
KRONOS module tests run in stub mode by default to avoid downloading large model checkpoints. To run the KRONOS integration module test, set a pre-downloaded model directory (must contain a `.pt` checkpoint) and run:
34+
35+
```bash
36+
export KRONOS_MODEL_DIR=/path/to/kronos_model_dir
37+
nf-test test modules/local/kronosembeddings/tests/main.nf.test --profile debug,test,apptainer --verbose
38+
```
39+
3340
When you create a pull request with changes, [GitHub Actions](https://github.com/features/actions) will run automatic tests.
3441
Typically, pull-requests are only fully reviewed when these tests are passing, though of course we can help out before then.
3542

‎.github/workflows/ci.yml‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,6 @@ jobs:
9292
- name: "Run pipeline with test data ${{ matrix.NXF_VER }} | ${{ matrix.test_name }} | ${{ matrix.profile }}"
9393
run: |
9494
mkdir -p work && chmod 777 work
95-
mkdir -p $HOME/.gradle && chmod 777 $HOME/.gradle
9695
if [ "${{ matrix.profile }}" == "singularity" ]; then
9796
export NXF_SINGULARITY_HOME_MOUNT=true
9897
fi

‎.github/workflows/python-tests.yml‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
name: Python unit tests
2+
3+
on:
4+
push:
5+
branches:
6+
- dev
7+
- main
8+
paths:
9+
- "bin/**.py"
10+
- "tests/python/**.py"
11+
- "pyproject.toml"
12+
- "uv.lock"
13+
pull_request:
14+
paths:
15+
- "bin/**.py"
16+
- "tests/python/**.py"
17+
- "pyproject.toml"
18+
- "uv.lock"
19+
20+
jobs:
21+
pytest:
22+
name: pytest
23+
runs-on: ubuntu-latest
24+
25+
steps:
26+
- name: Check out pipeline code
27+
uses: actions/checkout@v4
28+
29+
- name: Install uv
30+
uses: astral-sh/setup-uv@v5
31+
32+
- name: Set up Python
33+
run: uv python install 3.10
34+
35+
- name: Install dependencies
36+
run: uv sync --dev
37+
38+
- name: Run Python unit tests
39+
run: uv run pytest tests/python/ -v

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,3 +8,4 @@ testing*
88
*.pyc
99
null/
1010
.nf-test*
11+
*.egg-info

‎.nf-core.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ org_path: null
4646
repository_type: pipeline
4747
template:
4848
author: WEHI SODA Hub
49-
description: Cell segmentation and downstream analysis pipeline for the COMET and MIBI spatial platforms
49+
description: Cell segmentation and downstream analysis pipeline for the COMET, MIBI, and OPAL spatial platforms
5050
force: true
5151
is_nfcore: true
5252
name: spatialproteomics

‎CHANGELOG.md‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,53 @@
33
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
44
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
55

6+
## v0.4.0 - 2026-04-20
7+
8+
### Changed
9+
10+
- **cellmeasurement: Python replaces Groovy as the only implementation.** The original Groovy app
11+
(`https://github.com/WEHI-SODA-Hub/cellmeasurement`) is no longer used. All measurement logic
12+
is now in `bin/cellmeasurement.py`.
13+
14+
- **cellmeasurement: erosion measurement column names have changed.** The Groovy app produced
15+
columns named `{Channel}: {Compartment}: Eroded_{N}px: Mean/Median` where N was a fixed pixel
16+
depth. The Python implementation uses 5 _equal-area_ bins and names them
17+
`{Compartment}: ErosionBin_{N}: Mean/Median`. If you have existing Groovy-generated GeoJSON
18+
that you compare with new Python output, expect different column names.
19+
20+
- **cellmeasurement: `--erosion-steps` no longer accepts pixel-depth values.** Passing numeric
21+
values such as `--erosion-steps=4,7,11,14,18` (Groovy API) is not supported. Use the boolean
22+
flag `--erosion-steps` to enable the 5 equal-area bins.
23+
24+
### Added
25+
26+
- **KRONOS embedding output**: new `kronosembeddings` module extracts per-cell embeddings using
27+
the KRONOS foundation model. Produces `*_kronos_embeddings.csv` (cell IDs + 384 embedding
28+
dimensions), `*_marker_report.txt` (channel-to-marker match summary), and optionally
29+
`*_kronos_merged.geojson` (embeddings merged into the cellmeasurement GeoJSON).
30+
Controlled by `skip_kronos`, `kronos_model_path`, `kronos_marker_metadata`, and related
31+
`kronos_*` parameters. Disabled by default (`skip_kronos = true`).
32+
33+
- **CellSAM segmentation**: new `cellsam_segment` module adds support for the CellSAM foundation
34+
model as a third segmentation option alongside Mesmer and Cellpose. Supports tiled (WSI) and
35+
non-tiled inference via `cellsam_use_wsi`. Controlled by `cellsam_bbox_threshold`,
36+
`cellsam_block_size`, `cellsam_overlap`, `cellsam_iou_threshold`, and `cellsam_use_wsi`
37+
parameters.
38+
39+
- **Mask smoothing (`smooth_masks`)**: new optional `smoothmasks` module reduces polygon
40+
complexity. Two methods: `morphological` (disk close+open, default) and
41+
`shapely` (Douglas-Peucker polygon simplification). Enabled via `smooth_masks
42+
= true`; tunable with `smooth_method` and `smooth_kernel_size`.
43+
44+
- `dist_threshold` pipeline parameter: controls the maximum centroid distance (pixels) for
45+
matching a nucleus to a whole-cell ROI in cellmeasurement (default: 10.0).
46+
- `downsample_factor` pipeline parameter: integer downsample factor applied to image and masks
47+
before cellmeasurement to speed up processing on very large images (default: 1.0 = disabled).
48+
49+
### Removed
50+
51+
- `gradle_cache_dir` parameter and all references removed (leftover from the Groovy era).
52+
653
## v0.3.0 - 2025-11-13
754

855
- Rename to sp_segment

‎README.md‎

Lines changed: 125 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -19,12 +19,12 @@
1919
## Introduction
2020

2121
**WEHI-SODA-Hub/sp_segment** is a pipeline for running cell segmentation
22-
on COMET and MIBI data. For COMET, background subtraction can be performed
23-
followed by patched cellpose segmentation, or non-patched mesmer segmentation.
24-
For MIBI, mesmer segmentation can be run. Whole-cell and nuclear segmentations
25-
are run separately, and then consolidated into whole cells with nuclei with full
26-
shape and intensity measurements per compartment. The output GeoJSON files can
27-
be viewed in QuPath.
22+
on COMET, MIBI, and OPAL data. For COMET, background subtraction can be performed
23+
followed by patched cellpose segmentation, non-patched mesmer segmentation, or
24+
CellSAM foundation model segmentation. For MIBI, mesmer or CellSAM segmentation
25+
can be run. Whole-cell and nuclear segmentations are run separately, and then
26+
consolidated into whole cells with nuclei with full shape and intensity
27+
measurements per compartment. The output GeoJSON files can be viewed in QuPath.
2828

2929
<details>
3030
<summary>Click to view Mermaid diagram</summary>
@@ -69,9 +69,13 @@ The pipeline uses the following tools:
6969
-- background subtraction tool for COMET.
7070
- [MesmerSegmentation](https://github.com/WEHI-SODA-Hub/mesmersegmentation) -- a
7171
CLI for running Mesmer segmentation of MIBI and OME-XML TIFFs.
72+
- [CellSAM](https://github.com/vanvalenlab/cellSAM) -- a foundation model for
73+
cell segmentation across diverse imaging modalities.
7274
- [cellmeasurement](https://github.com/WEHI-SODA-Hub/cellmeasurement) -- a
73-
Groovy app that matches whole-cell segmentations with nuclei, and uses the
74-
QuPath API to calculate compartment measurements and intensities.
75+
Python app that matches whole-cell segmentations with nuclei and calculates
76+
compartment measurements and intensities.
77+
- [KRONOS](https://github.com/mahmoodlab/KRONOS) -- a foundation model for
78+
multiplex spatial proteomics that extracts rich embeddings for each cell.
7579
- [sopa](https://github.com/gustaveroussy/sopa) -- we use the sopa CLI tool to
7680
patchify images and perform cellpose segmentation.
7781
- [spatialVis](https://github.com/WEHI-SODA-Hub/spatialVis) -- R package for spatial
@@ -86,9 +90,6 @@ Please see the [docs for more detailed information on pipeline usage and output]
8690
8791
If you are running this pipeline from WEHI, it has been set up to run on [Seqera Platform](https://seqera.services.biocommons.org.au/).
8892

89-
> [!NOTE]
90-
> If you don't have a .gradle directory in your home, make sure you create it with `mkdir $HOME/.gradle` before runnning the pipeline. You con't need to do this if you are running via WEHI's Seqera Platform mentioned above.
91-
9293
Usage will depend on your desired steps. See [usage docs](docs/usage.md) for more detailed information.
9394

9495
### Background subtraction
@@ -101,9 +102,9 @@ Prepare a sample sheet as follows:
101102
`samplesheet.csv`:
102103

103104
```csv
104-
sample,run_backsub,run_mesmer,run_cellpose,tiff
105-
sample1,true,true,false,/path/to/sample1.tiff
106-
sample2,true,false,true,/path/to/sample2.tiff
105+
sample,run_backsub,run_mesmer,run_cellpose,run_cellsam,tiff
106+
sample1,true,true,false,false,/path/to/sample1.tiff
107+
sample2,true,false,false,true,/path/to/sample2.tiff
107108
```
108109

109110
You may also prefer to use YAML for your samplesheet, either is supported:
@@ -115,19 +116,20 @@ You may also prefer to use YAML for your samplesheet, either is supported:
115116
run_backsub: true
116117
run_mesmer: true
117118
run_cellpose: false
119+
run_cellsam: false
118120
tiff: /path/to/sample1.tiff
119121
- sample: sample2
120122
run_backsub: true
121123
run_mesmer: true
122124
run_cellpose: false
125+
run_cellsam: false
123126
tiff: /path/to/sample2.tiff
124127
```
125128
126129
> [!WARNING]
127-
> Please ensure that your image name and all directories in your file path do not contain spaces.
130+
> Please ensure that your image name and all directories in your path do not contain spaces.
128131
129-
If you don't specify any segmentation algorithm to run (mesmer or cellpose), the
130-
pipeline will run a background subtraction only.
132+
If you don't specify any segmentation algorithm to run (mesmer, cellpose, or cellsam), the pipeline will run a background subtraction only.
131133
132134
Now, you can run the pipeline using:
133135
@@ -167,10 +169,8 @@ You can also set the segmentation parameters for mesmer either via CLI
167169
(e.g., `--combine_method prod` or in a config file pass to the workflow
168170
via `-c`. See [usage](docs/usage.md) for a full list.
169171

170-
> [!WARNING]
171-
> You cannot run both Mesmer and Cellpose segmentation on the same sample (with
172-
> the same name). If you want to run both on a sample, put it on a different
173-
> line and give it a different sample name.
172+
> [!NOTE]
173+
> You cannot run multiple segmentation methods (Mesmer, Cellpose, or CellSAM) on the same sample (with the same name). If you want to run multiple methods on a sample, put it on a different line and give it a different sample name.
174174
175175
### Cellpose segmentation
176176

@@ -195,6 +195,110 @@ parameters.
195195
If you want to skip measurements (this may take some time for large images), you
196196
can use set the parameter `skip_measurements` to `true`.
197197

198+
### KRONOS embeddings
199+
200+
KRONOS is a foundation model for multiplex spatial proteomics that extracts rich embeddings for each cell. These embeddings capture cellular phenotype and microenvironment context, enabling downstream analysis like clustering, classification, and spatial analysis.
201+
202+
To enable KRONOS embeddings:
203+
204+
```bash
205+
nextflow run main.nf \
206+
--input samplesheet.csv \
207+
--skip_kronos false \
208+
--kronos_model_path /path/to/kronos_model \
209+
--kronos_marker_metadata /path/to/marker_metadata.csv \
210+
--kronos_merge_geojson true \
211+
...
212+
```
213+
214+
#### KRONOS parameters
215+
216+
- `--skip_kronos` (default: true): Set to `false` to enable KRONOS embedding extraction
217+
- `--kronos_model_path` (required): Path to the KRONOS model checkpoint (.pt file)
218+
- `--kronos_config_path` (optional): Path to KRONOS `config.json` (auto-detected from model directory if not set)
219+
- `--kronos_marker_metadata` (required): Path to marker metadata CSV file mapping marker IDs to names
220+
- `--kronos_merge_geojson` (default: false): Merge embeddings into the cellmeasurement GeoJSON output
221+
- `--kronos_patch_size` (default: 64): Patch size for cell-centered crops
222+
- `--kronos_batch_size` (default: 32): Batch size for model inference
223+
- `--kronos_num_workers` (default: 4): Number of DataLoader workers for parallel data loading
224+
- `--kronos_max_value` (default: 65535): Maximum intensity value for normalization
225+
- `--kronos_marker_mapping` (optional): JSON string mapping image marker names to KRONOS marker names
226+
- `--kronos_distance_threshold` (default: 5.0): Maximum centroid distance (pixels) used for GeoJSON merge fallback matching
227+
228+
#### Embeddings for filtered data with KRONOS
229+
230+
When `--kronos_merge_geojson` is enabled, the pipeline automatically creates a new segmentation mask directly from the GeoJSON polygons.
231+
232+
#### Output files
233+
234+
KRONOS produces the following outputs:
235+
236+
- `*_kronos_embeddings.csv`: CSV file with cell IDs, centroids, and 384 embedding dimensions
237+
- `*_marker_report.txt`: Report showing which image channels were matched to KRONOS markers
238+
- `*_kronos_merged.geojson` (if `--kronos_merge_geojson=true`): GeoJSON file with embeddings added as cell properties
239+
240+
The merged GeoJSON file contains all original cell measurements plus additional features (`kronos_emb_0` through `kronos_emb_#`), enabling integrated analysis of morphology, intensity, and KRONOS embeddings.
241+
242+
#### Marker matching
243+
244+
KRONOS expects specific marker names based on its training data. The pipeline automatically performs case-insensitive matching between your image channel names and the KRONOS marker metadata. For markers that don't auto-match, use `--kronos_marker_mapping`:
245+
246+
```bash
247+
--kronos_marker_mapping '{"CD3e": "CD3E", "PanCK": "PANCK"}'
248+
```
249+
250+
For COMET data with fluorophore suffixes in channel names, you can map them like this:
251+
252+
```bash
253+
--kronos_marker_mapping '{"DAPI": "DAPI", "FOXP3_T - TRITC": "FOXP3", "CD3_T - Cy5": "CD3"}'
254+
```
255+
256+
For more information about KRONOS, see the [KRONOS GitHub repository](https://github.com/mahmoodlab/KRONOS).
257+
258+
### CellSAM segmentation
259+
260+
CellSAM is a foundation model for cell segmentation that works across different
261+
imaging modalities. To use CellSAM as your segmentation algorithm, specify a
262+
config file like so:
263+
264+
```csv
265+
sample,run_backsub,run_cellsam,tiff,nuclear_channel,membrane_channels
266+
sample1,true,true,/path/to/sample1.tiff,DAPI,CD45:CD8
267+
sample2,false,true,/path/to/sample2.tiff,DAPI,CD45
268+
```
269+
270+
Nuclear channels only support one entry; membrane channels may have multiple
271+
values separated by `:` characters. If your channels have spaces in them, make
272+
sure that you surround your channel name with quotes.
273+
274+
CellSAM uses a tiling approach for large images and supports the following
275+
parameters:
276+
277+
- `--cellsam_bbox_threshold` (default: 0.4): Confidence threshold for cell detection
278+
- `--cellsam_block_size` (default: 600): Size of tiles for processing
279+
- `--cellsam_overlap` (default: 56): Tile overlap for merging
280+
- `--cellsam_iou_threshold` (default: 0.5): IOU threshold for label merging
281+
- `--cellsam_use_wsi` (default: true): Enable tiling for large images
282+
283+
#### Model weights
284+
285+
CellSAM can automatically download the latest model weights (v1.2) from
286+
[users.deepcell.org](https://users.deepcell.org). To use the latest weights:
287+
288+
1. Create an account at [users.deepcell.org](https://users.deepcell.org)
289+
2. Generate your access token
290+
3. Set it as a Nextflow secret:
291+
```bash
292+
nextflow secrets set DEEPCELL_ACCESS_TOKEN $YOUR_TOKEN
293+
```
294+
295+
If the token is not set, CellSAM will use the default bundled model weights.
296+
297+
> [!NOTE]
298+
> You cannot run both Mesmer/Cellpose and CellSAM segmentation on the same sample
299+
> (with the same name). If you want to run multiple methods on a sample, put it
300+
> on a different line and give it a different sample name.
301+
198302
## Dealing with large images
199303

200304
You can run the pipeline with different profiles for different size images:

‎assets/samplesheet.csv‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
sample,run_backsub,run_cellpose,run_mesmer,tiff,nuclear_channel,membrane_channels
2-
SAMPLE,boolean,boolean,boolean,/path/to/image.tif,nuclear_channel,membrane_channel1:membrane_channel2
1+
sample,run_backsub,run_cellpose,run_mesmer,run_cellsam,tiff,nuclear_channel,membrane_channels
2+
SAMPLE,boolean,boolean,boolean,boolean,/path/to/image.tif,nuclear_channel,membrane_channel1:membrane_channel2

‎assets/schema_input.json‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,11 @@
2828
"default": false,
2929
"errorMessage": "Run cellpose must be a boolean"
3030
},
31+
"run_cellsam": {
32+
"type": "boolean",
33+
"default": false,
34+
"errorMessage": "Run cellsam must be a boolean"
35+
},
3136
"tiff": {
3237
"type": "string",
3338
"format": "file-path",

0 commit comments

Comments
 (0)