Repository navigation
feat: add kronos_exclude_markers, and fix split-series channel reads - #42
Merged
Merged
Conversation
Adds a way to withhold channels that are not markers -- autofluorescence, blanks, empty cycles -- from KRONOS2, so they contribute nothing to the embedding. Until now the only escape for an out-of-vocabulary channel was --allow-novel-defaults, which still *feeds* it to the model normalised with default statistics: the opposite of what is wanted for a channel that carries no biology. This is scoped to the embedding step and deletes nothing. A withheld channel is simply never read, so it stays in the image, remains available to segmentation and every other process, and keeps whatever intensity measurements CELLMEASUREMENT wrote for it in the GeoJSON. Contrast remove_markers, which does strip channels out of the background-subtracted image. The log line, the marker report and the docs all say "not shown to KRONOS2" rather than "removed", so the scope is not mistakable. Names are comma separated and case sensitive, matched against the image's own channel names before any kronos_marker_mapping rename. A name matching nothing fails the run and lists the channels that are present; the nuclear channel cannot be withheld because it is the model's preferred_dapi. While wiring this up, channel indices became load-bearing and exposed a silent correctness bug on OME-TIFFs that give each channel its own 2D series. ChannelReader sized itself from series[0] alone, so an 8-series image read as 1 channel while the patch buffer was still sized from the 8-name marker list -- and numpy broadcast that one channel across all eight. The run completed, reported 8 channels, and produced 768-d vectors computed from eight copies of DAPI. Verified on the COMET _fixed images. The layout is now detected once and shared by the name list and the pixel reader, each channel is read from its own series, read_window refuses to return fewer channels than were asked of it, and the run fails outright if the name count and the image's channel count disagree. Channel names are also built positionally, so an OME Channel carrying neither Name nor ID no longer shifts every later marker onto the wrong channel. The marker report gains the embedded/not-shown split and names withheld channels in place, and is now written before the novel-marker gate can exit -- a failing run is when it is most useful. Adds tests/python/test_kronos2_common.py, the first module under the tests/python scaffold, plus a pytest job in the linting workflow so it runs, and a stub nf-test for the module, which had none. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two changes, one of which is a silent correctness bug worth reading first.
1.
kronos_exclude_markers— withhold non-marker channels from KRONOS2A COMET panel's
Autofluorescencechannel is not a marker and is not in KRONOS2's288-marker vocabulary, so the run fails:
There was no way to say "don't embed this channel".
--marker-mappingonly renames,and
--allow-novel-defaultsstill feeds the channel to the model, normalised withdefault statistics — the opposite of what is wanted for a channel carrying no biology.
--kronos_exclude_markers 'Autofluorescence'Scoped to the embedding step; nothing is deleted. A withheld channel is simply
never read, so it stays in the image, remains available to segmentation and every
other process, and keeps whatever intensity measurements CELLMEASUREMENT wrote for it
in the GeoJSON. This is deliberately unlike
remove_markers, which does stripchannels out of the background-subtracted image — the parameter shape is borrowed from
it, the blast radius is not, and the log line, marker report and docs all say "not
shown to KRONOS2" rather than "removed" so the distinction is not mistakable.
Names are comma separated and case sensitive, matched against the image's own channel
names before any mapping. A name matching nothing fails the run and lists the
channels that are present; the nuclear channel cannot be withheld because it is the
model's
preferred_dapi.2.⚠️ KRONOS2 embedded one channel under every marker's name
Found while making channel indices load-bearing for the above.
On OME-TIFFs that store each channel as its own 2D series,
ChannelReadersized itselffrom
series[0]alone, so an 8-series image read as 1 channel — whileCellPatchDatasetstill sized the patch buffer from the 8-name marker list. NumPy thenbroadcast that single channel across all eight slots.
The run completed, logged
Found 8 channels, and produced 768-d vectors computed fromeight copies of DAPI. No error, no warning. The tell in the log is
Found 8 channelson one line and
image 1x27454x26980a few lines later.Confirmed on the COMET
_fixedimages (inForm unmixing export): every one of the 8"markers" came back byte-identical to DAPI. Embeddings produced from such images by
earlier versions are invalid and need regenerating.
Now: the layout is detected once by
channel_layout()and shared between the name listand the pixel reader; each channel is read from its own series via its own zarr store;
read_windowrefuses to return fewer channels than it was asked for, so this class ofbug cannot recur silently; and the run fails outright if the channel-name count and the
image's channel count disagree.
Also fixed in passing: a channel whose OME
Channelelement carried neitherNamenorIDwas dropped from the name list but not from the image, sliding every later markeronto the wrong channel. Names are now built positionally, one per channel always.
Marker report
Gains the embedded/not-shown split and names withheld channels in place. It is now
written before the novel-marker gate can exit — a failing run is exactly when you
need it, since it lists the channel names to map or withhold.
Testing
tests/python/test_kronos2_common.pyis the first module under thetests/python/scaffold, which had a
conftest.pyand no tests since the cellmeasurement extraction.22 tests covering channel naming, selection, the reader and the report. Pure python —
no torch, no GPU, no gated weights — so a
pytestjob has been added to the lintingworkflow and it actually runs in CI.
The split-layout test is a genuine regression guard: against the pre-fix code its
fixture returns eight identical channels, against the fix it returns eight distinct
ones.
Also adds a stub
nf-testforKRONOS2EMBEDDINGS, which had none — the KRONOS1 testwas deleted in the v2 migration and never replaced.
Verified locally beyond the unit tests:
(7, 64, 64)with per-channel values matching the raw file exactly.nuclear channel, everything withheld).
ext.argsexpansion quotes names containing spaces ('Collagen 4'),trims whitespace and drops empty entries from a trailing comma.
pre-commit run --all-filespasses.Not run: the model forward itself, which needs the gated snapshot and a GPU. The
preprocesscontract was verified by reading the model source — it builds its statslist from
marker_namesand broadcasts, so it is channel-count agnostic.PR checklist
nf-core pipelines lint) —nf-corenot available locally;pre-commitpassesnextflow run . -profile test,docker --outdir <OUTDIR>) — not run locally (no container engine on the login node)docs/usage.mdis updated.docs/output.mdis updated.CHANGELOG.mdis updated.README.mdis updated — n/a, it delegates the parameter list todocs/usage.md🤖 Generated with Claude Code