Skip to content

Restore the original cell order after the FCS roundtrip - #128

Closed
rcannood wants to merge 3 commits into
mainfrom
fix-fcs-cell-order
Closed

Restore the original cell order after the FCS roundtrip#128
rcannood wants to merge 3 commits into
mainfrom
fix-fcs-cell-order

Conversation

@rcannood

Copy link
Copy Markdown
Member

Describe your changes

anndata_to_fcs() builds one flowFrame per sample, so flowCore::fsApply(fset, exprs) gives the cells back grouped per sample. gaussnorm and the six cytonorm_* methods then attach that matrix to the obs of the input AnnData, which is in the original cell order:

integrated_matrix <- fsApply(fset, exprs)     # grouped per sample

output <- anndata::AnnData(
  obs = adata$obs[, integer(0)],              # original cell order
  layers = list(integrated = integrated_matrix)
)

As long as the cells in the input happen to be contiguous per sample and in first-appearance order, the two orders coincide and everything is fine -- which is presumably why this hasn't shown up. But nothing in file_censored_split1.yaml guarantees it, and if it ever doesn't hold, every cell gets paired with another cell's expression values. Silently. It would look like a badly performing method rather than a bug.

batchadjust_* already guards against this explicitly with its Original_ID column; this PR does the equivalent for the other two families:

  • Adds fcs_cell_order() to src/utils/anndata_to_fcs.R, which returns the AnnData row indices in the order the concatenated matrix has them, looked up by the FlowSet's own sample names. It errors out if the FlowSet doesn't cover exactly the cells of the AnnData object.
  • Uses it in methods/gaussnorm and in the six methods/cytonorm_* variants.
  • In the cytonorm methods, also checks that CytoNorm.normalize() returned the same number of cells per sample as it was given, so the assumption that it preserves frame order fails loudly rather than silently.

This is a no-op when the current implicit assumption holds -- order(fcs_cell_order(...)) is the identity permutation for sample-contiguous input -- so it should not change any score unless there was a real problem to begin with. That also makes it a cheap way to find out whether there was one: if a score moves, the ordering was wrong.

I haven't been able to run viash test for these locally, so CI and a test benchmark run are the real check here.

Found while reviewing the task ahead of the next full benchmark run -- see also the sibling PRs.

Checklist before requesting a review

  • I have performed a self-review of my code

  • Check the correct box. Does this PR contain:

    • Breaking changes
    • New functionality
    • Major changes
    • Minor changes
    • Bug fixes
  • Proposed changes are described in the CHANGELOG.md

  • CI Tests succeed and look good!

* Add fcs_cell_order() to anndata_to_fcs.R
* Use it in gaussnorm and the six cytonorm methods
* Check that cytonorm returns the same number of cells per sample
@rcannood
rcannood requested review from LuLeom and ghar1821 July 28, 2026 10:59

@ghar1821 ghar1821 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes here would "reshuffle" the output of a method if the method has somehow reordered the cells when performing the correction. But we should be penalising this because it means the method has done something it should not have done. Thus we should not try to correct it.

Hence this pull request should not go ahead.

@LuLeom LuLeom left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BatchAdjust ordering check was meant to control the behaviour of the implementation, as it is the more fragile one. Methods that shuffle rows should be penalised

@ghar1821

Copy link
Copy Markdown
Contributor

Closing this without merging. See comments.

@ghar1821 ghar1821 closed this Jul 30, 2026
@rcannood
rcannood deleted the fix-fcs-cell-order branch September 8, 2026 09:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants