Skip to content

Milab-6847: fix calculations with empty abundance - #40

Closed
erohinaelena wants to merge 3 commits into
mainfrom
MILAB-6847_vj-usage-abundance-dtype
Closed

Milab-6847: fix calculations with empty abundance#40
erohinaelena wants to merge 3 commits into
mainfrom
MILAB-6847_vj-usage-abundance-dtype

Conversation

@erohinaelena

@erohinaelena erohinaelena commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR fixes usage normalization for empty or all-null abundance data by preserving the analysis table’s declared schema and updates the associated build tooling.

  • Replaces the untyped TSV build with buildForPT() and lets wf.frame consume its embedded format and schema metadata.
  • Updates CI identities to V/J Gene Usage and moves CI from Node 20 to Node 22.
  • Upgrades @platforma-sdk/block-tools from 2.12.6 to 2.14.3 and records patch releases for the workflow and block packages.
  • Important touched terms:
    • Abundance — the per-clonotype numeric weight used for weighted usage; its declared numeric type is now preserved even when every value is null or no rows remain.
    • Weighted usage — a gene group’s abundance divided by the matching sample’s total abundance; the change prevents its division expression from receiving a string-typed abundance column.
    • Unweighted usage — a gene group’s row count divided by the sample’s total row count; its calculation is unchanged.
    • PTabler schema — metadata describing table column and axis types; it is now carried by the table produced with buildForPT().
    • Analysis table — the sample, clonotype, V gene, J gene, and abundance input to the calculation stage; its construction changes from inferred TSV typing to declared typing.

Confidence Score: 5/5

The PR appears safe to merge, with no concrete blocking or independently actionable non-blocking issue identified.

The schema-aware table producer and frame consumer are changed together, their headers remain aligned with all usage aggregations, and the tooling updates reveal no established reachable failure.

Important Files Changed

Filename Overview
workflow/src/main.tpl.tengo Builds the calculation input with buildForPT() so declared axis and column types survive empty or all-null datasets.
workflow/src/usage.lib.tengo Loads the structural PTabler input directly instead of overriding it as a plain TSV.
pnpm-workspace.yaml Upgrades @platforma-sdk/block-tools to 2.14.3.
.github/workflows/build.yaml Corrects the application identity and moves the shared build workflow to Node 22.
.github/workflows/mark-stable.yaml Corrects the stable-release application identity and moves that workflow to Node 22.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Resolved sample, clonotype, V, J, and abundance columns] --> B[Build analysis table with buildForPT]
  B --> C[Schema-carrying PTabler input]
  C --> D[Load with wf.frame]
  D --> E[Calculate per-sample totals]
  D --> F[Aggregate V, J, and V/J groups]
  E --> G[Normalize weighted and unweighted usage]
  F --> G
  G --> H[Publish typed usage results]
Loading

Reviews (1): Last reviewed commit: "MILAB-6847: run CI on node 22, fix copy-..." | Re-trigger Greptile

Context used (3)

The input table was built with tsvFileBuilder.build(), which emits an untyped
TSV, so ptabler let polars infer the column types. An abundance column with no
parseable numeric value in the exported table -- an empty join result, or
all-null abundance -- is inferred as String; sum() keeps that dtype and the
per-sample normalisation then fails during query planning with
"division with 'String' datatypes is not allowed".

Build the table with buildForPT() instead, so column types come from the
p-column and axis specs and nulls are written as the "%%NA%%" sentinel that
ptabler declares per column. wf.frame() reads xsvType and the schema straight
off the structural input, so its options argument is no longer needed.

This changes the exported TSV bytes, so cached results for this block are
invalidated -- correct here, since the previous output was wrong.
The shared CI gates publication on @platforma-sdk/block-tools being at the
latest version (node-simple-pnpm.yaml, require-latest step), so the build is
rejected otherwise. pnpm-lock.yaml is updated alongside, which the same
workflow also enforces for any pnpm-workspace.yaml change.
Both workflows pinned node-version to '20.x'. Move them to '22.x', matching the
12 blocks already on it against the same shared workflows (node-simple-pnpm.yaml
and block-mark-stable.yaml at @v4).

Both also carried a copy-paste leftover naming this block "CDR3 SpectraType",
with app-name-slug 'block-cdr3-spectratype' colliding with the real
cdr3-spectratype block. Since the slug is part of the release file name and
app-name is what CI reports in messages, this block's builds and releases were
labelled as another one. Name it after itself instead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant