Milab-6847: fix calculations with empty abundance - #40
Closed
erohinaelena wants to merge 3 commits into
Closed
Conversation
The input table was built with tsvFileBuilder.build(), which emits an untyped TSV, so ptabler let polars infer the column types. An abundance column with no parseable numeric value in the exported table -- an empty join result, or all-null abundance -- is inferred as String; sum() keeps that dtype and the per-sample normalisation then fails during query planning with "division with 'String' datatypes is not allowed". Build the table with buildForPT() instead, so column types come from the p-column and axis specs and nulls are written as the "%%NA%%" sentinel that ptabler declares per column. wf.frame() reads xsvType and the schema straight off the structural input, so its options argument is no longer needed. This changes the exported TSV bytes, so cached results for this block are invalidated -- correct here, since the previous output was wrong.
The shared CI gates publication on @platforma-sdk/block-tools being at the latest version (node-simple-pnpm.yaml, require-latest step), so the build is rejected otherwise. pnpm-lock.yaml is updated alongside, which the same workflow also enforces for any pnpm-workspace.yaml change.
Both workflows pinned node-version to '20.x'. Move them to '22.x', matching the 12 blocks already on it against the same shared workflows (node-simple-pnpm.yaml and block-mark-stable.yaml at @v4). Both also carried a copy-paste leftover naming this block "CDR3 SpectraType", with app-name-slug 'block-cdr3-spectratype' colliding with the real cdr3-spectratype block. Since the slug is part of the release file name and app-name is what CI reports in messages, this block's builds and releases were labelled as another one. Name it after itself instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Greptile Summary
The PR fixes usage normalization for empty or all-null abundance data by preserving the analysis table’s declared schema and updates the associated build tooling.
buildForPT()and letswf.frameconsume its embedded format and schema metadata.@platforma-sdk/block-toolsfrom 2.12.6 to 2.14.3 and records patch releases for the workflow and block packages.buildForPT().Confidence Score: 5/5
The PR appears safe to merge, with no concrete blocking or independently actionable non-blocking issue identified.
The schema-aware table producer and frame consumer are changed together, their headers remain aligned with all usage aggregations, and the tooling updates reveal no established reachable failure.
Important Files Changed
buildForPT()so declared axis and column types survive empty or all-null datasets.@platforma-sdk/block-toolsto 2.14.3.Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart LR A[Resolved sample, clonotype, V, J, and abundance columns] --> B[Build analysis table with buildForPT] B --> C[Schema-carrying PTabler input] C --> D[Load with wf.frame] D --> E[Calculate per-sample totals] D --> F[Aggregate V, J, and V/J groups] E --> G[Normalize weighted and unweighted usage] F --> G G --> H[Publish typed usage results]Reviews (1): Last reviewed commit: "MILAB-6847: run CI on node 22, fix copy-..." | Re-trigger Greptile
Context used (3)