MILAB-6871: size analyze scratch space from the sample's reads

[DenKoren]

What

mixcr analyze now asks the backend for disposable disk space sized from the sample's own reads, instead of writing its temporary files into the working directory on shared storage. --use-local-temp is dropped, since TMPDIR points at that storage.

Size

S   = 2.0 x millionsOfReads x (lengthOfR1 + lengthOfR2) / 1000
C   = 11  x millionsOfReads x (lengthOfTheLongestRead / 1000) ^ 2
GiB = max(S, C)                                                    (rounded up)

S is two bytes per base of the pair, so it tracks the volume of the input. C is quadratic in read length and overtakes S only on long reads.

sample	S	C	request
72.6M pairs x 151 bp	43.85	18.21	44GiB
1M pairs x 2000 bp	8.00	44.00	44GiB
72.6M single-end x 151 bp	21.93	18.21	22GiB

None of those numbers is in the column metadata, so both read ends are measured in one pass before the run:

seqkit stats --threads {system.cpu} --tabular reads_R1_0.<ext> reads_R2_1.<ext>
  ->  csvtk --tabs csv2json

--threads reads the granted core count back from {system.cpu}, so it tracks the allocation rather than repeating a literal. Bases are summed rather than averaged — exact, and needs no weighting when a sample spans lanes. The read count comes from R1 alone, since it counts pairs.

Shape

file	role
workflow/src/read-stats.lib.tengo	new — the measurement and the size formula, in one place
workflow/src/run-mixcr.tpl.tengo	new — the previous mixcr-analyze body, plus a readStats input
workflow/src/mixcr-analyze.tpl.tengo	now measures, then renders run-mixcr
workflow/src/process.tpl.tengo	unchanged

The split is forced, not cosmetic: a template cannot read the result of a command it started itself, and exec.formula only knows size/lineCount on files.

Deduplication is preserved

pframes.processColumn still renders mixcr-analyze with the same inputs, and mixcr-analyze carries the hash_override every released block carries. The per-group dedup key is therefore what it has always been, so an analysis that already finished resolves to its existing result instead of running again. That is safe because scratch space changes only where MiXCR writes its temporary files, never what it produces.

run-mixcr carries a freshly generated override — never the one mixcr-analyze uses, since a shared override declares two templates interchangeable to the backend.

The inner render passes the CPU and memory overrides as meta inputs, matching how process.tpl.tengo passes them in via metaExtra, so a run can still be recovered across a change of either.

Also reverts an unreleased override rotation

86d727d (MILAB-6721) rotated mixcr-analyze's override from D70EDB25 to AD2092AB because the memory request formula changed. That was a mistake — a resource formula changes what a run asks for, never what it produces — and it never shipped: every release up to v2.22.0 carries D70EDB25, and git tag --contains 86d727d is empty. Left in place it would have re-analysed every sample in every project on the next block update.

Verified on lab.research.platforma.bio: a dev build off main re-ran samples the released block had already processed, and the DB showed AD2092AB appearing for the first time on that instance the moment the dev block loaded. This PR reverts the value and documents above it, in both templates that carry one, that changing it means "re-run every sample on the next block update".

Failure behaviour

The request is an optimisation, never a precondition. A deployment that cannot serve the size shrinks or grows it, one with no scratch storage ignores it, and a sample with no read end to measure asks for nothing. Analyze runs either way.

Tests

workflow/src/read-stats.test.tengo runs on the tengo VM via pl-tengo test and covers which term wins, single-end input, multi-lane summing, the longest read crossing ends, the round up to a whole GiB, and every case that must ask for nothing.

Verified on a live cluster

Ten samples through denkoren-test (EKS, EBS gp3 scratch class), against the earlier single-term formula. Requests were read from the backend's compute request was registered log line and came out per-sample distinct — 20, 22, 23, 23, 24, 24, 24, 25, 26 GiB — each then raised to the deployment's --k8s-min-scratch-request=32Gi floor, which is the EBS gp3 IOPS-ratio minimum for that storage class. The formula in this PR sizes those same samples at roughly twice that, so most will now clear the floor on their own.

Dependencies

• @platforma-sdk/workflow-tengo 6.8.2 -> 6.9.0 — first release carrying scratchFreeSpace
• @platforma-open/milaboratories.software-binary-collection 1.0.12 — new, supplies seqkit and csvtk