Close the skipped minors; list-metrics and guide for native 3-D files

CHANGELOG.md

@@ -45,10 +45,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
 ### Changed
 
 - **Sessions screen: a session tracked in separate intervals shows its real length.** The Frames column and the detail panel read "400 of 600" (rows tracked of video frames up to the end of the last interval), and the minutes in the detail panel and the summary line use the real elapsed time rather than the row count, which understated a file with gaps (a file with frames 0-199 and 400-599 showed 13 s instead of 20 s). Sessions without gaps look as before; the reader contract (`video.n_frames` is the row count) is unchanged.
-- **Clearer native 3-D wording on the command line.** `track2data validate` and `run` name the project setting (`calibration.water_column`) instead of the Calibration page when IL-15 waits for the water column; the Metrics screen and the run `README.md` keep their text. `track2data list-metrics` ends with a line saying that availability also depends on the project (3-D data, cm scale, pixel frame, number of animals) and pointing to `validate`. `track2data add --yes` prints a `check:` line for every required option the reader proposed rather than you gave (for example `unit=mm` for Anipose). `docs/METRICS_SPEC.md` gains an Availability note on IL-3, IL-9, IL-10, IL-14 and the group metrics for native 3-D projects.
+- **Calibration screen: native 3-D sessions are no longer described as having a pixel scale.** The per-session readiness list said "calibrated (1 px per unit)" and the note under Session calibration asked for a "Scale (pixels per cm)" import option that native readers do not have. A project with only native 3-D files now shows "<id> — in cm (converted when read)" (with the project unit) and a note that native files are already in physical units. Mixed lists and idtracker.ai projects keep their text.
+- **Clearer native 3-D wording on the command line.** `track2data validate` and `run` name the project setting (`calibration.water_column`) instead of the Calibration page when IL-15 waits for the water column; the Metrics screen and the run `README.md` keep their text. `track2data list-metrics` ends with a line saying that availability also depends on the project (3-D data, cm scale, pixel frame, number of animals) and pointing to `validate`, and, above that line, lists what a native 3-D project rules out (IL-3, IL-9, IL-10, IL-14, IL-15 until the water column is given, and every group metric) with the same reasons as the Metrics screen. `track2data add --yes` prints a `check:` line for every required option the reader proposed rather than you gave (for example `unit=mm` for Anipose). `docs/METRICS_SPEC.md` gains an Availability note on IL-3, IL-9, IL-10, IL-14 and the group metrics for native 3-D projects.
 
 ### Fixed
 
+- **Native 3-D readers: clearer refusals and scan warnings.** A hand-edited Anipose file with a duplicate keypoint column (for example two `a_x` columns) used to be read from the first of them; it is now refused with `ANIPOSE_DUPLICATE_COLUMN` (DeepLabCut 3-D already refused such files), and the scan lists "duplicate column <name>: the file will be refused when read" on the file's session. A file whose header is longer than 8 MB is still not detected, but reading it directly now raises `HEADER_TOO_LONG` instead of "not an Anipose / DeepLabCut 3-D table". `NATIVE_3D_MIXED` calls a folder that no native reader recognised "not a native 3-D file (no native reader recognised it)" instead of "pixel-tracked". No metric value changes.
+
+- **IL-17 3-D speed: bin-edge values now match the whole-session estimate.** The vertical speed was estimated inside each time bin, so the first and last frames of a bin used the estimator's edge behaviour instead of the whole-session value (IL-2's speed is computed once and sliced). It is now computed once on the whole session's Z with the project's kinematics settings and cut into the bins with the session. Only IL-17 values in binned runs change (at the bin edges); unbinned IL-17 and every other metric are unchanged.
+- **GL-1 and GL-16 no longer count the gap marker between tracking intervals as a skipped frame.** A session tracked in separate intervals keeps one NaN separator row at each unfilled gap; it is not a frame, but `n_skipped_frames` (GL-1) and `n_skipped_frames_3d` (GL-16) counted it as one skipped frame per gap (and, for one animal, counted it among the frames). They now count only real frames: frames with at least one animal missing (GL-1) or missing X, Y or Z (GL-16), and for a one-animal session the number of real frames. **Only sessions with at least one unfilled gap are affected** (each count falls by the number of gaps); a session without tracking intervals, or whose gaps were all filled, is unchanged, and no mean or median changes.
+- **GL-16 `mean_nnd_3d_bl` now follows GL-1's body-length rule.** GL-1 divides by the mean of the animals' body lengths (animals with no length are ignored) and gives NaN unless that mean is positive. GL-16 used to drop every animal whose own length was zero, negative or infinite before averaging. It now averages the raw lengths like GL-1: a zero length counts in the mean (lengths 10, 0, 20 px give a ratio from a mean of 10, not 15), a negative length counts and the ratio is valid when the mean is still positive, and an infinite length gives a ratio of 0.0, as in GL-1 (also when no other length is finite). **Only a session with a zero, negative or infinite body length for some animal changes**; positive and missing (NaN) lengths give exactly the same values as before.
 - **Native 3-D files read with `vertical_axis = y` no longer have a mirrored horizontal plane.** The two axes that are not vertical were taken in file order, which for `y` gives `(x, z)`, the mirror image of the right-handed `(z, x)`. Distances, speeds and every magnitude were unaffected, but any signed quantity was reflected: **IL-11's `mean_heading_rad` (θ became π/2 − θ) and the sign of `left_right_turn_bias`** for such files. The plane is now taken in cyclic order after the vertical axis (`z`: x, y; `x`: y, z; `y`: z, x), so it is right-handed seen from above whichever axis is vertical. `vertical_axis = z` (the default) and `x` are unchanged; **only files read with `y` change**, and only those two signed values. `flip` still reverses the vertical axis alone, which also mirrors the view from above; that is a documented limit, not changed here.
 - **Native 3-D projects refuse files on different vertical axes.** The project has one water column, entered on "the file's own vertical axis", but `vertical_axis` was only recorded in provenance, so two files read with different axes shared a surface and floor that meant different things for each. The import rule that already refuses mixed units now also refuses a file whose `vertical_axis` differs from the project's (`NATIVE_3D_AXIS_MISMATCH`, naming both axes and the fix), whether it comes in with the same batch, was already in the project, or is a session just read. A missing option counts as `z`, the default; `flip` may still differ because it is compensated per session. The rule runs whenever a session is read, as the unit rule does, so an existing project whose files already disagree is refused with the same message until they are re-added on one axis. The origin of the axis cannot be checked from the files, so the water column still assumes one shared origin.
 - **Native 3-D files: the water column now bounds every 3-D metric.** It already excluded out-of-column samples from IL-15's depth statistics, but IL-16, IL-17 and GL-16 read the raw z, so a mis-triangulated or out-of-tank sample (a reflection, a fish out of the water) stayed in the path length and speed. One helper, `vertical_cm`, now gives the vertical coordinate to all of them: a sample outside the water column is NaN there, so a step or frame that needs it is dropped (nothing is clipped). The exported `z_cm` column stays the file's own coordinate. **IL-16, IL-17 and GL-16 change for native 3-D projects that set a water column and have samples outside it**; without a column, or with every sample inside, they are unchanged. The Calibration page says so.

docs/3d-movement/2026-10-10-3d-metrics-design.md

@@ -104,8 +104,9 @@ Where the build differs from the text above, the build is right.
 - **IL-17 and the inequality.** `3-D speed >= IL-2` holds only over the frames where the vertical
   speed is finite (the forward-difference estimator gives none for the last frame; a depth segment
   shorter than the Savitzky-Golay window gives none). Frames without a vertical speed are left out of
-  IL-17, never counted as 0. In a binned run `vz` is estimated inside each bin, so the estimator's edge
-  frames can differ slightly from a whole-session run.
+  IL-17, never counted as 0. `vz` is estimated once on the whole session (`geometry3d.vertical_speed`, set as
+  `PreprocessedSession.z_speed` by the Engine) and sliced with the session, so a bin's edge frames match a
+  whole-session run.
 - **IL-16 and IL-1.** `IL-16 >= IL-1` holds over the steps where depth is also finite (`n_valid_steps`).
 - **GL-16 and GL-1.** GL-16 skips frames where any fish lacks depth that GL-1 keeps, so the
   inequality holds only per frame where both are valid, not for the two means.

docs/METRICS_SPEC.md

@@ -444,7 +444,7 @@ info-button modal (§6).
 | **Formula** | `v3[t, k] = sqrt(vh² + vz²)` with `vh = speed_px_s[t, k] / px_per_cm` (identical to the horizontal part of IL-2 in cm/s) and `vz` the vertical speed in cm/s of `Z = depth · tank_height_cm` (for a native file, its z after the same gap filling and smoothing as x and y, with samples outside the water column left out), from the same estimator and window as the pipeline's kinematics (Savitzky-Golay by default, forward difference when the project selects it); mean/median/max over frames where both `vh` and `vz` are finite |
 | **Output columns** | `individual_id`, `mean_speed_3d_cm_s`, `median_speed_3d_cm_s`, `max_speed_3d_cm_s`, `mean_speed_3d_bl_s` |
 | **Units** | cm/s / BL/s |
-| **Assumptions** | The depth comes from a side camera at right angles to the top camera. A frame needs both speeds finite: where the estimator gives no vertical speed (the last frame of the forward-difference estimator, a depth segment shorter than the window needs) the frame is left out of the 3-D speed, never counted as 0. An animal with no such frame is NaN. In a binned run `vz` is estimated inside each bin, so the estimator's edge frames differ slightly from the whole-session values |
+| **Assumptions** | The depth comes from a side camera at right angles to the top camera. A frame needs both speeds finite: where the estimator gives no vertical speed (the last frame of the forward-difference estimator, a depth segment shorter than the window needs) the frame is left out of the 3-D speed, never counted as 0. An animal with no such frame is NaN. `vz` is estimated once on the whole session's Z and sliced with the session, as IL-2's speed is, so in a binned run a bin's first and last frames carry the whole-session estimate (not the estimator's edge behaviour inside the bin); unbinned values are unchanged |
 | **Warnings** | Max speed is sensitive to jump artefacts and to depth noise. No refraction or parallax correction. A wrong tank height scales every vertical speed. Over the frames where the vertical speed is finite, the 3-D speed is at least IL-2's cm/s (not an unconditional inequality: frames without a vertical speed are in IL-2 but not here). Without depth or a cm scale all columns are NaN |
 | **Reference** | Standard kinematics |
 | **Supporting references** | Bjorneraas et al. 2010, J. Wildl. Manage. 74(6):1361-1366 (screening GPS location data for errors using animal movement characteristics) (DOI: 10.2193/2009-405) |
@@ -628,7 +628,7 @@ info-button modal (§6).
 | **Formula** | `nnd[t] = mean_k min_{j≠k} ‖xy[t, k] − xy[t, j]‖`, computed via `scipy.spatial.cKDTree` |
 | **Output columns** | `mean_nnd_px`, `mean_nnd_cm`, `mean_nnd_bl` (and time-series option) |
 | **Units** | px, cm, BL |
-| **Assumptions** | All `N_animals` are present in the frame (NaN-bearing frames are skipped) |
+| **Assumptions** | All `N_animals` are present in the frame (NaN-bearing frames are skipped). A separator row (the NaN row kept at an unfilled gap between tracking intervals) is not a frame: it is neither used nor counted in `n_skipped_frames`, which is the number of real frames with a NaN animal (for one animal, the number of real frames). `mean_nnd_bl` divides by the nan-mean of the per-animal body lengths in px, and is NaN unless that mean is positive (an infinite length gives 0.0) |
 | **Warnings** | With NaN ≥ 1 in a frame, frame is excluded; report % skipped |
 | **Reference** | Clark & Evans 1954, Ecology 35(4):445-453 (distance to nearest neighbor as a measure of spatial relationships in populations) — DOI [10.2307/1931034](https://doi.org/10.2307/1931034) |
 | **Supporting references** | Pitcher 1973, Anim. Behav. 21(4):673-686 (the three-dimensional structure of schools in the minnow, Phoxinus phoxinus) (DOI: 10.1016/S0003-3472(73)80091-0); Krause & Ruxton 2002, Living in Groups (Oxford University Press) (DOI: 10.1093/oso/9780198508175.001.0001) |
@@ -845,7 +845,7 @@ Group metrics (`GL-*`) are unavailable in native 3-D projects: see the Availabil
 | **Formula** | `P[t, k] = (x/px_per_cm, y/px_per_cm, depth · tank_height_cm)`; `nnd[t] = mean_k min_{j≠k} ‖P[t, k] − P[t, j]‖`, computed via `scipy.spatial.cKDTree` as for GL-1; mean and median over frames |
 | **Output columns** | `mean_nnd_3d_cm`, `median_nnd_3d_cm`, `mean_nnd_3d_bl`, `n_skipped_frames_3d`, `frac_frames_used_3d` |
 | **Units** | cm / BL / frames / fraction (0-1) |
-| **Assumptions** | The depth comes from a side camera at right angles to the top camera. A frame needs finite X, Y and Z for every animal; other frames are skipped and counted in `n_skipped_frames_3d`, and `frac_frames_used_3d` is the share of the real frames (separator rows excluded) that were used. GL-1 also skips a frame when any animal lacks X or Y, but not for a missing depth. Needs no identities (an unordered point set per frame, like GL-1). `mean_nnd_3d_bl` uses GL-1's convention: the mean body length over the animals. One animal, or no valid frame, gives NaN (never 0) and `n_skipped_frames_3d` is the number of frames in the first case |
+| **Assumptions** | The depth comes from a side camera at right angles to the top camera. A frame needs finite X, Y and Z for every animal; other frames are skipped and counted in `n_skipped_frames_3d`, and `frac_frames_used_3d` is the share of the real frames (separator rows excluded) that were used. GL-1 also skips a frame when any animal lacks X or Y, but not for a missing depth. Needs no identities (an unordered point set per frame, like GL-1). `mean_nnd_3d_bl` follows GL-1's rule: the nan-mean of the raw per-animal body lengths (in cm), NaN unless that mean is positive (an infinite length gives 0.0, as in GL-1). One animal, or no valid frame, gives NaN (never 0) and `n_skipped_frames_3d` is the number of frames in the first case |
 | **Warnings** | GL-1 and GL-16 are computed on different frames whenever depth is missing, and the frames lost are not a random sample: those in which every animal is tracked cleanly are disproportionately frames without crossings, where neighbours are far apart, so `mean_nnd_3d_cm` is biased upward relative to `mean_nnd_cm` by an amount that grows with the number of animals and with the loss. A warning is logged when GL-16 uses more than 5 % fewer frames than GL-1. GL-1 may keep frames GL-16 skips (missing depth), so "3-D NND ≥ GL-1" holds only over the frames where both are valid (and then only per frame, not for the two means). No refraction or parallax correction. A wrong tank height scales every vertical distance. With a constant depth GL-16 equals GL-1 in cm. Without depth or a cm scale all statistics are NaN |
 | **Reference** | Clark & Evans 1954, Ecology 35(4):445-453 (distance to nearest neighbor as a measure of spatial relationships in populations) — DOI [10.2307/1931034](https://doi.org/10.2307/1931034) |
 | **Supporting references** | Pitcher 1973, Anim. Behav. 21(4):673-686 (the three-dimensional structure of schools in the minnow, Phoxinus phoxinus) (DOI: 10.1016/S0003-3472(73)80091-0); Krause & Ruxton 2002, Living in Groups (Oxford University Press) (DOI: 10.1093/oso/9780198508175.001.0001) |