Research data infrastructure and model-development work built for the Salk Institute's mouse behavioral neuroscience program. The directories below cover the full path from raw homecage-monitor (HCM) recordings, through the iteratively-trained SLEAP models that process them, to GPU-processed pose-estimation output ready for behavioral analysis.
| Directory | What it does | Environment |
|---|---|---|
hcm-video-organization/ |
Moves raw MP4s off the recording workstation and sorts them onto network storage by cage and date | Windows workstation → NAS |
sleap-model-benchmarking/ |
Qualitatively evaluates iteratively-trained SLEAP models by rendering their predictions onto fixed video clips for PI review — the model-development loop behind the inference pipeline | 4-GPU Linux compute node |
sleap-inference/archive/ |
Single-node production pipeline — pins one notebook per GPU, parallel workers on each | 1 node, 4 GPUs |
sleap-inference/multi-node-inference/ |
Scale-out variant fanning the same workload across a heterogeneous cluster — a reconstruction, not original artifacts | 3 nodes, 10 GPUs |
hcm-video-organization/ and sleap-inference/archive/ ran in production across multiple experiment cohorts — 84 videos per cohort, across 4 recording cages. The models they run were developed and validated with the benchmarking workflow.
The 10-GPU multi-node variant is a reconstruction. This workload did run across 3 nodes and 10 GPUs, but those original scripts were not retained. The notebooks in
multi-node-inference/rebuild that design from the single-node pipeline that was kept. They document the approach, not the code that ran at the time.
The partitioning design is the most interesting thing here: the local GPU index (--gpu 0–3, which physical card to run on) is decoupled from the global partition index (0–9, which slice of the video list to claim), joined by a NODE_OFFSETS table mapping each hostname to its starting offset. Because every worker applies the same deterministic random.seed(42) shuffle before np.array_split, all ten workers on three separate machines independently derive disjoint, gap-free slices — no shared queue, no scheduler, no cross-node coordination at all.
A separate recording stream from the HCM work above — different assay, different acquisition, no shared code.
| Directory | What it does | Environment |
|---|---|---|
three-chamber-video-cropping/ |
Crops apparatus video to a per-cohort region before pose estimation. A GUI sets the crop box against a real frame; crop size is locked across cohorts so all output is uniform. | Linux node, ffmpeg |
Experiments use Bonsai to record continuous video from multiple homecage cameras, each mapping to a specific cage housing 2 mice. After recording, videos need to move from the acquisition workstation to a Linux NAS for compute. They are then processed with SLEAP — a machine-learning toolkit for multi-animal pose tracking — which extracts per-frame body keypoints for downstream behavioral analysis.
These directories are the infrastructure that makes that workflow repeatable across cohorts — from moving raw video, to developing and validating the pose models, to running them at scale.
Note on paths. Infrastructure identifiers throughout this repo — the NAS root (
/nas/...), home directories, cluster hostnames (node01–node03), and the Windows share (N:\) — are placeholders standing in for internal Salk values. They preserve the shape of the originals so the examples still read correctly, but the commands will not run as-is without repointing them. Command structure, flags, and arguments are unmodified.
A guard in .githooks/pre-commit keeps internal identifiers and notebook output from re-entering the repo. Install it once, from the repo root:
git config core.hooksPath .githooks
It does two things on every commit: clears outputs and execution counts from staged notebooks (re-staging what it rewrites), and fails the commit if a known internal identifier appears in a staged filename or anywhere in staged content. Matching on bare tokens rather than path shapes is deliberate — the notebooks build paths as os.path.join(os.sep, "...", ...), which no path-shaped pattern catches. Python 3 only; no nbstripout or other install required. Bypass with git commit --no-verify.