Barnes Maze Analysis Pipeline

• Live app: https://barnes-maze-pipeline.pages.dev
• Demo video: https://www.loom.com/share/ffe445ac34334ac099c95b025e38fd3a

Turns a folder of Barnes maze videos into a spreadsheet, in a browser, with no terminal. Everything runs client-side (WebCodecs + OpenCV.js): no backend, no GPU, and no video ever leaves your machine.

_{Real output for one sample trial: the tracked path, coloured by elapsed time. The per-hole visit raster and heat map are in examples/.}

Origin. I built this in September 2026 as my response to a public technical exercise from the Salk Institute's Center for AI and Research Computing (task: Barnes maze video analysis). I'm keeping it as a portfolio piece, as the exercise's terms allow. The sample videos belong to the Salk dataset and are not included here (see Credits). The full commit history is preserved, and the working notes - every wrong turn included - are in docs/process/.

Try it in 60 seconds

Open https://barnes-maze-pipeline.pages.dev and click Load an example (test50). That fetches one of the sample clips from the Salk pilot dataset and its saved analysis, and lands you on a tracked trial with real numbers - no video of your own, no account, no files to find. Everything after that (change a threshold, correct an event, export the spreadsheet) works exactly as it would on your own footage.

Who this is for

A lab student or rotating tech running Barnes maze trials for several labs. Today that means watching each video with a stopwatch and a clicker and typing the result into Excel by hand - slow, and inconsistent between people. This tool does the watching and counting automatically; the student's job becomes reviewing and correcting it, not starting from a blank spreadsheet. No terminal, no install, no account - open a link, drop in a video. Runs entirely on CPU - no GPU is required or used anywhere in the pipeline, so this works on whatever laptop is sitting at the rig, including an old one.

How it works

flowchart TD
    V[Video file] --> D["Frame-accurate decode<br/>(WebCodecs + mp4box.js, fps from container)"]
    D --> M["Median background<br/>(~60 frames)"]
    M --> R["Find platform + 20 holes<br/>(Hough circles)"]
    R --> T["Track the mouse<br/>(failed frames stay failed)"]
    T --> E["Detect events: two independent signals<br/>per-hole pixel change + blob area"]
    E --> C["Measures + search-strategy classifier"]
    C --> X["CSV / XLSX / PNG + reloadable project file"]
    H(["Human review:<br/>correct ROIs, events, verdicts"]) -.-> E
    H -.-> C

Loading

The design rule that shapes everything: two detectors that fail differently, and disagreement between them is flagged for a person instead of being silently resolved. Rationale and the wrong turns it took to get there: docs/process/DECISIONS.md.

What it does

Updated as pieces ship, not written once and left to drift:

• Load a video, decode it frame-accurately (exact fps from the container, not assumed)
• Work through multiple videos in one session - a roster on the left shows every video you've opened or pointed the tool at, click to switch between them, resume any of them (not just the last one) after closing the tab, and clear the list (with its saved work) in one place when you're done
• Compute a median background, detect the platform and all 20 holes automatically
• Confirm/correct the detected ROIs by hand (drag to fix, pick the target hole)
• Track the mouse across every frame (background subtraction), failed frames marked failed, never interpolated. Zoom the frame viewer (50-300%) to check exactly what happened near a hole
• Show where tracking failed across the whole video, not just how often - a timeline strip, so a real cluster reads differently from scattered misses
• Detect nose-poke/entry events per hole, with adjustable, visible thresholds. The poke signal is per-hole pixel change against the median background, not proximity between a tracked point and a hole - so a single body-centroid track never contaminates the poke/entry call the way a naive nose-vs-body distinction is usually meant to guard against. The centroid is used for path length, speed, and quadrant time, where whole-body position is the correct signal anyway. A low-confidence flag calls out events worth a second look, and a "False alarm" review option lets you dismiss one that turns out to be nothing (a shadow, a decode artifact) without forcing it into poke/entry/thigmotaxis
• Merge adjacent same-hole events into visits (handles re-engagement vs. genuine disengagement)
• Compute per-trial measures (primary/total latency, primary/total errors, working memory error, perseverative error, hole deviation score, commitment delay, target re-visit count, path length, speed, quadrant time) - definitions sourced from Gawel et al. 2019 and Illouz et al. 2020, not invented
• Classify search strategy (spatial/serial/random/unclear) per Gawel et al. 2019 (citing O'Leary & Brown), with a human-review override on the verdict itself
• Visualize the trial three ways alongside the frame viewer: a path plot coloured by elapsed time, an occupancy heat map, and a per-hole visit raster. All three use viridis (colourblind-safe, survives greyscale printing) and export a 2048px PNG redrawn at that size, not upscaled
• Correct any automatic call by hand at multiple levels (ROI, event classification, search-strategy verdict) - corrections survive a reload. A free-text note on any event, independent of the correction itself, for anything worth explaining that doesn't fit a dropdown - carried through to the project file and the event-detail export
• Export a tidy one-row-per-trial summary and a per-event detail sheet, as both CSV and XLSX, with field selection, audited-value labeling, and the exact parameters/tool version that produced them
• Reload a saved project file to re-run analysis on a trial without re-tracking

How to run it

Just use it: open https://barnes-maze-pipeline.pages.dev. Nothing to install, no account, no build step. This is how the intended user runs it.

Run it yourself from source: clone the repo and open app/index.html directly in a recent Chrome, Edge, or Firefox. No server, no npm install, no build step. Confirmed directly (not assumed): both WebCodecs and IndexedDB work correctly when index.html is opened straight from disk (file://), so double-clicking the file is genuinely sufficient, not just a simplification.

Sample videos are not committed here (see below). Run ./scripts/get-sample-data.sh to fetch the three clips into data/barnes-maze/, or use the Load an example button in the app, which needs nothing local at all.

Browser support: WebCodecs is required - the app checks on load and tells you if your browser doesn't have it. Confirmed working end to end in Firefox (load, ROI confirm, track, detect events, export): noticeably slower than Chrome, but every step completed correctly, including tracking/event-detection producing frame-for-frame matching results to the Chrome run. Edge (also Chromium-based, same engine as this app's primary testing) has not been separately spot-checked.

Example outputs

examples/ has the real, actually-generated outputs for all three sample clips, one subfolder per clip - nothing hand-written - so they can be read without re-running the pipeline. Ten files per clip: the reloadable .bmp.json project file, per-event detail (CSV + XLSX), the path/heat map/raster PNGs, and the trial summary (CSV + XLSX) twice - once as exported by default, once with the _labeled_audited suffix, which has audited-value labeling turned on so every value a human override touched reads "(audited)" in the cell itself. Deliberately named to look different at a glance, not like duplicate uploads. The .bmp.json format itself is documented in docs/PROJECT_FILE_FORMAT.md.

What I deliberately didn't build

The larger items are tracked as GitHub Issues.

A short, curated list - see KNOWN_LIMITATIONS.md for the full detail and reasoning behind each:

• Point-level trajectory correction (checked for real evidence of need across all three sample clips, found none)
• Cohort-scale analysis: learning curves across days and cohort comparisons. The brief asks for both by name. They need a second trial-level tool that merges many exports, not another panel in a single-trial app, so the honest version is a companion tool - designed in full at docs/COHORT_AGGREGATOR_DESIGN.md, deliberately not built this cycle. The per-trial exports are already shaped to feed it.
• Visible, adjustable trajectory smoothing/gap-filling as its own step (failed frames are honestly marked and never silently interpolated, which is the harder and more important half of this - but no optional smoothing pass exists on top of that)
• A confirmation step + timestamped audit log for overrides
• A publication-formatted (APA-style) table/chart export button

What leaves your machine

Nothing you generate leaves your machine. This app is entirely client-side - video decoding (WebCodecs), computer vision (OpenCV.js), and all analysis run in your browser. No video, frame, or measurement is ever uploaded anywhere. Session state (ROI corrections, thresholds, human-review overrides, tracked results) is stored locally in your browser's IndexedDB, never sent anywhere. This matters beyond convenience: footage of animal subjects is IACUC-governed and typically bound by institutional data-handling policy, and a tool that never transmits it off the machine that recorded it sidesteps that concern entirely rather than needing to justify a transfer.

Three requests do leave the machine, and that is the complete list. Loading the page fetches OpenCV.js and mp4box.js from their public CDNs (docs.opencv.org and jsdelivr.net), the same as any web page loading a script from a CDN. Clicking Load an example fetches test50.mp4 from the sample-data repository on GitHub (raw.githubusercontent.com) - only if you click it, and it is a download, not an upload. No data of yours - not your video, not a filename, nothing you've entered - is sent to any of the three.

Keys and cost: none. No API keys, no paid services, no login to provision. It's a static page, so cost doesn't scale with use: an institution's marginal cost for the tenth lab, or the thousandth video, is the same as for the first, because the computation happens on the machine that already belongs to the person doing the work. Hosting a static site is effectively free at any scale this would reach.

Accessibility

Keyboard-operable: all text inputs, dropdowns, and flag buttons; the target-hole dropdown works as a keyboard alternative to clicking the canvas; the frame controls (first/prev/next/±1s/jump-to-frame); the Raster visualization, which takes arrow keys to move between visits and Enter to seek there; and moving/resizing a platform or hole circle - select it from a dropdown next to the drag instructions, then arrow keys move it and +/- resizes it (Shift for bigger steps), no dragging required.

Click-to-seek on the Path plot, the Heat map, and the tracking-failure timeline is mouse-only, but each of those is a shortcut to a frame the keyboard frame controls can reach anyway.

No colour-only meaning: status messages use a border style and a symbol (⚠/✓) in addition to colour; the target hole is shape-distinguished on the canvas; raster marks are shaped by event type with the target row starred; tracked and failed frames are a crosshair and an X, not two colours.

200% browser zoom: checked on the current three-column layout (2026-09-06) - tab order intact, no clipping or overlap, reflows cleanly, and the layout drops to a single column below 1100px wide.

AI-assisted development and process

Built with Claude Code. Tools, config, and specific moments where the model was wrong and was overridden: docs/process/AI_NOTES.md. The full decision log - every wrong turn kept visible, not just the approaches that worked - is docs/process/DECISIONS.md. Two independent audits of the code are in docs/process/audits/, and each fix has a regression test in tests/ (npm test).

Known limitations

See KNOWN_LIMITATIONS.md.

License

MIT - see LICENSE.

Credits and sample data

• Sample videos (test50, test51, test53) were recorded at the Salk Institute in 2024 and published in https://github.com/salk-airc/rse-takehome-2026 under data/barnes-maze/. They are not committed here. Run ./scripts/get-sample-data.sh to fetch them locally. The images in examples/ are outputs generated from those clips.
• Frame-accurate seeking is adapted from the video-player tool in talmolab/vibes (BSD-3-Clause); see THIRD_PARTY.md.
• Measure definitions and search-strategy classes follow Gawel et al. 2019 and Illouz et al. 2020.