FlyRun: Drosophila-inspired navigation experiments

FlyRun combines a browser runner with hand-written avoidance rules and a sparse, reward-modulated associative controller. Separate Python experiments explore Leaky Integrate-and-Fire (LIF) neurons, contrast-transient escape responses, Hassenstein–Reichardt motion correlators, and three-factor STDP.

This is a bio-inspired prototype, not a reconstruction of a measured fly connectome. Gameplay does not establish biological fidelity, learning benefits, a sub-millisecond end-to-end deadline, or measured low-power operation.

What runs where

Component	Implemented behavior	Limits
3D Fly Character	Canonical Google DeepMind / HHMI Janelia flybody meshes (honey chitin, macrochaetae bristles, vermillion faceted compound eyes, striped abdominal tergites, claws, translucent veined wings)	Decimated for high-performance WebGL 60 FPS; kinematic wing flutter and anti-phase haltere stabilization
Neural Observatory HUD	2,400-soma MaleCNS v1.0 anatomical layout with Dual-State Sparsity, live 5-channel spike raster, and calcium fluorescence decay	Illustrative HUD reflection of controller state; not an emulated 166k-neuron connectome
Browser autopilot	Exact game-state features → 24 inputs → up to 9 active features among 128 KC-like units → four action preferences, combined with avoidance rules	Continuous activations, not a spiking network; rules use obstacle identity and distance directly
Browser learning	Reward × action eligibility × KC-like activation; one-second eligibility decay in simulated time	Not spike-timing-dependent plasticity; improvement has not been demonstrated against a frozen-learning baseline
Python circuits	Independent LIF, STDP and motion/contrast experiments	Hand-built connectivity; not connected to browser actuation
Python server	Static files and WebSocket incident logging	Connecting it does not enable a Python neural controller
Data exporter	NeuPrint extraction or explicitly requested synthetic examples, with provenance manifests	Exported graphs are not loaded by the current circuits; measured counts do not validate assumed effective weights

Quick Start

git clone https://github.com/gordonyfg/FlyRun.git
cd FlyRun

uv venv --python 3.12 .venv
source .venv/bin/activate
uv pip install -e .
python scripts/run_game_bridge.py

Open http://localhost:8080. Autopilot starts enabled; use the mode button for manual play (arrows/WASD). The optional logger writes incident files under data/. The HUD's controller compute time is measured locally and excludes rendering, physics, input waiting and transport. Control runs once per animation frame.

For a fresh, seeded run use http://localhost:8080/?seed=42&fresh=1; add &learning=off for a frozen-weight comparison. Seeded/fresh runs do not read or write saved sessions. The seed controls obstacle generation, not rendering or frame timing. Replaying the same initial seed is necessary but not sufficient for identical trajectories across frame rates.

🧠 Educational Guide: The Biology Behind FlyRun

1. Why a Small-Scale Model? (166k Whole Brain vs. Real-Time Micro-Circuit)

A biological adult Drosophila melanogaster brain contains approximately 139,255 to 166,606 neurons and more than 50 million synapses (as mapped in the Cambridge/Janelia MaleCNS v1.0 and FlyWire whole-brain connectomes).

• Simulating a full-scale connectome with biophysically realistic Hodgkin–Huxley or multi-compartment Leaky Integrate-and-Fire (LIF) equations requires supercomputer clusters or multi-GPU workstations, running thousands of times slower than real time.
• FlyRun's design philosophy: Instead of a heavy offline simulation, FlyRun isolates the exact behavioral sub-circuit that insects use for rapid 3D obstacle avoidance, optic flow stabilization, and associative learning.
• The 3D Neural Observatory visualizes 2,400 representative somas mapped to authentic stereotaxic anatomical coordinates, while the browser execution loop runs a compact 128-Kenyon-cell network on pure client CPU in under 0.05 milliseconds per frame. It requires zero GPUs, no cloud servers, and runs smoothly at 60 FPS on any laptop or smartphone.

2. What Brain Regions Are Activated & Why?

In nature, flies react to visual threats within 30 to 50 milliseconds (faster than human conscious perception). FlyRun models the primary biological pathways that make this possible:

                          VISUAL INPUT (Corridor & Hazards)
                                        │
             ┌──────────────────────────┴──────────────────────────┐
             ▼                                                     ▼
     LOBULA / MEDULLA (LC4)                               OPTOMOTOR (T4 / T5)
  (Exponential looming detector)                         (Wide-field motion flow)
             │                                                     │
             ▼                                                     ▼
   MUSHROOM BODY (24 PNs)                                CENTRAL COMPLEX (CX)
(Multi-hazard sensory encoding)                      (Steering compass / heading)
             │                                                     │
             ▼                                                     │
    128 KENYON CELLS (KCs)                                         │
 (Sparse expansion, ~7% active)                                    │
             │                                                     │
             ▼                                                     │
     4 MBONs + DOPAMINE (DANs)                                     │
(Learned action valences + reward)                                 │
             │                                                     │
             └──────────────────────────┬──────────────────────────┘
                                        ▼
                           DESCENDING NEURONS / VNC
                      (Jump, Slide, Bank-Left, Bank-Right)

1. Optic Lobes — LC4 (Lobula Columellar Neuron 4):
   • Role: The fly's canonical looming collision detector.
   • Why: Fires with exponential acceleration as an obstacle grows larger on the retina. If an impending collision crosses threshold, it triggers rapid steering bias or emergency braking.
2. Optic Lobes — T4/T5 Direction-Selective Neurons:
   • Role: Detects wide-field optic flow (motion energy from side walls and pillars).
   • Why: Provides biological optomotor feedback to center the fly in the corridor and regulate flight velocity.
3. Central Complex (CX — Fan-shaped Body & Ellipsoid Body):
   • Role: The insect's internal navigational compass and motor steering hub.
   • Why: Arbitrates lateral heading and converts sensorimotor discrepancies into banked lane-switching commands.
4. Mushroom Body (MB — Associative Plasticity & Memory):
   • Role: The adaptive learning center of the insect brain.
   • Why: Combines 24 multi-sensory Projection Neurons (PNs) into 128 Kenyon Cells (KCs). Using lateral feedback inhibition, only the top ~7% (up to 9 units) fire at any moment (Dual-State Sparsity). This sparse code projects onto 4 Mushroom Body Output Neurons (MBONs: Left, Right, Jump, Slide) modulated by Dopaminergic Neurons (DANs).
5. Giant Fiber (GF) & Thoracic VNC:
   • Role: The ballistic escape trigger down the ventral nerve cord.
   • Why: Directly excites thoracic motor units to actuate ballistic jumps (tergotrochanteral muscle TTM) or crouching slides.

3. Why Is This Appropriate for a Temple Runner?

Insects evolved to navigate high-density clutter (dense vegetation, moving branches, and swooping predators) at high forward velocity:

• Low Hurdles map to terrestrial obstacles requiring a ballistic jump.
• Overhead Spikes / Arches map to low-hanging canopy requiring tucked aerodynamic sliding.
• Tall Monoliths map to tree trunks and vertical boundaries requiring optic-flow-guided lateral evasion.
• The game loop mirrors the insect's real-life sensorimotor loop: continuous optic flow interspersed with discrete high-stakes collision threats that demand instantaneous motor decisions.

4. How Training Runs: Reward, Damage, and Plasticity

FlyRun does not use offline backpropagation or deep reinforcement learning. Instead, it uses Three-Factor Hebbian Plasticity with decaying eligibility traces, matching real dopamine-modulated synaptic transmission:

1. Eligibility Traces ($e$):
   • When Kenyon Cells fire in response to a hazard, an intracellular molecular trace is tagged in the synapse: $$e \leftarrow e \cdot \exp\left(-\frac{\Delta t}{1.0,\text{s}}\right)$$
   • Only the motor action selected by the arbiter tags its corresponding synapse row with $+0.35 \times \text{KC activity}$.
2. Clearing an Obstacle (+Reward):
   • Successfully crossing an obstacle without collision delivers a positive dopamine burst ($\text{Reward} = +1.0$ to $+2.0$).
   • Synapses tagged with active eligibility are reinforced: $$W \leftarrow \text{clip}(W + 0.08 \times \text{Reward} \times e,, 0.02,, 1.95)$$
   • The synaptic weight $W$ rises above its baseline ($0.350 \to 0.351 \to 0.352\dots$). Over dozens of successful clears, the fly learns strong associative preferences for the correct maneuver in that hazard context.
3. Collisions (-Damage & Dopamine Dip):
   • Colliding with an obstacle inflicts mechanical structural damage, reducing the fly's Integrity (e.g., $100% \to 60% \to 20%$).
   • A negative reinforcement signal is applied, causing long-term synaptic depression (LTD) on the misfiring action and preventing repeated mistakes.
4. What to Expect During a Run:
   • Initial phase (0–30 obstacles): The fly relies heavily on hand-crafted avoidance heuristics; synaptic weights remain near $W \approx 0.350$.
   • Trained phase (50–100+ obstacles): Active KC combinations become strongly associated with winning motor outputs. Jump and slide actions trigger with tighter, more confident timing, minimizing emergency braking events.

5. What Are the Brain's Capabilities & Limits?

Capabilities (What it excels at)	Boundaries (What it does not do)
Ultra-fast inference: Executes in $< 0.05,\text{ms}$ on CPU	No deep reasoning: Cannot plan complex multi-step puzzle sequences
Sample-efficient: Learns associations in $10\text{--}50$ trials	Short horizon: Operates on a $1\text{--}2$ second associative window
No catastrophic forgetting: Orthogonal sparse codes protect old memories	Not a full connectome: A targeted functional circuit, not 166k biophysical cells
Zero GPU / zero cloud: Runs anywhere, 100% client-side	No language/abstract cognition: Strictly sensorimotor motor control

6. What Is a "Win" and What Is a "Lose"?

• Win (Survival & Mastery):
  • Distance Milestone: Flying $> 1,000,\text{meters}$ without fatal structural damage.
  • Clean Clearance Streak: Clearing $50+$ obstacles consecutively without clipping.
  • High Velocity Maintenance: Cruising at top flight speed ($24.0,\text{m/s}$) with minimal emergency deceleration.
  • Synaptic Convergence: Observing the MB Synapse weight readout stabilize at specialized values, proving learned competence.
• Lose (Structural Breakdown):
  • Integrity Drops to 0%: Taking repeated collision damage destroys the exoskeleton and terminates the flight episode.
  • Black Box Trigger: A crash triggers an emergency telemetry capture, writing the final 5 seconds of pre-crash sensorimotor states to the flight recorder for post-mortem analysis.

Validate

pytest tests/ -v
node --test tests/test_browser.cjs
python scripts/run_mvp_looming.py
python benchmarks/benchmark_latency.py --steps 5000 --threads 1 --seed 0

The benchmark emits hardware/software settings, median/p95/p99/max and the fraction of steps missing a 1 ms target. Add --require-deadline to fail on any observed miss. Passing a finite benchmark does not guarantee future deadlines. The MVP tests an expanding synthetic disk, not successful physical navigation.

The escape circuit removes additive whole-field brightness shifts before integrating OFF contrast. It remains a contrast heuristic: selectivity against translation, patterned lighting, and other non-looming stimuli is not established. The Python KC model applies an explicit top-k spike cap alongside feedback; this engineering constraint is not a validated biological APL model.

Data provenance

export NEUPRINT_APPLICATION_CREDENTIALS="<YOUR_TOKEN>"
python data/export_subgraphs.py --dataset hemibrain:v1.2.1 --output-dir data/live
# Or explicitly generate unvalidated examples in a separate directory:
python data/export_subgraphs.py --force-synthetic --output-dir data/synthetic

Live extraction fails on missing credentials, query errors, empty graphs or unknown neurotransmitter polarity. It never substitutes synthetic data. Dataset coverage, neuron names and neurotransmitter fields must match the chosen server. A manifest records source, dataset and file hashes; verify hashes when consuming an export. Existing bundled Parquet files have unverified provenance and are not biological ground truth. No FlyWire-specific importer is implemented.

Performance and energy scope

CSR multiplication processes stored graph edges, not just synapses whose source neuron fired. LIF integration also visits the population. A simulated 1 ms step is not a 1 kHz wall-clock scheduler. Small-kernel CPU medians can be below 1 ms; no portable latency guarantee or edge CPU/GPU comparison is established.

EnergyEstimator sums assumed energy per modeled synaptic event and neuron update, then divides by simulated duration. Its output is a hypothetical operation-energy model, not measured CPU/GPU power. It omits rendering, memory, software overhead, idle power and other operations. The MVP counts LC4→GF events only; it does not account for all retinal processing.

Next validation work

Compare learning enabled and frozen on multiple held-out obstacle seeds, using the same initial weights. Report collision rate, damage per distance, survival distance and speed at several frame rates. Measure actual device energy before making energy-efficiency claims. Validate circuit selectivity with translation, recession, illumination and expansion controls, and document anatomical sources before claiming connectome-derived wiring.

See ARCHITECTURE.md for model equations and implementation boundaries.