Honesty pass: reproducible forecast claims, latent interface, complete [all]

Address four credibility/honesty findings from a world-model researcher's
usability test, keeping the small-&-beautiful tone and the legitimate
encoder/world-model framing.

- Forecast (卡点 5): drop the unreproducible "−35%" learned-vs-baseline number
  from the README hero caption and the forecast README (the training footage is
  not redistributed, only the checkpoint is). Reframe quick_forecast.py as the
  reproducible no-footage demo (it prints honest baseline-vs-baseline numbers
  from committed code) and clearly mark the real-video result as needing the
  user's own clip, with the export → train → render path documented.
- Latent channel (卡点 6): reconcile the README's present-tense latent claims
  with DESIGN.md — the latent vec is a shipped, serializable interface; the
  V-JEPA/ReID producers are roadmap. Add examples/latent_vec.py demonstrating
  the channel is usable today by attaching your own embedding (track.user["vec"]
  → WorldState entity.vec → serialize → round-trip), numpy-only.
- [all] extra (卡点 7): include the grounding deps (transformers/torch/pillow)
  so [all] genuinely installs every optional adapter, matching the README.
- Examples (卡点 8): note that examples ship with the source tree (git clone),
  not the installed wheel.

ruff clean; 92 passed / 1 skipped; both demos verified numpy-only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: raullenchai

CHANGELOG.md

@@ -8,6 +8,10 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ### Added
 
+- Example `examples/latent_vec.py`: populate the dual-state latent `vec` channel by
+  hand (attach your own embedding → `WorldState` entity → serialize → round-trip),
+  runnable with numpy only — shows the latent interface is usable before the
+  built-in producers ship.
 - `Detection.from_supervision(detections, class_names=None)`: ingest a Roboflow
   Supervision `sv.Detections` into `list[Detection]` by duck-typing (no
   `supervision` import), so Supervision users plug straight into Retina's event
@@ -21,6 +25,19 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
   webhook) and `examples/from_supervision.py` (ingest a Roboflow `sv.Detections`
   pipeline), both runnable with no model / GPU / network on synthetic input.
 
+### Changed
+
+- `[all]` extra now installs every optional adapter's deps, including `grounding`
+  (transformers + torch + pillow), so it is genuinely "everything" as the README says.
+- Honesty pass on forecast claims: dropped the unreproducible "−35%" learned-vs-baseline
+  number from the README hero caption and the forecast README (the training footage
+  isn't redistributed). `quick_forecast.py` is documented as the reproducible,
+  no-footage demo; the real-video result is clearly marked as needing your own clip.
+- README latent-channel wording reconciled with `DESIGN.md`: the latent `vec` is a
+  shipped, serializable *interface*; the automatic V-JEPA / ReID *producers* are
+  roadmap, not shipped.
+- Noted that examples ship with the source tree (`git clone`), not the installed wheel.
+
 ## [0.1.0] — 2026-06-17
 
 First public open-source release.

README.md

@@ -1,6 +1,6 @@
 # Trio Retina
 
-**Turn any perception model's output into one standard, queryable world-state — symbolic events + latent vectors.**
+**Turn any perception model's output into one standard, queryable world-state — symbolic events, with a latent-vector channel built in.**
 The model-agnostic state layer for world models.
 
 *A lightweight, model-agnostic **computer-vision pipeline** for **object detection & tracking** that emits structured **events** — zone intrusion, line-crossing, dwell, people-counting — from **YOLO**, **VLM**, or **Grounding DINO** detectors over video, files, or **RTSP**. Runs on CPU at the **edge**; feeds **digital twins**, dynamics models, and LLMs.*
@@ -14,11 +14,11 @@ The model-agnostic state layer for world models.
 
 ![Trio Retina computer-vision pipeline: YOLO object tracking with two dynamics models forecasting entity trajectories from one world-state](https://raw.githubusercontent.com/machinefi/trio-retina/main/media/retina_demo.gif)
 
-> One world-state from any detector → **two dynamics models forecast where each entity is headed** off the *same* state (gray = constant-velocity, magenta = learned, −35%). Swap the detector (YOLO → V-JEPA → DINO) or the dynamics model — the state in the middle is the constant.
+> One world-state from any detector → **two dynamics models forecast where each entity is headed** off the *same* state (gray = constant-velocity baseline, magenta = a learned model). Swap the detector (YOLO → V-JEPA → DINO) or the dynamics model — the state in the middle is the constant.
 
 ## 👋 hello
 
-**Trio Retina** (Retina for short) turns raw signals — video, sensor — into a **queryable world-state**: readable **events** (`zone.enter`, `dwell`, `line.cross`) *plus* optional **latent** vectors, on one small model-agnostic standard. Bring any model (YOLO, V-JEPA, DINO, a VLM, or none); Retina assembles its output into state a dynamics model, rule engine, or LLM can consume.
+**Trio Retina** (Retina for short) turns raw signals — video, sensor — into a **queryable world-state**: readable **events** (`zone.enter`, `dwell`, `line.cross`) *plus* a standardized **latent** `vec` channel on the same records, on one small model-agnostic standard. The latent channel is a real, serializable interface today (attach your own embedding — see [`examples/latent_vec.py`](examples/latent_vec.py)); the automatic *producers* (V-JEPA scene + per-object ReID) are on the [roadmap](#-roadmap). Bring any model (YOLO, V-JEPA, DINO, a VLM, or none); Retina assembles its output into state a dynamics model, rule engine, or LLM can consume.
 
 Think **OpenTelemetry for perception** — it doesn't build the sensors, it normalizes any of them into one state. In world-model terms it's the **encoder** (`s = Enc(x)`), and *only* the encoder; dynamics and policy build on top. → see [`DESIGN.md`](DESIGN.md).
 
@@ -52,7 +52,7 @@ for event in cam.run(video_frames("dock.mp4")):
     #  "label":"person","zone":"dock","dur":31.0,"conf":0.91}
 ```
 
-No model, no GPU? The [`examples/`](examples/) quickstarts run on synthetic detections — start with `python examples/quickstart.py` (the forecast / video demos need `[video]` + a clip).
+No model, no GPU? The [`examples/`](examples/) quickstarts run on synthetic detections — `git clone` the repo (they ship with the source, not the wheel) and start with `python examples/quickstart.py` (the forecast / video demos need `[video]` + a clip).
 
 ### compose models with `|`
 
@@ -134,7 +134,7 @@ The hero GIF above. [`examples/forecast/`](examples/forecast/) runs a dynamics m
 <details>
 <summary>All examples</summary>
 
-The top-level quickstarts run with **no model and no GPU** (synthetic detections):
+The examples live in this repo (not in the installed wheel) — `git clone` to run them. The top-level quickstarts run with **no model and no GPU** (synthetic detections):
 
 ```bash
 python examples/quickstart.py          # zone / line / count / dwell events
@@ -144,6 +144,7 @@ python examples/gate_savings.py        # a cheap gate cuts detector calls 100 ->
 python examples/pipeline_compose.py    # compose with | (n8n without a GUI)
 python examples/rtsp_to_webhook.py     # camera -> restricted-zone alert -> webhook
 python examples/from_supervision.py    # ingest a Roboflow sv.Detections pipeline
+python examples/latent_vec.py          # populate the latent vec channel by hand
 ```
 
 Real-footage / dynamics demos need a clip and the extras — `pip install 'trio-retina[all]'`:
@@ -189,7 +190,7 @@ Everything flows through one append-only data unit, the **`Frame`**. Each stage
 
 **Two senses of "encoder."** Foundation backbones (V-JEPA, DINO, SAM, YOLO) turn pixels into features — that race is theirs, and Retina rides it. Retina is the encoder *layer* on top: it **fuses** many models into one record, gives objects **persistent identity**, **structures** it into entities + relations + events, carries the **dual** symbolic + latent channels, as an **event-sourced stream** — one small, serializable, model-agnostic standard.
 
-**Dual state.** The same entities on two linked channels: *symbolic* (readable `events` / entity records, for rules / LLMs / dashboards) and *latent* (optional model-tagged embeddings, for a downstream dynamics model). Symbols you can read; vectors a model can predict on.
+**Dual state.** The same entities on two linked channels: *symbolic* (readable `events` / entity records, for rules / LLMs / dashboards) and *latent* (optional model-tagged embeddings, for a downstream dynamics model). Symbols you can read; vectors a model can predict on. The latent channel is a standardized, serializable interface shipping today — you can populate `entity.vec` with your own embedding now ([`examples/latent_vec.py`](examples/latent_vec.py)); the built-in *producers* (V-JEPA / ReID) are on the roadmap, not shipped yet.
 
 **vs DeepStream / Holoscan** — same good ideas (event semantics, metadata model, composable graph), none of the weight:
 

examples/forecast/README.md

@@ -9,26 +9,42 @@ pixels, so Retina is the interface between any backbone and any forecaster.
 behind one tiny `DynamicsModel` protocol (`observe` / `predict`):
 
 - `LinearForecaster` — constant-velocity baseline (gray arrow).
-- `LearnedForecaster` — a small trained MLP (magenta arrow), **−35 %** centroid
-  error vs the baseline on held-out entities.
+- `LearnedForecaster` — a small trained MLP (magenta arrow), which *can* anticipate
+  turns / slow-downs a constant-velocity model misses.
 
 Swap the detector (YOLO → V-JEPA → DINO) or the forecaster — the state in the
 middle is the constant.
 
-## Run it
+## Run it — the reproducible demo (no model, no GPU, no footage)
+
+```bash
+python examples/forecast/quick_forecast.py
+```
+
+This is the runnable evidence: it builds a synthetic `WorldState` stream, forecasts
+a few frames ahead, and prints the real baseline-vs-baseline comparison — a
+constant-velocity model vs a no-motion model — so you can see the WorldState is
+genuinely dynamics-ready. Whatever the numbers are, they come straight from
+committed code.
 
 ```bash
-python examples/forecast/quick_forecast.py     # synthetic — proves the loop, no model/GPU
-python examples/forecast/forecast_video.py v.mp4   # real-video constant-velocity baseline
 python examples/forecast/multi_consumer.py     # one WorldState -> rules + forecast + an LLM-judge stub
 ```
 
-Reproduce the trained model and the annotated GIF:
+### The headline real-video result needs your own footage
+
+The learned-vs-baseline comparison and the annotated GIF were measured on real
+traffic video, which we don't redistribute — so that specific result isn't
+reproducible from this repo alone (only the trained checkpoint is committed). A
+learned MLP does **not** automatically beat constant velocity; `train_dynamics.py`
+prints the honest win-or-tie on *your* data. To regenerate it end-to-end on a clip
+of your own:
 
 ```bash
-python examples/forecast/export_trajectories.py v.mp4   # WorldState tracks -> JSON
-python examples/forecast/train_dynamics.py traj.json dynamics.ckpt   # train the MLP (torch)
-python examples/forecast/render_demo.py v.mp4 out.mp4   # the two-arrow demo video
+python examples/forecast/export_trajectories.py v.mp4 traj.json   # WorldState tracks -> JSON ([video]+[yolo])
+python examples/forecast/train_dynamics.py traj.json dynamics.ckpt   # train + report baseline vs learned (torch)
+python examples/forecast/forecast_video.py v.mp4                  # real-video constant-velocity baseline
+python examples/forecast/render_demo.py v.mp4 out.mp4             # the two-arrow demo video
 ```
 
 ## Files
@@ -40,7 +56,7 @@ python examples/forecast/render_demo.py v.mp4 out.mp4   # the two-arrow demo vid
 | `export_trajectories.py` | Run Retina (YOLO) → per-entity centroid tracks → JSON. |
 | `render_demo.py` | Render the annotated two-arrow demo video. |
 | `quick_forecast.py` · `forecast_video.py` · `multi_consumer.py` | Synthetic proof · real-video baseline · one-state-many-consumers. |
-| `dynamics.ckpt` | The trained MLP weights (committed, so the GIF is reproducible). |
+| `dynamics.ckpt` | The trained MLP weights from the demo footage (committed for the GIF; retrain on your own clip with `train_dynamics.py`). |
 
 ## What this is (and isn't)
 

examples/latent_vec.py

@@ -0,0 +1,107 @@
+"""The latent channel, populated by hand — runs with NO model and NO GPU.
+
+Retina's state is **dual**: every entity carries a readable *symbolic* core AND an
+optional model-tagged *latent* `vec` on the *same* record (see DESIGN.md / SPEC.md
+"The latent channel"). The `Vec` type and the `entity.vec` slot are a real,
+serializable interface today — what's on the roadmap is the *producers* that fill
+it automatically (V-JEPA scene latents, per-object ReID). This example shows the
+interface is usable right now by attaching your **own** embedding:
+
+    track.user["vec"] = Vec(...).to_dict()   # any embedding you already have
+        → flows into WorldState  →  entity.vec  →  serialize  →  round-trips
+
+So if you have an embedder (a frozen backbone, a ReID head, anything), you can ride
+the latent channel today; you don't have to wait for the shipped producers.
+
+    python examples/latent_vec.py
+"""
+
+import numpy as np
+
+from retina import IoUTracker, Vec, WorldState
+from retina.detect import Detection
+from retina.nodes import DetectorNode, TrackerNode
+from retina.pipeline import Pipeline
+
+
+class Walker:
+    """One 'person' walking across the frame — one detection per call."""
+
+    def __init__(self):
+        self.f = 0
+
+    def __call__(self, image):
+        x, y = 60 + self.f * 8, 200 + self.f * 4
+        self.f += 1
+        return [Detection("person", (x - 25, y - 25, x + 25, y + 25), 0.9)]
+
+
+def my_embedding(track) -> Vec:
+    """Stand in for a real producer (ReID head / frozen V-JEPA ROI). Here we just
+    hash the track's box into a tiny deterministic vector — the point is the
+    *channel*, not the model. Swap this for your own embedder and nothing else
+    changes."""
+    x1, y1, x2, y2 = track.bbox
+    rng = np.random.default_rng(int(x1 + y1) % 2**32)
+    values = rng.standard_normal(8).round(3).tolist()
+    return Vec(model="demo-reid/v0", dim=8, values=values)
+
+
+def main() -> None:
+    pipe = Pipeline(
+        [DetectorNode(Walker()), TrackerNode(IoUTracker(min_hits=2))], source_id="cam"
+    )
+
+    entity = None
+    for i in range(4):
+        frame = pipe.process(np.zeros((400, 640, 3), np.uint8), float(i))
+        # Attach YOUR embedding to each track's open `user` slot. `WorldState`
+        # reads `track.user["vec"]` and carries it onto the entity automatically.
+        for trk in frame.tracks:
+            trk.user["vec"] = my_embedding(trk).to_dict()
+        ws = WorldState.from_frame(frame)
+        if ws.entities:
+            entity = ws.entities[-1]
+
+    assert entity is not None, "tracker should have confirmed the walker"
+    print("entity (symbolic core + latent vec):")
+    print(f"  id={entity.id}  type={entity.type}  bbox={tuple(round(v) for v in entity.bbox)}")
+    print(f"  vec.model={entity.vec.model}  vec.dim={entity.vec.dim}")
+    print(f"  vec.values={entity.vec.values}\n")
+
+    # The dual state serializes as one record — symbol + latent, never collapsed —
+    # and round-trips losslessly through JSON.
+    blob = ws.to_json()
+    print(f"serialized WorldState ({len(blob)} bytes):")
+    print(f"  {blob}\n")
+
+    rt = WorldState(**_load_entities(blob))
+    e2 = rt.entities[-1]
+    assert e2.vec is not None and e2.vec.values == entity.vec.values
+    print("round-trip ok: the latent vec survived serialize → parse, attached to its entity.")
+
+
+def _load_entities(blob: str) -> dict:
+    """Rebuild a WorldState from its JSON, reconstructing nested Entity/Vec."""
+    import json
+
+    from retina import Entity
+
+    d = json.loads(blob)
+    ents = []
+    for e in d.get("entities", []):
+        vec = Vec(**e["vec"]) if "vec" in e else None
+        ents.append(
+            Entity(
+                id=e["id"],
+                type=e["type"],
+                bbox=tuple(e["bbox"]) if "bbox" in e else None,
+                conf=e.get("conf"),
+                vec=vec,
+            )
+        )
+    return {"src": d["src"], "t": d["t"], "frame": d.get("frame"), "entities": ents}
+
+
+if __name__ == "__main__":
+    main()

pyproject.toml

@@ -39,7 +39,7 @@ yolo = ["ultralytics>=8.3"]
 norfair = ["norfair>=2.2"]
 grounding = ["transformers>=4.40", "torch>=2.0", "pillow>=10"]
 video = ["opencv-python>=4.8"]
-all = ["ultralytics>=8.3", "norfair>=2.2", "opencv-python>=4.8"]
+all = ["ultralytics>=8.3", "norfair>=2.2", "opencv-python>=4.8", "transformers>=4.40", "torch>=2.0", "pillow>=10"]
 dev = ["pytest>=8", "ruff>=0.6", "norfair>=2.2"]
 docs = ["mkdocs-material>=9.5", "mkdocstrings[python]>=0.26"]