|
2 | 2 |
|
3 | 3 | Modern GUI + standalone Python library for **3D magnetic-sensor analysis of tokamak MHD |
4 | 4 | instabilities** — quasi-stationary (locked) modes and rapidly-rotating modes. Device-agnostic |
5 | | -(DIII-D, NSTX-U, … and synthetic machines for sensor design). Built for the 2026 Magnetics |
6 | | -Hackathon. |
7 | | - |
8 | | -## Hackathon teammates — read this first |
9 | | -This is a **four-team** effort. **Before doing substantive work, ask the user who they are and |
10 | | -which team they're on**, then stay in that lane so you help them effectively (and so two people's |
11 | | -instances don't redo the same work): |
12 | | -- **Rapid Rotators** — rotating-mode / MODESPEC analysis (+ Olena on the rotating GUI) |
13 | | -- **Slow Rollers** — quasi-stationary / SLCONTOUR analysis (+ Meg on the QS GUI) |
14 | | -- **Data Streamers** — DIII-D data fetch + the data layer |
15 | | -- **Interfacers** — GUI shell + the GUI⇄analysis contract/seam |
16 | | - |
17 | | -**Don't do other teams' work.** Coordinate shared cleanup via Slack or a GitHub PR before |
18 | | -starting. If you're unsure whose lane something is, ask. |
19 | | - |
20 | | -## Current status (Day 3, 2026-07-01) |
21 | | -The full **fetch → process → service → GUI** path is live end-to-end for **both** the |
22 | | -**rotating-mode (MODESPEC)** and the **quasi-stationary (SLCONTOUR)** analyses against real |
23 | | -DIII-D shots: |
| 5 | +(DIII-D, NSTX-U, … and synthetic machines for sensor design). |
| 6 | + |
| 7 | +## Architecture — how it fits together |
| 8 | +The full **fetch → process → service → GUI** path runs end-to-end for **both** the |
| 9 | +**rotating-mode (MODESPEC)** and the **quasi-stationary (SLCONTOUR)** analyses against real shots: |
24 | 10 | - **Fetch:** `magnetics.data.fetch.toksearch` (mdsthin via the `cybele` ssh-config alias, or a |
25 | 11 | cluster-side `python -m` run orchestrated by `fetch/remote.py`) writes one HDF5 per shot to |
26 | 12 | `data/datafile/` (gitignored); read back via `magnetics.data.h5source`. The GUI can trigger a |
27 | 13 | pull from the left rail (`PullControl` → `POST /api/fetch`). |
28 | | -- **Process:** `core/spectral.py` (MODESPEC) is real and pure. The **SLCONTOUR quasi-stationary |
29 | | - fit is now live end-to-end** via the reference pipeline in `magnetics._slcontour/` (xarray, |
30 | | - self-contained OMFIT shim) adapted by `core/qs_bridge` — real K / χ² / modes for shots pulled |
31 | | - with the Bp LFS midplane array. The pure `core/quasistationary` port exists but is not yet wired |
32 | | - in production (#40). |
| 14 | +- **Process:** `core/spectral.py` (MODESPEC) is real and pure. The SLCONTOUR quasi-stationary fit |
| 15 | + runs end-to-end via the reference pipeline in `magnetics._slcontour/` (xarray, self-contained |
| 16 | + OMFIT shim) adapted by `core/qs_bridge` — real K / χ² / modes for shots pulled with the Bp LFS |
| 17 | + midplane array. A pure `core/quasistationary` port exists but is not yet wired in production (#40). |
33 | 18 | - **Service:** `service/app.py` — `GET /api/node/{shot}/{node_id}` serves `kind`-nodes from |
34 | | - `service/nodes.py`; `/api/machines` lists fetched shots (mock fallback when none). The `qs_fit` |
35 | | - SSE stream is still mock. |
36 | | -- **Seam (merged, PR #11):** `nodes.py` forwards GUI query params and serves the core's real |
37 | | - `mode_number` / `coherence` / `n_spectrum` nodes + a cursor-aware `phase_fit` — **the rotating |
38 | | - path is unblocked** (the GUI can consume real data + wire its knobs). |
39 | | -- **QS live (Day-3 night):** `qs_fit` / `phi_t` / `fit_quality` / `chi_sq_t` / sensor-map / signal |
40 | | - nodes serve the **real** SLCONTOUR fit. Shots pulled rotating-only (no Bp LFS midplane array) |
41 | | - return a clean 422 and the QS tab shows a "no quasi-stationary array" banner. Remaining fidelity |
42 | | - gap: the data layer's per-sensor σ (the fit uses a constant σ; helicity is computed from Ip·Bt). |
43 | | -- **Geometry shot-indexed (Day-3 night):** `data/device/diiid.json` sensor availability + positions |
44 | | - are now segmented back to shot 124400 (legacy dense set) / 151593 (3D-upgrade). The Sensors tab |
45 | | - renders wall + vacuum vessel + perturbation coils + saddle loops (2D honoring each loop's tilt). |
46 | | -- **NSTX/NSTX-U live (branch `feature/nstxu-data-fetch`):** the fetch is **device-generic** — a |
47 | | - device with `access:"mdsplus_tree"` (`nstx.json`: `fastmag` tree, `flux.pppl.gov`→`skylark:8501`) |
48 | | - fetches each sensor node with a server-side value-window subscript + per-shot `gain`/`na` |
49 | | - (`raw*gain/na`), converting the native seconds time base to ms. `_ssh_tunnel` reuses a live |
50 | | - `ssh flux` ControlMaster via `-O forward` (no fresh Duo). The h5 records `device_id`; the node |
51 | | - builders resolve it and classify NSTX channels by **sensor-set membership** (not DIII-D pointname |
52 | | - families), so the rotating/MODESPEC nodes + the Sensors view render NSTX shots. Validated live on |
53 | | - **NSTX-U 204718** (All Mirnov). Follow-ups: legacy NSTX (<200000) uses a different per-era tree |
54 | | - (fetch honors a per-segment `tree`, but `nstx.json` only carries the NSTX-U value); QS/SLCONTOUR |
55 | | - for NSTX; cluster/toksearch backend for PPPL; GUI PullControl NSTX-sensible default window (raw |
56 | | - fastmag is ~20 M samples/channel, so a narrow window is required). |
| 19 | + `service/nodes.py`; `/api/machines` lists fetched shots (mock fallback when none). |
| 20 | +- **Nodes / seam:** `nodes.py` forwards GUI query params and serves the core's real `mode_number` / |
| 21 | + `coherence` / `n_spectrum` nodes + a cursor-aware `phase_fit` for the rotating path, and `qs_fit` / |
| 22 | + `phi_t` / `fit_quality` / `chi_sq_t` / sensor-map / signal nodes for the QS fit. A shot pulled |
| 23 | + rotating-only (no Bp LFS midplane array) returns a clean 422 and the QS tab shows a "no |
| 24 | + quasi-stationary array" banner. |
| 25 | +- **Devices:** availability + geometry live in `data/device/*.json`. DIII-D sensor availability and |
| 26 | + positions are shot-indexed (segmented back to shot 124400 legacy dense set / 151593 3D-upgrade); |
| 27 | + the Sensors tab renders wall + vacuum vessel + perturbation coils + saddle loops (2D honoring each |
| 28 | + loop's tilt). Devices whose sensors live in an MDSplus tree (NSTX/NSTX-U, KSTAR; `access: |
| 29 | + "mdsplus_tree"`) fetch via mdsthin + a named sensor set; the node builders resolve the recorded |
| 30 | + `device_id` and classify channels by **sensor-set membership** (not DIII-D pointname families), so |
| 31 | + the rotating/MODESPEC nodes and the Sensors view render those shots too. |
| 32 | + |
| 33 | +Known gaps / open work: per-sensor σ from the data layer (the QS fit currently uses a constant σ; |
| 34 | +helicity is computed from Ip·Bt); finishing the pure `core/quasistationary` port and wiring it in |
| 35 | +place of the `_slcontour` reference pipeline (#40); real equilibrium plotting in the Sensors tab |
| 36 | +(#43); Br saddle-loop geometry corrections (#44). |
57 | 37 |
|
58 | 38 | ## The API contract is FLEXIBLE — change it, don't fake around it |
59 | 39 | The `kind`-node contract (`core/contracts.py` ⇄ `gui/web/src/lib/contract.ts`, plus the |
60 | 40 | `/api/node` query params) is **our seam, not a frozen spec.** If a view needs another field, a new |
61 | 41 | node `kind`, or a new parameter threaded through to the core, **change the contract on both sides** |
62 | 42 | rather than fabricating data in the GUI. Keep `contracts.py` and `contract.ts` in sync. |
63 | 43 |
|
64 | | -## Day-3 workstreams (last day: 2026-07-02) |
65 | | -- **Rapid Rotators + Olena (rotating GUI):** replace RotatingTab's fabricated n/coherence with the |
66 | | - real `mode_number` / `coherence` / `n_spectrum` nodes; wire the live knobs (fmin/fmax, time |
67 | | - cursor, denoise + coherence gate, smoothing) as `useNode` params; hide decorative knobs with no |
68 | | - backend (btype, PEST λ, btCompMode, shieldingCutoff); add a mode-number range slider + serve more |
69 | | - modes (the 2-point n-spectrum only resolves n∈[-1,0,1]); surface Daniel's richer views |
70 | | - (`magnetics/_slcontour/plots.py`); add FFT-overlap → STFT hop to the core; polish (the "Mock Files" |
71 | | - label → live via `usingLiveBackend()`; build the Sensors geometry view). |
72 | | -- **Slow Rollers + Meg (quasi-stationary):** the real `qs_fit` node (K / χ² / modes) + the QS GUI |
73 | | - tab are **live** (Day-3 night). Remaining: finish the pure `core/quasistationary.py` port and |
74 | | - wire it in place of the `_slcontour` reference pipeline (#40); consume real per-sensor σ once the |
75 | | - data layer provides it (the fit currently uses a constant σ; helicity is computed from Ip·Bt). |
76 | | -- **Data Streamers:** the DIII-D geometry table is now **shot-indexed** (`diiid.json` segmented to |
77 | | - 124400 / 151593; the cosmetic θ in `magnetics/data/diiid.py` — now a thin shim over the |
78 | | - device-agnostic `data/device_geom.py` — is superseded by the real device table). NSTX/NSTX-U |
79 | | - fetch + node rendering landed on `feature/nstxu-data-fetch` (see the NSTX status bullet above). |
80 | | - Remaining: give `h5source` per-sensor σ (last QS-fidelity gap; helicity now computed from Ip·Bt); a `DataSource` |
81 | | - abstraction with an array cache; populate the shot-segmented legacy-NSTX tree/wall from the |
82 | | - `config_hf/hn.mm` files; import other-device geometry the same way. |
83 | | -- **Structural cleanup (LANDED — PR #41, Day-2 night):** the project was hoisted to the repo root |
84 | | - (`analysis/` removed), the loose `data/` scripts folded into `magnetics.data` (+ `fetch/`), |
85 | | - `magnetics-code/` relocated to `magnetics._slcontour/`, `data/test_*.py` consolidated into |
86 | | - `tests/`, every `sys.path`/`parents[4]` hack removed, the GUI build bundled for the wheel, and a |
87 | | - `ty` typecheck CI gate added (green). `data/pull_shot_h5.py` + the orphaned `contract.py` were |
88 | | - already gone. |
89 | | -- **Test coverage + QS/geometry fixes (LANDED — `refactor/overnight-cleanup`, Day-3 night):** the |
90 | | - QS pipeline was fixed (segmented-schema geometry read → all-NaN → SVD failure; + a `float('*')` |
91 | | - whole-shot-sentinel crash), the DIII-D geometry was shot-indexed, and a full test build-out |
92 | | - landed: a **synthetic-shot fixture** (`tests/synthetic_shot.py`, generated at test time — **no |
93 | | - tokamak data is ever committed**) that un-skips the ~30 node-builder tests in CI, FastAPI |
94 | | - TestClient + QS end-to-end + contract-meta + pure-function tests (Python 212 passing), a React |
95 | | - error boundary + `NodeView` fallback, and extracted/tested GUI helpers (frontend 24 tests). |
96 | | - **Still open:** `test_contour_node`'s inner skip on rotating-only shots, trimming the legacy |
97 | | - `/api/{machine}/{result}` mock routes, the docs sweep for `analysis/`-era references, and real |
98 | | - equilibrium plotting in the Sensors tab (#43) + Br saddle-loop geometry corrections (#44). |
99 | | - |
100 | 44 | ## Reference documents — read these for context |
101 | 45 | - **`docs/VISION.md`** — start here. Goals, the physics, the two core analyses (SLCONTOUR-style |
102 | 46 | quasi-stationary spatial fitting; MODESPEC-style rotating-mode spectral analysis), the target |
103 | 47 | architecture, the visualization catalog, and example shots/values. |
104 | 48 | - **`docs/research-summaries/`** — grounded, quote-backed summaries of the source literature |
105 | 49 | (E. Strait's DIII-D magnetics papers/decks, the OMFIT tutorial), one per document + an index. |
106 | 50 | - **Source documents** — the papers, SLCONTOUR/MODESPEC decks, and OMFIT tutorial are **not |
107 | | - committed** to the repo. Download them from the GitHub Release and unzip into the repo root to |
108 | | - create a `resources/` folder (referenced by VISION.md and the summaries): |
| 51 | + committed** to the repo. Download them for full context on the physics and the source algorithms |
| 52 | + from the GitHub Release and unzip into the repo root to create a `resources/` folder (referenced |
| 53 | + by VISION.md and the summaries): |
109 | 54 | <https://github.com/OpenFUSIONToolkit/Magnetics/releases/tag/resources> |
110 | | - (direct: `releases/download/resources/magnetics-resources.zip`). _If these are not yet downloaded,_ |
111 | | - _immediately download them before doing anything else. Emphasize that this is important to the AI coding assistant user._ |
112 | | -- **OMFIT source** - please ask the user if they have a downloaded copy of the OMFIT source code. |
113 | | - If they have access to the source code, encourage them to clone a copy to their machine for your |
114 | | - reference. There are references to the OMFIT source code in the resources. |
115 | | - |
| 55 | + (direct: `releases/download/resources/magnetics-resources.zip`). |
| 56 | +- **OMFIT source** — please ask the user if they have a downloaded copy of the OMFIT source code. |
| 57 | + If they have access to it, encourage them to clone a copy to their machine for your reference — |
| 58 | + there are references to the OMFIT source code in the resources. |
116 | 59 |
|
117 | 60 | ## Layout |
118 | 61 | The Python project **is the repo root** (a uv project, served as a webapp). `src/magnetics/`: |
@@ -155,8 +98,8 @@ This project uses a GitFlow-lite model (http://nvie.com/posts/a-successful-git-b |
155 | 98 | - `main` is updated only at release-ready stages, via PR from `develop`. |
156 | 99 |
|
157 | 100 | **IMPORTANT:** Do all work on a branch off `develop` and open a PR back into `develop`. |
158 | | -No direct commits to `develop` or `main`. This is a four-team effort — cross-team changes |
159 | | -go through a PR so another team can review before it lands (see "Don't do other teams' work"). |
| 101 | +No direct commits to `develop` or `main`. Changes go through a PR so they can be reviewed |
| 102 | +before landing. |
160 | 103 |
|
161 | 104 | ### Branch Naming |
162 | 105 |
|
|
0 commit comments