Skip to content

Commit 6760754

Browse files
arunviswanathan91arunviswanathan91
committed
trivandrum night stramFLOW 2.0
1 parent 91e1c6a commit 6760754

33 files changed

Lines changed: 2274 additions & 292 deletions

‎.gitignore‎

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -82,8 +82,25 @@ test-assets/
8282
test-data/
8383
*.fcs
8484

85-
# ── Local Claude Code config (personal permissions + machine-specific paths) ──
85+
# ── AI assistant artifacts (never for the public repo) ───────────────────────
86+
# Local Claude Code config (personal permissions + machine-specific paths),
87+
# project instruction files, memories, and any other coding-assistant scratch.
8688
.claude/
89+
CLAUDE.md
90+
CLAUDE.local.md
91+
.claude.json
92+
*.local.md
93+
.cursor/
94+
.cursorrules
95+
.aider*
96+
.continue/
97+
.windsurf/
98+
.github/copilot-instructions.md
99+
# AI-generated planning / report scratch left at the repo root
100+
PLAN.md
101+
NOTES.md
102+
TODO.ai.md
103+
SCRATCH.md
87104

88105
# ── OS / editor cruft ────────────────────────────────────────────────────────
89106
Thumbs.db

‎README.md‎

Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
<div align="center">
2+
3+
<img src="streamflow-icon.png" alt="StreamFLOW" width="180"/>
4+
5+
# StreamFLOW
6+
7+
**Native JavaFX flow-cytometry analysis with a Python / FlowKit engine.**
8+
FlowJo-class gating, compensation and analysis — open, fast, and reproducible.
9+
10+
</div>
11+
12+
---
13+
14+
## What it is
15+
16+
StreamFLOW is a desktop flow-cytometry analysis application. The user interface is a native
17+
**JavaFX** app (instant, GPU-accelerated plotting and gating — no web stack), backed by a headless
18+
**Python** analysis engine built on [FlowKit](https://github.com/whitews/FlowKit) / FlowUtils /
19+
NumPy / SciPy / scikit-learn. The two communicate over a simple JSON line protocol, with bulk event
20+
data shipped as little-endian `float32` blobs for native rendering.
21+
22+
The goal is to match FlowJo's workflow where it counts and improve on it where it doesn't: an
23+
always-on audit trail, undo/redo that actually works, live gate statistics, a real before/after
24+
compensation preview, and analysis modules (cell cycle, proliferation, apoptosis, classifier,
25+
cross-sample) that are interactive rather than static images.
26+
27+
## Highlights
28+
29+
- **FlowJo-style gating** — Graph Windows with pseudocolor / dot / contour / density / histogram
30+
plots, polygon / rectangle / ellipse / interval / quadrant gates, a live gating tree, drill-down
31+
child windows, and gate sync across every open window.
32+
- **Axis transforms** — Linear, Log, Logicle (biexponential) and ArcSinh, with per-axis tuning.
33+
- **Compensation** — extract an embedded `$SPILL` matrix or **compute one from single-stain
34+
controls** (Bagwell-Adams) via a guided wizard; interactive heatmap + table editing; a true
35+
**before/after preview** on any channel pair; and a residual-correlation diagnostic that flags
36+
over/under-compensated pairs straight on the heatmap.
37+
- **Analysis modules** — Cell Cycle (Watson / Dean-Jett-Fox), Proliferation (dye-dilution
38+
generations), Apoptosis (Annexin/PI quadrants), Statistics, Cross-Sample (gate×sample matrix,
39+
consistency, radar, MFI drift), Sample Classifier (PCA + Random Forest), Kinetic, Longitudinal,
40+
Dimensionality Reduction and Clustering.
41+
- **Reproducible** — a session audit log, a Methods-paragraph generator from the gating tree,
42+
workspace save/load (`.sfw`) and GatingML export.
43+
- **Publication export** — high-DPI clipboard images and SVG vector output, with configurable DPI,
44+
font size and gate-label options.
45+
46+
## Screenshots
47+
48+
The app icon (above) and module screens use the StreamFLOW dark theme. See **Help → User Guide**
49+
inside the application for a guided tour.
50+
51+
## Architecture
52+
53+
```
54+
┌──────────────────────────┐ JSON line protocol ┌───────────────────────────┐
55+
│ JavaFX desktop app │ ───────────────────────────────▶ │ Python engine │
56+
│ (app/) │ commands + args (stdin) │ (engine/) │
57+
│ │ ◀─────────────────────────────── │ streamflow_engine.py │
58+
│ • CytoPlot native render │ results / progress (stdout) │ │
59+
│ • Graph Windows + gating │ │ • FlowKit / FlowUtils │
60+
│ • Compensation wizard │ event data as float32 blobs │ • NumPy / SciPy │
61+
│ • Analysis modules │ ◀─────────────────────────────── │ • scikit-learn, etc. │
62+
└──────────────────────────┘ └───────────────────────────┘
63+
```
64+
65+
- **`app/`** — the JavaFX application (Java 25, Maven). Plotting/gating run natively in
66+
`CytoPlot`; heavy event binning is computed off the FX thread.
67+
- **`engine/`** — the Python analysis engine (`streamflow_engine.py`) plus a portable virtual
68+
environment under `engine/py-env`. A legacy R engine (`streamflow_engine.R`) is retained for
69+
reference and a subset of tests.
70+
71+
## Requirements
72+
73+
- Windows 10/11 (primary target; the stack is cross-platform).
74+
- **Microsoft Build of OpenJDK 25** (bundled under `jdk-25.0.3+9` in dev setups).
75+
- **Maven** (a copy is bundled under `apache-maven-3.9.16`).
76+
- A staged Python environment under `engine/py-env` (see `engine/stage-python.ps1`).
77+
78+
## Build & run
79+
80+
From the `app/` directory:
81+
82+
```powershell
83+
# Run in development (builds, then launches via the JavaFX Maven plugin)
84+
./run-dev.ps1
85+
86+
# Build only
87+
./build.ps1
88+
89+
# Package a distributable
90+
./package.ps1
91+
```
92+
93+
Or directly with the bundled Maven:
94+
95+
```powershell
96+
$env:JAVA_HOME = "<repo>/jdk-25.0.3+9"
97+
./apache-maven-3.9.16/bin/mvn -f app/pom.xml javafx:run
98+
```
99+
100+
## Tests
101+
102+
```powershell
103+
$env:JAVA_HOME = "<repo>/jdk-25.0.3+9"
104+
./apache-maven-3.9.16/bin/mvn -f app/pom.xml test
105+
```
106+
107+
The suite includes `FxmlLoadTest` (loads every FXML through a real `FXMLLoader` to catch markup
108+
regressions) and `PyEngineTest` (exercises the live Python engine). Engine-dependent bridge tests
109+
skip automatically when their data or runtime is absent.
110+
111+
## Help
112+
113+
Inside the app: **Help → User Guide** for the end-to-end workflow, and **Help → Compensation Help**
114+
for the compensation method (controls, gating, Bagwell-Adams, residual diagnostic).
115+
116+
## Acknowledgements — standing on the shoulders of giants
117+
118+
StreamFLOW is built entirely on open-source software. **Without these projects and the people behind
119+
them, it simply would not exist.** Our deepest thanks to every maintainer and contributor. 🙏
120+
121+
### Analysis engine (Python)
122+
123+
| Project | What it gives StreamFLOW | Link |
124+
|---|---|---|
125+
| **FlowKit** | FCS handling, GatingML/FlowJo interop, compensation application | <https://github.com/whitews/FlowKit> |
126+
| **FlowUtils** | Low-level compensation & transform math (incl. OLS spectral) | <https://github.com/whitews/FlowUtils> |
127+
| **FlowIO** | Pure-Python FCS file reader | <https://github.com/whitews/FlowIO> |
128+
| **NumPy** | Array math underpinning every computation | <https://github.com/numpy/numpy> |
129+
| **SciPy** | Curve fitting, signal processing, statistics | <https://github.com/scipy/scipy> |
130+
| **pandas** | Event tables / dataframes | <https://github.com/pandas-dev/pandas> |
131+
| **scikit-learn** | PCA, Random Forest, clustering | <https://github.com/scikit-learn/scikit-learn> |
132+
| **matplotlib** | Server-side plot rendering | <https://github.com/matplotlib/matplotlib> |
133+
| **UMAP (umap-learn)** | Dimensionality reduction | <https://github.com/lmcinnes/umap> |
134+
| **openTSNE** | Fast t-SNE embeddings | <https://github.com/pavlin-policar/openTSNE> |
135+
| **PhenoGraph** | Graph-based clustering | <https://github.com/dpeerlab/PhenoGraph> |
136+
| **FlowSOM (Python)** | Self-organizing-map clustering | <https://github.com/saeyslab/FlowSOM_Python> |
137+
| **AnnData** | Annotated single-cell data structures | <https://github.com/scverse/anndata> |
138+
139+
### Desktop app (Java / JavaFX)
140+
141+
| Project | What it gives StreamFLOW | Link |
142+
|---|---|---|
143+
| **OpenJFX (JavaFX)** | The entire native UI, canvas rendering & 3D | <https://github.com/openjdk/jfx> |
144+
| **ControlsFX** | PopOver and richer JavaFX controls | <https://github.com/controlsfx/controlsfx> |
145+
| **Ikonli** | Icon fonts (FontAwesome 5, Material Design 2) | <https://github.com/kordamp/ikonli> |
146+
| **Jackson** | JSON protocol between the app and engine | <https://github.com/FasterXML/jackson> |
147+
| **SLF4J** | Logging API | <https://github.com/qos-ch/slf4j> |
148+
| **Logback** | Logging implementation | <https://github.com/qos-ch/logback> |
149+
| **JUnit 5** | Test framework | <https://github.com/junit-team/junit5> |
150+
| **TestFX** | JavaFX UI testing | <https://github.com/TestFX/TestFX> |
151+
152+
### Build & runtime
153+
154+
| Project | Role | Link |
155+
|---|---|---|
156+
| **Microsoft Build of OpenJDK** | Java 21+ runtime / toolchain | <https://github.com/microsoft/openjdk> |
157+
| **Apache Maven** | Build & dependency management | <https://github.com/apache/maven> |
158+
| **javafx-maven-plugin** | Run/package the JavaFX app | <https://github.com/openjfx/javafx-maven-plugin> |
159+
160+
### Methods & inspiration
161+
162+
The compensation workflow follows the **Bagwell-Adams** median method and is informed by the
163+
approaches in [flowStats](https://github.com/RGLab/flowStats) and
164+
[CytoExploreR](https://github.com/DillonHammill/CytoExploreR); the **Logicle** scale implements
165+
Parks–Moore–Roederer (Cytometry A, 2012). [FlowKit's](https://github.com/whitews/FlowKit) design and
166+
the [scverse](https://github.com/scverse) ecosystem (incl. [pytometry](https://github.com/scverse/pytometry))
167+
shaped how StreamFLOW thinks about reproducible cytometry. FlowJo and FCS Express set the bar we aim to clear.
168+
169+
## License
170+
171+
See repository for licensing details.

‎app/src/main/java/org/streamflow/StreamFlowApp.java‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@ public void start(Stage stage) throws Exception {
3838
stage.setMinWidth(1100);
3939
stage.setMinHeight(700);
4040
stage.setScene(scene);
41+
org.streamflow.ui.AppIcons.apply(stage);
4142
stage.show();
4243

4344
startEngine();

‎app/src/main/java/org/streamflow/ui/AnalysisChart.java‎

Lines changed: 52 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -39,13 +39,44 @@ public static final class Series {
3939
private double xMin = 0, xMax = 1; // current (zoomable) X view
4040
private double xDataMin = 0, xDataMax = 1; // full data extent
4141

42+
// optional draggable threshold (e.g. a compensation positive/negative split)
43+
private double thresholdX = Double.NaN;
44+
private boolean thresholdEnabled = false;
45+
private boolean draggingThreshold = false;
46+
private java.util.function.Consumer<Double> onThresholdChange;
47+
private double lastLeft = 60, lastPw = 1; // captured each redraw for pixel<->data mapping
48+
private boolean legendVisible = true; // hide on compact charts where it overlaps bars
49+
4250
private static final double ML = 60, MR = 16, MT = 30, MB = 46;
4351

4452
public AnalysisChart() {
4553
getChildren().add(canvas);
4654
widthProperty().addListener((o, a, b) -> redraw());
4755
heightProperty().addListener((o, a, b) -> redraw());
4856
setMinSize(200, 160);
57+
canvas.setOnMousePressed(e -> {
58+
if (thresholdEnabled && Math.abs(e.getX() - sx(thresholdX, lastLeft, lastPw)) < 8) draggingThreshold = true;
59+
});
60+
canvas.setOnMouseDragged(e -> { if (draggingThreshold) { thresholdX = dataXAt(e.getX()); redraw(); } });
61+
canvas.setOnMouseReleased(e -> {
62+
if (draggingThreshold) { draggingThreshold = false; if (onThresholdChange != null) onThresholdChange.accept(thresholdX); }
63+
});
64+
canvas.setOnMouseMoved(e -> {
65+
if (thresholdEnabled) canvas.setCursor(Math.abs(e.getX() - sx(thresholdX, lastLeft, lastPw)) < 8
66+
? javafx.scene.Cursor.H_RESIZE : javafx.scene.Cursor.DEFAULT);
67+
});
68+
}
69+
70+
/** Show or hide the series legend (hide on small charts where it overlaps the bars). */
71+
public void setLegendVisible(boolean b) { this.legendVisible = b; redraw(); }
72+
73+
/** Show a draggable vertical threshold at data-X {@code x}; NaN hides it. */
74+
public void setThreshold(double x) { this.thresholdX = x; this.thresholdEnabled = !Double.isNaN(x); redraw(); }
75+
/** Notified with the new data-X when the user finishes dragging the threshold. */
76+
public void setOnThresholdChange(java.util.function.Consumer<Double> c) { this.onThresholdChange = c; }
77+
private double dataXAt(double px) {
78+
double t = xMin + (px - lastLeft) / Math.max(1, lastPw) * (xMax - xMin);
79+
return Math.max(xMin, Math.min(xMax, t));
4980
}
5081

5182
// ---- data API ------------------------------------------------------------
@@ -167,14 +198,27 @@ private void redraw() {
167198
g.fillText(fmt(yv), 26, top + ph - k / 4.0 * ph + 3);
168199
}
169200

170-
// legend (top-right)
171-
double ly = top + 6, lx = left + pw - 150;
172-
g.setFont(Font.font("Segoe UI", 10));
173-
for (Series s : series.values()) {
174-
if (!s.visible) continue;
175-
g.setFill(s.color); g.fillRect(lx, ly - 8, 12, 10);
176-
g.setFill(Color.web("#222222")); g.fillText(s.name, lx + 16, ly + 1);
177-
ly += 15;
201+
// draggable threshold line (compensation positive/negative split)
202+
lastLeft = left; lastPw = pw;
203+
if (thresholdEnabled && !Double.isNaN(thresholdX)) {
204+
double tx = sx(thresholdX, left, pw);
205+
g.setStroke(Color.web("#00B4D8")); g.setLineWidth(1.6); g.setLineDashes(5, 4);
206+
g.strokeLine(tx, top, tx, top + ph);
207+
g.setLineDashes((double[]) null);
208+
g.setFill(Color.web("#00B4D8"));
209+
g.fillPolygon(new double[]{tx - 5, tx + 5, tx}, new double[]{top, top, top + 9}, 3);
210+
}
211+
212+
// legend (top-right) — optional, since on small/compact charts it can overlap the bars
213+
if (legendVisible) {
214+
double ly = top + 6, lx = left + pw - 150;
215+
g.setFont(Font.font("Segoe UI", 10));
216+
for (Series s : series.values()) {
217+
if (!s.visible) continue;
218+
g.setFill(s.color); g.fillRect(lx, ly - 8, 12, 10);
219+
g.setFill(Color.web("#222222")); g.fillText(s.name, lx + 16, ly + 1);
220+
ly += 15;
221+
}
178222
}
179223
}
180224

0 commit comments

Comments
 (0)