Improve TUI: web feedback UI, resizable columns, unified ui config - #493
Open
shreyashankar wants to merge 59 commits into
Open
Improve TUI: web feedback UI, resizable columns, unified ui config#493shreyashankar wants to merge 59 commits into
shreyashankar wants to merge 59 commits into
Conversation
…sed progress for non-TTY environments Addresses both items in #492: 1. Cost and documents now update live during operations instead of only appearing when an operation finishes. Added `tick_cost(delta)` to ProgressTracker, wired it through RichLoopBar.update(cost=) and directly in reduce, resolve, rank, and equijoin loops. The TUI's operations pane and header reflect real-time cost accumulation. Reduce and resolve also stream output documents via add_outputs(). 2. When `interactive_ui: True` but stdout is not a TTY (AI agents, CI, subprocesses), the runner now activates a log-based progress reporter instead of silently falling back to no progress. It prints compact structured lines showing operation status, progress, cost, and timing — readable by both humans and agents. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
The log reporter now prints actual document field values as they stream in (up to 3 per tick to avoid flooding). This lets a human watching in a non-TTY environment (e.g. via Claude Code) see what the pipeline is producing and decide whether to kill it or let it continue. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
When interactive_ui: True and stdout is not a TTY, the pipeline now starts a lightweight HTTP server on a free port and prints the URL. Opening it in a browser shows: - Live operation progress (status, cost, timing) - Document outputs streaming in as they're generated - Per-document feedback text inputs - Pipeline-level feedback input - Kill button to stop the pipeline with a reason Feedback is printed to stdout (so the agent reads it) and written to _docetl_feedback.json. The kill button sets a flag checked by RichLoopBar.update(), raising PipelineKilled to stop the pipeline cleanly between batches. No external web framework — uses Python's built-in http.server with Server-Sent Events for live updates. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
Uses the same light theme, blue primary color (hsl 211), emerald success, rose destructive, rounded-xl cards with subtle shadows, and system sans- serif font. Adds animated progress bars in the sidebar and fade-in animations for new document cards. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
Replace card-based layout with a proper data table featuring sortable columns, inline histograms (distribution + categorical), and a dedicated Visualize tab. Uses the playground's exact CSS variables and design tokens. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
Replace operation pills with a clean horizontal text list. Make feedback inputs inline in every table row instead of click-to-expand. Standardize border-radius to var(--radius). Add CSS scroll shadows for affordance. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
Replace crude CSS div bars with proper SVG bar charts that have rounded tops, axis labels, and native SVG title tooltips on hover. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
The pulsing dot + count text already convey running status without the bottom progress bar. Hide the ops strip on the Visualize tab since it's not relevant to the column distribution view. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
…ications, and skill instructions - FeedbackStore prints [FEEDBACK:doc] and [FEEDBACK:pipeline] to stdout in real-time so the agent can read feedback immediately - Add /message POST endpoint and toast notification UI so the agent can send messages back to the human - Add /messages?since=N GET endpoint for polling agent messages - Include agent_messages in SSE event stream for live toast delivery - Add "Human-in-the-Loop Feedback Workflow" section to SKILL.md with instructions for reading feedback, sending toasts, and iterating on pipelines https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
Every border (topbar, ops strip, tabs, table cells, buttons, inputs, viz cards, toasts, tooltip, statusbar, completion banner) replaced with subtle box-shadows or removed entirely. Toasts are now clean floating cards with no border at all. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
…e layout Toast changes: - Agent messages can include an `actions` list (e.g. ["Confirm re-run", "Dismiss"]) - Toasts with actions show clickable buttons that stay on screen until clicked - Clicking posts to /message/respond and prints [TOAST:response] to stdout - Toasts without actions auto-dismiss after 15s (info/status toasts) Table changes: - Remove table-layout: fixed — columns now auto-size - String-words columns get min-width 200px, 300px for long text - Number and low-cardinality columns stay narrow (80-100px) Update SKILL.md with action toast documentation. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
…external feedback Major UI redesign: - Remove operation column from table (info available in detail panel) - Click any row to open a slide-out detail panel with full field values, prev/next navigation, and feedback input - Add column search/filter (magnifying glass icon in headers, AND logic) - Green dot indicator on rows that have feedback - Feedback submitted via API now syncs to the UI via SSE doc_feedback - ESC closes the detail panel https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
Left/Right arrows navigate between rows (matching playground UI pattern). Disabled when typing in inputs. Shows "Row X of Y" and hotkey hints in the detail header. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
…l panel - Operations strip now shows → arrows between ops to clarify sequential flow - Detail panel feedback: existing feedback shown as green card, input is empty with placeholder (no more pre-filling that duplicated the text) https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
- POST /reset clears all doc outputs and increments reset_token - JS detects reset_token change and clears table/columns/state - Toast messages >100 chars are truncated to 2 lines with "Show more" toggle to expand/collapse full text - Toast expand button styled as subtle link below message text https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
When from_agent: True is set in the pipeline YAML, the runner always launches the browser-based feedback UI (web_reporter) regardless of TTY status. This is designed for agent-orchestrated pipelines where a human supervises via browser. interactive_ui: True continues to use the Textual TUI when a TTY is available. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
Consolidates two boolean flags into one string: ui: "none" (default), "tui" (Textual dashboard), "web" (browser feedback UI) No mixed types, no precedence ambiguity. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
The detail panel's "← → to navigate" hint span was blocking clicks on inputs underneath it. Adding pointer-events: none fixes it. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
Columns can be dragged wider or narrower via a resize handle at the right edge of each header cell. User-set widths persist across table re-renders. Uses table-layout: fixed for stable column sizing. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
Resolve conflicts in progress.py (keep set_description from main + cost param from branch) and test_tui.py (keep test_pipeline_label_from_yaml_path from main + test_ui_mode_routing from branch). https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
Instead of showing provenance as a separate section at the bottom of the document detail, render it as an inline key-value immediately after the status line. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
The web UI was only printing the URL to console. Now it calls webbrowser.open() so the page launches automatically. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
Instead of mixing all intermediate ops' docs together (map + unnest + resolve + reduce), the table now shows only the last operation that has outputs. When the source op changes (e.g. resolve finishes, reduce starts producing), the JS clears the old rows and streams the new op's outputs. This fixes the empty table issue and gives a cleaner UX. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
The previous logic picked the last running op as the table source even when it had 0 outputs yet, showing an empty table. Now we only pick ops that have actual output docs, falling back to the previous op until the current one starts producing. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
Prints doc counts and op output sizes to stderr on each SSE push so we can diagnose the empty table issue. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
const stats was declared inside an else block (block-scoped) but referenced outside it, causing a ReferenceError that killed both renderTableHead and renderTableBody on every call. Moved the declaration to the outer forEach scope. Also removed debug logging. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
- Each doc can now accumulate multiple feedback items instead of replacing the previous one. - Feedback cards show an × button to delete individual items. - Delete hits POST /feedback/doc/delete on the server side. - Textarea clears after sending; feedback count shown in label. - Green dot in table shown when doc has any feedback. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
Add ui: "web" to the pipeline YAML template and the feedback workflow section so agents always enable the browser feedback UI when they author a pipeline. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
Clarify that all data must be self-contained in the HTML — never generate href links to local files or external URLs. Use inline JSON data with onclick handlers and modals instead. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
The web UI server now supports two modes: - Persistent: `docetl serve` starts a long-lived server that survives across pipeline runs. The agent polls /feedback/poll for feedback. - Inline (fallback): the pipeline starts its own server and waits for "Done reviewing" before exiting. Pipeline runs with `ui: "web"` auto-detect a running persistent server and push state to it, keeping the browser connection alive across re-runs. Also trims the skill file's hardcoded operation docs to short pointers to the actual repo docs. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
- .claude/hooks/poll-feedback.sh: fires after every `docetl run`, polls the feedback server, injects feedback into agent context or tells the agent to wait for human review - .claude/settings.json: registers the hook - /feedback/wait endpoint: blocks until human clicks "Done reviewing" - Skill updated to document the automatic feedback hook flow https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
The hook now checks for new human feedback after every Bash command and after every agent turn, not just after `docetl run`. This means the agent gets notified whenever the human submits feedback, regardless of what the agent is currently doing. Deduplication ensures already-seen feedback is not re-injected. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
- Remove hook infrastructure (poll-feedback.sh, settings.json hooks) - FeedbackStore writes all events to .docetl_feedback.log - Skill instructs agent to run pipelines via background subagents - Skill instructs agent to Monitor(tail -f .docetl_feedback.log) for real-time feedback notifications - Fix /feedback/wait to return immediately if feedback already exists - Main agent stays free to react to feedback as it arrives https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
The true_labels array from comparisons contained Python bool objects, which numpy can't bitwise-AND with a float-derived boolean array on Python 3.11. Explicitly set dtype=bool. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
… clears stale data The push loop in _push_state_to_server only checked for kill requests but ignored doc/pipeline feedback — the subagent never saw user feedback. Now it tracks and prints new feedback events to stdout as they arrive, plus prints a summary at pipeline end. Also: /reset clears old feedback + log, skill uses tail -F for robustness, removes stale /feedback/wait instructions. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
The SSE broadcaster only pushed events when the pipeline's push loop called accept_state(). After the pipeline finished (or between pushes), toasts and feedback were stored but never broadcast to SSE subscribers — the browser never saw them. Added rebroadcast() method that re-sends the last known state with fresh feedback/messages. All mutation endpoints (feedback, messages, kill, done) now call it, so the browser updates instantly. Also consolidated skill instructions: Monitor starts once at session start (not per-pipeline), stays alive across all runs. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
…dpoint These were artifacts of the inline server mode's blocking wait pattern. With the persistent server architecture, the server stays alive across runs and feedback is delivered via the log file. No need for a "done" signal. Inline mode now exits after a brief pause instead of blocking for 30 minutes. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
…eeded Instead of requiring `docetl serve &` (which dies when the Bash shell exits), run_with_web_ui() now auto-starts a detached server subprocess if none is found. The browser opens once and all subsequent runs reuse the same server and tab — no new windows per run. The inline server mode remains as a fallback if auto-start fails. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
Three fixes for the UI not showing data during pipeline runs: 1. Send _last_event immediately when an SSE subscriber connects, so the browser catches up on current state instead of waiting for the next push. 2. Add X-Accel-Buffering: no header to prevent reverse proxies from buffering the SSE stream. 3. Reduce push/poll interval from 1s to 0.5s for snappier row streaming. 4. Log push errors to stderr instead of silently swallowing them. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
SSE is unreliable through reverse proxies in remote environments. Added: - JS polling for toast messages every 3s (independent of SSE) - JS polling for full state every 2s when SSE is disconnected - GET /state/current endpoint returning the last known state UI redesign: - Reduced border-radius from 0.5rem to 4px for a sharper, less bubbly look - Feedback inputs: larger (44px min-height), visible borders, white background - Feedback cards: colored left border accent instead of full rounded background - Toasts: left border accent by type (blue=info, green=success, amber=warning, red=error), wider max-width, better text hierarchy - Buttons: visible borders, slightly larger padding - Complete banner: left border accent like feedback cards https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
…ration tests - rebroadcast() now updates _last_event so /state/current and SSE catch-up include fresh toasts and feedback (was returning stale data without new msgs) - Removed all colored left/bottom borders per user feedback — using uniform border + subtle background tints instead - All 8 integration tests pass: push, SSE, toasts, feedback, polling, reset https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
- Move feedback input to top of panel for easy access - Make field values collapsible with chevron toggles (long fields collapsed by default) - Widen panel from 400px to 520px - Overlay Send button inside textarea instead of separate row - Add toggleDetailField() JS for expand/collapse behavior https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
User explicitly requires no colored borders. Feedback cards now use var(--card) background and var(--border) border instead of green hsl(152). https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
Each pipeline run now kills any lingering docetl serve processes before spawning a fresh server. Prevents zombie servers piling up. Also handles SIGTERM gracefully to clean up the port file on shutdown. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
Replace Unicode ▶ with a CSS border-triangle that renders crisply across all browsers. Switch Send button from absolute positioning to flex layout so it aligns properly with the textarea. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
Long field content now scrolls within a 300px container instead of expanding the panel indefinitely. https://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
shreyashankar
force-pushed
the
claude/wonderful-davinci-rHSLD
branch
from
June 9, 2026 06:58
c868d38 to
0f0ba5a
Compare
Previously agent toasts were delivered through three redundant paths: SSE events, /messages polling, and /state/current polling. All three included agent_messages, relying on client-side dedup via seenMsgIds. Now toasts are delivered exclusively via /messages?since=N polling (every 3s). SSE and /state/current handle pipeline state only. https://claude.ai/code/session_013WUwghjHVh6r5mRHEMuhGP
Clicking "Feedback: N" in the statusbar now opens a dropdown showing all submitted feedback (pipeline-level and doc-level) with timestamps and labels. Fetches from /feedback/poll on open, closes on outside click. https://claude.ai/code/session_013WUwghjHVh6r5mRHEMuhGP
The feedback log (.docetl_feedback.log) used a relative path, so if the server and the agent had different cwds, the agent's Monitor would watch the wrong file and never see feedback. Now FeedbackStore resolves the log path to an absolute path at init. The pipeline prints [FEEDBACK_LOG] <path> to stdout so the agent knows where to monitor. Added /feedback/log_path endpoint for discovery. https://claude.ai/code/session_013WUwghjHVh6r5mRHEMuhGP
shreyashankar
force-pushed
the
claude/wonderful-davinci-rHSLD
branch
from
June 9, 2026 07:01
e533382 to
03f5ca2
Compare
The end-to-end feedback flow was broken in three ways: 1. The port file (.docetl_server_port) was cwd-relative, so a pipeline run from a different directory could not find the running server, spawned a new one, and killed the old one in the process. 2. The feedback log was written in the server's cwd, not where the agent was tailing, so feedback silently went to the wrong file. 3. Toast button responses ([TOAST:response]) were only printed to the server's stdout — a never-read pipe when auto-started — so the agent could never see them. Simplified protocol, one channel per direction: - Human -> agent: ONE log file, <pipeline>.feedback.log, next to the pipeline YAML and named after it. The server appends every human event there: doc/pipeline feedback, kill, and toast button clicks. The run prints "[FEEDBACK_LOG] <path>" so the agent knows what to tail. Each run registers its log path with the server via /reset. - Agent -> human: POST /message (unchanged). - Discovery: one global port file at ~/.docetl/server_port so pipelines in any directory share one server. Also: auto-start now waits up to 30s for slow environments and detects early death of the server process; spawned server stdout goes to DEVNULL instead of a never-drained pipe. Verified end-to-end with a headless browser: UI feedback (both pipeline bar and per-doc detail panel) and toast button clicks all land in the pipeline's own feedback log; a second pipeline in another directory reuses the same server and retargets the log. https://claude.ai/code/session_013WUwghjHVh6r5mRHEMuhGP
The agent must start the feedback Monitor in the same parallel tool call as the pipeline subagent — not as a separate sequential step it can skip. The feedback log path is predictable from the YAML name, so no need to wait for [FEEDBACK_LOG] output. https://claude.ai/code/session_013WUwghjHVh6r5mRHEMuhGP
The feedback protocol had accumulated machinery beyond what it needs: two server modes (inline + persistent), SSE with subscriber queues plus two fallback polling loops in the browser. Strip it to the minimum that implements the same protocol: - Delete inline mode: the persistent server is the only mode. If it can't start, run without a UI instead of half-working. - Delete SSE and the _Broadcaster class. The browser polls GET /state once a second — one endpoint returns pipeline state, feedback counts, and agent toasts together. - Remove /events, /messages, /state/current endpoints. - Dedupe the pipeline push loop's snapshot serialization. External protocol unchanged: <pipeline>.feedback.log next to the YAML for human→agent, POST /message for agent→human. Verified end-to-end in a headless browser: table streams, feedback lands in the log, action toast renders within a second, button click is logged, re-run resets. Net -247 lines. All 29 TUI tests pass. https://claude.ai/code/session_013WUwghjHVh6r5mRHEMuhGP
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ui: "web") for agent-orchestrated pipelines — browser-based dashboard with live-streaming data table, progress bars, document detail panel, and human feedback collectionuiconfig field replacing the oldinteractive_uiboolean —"none"(default),"tui"(Textual terminal dashboard), or"web"(browser feedback UI)/resetendpoint — clears outputs and streams fresh dataConfig usage
Test plan
pytest tests/test_tui.py)ui: "web"and verify the browser UI loads/messageendpoint → toast notification appears/resetendpoint → table clears and accepts new dataui: "tui"→ Textual terminal dashboard launchesuifield → standard console output, no UIhttps://claude.ai/code/session_01M8PbcxpEsrD7MkffxDjFZX
Generated by Claude Code