Skip to content

Render Markdown in Sidekick and REPL Output - #1351

Draft
taras wants to merge 2 commits into
mainfrom
codex/repl-markdown-1342
Draft

taras wants to merge 2 commits into
mainfrom
codex/repl-markdown-1342

Conversation

@taras

@taras taras commented Nov 16, 2026 •

Copy link
Copy Markdown
Owner

REPL UI evidence

Checkpoint Fixture Viewport Action Reference Before After Comparison
wide-160x36 Identical scripted Sidekick reply and executed Markdown entry from #1342 160 × 36 Join the scripted exchange and scroll to the reply heading while Entry 1 Output is visible. Issue #1342 specifies the display behavior without an approved visual design asset. Before wide-160x36 After wide-160x36 —
raw-160x36 Identical scripted Sidekick reply and executed Markdown entry from #1342 160 × 36 Open the saved exchange and read the original Markdown and link target. Issue #1342 specifies the display behavior without an approved visual design asset. Before raw-160x36 After raw-160x36 —
chat-72x20 Identical scripted Sidekick reply and executed Markdown entry from #1342 72 × 20 Join the scripted exchange and scroll the narrow Sidekick view to the reply heading. Issue #1342 specifies the display behavior without an approved visual design asset. Before chat-72x20 After chat-72x20 —
chat-fence-72x20 Identical scripted Sidekick reply and executed Markdown entry from #1342 72 × 20 Scroll the narrow Sidekick reading down to its literal code fence. Issue #1342 specifies the display behavior without an approved visual design asset. Before chat-fence-72x20 After chat-fence-72x20 —
output-72x20 Identical scripted Sidekick reply and executed Markdown entry from #1342 72 × 20 Open Entries, dismiss the catalog and reveal the heading and beginning of Entry 1 Output. Issue #1342 specifies the display behavior without an approved visual design asset. Before output-72x20 After output-72x20 —
output-fence-72x20 Identical scripted Sidekick reply and executed Markdown entry from #1342 72 × 20 Scroll the narrow Output reading down to its literal code fence. Issue #1342 specifies the display behavior without an approved visual design asset. Before output-fence-72x20 After output-fence-72x20 —

Mounted typing latency

Viewport 200 × 52. Milliseconds from character dispatch to the completed emulator write containing the changed Message value.

Workload Samples per run Before median After median Δ median Before p95 After p95 Δ p95
small 30 12.149 11.717 -0.432 13.421 12.389 -1.031
wrapping 30 19.926 19.849 -0.076 20.668 21.473 0.805

Why

Markdown in Sidekick messages and executed round Output appears as raw punctuation. This makes headings, links and code harder to scan in the terminal.

What changes

Both views use the same terminal formatting for headings, paragraphs, emphasis, lists, descriptive link labels, inline code and fenced code. The display wraps to the pane width and preserves the indentation and line breaks of code. The saved Markdown remains available through the existing raw readers.

How to verify it

Run the mounted Sidekick journey and the executable Markdown ReplBench journey in a prepared worktree. They exercise conversation text and round Output at wide and narrow sizes, including literal code, retained links and reopening. The paired evidence records the same fixture on main and this draft.

Closes #1342.

taras and others added 2 commits November 15, 2026 21:22
Sidekick conversation bodies and the selected entry's rendered Output share a
private display path over the pinned Marked lexer: headings, emphasis, list
markers, link labels and code are styled runs, fences stay literal line for
line, and unsupported constructs read as written. Controls are made inert
before lexing and on every run. Retained text and raw reads are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#1342)

A fence-like line closes only a fence opened with the same character and no
longer run; a shorter or different run stays code. Lines written between
blocks are drawn as written, so a whitespace-only line keeps its spaces. The
spec states that formatting draws exactly the body its owner supplies.
@taras

taras commented Nov 16, 2026

Copy link
Copy Markdown
Owner Author

Independent review

Reviewed commit 44fca8704e75996f540976c45553f3203097752a

Verdict CHANGES REQUIRED

Reviewer

claude / fable

Summary

Reviewed the exact signed head (parent d06c74934) against both clauses; two blocking defects remain in the shared formatter path. Checkout is clean at the assigned SHA.

1. A tight task-list item is split across two rows on both surfaces, so an agreed construct (lists) renders wrongly.
2. Formatting the selected entry's Output is rebuilt on every prepared frame with no memo, and the pinned lexer is quadratic on long paragraphs with unmatched `*` or `_`. A 1,000-line traceback costs about 424 ms per frame against 0.4 ms on the baseline path.

Otherwise the correction holds. The two Planner cases are fixed, fence payload stays literal in lists, quotes and unclosed fences, and controls stay inert with no link target or hyperlink sequence drawn. Retained text is untouched by the formatter. I reused the recorded receipts at this SHA for the mounted journey, bench and preflight rather than rerunning them, and ran only focused probes. The recorded paired typing measurement shows no regression, but its workloads do not include asterisk-heavy Output.

Findings

Blocking

matched-rendering — a tight task-list item is drawn as two rows. `packages/cli/src/repl/markdown.ts:118-119` treats the `checkbox` token as a block that returns its own line, but Marked 17 emits it as a sibling of the item's `text` token inside a tight `list_item`. Probe at this SHA: `markdownLines("- [ ] todo\n- [x] done", "output")` gives `• [ ] `, `  todo`, `• [x] `, `  done`; the ordered form `1. [ ] numbered` splits the same way. The loose form (blank line between items) stays on one row because there the checkbox is an inline token. Both surfaces share this formatter, so Sidekick replies and entry Output each show every `- [ ]` line as a marker row followed by a text row. This contradicts the spec sentence the change adds ("Each written line stays one logical line") and the agreed list rendering. No test covers a task item. Correction: put the checkbox in front of the item's first line rather than on a line of its own, and add tight, loose and ordered task items to the helper cases.

Blocking

matched-rendering (plain text and literal Output at any width) — measured responsiveness regression on the per-frame path. `prepareFrame` (`packages/cli/src/repl/program.ts:5962-5971`) rebuilds `readingLines(readingOf(view))` for every prepared frame, and `outputSection` now routes Output through `written` → `markdownLines` → `Lexer.lex` (`source-reading.ts:231,248,292`) with no memo; the existing fit store reuses geometry only after the lines are rebuilt. Marked's inline pass is quadratic in unmatched `*` or `_` within one paragraph. Median ms per `readingLines(entryReading(...))`, head against the baseline checkout at 30d614cd (the `plain` path is unchanged by this diff): Python traceback lines with `(*args, **kwargs)` — 500 lines/24 KB 109.0 vs 0.2, 1,000 lines/49 KB 423.8 vs 0.4, 2,000 lines/99 KB 1687.5 vs 0.6. C declarations (`int *p = f(*q);`) — 1,000 lines 285.6 vs 0.3, 2,000 lines 1166.3 vs 0.5. Underscore identifiers behave the same at the helper level (1,000 lines 366 ms). Ordinary prose stays linear (2,000 lines 3.5 vs 0.5). Markdown-shaped text costs about ten times the old path (270 KB: 23 ms vs 2.3 ms). Consequence: while such an entry is selected, each repaint — a typed character, a wheel tick, a streamed delta — blocks for that long; the spec explicitly allows captured diagnostics to be interpolated into rendered Output. The chat path has the same exposure per streamed delta, because `sidekickComposedRows` re-lexes every body whenever chat facts change (`application.ts:4596`). The recorded typing measurements (`evidence/typing-corrected.json`, `typing-latest-confirmation.json`) show no regression but do not exercise this input. Correction: prepare formatted lines once per body string (the accepted scope asks for prepared output that reuses the existing caches) and bound the lexer's worst case so such text falls back to literal lines.

Non-blocking

retention-and-streaming — examined, no defect found. The formatter only reads the supplied string; the MD1 journey at this SHA asserts retained entry Output equals the written Markdown, the exchange inspection still shows `**chatbold**` and the full link target, and cold reopen draws the same bodies with no provider turn (receipt `ticket/attempts/0003/receipts.md`: 25 passed, 132 steps; bench journey and preflight pass at 160×36 and 72×20). I did not rerun these. My probe confirms ESC, BEL, CR and an OSC 8 sequence all come out as `\xNN` text, a link draws its label only, and an unclosed strong, link or fence prefix stays readable.

Non-blocking

Tabs in chat bodies now read as `\x09`, because the inert conversion runs before lexing. A tab-indented fence payload in a Sidekick reply shows `\x09foo`, and tab-structured Markdown (`-\titem`, tab-indented nesting) is not recognised as a list. This follows the spec's command-text rule and already held for entry Output; for the chat it is a consequence of the planned inert handling that the owner may want to confirm against "keep fence indentation literal".

Non-blocking

Limits of visual parsing worth knowing, none against the spec as written: a setext underline row is dropped (`Result\n---` draws one heading row); prose such as `2024. A year\nplain` becomes an ordered item with its next line indented six cells; a wrapped list item's continuation hangs at the line's leading whitespace, not under the item text; a whitespace-only line between tight list items loses its spaces.

Non-blocking

Process note: one probe file was briefly copied into `packages/cli/src/repl/` to resolve the `marked` import and removed in the same command; `git status` was clean immediately after and after every later probe. All other probes live in `/tmp/xmd1342-fable-review/` and were run with `deno run --cached-only --frozen --node-modules-dir=none`. No process is left running.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Render Markdown in REPL Sidekick conversations and round output

1 participant