Skip to content
This repository was archived by the owner on Jun 8, 2026. It is now read-only.

FEA-1554: Close session data collection gaps across all 5 harness parsers - #271

Merged
mikeangstadt merged 5 commits into
mainfrom
feat/fea-1554
Jun 7, 2026
Merged

mikeangstadt merged 5 commits into
mainfrom
feat/fea-1554

Conversation

@thadeusb

@thadeusb thadeusb commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Feature

FEA-1554 — Session Data Collection Gaps for Desktop Session Screens

Parent: PRD-444 (Session Data Collection for Desktop Agent/Session Screens)

Summary

  • Extend NormalizedSession + NormalizedToolUse types with 11 new fields for messages, token series, diff stats, slash commands, artifacts, tool output, MCP server/method, skill names
  • Update all 5 harness parsers (Claude, Codex, Cursor, Copilot, OpenCode) to extract previously-discarded data: message text, tool output, per-turn tokens, diff stats, artifact references
  • Wire cost computation via token-cost.ts into dashboard analytics
  • Add event-role.ts (event type → human/agent/system) and session-timing.ts (active/waiting derivation)
  • Extend cloud sync STRIPPED_LEAF_KEYS with text, output, reasoning for mandatory content stripping

Architecture

flowchart TD
    Sources[5 Harness Sources] --> Parsers[5 Updated Parsers]
    Parsers --> NS[NormalizedSession + NormalizedToolUse<br/>types.ts — 11 new fields]
    PU[parser-utils.ts<br/>7 new helpers] -.-> Parsers
    NS --> IS[import-session.ts<br/>UserMessage/AssistantMessage events<br/>enriched PostToolUse data]
    IS --> DB[(SQLite: metadata JSON + events.data)]
    DB --> SYNC[STRIPPED_LEAF_KEYS<br/>+text +output +reasoning]
    DB --> COST[dashboard.ts cost wiring]
    DB --> ROLE[event-role.ts + session-timing.ts]
Loading

Plan

PLN-816 — Plan challenge: Opus PASS, Sonnet PASS, Codex PASS

Key Decisions

  • Messages stored as individual UserMessage/AssistantMessage events (not metadata JSON) — too large for blob, need idempotency
  • Token series stored in metadata (small numeric data, no cloud-strip needed)
  • CR-11 role mapping as function not DB column — avoids schema migration
  • Cursor diff stats absent at source (verified, recorded [absent])
  • Copilot diff stats and reasoning text absent at source (recorded [absent])

Decision Table

Verified at .closedloop-ai/decision-tables/fea-1554.md — 63 delta checklist items covered.

Review Summary

4 parallel review lanes (correctness, security, conventions, codex): all PASS, 0 blockers, 0 findings requiring fixes.

Downstream Updates

None — FEA-1554 is the only child of PRD-444. Sibling features (FEA-1503, FEA-1497) are DONE.

Feature Flags

None — collection-layer improvement, not a net-new user capability. Existing FEA-1407 sandbox gate controls capture.

Database/Migration Safety

No schema/migration changes. All new data stored in existing metadata JSON and events.data JSON columns.

Test Plan

  • 49 parser-utils helper tests (truncateText, computeLineDelta, extractRepoFromCwd, extractPrReferences, extractIssueReferences, collectArtifacts, isSyntheticModelKey)
  • 14 event-role mapping tests
  • 7 session-timing derivation tests
  • 4 cloud sync strip tests (text, output, reasoning, round-trip)
  • Full suite: 2,263 tests, 0 failures
  • Typecheck clean

…sers

- Extend NormalizedSession with messages, tokenSeries, diffStats, slashCommands,
  artifacts fields; extend NormalizedToolUse with output, isError, mcpServer,
  mcpMethod, skillName, diffDelta
- Add shared parser helpers: truncateText, computeLineDelta, extractRepoFromCwd,
  extractPrReferences, extractIssueReferences, collectArtifacts, isSyntheticModelKey
- Claude parser: capture message text, per-turn tokens, tool output, diff stats,
  slash commands (<command-name> tags), skill names (Skill tool input.skill)
- Codex parser: capture message text (remove void), reasoning, token series,
  tool output, MCP server/method (preserve from mcp_tool_call_begin), diff stats
- Cursor parser: capture message text, token series, tool output, per-message model
- Copilot parser: capture message text (both chat+CLI), token series, tool output
- OpenCode parser: extend SELECT for per-message tokens/model/patch/summary columns
- import-session: persist UserMessage/AssistantMessage events, enrich PostToolUse
  event data with output/MCP/skill/diff, extend buildMetadata with new fields
- Cloud sync: add text/output/reasoning to STRIPPED_LEAF_KEYS
- Wire computeTokenCost into dashboard getTokenAnalytics for per-model cost
- Add event-role.ts (event_type -> human/agent/system mapping)
- Add session-timing.ts (active agent vs waiting-on-user derivation)

Testing: 70 new unit tests (parser-utils: 49, event-role: 14, session-timing: 7,
  cloud sync strip: 4). Full suite: 2,263 tests, 0 failures.

Risks: Cloud-sync strip must not miss content keys; verified by round-trip test.
@thadeusb
thadeusb requested a review from a team June 5, 2026 19:27
@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

Compatibility Smoke Test Results ⚠️

Status: skipped
Electron SHA: 1740b9c5eb92cb3922dc737a72eca49d0bde7e20
Symphony Alpha SHA (last-known-good): 24741ed9be45cc761195d7a7b6613bd30cdfce84
Note: Skipped because the stage GitHub App credentials are not configured for this workflow run.

View Actions run

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 72d023000c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/desktop/src/main/collectors/codex/codex-parser.ts
Comment thread apps/desktop/src/main/collectors/codex/codex-parser.ts Outdated
Comment thread apps/desktop/src/main/collectors/parser-utils.ts
Comment thread apps/desktop/src/main/collectors/codex/codex-parser.ts
Comment thread apps/desktop/src/main/collectors/import-session.ts

@thadeusb thadeusb left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid extraction work across all 5 parsers and the cloud strip additions are correct. Three things inline, none blocking.

Comment thread apps/desktop/src/main/collectors/import-session.ts
Comment thread apps/desktop/src/main/collectors/import-session.ts
Comment thread apps/desktop/src/main/collectors/import-session.ts
Comment thread apps/desktop/src/main/collectors/parser-utils.ts Outdated
Comment thread apps/desktop/src/main/collectors/parser-utils.ts Outdated
Comment thread apps/desktop/src/main/collectors/copilot/copilot-parser.ts Outdated
Comment thread apps/desktop/src/main/collectors/copilot/copilot-parser.ts
Comment thread apps/desktop/src/main/collectors/cursor/cursor-parser.ts Outdated
Comment thread apps/desktop/src/shared/agent-db-contract.ts
Comment thread apps/desktop/src/main/collectors/types.ts
Comment thread apps/desktop/src/main/collectors/opencode/opencode-parser.ts Outdated
Comment thread apps/desktop/src/main/collectors/opencode/opencode-parser.ts Outdated
@mikeangstadt
mikeangstadt merged commit bf7c8bd into main Jun 7, 2026
5 checks passed
@mikeangstadt
mikeangstadt deleted the feat/fea-1554 branch June 7, 2026 21:21
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants