From 8f3b84baf9bf53465ddca2d0c19e63ef5ec99bc9 Mon Sep 17 00:00:00 2001 From: willwearing Date: Sat, 5 Sep 2026 12:02:38 -0600 Subject: [PATCH] chore: clean up untracked working-tree files Four unrelated leftovers had been sitting untracked in the working tree. Resolving each rather than carrying them further. Deleted backend/debug-mastery.ts and debug-mastery2.ts. Standalone Prisma scripts from April that dump one hardcoded user's mastery state to the console. Nothing imports them and the UUIDs they target are stale. Deleted backend/src/config/cors-origin.ts and its spec. An unwired second implementation of CORS origin checking. main.ts imports buildStaticOrigins, createCorsOriginGuard and isOriginAllowed from ./config/cors, and nothing referenced createCorsOriginHandler. Keeping a duplicate of a security control next to the live one risks someone wiring up the wrong one later. The behaviour it was exploring, answering a disallowed origin by omitting CORS headers instead of returning 403, can be revisited in config/cors.ts directly if that turns out to be wanted. Gitignored .posthog-audit-checks.json. Regenerated wizard output, and the committed copy was already stale on SDK versions. Committed .claude/skills/audit and integration-nextjs-app-router. Same shape as the tracked integration-javascript_node skill; they were just never added. Co-Authored-By: Claude Opus 5 --- .claude/skills/audit/.posthog-wizard | 0 .claude/skills/audit/SKILL.md | 75 ++ .claude/skills/audit/references/1-version.md | 90 +++ .claude/skills/audit/references/2-init.md | 52 ++ .../audit/references/3-identification.md | 118 +++ .../audit/references/4-event-capture.md | 95 +++ .claude/skills/audit/references/5-report.md | 373 +++++++++ .../skills/audit/references/best-practices.md | 194 +++++ .../skills/audit/references/identify-users.md | 272 +++++++ .../.posthog-wizard | 0 .../integration-nextjs-app-router/SKILL.md | 59 ++ .../references/1-begin.md | 56 ++ .../references/2-edit.md | 36 + .../references/3-revise.md | 22 + .../references/4-conclude.md | 57 ++ .../references/EXAMPLE.md | 710 ++++++++++++++++++ .../references/identify-users.md | 272 +++++++ .../references/next-js.md | 383 ++++++++++ .gitignore | 3 + 19 files changed, 2867 insertions(+) create mode 100644 .claude/skills/audit/.posthog-wizard create mode 100644 .claude/skills/audit/SKILL.md create mode 100644 .claude/skills/audit/references/1-version.md create mode 100644 .claude/skills/audit/references/2-init.md create mode 100644 .claude/skills/audit/references/3-identification.md create mode 100644 .claude/skills/audit/references/4-event-capture.md create mode 100644 .claude/skills/audit/references/5-report.md create mode 100644 .claude/skills/audit/references/best-practices.md create mode 100644 .claude/skills/audit/references/identify-users.md create mode 100644 .claude/skills/integration-nextjs-app-router/.posthog-wizard create mode 100644 .claude/skills/integration-nextjs-app-router/SKILL.md create mode 100644 .claude/skills/integration-nextjs-app-router/references/1-begin.md create mode 100644 .claude/skills/integration-nextjs-app-router/references/2-edit.md create mode 100644 .claude/skills/integration-nextjs-app-router/references/3-revise.md create mode 100644 .claude/skills/integration-nextjs-app-router/references/4-conclude.md create mode 100644 .claude/skills/integration-nextjs-app-router/references/EXAMPLE.md create mode 100644 .claude/skills/integration-nextjs-app-router/references/identify-users.md create mode 100644 .claude/skills/integration-nextjs-app-router/references/next-js.md diff --git a/.claude/skills/audit/.posthog-wizard b/.claude/skills/audit/.posthog-wizard new file mode 100644 index 0000000..e69de29 diff --git a/.claude/skills/audit/SKILL.md b/.claude/skills/audit/SKILL.md new file mode 100644 index 0000000..a669013 --- /dev/null +++ b/.claude/skills/audit/SKILL.md @@ -0,0 +1,75 @@ +--- +name: audit +description: Audit an existing PostHog integration for correctness and best practices +metadata: + author: PostHog + version: 1.26.0 +--- + +# PostHog Audit + +This skill audits an existing PostHog integration for **data integrity** in event capture and identification. **Read-only** — the only file you create is the final audit report. + +Perform the checks described in the referenced skills and only the events referenced in the skills. + +## Workflow + +The audit runs as a 5-step chain: Installation (SDK + version) → init correctness → identification → event capture → report (which also uploads the report to a PostHog notebook). Each step file ends with a pointer to the next. Follow them in the order they are written. You must resolve them in order before any source-tree exploration. + +The audit ledger is already seeded with the 11 pending checks (10 correctness checks plus `upload-notebook`, which the report step resolves after mirroring the markdown into a PostHog notebook). Use `mcp__wizard-tools__audit_resolve_checks` to patch each one as you finish it. + +**Start by reading the path relative to this file at `references/1-version.md`.** Do not Glob, ls, or find the skill directory. Do not preload future steps. Do not re-read a step file once you've moved past it. Do not re-read SKILL.md. + +`ToolSearch` is only for loading a tool by exact name when the SDK has it deferred (e.g. `select:Grep`). Do **not** use it to browse for other tools — every tool the audit needs (`Glob`, `Grep`, `Read`, `Write`, `Bash`, and the named `mcp__wizard-tools__audit_*` tools) is already named in this skill. + +**Do not call `TaskCreate` / `TaskUpdate` / `TaskGet` / `TaskList`.** The audit doesn't track its own task list — progress comes from the audit ledger plus `[STATUS]` lines. + +## Live activity — `[STATUS]` + +The "Working on …" banner reads from `[STATUS]` lines you emit in plain text. Whenever you start a new sub-step, write a line like: + +``` +[STATUS] Scanning manifests +``` + +The wizard intercepts these and updates the spinner. Use them freely — they are cheap. Each step file lists the exact `[STATUS]` strings to emit at each sub-step. + +## Audit checks ledger + +The ledger lives at `.posthog-audit-checks.json` and is rendered live in the "Audit plan" tab. It is owned by MCP tools — **never `Write` this file directly**: + +- `mcp__wizard-tools__audit_resolve_checks({ updates })` — patch one or more checks by `id`. Each `update` is `{ id, status, file?, details? }`. Batch updates from the same step into a single call. + +All audit ledger calls are atomic and serialize internally — **concurrent calls from parallel subagents cannot lose updates**, so feel free to fan out runtime checks across `Agent` subagents when a step says so. + +### Check entry shape + +- `id` — stable kebab-case slug. Reuse the existing seeded ids exactly when calling `audit_resolve_checks`. +- `area` — short group name. The current core workflow uses `Installation`, `Identification`, and `Event Capture`. +- `label` — short human name. +- `status` — `pending` | `pass` | `error` | `warning` | `suggestion`. +- `file` — optional `path:line` for findings tied to a location. +- `details` — optional one-line explanation. + +After the report is written (Step 5), delete `.posthog-audit-checks.json`. + +## Severity levels + +- `error`: Must fix. Broken functionality, data corruption, or security issue. +- `warning`: Should fix. Pattern that causes subtle bugs or data-quality problems. +- `suggestion`: Nice to have. Best-practice improvement. + +## Key principles + +- **Read-only**: Do not edit project source files. The only file you create is the audit report. +- **Evidence-based**: Reference specific `file:line` for every non-pass finding. +- **Actionable**: Every finding states what to fix and how. + +## Abort statuses + +Report abort states with `[ABORT]` prefixed messages. The wizard catches these and terminates the run — do not halt yourself. +- No PostHog SDK found + +## Framework guidelines + +_No specific framework guidelines._ diff --git a/.claude/skills/audit/references/1-version.md b/.claude/skills/audit/references/1-version.md new file mode 100644 index 0000000..13784b0 --- /dev/null +++ b/.claude/skills/audit/references/1-version.md @@ -0,0 +1,90 @@ +# Step 1 — SDK installed + SDK up-to-date + +**Read ONLY this file.** Do not read any other reference file until this one tells you to. + +This step is intentionally narrow. It runs **before any other project work**. Resolve exactly two checks: `sdk-installed` and `sdk-up-to-date`. **Do not** read source code, locate init sites, look at `.env*` files, or scan for identify/capture call sites in this step — that all belongs to later steps. + +## Status + +Emit: + +``` +[STATUS] Scanning manifests +[STATUS] Checking SDK version +``` + +## Action + +### a. Find the PostHog SDK + +`Glob` for the project's dependency manifests across every language PostHog ships an SDK for. The full list: + +- `package.json` — npm / pnpm / yarn (Node, web, React, Next.js, Nuxt, Vue, Svelte, Angular, React Native, Expo) +- `requirements.txt`, `pyproject.toml`, `Pipfile`, `setup.py` — Python (Django, Flask, FastAPI, etc.) +- `Gemfile` — Ruby / Ruby on Rails +- `composer.json` — PHP / Laravel +- `go.mod` — Go +- `build.gradle`, `build.gradle.kts`, `pom.xml` — Java / Android +- `Podfile`, `Package.swift` — iOS / Swift +- `pubspec.yaml` — Flutter / Dart +- `*.csproj` — .NET +- `mix.exs` — Elixir + +Read enough of them to identify which PostHog SDK the project uses, what version, and what framework it sits on top of. + +If no PostHog SDK is anywhere in the project, emit `[ABORT] No PostHog SDK found` and stop. The wizard catches `[ABORT]` and terminates the run. + +### b. Install the matching integration skill + +Once you know the SDK + framework, install the matching integration skill so the rest of the audit has framework-specific install docs to reference instead of guessing: + +1. Call `mcp__wizard-tools__load_skill_menu({ category: "integration" })` once to list available integration skill IDs. +2. Call `mcp__wizard-tools__install_skill({ skillId: "" })` with the **single** ID that matches the framework you detected. Pick one — do not install multiple. + +If no integration skill matches the framework, skip this step. Step 2 will fall back to general framework knowledge. + +### c. Check latest published version + +For each detected SDK, run `Bash` once to look up the latest published version. Use the command that matches the SDK's registry: + +- **npm** (JS/TS, Node, React, Next.js, Nuxt, Vue, Svelte, Angular, React Native, Expo): `npm view version` +- **PyPI** (Python): `pip index versions ` (or `pip show ` if `index` is unavailable) +- **RubyGems** (Ruby / Rails): `gem search ^$ -r` +- **Packagist** (PHP / Laravel): `composer show --latest --available --format=json` +- **Go modules** (Go): `curl -s https://proxy.golang.org//@latest` (returns JSON with the latest `Version`) +- **Maven Central** (Java / Android): `curl -s "https://search.maven.org/solrsearch/select?q=g:+AND+a:&rows=1&wt=json"` and read `.response.docs[0].latestVersion` +- **CocoaPods** (iOS / Swift): `pod search ` (or check `https://cdn.cocoapods.org/all_pods_versions___.txt` for the spec mirror) +- **Swift Package Manager** (Swift): `gh release list --repo posthog/posthog-ios --limit 1` (SwiftPM resolves from GitHub tags) +- **pub.dev** (Flutter / Dart): `curl -s https://pub.dev/api/packages/ | jq -r .latest.version` +- **NuGet** (.NET): `curl -s https://api.nuget.org/v3-flatcontainer//index.json | jq -r '.versions[-1]'` +- **Hex** (Elixir): `mix hex.info ` + +## Resolution rules + +`sdk-installed`: +- `pass`: at least one PostHog SDK in a manifest. Record SDK + version in `details`. + +`sdk-up-to-date`: +- `pass`: at the latest minor. +- `suggestion`: patch-only behind. +- `warning`: more than one minor behind. +- `error`: one or more major versions behind. + +## Resolve + +Single call to `mcp__wizard-tools__audit_resolve_checks` with two updates and **nothing else**: + +``` +{ + "updates": [ + { "id": "sdk-installed", "status": "pass", "details": "@" }, + { "id": "sdk-up-to-date", "status": "pass|suggestion|warning|error", "details": "installed , latest " } + ] +} +``` + +Do not include `init-correct` in this call — it's resolved in Step 2. + +--- + +**Upon completion, continue with:** [2-init.md](2-init.md) \ No newline at end of file diff --git a/.claude/skills/audit/references/2-init.md b/.claude/skills/audit/references/2-init.md new file mode 100644 index 0000000..fbc7d7a --- /dev/null +++ b/.claude/skills/audit/references/2-init.md @@ -0,0 +1,52 @@ +# Step 2 — Init correctness + +**Read ONLY this file.** Do not read any other reference file until this one tells you to. + +This step resolves two checks: `init-correct` and `init-not-duplicated`. Manifests and SDK versions are already resolved (Step 1). Identification call sites belong to Step 3 and event-capture call sites to Step 4 — do not scan for them here. + +## Status + +Emit: + +``` +[STATUS] Locating PostHog initialization +``` + +## Action + +Locate every PostHog init site by issuing one `Grep` for `posthog\.init\(|new PostHog\(|posthog\.Posthog\(|Posthog\(` plus whatever `Read` calls are needed. Confirm at least one init exists, runs in the right runtime for the detected SDK + framework, and sources its token from an env variable (not hardcoded). Also check `.env*` files to confirm the token env var is actually set. Reverse-proxy / `api_host` configuration belongs to Step 4 — don't evaluate it here. + +Use the detected SDK + framework from Step 1 to know what to look for: the canonical init filename, runtime, and shape vary by framework. If the host project already ships a PostHog integration skill, use that as the source of truth. Skills are typically under `.claude/skills/`; if that directory doesn't exist (some projects keep skills under `agents/skills/`, plain `skills/`, etc.), discover any candidates with one `Glob` pattern: `**/skills/**/SKILL.md`. Read the matching skill before judging. + +When no integration skill is available, rely on general framework knowledge — and stay conservative on `init-correct` (prefer `warning` over `error` when the convention is unclear). + +For `init-not-duplicated`, count the init sites the grep found and group them by runtime (browser vs. server vs. mobile). The check fires when more than one init site exists in the same runtime — the browser bundle running `posthog.init()` twice will race to stamp `$set_once` properties like `$initial_pathname` and `$initial_referrer`, corrupting attribution at scale. A browser init plus a server init is fine; two browser inits is not. Treat conditional inits inside an `if (typeof window === 'undefined')` / `if (typeof window !== 'undefined')` branch on the same code path as ONE logical site per runtime (only one branch executes). + +## Resolution rules + +`init-correct`: +- `pass`: at least one init present, env-sourced token, runtime-appropriate location. +- `error`: init missing, hardcoded token, or wrong runtime (e.g. server-only init for a browser-side framework). +- `warning`: init present but in a non-canonical location for the framework. + +`init-not-duplicated`: +- `pass`: at most one init site per runtime. +- `warning`: two or more init sites in the same runtime that could plausibly both execute. Report the file:line of each and which runtime. The dual-init `$set_once` race corrupted attribution at scale at one multi-SDK SaaS customer (~165k blocked merges/day). +- `error`: two or more browser-runtime init sites confirmed to run on the same page (e.g. one in the root layout / provider, another in a child component that always mounts). + +## Resolve + +Single call to `mcp__wizard-tools__audit_resolve_checks` with two updates: + +``` +{ + "updates": [ + { "id": "init-correct", "status": "pass|error|warning", "file": "", "details": "..." }, + { "id": "init-not-duplicated", "status": "pass|warning|error", "file": "", "details": "..." } + ] +} +``` + +--- + +**Upon completion, continue with:** [3-identification.md](3-identification.md) \ No newline at end of file diff --git a/.claude/skills/audit/references/3-identification.md b/.claude/skills/audit/references/3-identification.md new file mode 100644 index 0000000..1e310d6 --- /dev/null +++ b/.claude/skills/audit/references/3-identification.md @@ -0,0 +1,118 @@ +# Step 3 — Identification + +**Read ONLY this file.** Do not read any other reference file until this one tells you to. + +This step resolves four identification checks **in parallel**, one subagent per check: + +- `identify-stable-distinct-id` +- `identify-not-late` +- `cross-runtime-distinct-id` +- `identify-reset-on-logout` + +Each subagent owns its own grep, reads, evaluates its single rule, and emits one `audit_resolve_checks` call with one update. The ledger's mutex serializes concurrent writes — there's no race. + +## Status + +Emit before dispatching: + +``` +[STATUS] Auditing identification +``` + +## Action — dispatch four subagents in one message + +Make **four `Agent` tool calls in a single message** so they run concurrently. Wait for all four to return, then continue to `4-event-capture.md`. Do not run any other tools between dispatch and the next step. + +The bundled `identify-users.md` reference holds PostHog's authoritative guidance on `distinct_id`, `identify()` ordering, and cross-runtime identity. It's typically at `.claude/skills/audit/references/identify-users.md`; if that path doesn't exist, discover it with `Glob` `**/skills/audit/references/identify-users.md`. Each subagent reads it once before judging. + +### Task A — `identify-stable-distinct-id` + +`description`: `Audit identify-stable-distinct-id` + +`prompt`: +``` +You are an audit subagent. Resolve exactly one rule and return: identify-stable-distinct-id. + +Read this skill's bundled `identify-users.md` reference once (typically `.claude/skills/audit/references/identify-users.md`; otherwise discover with `Glob` `**/skills/audit/references/identify-users.md`). + +Run **one** Grep: `posthog\.identify\(`. Read each file that contains a hit, once. Inspect the first argument passed to identify(). + +Rule: +- distinct_id must be a stable identifier (auth user id, account id), not a session UUID, ephemeral cookie, or device-only id. +- pass: sources from authenticated user (session.user.id, auth.uid(), etc.) +- error: sources from a session, request, or device id that resets +- warning: source unclear — flag for human review + +Emit one `mcp__wizard-tools__audit_resolve_checks` call with a single update for id `identify-stable-distinct-id`, including `file` (path:line) and `details` (one-line explanation). Return when the call completes. Do not write the audit report. +``` + +### Task B — `identify-not-late` + +`description`: `Audit identify-not-late` + +`prompt`: +``` +You are an audit subagent. Resolve exactly one rule and return: identify-not-late. + +Read this skill's bundled `identify-users.md` reference once (typically `.claude/skills/audit/references/identify-users.md`; otherwise discover with `Glob` `**/skills/audit/references/identify-users.md`). + +Run **two** Greps in parallel: +- `posthog\.identify\(` — where identity is established +- `posthog\.capture\(|getFeatureFlag\(|isFeatureEnabled\(` — where captures and flag evals happen + +Read each file that contains a hit, once. Compare the timing/ordering of identify() against the surrounding capture / flag-eval calls. + +Rule: +- identify() must be called before any posthog.capture for that user, and before any feature-flag eval depending on user identity. +- pass: identify runs at session start / right after login. Captures and flag evals come after. +- warning: identify runs lazily (e.g. settings-page mount), so early captures and flag evals are anonymous. + +Emit one `mcp__wizard-tools__audit_resolve_checks` call with a single update for id `identify-not-late`, including `file` (path:line of the identify call) and `details` (one-line explanation). Return when the call completes. Do not write the audit report. +``` + +### Task C — `cross-runtime-distinct-id` + +`description`: `Audit cross-runtime-distinct-id` + +`prompt`: +``` +You are an audit subagent. Resolve exactly one rule and return: cross-runtime-distinct-id. + +Read this skill's bundled `identify-users.md` reference once (typically `.claude/skills/audit/references/identify-users.md`; otherwise discover with `Glob` `**/skills/audit/references/identify-users.md`). + +Run **one** Grep: `posthog\.init\(|new PostHog\(|posthog\.Posthog\(|Posthog\(` — locate every PostHog initialization across runtimes. Read each file that contains a hit, once. Determine whether both client and server runtimes initialize PostHog, and if so, how distinct_id flows between them. + +Rule: +- If both client and server runtimes call PostHog, the same distinct_id must be used on both sides for the same user. +- pass: server-side captures source the client's distinct_id (cookie, session token, or explicit hand-off). +- error: server-side captures use a different identifier scheme. +- Skip (`pass` with details: "single runtime"): only one runtime initializes PostHog. + +Emit one `mcp__wizard-tools__audit_resolve_checks` call with a single update for id `cross-runtime-distinct-id`, including `file` (path:line of the most relevant init or capture site) and `details` (one-line explanation). Return when the call completes. Do not write the audit report. +``` + +### Task D — `identify-reset-on-logout` + +`description`: `Audit identify-reset-on-logout` + +`prompt`: +``` +You are an audit subagent. Resolve exactly one rule and return: identify-reset-on-logout. + +Read this skill's bundled `identify-users.md` reference once (typically `.claude/skills/audit/references/identify-users.md`; otherwise discover with `Glob` `**/skills/audit/references/identify-users.md`). + +Locate logout, sign-out, and account-switching flows by issuing whatever `Grep` and `Read` calls are needed in parallel. Determine whether those flows clear PostHog state with `posthog.reset()`. + +Rule: +- Logout or account-switching flows should call `posthog.reset()`. Without a reset, when user B logs in on the same device after user A, PostHog's anonymous ID is shared and the next `identify()` can merge both accounts into one person. +- pass: every detected logout/account-switch flow calls `posthog.reset()`. +- error: a logout/account-switch flow is missing `posthog.reset()`. +- Skip (`pass` with details: "no logout/account-switch flow found"): no detectable logout/account-switch flow exists. +- note: `posthog.reset(true)` is valid when a completely clean device ID reset is required. + +Emit one `mcp__wizard-tools__audit_resolve_checks` call with a single update for id `identify-reset-on-logout`, including `file` (path:line of the most relevant logout or reset site) and `details` (one-line explanation). Return when the call completes. Do not write the audit report. +``` + +--- + +**Upon completion, continue with:** [4-event-capture.md](4-event-capture.md) \ No newline at end of file diff --git a/.claude/skills/audit/references/4-event-capture.md b/.claude/skills/audit/references/4-event-capture.md new file mode 100644 index 0000000..6bb1ad9 --- /dev/null +++ b/.claude/skills/audit/references/4-event-capture.md @@ -0,0 +1,95 @@ +# Step 4 — Event capture + +**Read ONLY this file.** Do not read any other reference file until this one tells you to. + +This step resolves three event-capture checks **in parallel**, one subagent per check: + +- `capture-event-names-static` +- `capture-uses-proxy` +- `capture-growth-events` + +Each subagent owns its own grep, reads, evaluates its single rule, and emits one `audit_resolve_checks` call with one update. The ledger's mutex serializes concurrent writes. + +## Status + +Emit before dispatching: + +``` +[STATUS] Auditing event capture +``` + +## Action — dispatch three subagents in one message + +Make **three `Agent` tool calls in a single message** so they run concurrently. Wait for all three to return, then continue to `5-report.md`. Do not run any other tools between dispatch and the next step. + +The bundled `best-practices.md` reference holds PostHog's authoritative guidance on event-name shape, reverse-proxy setup, and growth-event coverage. It's typically at `.claude/skills/audit/references/best-practices.md`; if that path doesn't exist, discover it with `Glob` `**/skills/audit/references/best-practices.md`. Each subagent reads it once before judging. + +### Task A — `capture-event-names-static` + +`description`: `Audit capture-event-names-static` + +`prompt`: +``` +You are an audit subagent. Resolve exactly one rule and return: capture-event-names-static. + +Read this skill's bundled `best-practices.md` reference once (typically `.claude/skills/audit/references/best-practices.md`; otherwise discover with `Glob` `**/skills/audit/references/best-practices.md`). + +Run **one** Grep: `posthog\.capture\(`. Read each file that contains a hit, once. Inspect the first argument of every capture() call. + +Rule: +- Event names in posthog.capture("name", …) must be static strings, not template literals or dynamic variables. +- pass: all capture calls use string literals. +- error: any call uses a template literal or variable as the event name. + +Emit one `mcp__wizard-tools__audit_resolve_checks` call with a single update for id `capture-event-names-static`, including `file` (path:line of the first violation if any, otherwise of a representative capture call) and `details` (one-line explanation). Return when the call completes. Do not write the audit report. +``` + +### Task B — `capture-uses-proxy` + +`description`: `Audit capture-uses-proxy` + +`prompt`: +``` +You are an audit subagent. Resolve exactly one rule and return: capture-uses-proxy. + +Read this skill's bundled `best-practices.md` reference once (typically `.claude/skills/audit/references/best-practices.md`; otherwise discover with `Glob` `**/skills/audit/references/best-practices.md`). + +Run **one** Grep: `api_host`. Read each file that contains a hit, once. Determine the configured ingest host the SDK posts to, and whether any browser runtime initializes PostHog at all. + +Rule: +- A reverse proxy fronts PostHog's ingest endpoint via `api_host`, so events keep flowing when ad/tracking blockers would otherwise drop them. Without one, a meaningful share of browser captures never reach PostHog. +- pass: `api_host` resolves to a first-party domain on the project's own infra (e.g. `e.example.com`, `posthog.example.com`, `/ingest`-style same-origin path, or a known proxy SaaS like `app.example.com/relay-...`). +- warning: `api_host` is the default PostHog host (`https://us.i.posthog.com`, `https://eu.i.posthog.com`, `https://app.posthog.com`, or omitted entirely so the SDK default applies). +- Skip (`pass` with details: "server-only SDK"): only server-side runtimes init PostHog — a proxy isn't needed when no browser sends captures. + +Emit one `mcp__wizard-tools__audit_resolve_checks` call with a single update for id `capture-uses-proxy`, including `file` (path:line of the init that sets api_host) and `details` (one-line explanation). Return when the call completes. Do not write the audit report. +``` + +### Task C — `capture-growth-events` + +`description`: `Audit capture-growth-events` + +`prompt`: +``` +You are an audit subagent. Resolve exactly one rule and return: capture-growth-events. + +Read this skill's bundled `best-practices.md` reference once (typically `.claude/skills/audit/references/best-practices.md`; otherwise discover with `Glob` `**/skills/audit/references/best-practices.md`). + +Run **two** Greps in parallel: +- `posthog\.capture\(` — explicit capture calls +- `signup|signin|register|checkout|purchase|subscribe|onboard` — likely growth-funnel surfaces + +Read each file that contains a hit, once. Cross-reference: do the growth-funnel surfaces actually emit explicit capture calls? + +Rule: +- Signup, activation/first-key-action, and purchase/subscription should be tracked explicitly. Autocapture isn't enough for funnels. +- pass: at least signup + one activation + (purchase or subscribe) are captured explicitly. +- warning: one or more growth events missing — list which. +- Skip (`pass` with details: "no auth/billing paths detected"): no detectable signup/billing surfaces. + +Emit one `mcp__wizard-tools__audit_resolve_checks` call with a single update for id `capture-growth-events`, including `file` (path:line of the most relevant capture or growth-surface site) and `details` (one-line explanation, listing missing growth events when applicable). Return when the call completes. Do not write the audit report. +``` + +--- + +**Upon completion, continue with:** [5-report.md](5-report.md) \ No newline at end of file diff --git a/.claude/skills/audit/references/5-report.md b/.claude/skills/audit/references/5-report.md new file mode 100644 index 0000000..417fc44 --- /dev/null +++ b/.claude/skills/audit/references/5-report.md @@ -0,0 +1,373 @@ +# Step 5 — Generate the audit report (and upload it to a notebook) + +The audit report is rendered **directly from `.posthog-audit-checks.json`** — that file is the source of truth. Every check the wizard seeded ends up in the report, even passes; nothing is invented. After the markdown is written to disk, this step also writes the report into a PostHog notebook so it's shareable from inside PostHog. + +## Status + +Emit, in order: + +``` +[STATUS] Writing audit report +[STATUS] Building notebook payload +[STATUS] Uploading report to notebook +``` + +## MCP tools + +| MCP tool | When | Use | +|----------|------|-----| +| `mcp__posthog-wizard__notebooks-create` | (a) of "Upload to a PostHog notebook" | Create the notebook with a small placeholder skeleton (title + section headings + placeholder paragraphs). One call. | +| `mcp__posthog-wizard__notebook-edit` | (b) of "Upload to a PostHog notebook" | Replace one placeholder paragraph in the cloud notebook with a real ProseMirror node. **Called many times** (one per placeholder). Required because the model can't emit the full assembled tree in a single `notebooks-create` tool_use input — it self-truncates. | +| `mcp__posthog-wizard__notebooks-retrieve` | (c) of "Upload to a PostHog notebook" | Read the cloud notebook back to verify every placeholder has been replaced. | + +Load all three via `ToolSearch select:mcp__posthog-wizard__notebooks-create,mcp__posthog-wizard__notebook-edit,mcp__posthog-wizard__notebooks-retrieve` once, right before the upload sub-step — not at the top, so step 5 stays cheap when MCP isn't needed yet. `mcp__wizard-tools__audit_resolve_checks` is already loaded — you'll use it again after the upload. + +If `notebook-edit` doesn't appear in the search results, the project's `notebooks-collaboration` feature flag isn't enabled. Skip the notebook-upload sub-step entirely; emit `Notebook upload skipped: notebook-edit unavailable. The local report at posthog-audit-report.md is still the source of truth.` and resolve `upload-notebook` to `suggestion` with that reason. + +## Action + +`Read` the ledger once, then build the report **incrementally** — `Write` a skeleton with placeholder markers, then `Edit` each placeholder with its real section in a separate turn. **Do not compose the whole report in one turn.** A single sustained generation of the full document routinely drops the LLM streaming connection around the 10-minute mark; chunking via Write + Edit keeps every turn short and resets the SSE timer at each tool call. The on-disk file is the source of truth, so a dropped turn loses at most one section, not the whole report. + +**Do not delete `.posthog-audit-checks.json` yet** — the notebook-upload sub-step still resolves a ledger row. The cleanup happens at the very end of this step. + +The report has four sections in this order: + +1. **Summary** — one-paragraph overview, severity counts, and a problematic-items table. +2. **Recommended actions** — prioritized fixes with `file:line` and a docs link per item. +3. **Full audit** — every check the wizard ran, grouped by `area`, including passes. +4. **About this audit** — a short closing block explaining what the audit covered and how to interpret the report. *Static text — already baked into the skeleton.* + +For the Full audit section, group rows dynamically by each distinct `area` value in the ledger, preserving first-seen area order from the JSON. Today the core audit produces three areas — **Installation**, **Identification**, **Event Capture** — but the report must not hard-code that list; render whatever areas appear. + +For each area, write a one-paragraph framing immediately under the area heading, then the table. Use the canonical copy below verbatim when the area name matches; otherwise write a one-sentence summary derived from the area's check labels. + +### a. Write the skeleton + +One `Write` to `posthog-audit-report.md` with section headings and HTML-comment placeholders for the body of each non-static section. The About-this-audit text is identical every run, so it's baked in directly. + +```markdown +# PostHog Audit Report + +## Summary + + + +## Recommended actions + + + +## Full audit + + + +## About this audit + +The PostHog wizard runs a five-stage chain: SDK installation → init correctness → identification → event capture → this report. Each stage resolves one or more checks against the project's source tree, recording every result — pass or otherwise — in the ledger this report was generated from. + +- `error` items break correctness now (events lost, identity broken). Fix first. +- `warning` items work today but cause subtle data-quality bugs. Fix when convenient. +- `suggestion` items are best-practice improvements with measurable upside. + +Re-run `posthog-wizard audit` after applying fixes to refresh the ledger. +``` + +This Write should be small — just the structure above. Don't compose section bodies yet. + +### b. Fill the Summary section + +One `Edit`: + +- `old_string`: `` +- `new_string`: the Summary body — one-paragraph overview, then the counts list, then the problematic-items table (or the "no issues" line). See the Summary template below for the exact shape. + +Output for this turn is bounded by the Summary content alone (~500 tokens for most projects). + +### c. Fill the Recommended actions section + +One `Edit`: + +- `old_string`: `` +- `new_string`: the numbered list of actions in the format below, or `_Nothing to fix._` if there are none. + +### d. Fill the Full audit section + +One `Edit`: + +- `old_string`: `` +- `new_string`: the per-area headings + paragraphs + tables + per-area `#### Assumptions and blind spots` subsection, in ledger order. The blind-spots subsection lives directly under each area's table, following the per-area body template below. + +If the Full audit section is large (many areas, many checks), you may split it across multiple Edits by including per-area placeholders in the original skeleton and filling each with one Edit. Most audits fit in one Edit. + +## Section body templates + +Use these shapes when computing the `new_string` for each Edit above. + +### Summary body + +```markdown +[1–2 sentence overview: runtimes covered (client/server/both), overall health, and which areas had issues.] + +**Counts** + +- **Errors**: [N] (must fix) +- **Warnings**: [N] (should fix) +- **Suggestions**: [N] (nice to have) +- **Passes**: [N] + +**Problematic items** _(only `error`, `warning`, `suggestion` — no passes)_ + +| Severity | Area | Check | File | Details | +|----------|------|-------|------|---------| +| `error` | Installation | [label] | [file:line] | [details] | +``` + +If there are no problematic items, replace the table with `_No issues found — your PostHog setup looks healthy._`. + +### Recommended actions body + +Numbered list, ordered by severity (errors → warnings → suggestions), then by ledger order within a severity. Each item is **three sentences**, in this order: + +1. **What's wrong** — the finding, written as a one-sentence diagnosis derived from `details`. +2. **Why it matters** — one sentence on the data-quality consequence: which downstream artifact (funnels, retention, person count, billing, replays, experiments, etc.) this finding contaminates if left alone, and how. Use the canonical "why it matters" copy below verbatim when the check id matches; otherwise write one sentence rooted in the check's rule. +3. **How to fix** — one short imperative sentence pointing at `file:line` and the concrete change. End with a docs link. + +Format: + +```markdown +1. **[Area] · [label]** — [what's wrong]. _Why it matters:_ [why-it-matters]. _Fix:_ [how-to-fix at `file:line`]. See [docs]([area docs url]). +``` + +If there are no actions, write `_Nothing to fix._`. + +### Full audit body + +For each `area` from the ledger, in first-seen order: + +```markdown +### [Area from ledger] + +[Canonical paragraph for the area, see "Canonical area copy" below. If the area is not in the canonical list, write one short sentence summarizing what its checks verify.] + +| Check | Status | File | Details | +|-------|--------|------|---------| +| [label] | [status] | [file] | [details] | + +#### Assumptions and blind spots + +[Per the investigation standards in `posthog-best-practices/references/investigation-standards.md`, standard 3. ≤4 sentences answering: which code paths were not checked, which runtime assumptions are unproven by static code, what alternative explanations exist for the patterns found, and what to verify in the live PostHog project to confirm the most important findings. When the area produced only `pass` rows, write `_No findings to qualify; the standard checks for this area passed cleanly._` instead.] +``` + +After the report is written, emit a line so the wizard can surface the path to the user: + +``` +Created audit report: +``` + +### Resolve `write-report` + +Flip the `write-report` row to `pass` now that the markdown file exists on disk. The notebook-upload sub-step that follows can take a while (large ProseMirror payload), and resolving this row first lets the wizard sidebar advance to "Upload notebook" so the user can see what's happening. + +```json +{ + "updates": [ + { "id": "write-report", "status": "pass" } + ] +} +``` + +## Upload to a PostHog notebook + +The markdown report on disk is the source of truth. The notebook is a shareable, in-PostHog mirror so the reader can comment, link to it from insights, and discuss it without leaving the product. + +### Why two MCP tools instead of one + +Earlier versions of this skill called `notebooks-create` once with the full assembled ProseMirror tree as the `content` argument. The assistant turn that emits that tool_use has to *generate the tree as output tokens*, even if it's just copying from a file it just read. For a 12-check audit with tables and bullet lists the full tree is several thousand tokens — past the per-turn output budget for some runs. The model self-truncates and the notebook ships with sections missing. + +The fix is to **build the cloud notebook incrementally**. `notebooks-create` carries only a small skeleton (title + section headings + placeholder paragraphs). Then `notebook-edit` replaces one placeholder paragraph at a time with the real ProseMirror node. Each `notebook-edit` tool_use input is bounded — never more than one block-level node — so it always fits in one turn. The notebook is complete only after the last edit lands. + +There's no local notebook payload scratch file in this design. Section content is computed on demand from the ledger and the on-disk report. + +### Orientation: re-read the report + +`Read` `posthog-audit-report.md` once. You'll use it as a reference for what content to send in each edit. Don't translate the whole thing up front — translate per placeholder, as you fill each one. + +### Node mapping (apply per placeholder as you `notebook-edit`) + +| Markdown | ProseMirror node | +|---|---| +| `# / ## / ### heading` | `{"type":"heading","attrs":{"level":},"content":[{"type":"text","text":""}]}` | +| paragraph | `{"type":"paragraph","content":[{"type":"text","text":"<...>"}]}` | +| bulleted list | `{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"type":"text","text":""}]}]}, ...]}` | +| numbered list | `{"type":"orderedList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"type":"text","text":""}]}]}, ...]}` | +| inline `code` | text node with a `code` mark: `{"type":"text","marks":[{"type":"code"}],"text":""}` | +| `**bold**` | text node with a `bold` mark | +| `[label](url)` | text node with a `link` mark: `{"type":"text","marks":[{"type":"link","attrs":{"href":""}}],"text":"