Skip to content

Commit 8f3b84b

Browse files
willwearingclaude
andcommitted
chore: clean up untracked working-tree files
Four unrelated leftovers had been sitting untracked in the working tree. Resolving each rather than carrying them further. Deleted backend/debug-mastery.ts and debug-mastery2.ts. Standalone Prisma scripts from April that dump one hardcoded user's mastery state to the console. Nothing imports them and the UUIDs they target are stale. Deleted backend/src/config/cors-origin.ts and its spec. An unwired second implementation of CORS origin checking. main.ts imports buildStaticOrigins, createCorsOriginGuard and isOriginAllowed from ./config/cors, and nothing referenced createCorsOriginHandler. Keeping a duplicate of a security control next to the live one risks someone wiring up the wrong one later. The behaviour it was exploring, answering a disallowed origin by omitting CORS headers instead of returning 403, can be revisited in config/cors.ts directly if that turns out to be wanted. Gitignored .posthog-audit-checks.json. Regenerated wizard output, and the committed copy was already stale on SDK versions. Committed .claude/skills/audit and integration-nextjs-app-router. Same shape as the tracked integration-javascript_node skill; they were just never added. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 3e24444 commit 8f3b84b

19 files changed

Lines changed: 2867 additions & 0 deletions

.claude/skills/audit/.posthog-wizard

Whitespace-only changes.

.claude/skills/audit/SKILL.md

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
---
2+
name: audit
3+
description: Audit an existing PostHog integration for correctness and best practices
4+
metadata:
5+
author: PostHog
6+
version: 1.26.0
7+
---
8+
9+
# PostHog Audit
10+
11+
This skill audits an existing PostHog integration for **data integrity** in event capture and identification. **Read-only** — the only file you create is the final audit report.
12+
13+
Perform the checks described in the referenced skills and only the events referenced in the skills.
14+
15+
## Workflow
16+
17+
The audit runs as a 5-step chain: Installation (SDK + version) → init correctness → identification → event capture → report (which also uploads the report to a PostHog notebook). Each step file ends with a pointer to the next. Follow them in the order they are written. You must resolve them in order before any source-tree exploration.
18+
19+
The audit ledger is already seeded with the 11 pending checks (10 correctness checks plus `upload-notebook`, which the report step resolves after mirroring the markdown into a PostHog notebook). Use `mcp__wizard-tools__audit_resolve_checks` to patch each one as you finish it.
20+
21+
**Start by reading the path relative to this file at `references/1-version.md`.** Do not Glob, ls, or find the skill directory. Do not preload future steps. Do not re-read a step file once you've moved past it. Do not re-read SKILL.md.
22+
23+
`ToolSearch` is only for loading a tool by exact name when the SDK has it deferred (e.g. `select:Grep`). Do **not** use it to browse for other tools — every tool the audit needs (`Glob`, `Grep`, `Read`, `Write`, `Bash`, and the named `mcp__wizard-tools__audit_*` tools) is already named in this skill.
24+
25+
**Do not call `TaskCreate` / `TaskUpdate` / `TaskGet` / `TaskList`.** The audit doesn't track its own task list — progress comes from the audit ledger plus `[STATUS]` lines.
26+
27+
## Live activity — `[STATUS]`
28+
29+
The "Working on …" banner reads from `[STATUS]` lines you emit in plain text. Whenever you start a new sub-step, write a line like:
30+
31+
```
32+
[STATUS] Scanning manifests
33+
```
34+
35+
The wizard intercepts these and updates the spinner. Use them freely — they are cheap. Each step file lists the exact `[STATUS]` strings to emit at each sub-step.
36+
37+
## Audit checks ledger
38+
39+
The ledger lives at `.posthog-audit-checks.json` and is rendered live in the "Audit plan" tab. It is owned by MCP tools — **never `Write` this file directly**:
40+
41+
- `mcp__wizard-tools__audit_resolve_checks({ updates })` — patch one or more checks by `id`. Each `update` is `{ id, status, file?, details? }`. Batch updates from the same step into a single call.
42+
43+
All audit ledger calls are atomic and serialize internally — **concurrent calls from parallel subagents cannot lose updates**, so feel free to fan out runtime checks across `Agent` subagents when a step says so.
44+
45+
### Check entry shape
46+
47+
- `id` — stable kebab-case slug. Reuse the existing seeded ids exactly when calling `audit_resolve_checks`.
48+
- `area` — short group name. The current core workflow uses `Installation`, `Identification`, and `Event Capture`.
49+
- `label` — short human name.
50+
- `status``pending` | `pass` | `error` | `warning` | `suggestion`.
51+
- `file` — optional `path:line` for findings tied to a location.
52+
- `details` — optional one-line explanation.
53+
54+
After the report is written (Step 5), delete `.posthog-audit-checks.json`.
55+
56+
## Severity levels
57+
58+
- `error`: Must fix. Broken functionality, data corruption, or security issue.
59+
- `warning`: Should fix. Pattern that causes subtle bugs or data-quality problems.
60+
- `suggestion`: Nice to have. Best-practice improvement.
61+
62+
## Key principles
63+
64+
- **Read-only**: Do not edit project source files. The only file you create is the audit report.
65+
- **Evidence-based**: Reference specific `file:line` for every non-pass finding.
66+
- **Actionable**: Every finding states what to fix and how.
67+
68+
## Abort statuses
69+
70+
Report abort states with `[ABORT]` prefixed messages. The wizard catches these and terminates the run — do not halt yourself.
71+
- No PostHog SDK found
72+
73+
## Framework guidelines
74+
75+
_No specific framework guidelines._
Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# Step 1 — SDK installed + SDK up-to-date
2+
3+
**Read ONLY this file.** Do not read any other reference file until this one tells you to.
4+
5+
This step is intentionally narrow. It runs **before any other project work**. Resolve exactly two checks: `sdk-installed` and `sdk-up-to-date`. **Do not** read source code, locate init sites, look at `.env*` files, or scan for identify/capture call sites in this step — that all belongs to later steps.
6+
7+
## Status
8+
9+
Emit:
10+
11+
```
12+
[STATUS] Scanning manifests
13+
[STATUS] Checking SDK version
14+
```
15+
16+
## Action
17+
18+
### a. Find the PostHog SDK
19+
20+
`Glob` for the project's dependency manifests across every language PostHog ships an SDK for. The full list:
21+
22+
- `package.json` — npm / pnpm / yarn (Node, web, React, Next.js, Nuxt, Vue, Svelte, Angular, React Native, Expo)
23+
- `requirements.txt`, `pyproject.toml`, `Pipfile`, `setup.py` — Python (Django, Flask, FastAPI, etc.)
24+
- `Gemfile` — Ruby / Ruby on Rails
25+
- `composer.json` — PHP / Laravel
26+
- `go.mod` — Go
27+
- `build.gradle`, `build.gradle.kts`, `pom.xml` — Java / Android
28+
- `Podfile`, `Package.swift` — iOS / Swift
29+
- `pubspec.yaml` — Flutter / Dart
30+
- `*.csproj` — .NET
31+
- `mix.exs` — Elixir
32+
33+
Read enough of them to identify which PostHog SDK the project uses, what version, and what framework it sits on top of.
34+
35+
If no PostHog SDK is anywhere in the project, emit `[ABORT] No PostHog SDK found` and stop. The wizard catches `[ABORT]` and terminates the run.
36+
37+
### b. Install the matching integration skill
38+
39+
Once you know the SDK + framework, install the matching integration skill so the rest of the audit has framework-specific install docs to reference instead of guessing:
40+
41+
1. Call `mcp__wizard-tools__load_skill_menu({ category: "integration" })` once to list available integration skill IDs.
42+
2. Call `mcp__wizard-tools__install_skill({ skillId: "<id>" })` with the **single** ID that matches the framework you detected. Pick one — do not install multiple.
43+
44+
If no integration skill matches the framework, skip this step. Step 2 will fall back to general framework knowledge.
45+
46+
### c. Check latest published version
47+
48+
For each detected SDK, run `Bash` once to look up the latest published version. Use the command that matches the SDK's registry:
49+
50+
- **npm** (JS/TS, Node, React, Next.js, Nuxt, Vue, Svelte, Angular, React Native, Expo): `npm view <pkg> version`
51+
- **PyPI** (Python): `pip index versions <pkg>` (or `pip show <pkg>` if `index` is unavailable)
52+
- **RubyGems** (Ruby / Rails): `gem search ^<pkg>$ -r`
53+
- **Packagist** (PHP / Laravel): `composer show <pkg> --latest --available --format=json`
54+
- **Go modules** (Go): `curl -s https://proxy.golang.org/<module>/@latest` (returns JSON with the latest `Version`)
55+
- **Maven Central** (Java / Android): `curl -s "https://search.maven.org/solrsearch/select?q=g:<group>+AND+a:<artifact>&rows=1&wt=json"` and read `.response.docs[0].latestVersion`
56+
- **CocoaPods** (iOS / Swift): `pod search <pkg>` (or check `https://cdn.cocoapods.org/all_pods_versions_<x>_<y>_<z>.txt` for the spec mirror)
57+
- **Swift Package Manager** (Swift): `gh release list --repo posthog/posthog-ios --limit 1` (SwiftPM resolves from GitHub tags)
58+
- **pub.dev** (Flutter / Dart): `curl -s https://pub.dev/api/packages/<pkg> | jq -r .latest.version`
59+
- **NuGet** (.NET): `curl -s https://api.nuget.org/v3-flatcontainer/<pkg>/index.json | jq -r '.versions[-1]'`
60+
- **Hex** (Elixir): `mix hex.info <pkg>`
61+
62+
## Resolution rules
63+
64+
`sdk-installed`:
65+
- `pass`: at least one PostHog SDK in a manifest. Record SDK + version in `details`.
66+
67+
`sdk-up-to-date`:
68+
- `pass`: at the latest minor.
69+
- `suggestion`: patch-only behind.
70+
- `warning`: more than one minor behind.
71+
- `error`: one or more major versions behind.
72+
73+
## Resolve
74+
75+
Single call to `mcp__wizard-tools__audit_resolve_checks` with two updates and **nothing else**:
76+
77+
```
78+
{
79+
"updates": [
80+
{ "id": "sdk-installed", "status": "pass", "details": "<sdk>@<version>" },
81+
{ "id": "sdk-up-to-date", "status": "pass|suggestion|warning|error", "details": "installed <v>, latest <v>" }
82+
]
83+
}
84+
```
85+
86+
Do not include `init-correct` in this call — it's resolved in Step 2.
87+
88+
---
89+
90+
**Upon completion, continue with:** [2-init.md](2-init.md)
Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# Step 2 — Init correctness
2+
3+
**Read ONLY this file.** Do not read any other reference file until this one tells you to.
4+
5+
This step resolves two checks: `init-correct` and `init-not-duplicated`. Manifests and SDK versions are already resolved (Step 1). Identification call sites belong to Step 3 and event-capture call sites to Step 4 — do not scan for them here.
6+
7+
## Status
8+
9+
Emit:
10+
11+
```
12+
[STATUS] Locating PostHog initialization
13+
```
14+
15+
## Action
16+
17+
Locate every PostHog init site by issuing one `Grep` for `posthog\.init\(|new PostHog\(|posthog\.Posthog\(|Posthog\(` plus whatever `Read` calls are needed. Confirm at least one init exists, runs in the right runtime for the detected SDK + framework, and sources its token from an env variable (not hardcoded). Also check `.env*` files to confirm the token env var is actually set. Reverse-proxy / `api_host` configuration belongs to Step 4 — don't evaluate it here.
18+
19+
Use the detected SDK + framework from Step 1 to know what to look for: the canonical init filename, runtime, and shape vary by framework. If the host project already ships a PostHog integration skill, use that as the source of truth. Skills are typically under `.claude/skills/`; if that directory doesn't exist (some projects keep skills under `agents/skills/`, plain `skills/`, etc.), discover any candidates with one `Glob` pattern: `**/skills/**/SKILL.md`. Read the matching skill before judging.
20+
21+
When no integration skill is available, rely on general framework knowledge — and stay conservative on `init-correct` (prefer `warning` over `error` when the convention is unclear).
22+
23+
For `init-not-duplicated`, count the init sites the grep found and group them by runtime (browser vs. server vs. mobile). The check fires when more than one init site exists in the same runtime — the browser bundle running `posthog.init()` twice will race to stamp `$set_once` properties like `$initial_pathname` and `$initial_referrer`, corrupting attribution at scale. A browser init plus a server init is fine; two browser inits is not. Treat conditional inits inside an `if (typeof window === 'undefined')` / `if (typeof window !== 'undefined')` branch on the same code path as ONE logical site per runtime (only one branch executes).
24+
25+
## Resolution rules
26+
27+
`init-correct`:
28+
- `pass`: at least one init present, env-sourced token, runtime-appropriate location.
29+
- `error`: init missing, hardcoded token, or wrong runtime (e.g. server-only init for a browser-side framework).
30+
- `warning`: init present but in a non-canonical location for the framework.
31+
32+
`init-not-duplicated`:
33+
- `pass`: at most one init site per runtime.
34+
- `warning`: two or more init sites in the same runtime that could plausibly both execute. Report the file:line of each and which runtime. The dual-init `$set_once` race corrupted attribution at scale at one multi-SDK SaaS customer (~165k blocked merges/day).
35+
- `error`: two or more browser-runtime init sites confirmed to run on the same page (e.g. one in the root layout / provider, another in a child component that always mounts).
36+
37+
## Resolve
38+
39+
Single call to `mcp__wizard-tools__audit_resolve_checks` with two updates:
40+
41+
```
42+
{
43+
"updates": [
44+
{ "id": "init-correct", "status": "pass|error|warning", "file": "<path:line>", "details": "..." },
45+
{ "id": "init-not-duplicated", "status": "pass|warning|error", "file": "<path:line>", "details": "..." }
46+
]
47+
}
48+
```
49+
50+
---
51+
52+
**Upon completion, continue with:** [3-identification.md](3-identification.md)
Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,118 @@
1+
# Step 3 — Identification
2+
3+
**Read ONLY this file.** Do not read any other reference file until this one tells you to.
4+
5+
This step resolves four identification checks **in parallel**, one subagent per check:
6+
7+
- `identify-stable-distinct-id`
8+
- `identify-not-late`
9+
- `cross-runtime-distinct-id`
10+
- `identify-reset-on-logout`
11+
12+
Each subagent owns its own grep, reads, evaluates its single rule, and emits one `audit_resolve_checks` call with one update. The ledger's mutex serializes concurrent writes — there's no race.
13+
14+
## Status
15+
16+
Emit before dispatching:
17+
18+
```
19+
[STATUS] Auditing identification
20+
```
21+
22+
## Action — dispatch four subagents in one message
23+
24+
Make **four `Agent` tool calls in a single message** so they run concurrently. Wait for all four to return, then continue to `4-event-capture.md`. Do not run any other tools between dispatch and the next step.
25+
26+
The bundled `identify-users.md` reference holds PostHog's authoritative guidance on `distinct_id`, `identify()` ordering, and cross-runtime identity. It's typically at `.claude/skills/audit/references/identify-users.md`; if that path doesn't exist, discover it with `Glob` `**/skills/audit/references/identify-users.md`. Each subagent reads it once before judging.
27+
28+
### Task A — `identify-stable-distinct-id`
29+
30+
`description`: `Audit identify-stable-distinct-id`
31+
32+
`prompt`:
33+
```
34+
You are an audit subagent. Resolve exactly one rule and return: identify-stable-distinct-id.
35+
36+
Read this skill's bundled `identify-users.md` reference once (typically `.claude/skills/audit/references/identify-users.md`; otherwise discover with `Glob` `**/skills/audit/references/identify-users.md`).
37+
38+
Run **one** Grep: `posthog\.identify\(`. Read each file that contains a hit, once. Inspect the first argument passed to identify().
39+
40+
Rule:
41+
- distinct_id must be a stable identifier (auth user id, account id), not a session UUID, ephemeral cookie, or device-only id.
42+
- pass: sources from authenticated user (session.user.id, auth.uid(), etc.)
43+
- error: sources from a session, request, or device id that resets
44+
- warning: source unclear — flag for human review
45+
46+
Emit one `mcp__wizard-tools__audit_resolve_checks` call with a single update for id `identify-stable-distinct-id`, including `file` (path:line) and `details` (one-line explanation). Return when the call completes. Do not write the audit report.
47+
```
48+
49+
### Task B — `identify-not-late`
50+
51+
`description`: `Audit identify-not-late`
52+
53+
`prompt`:
54+
```
55+
You are an audit subagent. Resolve exactly one rule and return: identify-not-late.
56+
57+
Read this skill's bundled `identify-users.md` reference once (typically `.claude/skills/audit/references/identify-users.md`; otherwise discover with `Glob` `**/skills/audit/references/identify-users.md`).
58+
59+
Run **two** Greps in parallel:
60+
- `posthog\.identify\(` — where identity is established
61+
- `posthog\.capture\(|getFeatureFlag\(|isFeatureEnabled\(` — where captures and flag evals happen
62+
63+
Read each file that contains a hit, once. Compare the timing/ordering of identify() against the surrounding capture / flag-eval calls.
64+
65+
Rule:
66+
- identify() must be called before any posthog.capture for that user, and before any feature-flag eval depending on user identity.
67+
- pass: identify runs at session start / right after login. Captures and flag evals come after.
68+
- warning: identify runs lazily (e.g. settings-page mount), so early captures and flag evals are anonymous.
69+
70+
Emit one `mcp__wizard-tools__audit_resolve_checks` call with a single update for id `identify-not-late`, including `file` (path:line of the identify call) and `details` (one-line explanation). Return when the call completes. Do not write the audit report.
71+
```
72+
73+
### Task C — `cross-runtime-distinct-id`
74+
75+
`description`: `Audit cross-runtime-distinct-id`
76+
77+
`prompt`:
78+
```
79+
You are an audit subagent. Resolve exactly one rule and return: cross-runtime-distinct-id.
80+
81+
Read this skill's bundled `identify-users.md` reference once (typically `.claude/skills/audit/references/identify-users.md`; otherwise discover with `Glob` `**/skills/audit/references/identify-users.md`).
82+
83+
Run **one** Grep: `posthog\.init\(|new PostHog\(|posthog\.Posthog\(|Posthog\(` — locate every PostHog initialization across runtimes. Read each file that contains a hit, once. Determine whether both client and server runtimes initialize PostHog, and if so, how distinct_id flows between them.
84+
85+
Rule:
86+
- If both client and server runtimes call PostHog, the same distinct_id must be used on both sides for the same user.
87+
- pass: server-side captures source the client's distinct_id (cookie, session token, or explicit hand-off).
88+
- error: server-side captures use a different identifier scheme.
89+
- Skip (`pass` with details: "single runtime"): only one runtime initializes PostHog.
90+
91+
Emit one `mcp__wizard-tools__audit_resolve_checks` call with a single update for id `cross-runtime-distinct-id`, including `file` (path:line of the most relevant init or capture site) and `details` (one-line explanation). Return when the call completes. Do not write the audit report.
92+
```
93+
94+
### Task D — `identify-reset-on-logout`
95+
96+
`description`: `Audit identify-reset-on-logout`
97+
98+
`prompt`:
99+
```
100+
You are an audit subagent. Resolve exactly one rule and return: identify-reset-on-logout.
101+
102+
Read this skill's bundled `identify-users.md` reference once (typically `.claude/skills/audit/references/identify-users.md`; otherwise discover with `Glob` `**/skills/audit/references/identify-users.md`).
103+
104+
Locate logout, sign-out, and account-switching flows by issuing whatever `Grep` and `Read` calls are needed in parallel. Determine whether those flows clear PostHog state with `posthog.reset()`.
105+
106+
Rule:
107+
- Logout or account-switching flows should call `posthog.reset()`. Without a reset, when user B logs in on the same device after user A, PostHog's anonymous ID is shared and the next `identify()` can merge both accounts into one person.
108+
- pass: every detected logout/account-switch flow calls `posthog.reset()`.
109+
- error: a logout/account-switch flow is missing `posthog.reset()`.
110+
- Skip (`pass` with details: "no logout/account-switch flow found"): no detectable logout/account-switch flow exists.
111+
- note: `posthog.reset(true)` is valid when a completely clean device ID reset is required.
112+
113+
Emit one `mcp__wizard-tools__audit_resolve_checks` call with a single update for id `identify-reset-on-logout`, including `file` (path:line of the most relevant logout or reset site) and `details` (one-line explanation). Return when the call completes. Do not write the audit report.
114+
```
115+
116+
---
117+
118+
**Upon completion, continue with:** [4-event-capture.md](4-event-capture.md)

0 commit comments

Comments
 (0)