Repository navigation
Commit a67a4b1
authored
feat: identify every gitops API request with a User-Agent (#78)
## Value
**V.A.L.U.E. tier:** small — a behavior change: every API request now carries a new header. No blast-radius path.
- **Problem:** we can't measure how gitops is used beyond simulations. Only `npm run sim` and `npm run check` identify themselves. Setup, pull, push, apply, promote, cleanup, rollback, call and audit send Node's default User-Agent, `node`, which was 126k of 728k `api.vapi.ai` requests in one hour this morning. So "how many deploys come from gitops, from CI or from laptops" has no answer.
- **Who it affects:** the Vapi team measuring adoption after the public launch. Customers see no change in behavior.
- **What changes:**
- **`src/user-agent.ts`** builds `vapi-gitops-<command>/<version>`, plus ` (ci)` when `CI` or `GITHUB_ACTIONS` is set:
- `<command>` comes from a fixed list of this repo's commands, so a fork's own npm script names are never sent. The first process pins its label in `VAPI_GITOPS_COMMAND`, which child processes inherit: `npm run apply` labels its pull and push `apply`, a promotion's applies are `promote`, and the PR check's bindings pull is `check`. Promotion gate runs are labelled `promote`, apart from PR check runs.
- When a script is run directly, as the PR check workflow does, it falls back to the entry script's name.
- `sim` and `check` keep their fixed labels, which existing simulation analytics already counts by.
- **Every `fetch` to the Vapi API sends it:** `api.ts` (push, pull, apply, promote), cleanup, setup, the interactive pickers, rollback, call, and push's direct fetch.
- **Fixes two tests that called production.** `cleanup-safety` and `new-file-gate` sent about 12 requests per `npm test` to `api.vapi.ai`, with a fake key, getting 401s. That breaks the repo's own rule that tests never call the real API, and with this PR it would have counted every fork's CI run as gitops usage. They now point at a dead local address.
- **Docs:** `how-it-works.md` says exactly what the API sees (and that there is no other telemetry). `AGENTS.md` says every request must send the header.
## Evidence of value
**Live, through the Cloudflare request logs in Axiom (`cloudflare-logpush`):** a read-only `npm run setup -- ua-check --resources none` against the test org, run from a scratch copy:
| User-Agent | Requests | Status |
| --- | --- | --- |
| `vapi-gitops-setup/1.0.0` | 13 GET | 200 |
| `vapi-gitops-test/1.0.0` | 24 GET | 401 — the two leaky tests, from the two `npm test` runs before the fix |
Before this PR, both rows would have been indistinguishable `node` traffic.
**Tests:**
- **`tests/user-agent-coverage.test.ts`** reads every `fetch(` in `src/` and requires a `User-Agent`. On the parent branch it lists 10 call sites without one; here it lists none. It also runs `api.ts` against a local server with `npm_lifecycle_event=apply`.
- **`tests/user-agent.test.ts`** pins the format: fixed sim and check labels, npm script vs. entry script vs. `cli`, label cleaning, and the CI marker for `GITHUB_ACTIONS=true`, `CI=true` and `CI=1` (but not `false`, `0` or empty).
- **No more production calls:** running the full suite with a `fetch` trap that records any request to `vapi.ai` caught 12 requests before the test fix and none after.
## Testing plan
- `npm test` (527 tests) and `npx tsc --noEmit` pass.
- **Not covered:**
- **Distinct-org counts for non-simulation commands.** The request logs carry the User-Agent but not the org. Only simulation runs get an org ID, through PostHog. Joining the two needs an API-side change, outside this repo.
- **The version** is still `1.0.0`; bumping it is deliberately left out of this PR.
- **The `call` command's WebSocket audio connection** isn't a Vapi REST request and doesn't carry the header.
Refs TEST-141
## After review
- **Labels:** come only from the known command list, and are pinned for child processes (Chris's 🟠: the PR check's bindings pull was counted as a CI pull). `npx` and fork script names fall through to the entry script, then to `cli`.
- **GitHub status call:** now sends the User-Agent too, so every `fetch` in `src/` does. The coverage test checks each call on its own and scans subfolders.
- **`tests/no-vapi-api.ts`:** loads before every test file, with an unroutable base URL and no inherited real key. With a pretend key exported and a fetch trap on, the suite makes zero requests to vapi.ai.
- **Docs:** `how-it-works.md` matches the code on `(ci)` and `<command>`.
- **Formatting:** the unrelated Prettier changes are reverted.
🤖 Generated with [Claude Code](https://claude.com/claude-code)19 files changed
Lines changed: 303 additions & 20 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
321 | 321 | | |
322 | 322 | | |
323 | 323 | | |
| 324 | + | |
| 325 | + | |
324 | 326 | | |
325 | 327 | | |
326 | 328 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
145 | 145 | | |
146 | 146 | | |
147 | 147 | | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
148 | 159 | | |
149 | 160 | | |
150 | 161 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
26 | | - | |
| 26 | + | |
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
| 3 | + | |
3 | 4 | | |
4 | 5 | | |
5 | 6 | | |
| |||
97 | 98 | | |
98 | 99 | | |
99 | 100 | | |
| 101 | + | |
100 | 102 | | |
101 | 103 | | |
102 | 104 | | |
| |||
140 | 142 | | |
141 | 143 | | |
142 | 144 | | |
| 145 | + | |
143 | 146 | | |
144 | 147 | | |
145 | 148 | | |
| |||
188 | 191 | | |
189 | 192 | | |
190 | 193 | | |
| 194 | + | |
191 | 195 | | |
192 | 196 | | |
193 | 197 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| 5 | + | |
| 6 | + | |
5 | 7 | | |
6 | 8 | | |
7 | 9 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| 9 | + | |
9 | 10 | | |
10 | 11 | | |
11 | 12 | | |
| |||
362 | 363 | | |
363 | 364 | | |
364 | 365 | | |
| 366 | + | |
365 | 367 | | |
366 | 368 | | |
367 | 369 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| 8 | + | |
| 9 | + | |
8 | 10 | | |
9 | 11 | | |
10 | 12 | | |
| |||
84 | 86 | | |
85 | 87 | | |
86 | 88 | | |
| 89 | + | |
| 90 | + | |
87 | 91 | | |
88 | 92 | | |
89 | 93 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
14 | 15 | | |
15 | 16 | | |
16 | 17 | | |
| |||
29 | 30 | | |
30 | 31 | | |
31 | 32 | | |
32 | | - | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
33 | 37 | | |
34 | 38 | | |
35 | 39 | | |
| |||
67 | 71 | | |
68 | 72 | | |
69 | 73 | | |
70 | | - | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
71 | 78 | | |
72 | 79 | | |
73 | 80 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| 12 | + | |
12 | 13 | | |
13 | 14 | | |
14 | 15 | | |
| |||
223 | 224 | | |
224 | 225 | | |
225 | 226 | | |
226 | | - | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
227 | 231 | | |
228 | 232 | | |
229 | 233 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
153 | 153 | | |
154 | 154 | | |
155 | 155 | | |
156 | | - | |
| 156 | + | |
| 157 | + | |
157 | 158 | | |
158 | 159 | | |
159 | 160 | | |
| |||
0 commit comments