Skip to content

Commit fb421b1

Browse files
docs: edit the guides for new readers and fix inaccuracies
The previous change moved the README's long-form content into docs/guides/ verbatim; this one edits it. - workflows: merge the duplicated sections (pull without losing work appeared twice, pull by UUID three times, selective push separately from deploy), drop the old "How to Use This Repo" list now covered by the README, and turn the AI-agent callout into a note for the user (AGENTS.md already instructs agents). - commands: one sentence per command; fix the claim that every command is interactive; move migrate to an "Upgrading from an older version" section. - configuration: document every setting users actually set: per-org .env values and generated bindings, precedence, the promotion and PR check secrets and variables, config files, and CI/debug variables. - how it works: replace the stale project tree (a third of src/, 5 of 68 tests, internal labels) with a short "where things live" table; push deletes only with --force. - troubleshooting: don't advise deleting state entries (that makes the next deploy treat the file as new) or silently editing engine code; point at audit's suggested fixes and an issue. - Present-tense wording throughout (no "used to", "now refuses"). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 2011b05 commit fb421b1

6 files changed

Lines changed: 255 additions & 238 deletions

File tree

‎docs/guides/commands.md‎

Lines changed: 35 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,28 @@
11
# Commands
22

3-
Every command works in two modes:
3+
`setup`, `apply`, `pull`, `push`, `cleanup` and `call` work in two modes:
44

55
- **Interactive** — run without arguments, get prompted for org and resources
6-
- **Direct** — pass an org slug and flags for scripting / CI
7-
8-
| Command | Interactive | Direct | One-liner |
9-
| --- | --- | --- | --- |
10-
| `npm run setup` | ✅ | — | First-time org wizard — creates `.env.<org>` and `resources/<org>/`. |
11-
| `npm run validate` | — | `npm run validate -- <org>` | Schema-check local YAML/MD with no network call. **Run before every `apply`.** |
12-
| `npm run audit` | — | `npm run audit -- <org> [--type <t>]` | Read-only drift detector — orphan local YAML, state ghosts, UUID collisions, content-identical clusters, sibling base-slug clusters, dashboard orphans, assistants with inline `model.tools`. Exit 1 on any finding; safe to wire into CI. |
13-
| `npm run promote` | — | `npm run promote -- --pipeline <name> --from <org> --to <org> [--apply]` | Plan or apply a forward-only, dependency-aware promotion defined by `promotion.yml`. |
14-
| `npm run apply` | ✅ | `npm run apply -- <org> [--force]` | **Default deploy verb.** Pull → merge → push in one safe pass; resilient against dashboard drift. |
15-
| `npm run pull` | ✅ | `npm run pull -- <org> [flags]` | Fetch remote state into local files / state file. Local-first by default — won't clobber local edits. |
16-
| `npm run push` | ✅ | `npm run push -- <org> [flags]` | Raw push without a pre-pull. Refuses by default when local YAML files lack state entries (orphan-YAML gate); pass `--allow-new-files` to bypass after confirming intent. **Skip unless you just ran `pull` and are certain state is fresh** — otherwise prefer `apply`. |
17-
| `npm run cleanup` | ✅ | `npm run cleanup -- <org> [--force --confirm <org>]` | Inspect (default) or delete orphaned remote resources. Destructive run requires `--confirm <org>`. |
18-
| `npm run rollback` | — | `npm run rollback -- <org> --list` or `--to <ISO>` | Restore from a snapshot in `.vapi-state.<org>.snapshots/` (one is written before every push/apply). |
19-
| `npm run call` | ✅ | `npm run call -- <org> -a <name>` or `-s <squad>` | Start an interactive WebSocket call against an assistant or squad. |
20-
| `npm run sim` | — | `npm run sim -- <org> --suite <name> --target <name> [--timeout <min>]` | Run a simulation suite (or specific simulations) against a deployed assistant/squad. Prints the run link; exits 0 passed, 1 failed, 3 incomplete (timeout, Ctrl-C, missing results). |
21-
| `npm run check` | — | `npm run check -- <check>\|--all [--dry-run] [--changed-since <ref>] [--budget-minutes <n>] [--json <path>]` | Run the `vapi-checks.yml` simulation checks against the files on disk: each target is built inline (tools, handoffs, judges, personalities; tools mocked fail-closed, servers dead-ended) and run in one simulation run per target, so nothing is deployed. Posts `Vapi Evals` commit statuses when run by the PR workflow. `--dry-run` builds the payloads offline (no key, nothing sent; `--print-payload` writes them). Exits 0 passed, 1 failed, 2 config or build error, 3 incomplete (timeout, interrupt, budget, billing). |
22-
| `npm run migrate` | — | `npm run migrate` | One-time, all orgs at once: slim legacy state files to pure `name → uuid` and seed the per-developer `.vapi-state-hash/` baseline store from the old hashes. Required once after upgrading to the hash-store engine — `pull`/`push`/`apply` refuse legacy-shaped state until it runs. Idempotent. |
23-
| `npm run build` | — | — | Type-check the codebase (`tsc --noEmit`). |
24-
| `npm test` | — | — | Run regression tests (`node:test`). |
6+
- **Direct** — pass an org name and flags, for scripts and CI
7+
8+
The other commands are direct only.
9+
10+
| Command | Usage | What it does |
11+
| --- | --- | --- |
12+
| `npm run setup` | `npm run setup [-- <org>]` | Connect an org: creates `.env.<org>` and `resources/<org>/`. |
13+
| `npm run validate` | `npm run validate -- <org>` | Check resource files offline. Run it before every `apply`. |
14+
| `npm run apply` | `npm run apply -- <org> [types or paths]` | **The default deploy:** pull, merge, then push. See [workflows](workflows.md). |
15+
| `npm run pull` | `npm run pull -- <org> [--force] [--bootstrap]` | Sync platform changes down; never overwrites local edits unless `--force`. |
16+
| `npm run push` | `npm run push -- <org> [--dry-run]` | Push without pulling first. Prefer `apply`. |
17+
| `npm run rollback` | `npm run rollback -- <org> --list` or `--to <ISO>` | Restore a pre-deploy snapshot from `.vapi-state.<org>.snapshots/`. |
18+
| `npm run cleanup` | `npm run cleanup -- <org> [--force --confirm <org>]` | List platform resources with no file; delete them only with both flags. |
19+
| `npm run audit` | `npm run audit -- <org> [--type <type>]` | Report drift between files, state and the platform. Exits 1 on any finding, so it can run in CI. |
20+
| `npm run call` | `npm run call -- <org> -a <assistant>` or `-s <squad>` | Talk to an assistant or squad from your terminal. |
21+
| `npm run sim` | `npm run sim -- <org> --suite <name> --target <name>` | Run a simulation suite against deployed resources. Exits 0 passed, 1 failed, 3 incomplete. |
22+
| `npm run check` | `npm run check -- <check>` or `--all`, `[--dry-run]` | Run [PR checks](pr-checks.md) against local files, nothing deployed. Exits 0 passed, 1 failed, 2 config or build error, 3 incomplete. |
23+
| `npm run promote` | `npm run promote -- --pipeline <p> --from <org> --to <org> [--apply]` | Plan, or apply, a [promotion](promotion.md) between orgs. |
24+
| `npm run build` | `npm run build` | Type-check the code and tests. |
25+
| `npm test` | `npm test` | Run the test suite. |
2526

2627
## Interactive Mode
2728

@@ -86,3 +87,17 @@ npm run call -- my-org -a my-assistant
8687
# Call a squad
8788
npm run call -- my-org -s my-squad
8889
```
90+
91+
## Upgrading from an older version
92+
93+
Repositories created before the state-file format changed need a one-time
94+
migration. `pull`, `push` and `apply` refuse to run until it's done:
95+
96+
```bash
97+
npm run migrate
98+
```
99+
100+
It rewrites every org's `.vapi-state.<org>.json` to the current
101+
`{ "name": { "uuid": … } }` format and seeds your local
102+
`.vapi-state-hash/` drift baseline from the old file. It's safe to run more
103+
than once. Commit the rewritten state files.

‎docs/guides/configuration.md‎

Lines changed: 40 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,44 @@
11
# Configuration
22

3-
## Environment Variables
3+
## Per-org settings (`.env.<org>`)
44

5-
| Variable | Required | Description |
6-
| --------------- | -------- | ------------------------------------------------ |
7-
| `VAPI_PRIVATE_API_KEY` | ✅ | Vapi private API key from [Private API Keys](https://dashboard.vapi.ai/org/api-keys). The legacy name `VAPI_TOKEN` is still accepted. |
8-
| `VAPI_BASE_URL` | ❌ | API base URL (defaults to `https://api.vapi.ai`) |
5+
`npm run setup` creates one `.env.<org>` file per org. These files are
6+
gitignored: never commit them.
97

10-
These are stored in `.env.<org>` files, one per configured organization.
8+
| Variable | Required | Description |
9+
| --- | --- | --- |
10+
| `VAPI_PRIVATE_API_KEY` | ✅ | The org's private API key, from [Private API Keys](https://dashboard.vapi.ai/org/api-keys). The older name `VAPI_TOKEN` is also accepted. |
11+
| `VAPI_BASE_URL` | | API base URL. Defaults to `https://api.vapi.ai`; EU orgs use `https://api.eu.vapi.ai` (`npm run setup -- <org> --region eu` sets it). |
12+
| `VAPI_CREDENTIAL_<NAME>`, `VAPI_PHONE_NUMBER_<NAME>` | | Generated by `setup` and `pull` inside a marked block: the org's credential and phone-number IDs, by name. Values you set outside the block are kept and take precedence. |
13+
14+
A variable already set in your shell or CI environment takes precedence over
15+
the file. `.env.<org>.local` and `.env.local` are also read, after
16+
`.env.<org>`, and only fill in variables that are still unset.
17+
18+
## CI and workflow settings
19+
20+
Set these in GitHub under **Settings → Secrets and variables → Actions**.
21+
22+
| Name | Kind | Used by | Description |
23+
| --- | --- | --- | --- |
24+
| `VAPI_PROMOTION_TOKENS` | Secret | [Promotion](promotion.md) | JSON map of org name to private API key, for example `{"dev":"…","prod":"…"}`. |
25+
| `VAPI_PROMOTION_ENABLED` | Variable | Promotion | `true` to promote automatically after changes land on `main`. |
26+
| `VAPI_CHECK_TOKENS` | Secret | [PR checks](pr-checks.md) | JSON map of org name to private API key, for checks that run in several orgs or a CI org. |
27+
| `VAPI_PRIVATE_API_KEY` | Secret | PR checks | A single org's key, when every check runs in one org. |
28+
| `VAPI_CHECKS_ENABLED` | Variable | PR checks | `true` to run PR checks. The workflow does nothing without it. |
29+
30+
## Configuration files
31+
32+
| File | Purpose |
33+
| --- | --- |
34+
| `promotion.yml` | Promotion pipelines between orgs. Start from `promotion.example.yml`. |
35+
| `vapi-checks.yml` | PR checks. Start from `vapi-checks.example.yml`. |
36+
| `resources/<org>/.vapi-ignore` | Platform resources this repo shouldn't manage, as glob patterns. See `resources/.vapi-ignore.example` and [YAML conventions](../learnings/yaml-conventions.md). |
37+
38+
## Other environment variables
39+
40+
| Variable | Description |
41+
| --- | --- |
42+
| `CI=true` | Suppresses interactive warnings, such as the one `pull --force` prints. Set automatically by most CI systems. |
43+
| `VAPI_CALL_DEBUG=1` | Makes `npm run call` log message types it would otherwise ignore. |
44+
| `VAPI_GITOPS_ROOT` | Overrides the repository root that `npm run promote` and `npm run check` read from. Mainly for tests. |

‎docs/guides/how-it-works.md‎

Lines changed: 20 additions & 62 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
# How the engine works
22

3-
## Organization-Based Structure
3+
## One folder per org
44

5-
Resources are scoped by organization (not fixed `dev`/`stg`/`prod` names). Each org gets:
5+
Resources are scoped by organization, with names you choose (not fixed `dev`/`stg`/`prod`). Each org gets:
66

77
- `.env.<org>` — private API key and base URL
88
- `.vapi-state.<org>.json` — resource name ↔ UUID mappings (nothing else — committed)
@@ -40,7 +40,7 @@ files
4040

4141
**`pull`** — downloads platform state. Detects locally modified files and skips them (your work is preserved). Use `--force` to overwrite everything.
4242

43-
**`push`** — reads local files and syncs them to the platform. Handles creates, updates, and deletions.
43+
**`push`** — reads local files and syncs them to the platform: creates and updates. It deletes platform resources whose files you removed only when you pass `--force`.
4444

4545
**`apply`** — runs `pull` then `push` in sequence.
4646

@@ -110,62 +110,20 @@ Tracks resource ID ↔ Vapi UUID mappings per org:
110110

111111
Every resource type has a section. Keys are sorted, so diffs stay readable.
112112

113-
## Project Structure
114-
115-
```
116-
vapi-gitops/
117-
├── docs/
118-
│ ├── Vapi Prompt Optimization Guide.md
119-
│ ├── changelog.md
120-
│ └── learnings/ # Gotchas, recipes, troubleshooting per area
121-
│ ├── assistants.md
122-
│ ├── tools.md
123-
│ ├── squads.md
124-
│ ├── simulations.md
125-
│ └── ...
126-
├── src/
127-
│ ├── setup.ts # Setup wizard (interactive) + non-interactive setup
128-
│ ├── setup-args.ts # `npm run setup` argument parsing
129-
│ ├── interactive.ts # Interactive pull/push/apply/call/cleanup flows
130-
│ ├── searchableCheckbox.ts # Custom multi-select prompt component
131-
│ ├── pull.ts # Pull platform state
132-
│ ├── push.ts # Push local state to platform
133-
│ ├── apply.ts # Orchestrator: pull → merge → push
134-
│ ├── call.ts # WebSocket call script
135-
│ ├── cleanup.ts # Orphan cleanup
136-
│ ├── pull-cmd.ts # Entry point: interactive or direct pull
137-
│ ├── push-cmd.ts # Entry point: interactive or direct push
138-
│ ├── apply-cmd.ts # Entry point: interactive or direct apply
139-
│ ├── call-cmd.ts # Entry point: interactive or direct call
140-
│ ├── cleanup-cmd.ts # Entry point: interactive or direct cleanup
141-
│ ├── types.ts # TypeScript interfaces
142-
│ ├── config.ts # Environment & configuration
143-
│ ├── api.ts # Vapi HTTP client
144-
│ ├── state.ts # State file management
145-
│ ├── resources.ts # Resource loading (YAML, MD, TS)
146-
│ ├── resolver.ts # Reference resolution
147-
│ ├── credentials.ts # Credential resolution (name ↔ UUID)
148-
│ ├── delete.ts # Deletion & orphan checks
149-
│ └── check-cmd.ts # Entry point: PR simulation checks (check-*.ts)
150-
├── resources/
151-
│ └── <org>/ # One directory per configured org
152-
│ ├── assistants/
153-
│ ├── tools/
154-
│ ├── squads/
155-
│ ├── structuredOutputs/
156-
│ ├── evals/
157-
│ └── simulations/
158-
│ ├── personalities/
159-
│ ├── scenarios/
160-
│ ├── tests/
161-
│ └── suites/
162-
├── tests/
163-
│ ├── credentials.test.ts # Credential walker scoping (P0-1 regression suite)
164-
│ ├── clean-resource.test.ts # null-preservation in pull (P0-3 regression suite)
165-
│ ├── path-matching.test.ts # Short-form path matching (P0-7 regression suite)
166-
│ ├── cleanup-safety.test.ts # --confirm + empty-state gates (P0-4 regression suite)
167-
│ └── cli-arg-parsing.test.ts # Bare-id refusal, --confirm pass-through (P0-7)
168-
├── vapi-checks.example.yml # Copy to vapi-checks.yml for PR simulation checks
169-
├── .env.<org> # Private API key per org (gitignored)
170-
└── .vapi-state.<org>.json # State file per org
171-
```
113+
## Where things live
114+
115+
| Path | What it is |
116+
| --- | --- |
117+
| `resources/<org>/` | Your resources, one folder per org |
118+
| `.vapi-state.<org>.json` | Name → UUID mappings per org (committed) |
119+
| `.env.<org>` | API key and generated binding IDs (gitignored) |
120+
| `promotion.yml`, `vapi-checks.yml` | Promotion pipelines and PR checks (copy from the `*.example.yml` files) |
121+
| `resources/<org>/.vapi-ignore` | Platform resources this repo shouldn't manage (see `resources/.vapi-ignore.example`) |
122+
| `src/` | The engine; `package.json` scripts name each command's entry point |
123+
| `tests/` | The test suite (`npm test`) |
124+
| `docs/guides/` | These guides |
125+
| `docs/learnings/` | The Vapi field guide |
126+
| `docs/changelog.md` | A template for your own deployment's change log |
127+
| `examples/` | Copyable examples; never loaded by the engine |
128+
| `.github/workflows/` | CI, PR checks and promotion workflows |
129+
| `AGENTS.md`, `CLAUDE.md` | Instructions for coding agents |

‎docs/guides/promotion.md‎

Lines changed: 30 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -113,15 +113,41 @@ orgs:
113113
114114
- Plans print `check would run staging-core in example-staging (<n> simulations × <t> targets)`
115115
and run nothing.
116-
- On `--apply`, the check runs against `resources/example-staging/` at the
117-
promoted commit, in example-staging, with that org's key from
116+
- On `--apply`, the check runs against `resources/example-staging/` as this
117+
run leaves it (the promoted commit, plus anything promoted into it earlier in
118+
the same run), in example-staging, with that org's key from
118119
`VAPI_PROMOTION_TOKENS`, before any file is written to the destination. A
119120
failure, an incomplete run (timeout, billing) or a build error blocks the
120-
transition with the run link; transitions that already applied are still
121-
committed.
121+
transition, with the run link (or the reason, if nothing ran); transitions
122+
that already applied are still committed.
123+
- A block stops the whole run, including pipelines that don't involve the
124+
blocked org. Fix the check, or until it passes, promote the other
125+
pipelines a step at a time with `--pipeline <name> --from <org> --to <org>`.
122126
- A pass is reused for later transitions out of the same org in the same run,
123127
until something is promoted into it.
124128
- Transitions with no changes skip the check.
129+
- Each gated promotion runs its simulations again, on top of the PR check, and
130+
uses simulation minutes.
131+
132+
The check is validated when promotion starts, and these stop the run before
133+
anything applies:
134+
135+
- the check doesn't read and run in the gated org (`org` and `runOrg`);
136+
- `toolMocks: off` or `stripWebhooks: false`: a gate runs in the real org,
137+
never a CI org, so it must not reach real tools or the org's servers;
138+
- a `baseUrl` on the check that differs from the org's `baseUrl` in
139+
`promotion.yml` (the org's key only goes to the host promotion uses);
140+
- a gate on an org that is last in every pipeline, so nothing is ever promoted
141+
out of it (gate the org before production, not production itself);
142+
- gated checks whose combined budget exceeds 300 minutes per run. Each check
143+
can take `timeoutMinutes` for every batch of 3 targets.
144+
- an unknown key under an org, such as a misspelled `check:`.
145+
146+
Promotion can't yet carry a simulation that references a stock personality
147+
by UUID (`a0000000-…`): its dependency check reports "Referenced managed
148+
dependency is missing from source". Use personality files under
149+
`simulations/personalities/` in any org you promote out of, gated or not
150+
(`improvements.md` #38).
125151

126152
## Rolling Back a Promotion
127153

‎docs/guides/troubleshooting.md‎

Lines changed: 23 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -11,18 +11,22 @@ The referenced resource doesn't exist. Check:
1111

1212
## "Cannot delete resource - still referenced"
1313

14-
1. Find which resources reference it (shown in error)
15-
2. Remove the references
16-
3. Push again
17-
4. Then delete the resource file
14+
1. Find which resources reference it (shown in the error)
15+
2. Remove those references and deploy with `npm run apply -- <org>`
16+
3. Then delete the resource file and deploy again
1817

1918
## Resource not updating
2019

21-
Check the state file has correct UUID:
20+
The file may be mapped to the wrong platform resource, or to one that no
21+
longer exists.
2222

23-
1. Open `.vapi-state.<org>.json`
24-
2. Find the resource entry
25-
3. If incorrect, delete entry and re-run push
23+
Run `npm run audit -- <org>`. It reports state entries that point at a
24+
missing resource, or several files that point at the same one, with a
25+
suggested fix for each.
26+
27+
Don't delete a state entry by hand: the next deploy would treat the file as
28+
new, and stop at the new-file check (or create a duplicate if the check is
29+
bypassed).
2630

2731
## "Credential with ID not found" errors
2832

@@ -34,7 +38,13 @@ The credential UUID doesn't exist in the target org. Fix:
3438

3539
## "property X should not exist" API errors
3640

37-
Some properties can't be updated after creation. Add them to `UPDATE_EXCLUDED_KEYS` in `src/config.ts`.
41+
Some properties can't be changed after a resource is created, so the API
42+
rejects them on update. Please [open an issue](https://github.com/VapiAI/gitops/issues/new/choose)
43+
with the resource type and property, so the engine stops sending it.
44+
45+
As a stopgap you can add the property to `UPDATE_EXCLUDED_KEYS` in
46+
`src/config.ts`. That's an engine change, so expect to resolve it when you
47+
next pull in updates.
3848

3949
## "Refusing to run destructive cleanup" errors
4050

@@ -62,11 +72,11 @@ The interactive `npm run cleanup` flow handles both gates for you (it shows
6272
the dry-run preview, asks you to confirm, and forwards `--force --confirm
6373
<org>` automatically when you say yes).
6474

65-
## "Unrecognized argument" / push appears to do nothing
75+
## "Unrecognized argument" errors
6676

67-
If you typed `npm run push -- my-org foo` (a bare resource id with no folder
68-
or extension), the CLI now refuses with `Unrecognized argument: foo` rather
69-
than silently running a full apply. Pass either:
77+
A bare resource ID (`npm run push -- my-org foo`, with no folder or
78+
extension) is rejected with `Unrecognized argument: foo`, rather than
79+
falling through to a full deploy. Pass either:
7080

7181
- a resource type — `npm run push -- my-org assistants`, or
7282
- a path — `npm run push -- my-org assistants/foo.yml` (short form)

0 commit comments

Comments
 (0)