Skip to content

Commit 2b2d34b

Browse files
pmclSFclaude
andcommitted
docs(0.2.0): close 27 doc-audit findings (release-blockers + should-fix)
Comprehensive sweep through public-facing docs after the v0.2.0 release-readiness audit. Covers everything flagged across the four parallel review domains: README + top-level repo files, product narrative, release-shape docs (parity rubric/scores, known-gaps, release notes), and adopter-facing user-guides + schema + examples. Release-blocking fixes: - README.md, CONTRIBUTING.md: Tier-1 floor "≥ 4" → "≥ 3 in 0.2.0 (≥ 4 in 0.3 once labeled-corpus precision floor lands)"; matches rubric.yaml's actual gate. - CONTRIBUTING.md: "10 canonical commands" → "11" (matches binary). Removed two references to non-existent `docs/release/0.2.x-maturity-audit.md`. Expanded exit-code documentation to cover the full 0/1/2/4/5/6 scheme. - docs/product/ai-trust-boundary.md (lines 75-94) + ai-risk-tiers.md (line 101): rewritten away from the non-existent flags `terrain ai run --baseline / --ingest-only / --promptfoo-results`. Now documents the actual 0.2 escape hatch — `terrain analyze` with `--promptfoo-results / --deepeval-results / --ragas-results` for ingesting pre-computed eval JSON without invoking any framework, plus `terrain ai record` + `terrain ai baseline compare` for regression-aware gating. - docs/product/vision.md (primitives table): clarified that `--new-findings-only --baseline` is an analyze flag; the gate flow is `terrain analyze --fail-on=...`. - docs/examples/gate/github-action.yml: rewrote the recommended GitHub Action template — `--new-findings-only --baseline` now on the `analyze` step (where the flag actually exists); `terrain report pr` is just the comment renderer. The previous template would have failed on first run with "flag provided but not defined: -new-findings-only". - docs/schema/analysis.schema.json: replaced the stale `1.0.0` schema (which described meta/summary/files top-level keys that haven't existed since the analyze rewrite) with a permissive stub pinned to `schemaVersion: "1"` and pointing at `internal/analyze/analyze.go::Report` as the canonical source. Includes one-line descriptions for all real top-level keys emitted by `terrain analyze --json` today. - docs/schema/portfolio.md: TestAsset and Finding shapes regenerated against `internal/portfolio/model.go` — `paths` → `path` + `relatedPaths`; `confidence` is a string enum (`high`/`moderate`/`low`), not a float; dropped fictional `severity` and `coverageRatio`; added missing `testType`, `breadthClass`, `passRate`, `retryRate`, `coveredModules`, `ownersCovered`, `importedSources`, `instabilitySignals`, `hasRuntimeData`, `hasCoverageData`, `metadata`. - docs/examples/serve-local-dev.md: default port 7344 → 8421; removed non-existent `--snapshot` flag; added `--host` example. Should-fix cleanup: - README.md, CHANGELOG.md, vision.md: dropped `focus` from the `terrain report` verb list (the canonical surface has summary, insights, metrics, explain, show, impact, pr, posture, select-tests — `focus` is a legacy top-level alias only). - README.md, CONTRIBUTING.md, DESIGN.md: package count 47/49 → 53. - docs/cli-spec.md: surface table rebuilt against `terrain --help` output — 11 canonical commands (analyze, init, report, migrate, ai, debug, config, doctor, portfolio, serve, version), correct per-namespace verb lists, and a one-paragraph note on legacy aliases. - docs/release/parity/rubric.yaml: removed dead reference to `0.2.x-maturity-audit.md`; ai_eval_ingestion surface string updated to use the real `terrain ai record / baseline compare` workflow. - docs/release/parity/scores.yaml score-evidence corrections: - core_analyze.E1 5 → 4 (no `func Fuzz*` tests in repo; adversarial fixtures sit at the level-3↔5 midpoint) - core_analyze.E4 evidence updated — the `FIELD_TIERS.md` doc closes the previously-claimed "no tier markers" gap - summary_posture_metrics_focus.P7 5 → 4 (level-5 anchor "composes with neighbouring tools cleanly" is partial) - policy_governance.E4 4 → 3 — previous evidence cited the wrong schema doc (`eval-adapters.md` is for AI eval adapters, not policy); honest level-3 anchor now noted - docs/release/0.2-known-gaps.md: rows 29/31/41 reclassified from "0.3 deferred" to "(fixed in 0.2)" with code pointers — the `pairedConfidence` ramp, the `aiHallucinationRate` keyword expansion, and the `aiModelDeprecationRisk` comment-prefix expansion all shipped in 0.2. - docs/release/feature-status.md: `terrain compare` row marked experimental (matches the "output format may shift" caveat later in the file). Added a one-line orphan-signal pointer surfacing the 15 long-standing structural / quality / migration / governance signals carried over from 0.1.x — truth-verify now reviews 44 signals instead of 29. - docs/release/0.2.md: replaced the unmet "P50 < 10s on 10k-file repo" Tier-2 gate with the actual measured budget contract (per-stage timing visible via --verbose; bench-gate enforces no >10% regression vs `benchmarks/baseline.txt`; published per-size-class budgets land in 0.3). - docs/release/quality-bar-and-gates.md: `make js-release-verify` → `make npm-release-verify` (the actual Makefile target name). - docs/release/supply-chain.md: corrected the Homebrew formula description — it's a source-build formula updated by the `homebrew-update.yml` workflow, not generated by goreleaser. - docs/release/release-checklist-final.md: `report focus` removed from the smoke list; `report select-tests` and `report impact` added (matches actual canonical verb list). - docs/glossary.md: posture entry corrected — five real dimensions (health, coverage_depth, coverage_diversity, structural_risk, operational_risk) and real bands instead of the wrong "reliability/change-readiness/speed/governance/AI" + A/B/C/D vocabulary. - docs/product/terrain-overview.md: stale "22 signal types" / count-based framings replaced with order-of-magnitude prose ("70+ signal types") + a pointer to the auto-generated manifest. Canonical 11-command CLI surface re-stated. - docs/user-guides/ui-triage-flow.md: posture dimensions corrected to match `posture-model.md`. - docs/user-guides/writing-a-policy.md: removed the false claim that `terrain init` writes `.terrain/policy.yaml`; the real workflow is to copy a template from `docs/policy/examples/`. - docs/user-guides/ai-eval-onboarding.md: expected `terrain ai list` output replaced with the actual current shape. - docs/user-guides/impact-analysis-and-test-selection.md: `--base` default `main` → `HEAD~1` (matches binary). - docs/schema/pr-analysis.md: `scope` example corrected — real fields are `baselineRef` / `currentRef` (not `baseRef` / `headRef`); `changedFiles` is `[]ChangedFile` objects with `path` + `changeKind` + `isTestFile`, not bare strings. Verification: - make pillar-parity → understand=3 PASS, gate=3 PASS, align WARN soft - make truth-verify → 44 documented signals all resolve (was 29) - make docs-verify → manifest + rubric stubs in sync - make docs-linkcheck → all intra-repo links resolve - make voice-lint → clean - make release-gate → all snapshot tests pass Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent b924d4e commit 2b2d34b

27 files changed

Lines changed: 382 additions & 259 deletions

CHANGELOG.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ status with pillar + tier is [`docs/release/feature-status.md`]
2929
The release groups deliverables by the three pillars:
3030

3131
- **Understand** (Tier 1): full snapshot pipeline; `report
32-
summary/posture/metrics/focus/insights/explain`; AI surface
32+
summary/posture/metrics/insights/explain`; AI surface
3333
inventory; cross-repo views.
3434
- **Align** (Tier 1): framework migration with per-file confidence;
3535
alignment-first docs; multi-repo manifest format.

CONTRIBUTING.md

Lines changed: 13 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -13,9 +13,9 @@ go test ./cmd/... ./internal/...
1313
## Project Structure
1414

1515
```
16-
cmd/terrain/ CLI entry point (10 canonical commands + legacy aliases)
16+
cmd/terrain/ CLI entry point (11 canonical commands + legacy aliases)
1717
cmd/terrain-bench/ Benchmark harness
18-
internal/ 49 Go packages
18+
internal/ 53 Go packages
1919
├── analysis/ Repository scanning and code surface inference
2020
├── convert/ Go-native test conversion (25 directions)
2121
├── depgraph/ Dependency graph with 5 reasoning engines
@@ -60,7 +60,7 @@ make release-verify
6060
- JSON output uses `json.NewEncoder(os.Stdout)` with `SetIndent("", " ")`
6161
- Nil slices should be converted to empty slices before JSON encoding (serialize as `[]` not `null`)
6262
- Error messages go to stderr: `fmt.Fprintf(os.Stderr, "error: %v\n", err)`
63-
- Exit codes: 0 = success, 1 = error, 2 = usage error / policy violation
63+
- Exit codes: 0 = success, 1 = error, 2 = usage error / policy violation, 4 = AI gate block, 5 = entity not found, 6 = severity-gate block (`--fail-on`). See `docs/cli-spec.md` for the full table.
6464
- Commands with positional args use `reorderCLIArgs()` to support flags in any position
6565

6666
## Adding a New Conversion Direction
@@ -115,19 +115,23 @@ apply:
115115

116116
| Pillar | Floor | Block release? |
117117
|--------|-------|----------------|
118-
| Gate | every cell ≥ 4 | yes |
118+
| Gate | every cell ≥ 3 in 0.2.0 (≥ 4 in 0.3) | yes |
119119
| Understand | every cell ≥ 3 | yes |
120120
| Align | every cell ≥ 3 | soft (warn-only) |
121121

122+
The Gate floor is 3 in 0.2.0 because P2 / E2 axes at level 4/5 require a
123+
labeled real-repo precision floor — the central deliverable of the 0.3
124+
program. Gate floor=3 reflects recall-anchored synthetic calibration in
125+
0.2.0; the level-4/5 jump lands cell-by-cell as the 0.3 corpus work
126+
materializes.
127+
122128
### How to lift a cell in your PR
123129

124130
1. Find the cell you're improving in `docs/release/parity/scores.yaml`.
125131
2. Update the score (1–5) and replace the evidence line with a
126132
one-line pointer to the change you're making (file:line, test
127133
name, or short rationale).
128-
3. If your change touches the audit doc's narrative,
129-
`docs/release/0.2.x-maturity-audit.md` updates in the same PR.
130-
4. Run `make pillar-parity` locally — your change should move at
134+
3. Run `make pillar-parity` locally — your change should move at
131135
least one cell; CI will compare the diff.
132136

133137
### Source-of-truth split
@@ -138,9 +142,8 @@ apply:
138142
means lives here.
139143
- **Per-cell scores**: `docs/release/parity/scores.yaml`. Changes
140144
every parity-lift PR. The shape is `area_id → axis_id → {score,
141-
evidence}`.
142-
- **Human-readable companion**: `docs/release/0.2.x-maturity-audit.md`.
143-
Same data, prose form. Update both together.
145+
evidence}`. Each cell carries a one-line evidence pointer
146+
(file:line, test name, PR number) so the lift is auditable.
144147

145148
### Local commands
146149

DESIGN.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,8 +37,8 @@ See [docs/architecture.md](docs/architecture.md) for the full layered architectu
3737
## Package Map
3838

3939
```
40-
cmd/terrain/ CLI entry point (10 canonical commands + legacy aliases)
41-
internal/ 49 packages — see internal/README.md for the listing
40+
cmd/terrain/ CLI entry point (11 canonical commands + legacy aliases)
41+
internal/ 53 packages — see internal/README.md for the listing
4242
```
4343

4444
See [docs/engineering/detector-architecture.md](docs/engineering/detector-architecture.md) for the detector plugin system architecture.

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -253,7 +253,7 @@ this is the summary view.
253253

254254
Capabilities are tiered:
255255

256-
- **Tier 1** — covered by tests, documented behavior, claimed publicly. Floor ≥ 4 on the parity rubric.
256+
- **Tier 1** — covered by tests, documented behavior, claimed publicly. Floor ≥ 3 on the parity rubric in 0.2.0; Gate floor lifts to ≥ 4 in 0.3 once the labeled real-repo precision corpus lands.
257257
- **Tier 2** — shipping but explicitly experimental; useful but not yet hardened. Floor ≥ 3.
258258
- **Tier 3** — in development, opt-in, no public claim. Wait for promotion.
259259

@@ -262,7 +262,7 @@ Capabilities are tiered:
262262
**Understand** (Tier 1 unless noted):
263263

264264
- `terrain analyze` — snapshot + signals + posture
265-
- `terrain report summary / posture / metrics / focus / insights / explain` — read-side queries
265+
- `terrain report summary / posture / metrics / insights / explain` — read-side queries
266266
- `terrain compare` — snapshots over time
267267
- AI surface inventory — what AI surfaces exist, where they are, what evals cover them
268268
- `terrain serve` (Tier 2) — local HTTP report; localhost-only, no auth
@@ -616,7 +616,7 @@ Repository scan → Signal detection → Risk modeling → Reporting
616616
cmd/terrain/ CLI — canonical surface (analyze, report, migrate,
617617
convert, posture, doctor, ai, serve, version, help)
618618
plus legacy aliases retained through 0.2.x
619-
internal/ 47 Go packages covering analysis, signals, risk,
619+
internal/ 53 Go packages covering analysis, signals, risk,
620620
impact, depgraph, measurement, reporting, and more
621621
```
622622

docs/cli-spec.md

Lines changed: 17 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -13,22 +13,28 @@ It must be:
1313

1414
## Surface — canonical 11 + legacy aliases
1515

16-
0.2.0 introduces three new namespace dispatchers
17-
(`terrain report`, `terrain migrate`, `terrain config`) plus
18-
`terrain debug`. The canonical surface is 10 top-level verbs:
16+
0.2.0 introduces namespace dispatchers (`terrain report`,
17+
`terrain migrate`, `terrain config`, `terrain debug`,
18+
`terrain ai`). The canonical surface is **11 top-level commands**:
1919

2020
| Canonical | What it does |
2121
|---|---|
2222
| `terrain analyze` | Full snapshot pipeline; the headline command |
23-
| `terrain report <verb>` | Read-side views: summary, insights, explain, posture, portfolio, metrics, focus, show, impact, pr, select-tests |
24-
| `terrain migrate <verb>` | Conversion + migration: run, config, list, detect, shorthands, estimate, status, checklist, readiness, blockers, preview |
25-
| `terrain convert <file>` | Per-file conversion (legacy fall-through preserved) |
26-
| `terrain config <verb>` | feedback, telemetry, reset |
27-
| `terrain doctor` | Migration-readiness diagnostic |
28-
| `terrain ai <verb>` | AI surface inventory + eval orchestration |
29-
| `terrain serve` | Local HTTP server with HTML report + JSON API |
23+
| `terrain init [path]` | First-run scaffolder — runs analyze + emits a starter `.terrain/` layout |
24+
| `terrain report <verb>` | Read-side views: `summary`, `insights`, `metrics`, `explain`, `show`, `impact`, `pr`, `posture`, `select-tests` |
25+
| `terrain migrate <verb>` | Framework migration: `run`, `config`, `list`, `detect`, `shorthands`, `estimate`, `status`, `checklist`, `readiness`, `blockers`, `preview` |
26+
| `terrain ai <verb>` | AI inventory + eval orchestration: `list`, `run`, `replay`, `record`, `baseline`, `doctor` |
27+
| `terrain debug <verb>` | Dependency-graph drill-downs: `graph`, `coverage`, `fanout`, `duplicates`, `depgraph` |
28+
| `terrain config <verb>` | Workspace prefs: `feedback`, `telemetry` |
29+
| `terrain doctor [path]` | Diagnostics for current setup |
30+
| `terrain portfolio` | Single-repo (stable) and multi-repo manifest (experimental) portfolio analysis |
31+
| `terrain serve` | Local HTTP server with HTML report + JSON API (default port 8421, 127.0.0.1 only) |
3032
| `terrain version` | Version, commit, build date, snapshot schema version |
31-
| `terrain help` | Top-level help surface |
33+
34+
Legacy aliases (`terrain summary`, `terrain insights`,
35+
`terrain compare`, `terrain convert <file>`, `terrain focus`,
36+
etc.) continue to route to the same handlers through 0.2.x.
37+
Removal targets 0.3.
3238

3339
The legacy top-level commands documented in this file
3440
(`terrain summary`, `terrain insights`, etc.) continue to work

docs/examples/gate/github-action.yml

Lines changed: 16 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -58,31 +58,32 @@ jobs:
5858
terrain version
5959
6060
# ── Step 1 of the primary workflow: understand the test system ──
61-
- name: terrain analyze
61+
#
62+
# `analyze` is the gate. `--new-findings-only --baseline <path>`
63+
# filters to signals introduced AFTER the baseline snapshot, so
64+
# established repos with pre-existing debt don't get blocked on
65+
# day one. To gate, uncomment `--fail-on critical` (exit 6 on
66+
# any Critical finding). The default below is warn-only — it
67+
# produces output but does not fail the job.
68+
- name: terrain analyze (warn-only by default)
6269
run: |
6370
terrain analyze \
6471
--write-snapshot \
72+
--new-findings-only \
73+
--baseline terrain-snapshot.json \
6574
--json > terrain-snapshot.json
75+
# --fail-on critical # uncomment to gate the build (exit 6)
6676
67-
# ── Step 2: gate the PR ────────────────────────────────────────
68-
# Default: warn-only. The job stays green; the PR comment shows
69-
# what Terrain found. Established repos with debt won't brick CI
70-
# on day one.
71-
#
72-
# To enable blocking gates, uncomment the `--fail-on critical`
73-
# line below. Combined with `--new-findings-only --baseline ...`,
74-
# the gate fires only on findings introduced AFTER the snapshot
75-
# in the baseline file (see `terrain analyze --write-snapshot`
76-
# for how to persist the baseline).
77-
- name: terrain report pr (warn-only by default)
77+
# ── Step 2: render the PR comment ──────────────────────────────
78+
# `report pr` summarizes the gated state. The exit status of
79+
# this step is independent of the gating decision (which is
80+
# already determined by step 1's `--fail-on`).
81+
- name: terrain report pr
7882
run: |
7983
terrain report pr \
8084
--base ${{ github.event.pull_request.base.sha }} \
8185
--format markdown \
82-
--new-findings-only \
83-
--baseline terrain-snapshot.json \
8486
> terrain-pr-comment.md
85-
# --fail-on critical # uncomment to gate the build
8687
8788
- name: Post or update the PR comment
8889
uses: peter-evans/create-or-update-comment@v5

docs/examples/serve-local-dev.md

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,11 +18,11 @@ time.
1818
```bash
1919
# In your repo:
2020
terrain serve
21-
# → Listening on http://127.0.0.1:7344
21+
# → Listening on http://127.0.0.1:8421
2222
# → Press Ctrl-C to stop.
2323
```
2424

25-
Open `http://127.0.0.1:7344` in your browser. The page renders the
25+
Open `http://127.0.0.1:8421` in your browser. The page renders the
2626
same data shapes as `terrain analyze`, with sticky pillar
2727
navigation, signal cards, and tasteful typography.
2828

@@ -32,12 +32,15 @@ navigation, signal cards, and tasteful typography.
3232
# Custom port
3333
terrain serve --port 8000
3434

35+
# Bind to a different host (default is 127.0.0.1)
36+
terrain serve --host 127.0.0.1 --port 8421
37+
3538
# Allow request to mutate state (the default is read-only — most
3639
# adopters want this off for local dev too):
3740
terrain serve --read-only=false
3841

39-
# Use a saved snapshot instead of re-analyzing:
40-
terrain serve --snapshot .terrain/snapshots/latest.json
42+
# Run from a different repository root:
43+
terrain serve --root /path/to/other/repo
4144
```
4245

4346
## What's safe

docs/glossary.md

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -40,10 +40,18 @@ the snapshot and emits zero or more signals. See
4040
## Posture
4141

4242
The five-dimension health summary Terrain produces:
43-
**reliability**, **change-readiness**, **speed**, **governance**,
44-
and **AI**. Each dimension is backed by a band (A/B/C/D) plus
45-
contributing measurements. See
46-
[`docs/scoring-rubric.md`](scoring-rubric.md).
43+
**health**, **coverage_depth**, **coverage_diversity**,
44+
**structural_risk**, **operational_risk**. Each dimension is
45+
backed by a band (`strong` / `moderate` / `weak` for health-style
46+
dimensions; `low` / `moderate` / `elevated` / `critical` for
47+
risk-style dimensions, where lower = better). The companion view
48+
in `terrain insights` is the **Health Grade** (A / B / C / D), a
49+
distinct construct documented in
50+
[`docs/health-grade-rubric.md`](health-grade-rubric.md). See
51+
[`docs/product/posture-model.md`](product/posture-model.md) for
52+
the full dimension descriptions and
53+
[`docs/scoring-rubric.md`](scoring-rubric.md) for the underlying
54+
scoring constants.
4755

4856
## Severity clause
4957

docs/product/ai-risk-tiers.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,9 @@ artifacts in CI.
9898
- `schemaParseFailure`, `wrongSourceSelected` — pipeline metadata
9999

100100
These fire only when the corresponding eval artifact is present
101-
(via `terrain ai run` or `--ingest-only`). On a repo without eval
101+
(via `terrain ai run`, or via `terrain analyze` with
102+
`--promptfoo-results` / `--deepeval-results` / `--ragas-results`
103+
pointing at pre-existing eval output). On a repo without eval
102104
output, Terrain silently emits zero of these — that's the contract.
103105

104106
## How this surfaces in output

docs/product/ai-trust-boundary.md

Lines changed: 41 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -72,34 +72,61 @@ sandbox tool calls. Terrain is one layer above.
7272
shell command), the framework decides what's allowed. Terrain
7373
does not add a sandbox layer in 0.2.
7474

75-
### `terrain ai run --baseline <path>` / `--ingest-only`
75+
### Ingesting pre-existing eval results without invoking the framework
7676

77-
`--ingest-only` skips the framework invocation and only reads
78-
existing eval-output JSON files (via `--promptfoo-results`,
79-
`--deepeval-results`, `--ragas-results`). In this mode Terrain is
80-
fully passive — no child processes, no LLM calls.
77+
If you want Terrain to read eval-output JSON files without
78+
invoking any framework — useful when you run the eval framework
79+
yourself in your own sandbox — use `terrain analyze` with the
80+
adapter flags:
81+
82+
```
83+
terrain analyze --root . \
84+
--promptfoo-results path/to/promptfoo-output.json \
85+
--deepeval-results path/to/deepeval-output.json \
86+
--ragas-results path/to/ragas-output.json
87+
```
88+
89+
In this mode Terrain is fully passive: it parses the artifact
90+
files, surfaces ingestion diagnostics (per-field fallbacks), and
91+
flows the results through the same signals + posture pipeline as a
92+
fresh `analyze`. No child processes, no LLM calls.
93+
94+
For baseline-aware regression gating (compare a current run to a
95+
known-good snapshot), use:
96+
97+
```
98+
terrain ai record ./baselines/known-good.json # snapshot a good run
99+
terrain ai baseline compare --against ./baselines/known-good.json
100+
```
81101

82102
## Sandboxing roadmap
83103

84104
| Capability | 0.2 | 0.3 |
85105
|------------|-----|-----|
86106
| Read-only filesystem access in `terrain analyze` | yes | yes |
107+
| Pre-computed-artifact ingestion via `terrain analyze --*-results` | yes | yes |
87108
| Sandbox `terrain ai run` child processes | no | yes |
88109
| Tool-call allowlist for AI tool invocations | no (framework's job) | terrain-side allowlist |
89110
| Network egress controls during eval execution | no (framework's job) | optional terrain-side network policy |
90111

91-
If you need 0.3-grade sandboxing in 0.2, run `terrain ai run` with
92-
`--ingest-only` and execute the eval framework yourself in your
93-
preferred sandbox (Docker, gVisor, Firecracker, etc.). Terrain
94-
will read the framework's output without invoking it.
112+
If you need 0.3-grade sandboxing in 0.2, run the eval framework
113+
yourself in your preferred sandbox (Docker, gVisor, Firecracker,
114+
etc.) and use `terrain analyze --promptfoo-results` /
115+
`--deepeval-results` / `--ragas-results` to ingest the resulting
116+
JSON. Skip `terrain ai run` entirely — it's the only path that
117+
spawns a child process.
95118

96119
## Detector boundary (pure-static side)
97120

98-
The 12 AI risk detectors that ship in 0.2 (`aiPromptInjectionRisk`,
99-
`aiHardcodedAPIKey`, `aiToolWithoutSandbox`, etc.) are all **pure
100-
static analysis** — they read source code on disk, never invoke an
101-
LLM. False positives are heuristic; AST-grade taint analysis lands
102-
in 0.3.
121+
The AI risk detectors that ship in 0.2 (`aiPromptInjectionRisk`,
122+
`aiHardcodedAPIKey`, `aiToolWithoutSandbox`,
123+
`aiModelDeprecationRisk`, `aiFewShotContamination`,
124+
`aiNonDeterministicEval`, `aiPromptVersioning`,
125+
`aiSafetyEvalMissing`, `aiToolWithoutSandbox`,
126+
`aiEmbeddingModelChange`, plus the inventory-tier signals) are all
127+
**pure static analysis** — they read source code on disk, never
128+
invoke an LLM. False positives are heuristic; AST-grade taint
129+
analysis lands in 0.3.
103130

104131
## What Terrain doesn't promise about AI risk in 0.2
105132

0 commit comments

Comments
 (0)