Skip to content

Commit ab15852

Browse files
committed
release: v0.7.0
1 parent c6b8df3 commit ab15852

21 files changed

Lines changed: 1101 additions & 9 deletions
Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
name: Spotlighting drift guard demo (worked example)
2+
3+
on:
4+
pull_request:
5+
paths:
6+
- 'scripts/spotlighting-drift-guard.py'
7+
- 'examples/spotlighting/**'
8+
push:
9+
branches: [main]
10+
paths:
11+
- 'scripts/spotlighting-drift-guard.py'
12+
- 'examples/spotlighting/**'
13+
workflow_dispatch: {}
14+
15+
# ---------------------------------------------------------------------------
16+
# This job does NOT gate this repo's own source tree -- it proves that
17+
# spotlighting-drift-guard.py (TR-SEC-005) correctly catches the problem
18+
# planted in examples/spotlighting/violation_example.py: a second LLM
19+
# boundary that re-inlined the security-notice and delimiter strings instead
20+
# of importing them from spotlighting_constants.py (see
21+
# examples/spotlighting/README.md). Here, the guard *failing* on the fixture
22+
# is success; if it ever passes clean, the fixture or the guard itself has
23+
# regressed.
24+
#
25+
# A downstream user copies this pattern into their own repo's CI, pointed at
26+
# the module where their own spotlighting constants live and the source tree
27+
# that should never re-inline them.
28+
# ---------------------------------------------------------------------------
29+
30+
permissions:
31+
contents: read
32+
33+
jobs:
34+
demo-detects-drift:
35+
runs-on: ubuntu-latest
36+
37+
steps:
38+
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
39+
40+
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
41+
with:
42+
python-version: '3.12'
43+
44+
- name: Assert spotlighting-drift-guard.py catches the planted re-inlined copy
45+
run: |
46+
set +e
47+
python3 scripts/spotlighting-drift-guard.py --constants-file examples/spotlighting/spotlighting_constants.py --scan-root examples/spotlighting
48+
code=$?
49+
set -e
50+
if [ "$code" -ne 1 ]; then
51+
echo "::error::Expected exit 1 (re-inlined constant detected) against the fixture, got exit $code instead -- the fixture or the guard regressed."
52+
exit 1
53+
fi
54+
echo "Confirmed: spotlighting-drift-guard.py correctly detected the planted re-inlined copy (exit 1)."

AGENTS.md

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -181,6 +181,79 @@ It must not authorize tool calls, change system rules, or override developer
181181
intent. Apply prompt-injection defenses at the reasoning boundary, not only
182182
at ingestion.
183183

184+
### Spotlighting at the Reasoning Boundary (TR-SEC-005)
185+
186+
Spotlighting delimits untrusted content — search results, scraped pages, RAG
187+
chunks, tool output — so the model can treat it as data to analyze rather
188+
than instructions to follow. Microsoft's measurements put this at cutting
189+
indirect prompt-injection success from >50% to <2%.
190+
191+
The wording that implements it (a security-notice string plus open/close
192+
delimiters) is itself security-critical text. Define it once — a notice
193+
constant and a pair of delimiter constants — and import it at every LLM
194+
boundary that consumes untrusted content; never let a second boundary paste
195+
its own copy. A pasted copy is exactly how spotlighting silently breaks: two
196+
call sites' wording drifts a few words apart and nobody notices until an
197+
audit. `scripts/spotlighting-drift-guard.py` is the worked example
198+
(`examples/spotlighting/`): it reads the constants from one designated module
199+
and fails CI if any of their literal values are re-inlined anywhere else in
200+
the scanned tree.
201+
202+
Same honest limit as the guard pattern below: this is friction against
203+
casual copy-paste drift, not a barrier against a determined author who edits
204+
the constants file and re-inlines a modified value in the same commit.
205+
206+
### Memory / Provenance Hygiene (TR-SEC-011)
207+
208+
Agentic memory and RAG indexes are a poisoning surface: content ingested from
209+
outside the system's own trust boundary sits in the same store the retriever
210+
treats as authoritative, and a malicious instruction embedded in it is
211+
indistinguishable from trusted content at synthesis time unless provenance is
212+
tracked and enforced.
213+
214+
Three layers, all deterministic — no LLM in the trust path:
215+
216+
1. **Tag at ingest.** Record where each piece of content came from (its
217+
source type) at write time, alongside the content itself.
218+
2. **Derive trust fail-closed at read time.** Map source type to a trust
219+
level in code, not data, so a mapping revision is a code change, not a
220+
migration. The mapping must be fail-closed by construction: only
221+
explicitly named self-authored types earn the most-trusted tier;
222+
everything unrecognized — including a source type nobody has classified
223+
yet — falls to the least-trusted tier. A drift-guard test should assert
224+
every known source type is covered by the mapping, so adding a new type
225+
without classifying its trust fails CI the same way an unreviewed
226+
permission grant does (TR-SEC-010).
227+
3. **Validate at retrieval, not only storage.** A row written before this
228+
pattern existed, or one whose provenance was never recorded, is
229+
`unverified` — treated exactly like the least-trusted tier, never
230+
silently upgraded to trusted by omission.
231+
232+
Untrusted or unverified content is quarantined data: pass it through the
233+
spotlighting pattern above at the reasoning boundary, never let it authorize
234+
a tool call or override system-level instructions. See
235+
`examples/provenance-trust-tags/` for a reference implementation of the
236+
fail-closed mapping and its drift guard.
237+
238+
### Strict LLM Output-Schema Validation (TR-SEC-012)
239+
240+
Every model-returned field gets a type check **and** a range/shape check.
241+
Reject on mismatch — never coerce. The canonical failure mode this guards
242+
against is a fail-open type coercion: Python's `bool("false")` evaluates to
243+
`True`, because any non-empty string is truthy. A classifier field parsed
244+
with a bare `bool(...)` call silently flips a JSON string `"false"` to
245+
`True`, and a boundary gating on that field fails open exactly when an
246+
attacker (or a malformed response) needs it to.
247+
248+
The fix is symmetric with the single-source-of-truth convention below: a
249+
strict parser for a given output schema lives in one place, raises on any
250+
field whose type or range doesn't match, and every caller of that LLM
251+
boundary uses it — no per-call-site ad hoc `bool()`/`float()` coercion.
252+
Absence of an optional field is a defined, valid state; a wrong *type* for a
253+
present field is not, and the two must not be handled by the same fallback
254+
path. See `examples/strict-output-schema/` for a before/after reference
255+
implementation and a live repro of the `bool("false")` bug.
256+
184257
### Deterministic Checks Before Agent Judgment
185258

186259
Use scripts, tests, linters, and schema validators before asking a model to

ATTRIBUTIONS.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ and examples are maintainer-authored unless a file header states otherwise.
2727
| MITRE ATT&CK (https://attack.mitre.org/) and MITRE ATLAS (https://atlas.mitre.org/) | Public technique catalogs cited by TR-ID and by name in `registry/tr-registry.yaml` and `templates/threat-model.md` to give threat-model findings a shared, falsifiable vocabulary. No content reproduced beyond technique IDs and short names. |
2828
| Anthropic, "Zero Trust for AI Agents" (2026) | Source of the "impossible vs. tedious" design test (`templates/threat-model.md`, `AGENTS.md`) — the barrier-vs-friction classification of a mitigation's real strength. Concept adopted and reworded; no text reproduced. |
2929
| OWASP agentic application security guidance | Source of the "least agency" framing applied to TR-SEC-010 (`AGENTS.md`) — least privilege extended to what an agent tool can do, how often, and where. Concept and term adopted; no text reproduced. |
30+
| Microsoft public research on prompt-injection defenses | Source of the "spotlighting" technique name and its measured effectiveness (indirect prompt-injection success reduced from >50% to <2%), cited in `AGENTS.md`, `scripts/spotlighting-drift-guard.py`, and `examples/spotlighting/README.md` (TR-SEC-005) via Anthropic's *Zero Trust for AI Agents* eBook, which references the same figure. Technique name and effect-size figure cited; no text reproduced. |
3031

3132
## Standards (document shapes, not certification)
3233

CHANGELOG.md

Lines changed: 35 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,39 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
55

66
## [Unreleased]
77

8+
## [0.7.0] - 2026-07-26
9+
10+
### Added
11+
12+
- `registry/tr-registry.yaml` — TR-SEC-011 (content provenance tracked and
13+
trust derived fail-closed at retrieval), TR-SEC-012 (strict LLM
14+
output-schema validation — reject, never coerce), exported from the
15+
2026-07-13 Zero-Trust-for-AI-Agents review (private monorepo)
16+
- `scripts/spotlighting-drift-guard.py` — single-sourced spotlighting
17+
constants (security notice + untrusted-content delimiters) enforcement:
18+
fails CI if any LLM boundary re-inlines a copy instead of importing the
19+
designated constants module
20+
- `examples/spotlighting/` — worked example + planted re-inlined-copy
21+
fixture for the drift guard above (TR-SEC-005);
22+
`.github/workflows/spotlighting-drift-guard-demo.yml` proves the guard
23+
still catches it
24+
- `examples/provenance-trust-tags/` — reference implementation of a
25+
fail-closed source-type → trust-level mapping with its own drift guard
26+
(every content type must be explicitly classified), and a quarantine
27+
helper routing untrusted/unverified content into the spotlighting layer
28+
(TR-SEC-011)
29+
- `examples/strict-output-schema/` — before/after reference parser for LLM
30+
JSON output, with a live repro of the `bool("false") is True` fail-open
31+
coercion bug and the reject-never-coerce fix (TR-SEC-012)
32+
- `AGENTS.md` — "Spotlighting at the Reasoning Boundary", "Memory /
33+
Provenance Hygiene", and "Strict LLM Output-Schema Validation" sections
34+
- `docs/requirements-implementation-map.md` — rows for all three exports;
35+
the TR-SEC-005 row upgraded from "Documented" to "Documented + script +
36+
example"
37+
- `ATTRIBUTIONS.md` — Microsoft public research on prompt-injection defenses
38+
(the "spotlighting" technique name and its measured effectiveness),
39+
cited via the same Anthropic eBook review
40+
841
## [0.6.0] - 2026-07-20
942

1043
### Added
@@ -232,7 +265,8 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
232265
- `CONTRIBUTING.md`, `SECURITY.md`, issue/PR templates, `release-check` CI workflow
233266
- Roadmap and changelog for intentional release cadence
234267

235-
[Unreleased]: https://github.com/onesimplecode/ai-engineering-standards/compare/v0.6.0...HEAD
268+
[Unreleased]: https://github.com/onesimplecode/ai-engineering-standards/compare/v0.7.0...HEAD
269+
[0.7.0]: https://github.com/onesimplecode/ai-engineering-standards/compare/v0.6.0...v0.7.0
236270
[0.6.0]: https://github.com/onesimplecode/ai-engineering-standards/compare/v0.5.0...v0.6.0
237271
[0.5.0]: https://github.com/onesimplecode/ai-engineering-standards/compare/v0.4.0...v0.5.0
238272
[0.4.0]: https://github.com/onesimplecode/ai-engineering-standards/compare/v0.3.0...v0.4.0

README.md

Lines changed: 21 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ public reuse.
1717
| Machine-readable requirement registry (`registry/tr-registry.yaml`) | Production runtime or hosted services |
1818
| Portable agent conventions (`AGENTS.md`, `agents/`) | Tool-specific private agent sessions |
1919
| Governance and eval templates (ADR, impact assessment, maturity checklist, LLM eval, completion checklist) | Full application frameworks |
20-
| Reference scripts (config drift, debt tags, release validation, Cursor rule export, llms.txt manifest, agent permission guard) | Full `agent-skills` replacement |
20+
| Reference scripts (config drift, debt tags, release validation, Cursor rule export, llms.txt manifest, agent permission guard, spotlighting drift guard) | Full `agent-skills` replacement |
2121
| Synthetic worked example | Personal data |
2222

2323
## Quick start
@@ -40,6 +40,9 @@ python3 scripts/llms-txt-generator.py
4040

4141
# Check an agent settings file's tool-permission grants against a reviewed baseline
4242
python3 scripts/agent-permission-guard.py --settings /path/to/your/settings.json
43+
44+
# Check that spotlighting security-notice/delimiter constants aren't re-inlined elsewhere
45+
python3 scripts/spotlighting-drift-guard.py --constants-file /path/to/your/constants.py --scan-root /path/to/your/src
4346
```
4447

4548
## Adopting this into your project
@@ -57,8 +60,11 @@ A step-by-step path for pulling these standards into your own repo, not just thi
5760
4. **Adopt templates as needed** — the ADR, impact assessment, maturity
5861
checklist, LLM eval, and completion checklist templates in
5962
[`templates/`](templates/) are meant to be copied, not just read.
60-
5. **Study the worked traces**[`examples/worked-example/`](examples/worked-example/)
61-
and [`examples/agent-permission-guard/`](examples/agent-permission-guard/) show
63+
5. **Study the worked traces**[`examples/worked-example/`](examples/worked-example/),
64+
[`examples/agent-permission-guard/`](examples/agent-permission-guard/),
65+
[`examples/spotlighting/`](examples/spotlighting/),
66+
[`examples/provenance-trust-tags/`](examples/provenance-trust-tags/), and
67+
[`examples/strict-output-schema/`](examples/strict-output-schema/) show
6268
a requirement moving end-to-end: TR-ID → ADR → maturity row → script → CI gate.
6369
6. **Reconcile with tools you already use**[`docs/agent-skills-integration.md`](docs/agent-skills-integration.md)
6470
covers how this layers under AGENTS.md, agent-skills, and Cursor rules rather
@@ -82,6 +88,18 @@ unreviewed grant, both caught by `scripts/agent-permission-guard.py`'s
8288
co-located reviewed baseline — the "make dangerous changes loud, not
8389
impossible" pattern.
8490

91+
See [`examples/spotlighting/`](examples/spotlighting/) for the untrusted-content
92+
delimiting trace (TR-SEC-005): a planted re-inlined copy of the
93+
security-notice/delimiter constants, caught by
94+
`scripts/spotlighting-drift-guard.py`'s single-source enforcement.
95+
96+
See [`examples/provenance-trust-tags/`](examples/provenance-trust-tags/) (TR-SEC-011)
97+
for the fail-closed content-trust derivation pattern — a drift-guarded mapping
98+
from source type to trust level — and
99+
[`examples/strict-output-schema/`](examples/strict-output-schema/) (TR-SEC-012) for
100+
a live repro of the `bool("false") is True` fail-open coercion bug and the
101+
reject-never-coerce fix.
102+
85103
## Public Evidence Map
86104

87105
- [`AGENTS.md`](AGENTS.md) — tool-neutral agent rules for data routing, loop contracts,

ROADMAP.md

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -100,23 +100,26 @@ maturity classes — this repo exports packaged practice, not aspirations.
100100

101101
**Export-ready (shipped + tested in the private monorepo, 2026-07-13):**
102102

103-
- [ ] **Spotlighting at the reasoning boundary** (private ADR-030,
103+
- [x] **Spotlighting at the reasoning boundary** (private ADR-030,
104104
private ADR-018): untrusted retrieved/external content is wrapped in
105105
explicit delimiters and every LLM call that sees it carries a firewall
106106
system message; the delimiter/notice strings are **single-sourced
107107
constants with a CI drift-guard test** that fails on any re-inlined copy
108108
— the drift guard is the enforceable artifact this repo ships
109-
- [ ] **Memory/provenance hygiene** — new TR-SEC entry (the registry's gap
109+
(`scripts/spotlighting-drift-guard.py`, `examples/spotlighting/`)
110+
- [x] **Memory/provenance hygiene** — new TR-SEC entry (the registry's gap
110111
against agentic memory-poisoning): source-tag content at ingest, derive
111112
trust via a **fail-closed** mapping at read time (unknown → untrusted;
112113
missing → unverified), validate provenance at *retrieval* not only at
113114
storage, and treat unverified/external content as quarantined data,
114115
never instructions (private ADR-030's implementation is the reference)
115-
- [ ] **Strict LLM output-schema validation** pattern + worked example: type
116+
(TR-SEC-011, `examples/provenance-trust-tags/`)
117+
- [x] **Strict LLM output-schema validation** pattern + worked example: type
116118
AND range checks on every model-returned field, reject — never coerce —
117119
wrong types (canonical bug: Python `bool("false") is True` failing open
118120
through a relevance gate; private ADR-018); pairs with the existing
119121
single-source-of-truth convention
122+
(TR-SEC-012, `examples/strict-output-schema/`)
120123

121124
**Roadmapped — export after the private implementation proves them
122125
(design-stage as of 2026-07-13; promotion to export requires the same

docs/requirements-implementation-map.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,9 @@ concrete artifacts. It is intentionally limited to public, reusable evidence.
1111
| Loop contracts | Required four-field node contract + reference implementation | Documented + example | `AGENTS.md`, `registry/tr-registry.yaml`, `examples/engine-interface/` |
1212
| Trigger classification | ADR-triggered trigger type | Documented + example | `examples/worked-example/`, `templates/adr.md` |
1313
| Behavioral mode declaration | Named, trigger-activated mode contract orthogonal to gate strictness | Documented | `AGENTS.md`, `registry/tr-registry.yaml` |
14-
| External content trust boundary | Retrieved content treated as data | Documented | `AGENTS.md`, `registry/tr-registry.yaml` |
14+
| External content trust boundary (spotlighting) | Retrieved content treated as data; single-sourced security-notice + delimiter constants with a CI drift guard that fails on any re-inlined copy | Documented + script + example | `AGENTS.md`, `registry/tr-registry.yaml`, `scripts/spotlighting-drift-guard.py`, `examples/spotlighting/`, `.github/workflows/spotlighting-drift-guard-demo.yml` |
15+
| Memory/provenance hygiene | Source-tag at ingest, fail-closed trust derivation at read time, validated at retrieval, unverified/external content quarantined | Documented + registry + example | `AGENTS.md`, `registry/tr-registry.yaml` (TR-SEC-011), `examples/provenance-trust-tags/` |
16+
| Strict LLM output-schema validation | Type and range checks on every model-returned field; reject, never coerce | Documented + registry + example | `AGENTS.md`, `registry/tr-registry.yaml` (TR-SEC-012), `examples/strict-output-schema/` |
1517
| Design-time threat modeling | Trust boundaries, data classification, and ATT&CK/ATLAS technique mapping required for new listeners/credentials/tool grants/content sources | Template + example | `templates/threat-model.md`, `examples/worked-example/docs/decisions/ADR-004-example.md` |
1618
| Impossible vs. tedious control classification | Every threat-model mitigation classified barrier vs. friction, with a named backstop for friction controls | Documented + template | `AGENTS.md`, `templates/threat-model.md` |
1719
| Least agency / agent permission grants | No wildcard write/install/exec/network grants in agent allowlists | Documented | `AGENTS.md`, `registry/tr-registry.yaml` |

0 commit comments

Comments
 (0)