Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 78 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,84 @@ jobs:
--estimates target/criterion/receipt_hash/compute_receipt_hash_mid_chain/new/estimates.json \
--threshold 0.20

# ── ADR-0006/0007/0008: event primitive safety gates ────────────────────────
# The ring and published-prefix algorithms compile once with native std
# atomics/UnsafeCell and once with Loom's modeled equivalents. This job stays
# separate from the gateway matrix so the hot-path crate cannot accidentally
# acquire Tokio, SQLx, protobuf, or system-service dependencies. CRC32C is the
# only default runtime edge; Loom is optional model-checking infrastructure.
event-concurrency:
name: Event primitives native + Loom
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
with:
components: clippy
- uses: Swatinem/rust-cache@v2
with:
workspaces: .
key: event-concurrency
- name: Native correctness and stress tests
run: cargo test -p aegis-event
- name: All-feature correctness and feature-isolation tests
run: cargo test -p aegis-event --all-features
- name: Loom ring and published-prefix publication models
run: cargo test -p aegis-event --features loom loom_
- name: All-feature Clippy
run: cargo clippy -p aegis-event --all-targets --all-features -- -D warnings
- name: Assert default and all-feature runtime allowlists are exact
run: |
test "$(cargo tree -p aegis-event --edges normal --depth 1 --prefix none | tail -n +2)" = "crc32c v0.6.8"
test "$(cargo tree -p aegis-event --edges normal --depth 1 --prefix none --all-features | tail -n +2)" = "$(printf 'crc32c v0.6.8\nloom v0.7.2')"
- name: Compile diagnostic event benchmarks
run: cargo bench -p aegis-event --bench spsc_ring --bench published_slab --no-run

event-miri:
name: Event primitives Miri
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@nightly
with:
components: miri
- uses: Swatinem/rust-cache@v2
with:
workspaces: .
key: event-miri
- name: Miri ring, sealed-page, and published-prefix ownership suite
run: cargo miri test -p aegis-event

# Rust nightly currently exposes ASan and TSan but no `undefined` sanitizer.
# Miri is therefore the Rust UB/provenance gate; any future C/C++ ABI must add
# a real UBSan lane before that boundary can be accepted.
event-sanitizers:
name: Event published-prefix ${{ matrix.sanitizer }} sanitizer
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
sanitizer: [address, thread]
steps:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@nightly
with:
components: rust-src
- uses: Swatinem/rust-cache@v2
with:
workspaces: .
key: event-${{ matrix.sanitizer }}-sanitizer
- name: Native raw-pointer and cross-thread stress
env:
RUSTFLAGS: -Zsanitizer=${{ matrix.sanitizer }}
ASAN_OPTIONS: detect_leaks=1:halt_on_error=1
TSAN_OPTIONS: halt_on_error=1
run: >-
cargo test -Zbuild-std
--target x86_64-unknown-linux-gnu
-p aegis-event
--test published_slab

# ── #1194 (Postgres GA): live Postgres integration smoke test ────────────
# Closes the biggest concrete gap named in
# docs/adr/0002-sqlite-first-storage.md for Postgres GA: until this job,
Expand Down Expand Up @@ -614,4 +692,3 @@ jobs:
--set secret.create=true \
--set secret.apiToken=example-token \
> /dev/null

52 changes: 52 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ members = [
"src/canon",
"lib/common",
"lib/api",
"lib/event",
"lib/storage",
"lib/policy",
"lib/soc",
Expand Down Expand Up @@ -37,3 +38,4 @@ hmac = "0.12"
axum = { version = "0.7.5", features = ["macros", "ws"] }
cedar-policy = "3.2.1"
utoipa = { version = "4.2", features = ["axum_extras", "uuid", "chrono"] }
crc32c = "=0.6.8"
11 changes: 8 additions & 3 deletions MIGRATION_MATRIX.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,16 @@

**Status:** normative migration blueprint; target components are not shipped until their exit gates pass

**Audit date:** 2026-07-12
**Audit date:** 2026-07-13

**Scope:** repository state at this checkout versus the Thread-Per-Core + HCMT target

**Audited HEAD:** `f027d07` (`feat(tool-broker): extract connector execution into standalone aegis-tool-broker binary`)
**Audited baseline HEAD:** `f027d07` (`feat(tool-broker): extract connector execution into standalone aegis-tool-broker binary`)

**Post-audit delta:** the `current` branch adds unwired `aegis-event` SPSC,
sealed generation-tagged slab-page, and append-only published-prefix prototypes
under ADR-0006 through ADR-0008. The lexical counts below remain the frozen
baseline so future excision is measured against one reproducible commit.

**Companion documents:** [HLD](ARCHITECTURE.md), [LLD](docs/LLD.md), [Roadmap](ROADMAP.md)

Expand Down Expand Up @@ -240,7 +245,7 @@ The v1 router’s 146 path literals cover agents, tools, MCP, authorization, ing
| Target capability | Current evidence | Gap |
|---|---|---|
| Thread-per-core reactor | Tokio multi-thread runtime; no affinity crate/config | No core ownership, per-core listener, NUMA allocation, or io_uring reactor |
| Disruptor/SPSC event fabric | Tokio bounded MPSC | No sequence barriers, cache padding, slab ownership, or epoch retirement |
| Disruptor/SPSC event fabric | Tokio bounded MPSC remains `current`; unwired `lib/event` adds 64-byte-separated cursors, Acquire/Release sequences, 32-byte descriptors, bounded closure/drop behavior, a safe seal-before-publish differential oracle, and an append-only page whose packed Release/Acquire state publishes descriptor count, byte watermark, and closure for immediate immutable-prefix resolution. Evidence includes native stress, the same publication algorithm under Loom, full Miri, defined ASan/TSan CI lanes, and a zero-allocation append-plus-resolve test. | The production fabric remains `target`, neither `shadow` nor `qualified`, and has no performance result. Blockers are green sanitizer CI artifacts, ADR acceptance/security review, composite ring reservation/admission, authenticated registry lookup, bounded page rotation/outstanding pages, generation reuse/epochs, NUMA-owner reclamation, priority lanes, production shadow wiring, UBSan support in the Rust toolchain, and qualification. |
| HCMT/Arrow SSTables | SQLite/PostgreSQL rows; JSON/TEXT payloads | No Arrow dependency, WAL format, memtable, segment manifest, compactor, or mmap query path |
| Gorilla timestamp codec | none | Codec, block restart points, fallback-to-raw rule, corpus absent |
| Roaring pruning | none | Bitmap build/serialization/planner absent |
Expand Down
7 changes: 5 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,12 +77,15 @@ Authorization compute and protected commit are measured separately. A warm deter
| APIs | REST JSON plus partial tonic/protobuf | protobuf-first parity; binary fast path; REST compatibility off benchmark path |
| Control storage | SQLite/PostgreSQL via `StorageBackend` | split transactional `ControlStore`/`ReceiptLog` |
| Telemetry storage | row tables and JSON/TEXT fields | WAL + Arrow-compatible HCMT SSTables |
| Event bus | Tokio bounded MPSC | NUMA-local cache-padded SPSC ring matrix |
| Event bus | Tokio bounded MPSC; `current`, unwired `aegis-event` SPSC, safe sealed-page, and append-only published-prefix prototypes with packed Release/Acquire count, byte-watermark, and closure publication | `target` NUMA-local cache-padded SPSC/slab matrix after composite admission, registry, rotation, epoch/reclamation, shadow, safety, and qualification gates |
| Detection | structured scalar rules; optional Qdrant | Aho DFA plus owned HNSW/PQ and isolated INT8 ONNX |
| Host sensor | procfs polling, spool, signed commands | CO-RE eBPF telemetry/containment with truthful fallback |
| Console | React JSON polling and SVG | Arrow IPC worker, Rust WASM, WebGL2 instancing |

No target row in this table is a shipped claim until its roadmap gate passes.
The event prototypes are `current` only as isolated, unwired code. The
production fabric remains `target`, neither `shadow` nor `qualified`, carries
no protected evidence, and has no performance claim. No target row in this
table is a shipped claim until its roadmap gate passes.

## Quick start

Expand Down
13 changes: 13 additions & 0 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,19 @@ Gate: legacy versus typed authorization decisions, hashes, approvals, receipts a

### Week 4 — SPSC ring and slab prototype

Progress (2026-07-13): the `current` checkout has unwired ring, safe sealed-page
oracle, and append-only published-prefix prototypes under Proposed ADR-0006
through ADR-0008. Packed Release/Acquire state publishes descriptor count, byte
watermark, and closure for immediate immutable-prefix resolution. Evidence
includes a safe sealed differential corpus, native stress, the same publication
algorithm under Loom, full Miri, defined ASan/TSan CI lanes, and a zero-allocation
append-plus-resolve test. The production fabric remains `target`, neither
`shadow` nor `qualified`, and has no performance result. ADR acceptance and
security review, green sanitizer CI artifacts, composite ring reservation/admission, authenticated registry,
bounded page rotation/outstanding pages, generation reuse/epochs, NUMA-owner
reclamation, production shadow wiring, UBSan support in the current Rust
toolchain, and qualification remain blockers.

Deliverables:

- implement cache-padded SPSC ring, 32-byte descriptor and NUMA-local slab prototype;
Expand Down
15 changes: 9 additions & 6 deletions docs/Documentation_Quality_Report.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ This inventory makes the all-documentation improvement program measurable. Each
|---|---:|---:|---:|
| authoring | 2 | 90% | 0 |
| component | 15 | 96% | 0 |
| decision | 7 | 95% | 0 |
| decision | 10 | 95% | 0 |
| flow | 11 | 95% | 0 |
| guide | 53 | 67% | 38 |
| landing | 10 | 93% | 0 |
Expand All @@ -21,7 +21,7 @@ This inventory makes the all-documentation improvement program measurable. Each
| reference | 11 | 85% | 0 |
| runbook | 7 | 100% | 0 |

**Total:** 123 active Markdown pages · **Migration backlog:** 38 pages below 75%.
**Total:** 126 active Markdown pages · **Migration backlog:** 38 pages below 75%.

## Scoring signals

Expand Down Expand Up @@ -84,7 +84,10 @@ Pages are sorted by structural coverage, then path. Improve factual accuracy and
| [adr/0003-aegis-jcs-1-canonicalization.md](adr/0003-aegis-jcs-1-canonicalization.md) | decision | 90 | A | 100% |
| [adr/0004-ed25519-receipt-signing.md](adr/0004-ed25519-receipt-signing.md) | decision | 85 | B | 83% |
| [adr/0005-fail-closed-defaults.md](adr/0005-fail-closed-defaults.md) | decision | 97 | A | 100% |
| [adr/index.md](adr/index.md) | decision | 44 | B | 83% |
| [adr/0006-cache-padded-spsc-event-fabric.md](adr/0006-cache-padded-spsc-event-fabric.md) | decision | 189 | B | 83% |
| [adr/0007-sealed-generation-tagged-slab-pages.md](adr/0007-sealed-generation-tagged-slab-pages.md) | decision | 267 | A | 100% |
| [adr/0008-append-only-published-prefix-slab-pages.md](adr/0008-append-only-published-prefix-slab-pages.md) | decision | 409 | A | 100% |
| [adr/index.md](adr/index.md) | decision | 50 | B | 83% |
| [adr/template.md](adr/template.md) | decision | 46 | A | 100% |
| [AegisAgent_Agent_Cage.md](AegisAgent_Agent_Cage.md) | guide | 581 | B | 78% |
| [AegisAgent_Agent_SOC_Design.md](AegisAgent_Agent_SOC_Design.md) | guide | 757 | D | 56% |
Expand Down Expand Up @@ -134,7 +137,7 @@ Pages are sorted by structural coverage, then path. Improve factual accuracy and
| [components/Tool_Broker.md](components/Tool_Broker.md) | component | 66 | A | 100% |
| [concepts.md](concepts.md) | guide | 415 | B | 78% |
| [contributing/documentation-standard.md](contributing/documentation-standard.md) | authoring | 207 | B | 80% |
| [current-vs-roadmap.md](current-vs-roadmap.md) | reference | 144 | A | 100% |
| [current-vs-roadmap.md](current-vs-roadmap.md) | reference | 146 | A | 100% |
| [database-schema.md](database-schema.md) | reference | 289 | A | 100% |
| [demo-github-attack.md](demo-github-attack.md) | guide | 220 | B | 78% |
| [deployment-guide.md](deployment-guide.md) | guide | 302 | A | 100% |
Expand All @@ -161,12 +164,12 @@ Pages are sorted by structural coverage, then path. Improve factual accuracy and
| [github-integration.md](github-integration.md) | guide | 125 | D | 56% |
| [Glossary.md](Glossary.md) | landing | 100 | A | 100% |
| [How_It_Works.md](How_It_Works.md) | landing | 75 | A | 100% |
| [Implementation_Status.md](Implementation_Status.md) | reference | 107 | B | 80% |
| [Implementation_Status.md](Implementation_Status.md) | reference | 126 | B | 80% |
| [index.md](index.md) | landing | 109 | B | 86% |
| [installation.md](installation.md) | guide | 127 | A | 100% |
| [Issue_Backlog_Execution_Plan.md](Issue_Backlog_Execution_Plan.md) | guide | 208 | D | 44% |
| [Last_Mile_System_Walkthrough.md](Last_Mile_System_Walkthrough.md) | guide | 147 | C | 67% |
| [LLD.md](LLD.md) | guide | 1454 | A | 100% |
| [LLD.md](LLD.md) | guide | 1537 | A | 100% |
| [Local_Development.md](Local_Development.md) | guide | 87 | D | 44% |
| [mcp-defense-architecture.md](mcp-defense-architecture.md) | guide | 145 | C | 67% |
| [mission.md](mission.md) | guide | 153 | D | 44% |
Expand Down
19 changes: 19 additions & 0 deletions docs/Implementation_Status.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,25 @@
> - **Unknown-agent runtime control** (cage execution loop + real sensor telemetry + forced egress) is **not** production-complete.
> - **Multi-replica K8s** requires Postgres GA (#1194); Helm defaults remain `replicaCount: 1`.

## v2 architecture migration ledger

This ledger uses the canonical `current`, `shadow`, `target`, and `qualified`
vocabulary from [architecture.md](architecture.md). The larger v1 capability
matrix below retains its historical release labels until it is migrated as a
separate documentation change.

| Artifact | Status | Evidence in this checkout | Authority / traffic | Remaining gates |
|---|---|---|---|---|
| Unwired SPSC, sealed-page oracle, and published-prefix prototypes | current | `lib/event/`; ring FIFO/full/wrap/drop/layout; safe sealed differential corpus; packed Release/Acquire descriptor-count, byte-watermark, and closure publication with immediate immutable-prefix resolution; native stress; same-algorithm Loom; full Miri; ASan/TSan CI lanes defined; zero-allocation append-plus-resolve test; ADR-0006 through ADR-0008 | No production or `shadow` traffic; cannot carry protected evidence; no performance claim | Green sanitizer CI artifacts, ADR acceptance/security review, composite ring reservation/admission, authenticated registry, bounded page rotation/outstanding pages, generation reuse/epochs, NUMA-owner reclamation, production shadow wiring, UBSan support, qualification |
| Thread-per-core reactor | target | `ARCHITECTURE.md`, `docs/LLD.md` | None | runtime ADR, core-affinity/io_uring implementation, migration and benchmark gates |
| HCMT telemetry store | target | `ARCHITECTURE.md`, `docs/LLD.md` | None; SQL remains authoritative/current | WAL/segment ADR, recovery corpus, dual write, shadow equality, qualification |

No v2 component is `qualified` in this checkout. The SPSC, sealed-page, and
published-prefix prototypes being `current` means only that their isolated,
unwired code and listed tests are present. The production event fabric remains
`target`, is neither `shadow` nor `qualified`, and is not
production-authoritative.

| Capability | Status | Current files | Missing pieces | Related issues | Test coverage | Prod-ready | Next PR |
|---|---|---|---|---|---|---|---|
| Gateway authorize | Implemented | `src/src/routes/authorize.rs`, `authorize_decision.rs`, `authorize_canon.rs` | — | #1305–#1313 | unit + integration + bench + fuzz (canon) | prod | perf follow-ups |
Expand Down
Loading
Loading