Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 21 additions & 16 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -104,12 +104,13 @@ jobs:
--estimates target/criterion/receipt_hash/compute_receipt_hash_mid_chain/new/estimates.json \
--threshold 0.20

# ── ADR-0006/0007/0008: event primitive safety gates ────────────────────────
# The ring and published-prefix algorithms compile once with native std
# atomics/UnsafeCell and once with Loom's modeled equivalents. This job stays
# separate from the gateway matrix so the hot-path crate cannot accidentally
# acquire Tokio, SQLx, protobuf, or system-service dependencies. CRC32C is the
# only default runtime edge; Loom is optional model-checking infrastructure.
# ── ADR-0006/0007/0008/0009: event primitive safety gates ────────────────────
# The ring, published-prefix, permit/claim, and composite-admission algorithms
# compile once with native std atomics/UnsafeCell and once with Loom's modeled
# equivalents. This job stays separate from the gateway matrix so the hot-path
# crate cannot accidentally acquire Tokio, SQLx, protobuf, or system-service
# dependencies. CRC32C is the only default runtime edge; Loom is optional
# model-checking infrastructure.
event-concurrency:
name: Event primitives native + Loom
runs-on: ubuntu-latest
Expand All @@ -126,7 +127,7 @@ jobs:
run: cargo test -p aegis-event
- name: All-feature correctness and feature-isolation tests
run: cargo test -p aegis-event --all-features
- name: Loom ring and published-prefix publication models
- name: Loom ring, published-prefix, and admission models
run: cargo test -p aegis-event --features loom loom_
- name: All-feature Clippy
run: cargo clippy -p aegis-event --all-targets --all-features -- -D warnings
Expand All @@ -135,7 +136,7 @@ jobs:
test "$(cargo tree -p aegis-event --edges normal --depth 1 --prefix none | tail -n +2)" = "crc32c v0.6.8"
test "$(cargo tree -p aegis-event --edges normal --depth 1 --prefix none --all-features | tail -n +2)" = "$(printf 'crc32c v0.6.8\nloom v0.7.2')"
- name: Compile diagnostic event benchmarks
run: cargo bench -p aegis-event --bench spsc_ring --bench published_slab --no-run
run: cargo bench -p aegis-event --bench spsc_ring --bench published_slab --bench admission --no-run

event-miri:
name: Event primitives Miri
Expand All @@ -149,14 +150,14 @@ jobs:
with:
workspaces: .
key: event-miri
- name: Miri ring, sealed-page, and published-prefix ownership suite
- name: Miri ring, slab-page, and admission ownership suite
run: cargo miri test -p aegis-event

# Rust nightly currently exposes ASan and TSan but no `undefined` sanitizer.
# Miri is therefore the Rust UB/provenance gate; any future C/C++ ABI must add
# a real UBSan lane before that boundary can be accepted.
event-sanitizers:
name: Event published-prefix ${{ matrix.sanitizer }} sanitizer
name: Event slab/admission ${{ matrix.sanitizer }} sanitizer
runs-on: ubuntu-latest
strategy:
fail-fast: false
Expand All @@ -171,16 +172,20 @@ jobs:
with:
workspaces: .
key: event-${{ matrix.sanitizer }}-sanitizer
- name: Native raw-pointer and cross-thread stress
- name: Native slab and composite-admission stress
env:
RUSTFLAGS: -Zsanitizer=${{ matrix.sanitizer }}
ASAN_OPTIONS: detect_leaks=1:halt_on_error=1
TSAN_OPTIONS: halt_on_error=1
run: >-
cargo test -Zbuild-std
--target x86_64-unknown-linux-gnu
-p aegis-event
--test published_slab
run: |
cargo test -Zbuild-std \
--target x86_64-unknown-linux-gnu \
-p aegis-event \
--test published_slab
cargo test -Zbuild-std \
--target x86_64-unknown-linux-gnu \
-p aegis-event \
--test admission

# ── #1194 (Postgres GA): live Postgres integration smoke test ────────────
# Closes the biggest concrete gap named in
Expand Down
6 changes: 6 additions & 0 deletions .github/workflows/sast.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,16 @@ jobs:
steps:
- uses: actions/checkout@v7
- name: Run semgrep (custom rules + Rust/Python security rulesets)
# The community blanket unsafe-usage rule is replaced by the
# path-scoped rust-unsafe-outside-reviewed-crates rule in
# .semgrep/aegisagent-rust.yml: `lib/event` carries isolated,
# SAFETY-documented unsafe under ADR-0006..0009 with its own Miri,
# Loom, ASan and TSan CI lanes; everywhere else unsafe still blocks.
run: |
semgrep scan \
--config .semgrep/ \
--config p/rust \
--config p/python \
--config p/secrets \
--exclude-rule rust.lang.security.unsafe-usage.unsafe-usage \
--error
29 changes: 29 additions & 0 deletions .semgrep/aegisagent-rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,35 @@ rules:
references:
- .claude/rules/rust_standards.md

- id: rust-unsafe-outside-reviewed-crates
languages: [rust]
severity: ERROR
message: >-
`unsafe` outside a crate with an accepted/proposed unsafe-review scope.
Repository law (docs/architecture.md §8, CONTRIBUTING.md) requires every
unsafe block to be isolated, justified with a `// SAFETY:` comment, and
covered by Miri/sanitizers (plus Loom for atomics) — and the crate must
be allowlisted here once that evidence exists in CI. Currently only
`lib/event` (ADR-0006..0009: SPSC ring, published-prefix slab pages,
failure-atomic admission; gated by the event-concurrency, event-miri and
event-sanitizers CI lanes) is allowlisted. This replaces the community
rust.lang.security.unsafe-usage rule, excluded in
.github/workflows/sast.yml.
patterns:
- pattern-either:
- pattern: unsafe { ... }
- pattern: unsafe fn $F(...) { ... }
paths:
exclude:
- "lib/event/*"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The glob pattern lib/event/* is non-recursive in Semgrep and will only exclude files directly under the lib/event/ directory (such as Cargo.toml). It will not exclude files in subdirectories like lib/event/src/ring.rs, which will cause Semgrep to fail on the unsafe blocks in those files. Use lib/event/** to recursively exclude the entire directory.

        - "lib/event/**"

metadata:
category: security
references:
- docs/architecture.md
- CONTRIBUTING.md
- docs/adr/0006-cache-padded-spsc-event-fabric.md
- docs/adr/0009-failure-atomic-slab-ring-admission.md

- id: rust-unredacted-secret-logging
languages: [generic]
severity: ERROR
Expand Down
18 changes: 17 additions & 1 deletion ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,22 @@ There is one SPSC ring for each producer→consumer edge. A writer shard polls i

The fixed ring does not need general garbage collection: its sequence barrier proves slot reuse. `crossbeam-epoch` is limited to slab-page retirement, snapshot publication, manifest generations, and readers that can outlive a ring slot.

The `current`, unwired `VolatileAdmissionChannel` prototype binds one
preallocated page to one preallocated ring. It validates page capacity before
reserving a ring slot, Release-publishes the complete immutable page prefix,
and only then Release-publishes the descriptor ring head. Its consumer claims a
descriptor without releasing ring capacity, validates canonical page
membership, bounds, and CRC32C, and advances the consumed tail only when a
must-use frame lease commits. Explicit clean/faulted terminal state prevents an
orphaned page prefix or ordinary producer drop from being reported as a clean
stream. This process-local prototype carries no production or `shadow`
traffic, cannot carry protected evidence, is not `qualified`, and establishes
no performance result. The production event fabric remains `target`; WAL
durability, formal ADR/security review, green hosted ASan/TSan artifacts, real
UBSan support, authenticated registry lookup, bounded page rotation, reuse,
epoch/NUMA reclamation, shadow evidence, rollback, and qualification remain
separate gates.

### Event admission states

```mermaid
Expand All @@ -193,7 +209,7 @@ stateDiagram-v2
Validating --> Spooling: critical ring saturated
Validating --> Rejected: normal ring saturated
Reserved --> Published: release sequence
Published --> Consumed: writer acquire sequence
Published --> Consumed: validate and commit frame lease
Consumed --> Durable: WAL policy satisfied
Durable --> Indexed: memtable/segment visible
Spooling --> Published: capacity restored
Expand Down
9 changes: 5 additions & 4 deletions MIGRATION_MATRIX.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,10 @@
**Audited baseline HEAD:** `f027d07` (`feat(tool-broker): extract connector execution into standalone aegis-tool-broker binary`)

**Post-audit delta:** the `current` branch adds unwired `aegis-event` SPSC,
sealed generation-tagged slab-page, and append-only published-prefix prototypes
under ADR-0006 through ADR-0008. The lexical counts below remain the frozen
baseline so future excision is measured against one reproducible commit.
sealed generation-tagged slab-page, append-only published-prefix, and
failure-atomic single-page slab/ring admission prototypes under ADR-0006
through ADR-0009. The lexical counts below remain the frozen baseline so future
excision is measured against one reproducible commit.

**Companion documents:** [HLD](ARCHITECTURE.md), [LLD](docs/LLD.md), [Roadmap](ROADMAP.md)

Expand Down Expand Up @@ -245,7 +246,7 @@ The v1 router’s 146 path literals cover agents, tools, MCP, authorization, ing
| Target capability | Current evidence | Gap |
|---|---|---|
| Thread-per-core reactor | Tokio multi-thread runtime; no affinity crate/config | No core ownership, per-core listener, NUMA allocation, or io_uring reactor |
| Disruptor/SPSC event fabric | Tokio bounded MPSC remains `current`; unwired `lib/event` adds 64-byte-separated cursors, Acquire/Release sequences, 32-byte descriptors, bounded closure/drop behavior, a safe seal-before-publish differential oracle, and an append-only page whose packed Release/Acquire state publishes descriptor count, byte watermark, and closure for immediate immutable-prefix resolution. Evidence includes native stress, the same publication algorithm under Loom, full Miri, defined ASan/TSan CI lanes, and a zero-allocation append-plus-resolve test. | The production fabric remains `target`, neither `shadow` nor `qualified`, and has no performance result. Blockers are green sanitizer CI artifacts, ADR acceptance/security review, composite ring reservation/admission, authenticated registry lookup, bounded page rotation/outstanding pages, generation reuse/epochs, NUMA-owner reclamation, priority lanes, production shadow wiring, UBSan support in the Rust toolchain, and qualification. |
| Disruptor/SPSC event fabric | Tokio bounded MPSC remains `current`; unwired `lib/event` adds 64-byte-separated cursors, Acquire/Release sequences, 32-byte descriptors, bounded closure/drop, cancelable producer permits, commit-delayed consumer claims, a safe sealed-page oracle, an append-only published prefix, and `VolatileAdmissionChannel` composition. The composite validates before reservation, publishes the page before the ring, withholds capacity until a validated frame lease commits, and distinguishes clean, faulted, and orphaned-prefix termination. Test sources include safe short-trace differential coverage, native tiny-ring stress, shipping-algorithm Loom models, Miri-oriented lifetime cases, defined ASan/TSan CI lanes, and zero-allocation admission/claim checks. | The production fabric remains `target`, neither `shadow` nor `qualified`, carries no protected evidence, and has no performance result. Blockers are formal ADR acceptance/security review, green hosted sanitizer artifacts, real UBSan support, authenticated registry lookup, bounded page rotation/outstanding pages, WAL durability/replay, generation reuse/epochs, NUMA-owner reclamation, priority lanes, production shadow wiring, release-artifact rollback, and qualification. |
| HCMT/Arrow SSTables | SQLite/PostgreSQL rows; JSON/TEXT payloads | No Arrow dependency, WAL format, memtable, segment manifest, compactor, or mmap query path |
| Gorilla timestamp codec | none | Codec, block restart points, fallback-to-raw rule, corpus absent |
| Roaring pruning | none | Bitmap build/serialization/planner absent |
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ Authorization compute and protected commit are measured separately. A warm deter
| APIs | REST JSON plus partial tonic/protobuf | protobuf-first parity; binary fast path; REST compatibility off benchmark path |
| Control storage | SQLite/PostgreSQL via `StorageBackend` | split transactional `ControlStore`/`ReceiptLog` |
| Telemetry storage | row tables and JSON/TEXT fields | WAL + Arrow-compatible HCMT SSTables |
| Event bus | Tokio bounded MPSC; `current`, unwired `aegis-event` SPSC, safe sealed-page, and append-only published-prefix prototypes with packed Release/Acquire count, byte-watermark, and closure publication | `target` NUMA-local cache-padded SPSC/slab matrix after composite admission, registry, rotation, epoch/reclamation, shadow, safety, and qualification gates |
| Event bus | Tokio bounded MPSC; `current`, unwired `aegis-event` SPSC, safe sealed-page, append-only published-prefix, and failure-atomic single-page volatile admission prototypes with cancelable permits, commit-delayed claims, and explicit clean/faulted termination; no protected evidence or performance claim | `target` NUMA-local SPSC/slab matrix after formal ADR/security review, hosted sanitizer and UBSan evidence, authenticated registry, bounded rotation, WAL durability/replay, epochs/NUMA reclamation, shadow, rollback, and qualification gates |
| Detection | structured scalar rules; optional Qdrant | Aho DFA plus owned HNSW/PQ and isolated INT8 ONNX |
| Host sensor | procfs polling, spool, signed commands | CO-RE eBPF telemetry/containment with truthful fallback |
| Console | React JSON polling and SVG | Arrow IPC worker, Rust WASM, WebGL2 instancing |
Expand Down
40 changes: 23 additions & 17 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,26 +66,32 @@ Gate: legacy versus typed authorization decisions, hashes, approvals, receipts a
### Week 4 — SPSC ring and slab prototype

Progress (2026-07-13): the `current` checkout has unwired ring, safe sealed-page
oracle, and append-only published-prefix prototypes under Proposed ADR-0006
through ADR-0008. Packed Release/Acquire state publishes descriptor count, byte
watermark, and closure for immediate immutable-prefix resolution. Evidence
includes a safe sealed differential corpus, native stress, the same publication
algorithm under Loom, full Miri, defined ASan/TSan CI lanes, and a zero-allocation
append-plus-resolve test. The production fabric remains `target`, neither
`shadow` nor `qualified`, and has no performance result. ADR acceptance and
security review, green sanitizer CI artifacts, composite ring reservation/admission, authenticated registry,
bounded page rotation/outstanding pages, generation reuse/epochs, NUMA-owner
reclamation, production shadow wiring, UBSan support in the current Rust
toolchain, and qualification remain blockers.

Deliverables:

- implement cache-padded SPSC ring, 32-byte descriptor and NUMA-local slab prototype;
oracle, append-only published-prefix, and failure-atomic single-page admission
prototypes under Proposed ADR-0006 through ADR-0009. The volatile composite
validates before reservation, Release-publishes the page before the ring,
withholds capacity until a must-use validated frame lease commits, and reports
clean, faulted, and orphaned-prefix terminal states. Test sources include safe
differential oracles, native tiny-ring stress, shipping-algorithm Loom,
Miri-oriented borrow/drop cases, defined ASan/TSan CI lanes, and zero-allocation
admission/claim checks. It carries no production or `shadow` traffic, cannot
carry protected evidence, is not `qualified`, and has no performance result.
The production fabric remains `target`. Formal ADR acceptance/security review,
green hosted sanitizer artifacts, real UBSan support, authenticated registry,
bounded page rotation/outstanding pages, WAL durability/replay, generation
reuse/epochs, NUMA-owner reclamation, priority lanes, production shadow wiring,
release-artifact rollback, and qualification remain blockers.

Deliverables:

- implement cache-padded SPSC ring, 32-byte descriptor, bounded slab, and single-page volatile admission prototype;
- document linearization, memory ordering, shutdown, wrap, drop and epoch rules;
- add scalar reference queue, Loom model, Miri tests and native stress benchmark;
- add safe differential oracles, Loom models, Miri tests and native stress benchmark;
- instrument allocations, copied bytes, cache misses and cycles/op.

Gate: zero lost/duplicated descriptors; zero steady-state allocations; safety suite green; no false sharing in layout/perf evidence.
Gate: zero lost/duplicated/reordered descriptors; validation before tail
acknowledgement; zero steady-state allocations; safety suite and hosted
sanitizer evidence green; no false sharing in layout/perf evidence. These gates
do not make the volatile prototype durable or authorize protected evidence.

### Week 5 — CoreReactor prototype

Expand Down
4 changes: 2 additions & 2 deletions docs/AegisAgent_World_Class_HLD.md
Original file line number Diff line number Diff line change
Expand Up @@ -451,7 +451,7 @@ evidence:

## Installation

This HLD does not replace the product installation guide. Use [Installation](Installation.md) for prerequisites and [Deployment Guide](deployment-guide.md) for Docker, Helm, and production configuration. Both REST 8080 and gRPC 6334 must be exposed where the deployment model permits access.
This HLD does not replace the product installation guide. Use [Installation](installation.md) for prerequisites and [Deployment Guide](deployment-guide.md) for Docker, Helm, and production configuration. Both REST 8080 and gRPC 6334 must be exposed where the deployment model permits access.

## Quick Start

Expand Down Expand Up @@ -513,7 +513,7 @@ Any missing dual-protocol operation is a contract gap, not permission to impleme

## CLI

Operational CLI commands must expose the same typed configuration, support a read-only `config validate`, and provide benchmark modes that report decision class, protocol, offered load, achieved throughput, latency histogram, error class, and queue utilization. Exact shipped setup and invocation commands remain documented in [Installation](Installation.md) and [Deployment Guide](deployment-guide.md).
Operational CLI commands must expose the same typed configuration, support a read-only `config validate`, and provide benchmark modes that report decision class, protocol, offered load, achieved throughput, latency histogram, error class, and queue utilization. Exact shipped setup and invocation commands remain documented in [Installation](installation.md) and [Deployment Guide](deployment-guide.md).

## Configuration Reference

Expand Down
2 changes: 1 addition & 1 deletion docs/AegisAgent_World_Class_LLD.md
Original file line number Diff line number Diff line change
Expand Up @@ -343,7 +343,7 @@ Validation rules:

## Installation

No new endpoint is considered installed until protobuf generation tools are present and both protocol suites pass. Follow [Installation](Installation.md); PostgreSQL target mode additionally requires migrations, pooling, backups, and a completed backend qualification matrix.
No new endpoint is considered installed until protobuf generation tools are present and both protocol suites pass. Follow [Installation](installation.md); PostgreSQL target mode additionally requires migrations, pooling, backups, and a completed backend qualification matrix.

## Quick Start

Expand Down
Loading
Loading