Skip to content

AGENTS.md

Latest commit

 

History

History
200 lines (163 loc) · 16.8 KB

AGENTS.md

File metadata and controls

200 lines (163 loc) · 16.8 KB
Raw

contextdb — Agent Rules

Two different jobs bring an agent here. Pick one; you do not need the other half.

  • Use contextdb — drive the database from the CLI or embed it as a Rust library. Go to Use contextdb. No verification gate applies to you.
  • Contribute to contextdb — change the code in this repo. Go to Contribute. The gate there is binding.

Safety boundaries (both jobs, read before you run anything)

Reading is safe by default

Plain contextdb <path> is a bounded read session; the --write flag, missing-store refusal, owner-vs-file routing, and that .help rewrites nothing are in docs/cli.md.

contextdb ./their.db                              # bounded read session, never modifies
echo ".tables" | contextdb ./their.db --json       # reads without copying anything
contextdb ./their.db --write                       # explicit read-write session

Destructive commands need explicit confirmation

Every row below changes or destroys durable data. Run none of them against a store you did not create for this task until the operator confirms that exact path. Sole exception: an ephemeral, task-scoped store you created yourself (:memory:, or a file under $(mktemp -d)) — there, run them freely and without asking.

Goal Sanctioned command Guardrail
Find out why a store will not open contextdb diagnose <path> Read-only. Always safe. Start here.
Recreate a wedged or corrupt store contextdb reset <path> --force Destroys all data. Recover what you need from a backup or a healthy sync peer first. --force is mandatory; without it the CLI refuses and exits 2.
Take a backup before any of the above contextdb snapshot export <path> <dest> Purge-fenced; the safe prerequisite for reset.
Apply declared RETAIN windows now instead of waiting for the background cycle .maintenance run (REPL meta-command) Synchronous; expires rows the table's own DDL already declared expirable.
Erase rows permanently contextdb purge <path> --table <t> --force Purge-fenced; refuses without --force (exit 2). Only legal on a store that has never connected to a sync hub.
Make a route, sink or schedule actually fire Run the DDL in a :memory: CLI session; inspect with .events status In an ephemeral :memory: session the CLI registers default callbacks, so routes deliver and schedules fire observably. On a file-backed store, events queue durably until your own program registers its callbacks (register_sink / register_cron_callback) — that durable queueing is the contract, not a gap.

One cargo build at a time, and check the disk first

cargo test --workspace grows target/ to 55–90 GB, and incremental rebuilds keep growing it. Never run two cargo commands at once (they deadlock on the target/ lock) and never start a workspace build while another is live on the box — chain with && and wait. First:

df -h .                             # want ~100G free for a full workspace test run
export CARGO_PROFILE_DEV_DEBUG=0    # roughly halves target/, keeps backtraces usable

Use contextdb

Need binaries? cargo build --release -p contextdb-cli -p contextdb-server (minutes on a cold cache), or use an existing target/release; install options in docs/getting-started.md.

Throwaway store to try anything against, machine-readable output:

db="$(mktemp -d)/scratch.db"
printf "%s\n" \
  "CREATE TABLE decisions (id UUID PRIMARY KEY, status TEXT NOT NULL);" \
  "INSERT INTO decisions VALUES ('550e8400-e29b-41d4-a716-446655440000', 'draft');" \
  "SELECT * FROM decisions;" | contextdb "$db" --write --json
# {"rows_affected":0}
# {"rows_affected":1}
# {"result":{"columns":["id","status"],"rows":[{"id":"550e8400-e29b-41d4-a716-446655440000","status":"draft"}]}}

--write is what creates the store and authorizes the CREATE/INSERT; drop it once you are only reading the store back.

Prefer --json and read named fields; never scrape the human table output, which reflows freely. JSON document shapes and the four exit codes are in docs/cli.md.

Task-shaped recipes, copy-paste runnable. Read the one matching what you are doing.

Skill What it's for Path
using-contextdb Open a database, define tables, run SQL (including multi-line paste), read --json, recover after a refused statement, branch on exit codes. skills/using-contextdb/SKILL.md
querying-the-graph Declare a DAG edge table, traverse it with GRAPH_TABLE, get cycle inserts refused, enforce STATE MACHINE transitions with PROPAGATE cascades. skills/querying-the-graph/SKILL.md
running-triggers-and-schedules Declare CREATE EVENT TYPE/SINK/ROUTE/SCHEDULE, see them fire observably in :memory:, and wire register_sink/register_cron_callback so a file-backed store's durably-queued events actually deliver. skills/running-triggers-and-schedules/SKILL.md
sync Stand up a hub, enroll an edge with its ticket, push/pull, change synced columns explicitly, and read the applied / skipped / conflicts counts correctly. skills/sync/SKILL.md
vector-search Embedding columns, <=> nearest-neighbour search, schema-declared USE RANK policies, and the hybrid graph + vector query. skills/vector-search/SKILL.md
work-fabric Hand a job to another machine over the work ledger and move the bytes it needs over the blob plane. Library API — no CLI. skills/work-fabric/SKILL.md
operating-a-store Enforce RETAIN windows on demand with .maintenance run, permanently erase rows with purge (never-connected stores only), back up before a destructive op, and read pull_pages_read sync liveness. skills/operating-a-store/SKILL.md

Reference docs, when a skill is not enough:

Doc What it covers
docs/getting-started.md Install, first REPL session, embedding as a library, two-machine sync
docs/cli.md Every flag and meta-command, the --json document shapes, the exit-code table
docs/query-language.md SQL surface, GRAPH_TABLE traversal, vector search, constraints, what is unsupported
docs/architecture.md Crate map, MVCC, sync protocol, work ledger and blob plane, store format and recovery
docs/usage-scenarios.md Walkthroughs with SQL
docs/why-contextdb.md Problem statement and comparison with alternatives
docs/capability-index.md One page: what contextdb is, is not, and the numbers it stops at
docs/agent-readiness.md How this repo measures whether AI assistants can use and contribute to it

contextdb ships no built-in schema; the decisions example and that framing live in the README. Conflict policy (KEEP FIRST vs KEEP LATEST) is in docs/query-language.md, with one recipe in skills/sync.

Contribute (change this repo)

Verification gate

The nine-command verification gate is in CONTRIBUTING.md. All nine must pass before any commit, release, or "done" claim. Read the disk and single-build rules in Safety boundaries before you start cargo test --workspace. Narrower suites while iterating: cargo test -p contextdb-engine --test acceptance, --test integration, --test sql_surface.

How CI is organised — its paths, each PR check, and the local bash scripts/verify command behind each check — is in .github/CI.md. If you edit anything under .github/ or a CI script (scripts/verify, scripts/ci-*), also read .github/AGENTS.md; its rules cover those scripts even though they live outside .github/.

Where a change lives

Describe the capability, then go to the crate that owns it — do not go looking by filename.

Change Where it lands
A new scalar SQL function ONE match arm in the engine crate's executor (eval_function), plus the ORDER BY allowlist (is_known_scalar_function), plus tests. The parser takes function calls generically and the planner needs nothing.
A function that works in SELECT but is refused in ORDER BY You added it to eval_function only. Those two lists are hand-synced with no cross-reference — add every new function to both.
Grammar, DDL clauses, PROPAGATE Parser crate; grammar reference is docs/query-language.md.
Partial synced ALTER, explicit column drop/rename, ordered schema replay Engine database.rs (author markers, schema projection and received-unit replay); durable installation in persistence.rs. Never infer a drop or rename from omitted columns.
Sync protocol, conflict arbitration, work ledger Engine crate owns the implementations; the server crate re-exports them. Never add a mirror module in the server — extend the engine and re-export.
How a bounded read session opens and which route it takes Engine crate, read_session.rs — it assembles the one public ReadSession and its two routes. Adding a third route, or a second way to open a store for reading, is out of bounds.
Reading a store nobody owns (the committed-snapshot file route) Engine crate, direct_file_reader.rs.
Serving readers from a live writer (admission, client, service) Engine crate, owner_read/.
Addressing, authenticating, framing or timing the owner channel Engine crate, local_transport/. Never open a second transport beside it.
How a read is actually executed and charged against its budgets Engine crate, executor/bounded.rs — the one execution kernel both routes share. Never give a consumer its own execution semantics.
The trusted companion, the claim window, reader holds, published reader identities Engine crate, persistence.rs.
A refusal class or kind a reader can branch on Core crate, read_contract.rs — the shared typed vocabulary. Never define a second refusal type downstream of it.
Vector search: routes, partition layouts, the global top-k merge, generation lifecycle, per-column vector policy Vector crate (store.rs, hnsw.rs, mem.rs). Raw payload sharing and each column's charge ownership live in store.rs / quantized.rs; generation persistence and maintenance scheduling stay in the engine crate.
Vector generation admission, catalog inventory, journal cleanup or cancellation Engine database.rs, database/vector_maintenance_inventory.rs, database/vector_source_page_admission.rs and persistence.rs. Vector store.rs selects resident frontier-live inputs; cold snapshots and delete-only carriers retain their raw-history path.
A graph primitive vector search cannot build on top of the HNSW fork HNSW crate, as the smallest upstream-shaped change, plus its upstream-0.3.4.patch and MAINTENANCE.md. Anything composable goes in the vector crate's hnsw.rs instead.
A storage primitive (compaction step, read admission, bounded byte-table lookup or cursor) the engine cannot build on top of redb redb fork crate, same rule: smallest change, patch and MAINTENANCE.md updated. Byte-node validation and reject-only windows live in tree_store/bounded_bytes.rs; compaction and inventory policy stay in the engine.
A contextdb-server flag, exit code, automatic push trigger, or worker poll loop Server crate (main.rs, exit_codes.rs, sync_plugin.rs, work_ledger.rs).
Anything else docs/architecture.md maps all 11 crates and the dependency direction.

Testing discipline

  • No nondeterministic or wall-clock-dependent tests. Assert on state, counters, or events — never on elapsed time, and never sleep as synchronization. A negative claim ("nothing fired") is a counter read after a deterministic drive. One documented exception: commit_index_reconstruction_is_not_quadratic, where the time bound IS the promise.
  • Time-dependent behavior uses the clock seam. Two audits enforce it on every gate; the mechanics (Wallclock::test_clock_guard, the thread-local limitation, the ratchet you may lower but never raise) are in crates/contextdb-engine/AGENTS.md.
  • When timestamp_audit fails on a change that touched no timestamp columns: its whitelist is pinned by declaration text, so a line-count shift in a whitelisted file does not trip it. A new created_at/valid_from/valid_to TIMESTAMP declaration, or a removed whitelisted declaration, does. Do not disable or re-scope the audit. It scans git-tracked files only.
  • Testability touches in src/ are production-dead — #[doc(hidden)] ..._for_test accessors or unused-in-production seams. Anything else is a behavior change and needs its own proof (benchmark + full gate), stated in the commit.
  • Never record test timing with a redirected TMPDIR. Temp DBs on a real disk pay ~40× fsync cost vs tmpfs and silently corrupt every figure. Unset it before timing, and state the TMPDIR/filesystem alongside any recorded timing.
  • Deleting or merging tests requires mutation-testing evidence that coverage is preserved (compare per-mutant results before and after, not summary counts).

Crate guides

Every crate carries a guide: what the crate owns and must not own, its seams, each invariant with the test that guards it, where a change lands, and the fast test command. A guide adds to this file rather than overriding it. Read the matching one before you edit that crate. docs/architecture.md maps every crate and the dependency direction.

Crate Guide Rule to know before editing
contextdb-cli crates/contextdb-cli/AGENTS.md The CLI and REPL surface; behavior belongs to the engine.
contextdb-core crates/contextdb-core/AGENTS.md Shared types, errors and the read-contract vocabulary; the workspace audits live in its tests/.
contextdb-engine crates/contextdb-engine/AGENTS.md Clock-seam discipline: every persisted timestamp goes through Wallclock::now(); the test-estate ratchet audit enforces it.
contextdb-graph crates/contextdb-graph/AGENTS.md Adjacency index and bounded BFS; DAG cycle refusal on insert.
contextdb-hnsw crates/contextdb-hnsw/AGENTS.md Maintained fork of hnsw_rs 0.3.4: the delta stays minimal and additive code lives in contextdb-vector; see its MAINTENANCE.md.
contextdb-parser crates/contextdb-parser/AGENTS.md Char-boundary discipline: every fixed-width lookahead over input must be boundary-safe, or multi-byte UTF-8 panics the parser.
contextdb-planner crates/contextdb-planner/AGENTS.md Turns the parsed statement into a physical plan.
contextdb-redb crates/contextdb-redb/AGENTS.md Maintained fork of redb 4.1.0, outside the workspace: the delta stays minimal and additive code lives in the engine; see its MAINTENANCE.md.
contextdb-relational crates/contextdb-relational/AGENTS.md Row storage, scan, insert, upsert and delete.
contextdb-server crates/contextdb-server/AGENTS.md The sync mover: sync semantics are the engine's and re-exported; never a mirror module.
contextdb-tx crates/contextdb-tx/AGENTS.md MVCC transaction manager and deferred-apply write sets.
contextdb-vector crates/contextdb-vector/AGENTS.md Partitioned maintained vector search; most of its tests need --features test-seams.

Releases

Needs cargo install cargo-release. Run cargo release {patch|minor|major} --execute --workspace --no-publish — patch for bug fixes, minor for backward-compatible features, major for breaking API changes. It bumps every crate, commits, tags, and pushes; CI (org context-graph-ai, repo contextdb, runs on main and PRs) then publishes crates, Docker images, GitHub releases, and the contextdb.tech docs deploy. No cargo login needed — CI holds the token.