Every ambiguity in SPEC.md resolved to the simplest reasonable interpretation
(SPEC §0 rule 4), with rationale.
Ambiguity: SPEC §4.2 says a fallback module chunk's end_line is "number
of lines", which is ambiguous for trailing newlines.
Decision: end_line = max(1, content.lines().count()) — Rust's .lines()
counts logical lines and ignores a single trailing newline, matching the
splitlines() semantics most implementations use. The fixture README.md
("# Demo\nPayment and authentication utilities.\n") yields end_line = 2, as
expected. Empty content yields 1.
Decision: A single JSON file (<store>/index.json) via serde. Chosen for
determinism and debuggability over SQLite/bincode; corpora are small (SPEC §1.2).
Embeddings are stored inline so search needs no re-embedding of the corpus.
Ambiguity: SPEC §7 requires idempotent re-indexing that "replaces prior data
for changed/removed files."
Decision: index rebuilds the whole store each run. Because chunk IDs are
deterministic, a full rebuild is idempotent and trivially handles changed/removed
files. Incremental indexing was out of scope for the benefit.
Ambiguity: SPEC §6.1 lists regex-ish triggers where is , find .* function,
.* defined.
Decision: Implemented as: contains "where is "; contains "defined";
contains "find" with "function" occurring after it. The keyword set
{function, class, method, def} is matched as whole tokens via the shared
tokenizer. None of the three conformance queries trigger CODE_LOOKUP, so this
interpretation does not affect the conformance gate.
Ambiguity: SPEC §6.5 mentions "file-hint" extraction but permits treating it
as none.
Decision: File hints = whitespace-split query terms containing a .
(lowercased), checked as substrings of file_path. The substring-of-content rule
still applies. Conformance queries contain no dotted terms, so this is inert
there.
Decision: keyword_distance lowercases chunk content and file paths with
ASCII case-folding (to_ascii_lowercase), consistent with the tokenizer's
ASCII-only lowercasing (SPEC §4.1). Query tokens are already ASCII-lowercased.
Ambiguity: SPEC §9 says invalid/empty inputs must not crash. Decision: A query that tokenizes to nothing (empty or all-separator) returns an empty result list rather than ranking by a zero vector.
Ambiguity: SPEC §10.1 says "Index the repo's Python sources," while cce index
indexes all text files.
Decision: cce bench filters the walk to *.py (via
Index::build_from_dir_filtered). This matches the spec's wording and keeps the
recall/token-savings numbers comparable to the Ruby implementation. General
cce index still indexes everything (module fallback for non-parsed files).
Ambiguity: SPEC §4.2 gives "react" → react and "./auth" → auth.
Decision: Split the specifier on /, drop empty / . / .. segments, take
the first remaining. Scoped packages (@scope/pkg) therefore resolve to
@scope; an acceptable edge case not exercised by the spec.
Ambiguity: cce search has no --embedder flag (SPEC §9) but ollama vectors
are model-specific.
Decision: The store records which embedder built it, and search uses the
same backend. Originally an unreachable server meant a warning and a hash-embedded
query — but that cosines a hash query vector against ollama-built embeddings, two
unrelated vector spaces, so retrieval quality collapsed silently. Revised
(issue #30): cce search on an ollama-built index with Ollama down now errors
with guidance (start Ollama, or re-index with the hash embedder); the MCP
context_search degrades to BM25-only under a pinned NOTICE: line. Index time
also fails loud: an embedding failure aborts cce index and writes no store.
Hash indexes (the default and conformance path) always use the hash embedder.
Decision: cargo llvm-cov for line/region coverage. CLI wiring (main.rs)
and the Ollama HTTP paths (network) are intentionally the least-covered; core
non-CLI logic exceeds the ≥85% target (SPEC §12).
Decision: serde #[derive(Serialize)] structs with fields declared in the
exact order shown in the SPEC §8.3 example, emitted with to_string_pretty.
serde serializes struct fields in declaration order, giving a deterministic,
spec-shaped layout every run (verified byte-identical across two runs).
Decision: Raised line coverage well above target by adding meaningful,
hermetic tests only — no behavior changed and conformance.json is byte-identical
to before. The gaps closed were the CLI wiring in main.rs (driven end-to-end
through the built binary in tests/cli.rs: default store-path resolution, human
vs JSON output, empty-query "(no results)", --embedder ollama fallback,
empty/missing-store stats, and bench/conformance against a tiny local temp
repo — never the flask corpus), plus config/store edge functions.
Ollama failure path: exercised without ever contacting a real
server by pointing OllamaEmbedder at a closed local port (127.0.0.1:1,
instant connection-refused). This covers try_embed_batch's error branch and
healthy() == false. (The silent empty-vector fallbacks this pass also covered
were removed by issue #30 — failures now propagate as errors; the loud-failure
policy is pinned hermetically in tests/ollama.rs against a loopback HTTP stub
via CCE_OLLAMA_URL.)
The Ollama HTTP success path (response parsing) remains uncovered by design —
it requires a live model server and is out of scope for the hermetic suite, so
embedder.rs is intentionally the last file below 100%.
Ambiguity: This clean-room package shipped without a rustfmt.toml, yet the
committed sources use a compact style (wide single-line calls/struct literals,
if/else wrapping ~60 cols) and a logical — not alphabetical — module order. Under
stock rustfmt 1.8.0 the tree is therefore not cargo fmt --check-clean, so the
formatting gate cannot pass as delivered.
Decision: Restored a minimal rustfmt.toml
(reorder_imports=false, reorder_modules=false, use_small_heuristics="Max",
single_line_if_else_max_width=60) that best reproduces the intended house style,
then ran cargo fmt. This makes cargo fmt --check genuinely clean while
preserving the compact style and module ordering rather than blowing them away to
stock defaults. Formatting-only; no logic or spec behavior changed.
The following ambiguities in DASHBOARD-SPEC.md were resolved to the simplest
reasonable reading and are recorded here.
HTTP server = hand-rolled on std::net::TcpListener. The spec offers a
choice between a minimal crate (e.g. tiny_http) and raw std::net. Raw std is
the smallest thing that works, adds no dependency (nothing new to pin or let
Dependabot chase), and the server surface is tiny (one request line, four routes,
Connection: close). Charts are hand-drawn inline SVG as required.
.jsonl files are excluded from indexing. The spec says to ship
test/fixture/metrics_sample.jsonl, but the conformance harness indexes
test/fixture/ and must keep producing exactly 7 chunks with a byte-identical
conformance.json. A .jsonl file is runtime log data, never source to be
chunked, so the walker now skips .jsonl (like it skips .cce/). This reconciles
both requirements: the fixture ships at the spec'd path and conformance.json is
unchanged. Verified byte-identical before and after.
--json search output becomes an object. DASHBOARD-SPEC §5 says to add a
top-level query_id field to --json. A "top-level field" implies an object, so
cce search --json now emits {"query_id": "...", "results": [ ... ]} instead of
a bare array. This is the documented v1.1 shape; query_id is null when metrics
are disabled.
Feedback for an unknown id warns but still records (exit 0). The spec allows either behaviour. Recording anyway is the more forgiving choice — feedback is cheap, ids are opaque, and a later re-index or log inspection can still make use of it. A warning is printed to stderr.
Metrics location derives from the store. The log lives beside the index at
<store-dir>/metrics.jsonl. --store PATH points at the index file, so the log
is PATH's parent directory plus metrics.jsonl; --dir D uses D/.cce/; and
--metrics PATH overrides both (dashboard/feedback only).
metrics.enabled config key vs --no-metrics. The base engine (SPEC v1.0)
ships no config-file loader — all configuration is compile-time constants — so
the runtime switch is the --no-metrics flag on index/search. A config-file
metrics.enabled=false is honoured in spirit by that flag; wiring an actual
config file is out of scope for v1.1 and would be a separate change.
Delta direction uses the rounded window means. delta_ratio /
delta_top_score are computed from the 6-decimal-rounded current/prior means, so
both language implementations reach an identical value and direction from the same
inputs (no last-ULP divergence). This matches the §4.1 anchor exactly.
Only named AST nodes become chunks. SPEC-V2 §1 says "for every node whose type
is in function_types/class_types emit a chunk". Some grammars name a
definition node the same string as its keyword token — e.g. tree-sitter-ruby's
class definition node and the anonymous class keyword both report
node.kind() == "class". Emitting for every matching node would double-count the
class (one chunk for the keyword token at its single line). The chunker therefore
guards on node.is_named(), so only real AST nodes are candidates. This is
correct for every pack (all function/class definition nodes are named) and keeps
counts sane. Ruby needs it; the others are unaffected.
A class node's references are emitted too (structural, not semantic). A pack
declares node types, not predicates. So a C struct Node *n parameter — a
bodyless struct_specifier reference — is emitted as a class chunk just like the
struct Node { … } definition. This keeps packs declarative and identical across
languages; the cost is a few noisy reference chunks. Both implementations follow
the same rule, so conformance stays byte-identical.
Conformance v2 drops the query section. SPEC-V2 §7 permits keeping or dropping
the base query section and makes the chunk array the equivalence gate. We drop it:
the samples are a multi-language corpus for which the old Python-specific queries
("hash password", …) are meaningless, and the orchestrator diffs the chunk
arrays. spec_version is "2.0", and each chunk gains kind.
spec_version for conformance is a dedicated constant. The persisted index
tag stays SPEC_VERSION = "1.0" (internal, not cross-checked); conformance emits
CONFORMANCE_SPEC_VERSION = "2.0" so both implementations agree on the v2 gate.
The base v1 fixture moved to test/fixture/base/. SPEC-V2 §6 places the
samples under test/fixture/samples/. Since the walker recurses, leaving the v1
fixture (auth.py, payments.py, README.md, metrics_sample.jsonl) at
test/fixture/ would fold the samples into every base-fixture test. Relocating the
v1 fixture into a sibling base/ keeps the two corpora independent; file paths
inside each stay root-relative and unchanged.
Import extraction is per-pack and NOT part of conformance. Imports feed only
the graph and each pack's own expected.imports self-test; they are absent from
conformance.json. So import rules need not agree byte-for-byte across languages,
only satisfy each pack's sample. Rust's optional mod name; import is therefore
omitted (the sample does not need it), keeping the rule to the first use-path
segment.
kind on the dashboard. SPEC-V2 §3 lists the dashboard among kind's
surfaces. The dashboard aggregates query-level metrics events, which carry no
per-chunk data, so there is no natural per-chunk kind to show without changing
the event schema and its cross-language §4.1 anchor. kind is therefore surfaced
where chunks are surfaced — search (human + --json), stats (by-kind
breakdown), and conformance — and carried through persistence; the dashboard is
left byte-stable. Recorded here as the resolution of that ambiguity.
Grammar crate versions. July-2026 latest that share the pinned tree-sitter
core's tree-sitter-language 0.1.x ABI: tree-sitter-ruby 0.23.1,
tree-sitter-rust 0.24.2, tree-sitter-typescript 0.23.2, tree-sitter-c 0.24.2
(TypeScript exposes LANGUAGE_TYPESCRIPT; the pack binds that, not the TSX
variant). All pinned with = in Cargo.toml.
The following ambiguities in SPEC-V2.1.md were resolved to the simplest
reasonable reading and are recorded here.
Generic pattern 10 never re-redacts an already-redacted value. SPEC-V2.1 §1
runs the nine specific patterns, then the generic key = value assignment. Its
example — token = "ghp_…" → token = "[REDACTED:GITHUB_TOKEN]" (not
[REDACTED:SECRET]) — is only reachable if the generic step skips a value the
specific step already turned into [REDACTED:…]. So the placeholder guard also
treats a leading [REDACTED: as "leave alone". This is forced by the spec's own
worked example, keeps redaction idempotent, and preserves the more precise label.
A leading-dot-only filename has no "extension". SPEC-V2.1 §1 compares "the
file's final extension". For a name like .env or .key we follow OS convention
(std::path::Path::extension), which reports no extension for a leading-dot
name. .env is handled by the dotenv rule and .pgpass/id_rsa/… by the
exact-basename rule; a bare .key (a hidden file, not a *.key secret) is not
treated as sensitive. This avoids surprising matches while covering every case the
fixture and spec name.
Layers are threaded through one builder, on by default. Index::build_protected
takes a protect_secrets bool; build_from_dir/build_from_dir_filtered keep
their signatures and pass true, so every existing caller (conformance, bench,
tests) stays secure-by-default with no change. Only cce index --allow-secrets
passes false. conformance therefore runs with protection on; because the
samples contain no secrets it is a no-op there and conformance.json stays
byte-identical (re-verified).
--allow-secrets scope = index. SPEC-V2.1 §2 says the flag applies to
index "and any command that indexes". The only user-facing indexing command is
index; conformance and bench index deterministically over fixtures with no
secrets, so they need no opt-out and keep protection on. Adding the flag solely to
index satisfies the requirement without widening the surface.
regex crate, pinned. The redaction patterns need real regex (lazy [\s\S]*?
for the private-key block, a closure over the generic match). regex = "=1.12.4"
was already resolved transitively, so promoting it to a direct, =-pinned
dependency adds no new code to the tree.
The following SPEC-V2.2.md ambiguities were resolved to the simplest reasonable
reading.
Federation is realised as a union corpus with member-namespaced paths. The
spec defines a workspace search as exactly the §6 retrieval over the union of
members' chunks, with a (member, file_path) diversity key. Rather than fork the
retriever, retriever::search is split into rank_core (the §6 pipeline without
graph expansion) and a thin search wrapper; federation builds a combined Index
whose chunk paths are namespaced <member>/<rel> and calls the same rank_core.
Namespacing makes the diversity key naturally (member, file_path) and lets BM25
statistics span the union — so the "single index over A+B" equivalence holds by
construction, and the namespace is stripped for output.
The combined graph is the union of per-member graphs, not a rebuild over the
union. Building one import graph over all namespaced files could resolve a module
name in member A to a same-stemmed file in member B, inventing a cross-member file
edge the spec does not want. Instead each member's own intra-store graph is unioned
(namespaced) via Graph::{out_pairs,from_pairs}; the only cross-member links are
the declared dependency edges, applied by a separate member-level expansion step.
The §8 fixture carries the engine/tsconfig markers the assertions require. The
§8 sketch lists billing as a ruby-engine and web as typescript, but the §3
detection rules only yield those types given a lib/**/engine.rb (engine) and a
tsconfig.json (typescript). The shipped fixture therefore includes
engines/billing/lib/billing/engine.rb and web/tsconfig.json — the minimal
markers that make the normative detection produce the types §8 asserts.
Workspace [<dir>] is an optional positional for search/stats/dashboard.
index --workspace [<dir>] already had a positional dir; to match the spec's
[<dir>] notation on the other workspace commands (whose single-repo forms use
--dir/--store), an optional positional DIR was added, preferred over --dir
in workspace mode. Single-repo behaviour is untouched.
Workspace search is read-only over member stores (no metrics write). A
federated search reads the members' stores and logs but does not append its own
search event to any member's metrics.jsonl (which member would own it?). The
--json output still carries a top-level query_id for shape compatibility; it is
generated, not persisted. The federated dashboard aggregates each member's existing
per-member events.
serde_yaml for reading, a hand-rolled writer for byte-determinism. The
manifest is emitted by a small canonical writer so the exact bytes are under our
control (and match across languages); serde_yaml = "=0.9.34" parses hand-written
manifests back. Emitting via serde_yaml was avoided because its formatting is not
guaranteed stable across versions or identical to another language's YAML library.
The two engines first diverged on the interchange artifact, so
SPEC-SYNC-RECONCILE.md pinned a single canonical
format. The decisions below reflect that reconciled format.
The interchange artifact is a hand-built canonical stream, not serde's default.
Cross-engine byte-identity (SPEC-SYNC §10) is a hard requirement. The artifact is
assembled explicitly: the manifest line, one object per chunk (sorted by
(file_path, start_line, id)), then the graph line — each an LF-terminated
compact JSON object, including the last line. Sorted keys come free from
serde_json's default Map (a BTreeMap) plus to_string (no whitespace); the
preserve_order feature is deliberately not enabled. The stream is a pure function
of its content.
Provenance is REMOVED entirely — no built_at, no built_by. An earlier draft
carried a git-derived built_at and a neutral built_by; the reconciliation
dropped both. Any provenance risks non-reproducibility and there is no need for it —
the checksum plus (repo_id, sha) already identify the artifact. Removing it makes
the manifest exactly {cce_version, checksum, chunk_count, embedder, file_tokens, pack_set_id, repo_id, sha} (sorted), which both engines produce identically.
The checksum covers the whole stream with checksum:"". checksum =
lowercase-hex SHA-256 over the entire canonical stream serialized with the
manifest's checksum field set to the empty string; the real hex is then written
in. Verify sets checksum to "", re-hashes, and compares. This is simpler than
the earlier "omit the field" rule and identical across engines (there is no
provenance to special-case). Independently reproduced with a standalone Python
SHA-256 over the emitted bytes.
Embeddings are standard base64 (with padding) of little-endian f64 bytes, never
decimals. Float→string formatting differs across languages, so serializing the
256-d vector as decimals would break byte-identity even though the vectors are
bit-equal (the hash embedder is deterministic). Encoding the raw IEEE-754 bytes
(f64::to_le_bytes) as RFC-4648 base64 with padding sidesteps formatting
entirely. base64 = "=0.22.1" is pinned; the standard alphabet + padding is
identical across engines (2048 bytes → 2732 base64 chars).
file_tokens lives in the MANIFEST (not the graph line). The dashboard's
baseline-tokens counterfactual (DASH §3) needs each file's whole-file token count,
which cannot be recomputed from chunks after import. It is fully deterministic
(max(1, bytes/4), a SPEC §3 constant both engines share). The canonical format
places it in the manifest as a sorted-key {path: int} object, keeping the
export→import round-trip lossless.
The graph line is {"edges":[…],"nodes":[…]} over the RESOLVED imports (base SPEC
§6.7). nodes are every indexed file ({"id": path}, sorted by id — derived from
file_tokens' keys); edges are the resolved file → file edges
({"source", "target", "type":"import"}, sorted by (source, target, type)): an
edge A → B exists only when a module imported by A resolves — by the same
stem-matching the retriever's graph expansion uses — to a corpus file B. External /
unresolved imports (os, fs, std, …) produce no edge, so samples yields
edges:[]. On import, file_imports is reconstructed by mapping each resolved
target back to a module name (its file stem) and grouping by source; re-building
the graph over those stems reproduces the identical file→file edges, so
search-expansion behaviour is preserved (the dropped external imports never produced
a hop). An earlier draft emitted the raw file → module edges; the reconciliation
pinned resolved file→file to match Ruby byte-for-byte.
pack_set_id is the literal sorted, comma-joined pack names. Not a hash: the
reconciled format uses the string c,javascript,python,ruby,rust,typescript
verbatim, so it is human-legible and trivially identical across engines (both
register the same six packs).
A per-branch ref pointer implements latest; content is addressed by sha.
Distinct shas are distinct files, so the artifacts never conflict in content. To
resolve "latest main," push also writes …/<repo_id>/refs/<branch> = sha in the
same commit, and pull --latest reads it. The pointer is a fixed path rewritten
by every push, so racing pushes genuinely conflict there (#92): every key is
whole-file last-writer-wins, and the git backend retries a lost push race by
re-applying the write on the freshly fetched remote state (proven by
deterministic race tests over both the code and knowledge keyspaces).
git is invoked via std::process, not a git library. The remote is "just a
git repo," and shelling out keeps CCE dependency-light and uses the user's real git
credentials/transport for free. Commits carry a fixed identity (-c user.name/email)
so they work in a bare CI/test environment.
git-LFS is default-on but never required by the core. sync init writes the
*.cce LFS .gitattributes and runs git lfs install when LFS is on. But the
whole test suite exercises artifact/push/pull/verify over plain git, so it needs
no git-lfs binary; LFS lives behind one smoke test that SKIPS gracefully when
git-lfs is absent (SPEC-SYNC §11).
The working-clone home is $CCE_HOME/sync (or ~/.cce/sync), overridable for
tests. Hermetic tests point CCE_HOME at a temp dir so a working clone never
touches the real ~/.cce. Because CCE_HOME is process-global and Cargo runs tests
in parallel threads, the sync tests serialize env access through one shared mutex.
The shared golden is on test/fixture/samples with a forced identity. The
cross-engine anchor indexes samples and builds the artifact with
repo_id = "cce/demo", sha = "0"*40. The test asserts the checksum
(581cbd0ff682a38d7d1250f3eec44f4ce456bdd660d4cb29aaaadd9e95072f48, confirmed equal
to Ruby's) and writes the raw bytes to /tmp/cce_artifact_rust.cce so the
orchestrator can diff it against Ruby byte-for-byte.
The branch-overlay for WIP is deferred (v1 fallback = full local index). If the
working tree differs from the pulled sha, pull says so and the user runs a normal
cce index. The incremental "reindex only changed files on top of the pulled base"
is a documented fast-follow, out of scope for v1 (SPEC-SYNC §7/§12).
Hand-rolled JSON-RPC 2.0 over stdio, no MCP SDK crate. The MCP stdio transport is
newline-delimited JSON-RPC 2.0. Rather than add an unvetted, larger MCP SDK, the
server hand-rolls exactly the slice it needs with serde_json (already a dependency)
— the same choice the rest of the engine makes for its hand-rolled HTTP/YAML writers.
This keeps every wire byte under our control, the dependency set pinned and minimal,
and the protocol trivially testable by piping strings. src/mcp/protocol.rs owns
request parsing and success/error encoding; src/mcp/server.rs owns the dispatch.
Protocol version pinned to 2025-06-18. The server advertises this MCP revision
in initialize and both engines pin the same value, so an agent negotiates an
identical protocol regardless of backend. The dispatch loop is transport-generic
(run<R: BufRead, W: Write>), so unit tests drive it in-process and the integration
suite pipes JSON-RPC to the real binary's stdin.
The dispatch loop is transport-generic; serve() wires it to process stdio. This
separation is what makes the server hermetically testable: handle_line is pure
(string in, optional string out), run loops over any reader/writer, and only
serve touches std::io::stdin/stdout (after the best-effort sync warm).
A missing index / unknown tool is a tool result, not a protocol error.
context_search over an unbuilt index returns a friendly "run cce index" text with
isError: false (it is a normal state, not a failure); an unknown tool name returns
isError: true. Only a malformed call (no query, no helpful, no tool name) is a
real error. This keeps an agent's session alive and steers it, rather than crashing.
context_search reuses the CLI's exact retrieval and logs an identical metrics
event. retriever::build_search_record was lifted out of main.rs into the library
so the CLI search and the MCP tool emit a byte-identical cce.metrics/v1 event —
that identity is what lets cce dashboard surface agent usage the same way it
surfaces CLI use. The MCP default top_k is 8 (tighter than the CLI's 10) because
an agent pays per token. Workspace metrics land in the workspace-root log so the root
cce dashboard --workspace sees them.
cce init merges idempotently by owning a stable key/marker. .mcp.json keeps
its mcpServers.cce entry (other servers preserved); CLAUDE.md keeps a single
<!-- BEGIN CCE MCP --> … <!-- END CCE MCP --> block whose region is replaced in
place. Re-running produces byte-identical files. Workspace detection is
.cce/workspace.yml exists → the server args become ["mcp", "--workspace"].
Sync is a soft dependency, gated on config. On startup cce mcp warms the index
via sync pull --latest only when a remote is configured and sync.auto_pull is
on, with force = false (never clobber a WIP local cache) and every error swallowed
— offline, no-remote, cache-miss, and sha-mismatch all fall back silently to the
local index. index_status freshness (source/sha/behind-remote) is a new
sync::commands::freshness that touches the network only when a remote is configured.
MCP works fully with no Sync present.
The sync artifact format version is decoupled from the app version. The artifact
format did not change in v2.4 (CCE MCP is purely additive), so its compatibility
version must not move with the release. The old cce_version_minor() derived it from
the crate version, which would have made every release invalidate everyone's cache and
diverge from Ruby. It is replaced by a dedicated constant SYNC_FORMAT_VERSION = "2.3"
that names the artifact format, used everywhere the sync layer stamps the version
(the content address hash/2.3/… and the manifest cce_version field). It moves only
when the artifact bytes actually change shape — then both engines bump it in lockstep.
The shared golden checksum on test/fixture/samples therefore stays
581cbd0ff682a38d7d1250f3eec44f4ce456bdd660d4cb29aaaadd9e95072f48, equal to Ruby's.
The app/crate version is 2.4.1 (Cargo.toml, CITATION.cff) — the v2.4.1 dashboard
refresh + docs sweep is additive and does not touch SYNC_FORMAT_VERSION, so the
golden checksum above is unchanged; conformance.json is independent of both and stays
byte-identical.
The metrics schema grows only by adding fields. The reader already tolerated
unknown/absent fields, so v2.4.1 extends it in place rather than versioning it:
search events gain source, index events gain sha/source/sensitive_skipped.
A pre-v2.4.1 log still parses — a search with no source normalises to "cli", an
index event to "local". No cce.metrics/v2; the schema tag stays cce.metrics/v1.
Agent-vs-human bucketing is a single crisp rule. Only the exact value "mcp"
counts as an agent search; every other source ("cli", empty, unknown) is a human
search. This keeps the CLI path (cce search → "cli") and the MCP path
(context_search → "mcp") as the two buckets, and makes an ambiguous/old event fall
into the human bucket deterministically — the same rule in both engines. The top-level
key is by_source (the reconciled cross-engine name).
index_freshness is PURELY log-derived — the dashboard makes zero network calls.
Its shape is exactly {indexes, source, sha, indexed_ts}, computed from the latest index
event, so both engines reproduce it identically and cce dashboard stays self-contained
and fully offline. It deliberately carries no remote_latest/behind_remote: a
live remote comparison would mean a git fetch on the request path, which breaks the
offline/self-contained posture. That comparison lives only in cce sync status and MCP
index_status, which are expected to consult the remote. To make the pulled state
observable without a live lookup, cce sync pull records a source: "sync-pull" index
event in the log (with the pulled sha) — so index_freshness.source reads "local"
(built by cce index) or "sync-pull" (installed by cce sync pull) straight from the
log.
The dashboard stays self-contained and offline. The four new panels
(agent-vs-human, per-package, index-freshness, secret-safety) render from the same
/api/metrics body with inline JS/SVG — no new endpoint, no external asset, no network
call, still loopback-only and read-only. secret_safety.sensitive_skipped sums the
index events' skip counts (the only source of that datum), so it needs the additive
index.sensitive_skipped field. by_package (workspace only) is an array of objects,
each with a package field, sorted by package for deterministic cross-engine order,
and gains mean_top_score so the per-member panel shows quality, not just savings.
Compact is the default, and expand-on-demand is what makes that safe. Retrieval
alone does not reliably beat a modern agent's grep-and-read, so context_search
serves compact chunks by default (retrieval.detail: compact) — a signature +
doc + first body line + an elision marker — and every result carries a chunk_id.
The agent reads a full body only by calling expand_chunk(chunk_id, scope=body),
which round-trips the exact detail:full bytes. The store always keeps the full
body; compaction is a serialization-time transform only, so nothing is lost.
This is the guiding thesis (all necessary context, no unnecessary context) made
concrete: default to less, expand on demand.
The structural-compact fix (a real regression, then its cure). The first
compact format truncated a chunk to its first N physical lines. In practice that
often cut a signature mid-declaration or dropped the member an agent was looking
for, and the agent's rational response was to re-issue context_search for the
same target — which cost more tokens than serving the body would have. Compression
that triggers a re-search is a net loss. The cure was to make compact structural,
not line-count-based: driven by the language pack's AST rules — a container renders
its header + doc + member signatures (for Ruby including the has_many /
belongs_to / validates DSL declarations), a leaf renders signature + doc + first
non-trivial line. And the tool descriptions were pinned with an expand-first rule
(“do NOT re-issue context_search for a target you already found; call
expand_chunk”). Compact must carry enough structure that the agent expands rather
than re-searches — that is the property being defended.
Memory is validated-only and precision-recalled — anti-pollution over volume. A
naive "save every answer and replay it" memory lowers quality: a wrong answer
re-injected makes the next answer worse, not cheaper. So record_decision is an
explicit call for a validated decision only — never an auto-capture of raw
model output — and session_recall is precision-filtered (score ≥ 0.30 and a
shared query token, small top_k) and returns entries the agent chooses to use,
never an auto-injected blob. Returning nothing on a weak match is the correct
behaviour. Memory that lowers answer quality is a bug even if it lowers tokens. The
store is local-only .cce/memory.jsonl, secret-redacted before write, and never
pushed by Sync (it is conversational, not reproducible).
Session digests are deterministic and structured — not an LLM summary.
summarize_context renders the server's per-session ledger (files/chunks touched,
queries issued, decisions recorded — deduped, sorted, bounded with … (+N more))
into a byte-deterministic digest. It is explicitly not an LLM-written summary:
the same sequence of tool calls always yields the same bytes, so it needs no model
and no network, stays testable with golden bytes, and cannot itself hallucinate. The
backing ledger is in-memory and wall-clock-free (order-preserving, no
timestamps), so determinism holds regardless of timing.
Grammar savings are self-measured against a pinned verbose baseline. Unlike the
output layer (measured against an off control in the eval harness), the grammar
bucket is computed inside the engine: the byte-pinned compact result grammar vs a
pinned verbose v2.4-style format, both counted with cce.tokens/v1. Because the
compact grammar is the single source of the bytes actually served (the result header
is rendered from grammar::compact_line), the measured saving is exactly the saving
delivered — no drift between what is reported and what is served.
The token counter is a deterministic estimator, labelled as such. All savings
use one cross-language counter, cce.tokens/v1 = max(1, floor(bytes / 4)), pinned
byte-exactly so Ruby and Rust agree on every count. It is an estimator, not a
model tokenizer, and every surface that reports savings says so, alongside the
"vs full-file baseline — not your real end-to-end agent cost" note. The real
end-to-end number is a separate, deliberately different measurement: the cce eval
A/B harness (correctness-gated so cheap non-answers never count; cost-primary,
including sub-agents).
SYNC_FORMAT_VERSION stays 2.3, decoupled from the app version. v2.5 is
additive — it changes neither the chunk conformance.json nor the CCE Sync artifact
format — so the artifact-format version stays 2.3 and the content address stays
hash/2.3/…. Bumping the crate to 2.5.x does not invalidate existing caches or
diverge from Ruby; the app version and the interchange-format version are two
separate numbers on purpose (the same discipline as the v2.4.1 consolidation).
Rust-first; cce-ruby reconciles later. This track ships on cce-rust as the reference implementation. Every transform is deterministic and byte-pinned, so the golden bytes authored here become the target for cce-ruby's later catch-up — the cross-language byte-identity is deferred, not abandoned. cce-ruby stays at 2.4.1 with the spec filed as backlog.
A docs-only consolidation, verified cold. Like the v2.4.1 sweep, v2.5.5 changes
no engine behaviour: it brings every doc current to the complete v2.5 track (the
seven layers, the nine MCP tools, cce savings, the honest framing) and re-verifies
the documented commands from a cold start (recorded in VERIFIED.md).
conformance.json and the Sync artifact are byte-identical; only the app version
moves, to 2.5.5.
Fully additive; the code index is byte-frozen. The markdown-heading chunker is
used ONLY by the new knowledge ingest and is deliberately not registered as a
LanguagePack, so Chunker::chunk_file still treats a code-repo .md as one
whole-file module chunk. conformance.json and the Sync code artifact are proven
byte-identical (regenerated + diffed in tests/knowledge_ingest.rs). Heading-chunking
code-repo .md too is a deferred future opt-in that would carry its own conformance
bump.
tree-sitter-markdown over a line-based splitter. The block grammar is consistent
with the code packs, shares the same tree-sitter ABI (crate tree-sitter-md =0.5.3,
one added dependency), and is robust to the cases a regex trips on — a # inside a
fenced code block is a code token, not a heading. The chunker collects heading nodes
(atx_heading / setext_heading) and applies its own same-or-higher boundary and
budget-split logic rather than trusting the grammar's section nesting, because the
grammar flattens setext headings; this keeps the rule uniform and byte-pinned.
Boundary/line rules (byte-pinned). A chunk's stored bytes are the section's bytes
with trailing whitespace trimmed (leading preserved for line accuracy); end_line = start_line + newlines(trimmed content), so inter-section blank lines never inflate
a span. The breadcrumb name reconstructs the markers from the heading level
(# Title › ## Section, joined by U+203A with spaces); the preamble chunk uses the
sentinel (preamble) for both kind and name.
A separate, snapshot-keyed knowledge store. Knowledge is mutable, so it can never
enter the repo@sha code cache. The store lives at .cce/knowledge/<snapshot>.json
with a current pointer; the snapshot id is the first 16 hex of SHA-256 over the
input feed bytes — location-independent, so the persisted store is byte-identical
regardless of where the feed lives. A newer ingest supersedes the old via the pointer.
Redact before chunk. The v2.1 redactor runs on the rendered # <title>\n\n<body>
document before M1 chunking, mirroring the code index's Layer 2: the store never
sees a secret and chunk ids/token counts derive from redacted text.
Chunk identity reuses the existing scheme. Knowledge chunk ids use the same
SHA-256(path:start:end:prefix) function, with the record id as the synthetic
document path — so ids are stable across snapshots as long as a record's rendered,
redacted content is unchanged.
Consumer mode is a smarter client, not a context server. The obvious way to
serve indexed context to machines that hold no source is a server: a daemon that
owns the indexes and answers queries over the network. That was rejected. The
determinism + content-addressing decisions already made the cache itself the
service: an artifact for repo@sha is byte-identical whoever built it, addressed
by a pure function of its identity, and complete (chunks, embeddings, whole-file
reconstruction, import graph — everything search needs). So a consumer needs no
protocol beyond get from a git remote — cce sync list enumerates the cache,
pull --all materializes it into a synthesized workspace, and search/MCP run
fully offline on the result. A server would have inverted every posture the
project holds: online-at-query-time instead of offline-first, a new auth/RBAC
surface instead of delegating permissions to git, an availability dependency
instead of a file cache, and a second query path to keep conformant across two
engines. Consumer mode adds zero infrastructure: the same cache, read by a
smarter client.
Integrity without source is a recorded-at-install checksum, deliberately
version-independent. Full verify (rebuild-and-compare) needs the source, so
repo-less consumers get verify --checksum-only: pull records the SHA-256 of
the exact bytes it installs, and verify re-hashes the on-disk file against that
record. The baseline is the installed file, never a re-export through the current
code — so artifacts pushed by any older cce verify identically ("has this file
changed since pull"). The caveat is stated wherever the flag is documented:
this detects corruption, not a malicious build; artifact == build(sha) stays
with source-holders/CI. The honest trust posture for consumers is CI as the
canonical pusher plus the git host's access control.
The cache self-describes through additive well-known keys. Publishing the
workspace manifest and cross-member graph under the base repo_id (SPEC-SYNC §3)
was chosen over embedding workspace metadata in the artifact (a format change —
SYNC_FORMAT_VERSION would move and every existing cache would miss) and over
requiring consumers to hold a manifest (defeats repo-less). Keys that are neither
artifacts nor ref pointers are normatively additive: old clients ignore them,
old caches simply lack them, and their absence is never an error.