Skip to content

Commit 6ec8d58

Browse files
committed
Merge branch 'sync/upstream-13m1-20261108' into sync/upstream-13m2-20261108
2 parents bff6508 + 3cbefe2 commit 6ec8d58

12 files changed

Lines changed: 702 additions & 57 deletions

‎docs/CONFIGURATION.md‎

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -78,13 +78,15 @@ codebase-memory-cli config get auto_index
7878
codebase-memory-cli config set auto_index true
7979
codebase-memory-cli config set auto_index_limit 50000
8080
codebase-memory-cli config set watcher_enabled false
81+
codebase-memory-cli config set ignore_worktrees true
8182
codebase-memory-cli config set doc_candidates false
8283
codebase-memory-cli config reset doc_candidates
8384
```
8485

8586
`doc_candidates` is `auto` by default: unset, the index mode decides. A single
8687
`codebase-memory-cli index --doc-candidates true|false` also overrides it for
87-
that index only.
88+
that index only. With `ignore_worktrees` on, `codebase-memory-cli index
89+
--index-worktree` indexes a linked git worktree for that one call.
8890

8991
Current keys:
9092

@@ -94,6 +96,7 @@ Current keys:
9496
| `auto_index_limit` | `50000` | Maximum file count allowed for automatic indexing of a new project. |
9597
| `auto_watch` | `true` | Register an active project with the retained background watcher when a warm runtime/hook integration supplies project context. A daemon-started watcher can retain registered projects after the initiating session ends until daemon shutdown, project deletion/root disappearance, or this setting is disabled. The ordinary one-shot CLI does not require this. |
9698
| `doc_candidates` | `auto` | Score documentation sections against code to store doc<->code candidates (the possible targets of a section, and the sections that may describe a function or file, each with a judged probability). Unset (`auto`), the index mode decides: on for `full` indexes (the default mode), off for `moderate` and `fast`. Set `true` or `false` to choose for every mode; `config reset doc_candidates` returns to `auto`. The scoring grows with sections x functions: under a second up to ~100k functions, ~15 s at ~400k, ~3 minutes on the Linux kernel. Off stores no candidates and leaves the rest of the graph unchanged; on in `fast` mode stores candidates without adding `fast`'s skipped `SIMILAR_TO` / `SEMANTICALLY_RELATED` edges. `codebase-memory-cli index --doc-candidates true|false` sets it for a single index and wins over this setting. Applies from the next index. |
99+
| `ignore_worktrees` | `false` | Skip linked git worktrees when indexing automatically, and refuse an explicit `codebase-memory-cli index` of one unless `--index-worktree` is passed. See [`ignore_worktrees`](#ignore_worktrees). |
97100
| `watch_non_git` | `false` | Poll non-Git roots with the indexer discovery rules and an adaptive file-tree scan. Skipped paths, including `.codebase-memory/`, do not trigger indexing. The first poll after daemon startup reindexes once; later changes trigger one reindex. This walks the whole tree each poll and is read when the daemon starts. |
98101
| `watcher_enabled` | `true` | Master switch for the retained background watcher subsystem. Set `false` to stop its poll thread/project registration. Reindex manually with `codebase-memory-cli index` when disabled. |
99102
| `index_max_files` | `off` | Optional maximum number of accepted source files in one discovery run. |
@@ -132,24 +135,24 @@ stored index of what is largely the same repository.
132135
Enable the key to keep those checkouts out of the index:
133136

134137
```bash
135-
codebase-memory-mcp config set ignore_worktrees true
138+
codebase-memory-cli config set ignore_worktrees true
136139
```
137140

138141
With it enabled:
139142

140143
- automatic indexing skips a session whose root is a linked worktree;
141144
- the `hook-augment` context says the worktree is unindexed on purpose instead
142-
of telling the agent to run `index_repository`;
143-
- an explicit `index_repository` call on a linked worktree is refused, and names
144-
both ways forward — pass `index_worktree=true` for that one call, or turn the
145-
key back off.
145+
of telling the agent to run `codebase-memory-cli index`;
146+
- an explicit `codebase-memory-cli index` of a linked worktree is refused with an
147+
error that names both ways forward: pass `--index-worktree` for that one call,
148+
or run `codebase-memory-cli config set ignore_worktrees false`.
146149

147150
The main checkout of the same repository is unaffected, as are ordinary clones
148151
and submodules. Detection is git plumbing only: a linked worktree's `.git` is a
149152
file pointing at a gitdir that contains a `commondir` entry.
150153

151154
A subdirectory counts as the worktree it sits in. Detection walks up from the
152-
given path to the nearest `.git` entry, so `index_repository` on any
155+
given path to the nearest `.git` entry, so `codebase-memory-cli index` on any
153156
subdirectory of a linked worktree is refused the same way, and a session root or
154157
hook working directory inside one is treated as that worktree. The walk stops at
155158
the first `.git` it finds; a main checkout or a submodule ends it as "not a

‎docs/EVALUATION_PLAN.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -283,7 +283,7 @@ for lang in $ALL_LANGS; do # ALL_LANGS = full 159-name list
283283
# --- step 2: cold index in the main channel, TIMED (key metric) ---
284284
t0=$(now_ms)
285285
CBM_BENCH_KEEP_RUNTIME=1 \
286-
scripts/benchmark-index.sh ~/.local/bin/codebase-memory-mcp "$lang" /tmp/bench/"$lang" /tmp/eval-results
286+
scripts/benchmark-index.sh ~/.local/bin/codebase-memory-cli "$lang" /tmp/bench/"$lang" /tmp/eval-results
287287
index_ms=$(( $(now_ms) - t0 )) # clone+index wall-clock → manifest + report (§5)
288288
# The harness indexes into a run-private root (#1696). CBM_BENCH_KEEP_RUNTIME leaves that
289289
# root behind on success and records its paths; the graph session in steps 4-7 must be
@@ -878,7 +878,7 @@ scripts/clone-bench-repos.sh /tmp/bench
878878
mkdir -p /tmp/eval-results
879879
for lang in $ALL_LANGS; do
880880
CBM_BENCH_KEEP_RUNTIME=1 \
881-
scripts/benchmark-index.sh ~/.local/bin/codebase-memory-mcp "$lang" /tmp/bench/"$lang" /tmp/eval-results
881+
scripts/benchmark-index.sh ~/.local/bin/codebase-memory-cli "$lang" /tmp/bench/"$lang" /tmp/eval-results
882882
done
883883

884884
# 3. Cross-repo pass for the 9 LSP pairs (index each service dir, then cross-repo-intelligence)

‎docs/MEASURING_SAVINGS.md‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -104,10 +104,12 @@ codebase-memory-cli status --project PROJECT_NAME --verbose
104104
```
105105

106106
Confirm that `root_path` is `GRAPH_REPO`, `git.head_sha` is `SHA`, and the Git
107-
context is detached. `index_status` describes the project root and its current
108-
filesystem Git context; it does **not** by itself prove that indexed records
109-
were produced from that revision. The immediately preceding successful fresh
110-
`index_repository` call is the freshness requirement.
107+
context is detached. The verbose `freshness` object compares the checkout SHA
108+
recorded when the index was written with the live HEAD and a bounded worktree
109+
status; `verdict` is `current`, `stale`, or `unknown` (report-only, it never
110+
indexes). Treat anything other than `current` as not proven. The immediately
111+
preceding successful fresh `index_repository` call remains the freshness
112+
requirement.
111113

112114
Finally, run one or more representative queries whose expected symbols you
113115
have verified directly at the recorded SHA:

‎scripts/benchmark-index.sh‎

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
set -euo pipefail
33

44
# Index a single benchmark repository and capture metrics.
5-
# Usage: benchmark-index.sh <binary> <lang> <repo_path> <results_dir>
5+
# Usage: benchmark-index.sh <codebase-memory-cli binary> <lang> <repo_path> <results_dir>
66

77
BINARY="${1:?Usage: benchmark-index.sh <binary> <lang> <repo_path> <results_dir>}"
88
LANG="${2:?}"
@@ -66,10 +66,6 @@ DAEMON_START_LOG="$BENCH_TMP/daemon-start.log"
6666

6767
# Resolve symlinks
6868
REPO=$(cd "$REPO" && pwd -P)
69-
# One pre-escaped spelling of the path for every request below, the way the
70-
# soak harness builds its own: a repository path may legitimately contain a
71-
# quote or a backslash, and hand-built JSON turns that into a parse error.
72-
REPO_JSON=$(python3 -c 'import json,sys; print(json.dumps(sys.argv[1]))' "$REPO")
7369

7470
# Elapsed time is read from a monotonic clock, never the wall clock: an NTP
7571
# step mid-run would otherwise skew — or negate — a figure whose whole purpose
@@ -115,7 +111,12 @@ fi
115111
# Index via CLI and capture timing
116112
START_MS=$(bench_now_ms)
117113

118-
INDEX_JSON=$("$BINARY" index "$REPO" --mode full --json 2>/dev/null || echo '{"error":"index failed"}')
114+
# The path is one argv element, so a quote or backslash in it needs no escaping.
115+
# The exit status is kept (a bare `|| echo` would hide it) and stderr goes to
116+
# INDEX_ERR for the failure path below.
117+
INDEX_RC=0
118+
INDEX_JSON=$("$BINARY" index "$REPO" --mode full --json 2>"$INDEX_ERR") || INDEX_RC=$?
119+
[ -n "$INDEX_JSON" ] || INDEX_JSON='{"error":"index failed"}'
119120

120121
END_MS=$(bench_now_ms)
121122
ELAPSED=$((END_MS - START_MS))

‎scripts/benchmark-search-graph.sh‎

Lines changed: 4 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
# codebase-memory-cli binary to measure regex / BM25 search performance.
44
#
55
# Usage:
6-
# scripts/benchmark-search-graph.sh <binary-path> <project-name>
6+
# scripts/benchmark-search-graph.sh <binary-path> <repo-path>
77

88
set -euo pipefail
99

@@ -30,20 +30,14 @@ if ! "$BINARY" daemon start >"$DAEMON_START_LOG" 2>&1; then
3030
fi
3131
exit 1
3232
fi
33-
# One pre-escaped spelling of the path, the way the soak harness builds its
34-
# own: a repository path may legitimately contain a quote or a backslash, and
35-
# hand-built JSON turns that into a parse error.
36-
REPO_JSON=$(python3 -c 'import json,sys; print(json.dumps(sys.argv[1]))' "$REPO")
3733
# Index and parse keep their stderr instead of discarding it: without it the
3834
# failure below names only its symptom, and the cause — an unreadable
39-
# repository, a refused daemon, a malformed envelope — is unrecoverable.
40-
INDEX_JSON=$("$BINARY" cli index_repository "{\"repo_path\":$REPO_JSON,\"mode\":\"full\"}" \
41-
2>"$INDEX_ERR" || echo '{}')
35+
# repository, a refused daemon, a malformed envelope — is unrecoverable. The
36+
# repository path is one argv element, so a quote or backslash needs no escaping.
37+
INDEX_JSON=$("$BINARY" index "$REPO" --mode full --json 2>"$INDEX_ERR" || echo '{}')
4238
PROJECT=$(echo "$INDEX_JSON" | python3 -c "
4339
import json, sys
4440
d = json.load(sys.stdin)
45-
if 'content' in d:
46-
d = json.loads(d['content'][0]['text'])
4741
print(d.get('project', ''))
4842
" 2>>"$INDEX_ERR" || echo "")
4943
if [ -z "$PROJECT" ]; then

‎src/cli/hook_augment.c‎

Lines changed: 55 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,8 @@
2323
#include "foundation/constants.h"
2424
#include "foundation/mem.h"
2525
#include "foundation/platform.h"
26+
#include "foundation/str_util.h"
27+
#include "git/git_context.h"
2628
#include "pipeline/pipeline.h"
2729
#include "yyjson/yyjson.h"
2830

@@ -1715,6 +1717,53 @@ static bool ha_invocation_supported(ha_lifecycle_dialect_t dialect, const char *
17151717
static CBM_TLS int *g_ha_worktree_gate_counter = NULL;
17161718
#endif
17171719

1720+
/* Reads ignore_worktrees from the runtime config store only when the store
1721+
* already exists: the hook must not create _config.db as a side effect, and
1722+
* with the key off (the default) it does nothing else for worktrees. */
1723+
static bool ha_ignore_worktrees_enabled(void) {
1724+
const char *cache_dir = cbm_resolve_cache_dir();
1725+
if (!cache_dir || !cache_dir[0]) {
1726+
return false;
1727+
}
1728+
char db_path[4096];
1729+
int written = snprintf(db_path, sizeof(db_path), "%s/" CBM_CONFIG_DB_FILENAME, cache_dir);
1730+
if (written <= 0 || (size_t)written >= sizeof(db_path) || !cbm_file_exists(db_path)) {
1731+
return false;
1732+
}
1733+
cbm_config_t *cfg = cbm_config_open(cache_dir);
1734+
bool enabled = cbm_config_get_bool(cfg, CBM_CONFIG_IGNORE_WORKTREES, false);
1735+
cbm_config_close(cfg);
1736+
return enabled;
1737+
}
1738+
1739+
/* Project resolution once ignore_worktrees is on; the caller runs this only
1740+
* after reading the key. A symlink alias has no .git of its own, so cwd is
1741+
* canonicalized as the index operation does before the same gate. A linked
1742+
* worktree is refused even inside an indexed checkout or another linked
1743+
* worktree, so a match is kept only when its root belongs to this cwd's own
1744+
* linked worktree, not a parent graph. */
1745+
static char *ha_resolve_project_ignoring_worktrees(const char *cwd, bool *drop_ancestor,
1746+
bool *worktree_ignored) {
1747+
#ifdef CBM_CLI_ENABLE_TEST_API
1748+
if (g_ha_worktree_gate_counter) {
1749+
(*g_ha_worktree_gate_counter)++;
1750+
}
1751+
#endif
1752+
char canonical_cwd[4096];
1753+
if (ha_canonical_path(cwd, canonical_cwd, sizeof(canonical_cwd))) {
1754+
cwd = canonical_cwd;
1755+
}
1756+
char project_root[4096];
1757+
project_root[0] = '\0';
1758+
char *project = ha_resolve_indexed_project_with_root(cwd, project_root, sizeof(project_root));
1759+
bool linked = cbm_git_is_linked_worktree(cwd);
1760+
/* Drop a parent graph without a second free(): this file's raw free count
1761+
* is ratcheted, and the caller's single free(project) still releases it. */
1762+
*drop_ancestor = linked && project && !cbm_git_same_linked_worktree(cwd, project_root);
1763+
*worktree_ignored = linked && (!project || *drop_ancestor);
1764+
return project;
1765+
}
1766+
17181767
static char *ha_lifecycle_json_from_root(const char *session_root, yyjson_val *root,
17191768
const char *forced_event, ha_lifecycle_dialect_t dialect) {
17201769
if (!root || !yyjson_is_obj(root)) {
@@ -1728,10 +1777,14 @@ static char *ha_lifecycle_json_from_root(const char *session_root, yyjson_val *r
17281777
char cwd_buffer[4096];
17291778
const char *cwd =
17301779
ha_normalized_cwd_with_root(root, session_root, cwd_buffer, sizeof(cwd_buffer));
1731-
char *project = cwd ? ha_resolve_indexed_project(cwd) : NULL;
1732-
/* Upstream #1430 worktree gate; set by the ignore_worktrees resolution. */
17331780
bool drop_ancestor = false;
17341781
bool worktree_ignored = false;
1782+
char *project = NULL;
1783+
if (cwd && ha_ignore_worktrees_enabled()) {
1784+
project = ha_resolve_project_ignoring_worktrees(cwd, &drop_ancestor, &worktree_ignored);
1785+
} else if (cwd) {
1786+
project = ha_resolve_indexed_project(cwd);
1787+
}
17351788

17361789
char context[2048];
17371790
const char *scope = "Session";

‎src/operations/index.c‎

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@
1818
#include "foundation/platform.h"
1919
#include "foundation/str_util.h"
2020
#include "foundation/workspace.h"
21+
#include "git/git_context.h"
2122
#include "pipeline/artifact.h"
2223
#include "pipeline/pipeline.h"
2324
#include "store/store.h"
@@ -1319,6 +1320,27 @@ static bool index_call_mode_arg(const char *json, const char *key, bool *out) {
13191320
return valid;
13201321
}
13211322

1323+
/* ignore_worktrees: an explicit index of a linked worktree is refused, not
1324+
* silently skipped; index_worktree overrides it for this call. Fills error and
1325+
* returns true on refusal. Reads no config unless repo_path is a linked
1326+
* worktree. */
1327+
static bool index_worktree_refused(const char *json, const char *repo_path, char *error,
1328+
size_t error_size) {
1329+
if (index_bool_arg(json, "index_worktree") || !cbm_git_is_linked_worktree(repo_path)) {
1330+
return false;
1331+
}
1332+
cbm_config_t *config = cbm_config_open(cbm_resolve_cache_dir());
1333+
bool ignore_worktrees = cbm_config_get_bool(config, CBM_CONFIG_IGNORE_WORKTREES, false);
1334+
cbm_config_close(config);
1335+
if (ignore_worktrees) {
1336+
snprintf(error, error_size,
1337+
"repo_path is a linked git worktree and ignore_worktrees is enabled. Pass "
1338+
"--index-worktree to index it anyway, or run: codebase-memory-cli config set "
1339+
"ignore_worktrees false");
1340+
}
1341+
return ignore_worktrees;
1342+
}
1343+
13221344
cbm_operation_result_t cbm_index_operation_execute(const char *args_json,
13231345
const cbm_operation_runtime_t *runtime) {
13241346
const char *json = args_json ? args_json : "{}";
@@ -1360,7 +1382,8 @@ cbm_operation_result_t cbm_index_operation_execute(const char *args_json,
13601382
: getenv("CBM_ALLOWED_ROOT");
13611383
char boundary[CBM_SZ_1K];
13621384
if (!cbm_workspace_root_allowed(repo_path, cbm_workspace_home_dir(), cbm_workspace_cache_dir(),
1363-
allowed_root, boundary, sizeof(boundary))) {
1385+
allowed_root, boundary, sizeof(boundary)) ||
1386+
(!status_mode && index_worktree_refused(json, repo_path, boundary, sizeof(boundary)))) {
13641387
free(repo_path);
13651388
return index_text_error(boundary);
13661389
}

0 commit comments

Comments
 (0)