You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit d3b8d25
Browse filesBrowse the repository at this point in the historyBrowse files
feat(index): doc_candidates setting and index_repository parameter
The doc<->code candidate scoring (DeusData#2574, doc_section_worker in
semantic_edges) scores every documentation section against the functions
that share a key term with it, key terms being those held by at most
funcs/20 functions. Its cost therefore grows with sections x functions.
Measured on 11 corpora (10-09 main vs 4d9592b, semantic_edges): under a
second up to ~100k functions, +12.6 s on java (325k), +14.6 s on csharp
(451k), +174 s on the Linux kernel (784k functions, 28k sections), almost
all of the kernel's regression against v0.11.0.
A new boolean setting, doc_candidates (default true), and an optional
index_repository argument of the same name switch the scoring off: no
candidate rows are stored and nodes and edges are unchanged. The argument
wins over the setting. The index handler resolves the value once and
passes it to a daemon job or supervised worker as an argument, the same way
as the resource policy, so every route sees one decision; the session
auto-index reads the setting. With the switch off, a delta-route reindex
replaces the candidate rows it would otherwise keep, so turning it off
leaves no stale candidates. The default output is unchanged.
Tests (each fails on its assertion when the switch has no effect):
- pipeline_doc_candidates_off_stores_none_and_keeps_graph: no rows, same
node and edge counts as the default run, which has rows;
- pipeline_doc_candidates_off_clears_rows_on_delta: a closure-repair
reindex with the switch off removes the earlier rows;
- tool_index_repository_doc_candidates_switch: argument false -> none;
setting default -> rows; setting false -> none; argument true over
setting false -> rows.
Proof, release-built binaries on macOS arm64 (M5 Pro 18c/48 GB):
- django, main 4d9592b vs this change with defaults: nodes (63,441),
edges (307,584) and doc_link_candidates rows (11,304) hash-identical;
- django with doc_candidates false: nodes and edges hash-identical, no
candidate rows;
- Linux kernel with doc_candidates false: semantic_edges 17.9 s (191.9 s on
main, 17.6 s before DeusData#2574), whole index 210.9 s (v0.11.0: ~234 s), no
candidate rows.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Copy file name to clipboardExpand all lines: docs/CONFIGURATION.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -88,6 +88,7 @@ Current keys:
88
88
|`auto_index`|`false`| Automatically index new projects when an MCP session starts. |
89
89
|`auto_index_limit`|`50000`| Maximum file count allowed for automatic indexing of a new project. |
90
90
|`auto_watch`|`true`| Register the session's project with the background git watcher on connect. Set `false` to keep a session from registering its project (the watcher still runs for other projects). |
91
+
|`doc_candidates`|`true`| Score documentation sections against code to store doc↔code candidates (the possible targets of a section, and the sections that may describe a function or file, each with a judged probability). The scoring grows with sections × functions: under a second up to ~100k functions, ~15 s at ~400k, ~3 minutes on the Linux kernel. Set `false` to skip it; no candidates are stored and the rest of the graph is unchanged. `index_repository` also takes `doc_candidates` for a single index, which wins over this setting. Applies from the next index. |
91
92
|`watcher_enabled`|`true`| Master switch for the background watcher subsystem. Set `false` to stop the watcher from starting at all — no poll thread and no project registration. Reindex manually with `index_repository` when disabled. Only an open MCP session's own project is ever watched; `index_status` reports the current state in its `watch` object. |
92
93
|`watch_non_git`|`false`| Also poll project roots that are **not git repositories**. By default the watcher only follows git projects, so a project indexed from a plain directory is never refreshed after its first index — reindex it manually with `index_repository`. Set `true` to poll such roots on the same adaptive cadence with a file-tree scan: the indexer's own discovery walk (same skip lists, `.gitignore` and `.cbmignore` rules) hashed over each file's path, size and mtime. Any change reindexes once; paths the indexer skips (including cbm's own `.codebase-memory/` output) never trigger. The first poll after the daemon starts reindexes each such project once, since nothing records which tree state the index holds. The scan walks the whole tree every poll, so it costs more than git polling on very large trees. Read once when the daemon starts, like `watcher_enabled`. |
93
94
|`index_max_files`|`off`| Optional maximum number of accepted source files in one discovery run. |
0 commit comments