Skip to content

Commit e5f3aab

Browse files
feat(pipeline): index-time importance scoring (weighted degree)
Score every Function/Method/Class node during indexing and persist the result as a numeric "importance" key inside the node's existing properties_json: importance = sqrt(num_refs) * priv * generic * distinct * test_penalty num_refs is the incoming CALLS + USAGE degree. The multipliers demote private (leading-underscore) and generic names (defined in >= 5 distinct files), promote distinctive identifiers (snake_case or camelCase, len >= 8), and demote test scaffolding -- symbols in a test file per the graph's own cbm_is_test_path() classifier, or targets of an incoming TESTS edge. Weighted degree only; a transitive (PageRank) refinement is deliberately not built here until it can be shown to beat this on a fixed judgment set. No schema change and no CBM_INDEX_FORMAT_VERSION bump: the score lives in the existing properties_json TEXT column, so no index is forced to rebuild and indexes written by older builds simply lack the key. Three failure modes this shape invites, all of them silent, are addressed directly: * Cost. The generic-name multiplier needs |{files a name is defined in}|. Computing that per NODE with a pairwise file-path scan is O(k^3) in a same-name group of size k -- ruinous on real corpora, where a single Java name group reaches k > 4000. Here the count is computed once per distinct NAME and memoized, and file paths are deduplicated through a hash set, so total distinct-file work is linear in the graph. Measured cold full index of a 666 MB Java corpus, same -O2 binary: 85 s with the pass disabled, 80 s with this implementation, 481 s with the per-node shape. * Registration. PREDUMP_PASS_COUNT is now derived from the pass table with sizeof instead of being hand-written. A hand-written count that lags an appended entry silently skips the last-registered pass with every test still green. * Incremental. The incremental post-pass sequence runs the pass too, over a graph rehydrated from the store whose nodes already carry the key. The write-back overwrites an existing key in place instead of appending a second one, which would otherwise corrupt properties_json into {"importance":1.0,...,"importance":2.0}. tests/test_importance.c covers the formula and binds all three failure modes; the complexity suite gains a GATED counter asserting the pass's same-name-group work stays linear (~2x, not ~4x) under a corpus doubling. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> Co-authored-by: petercoxphoto <20230727+petercoxphoto@users.noreply.github.com>
1 parent 31b611a commit e5f3aab

8 files changed

Lines changed: 1142 additions & 5 deletions

File tree

‎Makefile.cbm‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -354,6 +354,7 @@ PIPELINE_SRCS = \
354354
src/pipeline/pass_similarity.c \
355355
src/pipeline/pass_semantic_edges.c \
356356
src/pipeline/pass_complexity.c \
357+
src/pipeline/pass_importance.c \
357358
src/pipeline/pass_cross_repo.c \
358359
src/pipeline/artifact.c \
359360
src/pipeline/pass_ensemble_routing.c \
@@ -560,7 +561,7 @@ TEST_DISCOVER_SRCS = \
560561

561562
TEST_GRAPH_BUFFER_SRCS = tests/test_graph_buffer.c
562563

563-
TEST_PIPELINE_SRCS = tests/test_registry.c tests/test_pipeline.c tests/test_cross_repo.c tests/test_fqn.c tests/test_route_canon.c tests/test_path_alias.c tests/test_configlink.c tests/test_infrascan.c tests/test_worker_pool.c tests/test_parallel.c tests/test_index_resilience.c tests/test_index_format.c tests/test_call_reference_contract.c tests/repro/repro_call_scope_usages.c tests/repro/repro_call_argument_usages.c tests/repro/repro_reference_precision.c tests/repro/repro_lexical_binding_precision.c tests/repro/repro_call_argument_matrix_a.c tests/repro/repro_call_argument_matrix_b.c tests/repro/repro_call_node_behaviors.c tests/repro/repro_language_registry.c tests/repro/repro_call_node_manifest.c tests/repro/repro_lsp_ordered_signatures.c tests/repro/repro_lsp_ordered_local.c tests/repro/repro_ts_overload_return_chains.c tests/repro/repro_harness_cleanup.c tests/repro/repro_runner_filter.c
564+
TEST_PIPELINE_SRCS = tests/test_registry.c tests/test_pipeline.c tests/test_importance.c tests/test_cross_repo.c tests/test_fqn.c tests/test_route_canon.c tests/test_path_alias.c tests/test_configlink.c tests/test_infrascan.c tests/test_worker_pool.c tests/test_parallel.c tests/test_index_resilience.c tests/test_index_format.c tests/test_call_reference_contract.c tests/repro/repro_call_scope_usages.c tests/repro/repro_call_argument_usages.c tests/repro/repro_reference_precision.c tests/repro/repro_lexical_binding_precision.c tests/repro/repro_call_argument_matrix_a.c tests/repro/repro_call_argument_matrix_b.c tests/repro/repro_call_node_behaviors.c tests/repro/repro_language_registry.c tests/repro/repro_call_node_manifest.c tests/repro/repro_lsp_ordered_signatures.c tests/repro/repro_lsp_ordered_local.c tests/repro/repro_ts_overload_return_chains.c tests/repro/repro_harness_cleanup.c tests/repro/repro_runner_filter.c
564565

565566
TEST_WATCHER_SRCS = tests/test_watcher.c
566567

0 commit comments

Comments
 (0)