Repository navigation
fix(pipeline): keep both nodes when a file and a directory share a stem - #2624
Merged
Merged
Conversation
A file and the directory beside it with the same stem (genproto.sh next to genproto/) got one qualified name: the file's Module QN equals the directory's Folder QN, and the graph buffer keeps the structural node, so the file's Module was dropped, and with it the edges that pointed at it. On 17 infrastructure repositories this lost 27 Modules (shell scripts, YAML and JSON manifests, a Makefile, Rust, Elixir, Python, Terraform). One side now yields, decided once per run from the discovered files: - the Folder is spelled proj.pkg.foo/ (no module QN ends in a slash); - where the directory is a Java/Go package it IS that package's module and Go imports resolve to it, so the Folder keeps its QN and the file's Module is spelled proj.pkg.foo#sh instead. Symbol QNs do not change. The closure-repair planner declines when a deletion touches a collision on either side, since the spelling depends on the file set. The kustomize resolver binds a directory entry to its Folder even when a same-stem manifest sits beside it. CBM_SEMANTIC_INDEX_VERSION 7 -> 8, so existing indexes rebuild once. Measured on main 56700bd over the 17 corpora: colliding files without a Module 27 -> 0 (19 marked Folders, 8 #ext Modules). k8s.io resolves 3 more kustomize entries, and deckhouse's `source shell_lib.sh` imports land on the script instead of the directory. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
DeusData
force-pushed
the
fix/stem-folder-collision
branch
from
October 13, 2026 17:05
d39ff62 to
f96001f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A file and the directory beside it with the same stem got one qualified name:
src/frontend/genproto.shproj.src.frontend.genprotosrc/frontend/genproto/proj.src.frontend.genprotoThe graph buffer keeps the structural node of a QN on purpose (Java/Go modules ARE directories), so the file's Module was dropped, together with every edge that should have pointed at it. On 17 infrastructure repositories that lost 27 Modules: shell scripts, YAML and JSON manifests, a Makefile, Rust, Elixir, Python, Terraform.
Fix
One side yields. The choice is made once per run from the discovered files (
cbm_stem_marks_t,src/pipeline/pipeline.c):proj.pkg.foo/. No module QN ends in a slash.proj.pkg.foo#sh(the file's extension).foo.rsbesidefoo/keepsproj.foo, and the Cfoo.h/foo.cstem pair is untouched. A dotted stem (foo.test.tsbesidefoo/test/) collides by QN and is handled the same way.Around it:
stem_mark_changed, full rebuild) when a deletion touches a collision on either side. Adding a file already declines. The repair route builds the same set, so a re-extracted file keeps its spelling.dashboards) beside a same-stem manifest (dashboards.yaml) must not bind to the manifest's now-visible Module. The resolver accepts a Module only for the entry's own file, and otherwise looks up the Folder in either spelling.CBM_SEMANTIC_INDEX_VERSION7 → 8 (history entry), so existing indexes rebuild once.Evidence (17 infra corpora, main 56700bd vs this branch, fresh index roots)
…/), each with its Module#extModules beside a Go package#sh×5,#mk,#json×2)Graph changes, identity diff:
source shell_lib.shimports now land on the script, not on theshell_lib/directory.dashboards.yaml, unresolved on main).genprotopackages are unchanged.Tests (each RED without its part of the change)
pipeline:pipeline_stem_folder_collision_keeps_both_nodescovers every collision shape: Folder side, Go-package side, Rust, the C stem pair, the dotted stem, and a Go import onto the package Folder. It runs once small (sequential extraction) and once pastMIN_FILES_FOR_PARALLEL(parallel workers), since the two insert Modules on separate paths.pipeline:pipeline_kustomize_entry_beside_same_stem_file: a directory entry lands on the marked Folder, a file entry on the file's Module.pipeline_semantic_manifest_repro:incremental_stem_mark_deletion_rebuilds:Revert checks, each part removed on its own (macOS, ASan test runner):
#extinprocess_def(sequential)#extininsert_def_into_gbuf(parallel)#extlookup inregister_and_link_defgenproto.shdeletion takes the closure repairKnown limits
#extModule by its computed QN still lands on the Go Folder, exactly as on main.foo.pybeside afoo/package: the file wins forimport pkg.foo, as JS/TS resolution does. Python itself prefers the package; the layout is rare and is itself a bug in the user's code.Local legs
Run locally:
incremental_deleted_spawner_drops_its_process(closure declinedpython_namespace_changed), is pre-existing on main 56700bd and fixed by fix: never qsort an empty k8s vector; split the spawner deletion test #2617;pipeline.cbaseline lowered 134 → 132), the NOLINT check and the security audit.Skipped, with PR CI as the gate: the Linux and Windows legs. The change is platform-neutral C with no new I/O, process or memory-lifetime risk beyond the per-run arena owned by
run_extraction_phase/run_closure_delta.