feat(code-intel): coverage matches the index; anchors and semantics documented (PLN-2027 PR 4)

[wongk]

Campaign chunk C4 of 7 (PRD-739 → PLN-2027), built one chunk at a time against main (which already includes C1 #15, C2 #17 and C3 #18).

C4 — Coverage matches the index; anchors and semantics documented (FR7, FR8, FR9, FR10)

Lands PLN-2027 PR 4.

• Coverage follows the indexer. code_coverage_check applies the same file rules the index applies — skipped directories, .lemoncrow/.ignore, version-control ignores, unrecognised types, the free-tier cap — and every excluded verdict names the rule that skipped it. The rule ladder lives in a new fork-only module, src/lemoncrow/infra/code_intel/inclusion.py.
• The index applies those rules too. _reindex_files, the incremental path behind every edit, re-indexed whatever paths it was handed with no file selection at all, so gitignored files — a nested worktree under .claude/, for example — entered the index and showed up as extra callers. It now filters through the same ladder. A path coverage calls excluded can no longer have rows in the index.
• Anchors and semantics documented. code_changes states its fork point and that uncommitted changes are included; tool_relations points at code_query select=callers for call-site columns; match_kind is documented as applying to symbol-edge enumerations only.

Acceptance ledger

Criteria	Met	Met differently	Not met
27	19	8	0

Built in two attempts. Attempt 2 added the index-inclusion fix after the PRD-739 GA check found the main checkout's index holding a campaign worktree's files (PLN-2027 v4, PR 4 change 5).

Live re-check on this repository: 2,143 paths, 0 disagreements between coverage and the indexer's file scan, every excluded verdict carrying a rule.

Deviations from the plan

• Plan: Classify each unindexed, on-disk path by five rules, first match wins: should_skip_relative_path, .lemoncrow/.ignore, git-ignored, unrecognised type, free-tier cap.
  Did: iter_source_files membership decides excluded vs missing. The four scan rules only name why a path the scan did not select was skipped, and the cap applies only to selected paths. A sixth remainder rule, source-file-scan (reason not selected by the index's source-file scan), names unselected paths that no narrower rule explains.
  Why: The rules alone disagree with the scan. The language registry lowercases suffixes but the scan's fnmatch patterns are case-sensitive, so src/SHOUT.PY is python yet never selected; untracked files inside submodules are not listed either. Rules-only classification would report those missing and break the Exit criterion.
• Plan: Replace the _EXCLUSION_SOURCE string with the list of rules applied.
  Did: Kept exclusion_source and its value; added exclusion_rules (the rule list) beside it, and a per-path rule on excluded verdicts.
  Why: Campaign decision: the field contract is additive only (PLN-2027 section 0), and PLN-1677 N0 captured the code_coverage_check response shape.
• Plan: Add that interpretation to PRD-739 FR10 (open decision 3).
  Did: Did not edit PRD-739; the exact FR10 wording is in next_action.
  Why: Campaign decision: the orchestrator amends PRD-739 when this chunk's PR opens.
• Plan: Rule order puts should_skip_relative_path (skipped directory: <part>) before git-ignored.
  Did: Moved the existing test_git_ignored_files_are_excluded_not_missing fixture from build/ to generated/, keeping its assertions and adding rule == "git-ignore".
  Why: build is a skipped directory, so under first-match order that fixture now names skipped-directory; the test's subject is the git-ignore rule.
• Plan: test_prompt_txt_reports_unrecognised_type, which matches PLN-1677's N0 probe
  Did: The fixture is a git-tracked prompts/prompt.txt.
  Why: N0's probe path is not recorded in PLN-2027, PRD-739 or the manifest; the fixture reproduces the probed case, a tracked prompt file with no registered language.
• Plan: PRD-739 Edge Cases: Non-code files (prompts, markdown): Coverage reports excluded with the unsupported-type rule.
  Did: Markdown reports indexed, unparsed or missing, never excluded; only unregistered types (.txt, .svg, .gitignore) report unrecognised-file-type.
  Why: markdown is a registered language the indexer's scan selects (language_for_path('README.md') is markdown), and FR8 requires verdicts to follow the index's rules. test_verdicts_agree_with_iter_source_files keeps src/notes.md out of the excluded set.
• Plan: PRD-739 User Story 3 AC: states that uncommitted changes are included.
  Did: Emits includes_uncommitted: true; the tool_code_changes docstring states that untracked files are not in the diff.
  Why: git diff <fork point> compares tracked paths only: a scratch-repo probe listed a staged file and not an untracked one.
• Plan: Section 0: keep edits to upstream-owned files (mcp_server.py, engine.py, call_graph.py, repo_map/graph.py) small, with new logic in fork-only files; change 1 adds the graph.py alias as 'the only upstream-file edit'.
  Did: Change 5 adds 9 lines to engine.py (_reindex_files: a 3-line filter plus a 5-line comment), on top of change 1's 3-line graph.py alias and changes 2-3's 19 docstring lines in mcp_server.py. All new logic is in the new fork-only module src/lemoncrow/infra/code_intel/inclusion.py (149 lines).
  Why: The entry point change 5 names is _reindex_files, which lives in engine.py; nothing outside that file can filter what it re-extracts. The comment was trimmed from 7 lines to 5 before commit to keep the upstream conflict surface small.

Review

/code-review:deep --base origin/main (code-review 3.10, review_root = this worktree, code intelligence on): NEEDS_ATTENTION — verified 4, justified 0, rejected 0, pending 0, coverage gaps 0: 1 HIGH and 3 MEDIUM.

Fixed (all four, in 3b48dd65)

• design_critic_f0 [HIGH] src/lemoncrow/infra/code_intel/inclusion.py:113 -- New inclusion.py inverts the infra/pro.capabilities dependency direction, creating a package-level import cycle
• bha_p0_f0 [MEDIUM, invariant-critical] src/lemoncrow/infra/code_intel/inclusion.py:149 -- indexable_paths admits the source-file-scan remainder, so excluded paths can still be indexed
• design_critic_f1 [MEDIUM] src/lemoncrow/infra/code_intel/coverage.py:194 -- Free-tier file-cap selection logic duplicated across coverage.py and engine.py instead of shared
• bhb_f0 [MEDIUM] src/lemoncrow/infra/code_intel/coverage.py:194 -- Free-tier cap boundary in _index_selection ignores exclude_globs, can mislabel an exclude_globs-dropped file as 'free-tier-file-cap' instead of 'missing'

The import-direction fix moves the rule primitives down into infra and has repo_map/graph.py import and re-export them, so the ladder keeps one definition and inclusion.py imports no lemoncrow.pro at any scope (pinned by tests/infra/code_intel/test_inclusion_layering.py). The invariant fix carries a negative control: disabling the new scan rung makes test_an_excluded_path_never_enters_the_index fail on exactly one path, and restoring it passes.

Left open on purpose

• Residual of bha_p0_f0, deliberately left: the scan's git-visibility rung is still approximated by git check-ignore, so an untracked file inside a submodule -- which git ls-files --others does not recurse into and the scan therefore drops -- is still admitted by indexable_paths while coverage reports it excluded under source-file-scan. Closing it costs either an O(repo) git ls-files on the post-edit path whose entire purpose is O(edited files) latency, or a --pathspec-from-file dependency (git >= 2.25). The class is named in coverage._exclusion's docstring so the next reader is not surprised by it.

Comment pruning (/prune-comments): reviewed 63, cut 5, kept 56, rewritten 2.

Re-review after the fixes (--since-last-review, 62d25ef...HEAD): APPROVED — 0 findings in every bucket.

Test pruning (/prune-tests): reviewed 9 new tests, cut 0, kept 9.

Verification

• uv run pytest -q tests/infra/code_intel/test_coverage.py tests/infra/code_intel/test_change_impact.py tests/gateway/test_completeness_contract.py -p no:cacheprovider — pass: 114 passed, 14 warnings in 29.47s on HEAD 62d25ef; includes attempt 2's two new tests.
• FILES=$(git grep -l -E '_reindex_files|indexable_paths|code_intel\.inclusion|iter_source_files|index_repo\(' -- tests/ | grep -v '^tests/benchmarks/'); uv run pytest -q $ — pass: 17 files (test_code_context.py, test_repo_map.py, test_git_submodule_indexing.py, test_mcp_tool_handlers.py among them): 1 failed, 539 passed, 2 deselected in 171.17s on HEAD 62d25ef. The one failure, tests/core/test_code_context_index_workers.py::test_index_pool_does_not_fork_l
• Pre-existing-failure check on origin/main 4bf10c51 in a scratch worktree: uv run pytest -q tests/core/test_code_context_index_workers.py — pass: Reproduced: 1 failed, 5 passed with the identical assert 'spawn' == 'forkserver' at test_code_context_index_workers.py:52, on origin/main without this branch's diff (macOS default multiprocessing start method). It also fails in isolation on the branch, so it is not test-order
• uv run ruff check src/lemoncrow/infra/code_intel/inclusion.py src/lemoncrow/infra/code_intel/coverage.py src/lemoncrow/pro/capabilities/code_context/engine.py tests/infra — pass: All checks passed! The .githooks pre-commit hook then ran ruff --fix, black and mypy over the 4 staged files at commit time: 'All checks passed!', '4 files left unchanged', 'Success: no issues found in 3 source files'.
• uv run mypy --strict src/lemoncrow/infra/code_intel/inclusion.py src/lemoncrow/infra/code_intel/coverage.py ; uv run mypy --strict src/lemoncrow/pro/capabilities/code_con — pass: No new errors. inclusion.py + coverage.py: 'Success: no issues found in 2 source files'. engine.py: 7 errors on the branch and the same 7 on origin/main 4bf10c5 in a scratch worktree (since removed) -- identical codes and messages, the last one at 14862 on main and 14873 on the
• uv run pytest -q tests/test_mypyc_compile_safety.py — pass: 3 passed on HEAD 62d25ef.
• Counterfactuals, attempt 2: 4 one-line mutations, each target test run in isolation (backup with cp, mutate, pytest <node id>, restore, shasum -c) — pass: All 4 reported 1 failed. (A) indexable_paths applying no rules failed test_an_excluded_path_never_enters_the_index; (B) indexable_paths returning [] failed test_on_demand_indexing_still_takes_a_new_source_file; (C) the skipped-directory rule disabled in inclusion.exclusion_rule
• Exit criterion live re-check: check_coverage(repo_root=main checkout) whole-repo vs iter_source_files, run from this worktree, read-only — pass: engine_index_version 591, 2143 verdicts in 0.90s (1955 indexed, 94 unparsed, 92 excluded, 1 missing, 1 stale); 0 excluded paths selected by the scan, 0 on-disk missing paths outside it, 0 excluded verdicts without a rule (90 unrecognised-file-type, 2 skipped-directory). It also m
• Reproduction of the index-inclusion bug before fixing it, in a temp git repo — pass: With a gitignored .claude/worktrees/wt/src/app.py and a tracked prompts/prompt.txt, index_repo indexed only src/app.py, then the unfixed _reindex_files added files rows for BOTH excluded paths (and symbols for the nested copy). This is the minimal form of what the GA check saw.
• git diff --check origin/main...HEAD — pass: clean
• uv run pytest -q tests/gateway/test_mcp_jsonrpc_e2e.py — skipped: Excluded on purpose by the campaign verification scope; CI runs it.
• uv run pytest -q -m "not slow" — skipped: Excluded by the campaign verification scope; no PR 4 ledger criterion requires a full-suite run. make typecheck over all of src and make pre-commit were likewise excluded; CI runs them. Carried unchanged from attempt 1.
• git -C /Users/kris.wong/Dev/lemoncrow status --porcelain; git -C <worktree> status --porcelain — pass: both empty after the commit, and after every edit batch and counterfactual restore in between
• Resolve: uv run pytest -q tests/infra/code_intel/test_coverage.py tests/infra/code_intel/test_inclusion_layering.py tests/infra/code_intel/test_change_impact.py tests/gateway/test — pass
• Resolve: git grep -l -E 'inclusion|indexable_paths|_reindex_files|iter_source_files|should_skip_relative_path' -- tests/ then uv run pytest -q tests/core/test_git_submodule_indexi — pass
• Resolve: uv run ruff check src/lemoncrow/infra/code_intel/inclusion.py src/lemoncrow/infra/code_intel/coverage.py src/lemoncrow/pro/capabilities/repo_map/graph.py src/lemoncrow/pr — pass
• Resolve: uv run mypy --strict <changed src files>, per-file — fail
• Resolve: uv run pytest -q tests/test_mypyc_compile_safety.py — pass
• Resolve: import-direction proof: uv run python -c 'import lemoncrow.infra.code_intel.inclusion, lemoncrow.infra.code_intel.coverage' with a sys.modules scan, plus grep -rn '^from — pass
• Resolve: negative control on the new invariant test: disable the scan rung in exclusion_rule, then run tests/infra/code_intel/test_coverage.py::test_an_excluded_path_never_enters_ — pass
• Resolve: pre-commit hook on commit (format + lint + typecheck + large-file), no --no-verify — pass

The one fail row is per-file mypy --strict on files that are not clean on origin/main either: 1 error in repo_map/graph.py (identical against the base copy) and 7 in engine.py, none on a line this diff touches. The repo's own pre-commit type check passed on every commit.

Artifact updates this PR triggers

• PRD-739 FR10 is reworded to say match_kind covers symbol-edge enumerations only (PLN-2027 §11 decision 3).
• PRD-739's "Non-code files" edge case is corrected: markdown is indexed, so only unregistered types report excluded.

Still owed by later chunks

• C5 (PR 5): keep operators on the fork build (FR11).
• C6 (PR 6): repo-correct PR risk and review-module warm-up. pr_risk stays refused through the broker because _file_complexity writes the machine-wide semantic index.
• C7 (PR 7): remove the fork-only lc code export / import.
• C9 (PR 8): the three findings the GA review confirmed in feat(mcp): tool broker reaches read-only tools only (PLN-2027 PR 3) #18 — the Plan-Mode hook auto-allowing search/context, the broker's copied graph default, and the savings panel on list-only hosts.
• GA sign-off: held until this merges, then the check re-runs against an index that cannot hold worktree files.

Artifacts: PRD-739 · PLN-2027