ctags export recovery + caller-graph defs cache: eval hits 65/65 (v1.4.0)

Two tool-behavior improvements, both kernel-verified.

1. ctags export recovery — closes the last eval known-fail.
   GNU Global's C parser derails on sparse annotations
   (__acquires(lock) on a forward declaration) and silently drops
   every definition below the derail point; find_definition
   ("mutex_lock") returned 7 shadows/variants but never the real
   kernel/locking/mutex.c:314. EXPORT_SYMBOL*(sym) always sits in the
   defining .c and IS indexed, so: an export site whose file has no
   definition record is proof of a parser miss — confirm with a
   (cached) Universal Ctags scan and prepend the definition, flagged
   resolved_via "ctags:EXPORT_SYMBOL".
   - macros.export_sites() extracts (path, line, variant) per export.
   - enrich.definition_tag() picks definition-shaped ctags tags
     (prototypes/externvars rejected).
   - server._export_recovery() + shared _resolve_definitions(): wired
     into find_definition, symbol_info (free — refs already fetched),
     get_symbol_body, and find_callees, so the recovered definition is
     also readable and analyzable. -sx retry covers fully-missed
     symbols whose occurrences land in GSYMS. Disabled with the
     existing enrichment knob (--no-enrich / GTAGS_MCP_ENRICH /
     enrich=false); best-effort, never fails a query.
   - def-mutex_lock (known-fail since v1.0.0) now PASSES and asserts
     top_path + resolved_via: eval is 65/65 = 100% recall,
     15/15 = 100% precision@1 on the mid-2026 master snapshot.

2. Per-file `global -fx` definition cache for the caller graph.
   reachability/blast_radius run _callers_of once per BFS node and
   kept re-running identical `global -fx <file>` subprocesses for hot
   files. New LRU (capacity 4096) keyed by (root, index generation,
   path) — generation bumps on every build/refresh path, so entries
   die with the index state they were read from; negative results
   cached; thread-safe. Measured: warm reachability 552ms -> 46ms,
   warm find_callers on a 200-caller symbol 61ms -> 3ms.

Tests 293 -> 304: export_gap_project fixture reproduces the parser
derail end-to-end (skips if upstream fixes it), recovery via
find_definition/symbol_info/get_symbol_body/find_callees, opt-out
inertness, -fx run-counting (one run per file per generation),
cache invalidation via update_index, generation-bump coverage, plus
pure export_sites/definition_tag units.

Stability exercise: 28-call matrix (new def-recovered-mutex_lock
case), 28/28 sane, zero anomalies; docs/capability.md operational
table and known-limitations rewritten accordingly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Author: harshithsunku

README.md

@@ -25,7 +25,7 @@ Every AI coding agent — Claude Code, Cursor, Codex, you name it — answers *"
 
 - **~100× faster per query** — milliseconds instead of seconds, at any codebase size
 - **Radically less noise** — the definition, not 7,873 lines of matches
-- **Speaks kernel** — `#ifdef` guard stacks with `.config` filtering, and macro-generated symbols (`sys_read` → its `SYSCALL_DEFINE3` site) that no other tagging tool resolves
+- **Speaks kernel** — `#ifdef` guard stacks with `.config` filtering, macro-generated symbols (`sys_read` → its `SYSCALL_DEFINE3` site) that no other tagging tool resolves, and definitions the parser misses recovered from their `EXPORT_SYMBOL` site via ctags
 - **Zero index management** — first query builds the index, every query auto-refreshes it
 - **Correctness measured in CI** — a 65-case golden eval covering all 11 tools against a pinned kernel: 100% recall, 100% precision@1 ([docs/capability.md](docs/capability.md))
 - **Works everywhere MCP does** — Claude Code, Claude Desktop, Cursor, any MCP client
@@ -42,6 +42,7 @@ Measured on a full Linux kernel checkout — **65,163 C/C++ files, 37.1 million
 | Show me `tcp_v4_rcv`'s implementation | *read a 3,500-line file* | **`get_symbol_body`** | **exactly the 271-line function** |
 | Who calls `ext4_mark_inode_dirty`? | 245 raw match lines | **`find_callers`** | **62 deduped caller functions, with counts** |
 | Where is `sys_read` *really* defined? | *no answer — the name is macro-generated* | **0.03 s** | `fs/read_write.c SYSCALL_DEFINE3(read, ...)`, flagged `resolved_via` |
+| Where is `mutex_lock` defined? | 24,774 noisy match lines | **0.2 s** | `kernel/locking/mutex.c:314` — recovered via `EXPORT_SYMBOL` + ctags after gtags' parser derails on it |
 | Does `ksys_read` ever reach `rw_verify_area`? | *N rounds of grep + reading* | **0.6 s** | the shortest call chain, with every call site's file:line |
 | What does my uncommitted diff impact? | *not answerable* | **0.1 s** | `blast_radius`: changed functions + callers, ranked by distance |
 
@@ -53,7 +54,7 @@ One-time index build: **66 s** for the whole kernel. Incremental refresh after e
 
 The speed is nice. The real win is **precision**: an agent that gets 5 exact lines instead of 7,873 noisy ones keeps its context window for actual reasoning.
 
-And the answers are *measured*, not assumed: CI runs a [65-case golden eval](evals/golden.jsonl) against pinned kernel v6.16 on every push — currently **100% recall, 100% precision@1** across definitions, macro resolution, references, callers, callees, definition bodies, `#ifdef` guards, and reachability, covering all 11 tools. The full methodology, numbers, and honest limitations live in [docs/capability.md](docs/capability.md).
+And the answers are *measured*, not assumed: CI runs a [65-case golden eval](evals/golden.jsonl) against pinned kernel v6.16 on every push — currently **100% recall, 100% precision@1** across definitions, macro resolution, export recovery, references, callers, callees, definition bodies, `#ifdef` guards, and reachability, covering all 11 tools. The full methodology, numbers, and honest limitations live in [docs/capability.md](docs/capability.md).
 
 ## Quick start (60 seconds)
 

ROADMAP.md

@@ -117,14 +117,18 @@ Trust through measurable quality — reported on its own terms.
 **Done when:** an eval command prints a score in CI, and the capability writeup is live.
 ✅ `mcp-gtags-server eval --golden evals/golden.jsonl --root <kernel>` runs 65
 path-level cases covering all 11 tools in ~14s — 100% recall and 100% precision@1
-in CI on pinned v6.16; 98.5% recall on a 2026 master snapshot, where the one
-deliberate known-fail (a GNU Global parser gap triggered by the new `__acquires()`
-annotations) keeps the number honest. CI (`eval.yml`) clones the pinned kernel tag
-weekly and on main pushes and publishes the score in the job summary. A second
-operational layer, `scripts/stability_exercise.py`, measures per-tool latency and
-response size against the kernel (27-call matrix, zero anomalies — table in the
-writeup). The writeup lives at [docs/capability.md](docs/capability.md); posting
-it to the blog/HN is the Reach item below.
+on pinned v6.16 AND on a 2026 master snapshot. The one long-standing known-fail
+(a GNU Global parser gap triggered by the new `__acquires()` annotations that
+hid `mutex_lock`'s real definition) was closed in v1.4.0 by ctags export
+recovery: definitions the parser misses are restored from their `EXPORT_SYMBOL*`
+site, flagged `resolved_via: "ctags:..."`. v1.4.0 also added a per-file
+definition cache keyed by index generation, dropping warm reachability from
+~0.55s to ~46ms. CI (`eval.yml`) clones the pinned kernel tag weekly and on main
+pushes and publishes the score in the job summary. A second operational layer,
+`scripts/stability_exercise.py`, measures per-tool latency and response size
+against the kernel (28-call matrix, zero anomalies — table in the writeup). The
+writeup lives at [docs/capability.md](docs/capability.md); posting it to the
+blog/HN is the Reach item below.
 
 ---
 

docs/capability.md

@@ -21,15 +21,16 @@ path-level so they hold across kernel versions.
 Current scores:
 
 ```
-local, mid-2026 master snapshot: 64/65 = 98.5% recall, 14/14 = 100.0% precision@1, 14s
-CI, kernel v6.16 (pinned):       published on every push (100.0% on the previous 50-case set)
+local, mid-2026 master snapshot: 65/65 = 100.0% recall, 15/15 = 100.0% precision@1, 14s
+CI, kernel v6.16 (pinned):       published on every push
 ```
 
-The one failure on newer kernels is kept in the set deliberately: recent
-kernels added sparse `__acquires()` annotations to mutex forward
-declarations, and GNU Global's C parser derails on them — the eval caught a
-real upstream parser regression triggered by a new kernel annotation style
-(see "Known limitations").
+One case (`def-mutex_lock`) was a deliberate known-fail from v1.0.0 through
+v1.3.1: recent kernels added sparse `__acquires()` annotations to mutex
+forward declarations, and GNU Global's C parser derails on them, missing the
+real `mutex_lock` definition. v1.4.0 closes it with **ctags export
+recovery** (see "What resolves"), and the case now also asserts the recovery
+mechanism itself.
 
 ## What resolves
 
@@ -67,6 +68,24 @@ server resolves them from the index alone and flags the mechanism:
 Resolution costs 20–75 ms per query on the kernel; a miss costs ~9 ms.
 `symbol_info` also reports which `EXPORT_SYMBOL*` variant exports a symbol.
 
+**Parser-missed definitions recover via their export site** (v1.4.0). When
+GNU Global's parser derails mid-file (sparse annotations like
+`__acquires(lock)` on a forward declaration are the known trigger), real
+definitions below that point vanish from the index. But `EXPORT_SYMBOL*(sym)`
+always sits in the `.c` file that defines `sym` — so when an export site's
+file has no definition record, a (cached) Universal Ctags scan of that file
+confirms and restores the definition, ranked first and flagged
+`resolved_via: "ctags:EXPORT_SYMBOL"`:
+
+| query | gtags alone | with export recovery |
+|---|---|---|
+| `mutex_lock` | 7 records, none the real one | `kernel/locking/mutex.c:314` first, kind `function` |
+
+The recovered definition flows through the whole tool surface —
+`get_symbol_body` reads its real body, `find_callees` analyzes it — and
+costs ~0.25 s once (then cached). Disabled together with enrichment
+(`--no-enrich`).
+
 **Call-graph questions** collapse into single calls:
 
 - `reachability(ksys_read, rw_verify_area)` → the shortest chain, with the
@@ -80,47 +99,51 @@ Resolution costs 20–75 ms per query on the kernel; a miss costs ~9 ms.
 ## Operational profile (measured)
 
 [scripts/stability_exercise.py](../scripts/stability_exercise.py) runs a
-27-call matrix over all 11 tools against a real kernel tree and records
+28-call matrix over all 11 tools against a real kernel tree and records
 latency and response size (`--json` for diffing between runs). Measured on
-the 37M-line mid-2026 snapshot, 8 GB machine, warm index — **27/27 sane,
+the 37M-line mid-2026 snapshot, 8 GB machine, warm index — **28/28 sane,
 zero anomalies**:
 
 | call | warm latency | response size |
 |---|---|---|
-| `find_definition` (indexed hit) | ~2 ms | 0.5–1.7 KB |
-| `find_definition` (macro-generated, e.g. `sys_read`) | ~47 ms | 0.8 KB |
-| `find_references` (`kmalloc`, 2 744 refs, one page) | ~60 ms | 23 KB |
-| `get_symbol_body` (function / struct / macro) | ~2 ms | 0.6–7.5 KB |
-| `find_callers` (typical) | 13–61 ms | 0.5–5 KB |
-| `find_callers` (worst allowed: `schedule`, 610 callers / ~430 files) | ~1.5 s | — |
-| `find_callers` (>500 files: guard refuses gracefully) | ~53 ms | 0.3 KB |
-| `summarize_references` (`kmalloc`, 1 553 files) | ~56 ms | 0.8 KB |
-| `find_callees` | 15–40 ms | 0.9–1.9 KB |
-| `symbol_info` | 5–56 ms | 1.4 KB |
-| `reachability` (3-hop chain found) | ~0.6 s | 0.6 KB |
+| `find_definition` (indexed hit) | 4–45 ms | 0.5–1.7 KB |
+| `find_definition` (macro-generated, e.g. `sys_read`) | ~40 ms | 0.8 KB |
+| `find_definition` (parser-missed, ctags-recovered: `mutex_lock`) | ~0.22 s | 2.6 KB |
+| `find_references` (`kmalloc`, 2 744 refs, one page) | ~52 ms | 23 KB |
+| `get_symbol_body` (function / struct / macro) | 4–120 ms | 0.6–7.5 KB |
+| `find_callers` (typical, `-fx` cache warm) | 2–3 ms | 0.5–5 KB |
+| `find_callers` (>500 files: guard refuses gracefully) | ~54 ms | 0.3 KB |
+| `summarize_references` (`kmalloc`, 1 553 files) | ~52 ms | 0.8 KB |
+| `find_callees` | 17–45 ms | 0.9–1.9 KB |
+| `symbol_info` | 5–48 ms | 1.4 KB |
+| `reachability` (3-hop chain found) | ~46 ms warm (~0.35 s cold) | 0.6 KB |
 | `list_file_symbols` (80-symbol file) | ~4 ms | 27 KB |
 | `blast_radius` (real one-line edit, depth 1) | ~0.1 s | 2.5 KB |
 | `update_index` (synchronous incremental, 37M lines) | 6–10 s | 0.3 KB |
 
-Every response an agent sees stays comfortably under typical MCP client
-timeouts, wide queries are bounded by pagination and breadth guards, and the
-one deliberately slow call (`update_index`) is the explicit freshness barrier
-— background refresh keeps normal queries off that path.
+The caller-graph tools share a per-file definition cache keyed by index
+generation (v1.4.0): a reachability walk that re-visits the same hot files
+went from ~0.55 s to ~46 ms warm, and repeated `find_callers` calls in one
+agent session cost single-digit milliseconds. Every response stays
+comfortably under typical MCP client timeouts, wide queries are bounded by
+pagination and breadth guards, and the one deliberately slow call
+(`update_index`) is the explicit freshness barrier — background refresh
+keeps normal queries off that path.
 
 ## Known limitations (measured, not hidden)
 
 - **Function pointers are invisible to static tagging.** `vfs_read` never
   statically reaches `ext4_file_read_iter` — the route is `f_op->read_iter`.
   `reachability` says so explicitly rather than returning a wrong path; the
   golden set asserts this honest "no" (`reach-fnptr-honesty`).
-- **One upstream parser gap, kept as a failing eval case:** GNU Global's C
-  parser derails on the sparse annotation kernels newer than ~v6.16 put on
-  forward declarations
+- **Upstream parser gaps are recovered, not fixed.** GNU Global's C parser
+  derails on the sparse annotation kernels newer than ~v6.16 put on forward
+  declarations
   (`static void __sched __mutex_lock_slowpath(...) __acquires(lock);`) and
-  misses the real `mutex_lock` definition at `kernel/locking/mutex.c:314`.
-  The `CONFIG_DEBUG_LOCK_ALLOC` macro variant and the `PREEMPT_RT` variant
-  are still found, and older kernels are unaffected (hence 100% in CI on
-  v6.16 vs 98% on a 2026 master snapshot).
+  misses the real `mutex_lock` definition. Since v1.4.0, export recovery
+  restores such definitions from their `EXPORT_SYMBOL*` site via ctags — but
+  a parser-missed definition that is *not* exported (static helpers below
+  the derail point) stays invisible until the upstream parser is fixed.
 - **Prototypes rank alongside definitions.** `find_definition` can return a
   header prototype before the `.c` definition (both are index "definitions");
   the ctags `kind` field distinguishes them.

evals/golden.jsonl

@@ -12,12 +12,12 @@
 {"id": "def-tcp_v4_rcv", "category": "definition", "tool": "find_definition", "args": {"symbol": "tcp_v4_rcv"}, "expect": {"paths": ["net/ipv4/tcp_ipv4.c"], "top_path": ["net/ipv4/tcp_ipv4.c"]}}
 {"id": "def-ext4_map_blocks", "category": "definition", "tool": "find_definition", "args": {"symbol": "ext4_map_blocks"}, "expect": {"paths": ["fs/ext4/inode.c"]}}
 {"id": "def-kernel_clone", "category": "definition", "tool": "find_definition", "args": {"symbol": "kernel_clone"}, "expect": {"paths": ["kernel/fork.c"], "top_path": ["kernel/fork.c"]}}
-# def-mutex_lock is a deliberate KNOWN-FAIL: GNU Global's C parser derails on the
-# sparse annotation in "static void __sched __mutex_lock_slowpath(...) __acquires(lock);"
-# and misses the real mutex_lock definition at kernel/locking/mutex.c:314 entirely
-# (the CONFIG_DEBUG_LOCK_ALLOC macro and PREEMPT_RT variants are still found).
-# Tracked honestly instead of hidden — see docs/capability.md.
-{"id": "def-mutex_lock", "category": "definition", "tool": "find_definition", "args": {"symbol": "mutex_lock"}, "expect": {"paths": ["kernel/locking/mutex.c"]}}
+# def-mutex_lock exercises ctags export recovery (v1.4.0): GNU Global's C parser
+# derails on the sparse annotation in "static void __sched __mutex_lock_slowpath(...)
+# __acquires(lock);" and misses the real definition at kernel/locking/mutex.c:314 —
+# the server recovers it from the EXPORT_SYMBOL(mutex_lock) site via ctags and ranks
+# it first. This case was a deliberate known-fail from v1.0.0 to v1.3.1.
+{"id": "def-mutex_lock", "category": "definition", "tool": "find_definition", "args": {"symbol": "mutex_lock"}, "expect": {"paths": ["kernel/locking/mutex.c"], "top_path": ["kernel/locking/mutex.c"], "resolved_via": "ctags:EXPORT_SYMBOL"}}
 {"id": "def-skb_clone", "category": "definition", "tool": "find_definition", "args": {"symbol": "skb_clone"}, "expect": {"paths": ["net/core/skbuff.c"], "top_path": ["net/core/skbuff.c"]}}
 {"id": "def-do_dentry_open", "category": "definition", "tool": "find_definition", "args": {"symbol": "do_dentry_open"}, "expect": {"paths": ["fs/open.c"], "top_path": ["fs/open.c"]}}
 {"id": "def-wait_event", "category": "definition", "tool": "find_definition", "args": {"symbol": "wait_event"}, "expect": {"paths": ["include/linux/wait.h"]}}

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "mcp-gtags-server"
-version = "1.3.1"
+version = "1.4.0"
 description = "Indexed code navigation for AI coding agents — replace grep scans with GNU Global (gtags) lookups over MCP. ~100x faster and radically less noise on million-line C/C++ codebases."
 readme = "README.md"
 requires-python = ">=3.10"

scripts/stability_exercise.py

@@ -92,11 +92,24 @@ def status_ok(data):
     def has_suggestions(data):
         return None if data.get("suggestions") else "expected suggestions on the miss"
 
+    def recovered_from_export(data):
+        paths = [r["path"] for r in data["results"]]
+        if "kernel/locking/mutex.c" not in paths:
+            return f"expected recovered kernel/locking/mutex.c, got {paths[:3]}"
+        if not str(data.get("resolved_via", "")).startswith("ctags:"):
+            return f"expected ctags resolved_via, got {data.get('resolved_via')!r}"
+        return None
+
     return [
-        # find_definition: plain, multi-def guarded, macro-generated, miss.
+        # find_definition: plain, multi-def guarded, macro-generated,
+        # parser-missed (ctags export recovery), miss.
         Case("def-plain", "find_definition", {"symbol": "vfs_read"}),
         Case("def-guarded-kmap", "find_definition", {"symbol": "kmap"}, min_results=2),
         Case("def-macro-sys_read", "find_definition", {"symbol": "sys_read"}),
+        Case(
+            "def-recovered-mutex_lock", "find_definition", {"symbol": "mutex_lock"},
+            extra_check=recovered_from_export,
+        ),
         Case("def-struct", "find_definition", {"symbol": "task_struct"}),
         Case(
             "def-miss-suggestions", "find_definition", {"symbol": "tcp_v4_r"},

server.json

@@ -6,14 +6,14 @@
     "url": "https://github.com/harshithsunku/mcp-gtags-server",
     "source": "github"
   },
-  "version": "1.3.1",
+  "version": "1.4.0",
   "websiteUrl": "https://github.com/harshithsunku/mcp-gtags-server",
   "packages": [
     {
       "registryType": "pypi",
       "registryBaseUrl": "https://pypi.org",
       "identifier": "mcp-gtags-server",
-      "version": "1.3.1",
+      "version": "1.4.0",
       "runtimeHint": "uvx",
       "transport": {
         "type": "stdio"

src/gtags_mcp/__init__.py

@@ -1,3 +1,3 @@
 """MCP server exposing GNU Global (gtags) code navigation for C/C++ codebases."""
 
-__version__ = "1.3.1"
+__version__ = "1.4.0"

src/gtags_mcp/enrich.py

@@ -235,3 +235,22 @@ def best_tag(tags: dict[str, list[dict]], name: str, line: int) -> dict | None:
     if len(entries) == 1:
         return entries[0]
     return None
+
+
+# Declaration-shaped kinds: they point AT a definition, they are not one.
+_NON_DEFINITION_KINDS = frozenset({"prototype", "externvar"})
+
+
+def definition_tag(tags: dict[str, list[dict]], name: str) -> dict | None:
+    """The best definition-shaped tag for `name` in a file's tags, or None.
+
+    Prototypes and extern declarations are rejected — export recovery needs
+    the site that actually defines the symbol. Ties break on kind priority,
+    then the earliest line.
+    """
+    entries = [
+        t for t in tags.get(name, []) if t["kind"] not in _NON_DEFINITION_KINDS
+    ]
+    if not entries:
+        return None
+    return min(entries, key=lambda t: (_kind_rank(t["kind"]), t["line"]))

src/gtags_mcp/macros.py

@@ -186,6 +186,24 @@ def exported_via(symbol: str, sources: Iterable[str]) -> str | None:
     return None
 
 
+def export_sites(symbol: str, cxrefs: Iterable[Cxref]) -> list[tuple[str, int, str]]:
+    """Every EXPORT_SYMBOL* site of `symbol` among its occurrence cxrefs.
+
+    Returns (path, line, variant) tuples, deduplicated by path, source order
+    kept. EXPORT_SYMBOL* always sits in the .c file that defines the symbol,
+    so these paths are ground truth for where a definition must exist — the
+    signal export recovery uses when the parser missed one.
+    """
+    sites: list[tuple[str, int, str]] = []
+    seen: set[str] = set()
+    for _, lineno, path, source in cxrefs:
+        match = _EXPORT_RE.search(source)
+        if match and match.group("name") == symbol and path not in seen:
+            seen.add(path)
+            sites.append((path, lineno, match.group(0)))
+    return sites
+
+
 def resolve(
     symbol: str, query: Query, direct_empty: bool = True
 ) -> tuple[list[Cxref], str | None]: