Repository navigation
[pull] main from DeusData:main - #136
Merged
Merged
Conversation
The Vue change (#1852) built the shared embedded-script seam — the included-ranges parse over the original buffer and the lang=/src= resolver — but gated structural extraction on the host being Vue. A Svelte component, an HTML page or an Astro file therefore still yielded only File + Module nodes and IMPORTS edges from its <script> blocks: no Function definitions, no CALLS, and a false dead-code verdict for anything used only from a component. Widen the seam to every host that declares an embedded spec: - Structural extraction (definitions + calls) runs for every embedded block; the per-host boolean is gone. - The Vue attribute resolver becomes script_embedded_language() and runs for all four markup hosts. It starts from the spec row's language instead of hard-coding JavaScript, and learns HTML's type= attribute: absent or empty, module, text/javascript and application/javascript pass; application/json, importmap, text/x-template and the like bail, so data blocks cannot leak garbage definitions into the graph. src= and unsupported lang= values bail as before. CFML's <cfscript> carries no attributes and keeps its spec-fixed language. - Astro's frontmatter fence and <script> bodies default to TypeScript: Astro type-checks both as TS with no lang= attribute to say so, and the fence cannot carry one. The TypeScript grammar parses untyped JavaScript unchanged. Tests: the #1852 control that pinned Svelte/HTML/Astro at zero functions and zero calls now asserts the real symbols; stay-at-zero controls cover type="application/json" and <script src> in every host; both Svelte blocks (context="module", and Svelte 5's bare module) contribute in host-file coordinates; HTML's accepted type= forms and Astro's TS frontmatter are covered; an end-to-end CALLS edge inside a .svelte file is asserted. The Svelte and Astro label goldens gain the Variable their fixtures always declared. Verified on a 213-component Vue corpus: node counts per label and per host language, edge counts and trace_path output are identical before and after. Fixes #1807 Signed-off-by: Brian McNaboe <bmcnaboe@gmail.com>
…963) src/cli/cli.c reported an error range of 1-13047 — the whole file. The file indexed fine; the report was wrong. Three #ifndef _WIN32 blocks split a brace (two `if` headers, one closing brace), so the raw tree-sitter parse cannot resync at file scope, the root node becomes ERROR, and cbm.c takes its whole-file branch. The pipeline already parses these files a second time after preprocessing, and that parse is clean. The report just never consulted it. Build one byte per original line from the preprocessed pass, then cut each raw error range down to the runs of lines the second parse could not vouch for. Three rules, all found by running it and all load-bearing: - An expanded line only vouches for its original line when it HAS TEXT. The preprocessor emits a blank line where it dropped a branch; treating that blank as proof suppressed every C range in the suite. - Preprocessor directive lines (with backslash continuations) never count as missing code — the preprocessor consumes them, so the second parse can never vouch for one. Without this every #include block reported as a miss. Known cost: a #define the raw parse really dropped no longer shows up on its own. - A TOP-LEVEL macro invocation line never counts as vouched-for even when the expanded line parses clean. The macro can expand to a whole definition that the recovery walker deliberately refuses to adopt (#949), so a clean second parse there proves nothing. An in-body invocation is the benign #1071 case and is left to the existing macro subtraction. The order of the three coverage steps is now settled by where each one's evidence lives: recovery subtraction -> before the refinement; its evidence is a whole definition that STARTS inside the range, so it must be asked while the range still matches the construct the refinement -> middle #1071 macro rule -> after the refinement; its evidence is per-line, so a narrow range points at the call itself Measured on this repo: src/cli/cli.c goes from one whole-file range to 64 ranges over ~9.8% of the file, tests/test_cli.c from 48.6% to ~2.9%, src/cli/activation_transaction.c from 38% to 7.5%. What survives is honest — the biggest remaining ranges in cli.c are genuinely discarded #ifdef _WIN32 and #ifdef CBM_CLI_ENABLE_TEST_API blocks, absent from the graph on this platform. Both percentages above are floors, not measurements: cli.c and test_cli.c now land on exactly 64 ranges, which is CBM_MAX_ERROR_REGIONS. That cap drops regions with no signal, and a follow-up raises it and adds a truncation marker. Five tests, all red before the change: the range narrows to the dropped branch; lines the preprocessor explained are excluded; a range never starts or ends on a directive; real garbage beside a split brace stays flagged; a clean file stays unflagged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
…-file failures (#963) Two silent failures in the parse-coverage report, both made visible by the Phase 2 range refinement that came before this. Two caps sat in series and both returned early without saying anything: CBM_MAX_ERROR_REGIONS = 64 internal/cbm/cbm.c COVERAGE_RANGE_MAX = 128 src/mcp/mcp.c Raising only the first would have moved the clip from 64 to 128, so both move to 256. This was live behaviour, not a theoretical limit: after Phase 2 split one whole-file range into many small ones, src/cli/cli.c and tests/test_cli.c both reported exactly 64 ranges — the cap binding, dead-on, twice. Every coverage figure measured before this change was a floor. With the cap at 256 the true numbers are cli.c 13.9% (not 9.8%) and test_cli.c 3.1%, and the longest list in the repo is 85 ranges. A raised cap is still a cap, so the report now says when it clipped: - cbm_error_regions_t gained a `dropped` counter, and cbm_collect_error_regions walks to the end instead of stopping at the cap, so the count is exact rather than a lower bound. That costs little — the walk never descends into an ERROR subtree. - cbm_error_ranges_str appends ",+<N>" when N ranges were thrown away. - coverage_add_ranges reads that marker and sets "truncated": true, and also sets it when its own limit stops the loop. Before this the marker was invisible: the parser stopped at the '+' with no error and no leftover, so a clipped list arrived looking complete. - objectscript_export_append_error_ranges strips markers off both operands before joining two Studio Export parts and adds one back at the end. A marker left mid-string would make every reader stop there and silently lose every range after it. "Look at lines 1 to 13047" of a 13046-line file tells a reader nothing. Those files now carry their own kind rather than being described as partially covered. New `parse_unusable` field in CBMFileResult, set when one range covers 80% or more of the file. Its customers are non-C languages: the Phase 2 refinement that narrows a whole-file range using the preprocessed parse only runs for C, C++ and CUDA, so a Python, Java, Ruby or TypeScript file whose root node is ERROR still reports 1-N. Verified against real files in all four. The kind is `parse_unusable`, not `parse_failed`. index_coverage.kind already means one of two things — indexed-but-partial, or a skip phase saying the file was never indexed at all — and `parse_failed` reads as the second when it is the first. The store.c schema comment, which is the only written record of this vocabulary, now describes all three classes and says why. Two places would have mislabelled the new kind as "skipped", which is exactly that confusion: coverage_status fell through to its catch-all pass, and add_coverage_report fell into its else branch. A reader who finds a file under "skipped" believes it is absent from the graph, when it was indexed. Both now have explicit branches. index_status gained parse_unusable_count so a CI gate can read it without parsing anything else, get_code_snippet says "read the source directly" instead of naming useless ranges, and the three tool descriptions that listed two coverage kinds now list three. Seven added. The cap test moved from 64 to 256; a new test asserts the marker carries a real drop count and that nothing follows it; an inverse test asserts an under-cap file carries no marker at all. For the new kind: a Python file whose root is ERROR is unusable, a file with a local parse failure stays partial, a clean file is neither, and — the one that matters most — the that last one ever flips, the Phase 2 refinement has stopped working. Full suite: 7732 passed, 28 failed, 7 skipped. The 28 are pre-existing agent-client install/uninstall failures in the cli suite, identical in count and identity at clean HEAD. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
…e grammar limit (#963) Phase 5. Four test groups, each checked RED before it was kept. - The Studio Export range join puts ONE ",+<N>" marker at the end with the summed drop count. A marker left mid-string makes every reader stop there and silently lose the ranges after it. Reaching the join through the pipeline needs an export file with 256+ error regions across two <Class> elements, so it goes through a test seam, following the pattern already in this repo (CBM_COVERAGE_MARKER_TEST_API). - check_index_coverage emits every range in front of a marker, never turns the marker's digits into a range, and reports "truncated" from BOTH caps — the producer's and its own 256 limit. - test_index_resilience now has a ceiling beside its floor: exactly one of the two fixture files is flagged, the clean neighbour is absent, and the range does not cover the whole file. - The three _Thread_local forms are pinned as measured. Only the array form fails today; the plan's Phase 0 also listed the pointer form, and that is wrong on the grammar shipped now. Also fixes 13 clang-format violations the earlier commits on this branch left in cbm.c, mcp.c and pass_definitions.c. `make -f Makefile.cbm lint-format` would have failed CI. The changes are whitespace only — the two reflowed tool descriptions concatenate byte-identically, so no output moved. Full suite: 7735 passed, 28 failed, 7 skipped. The 28 are the pre-existing cli install/uninstall failures, identical at clean HEAD. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
…#963) Review follow-up on this branch. Each parse_unusable entry carries one number, and the field was called "lines". That reads as the length of the file, and the two are not the same number: a grammar can end an error node past the last line, which this repo has already met — scripts/setup-windows.ps1 has 326 lines and its range ends at 327. A report whose whole thesis is honest reporting should not name that number after the wrong thing. "lines" also already means something else in this same response. Every search result carries a "lines" field holding a definition's line span. One word, two meanings, one document. The field is now "range_end", at both places that emit it — add_coverage_report reading the persisted rows, and add_parse_unusable_summary reading the per-run errors. The comment beside each one says the number can exceed the file, so the next reader does not have to rediscover it. Deriving the real file length instead was the other option and is not available here: neither cbm_file_error_t nor cbm_coverage_row_t carries it, only the path and the range string. One test, proved RED first — "range_end is NULL" against the old field name. It reads the end line from the persisted coverage row rather than a constant, so it states the property and not a measurement, and it asserts the old name is gone rather than kept beside the new one. index_resilience, parse_coverage and mcp: 283 passed, 4 skipped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
…le (#963) cbm_refine_regions_with_pp_lines asks cbm_line_is_toplevel_macro_call about every preprocessor-parsed line in a region. That check walked the source from byte 0 to find the line, so the refinement cost bytes times lines — on exactly the whole-file-ERROR C shape the refinement exists to narrow. Build one line-offset table per file, and hand the byte span to the macro check. The line-based form stays for the two callers that ask about a whole region, where a single walk is cheaper than a table. A file with no table (the allocation failed) still gets the old walk, so the answer never changes. Measured on one C file whose file-scope brace is split by an #ifndef, padded to 5k / 10k / 20k / 40k lines. Release build, one machine, seconds: lines main branch this commit 5000 0.98 1.14 0.98 10000 3.30 4.02 3.41 20000 11.64 14.77 12.56 40000 44.04 58.58 46.90 The branch ran 33% over main at 40k lines and now runs 6.5% over. The reported ranges do not change: 8-8, where main reports the whole file as 5-40000. A profile of the 40k run no longer reaches the macro check. What is left is cbm_line_contains and cbm_span_contains_callable_def, which main carries too and this branch does not touch. The fixture holds 40k definitions, which makes those two cost more here than on a smaller file. parse_coverage, extraction, pipeline, mcp and index_resilience: 964 passed, 4 skipped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
…no longer refuses `uninstall` failed with rc 1 on any macOS where Hermes is installed: Hermes is detected by its binary on PATH, not by its home, so a HOME without `.hermes/` (every test HOME, and a fresh user's) reached the Hermes uninstall ops with `~/.hermes/config.yaml` absent. The JSON removers already treat a missing config as "nothing to remove"; the YAML removers acquired the lock first, and the lock directory is created beside the target — with the parent directory absent that failed, the adapter recorded error: agent_config agent=Hermes op=pre_llm_hook_uninstall ... (target: does not exist or cannot be inspected) error: agent_config agent=Hermes op=mcp_uninstall ... and uninstall refused to remove the executable and the indexes. This is what made cli_uninstall_quiesces_active_cohort_before_removing_binary_and_index fail on such a machine (7837 passed, 1 failed on a pristine main). All four YAML removers short-circuit on a genuinely absent path and create nothing: lstat + ENOENT on POSIX, and CreateFileW with FILE_FLAG_OPEN_REPARSE_POINT on Windows, mirroring toml_read_file. Opening the reparse point rather than following it keeps a dangling symlink out of the short-circuit on both platforms, so it still reaches the reader and its byte-identical rejection. The regression test covers the four removers and runs on every platform: it asserts "nothing was created" through cbm_path_info_utf8 rather than lstat, so the Windows branch is exercised rather than compiled out. Signed-off-by: David Gil <dgilperez@gmail.com>
On Windows cbm_canonical_path answers in the platform's native backslash form, and the daemon stored that spelling verbatim for the client's session_root and allowed_root (cbm_mcp_server_set_session_context) and for the auto-index job's repo_path (application.c canonicalizes with the bare call at three sites). Every tool handler, however, normalizes separators after canonicalizing. One directory therefore had two names, and the exact comparison in application_index_args_equal never matched: an explicit index_repository request for the session root was refused as OPTIONS_CONFLICT instead of joining the auto-index job already running for it. Normalize the separator spelling of both session roots in set_session_context, and canonicalize plus normalize at the three application.c sites through one helper, application_canonical_root(). Separators fold on every platform, so the new test daemon_session_context_keeps_one_spelling_of_a_root binds wherever the suite runs; it fails on main with "C:\repos\cbm" != "C:/repos/cbm". Distilled from #1726 with co-author credit. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> Co-authored-by: 刘冲 <mail@liuchong.dev>
…any machine cli_uninstall_quiesces_active_cohort_before_removing_binary_and_index moved HOME and CBM_CACHE_DIR into a temporary directory but left PATH alone. Agent detection reads PATH first, so a real agent binary on the developer's machine was found even though HOME pointed at the empty fixture. Uninstall then tried to edit that agent's config file inside the fixture, failed because the fixture never created one, and stopped before removing the binary and the index. The test measures exactly that removal, so it failed. On this machine the agent is Goose, at /Users/<me>/go/bin/goose. Same tree and same built runner, only PATH differs: goose on PATH 291 passed, 1 failed goose off PATH 292 passed, 0 failed With this change the suite reports 292 passed, 0 failed while goose is still on PATH, so the fix is the redirect and not a quieter environment. This only stops the suite from depending on what the developer has installed. The abort it exposed is a real defect and is reported in #1954: one agent config that cannot be edited stops teardown for every other agent and for the executable and indexes. Eleven other tests that call cli_activation_save_env leave PATH open the same way. They pass today, so I left them alone rather than change what they exercise without evidence. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
…pelling the policy root The first cut on this branch (c1b9c45) respelled the session root and the allowed root with forward slashes on the way into the session policy (cbm_mcp_server_set_session_context) and at the three application.c sites that canonicalize a root. CI's Windows shard then failed two tests: tests/test_daemon_application.c:2256: ASSERT(sensitive_blocked) tests/test_daemon_application.c:2380: ASSERT(sensitive_blocked) Both set HOME to the cbm_canonical_path spelling of a directory - the platform's native form, backslashes on Windows - and expect a session rooted there to be refused for auto-index and for watch. The sensitive-root and allowed-root containment walk (ws_is_ancestor_or_equal in src/foundation/workspace.c) compares prefixes byte-exact, so a root that now read C:/Users/... no longer matched a HOME of C:\Users\...: the sensitive refusal was lost and $HOME was admitted. Respelling the policy root was the wrong layer. This commit keeps the policy roots in their canonical native spelling (mcp.c and the three application.c sites are back to what main runs) and folds the separator spelling at the one comparison the original defect lives in: application_index_args_equal, where the running job's args are compared with an explicit index_repository request's. The auto-index job spells repo_path the way the session policy holds it; the handler spells it with forward slashes. On Windows the two never matched, and the request was refused as OPTIONS_CONFLICT instead of joining the job already running for its root. The fold acts on the parsed copies only (yyjson_mut_obj_replace with a cbm_normalize_path_sep'd string), so nothing the daemon stores changes spelling and every other option stays exact. Why this is correct on Windows without a local Windows run: the policy spelling is untouched, so the containment path is byte-for-byte what main runs there today; cbm_normalize_path_sep folds backslashes on every platform, so the new fold is exercised by the new test wherever the suite runs. Tests: daemon_session_context_keeps_one_spelling_of_a_root asserted that the policy respells a root with forward slashes. That was the wrong contract: the policy's containment checks compare its spelling byte-exact against HOME and the granted roots, so a respelled root is exactly what stops matching them. It is replaced by daemon_session_context_keeps_the_policy_spelling_of_a_root, which binds the opposite: the policy stores the root as given. The comparison is exposed through application_internal.h as cbm_daemon_application_index_args_equal_for_test and bound by daemon_application_index_args_compare_repo_path_separator_equivalently: a backslash and a forward-slash repo_path compare equal; a different path, a sub-path, and a different mode still do not. Refs #1726 Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…mars Visual Basic 6 shares .frm with FORM and .cls with Apex/ObjectScript, so VB6 forms were parsed by the FORM grammar (0 defs) and class modules by the Apex grammar (junk nodes, extract timeouts). Recognise the VB6 export header (VERSION x.yy / VERSION 1.0 CLASS / Begin VB. / Attribute VB_Name) and report such files as unsupported (CBM_LANG_COUNT) until a Visual Basic grammar exists: new cbm_disambiguate_frm(), VB6 check added to cbm_disambiguate_cls() ahead of its ObjectScript scan. Defaults on any doubt are unchanged (FORM / Apex). Short-term step agreed in #721; the grammar follows separately. Refs #721 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Tokarzewski <bartlomiej.tokarzewski@gmail.com>
…-structure fix(extract): extract structure from Svelte/HTML/Astro inline scripts
…isolation test(cli): point PATH at the fixture so uninstall isolation holds on any machine
…tors fix(daemon): keep one separator spelling for the session root on Windows
…absent-config fix(cli): YAML removers answer OK for an absent config, so uninstall no longer refuses
…guation fix(discover): keep VB6 .frm/.cls files out of the FORM and Apex grammars
…t-ranges fix(coverage): narrow parse-error ranges, and stop the report hiding what it dropped (#963)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )