Skip to content

Commit a953509

Browse files
fix(coverage): stop the report hiding what it dropped, and name whole-file failures (DeusData#963)
Two silent failures in the parse-coverage report, both made visible by the Phase 2 range refinement that came before this. Two caps sat in series and both returned early without saying anything: CBM_MAX_ERROR_REGIONS = 64 internal/cbm/cbm.c COVERAGE_RANGE_MAX = 128 src/mcp/mcp.c Raising only the first would have moved the clip from 64 to 128, so both move to 256. This was live behaviour, not a theoretical limit: after Phase 2 split one whole-file range into many small ones, src/cli/cli.c and tests/test_cli.c both reported exactly 64 ranges — the cap binding, dead-on, twice. Every coverage figure measured before this change was a floor. With the cap at 256 the true numbers are cli.c 13.9% (not 9.8%) and test_cli.c 3.1%, and the longest list in the repo is 85 ranges. A raised cap is still a cap, so the report now says when it clipped: - cbm_error_regions_t gained a `dropped` counter, and cbm_collect_error_regions walks to the end instead of stopping at the cap, so the count is exact rather than a lower bound. That costs little — the walk never descends into an ERROR subtree. - cbm_error_ranges_str appends ",+<N>" when N ranges were thrown away. - coverage_add_ranges reads that marker and sets "truncated": true, and also sets it when its own limit stops the loop. Before this the marker was invisible: the parser stopped at the '+' with no error and no leftover, so a clipped list arrived looking complete. - objectscript_export_append_error_ranges strips markers off both operands before joining two Studio Export parts and adds one back at the end. A marker left mid-string would make every reader stop there and silently lose every range after it. "Look at lines 1 to 13047" of a 13046-line file tells a reader nothing. Those files now carry their own kind rather than being described as partially covered. New `parse_unusable` field in CBMFileResult, set when one range covers 80% or more of the file. Its customers are non-C languages: the Phase 2 refinement that narrows a whole-file range using the preprocessed parse only runs for C, C++ and CUDA, so a Python, Java, Ruby or TypeScript file whose root node is ERROR still reports 1-N. Verified against real files in all four. The kind is `parse_unusable`, not `parse_failed`. index_coverage.kind already means one of two things — indexed-but-partial, or a skip phase saying the file was never indexed at all — and `parse_failed` reads as the second when it is the first. The store.c schema comment, which is the only written record of this vocabulary, now describes all three classes and says why. Two places would have mislabelled the new kind as "skipped", which is exactly that confusion: coverage_status fell through to its catch-all pass, and add_coverage_report fell into its else branch. A reader who finds a file under "skipped" believes it is absent from the graph, when it was indexed. Both now have explicit branches. index_status gained parse_unusable_count so a CI gate can read it without parsing anything else, get_code_snippet says "read the source directly" instead of naming useless ranges, and the three tool descriptions that listed two coverage kinds now list three. Seven added. The cap test moved from 64 to 256; a new test asserts the marker carries a real drop count and that nothing follows it; an inverse test asserts an under-cap file carries no marker at all. For the new kind: a Python file whose root is ERROR is unusable, a file with a local parse failure stays partial, a clean file is neither, and — the one that matters most — the that last one ever flips, the Phase 2 refinement has stopped working. Full suite: 7732 passed, 28 failed, 7 skipped. The 28 are pre-existing agent-client install/uninstall failures in the cli suite, identical in count and identity at clean HEAD. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Joshua Richter <jrichter5781@gmail.com>
1 parent cb167b9 commit a953509

7 files changed

Lines changed: 442 additions & 45 deletions

File tree

‎internal/cbm/cbm.c‎

Lines changed: 72 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -778,16 +778,24 @@ static bool cbm_source_nesting_exceeds(const char *source, int source_len, int c
778778
* nodes (does not descend into an error subtree — one range per failed region).
779779
* Bounded by CBM_MAX_ERROR_REGIONS so pathological input can't blow up the
780780
* output. The ranges mark where constructs were dropped; they are a detection
781-
* aid, never a completeness proof. */
782-
#define CBM_MAX_ERROR_REGIONS 64
781+
* aid, never a completeness proof.
782+
*
783+
* `dropped` counts the ranges the cap threw away. It exists so a clipped list
784+
* cannot read as a complete one: cbm_error_ranges_str turns a non-zero count
785+
* into a trailing "+<N>" marker. Phase 2 split one whole-file range into many
786+
* small ones, which pushed real files straight into a cap that used to be
787+
* unreachable, so the clip is live behaviour and not a theoretical limit. */
788+
#define CBM_MAX_ERROR_REGIONS 256
783789
typedef struct {
784790
uint32_t starts[CBM_MAX_ERROR_REGIONS];
785791
uint32_t ends[CBM_MAX_ERROR_REGIONS];
786792
int count;
793+
int dropped;
787794
} cbm_error_regions_t;
788795

789796
static void cbm_error_regions_push(cbm_error_regions_t *acc, TSNode n) {
790797
if (acc->count >= CBM_MAX_ERROR_REGIONS) {
798+
acc->dropped++;
791799
return;
792800
}
793801
acc->starts[acc->count] = ts_node_start_point(n).row + 1;
@@ -851,13 +859,14 @@ static bool cbm_is_eof_terminator_miss(TSNode n, const char *source, int source_
851859
return true;
852860
}
853861

862+
/* Walks to the end even after the cap is full, so `dropped` is the real number
863+
* of ranges lost rather than a lower bound. This costs little: the walk never
864+
* descends into an ERROR subtree — it records the top-most node and moves on —
865+
* so it only visits the spine of nodes that contain an error, plus one level. */
854866
static void cbm_collect_error_regions(TSNode n, cbm_error_regions_t *acc, const char *source,
855867
int source_len) {
856-
if (acc->count >= CBM_MAX_ERROR_REGIONS) {
857-
return;
858-
}
859868
uint32_t k = ts_node_child_count(n);
860-
for (uint32_t i = 0; i < k && acc->count < CBM_MAX_ERROR_REGIONS; i++) {
869+
for (uint32_t i = 0; i < k; i++) {
861870
TSNode c = ts_node_child(n, i);
862871
if (ts_node_is_missing(c) || strcmp(ts_node_type(c), "ERROR") == 0) {
863872
if (cbm_is_eof_terminator_miss(c, source, source_len)) {
@@ -1298,7 +1307,11 @@ static void cbm_push_trimmed_run(cbm_error_regions_t *out, uint32_t start, uint3
12981307
while (end >= start && end <= line_count && (map[end] & CBM_LINE_NO_CODE)) {
12991308
end--;
13001309
}
1301-
if (start > end || out->count >= CBM_MAX_ERROR_REGIONS) {
1310+
if (start > end) {
1311+
return; /* nothing but blank, comment or directive lines — no construct lost */
1312+
}
1313+
if (out->count >= CBM_MAX_ERROR_REGIONS) {
1314+
out->dropped++;
13021315
return;
13031316
}
13041317
out->starts[out->count] = start;
@@ -1330,7 +1343,7 @@ static bool cbm_line_is_toplevel_macro_call(const char *src, int src_len, uint32
13301343
static void cbm_refine_regions_with_pp_lines(cbm_error_regions_t *regs, const uint8_t *map,
13311344
uint32_t line_count, const char *src, int src_len,
13321345
const CBMDefArray *defs) {
1333-
cbm_error_regions_t out = {{0}, {0}, 0};
1346+
cbm_error_regions_t out = {{0}, {0}, 0, regs->dropped};
13341347
for (int i = 0; i < regs->count; i++) {
13351348
uint32_t run_start = 0;
13361349
uint32_t run_end = 0;
@@ -1357,12 +1370,40 @@ static void cbm_refine_regions_with_pp_lines(cbm_error_regions_t *regs, const ui
13571370
}
13581371

13591372
/* Serialize collected regions as "start-end,start-end,..." into the arena. */
1373+
/* Share of a file one range must cover before the range stops being advice and
1374+
* becomes noise. 80% is well clear of anything real: the widest single range in
1375+
* this repo covers 25.5% of its file, and the next widest 3.9%. */
1376+
#define CBM_UNUSABLE_PCT 80
1377+
1378+
/* Number of 1-based lines in `src`. A file that does not end with a newline
1379+
* still has a last line, so the count is separators plus one. */
1380+
static uint32_t cbm_count_lines(const char *src, int src_len) {
1381+
uint32_t n = 1;
1382+
for (int i = 0; i < src_len; i++) {
1383+
if (src[i] == '\n' && i + 1 < src_len) {
1384+
n++;
1385+
}
1386+
}
1387+
return n;
1388+
}
1389+
1390+
/* Serialize collected regions as "start-end,start-end,...", with a trailing
1391+
* ",+<N>" when the cap threw N ranges away.
1392+
*
1393+
* The marker must stay a SUFFIX and nothing else. Every reader stops at the
1394+
* first token that is not a range, so a marker in the middle of a string
1395+
* silently hides everything after it. objectscript_export_append_error_ranges
1396+
* strips markers before joining two parts for exactly that reason.
1397+
*
1398+
* N can be non-zero while the kept list is short, because the recovery and
1399+
* macro rules run after collection and remove ranges the cap never saw. That
1400+
* still reports honestly: the cap bound, so what was lost is unknown. */
13601401
static const char *cbm_error_ranges_str(CBMArena *a, const cbm_error_regions_t *regs) {
1361-
if (regs->count <= 0) {
1402+
if (regs->count <= 0 && regs->dropped <= 0) {
13621403
return NULL;
13631404
}
13641405
enum { RANGE_MAX = 24 }; /* "4294967295-4294967295," */
1365-
char *buf = (char *)cbm_arena_alloc(a, (size_t)regs->count * RANGE_MAX);
1406+
char *buf = (char *)cbm_arena_alloc(a, (size_t)(regs->count + 1) * RANGE_MAX);
13661407
if (!buf) {
13671408
return NULL;
13681409
}
@@ -1371,6 +1412,9 @@ static const char *cbm_error_ranges_str(CBMArena *a, const cbm_error_regions_t *
13711412
off += (size_t)snprintf(buf + off, RANGE_MAX, "%s%u-%u", i ? "," : "", regs->starts[i],
13721413
regs->ends[i]);
13731414
}
1415+
if (regs->dropped > 0) {
1416+
snprintf(buf + off, RANGE_MAX, "%s+%d", off ? "," : "", regs->dropped);
1417+
}
13741418
return buf;
13751419
}
13761420

@@ -1706,7 +1750,7 @@ static CBMFileResult *extract_file_ex_body(const char *source, int source_len, C
17061750
* the raw source line, and whose QN the raw pass did not
17071751
* already extract. */
17081752
if (ts_node_has_error(root)) {
1709-
cbm_error_regions_t raw_regs = {{0}, {0}, 0};
1753+
cbm_error_regions_t raw_regs = {{0}, {0}, 0, 0};
17101754
cbm_collect_error_regions(root, &raw_regs, source, source_len);
17111755
if (raw_regs.count > 0) {
17121756
int defs_before = result->defs.count;
@@ -1915,7 +1959,7 @@ static CBMFileResult *extract_file_ex_body(const char *source, int source_len, C
19151959
* miss, and a fully recovered file is not flagged at all. Detection aid
19161960
* only: the absence of this flag is NOT a completeness guarantee. */
19171961
if (ts_node_has_error(root)) {
1918-
cbm_error_regions_t regs = {{0}, {0}, 0};
1962+
cbm_error_regions_t regs = {{0}, {0}, 0, 0};
19191963
if (strcmp(ts_node_type(root), "ERROR") == 0) {
19201964
cbm_error_regions_push(&regs, root); /* whole file unparseable */
19211965
} else {
@@ -1940,10 +1984,25 @@ static CBMFileResult *extract_file_ex_body(const char *source, int source_len, C
19401984
* refinement, because its evidence is per-line: a narrow range points at
19411985
* the call itself instead of the whole blob around it. */
19421986
cbm_subtract_macro_invocation_regions(&regs, &result->defs, source, source_len);
1943-
if (regs.count > 0) {
1987+
/* A file whose kept list is empty but whose cap still bound is NOT clean:
1988+
* the ranges the cap threw away were never judged by the two rules
1989+
* above, so nothing proves they were recovered. Flag it. */
1990+
if (regs.count > 0 || regs.dropped > 0) {
19441991
result->parse_incomplete = true;
19451992
result->error_region_count = regs.count;
19461993
result->error_ranges = cbm_error_ranges_str(a, &regs);
1994+
/* One range covering nearly the whole file is not advice, it is
1995+
* noise: "look at lines 1 to 13047" of a 13046-line file tells a
1996+
* reader nothing they did not already know. Mark those separately
1997+
* so the report can say "read the source" instead. See
1998+
* parse_unusable in cbm.h for which files land here and why. */
1999+
if (regs.count == 1 && regs.dropped == 0) {
2000+
uint32_t total = cbm_count_lines(source, source_len);
2001+
uint32_t span = regs.ends[0] - regs.starts[0] + 1;
2002+
if (total > 0 && span * 100 >= total * CBM_UNUSABLE_PCT) {
2003+
result->parse_unusable = true;
2004+
}
2005+
}
19472006
}
19482007
}
19492008

‎internal/cbm/cbm.h‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -522,6 +522,21 @@ typedef struct CBMFileResult {
522522
* completeness guarantee. Callers should treat a flagged file as "prefer
523523
* grep here", never treat an unflagged file as provably complete. */
524524
bool parse_incomplete;
525+
/* True when the ranges cover so much of the file that they are no longer
526+
* useful advice — one range over 80% of the line count. The file WAS
527+
* indexed, but pointing a reader at almost every line tells them nothing,
528+
* so the report says "read the source" instead of listing the range.
529+
*
530+
* Its main customers are non-C languages. The refinement that narrows a
531+
* whole-file range using the preprocessed parse only runs for C, C++ and
532+
* CUDA, so a Python, Java or Ruby file whose root node is ERROR still
533+
* reports 1-N.
534+
*
535+
* Note the naming: this field and the phase string it produces are both
536+
* `parse_unusable`. The older `parse_incomplete` field emits the phase
537+
* `parse_partial` instead. That mismatch is historical, not deliberate —
538+
* do not copy it. */
539+
bool parse_unusable;
525540
const char *error_ranges;
526541
int error_region_count;
527542
bool is_test_file;

0 commit comments

Comments
 (0)