Skip to content

feat(discovery)!: match exclude patterns with picomatch under an owned glob grammar - #462

Merged
Vivswan merged 1 commit into
mainfrom
refactor/discovery-picomatch
Oct 5, 2026
Merged

Vivswan merged 1 commit into
mainfrom
refactor/discovery-picomatch

Conversation

@Vivswan

@Vivswan Vivswan commented Oct 5, 2026 •

Copy link
Copy Markdown
Owner

What this changes

The exclude discovery filter now matches through picomatch inside a grammar the input boundary owns. Documented forms keep their meaning. Anything else is refused by sentences that each name one fix, and one round yields acceptance.

pattern          today                      with picomatch
tmp-*            name starts with tmp-      same
octo/*           owner octo, any name       same
*                any name, .github too      same
tmp-?            literal "?" (none)         one character after tmp-
svc-[ab]         literal (none)             svc-a or svc-b
svc-[!ab]        literal (none)             svc- plus one other character
!svc-*           literal (none)             every name except svc-*
./tmp-**         literal (none)             refused: drop every leading "./", write one "*"
svc-(a)+  \d     literal (none)             refused: holds "(", ")", "+", remove them
[z-a]            literal (none)             refused: backwards range "z-a", write "[a-z]"
x/..  !!x        literal or refused         refused: never a name, "!" at 1 negates nothing
a/b/c  octo/     refused                    refused

How

  • One compile per pattern at parse time: compileExcludePattern keeps the text, the scope, and the regex, so an argv config equals an env config.
  • The boundary owns the grammar: name characters, one / at most, *, ?, [abc], [!abc], one leading !. Picomatch reads braces, quotes, and backslashes as regex, so anything else is refused.
  • A refusal is a program: a pipeline of steps, each reading only the view the earlier fixes leave, with a property test applying the fixes mechanically.
  • Linear for every input, the engine's refusal named in the same round.

Proof

  • tsc, biome, lint:arch, knip, build:check, merge-tree, smoke green.
  • 1174 targeted tests, 4 discovery e2e scenarios.
  • Independent codex review converged.
Technical details

Line accounting (git diff --numstat origin/main...HEAD)

kind                 +     -
source (src/)      397    30
tests              363    14
e2e scenarios      102     0
docs (hand)         12     1
generated           10     3
package.json         2     0
total              886    48
bun.lock (separate)  4     0

Dependency: picomatch 4.0.7 (runtime, exact), @types/picomatch 4.0.3 (dev, exact). The main entry with windows: false, since the extensionless picomatch/posix subpath fails Node ESM resolution for the externalized library (package smoke) and the main entry otherwise follows the host OS. literalBrackets: false, since picomatch otherwise emits (?:\[a\]|[a]) per class, an alternation no name can match that made 20,000 classes quadratic and reached the engine's size limit.

Deliberate readings

  • Refuse, never heal: the pattern is the user's workflow input (AGENTS.md hard rule).
  • The flaw pass is EXCLUDE_STEPS, a pipeline of (view) => Pass functions folded by judgeExcludePattern, the only scope holding the raw pattern. Order: trim, remove every stray character (listed), remove unpaired brackets (by position), remove a ! or ^ that neither negates nor opens a class (by position), per class in one rebuild (remove * and ?, delete an empty class, move a literal - last, turn a backwards range), drop every leading ./, collapse a star run. Then what the user has to write, named with no fix applied: the length cap, an empty body (ends the pass), the segment count, a dot-only side, and a regex the engine itself refuses.
  • One compile per pattern: the engine step compiles the fixed view, runs regex.test("") so V8's lazy compile surfaces there, and hands the regex out through the fold; compileExcludePattern keeps it. A test spies on picomatch.makeRe and sees one call for svc-[ab]; a clean result with no regex is a BUG: throw, since the engine step is last and runs whenever no earlier step ended the pass.
  • A position counts from 0 in the pattern as the earlier fixes leave it, and the message says so when one appears.
  • The class grammar allows - only as a range between two name characters or a literal first or last, so turning a range around can never pair a new one.
  • Every pass is linear: one indexOf walk collects classes, a flag array marks unpaired brackets, removal skips through a Set on code units. Worst inputs at 65,536 characters: [ runs 8 ms, ! runs 7 ms, ] runs 11 ms, 20,000 classes 48 ms under node including the engine probe.
  • The property test parses each refusal's sentences into appliers (trim, remove listed characters, remove at positions, drop ./, collapse stars, replace or delete a quoted class), applies them in order over 300 seeded patterns (seed 20261005, 36 pieces), and asserts acceptance or only remainder sentences the first round already named; a sentence with no applier throws. Named one-round cases include !!./tmp-**, x/.!, *[]*, []./x, ./ + 60,000 ?, and the over-cap inputs.
  • Builder controls, rerunnable with the test's own applyFix and oneRound: 4000 patterns with a 65,540-character piece (seed 20261005), 50,000 patterns over 96 pieces up to 12 per pattern (seed 0x9e3779b9), 50,000 over 104 pieces (seed 0xa57a1c21), 30,000 over 114 pieces with emoji and lone surrogates (seed 0x6a09e667). All clean.
  • The stray character and the displayed pattern go through quote (moved to src/text.ts), so a " or \ in the input stays distinguishable from the message's punctuation.
  • matchBase was rejected: under it o/* stops matching o/x. The scope split stays our code.
  • ExcludePattern stores picomatch's RegExp, not a closure, so toEqual holds across the argv and env paths.
  • One problem code, input-exclude-pattern-invalid { pattern, reason }.
  • Migration: docs/upgrading/v2-to-v3.md section 61 in six lines, since main sits at 1292 of the 1300-line cap.

Recorded, not built

  • The step type hides the raw pattern by lexical scope only: a step could still compile an unrelated literal and satisfy the type, and nothing but review rejects that. That is the structure's honest limit.
  • A solver proving a class can match a legal name: [.] or [!A-Za-z0-9_.-] compile correctly and match nothing.
  • Unicode in patterns: GitHub names are ASCII, so the alphabet rule refuses it with the character named.
  • A fuzz-generator roll for ? or ! patterns: the oracle supports them, the generator still rolls literal names.
  • An exact-text row for the engine refusal: bun's engine accepts what V8 refuses, so the case is a one-round row only.
  • Folding the default picomatch matching rows: the scope, case, leading-dot, and fast-path controls stay as rows.

Files: src/discovery/discover.ts, src/flows/input-readers.ts, src/flows/inputs.ts, src/problem.ts, src/text.ts, src/index.ts, test/discovery/discover.test.ts, test/e2e/oracle.ts, two e2e scenarios, docs/upgrading/v2-to-v3.md, docs/operate/multi-repo.md, docs/reference/library.md, regenerated action.yml and docs/reference/inputs.md.

BEGIN_COMMIT_OVERRIDE
feat(discovery)!: match exclude patterns with picomatch under an owned glob grammar

BREAKING CHANGE: In an exclude pattern, "?" now matches one character, "[abc]" and "[!abc]" are classes, and a leading "!" negates the pattern; each matched nothing before, so a pattern that relied on one of them matching literally is deleted and one that meant a glob is rewritten in that grammar. A leading "./" (drop it), a run of stars (write one "*"), surrounding whitespace from a library caller (trim it), a "." or ".." side, a second "/", a doubled "!", and any regex token ("(a)+", "\d", "{a..b}", quotes) fail the run naming the fix. Library callers supply exclude as ExcludePattern values from compileExcludePattern instead of strings.
END_COMMIT_OVERRIDE

Copilot AI balanced review requested due to automatic review settings October 5, 2026 05:38

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The implementation, validation boundary, public exports, documentation, and test coverage are consistent and complete.

Review effort: Balanced
Findings: None

What changed in this PR

Adds validated picomatch-based glob filtering to repository discovery while preserving existing documented behavior.

Changes:

  • Compiles and validates exclude patterns at input parsing.
  • Adds ?, character classes, and negation support.
  • Updates tests, e2e scenarios, documentation, and dependencies.
File Description
src/​discovery/​discover.ts Implements validated compiled glob matching.
src/​flows/​inputs.ts Compiles patterns during input parsing.
src/​problem.ts Adds detailed invalid-pattern errors.
src/​index.ts Exports the new public API.
test/​discovery/​discover.test.ts Tests grammar and matching semantics.
test/​action/​run.test.ts Tests action-level validation errors.
test/​cli/​inputs.test.ts Adapts CLI input assertions.
test/​problem.test.ts Updates problem-message specimens.
test/​private.test.ts Records the reviewed export surface.
test/​e2e/​oracle.ts Supports ? and negation independently.
test/​e2e/​scenarios/​multi-discovery-exclude.yml Covers scoped and single-character globs.
test/​e2e/​scenarios/​multi-discovery-exclude-negation.yml Covers negated exclusion.
docs/​operate/​multi-repo.md Documents the owned glob grammar.
docs/​reference/​library.md Documents new library exports.
docs/​reference/​inputs.md Updates generated input reference.
action.yml Updates generated action input help.
package.json Adds picomatch and its types.
bun.lock Locks the added dependencies.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

File size check

0 over a hard cap (fails), 11 warning(s).

File Size Tier Cap
.github/scripts/gen-inputs-table.ts:54 11 comment lines warn 10
.github/scripts/release-pipeline.ts:1 34 comment lines (header) warn 25
.github/scripts/release-pipeline.ts:453 14 comment lines warn 10
docs/upgrading/v2-to-v3.md 1298 lines warn 1040
src/discovery/discover.ts:91 11 comment lines warn 10
src/github/secret-scan.ts:45 11 comment lines warn 10
src/sections/contract/errors.ts:13 14 comment lines warn 10
src/sections/shared/roles.ts:43 13 comment lines warn 10
test/e2e/generators.ts 2634 lines warn 2560
test/e2e/generators.ts:1667 12 comment lines warn 10
test/sections/refusal-messages.test.ts:61 19 comment lines warn 10

Split the file, wrap the line, shorten or exempt the comment, or list the path in .file-size-allow.local with a # reason.

5 managed file(s) skipped; repo-platform owns them.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unescaped pattern values can produce ambiguous diagnostics and multiline healing notices.

Review effort: Balanced
Findings: 1 Medium severity · 2 Low severity

Open (3)

Comment thread src/discovery/discover.ts Resolved
Comment thread src/discovery/discover.ts Resolved
Comment thread src/problem.ts Resolved
Copilot AI balanced review requested due to automatic review settings October 5, 2026 06:56
@Vivswan
Vivswan force-pushed the refactor/discovery-picomatch branch from 2320431 to d8dad6e Compare October 5, 2026 06:56

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Validation omits some simultaneous flaws, refuses a required whitespace repair, and produces ambiguous quoted diagnostics.

Review effort: Balanced
Findings: 1 Medium severity · 2 Low severity

Open (3)
Resolved since last review (1)

Comment thread src/discovery/discover.ts Resolved
Copilot AI balanced review requested due to automatic review settings October 5, 2026 07:18
@Vivswan
Vivswan force-pushed the refactor/discovery-picomatch branch from d8dad6e to 07b31db Compare October 5, 2026 07:18

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The implementation, public API, documentation, and comprehensive tests consistently enforce the declared glob grammar.

Review effort: Balanced
Findings: 2 Low severity

Open (2)
Resolved since last review (1)

Copilot AI balanced review requested due to automatic review settings October 5, 2026 07:58
@Vivswan
Vivswan force-pushed the refactor/discovery-picomatch branch from 07b31db to 8cdf561 Compare October 5, 2026 07:58

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The validation, matching, public API, documentation, and comprehensive tests are consistent with the stated breaking change.

Review effort: Balanced
Findings: None

Resolved since last review (2)

Copilot AI balanced review requested due to automatic review settings October 5, 2026 08:26
@Vivswan
Vivswan force-pushed the refactor/discovery-picomatch branch from 8cdf561 to 4017d15 Compare October 5, 2026 08:26
Comment thread src/discovery/discover.ts Fixed
Comment thread src/discovery/discover.ts Fixed
Comment thread src/discovery/discover.ts Fixed

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Maximum-length malformed patterns can trigger quadratic validation work before producing an error.

Review effort: Balanced
Findings: 1 High severity

Open (1)

Comment thread src/discovery/discover.ts Resolved
Copilot AI balanced review requested due to automatic review settings October 5, 2026 08:52
@Vivswan
Vivswan force-pushed the refactor/discovery-picomatch branch from 4017d15 to 58a4d99 Compare October 5, 2026 08:52

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Accepted patterns are compiled twice despite the stated one-compile-per-pattern contract.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
Resolved since last review (1)

Comment thread src/discovery/discover.ts Resolved
Copilot AI balanced review requested due to automatic review settings October 5, 2026 09:06
@Vivswan
Vivswan force-pushed the refactor/discovery-picomatch branch from 58a4d99 to d7709af Compare October 5, 2026 09:06

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The parser accepts a provably non-matching class, and its error grammar omits one supported class form.

Review effort: Balanced
Findings: 2 Low severity

Open (2)
Resolved since last review (1)

Comment thread src/discovery/discover.ts Resolved
Comment thread src/problem.ts Resolved
Copilot AI balanced review requested due to automatic review settings October 5, 2026 09:17
@Vivswan
Vivswan force-pushed the refactor/discovery-picomatch branch from d7709af to fbdb52e Compare October 5, 2026 09:17

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The matcher change is consistently implemented, documented, and covered by focused unit and end-to-end tests.

Review effort: Balanced
Findings: None

Resolved since last review (2)

@Vivswan
Vivswan marked this pull request as ready for review October 5, 2026 09:26
@Vivswan Vivswan added the merge-when-green Owner approved: merge once every gate is green label Oct 5, 2026
@Vivswan Vivswan closed this Oct 5, 2026
@Vivswan Vivswan reopened this Oct 5, 2026
…d glob grammar

The exclude discovery filter compiled its own RegExp from each pattern, with "*" the only operator and every other character literal. Each pattern is now compiled once at the input boundary with picomatch (the posix entry, nocase, dot, posix classes, no fast paths, debug), and the boundary owns the grammar: name characters, at most one "/", "*", "?", "[abc]", "[!abc]" or "[^abc]", and one leading "!". Anything else is refused when the inputs are parsed, naming the written pattern, the reason, and the fix, because the pattern is the user's input and because picomatch reads braces, parentheses, quotes, backslashes, a leading "./", and a run of stars as regex syntax through paths no option closes, compiling a wrong or never-matching filter silently.

DiscoveryFilters.exclude carries the compiled ExcludePattern (pattern, scope, regex) instead of strings, so a config built from argv still equals one built from the environment, and compileExcludePattern with ExcludePattern join the library surface. The e2e oracle's independent matcher learns "?" and a leading "!", and two scenarios pin the new forms end to end. The upgrading guide gains section 61.

BREAKING CHANGE: In an exclude pattern, "?" now matches one character, "[abc]" and "[!abc]" are classes, and a leading "!" negates the pattern; each matched nothing before, so a pattern that relied on one of them matching literally is deleted and one that meant a glob is rewritten in that grammar. A leading "./" (drop it), a run of stars (write one "*"), surrounding whitespace from a library caller (trim it), a "." or ".." side, a second "/", a doubled "!", and any regex token ("(a)+", "\d", "{a..b}", quotes) fail the run naming the fix. Library callers supply exclude as ExcludePattern values from compileExcludePattern instead of strings.
@Vivswan
Vivswan force-pushed the refactor/discovery-picomatch branch from fbdb52e to 474dc3b Compare October 5, 2026 10:58
@Vivswan Vivswan closed this Oct 5, 2026
@Vivswan Vivswan reopened this Oct 5, 2026
@Vivswan
Vivswan merged commit 836f6cb into main Oct 5, 2026
51 of 53 checks passed
@Vivswan
Vivswan deleted the refactor/discovery-picomatch branch October 5, 2026 12:39
@Vivswan
Vivswan restored the refactor/discovery-picomatch branch October 5, 2026 12:42
@Vivswan
Vivswan deleted the refactor/discovery-picomatch branch October 5, 2026 12:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-when-green Owner approved: merge once every gate is green

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants