Skip to content

Commit 07b31db

Browse files
committed
feat(discovery)!: match exclude patterns with picomatch under an owned glob grammar
The exclude discovery filter compiled its own RegExp from each pattern, with "*" the only operator and every other character literal. Each pattern is now compiled once at the input boundary with picomatch (the posix entry, nocase, dot, posix classes, no fast paths, debug), and the boundary owns the grammar: name characters, at most one "/", "*", "?", "[abc]", "[!abc]" or "[^abc]", and one leading "!". Anything else is refused when the inputs are parsed, naming the written pattern, the reason, and the fix, because the pattern is the user's input and because picomatch reads braces, parentheses, quotes, backslashes, a leading "./", and a run of stars as regex syntax through paths no option closes, compiling a wrong or never-matching filter silently. DiscoveryFilters.exclude carries the compiled ExcludePattern (pattern, scope, regex) instead of strings, so a config built from argv still equals one built from the environment, and compileExcludePattern with ExcludePattern join the library surface. The e2e oracle's independent matcher learns "?" and a leading "!", and two scenarios pin the new forms end to end. The upgrading guide gains section 61. BREAKING CHANGE: In an exclude pattern, "?" now matches one character, "[abc]" and "[!abc]" are classes, and a leading "!" negates the pattern; each matched nothing before, so a pattern that relied on one of them matching literally is deleted and one that meant a glob is rewritten in that grammar. A leading "./" (drop it), a run of stars (write one "*"), surrounding whitespace from a library caller (trim it), a "." or ".." side, a second "/", a doubled "!", and any regex token ("(a)+", "\d", "{a..b}", quotes) fail the run naming the fix. Library callers supply exclude as ExcludePattern values from compileExcludePattern instead of strings.
1 parent 1b3b744 commit 07b31db

20 files changed

Lines changed: 467 additions & 41 deletions

‎action.yml‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -146,8 +146,15 @@ inputs:
146146
default: ""
147147
exclude:
148148
description: >-
149-
Comma- or newline-separated wildcard patterns removing repositories from
150-
repos: "*" discovery.
149+
Comma- or newline-separated glob patterns removing repositories from
150+
repos: "*" discovery, compared case-insensitively against owner/name
151+
when the pattern has a "/" and against the name alone otherwise. A star
152+
matches any run of characters, "?" one character, "[abc]" one of a set,
153+
"[!abc]" or "[^abc]" one outside it, and one leading "!" negates the
154+
pattern. Beyond those a pattern holds only letters, digits, ".", "-",
155+
and "_", with at most one "/", nothing empty around it, and no side that
156+
is just "." or ".."; anything else (a leading "./", a run of stars, a
157+
regex token such as "(a)+") fails the run naming the fix.
151158
required: false
152159
default: ""
153160
topics:

‎bun.lock‎

Lines changed: 4 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎docs/operate/multi-repo.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,10 @@ Discovery takes six filter inputs that apply only to `repos: "*"`; setting any o
5757
- `archived` defaults to `skip`, because settings writes fail on archived repositories; `archived: only` is mostly useful with `mode: check`.
5858
- `forks` includes, excludes, or keeps only forks.
5959
- `topics` keeps repositories carrying at least one listed topic, so a single marker topic can opt repositories in. It is unrelated to the `topics` settings section.
60-
- `exclude` takes wildcard patterns where `*` matches anything: a pattern containing `/` is matched against the full `owner/name`, any other against the name alone, case-insensitively.
60+
- `exclude` takes glob patterns, matched case-insensitively: a pattern containing `/` against the full `owner/name`, any other against the name alone.
61+
- `*` matches any run of characters, `?` one character, `[abc]` one of a set, `[!abc]` or `[^abc]` one outside it.
62+
- One leading `!` negates the pattern, so `!svc-*` drops everything except the services.
63+
- Beyond those operators a pattern holds only name characters (letters, digits, `.`, `-`, `_`), at most one `/` with a non-empty glob on each side, and no side that is just `.` or `..`. Anything else (a leading `./`, a run of stars, a brace, a parenthesis, a backslash, a quote, a second `/`, a class holding `*`) fails the run before discovery starts, naming the fix, as does a class like `[z-a]`. The matcher is [picomatch](https://github.com/micromatch/picomatch), confined to that grammar.
6164
- `affiliation` selects which relationships to the token's user qualify, passed to GitHub's `/user/repos` listing: `owner` (the default), `collaborator`, or `organization_member`. The list replaces the default, so widening discovery beyond owned repositories takes `owner,collaborator`.
6265

6366
## Fallback for repositories without a settings file

‎docs/reference/inputs.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ A list input (`sections`, `required-sections`, `repos`, `exclude`, `topics`, `af
3838
| `visibility` | <p>Keeps only repositories of this visibility in repos: "*" discovery: all (default), public, private, or internal.</p> | `false` | `""` |
3939
| `archived` | <p>Archived-repository policy for repos: "*" discovery: skip (default), include, or only.</p> | `false` | `""` |
4040
| `forks` | <p>Fork policy for repos: "*" discovery: include (default), exclude, or only.</p> | `false` | `""` |
41-
| `exclude` | <p>Comma- or newline-separated wildcard patterns removing repositories from repos: "*" discovery.</p> | `false` | `""` |
41+
| `exclude` | <p>Comma- or newline-separated glob patterns removing repositories from repos: "*" discovery, compared case-insensitively against owner/name when the pattern has a "/" and against the name alone otherwise. A star matches any run of characters, "?" one character, "[abc]" one of a set, "[!abc]" or "[^abc]" one outside it, and one leading "!" negates the pattern. Beyond those a pattern holds only letters, digits, ".", "-", and "_", with at most one "/", nothing empty around it, and no side that is just "." or ".."; anything else (a leading "./", a run of stars, a regex token such as "(a)+") fails the run naming the fix.</p> | `false` | `""` |
4242
| `topics` | <p>Comma- or newline-separated topics, of which repos: "*" discovery keeps the repositories carrying at least one.</p> | `false` | `""` |
4343
| `affiliation` | <p>Comma- or newline-separated affiliations for repos: "*" discovery: owner (default), collaborator, or organization_member.</p> | `false` | `""` |
4444
<!-- action-docs-inputs source="action.yml" -->

‎docs/reference/library.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -246,6 +246,8 @@ const renderExitCode = runRender(
246246
| `discoverRepos` | function | The repositories a token can see, filtered |
247247
| `DiscoveryFilters` | type | The filters: `visibility`, `archived`, `forks`, `affiliation`, `topics`, `exclude` |
248248
| `DEFAULT_DISCOVERY_FILTERS` | const | The action's defaults |
249+
| `compileExcludePattern` | function | One `exclude` glob compiled once, or the problem naming why it is unusable and the fix |
250+
| `ExcludePattern` | type | A compiled `exclude` entry: the operator's text, the scope its slash chose, and the regex |
249251
| `parseReposInput` | function | The `repos` input form: slugs, or `"*"` for discovery |
250252
| `resolveCentralTargets` | function | The `<owner>/<name>.yml` files of a repos-dir as targets, with warnings for the files it skipped |
251253
| `dedupeTargets` | function | Merge the central and the remote targets, central first; a remote target whose slug a central one already names is dropped with a notice |

‎docs/upgrading/v2-to-v3.md‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1281,6 +1281,38 @@ v3 tryList("/repos/o/r/labels", { perPage: 100 }) // re
12811281

12821282
Fix: add `tryList` to every `GitHubClient` double: answer the pages your `tryRequest` route table would (`per_page=N&page=K` from 1, until a page is short) and return their bodies as `{ data: [...] }`; a client over another transport returns each page's body as received, and honors `until` by stopping after the first page it accepts.
12831283

1284+
## 61. Discovery: `exclude` patterns are globs under an owned grammar
1285+
1286+
For every `repos: "*"` workflow that sets `exclude`, and for `@vivswan/github-settings-as-code` consumers building `DiscoveryFilters`. In v2 an `exclude` pattern knew one operator, `*`, and matched every other character literally, so a `?`, a `[`, or a leading `!` could only ever match nothing. In v3 the pattern is a glob matched by picomatch inside a grammar the input boundary owns: `*` any run of characters, `?` one character, `[abc]` one of a set, `[!abc]` or `[^abc]` one outside it, one leading `!` negating the whole pattern, and beyond those only name characters (letters, digits, `.`, `-`, `_`) with at most one `/`. The scope rule is unchanged: a pattern with `/` matches `owner/name`, any other the name alone, case-insensitively.
1287+
1288+
Two kinds of pattern move:
1289+
1290+
- A pattern that gains glob meaning: `tmp-?`, `svc-[ab]`, `!svc-*`. Each matched nothing in v2; in v3 `!svc-*` excludes every repository that is not a service.
1291+
- A pattern outside the grammar fails the run before discovery starts, naming the fix: a leading `./` (drop it), a run of stars (write one `*`), a `.` or `..` side (`x/..`), a second `/`, a doubled `!!`, or a regex token (`(a)+`, `\d`, `{a..b}`, a quote). The pattern is your input, so it is refused, never repaired.
1292+
1293+
```text
1294+
v2 exclude: "!svc-*" -> matched nothing, every repository discovered
1295+
v3 exclude: "!svc-*" -> excludes every repository whose name is not svc-*
1296+
notice: repos: "*" discovery skipped 2 repositories by exclude pattern "!svc-*": o/tooling, o/website
1297+
1298+
v2 exclude: "./tmp-**" -> matched nothing
1299+
v3 exclude: "./tmp-**" -> error: the "exclude" input pattern "./tmp-**" is not a usable glob: it starts with "./", which names no owner, so drop every leading "./", and it holds a run of stars, which matches no more than one "*" does, so write one "*". Write ...
1300+
1301+
v3 exclude: "svc-(a)+" -> error: the "exclude" input pattern "svc-(a)+" is not a usable glob: it holds "(", ")", "+", which no owner or repository name contains, so remove them. Write ...
1302+
```
1303+
1304+
Fix: delete a pattern that relied on `?`, `[`, `]`, or `!` matching literally, since it never matched a repository; rewrite one that meant a glob in the grammar above, following the fix each refusal names.
1305+
1306+
For library callers, `DiscoveryFilters.exclude` takes compiled `ExcludePattern` values instead of strings: run each pattern through `compileExcludePattern` at your input boundary.
1307+
1308+
```text
1309+
v2 discoverRepos(client, { ...DEFAULT_DISCOVERY_FILTERS, exclude: ["tmp-*"] });
1310+
1311+
v3 const exclude = compileExcludePattern("tmp-*");
1312+
if (exclude.isErr()) throw new Error(describeProblem(exclude.error));
1313+
discoverRepos(client, { ...DEFAULT_DISCOVERY_FILTERS, exclude: [exclude.value] });
1314+
```
1315+
12841316
## Order of operations
12851317

12861318
1. Rename any settings file whose path contains a comma, rename `undeclared` to `_undeclared` in every settings file, and move every other underscore key into a YAML comment; the v2 line accepts the old spellings only, so do all three together with the pin move.

‎package.json‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,7 @@
7777
"consola": "3.4.2",
7878
"neverthrow": "8.2.0",
7979
"picocolors": "1.1.1",
80+
"picomatch": "4.0.7",
8081
"yaml": "2.9.1",
8182
"zod": "4.6.5"
8283
},
@@ -90,6 +91,7 @@
9091
"@octokit/openapi-webhooks": "12.1.0",
9192
"@types/bun": "1.4.2",
9293
"@types/node": "26.6.4",
94+
"@types/picomatch": "4.0.3",
9395
"action-docs": "2.5.1",
9496
"ajv": "8.20.0",
9597
"ajv-formats": "3.0.1",

‎src/discovery/discover.ts‎

Lines changed: 126 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,8 @@
55
*/
66

77
import type { components } from "@octokit/openapi-types";
8-
import { err, ok, ResultAsync } from "neverthrow";
8+
import { err, ok, type Result, ResultAsync } from "neverthrow";
9+
import picomatch from "picomatch/posix";
910
import type { GitHubClient } from "../github/api.js";
1011
import { classifyApiError } from "../github/api-error.js";
1112
import { classifyVisibility } from "../github/repo-visibility.js";
@@ -63,7 +64,7 @@ export interface DiscoveryFilters {
6364
forks: (typeof FORKS_FILTERS)[number];
6465
affiliation: string[];
6566
topics: string[];
66-
exclude: string[];
67+
exclude: ExcludePattern[];
6768
}
6869

6970
export const DEFAULT_DISCOVERY_FILTERS: DiscoveryFilters = {
@@ -75,14 +76,129 @@ export const DEFAULT_DISCOVERY_FILTERS: DiscoveryFilters = {
7576
exclude: [],
7677
};
7778

78-
function compileExcludePattern(pattern: string): RegExp {
79-
const escaped = pattern.replace(/[.+^${}()|[\]\\?]/g, "\\$&").replace(/\*/g, ".*");
80-
return new RegExp(`^${escaped}$`, "i");
79+
/**
80+
* One `exclude` entry, compiled once at the input boundary. Plain data on purpose: a config built from argv must
81+
* equal one built from the environment, so the matcher is picomatch's RegExp, not a closure over it.
82+
*/
83+
export interface ExcludePattern {
84+
/** The operator's text, for the skip notice. */
85+
readonly pattern: string;
86+
/** "slug" when the pattern has a "/" and is compared against owner/name, "name" when against the name alone. */
87+
readonly scope: "slug" | "name";
88+
readonly regex: RegExp;
89+
}
90+
91+
/**
92+
* `nocase`: repository names compare case-insensitively on GitHub. `dot`: a leading "." is an ordinary character
93+
* in a repository name (".github"), not the hidden-file marker picomatch skips by default. `posix`: "[!ab]" is a
94+
* negated class, as in a shell, instead of a class holding "!". `fastpaths: false`: the shortcut regexes picomatch
95+
* mints for star-led patterns demand a character after a literal dot, so "*.*" would miss "archive."; the full
96+
* parser keeps "*" any run of characters. `debug`: an invalid class like "[z-a]" throws instead of compiling to a
97+
* never-matching regex. The posix entry point keeps the matcher platform-independent, since a slug is a GitHub
98+
* identifier, not a path.
99+
*/
100+
const EXCLUDE_GLOB_OPTIONS = {
101+
nocase: true,
102+
dot: true,
103+
posix: true,
104+
fastpaths: false,
105+
debug: true,
106+
} as const;
107+
108+
/** Every character no owner or repository name holds and no glob operator here uses. */
109+
const EXCLUDE_STRAYS = /[^\w.\-/*?[\]!^]/g;
110+
111+
/** One side of the "/": name characters, "*", "?", and classes "[abc]" or "[!abc]" over name characters. */
112+
const EXCLUDE_SEGMENT = /^(?:[\w.\-*?]|\[[!^]?[\w.-]+\])+$/;
113+
114+
/**
115+
* The grammar is owned here, not left to picomatch, because picomatch accepts far more than a repository name can
116+
* hold ("(a)+" a quantifier, "\d" a digit class, "{az..bz}" a range, "./" silently dropped, "***" losing the
117+
* escape of a later dot) and compiles several such inputs to a wrong or never-matching regex with no option to
118+
* refuse them. The pattern is the user's input, so a flaw is refused naming its fix, never repaired, and every
119+
* flaw of one pattern is named at once, so N flaws cost one run to discover, not N: each check runs on the
120+
* pattern as the fixes named before it would leave it, and picomatch's own refusal is probed on that view too.
121+
* Only an empty body ends the pass, since nothing is left to judge.
122+
*/
123+
function excludePatternFlaws(pattern: string): string[] {
124+
const flaws: string[] = [];
125+
const trimmed = pattern.trim();
126+
if (trimmed !== pattern) {
127+
flaws.push("it is wrapped in whitespace, so trim it");
128+
}
129+
const strays = [...new Set(trimmed.match(EXCLUDE_STRAYS) ?? [])];
130+
if (strays.length > 0) {
131+
const listed = strays.map((stray) => `"${stray}"`).join(", ");
132+
const them = strays.length === 1 ? "it" : "them";
133+
flaws.push(`it holds ${listed}, which no owner or repository name contains, so remove ${them}`);
134+
}
135+
const view = trimmed.replace(EXCLUDE_STRAYS, "");
136+
const negated = view.startsWith("!");
137+
let body = negated ? view.slice(1) : view;
138+
if (body.startsWith("./")) {
139+
flaws.push(`it starts with "./", which names no owner, so drop every leading "./"`);
140+
body = body.replace(/^(?:\.\/)+/, "");
141+
}
142+
if (body === "") {
143+
flaws.push(negated ? `"!" has nothing to negate` : "it is empty");
144+
return flaws;
145+
}
146+
const segments = body.split("/");
147+
if (segments.length > 2 || segments.includes("")) {
148+
flaws.push(`it takes at most one "/", with a non-empty glob on each side of it`);
149+
}
150+
const sides = segments.filter((segment) => segment !== "");
151+
if (sides.some((side) => side === "." || side === "..")) {
152+
flaws.push(`"." and ".." are never names`);
153+
}
154+
if (body.includes("**")) {
155+
flaws.push(
156+
`it holds a run of stars, which matches no more than one "*" does, so write one "*"`,
157+
);
158+
}
159+
if (!sides.every((side) => EXCLUDE_SEGMENT.test(side))) {
160+
flaws.push(
161+
`"!" negates only at the start or as the first character of a class, "^" only as the first character ` +
162+
`of a class, and a class is "[", name characters, "]" on one side of the "/"`,
163+
);
164+
}
165+
if (flaws.length > 0) {
166+
const fixed = `${negated ? "!" : ""}${body.replace(/\*{2,}/g, "*")}`;
167+
try {
168+
picomatch.makeRe(fixed, EXCLUDE_GLOB_OPTIONS);
169+
} catch (error) {
170+
flaws.push(error instanceof Error ? error.message : String(error));
171+
}
172+
}
173+
return flaws;
174+
}
175+
176+
/**
177+
* The scope split mirrors the repos-dir `<name>.yml` vs `<owner>/<name>.yml` pairing; picomatch's own basename
178+
* mode cannot express it (a slashed pattern stops matching the whole slug under it).
179+
*/
180+
export function compileExcludePattern(
181+
pattern: string,
182+
): Result<ExcludePattern, ProblemOf<"input-exclude-pattern-invalid">> {
183+
const flaws = excludePatternFlaws(pattern);
184+
if (flaws.length > 0) {
185+
return err({ code: "input-exclude-pattern-invalid", pattern, reason: flaws.join(", and ") });
186+
}
187+
const scope = pattern.includes("/") ? "slug" : "name";
188+
try {
189+
return ok({ pattern, scope, regex: picomatch.makeRe(pattern, EXCLUDE_GLOB_OPTIONS) });
190+
} catch (error) {
191+
return err({
192+
code: "input-exclude-pattern-invalid",
193+
pattern,
194+
reason: error instanceof Error ? error.message : String(error),
195+
});
196+
}
81197
}
82198

83-
export function excludeMatches(pattern: string, slug: string): boolean {
84-
const candidate = pattern.includes("/") ? slug : (slug.split("/")[1] ?? slug);
85-
return compileExcludePattern(pattern).test(candidate);
199+
export function excludeMatches(exclude: ExcludePattern, slug: string): boolean {
200+
const candidate = exclude.scope === "slug" ? slug : slug.slice(slug.indexOf("/") + 1);
201+
return exclude.regex.test(candidate);
86202
}
87203

88204
export interface DiscoveryResult {
@@ -188,8 +304,8 @@ function applyFilters(
188304
return null;
189305
},
190306
(repo) => {
191-
const hit = filters.exclude.find((pattern) => excludeMatches(pattern, repo.full_name));
192-
return hit ? `exclude pattern "${hit}"` : null;
307+
const hit = filters.exclude.find((exclude) => excludeMatches(exclude, repo.full_name));
308+
return hit ? `exclude pattern "${hit.pattern}"` : null;
193309
},
194310
];
195311
const kept: DiscoveredRepoRef[] = [];

‎src/flows/input-readers.ts‎

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,10 @@ import { err, ok, type Result, safeTry } from "neverthrow";
77
import {
88
AFFILIATIONS,
99
ARCHIVED_FILTERS,
10+
compileExcludePattern,
1011
DEFAULT_DISCOVERY_FILTERS,
1112
type DiscoveryFilters,
13+
type ExcludePattern,
1214
FORKS_FILTERS,
1315
VISIBILITY_FILTERS,
1416
} from "../discovery/discover.js";
@@ -196,13 +198,9 @@ export function readDiscoveryFilters(
196198
allowed: AFFILIATIONS,
197199
});
198200
}
199-
const exclude = input.list("exclude");
200-
const unmatchable = exclude.find((pattern) => {
201-
const parts = pattern.split("/");
202-
return parts.length > 2 || (parts.length === 2 && (!parts[0] || !parts[1]));
203-
});
204-
if (unmatchable !== undefined) {
205-
return err({ code: "input-exclude-pattern-invalid", pattern: unmatchable });
201+
const exclude: ExcludePattern[] = [];
202+
for (const pattern of input.list("exclude")) {
203+
exclude.push(yield* compileExcludePattern(pattern));
206204
}
207205
const discoveryFilters: DiscoveryFilters = {
208206
visibility,

‎src/flows/inputs.ts‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -181,8 +181,13 @@ export const INPUT_DECLS = {
181181
},
182182
exclude: {
183183
description:
184-
'Comma- or newline-separated wildcard patterns removing repositories from repos: "*" ' +
185-
"discovery.",
184+
'Comma- or newline-separated glob patterns removing repositories from repos: "*" discovery, compared ' +
185+
'case-insensitively against owner/name when the pattern has a "/" and against the name alone ' +
186+
'otherwise. A star matches any run of characters, "?" one character, "[abc]" one of a set, "[!abc]" ' +
187+
'or "[^abc]" one outside it, and one leading "!" negates the pattern. Beyond those a pattern holds ' +
188+
'only letters, digits, ".", "-", and "_", with at most one "/", nothing empty around it, and no side ' +
189+
'that is just "." or ".."; anything else (a leading "./", a run of stars, a regex token such as ' +
190+
'"(a)+") fails the run naming the fix.',
186191
default: "",
187192
list: true,
188193
},

0 commit comments

Comments
 (0)