Skip to content

Commit 2320431

Browse files
committed
feat(discovery)!: match exclude patterns with picomatch under an owned glob grammar
The exclude discovery filter compiled its own RegExp from each pattern, with "*" the only operator and every other character literal. Each pattern is now healed and compiled once at the input boundary with picomatch (the posix entry, nocase, dot, posix classes, noglobstar, no fast paths, debug), and the boundary owns the grammar: name characters, at most one "/", "*", "?", "[abc]", "[!abc]" or "[^abc]", and one leading "!". Malformed input with one unambiguous repair (a leading "./", a run of stars, and for library callers surrounding whitespace) is repaired and the run prints a notice naming the pattern as written and as matched. Anything else is refused when the inputs are parsed, naming the reason and the fix, because picomatch reads braces, parentheses, quotes, backslashes, and "***" as regex syntax through paths no option closes and would otherwise compile a wrong or never-matching filter silently. DiscoveryFilters.exclude carries the compiled ExcludePattern (written, pattern, scope, regex) instead of strings, so a config built from argv still equals one built from the environment, and compileExcludePattern, ExcludePattern, and formatHealNotice join the library surface. The e2e oracle's independent matcher learns "?", a leading "!", and the heals, and two scenarios pin the new forms end to end. The upgrading guide gains section 61. BREAKING CHANGE: In an exclude pattern, "?" now matches one character, "[abc]" and "[!abc]" are classes, and a leading "!" negates the pattern; each matched nothing before, so a pattern that relied on one of them matching literally is deleted and one that meant a glob is rewritten in that grammar. A leading "./" and a run of stars are healed and noticed in the run output, so "./tmp-**" runs as "tmp-*". A "." or ".." side, a second "/", a doubled "!", and any regex token ("(a)+", "\d", "{a..b}", quotes) fail the run naming the fix. Library callers supply exclude as ExcludePattern values from compileExcludePattern instead of strings.
1 parent 009d788 commit 2320431

22 files changed

Lines changed: 510 additions & 46 deletions

‎action.yml‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -146,8 +146,16 @@ inputs:
146146
default: ""
147147
exclude:
148148
description: >-
149-
Comma- or newline-separated wildcard patterns removing repositories from
150-
repos: "*" discovery.
149+
Comma- or newline-separated glob patterns removing repositories from
150+
repos: "*" discovery, compared case-insensitively against owner/name
151+
when the pattern has a "/" and against the name alone otherwise. A star
152+
matches any run of characters, "?" one character, "[abc]" one of a set,
153+
"[!abc]" or "[^abc]" one outside it, and one leading "!" negates the
154+
pattern. A leading "./" and a run of stars are repaired and noted in the
155+
run output. Beyond those a pattern holds only letters, digits, ".", "-",
156+
and "_", with at most one "/", nothing empty around it, and no side that
157+
is just "." or ".."; anything else, a regex token such as "(a)+"
158+
included, fails the run naming the fix.
151159
required: false
152160
default: ""
153161
topics:

‎bun.lock‎

Lines changed: 4 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎docs/operate/multi-repo.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,11 @@ Discovery takes six filter inputs that apply only to `repos: "*"`; setting any o
5757
- `archived` defaults to `skip`, because settings writes fail on archived repositories; `archived: only` is mostly useful with `mode: check`.
5858
- `forks` includes, excludes, or keeps only forks.
5959
- `topics` keeps repositories carrying at least one listed topic, so a single marker topic can opt repositories in. It is unrelated to the `topics` settings section.
60-
- `exclude` takes wildcard patterns where `*` matches anything: a pattern containing `/` is matched against the full `owner/name`, any other against the name alone, case-insensitively.
60+
- `exclude` takes glob patterns, matched case-insensitively: a pattern containing `/` against the full `owner/name`, any other against the name alone.
61+
- `*` matches any run of characters, `?` one character, `[abc]` one of a set, `[!abc]` or `[^abc]` one outside it.
62+
- One leading `!` negates the pattern, so `!svc-*` drops everything except the services.
63+
- A leading `./` and a run of stars are repaired, not refused: `./tmp-**` runs as `tmp-*`, and a notice names the pattern as written and as matched.
64+
- Beyond those operators a pattern holds only name characters (letters, digits, `.`, `-`, `_`), at most one `/` with a non-empty glob on each side, and no side that is just `.` or `..`. Anything else (a brace, a parenthesis, a backslash, a quote, a second `/`, a class holding `*`) fails the run before discovery starts, as does a class like `[z-a]`. The matcher is [picomatch](https://github.com/micromatch/picomatch), confined to that grammar.
6165
- `affiliation` selects which relationships to the token's user qualify, passed to GitHub's `/user/repos` listing: `owner` (the default), `collaborator`, or `organization_member`. The list replaces the default, so widening discovery beyond owned repositories takes `owner,collaborator`.
6266

6367
## Fallback for repositories without a settings file

‎docs/reference/inputs.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ A list input (`sections`, `required-sections`, `repos`, `exclude`, `topics`, `af
3838
| `visibility` | <p>Keeps only repositories of this visibility in repos: "*" discovery: all (default), public, private, or internal.</p> | `false` | `""` |
3939
| `archived` | <p>Archived-repository policy for repos: "*" discovery: skip (default), include, or only.</p> | `false` | `""` |
4040
| `forks` | <p>Fork policy for repos: "*" discovery: include (default), exclude, or only.</p> | `false` | `""` |
41-
| `exclude` | <p>Comma- or newline-separated wildcard patterns removing repositories from repos: "*" discovery.</p> | `false` | `""` |
41+
| `exclude` | <p>Comma- or newline-separated glob patterns removing repositories from repos: "*" discovery, compared case-insensitively against owner/name when the pattern has a "/" and against the name alone otherwise. A star matches any run of characters, "?" one character, "[abc]" one of a set, "[!abc]" or "[^abc]" one outside it, and one leading "!" negates the pattern. A leading "./" and a run of stars are repaired and noted in the run output. Beyond those a pattern holds only letters, digits, ".", "-", and "_", with at most one "/", nothing empty around it, and no side that is just "." or ".."; anything else, a regex token such as "(a)+" included, fails the run naming the fix.</p> | `false` | `""` |
4242
| `topics` | <p>Comma- or newline-separated topics, of which repos: "*" discovery keeps the repositories carrying at least one.</p> | `false` | `""` |
4343
| `affiliation` | <p>Comma- or newline-separated affiliations for repos: "*" discovery: owner (default), collaborator, or organization_member.</p> | `false` | `""` |
4444
<!-- action-docs-inputs source="action.yml" -->

‎docs/reference/library.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -244,6 +244,9 @@ const renderExitCode = runRender(
244244
| `discoverRepos` | function | The repositories a token can see, filtered |
245245
| `DiscoveryFilters` | type | The filters: `visibility`, `archived`, `forks`, `affiliation`, `topics`, `exclude` |
246246
| `DEFAULT_DISCOVERY_FILTERS` | const | The action's defaults |
247+
| `compileExcludePattern` | function | One `exclude` glob healed and compiled once, or the problem naming why it is unusable |
248+
| `ExcludePattern` | type | A compiled `exclude` entry: the pattern as written, as matched after the heals, the scope its slash chose, and the regex |
249+
| `formatHealNotice` | function | The notice the action prints for a healed `exclude` entry, naming it as written and as matched, or null when nothing was healed |
247250
| `parseReposInput` | function | The `repos` input form: slugs, or `"*"` for discovery |
248251
| `resolveCentralTargets` | function | The `<owner>/<name>.yml` files of a repos-dir as targets, with warnings for the files it skipped |
249252
| `dedupeTargets` | function | Merge the central and the remote targets, central first; a remote target whose slug a central one already names is dropped with a notice |

‎docs/upgrading/v2-to-v3.md‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1265,6 +1265,40 @@ v3 merge build(deps): bump yaml -> green -> no release-PR refresh
12651265

12661266
Fix: nothing for a consumer of `@next`. For a build of one exact commit, install the packaged commit (`github:Vivswan/github-settings-as-code#<packaged sha>`, from a `build/<position>.<sha7>` tag, the ten newest kept, or a release tag).
12671267

1268+
## 61. Discovery: `exclude` patterns are globs under an owned grammar
1269+
1270+
For every `repos: "*"` workflow that sets `exclude`, and for `@vivswan/github-settings-as-code` consumers building `DiscoveryFilters`. In v2 an `exclude` pattern knew one operator, `*`, and matched every other character literally, so a `?`, a `[`, or a leading `!` could only ever match nothing. In v3 the pattern is a glob matched by picomatch inside a grammar the input boundary owns: `*` any run of characters, `?` one character, `[abc]` one of a set, `[!abc]` or `[^abc]` one outside it, one leading `!` negating the whole pattern, and beyond those only name characters (letters, digits, `.`, `-`, `_`) with at most one `/`. The scope rule is unchanged: a pattern with `/` matches `owner/name`, any other the name alone, case-insensitively.
1271+
1272+
Three kinds of pattern move:
1273+
1274+
- A pattern that gains glob meaning: `tmp-?`, `svc-[ab]`, `!svc-*`. Each matched nothing in v2; in v3 `!svc-*` excludes every repository that is not a service.
1275+
- A pattern with one unambiguous repair is healed and the run says so in a notice: a leading `./` drops and a run of stars becomes one `*`. `./tmp-**` runs as `tmp-*`.
1276+
- A pattern outside the grammar fails the run before discovery starts, naming the fix: a `.` or `..` side (`x/..`), a second `/`, a doubled `!!`, or a regex token (`(a)+`, `\d`, `{a..b}`, a quote).
1277+
1278+
```text
1279+
v2 exclude: "!svc-*" -> matched nothing, every repository discovered
1280+
v3 exclude: "!svc-*" -> excludes every repository whose name is not svc-*
1281+
notice: repos: "*" discovery skipped 2 repositories by exclude pattern "!svc-*": o/tooling, o/website
1282+
1283+
v2 exclude: "./tmp-**" -> matched nothing
1284+
v3 exclude: "./tmp-**" -> runs as tmp-*
1285+
notice: repos: "*" discovery healed the exclude pattern "./tmp-**" to "tmp-*"
1286+
1287+
v3 exclude: "svc-(a)+" -> error: the "exclude" input pattern "svc-(a)+" is not a usable glob: it holds "(", ...
1288+
```
1289+
1290+
Fix: delete a pattern that relied on `?`, `[`, `]`, or `!` matching literally, since it never matched a repository; rewrite one that meant a glob in the grammar above, and check the notices of the first run for a heal you did not intend.
1291+
1292+
For library callers, `DiscoveryFilters.exclude` takes compiled `ExcludePattern` values instead of strings: run each pattern through `compileExcludePattern` at your input boundary, and read `formatHealNotice` for the repair line the action prints.
1293+
1294+
```text
1295+
v2 discoverRepos(client, { ...DEFAULT_DISCOVERY_FILTERS, exclude: ["tmp-*"] });
1296+
1297+
v3 const exclude = compileExcludePattern("tmp-*");
1298+
if (exclude.isErr()) throw new Error(describeProblem(exclude.error));
1299+
discoverRepos(client, { ...DEFAULT_DISCOVERY_FILTERS, exclude: [exclude.value] });
1300+
```
1301+
12681302
## Order of operations
12691303

12701304
1. Rename any settings file whose path contains a comma, rename `undeclared` to `_undeclared` in every settings file, and move every other underscore key into a YAML comment; the v2 line accepts the old spellings only, so do all three together with the pin move.

‎package.json‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,7 @@
7676
"consola": "3.4.2",
7777
"neverthrow": "8.2.0",
7878
"picocolors": "1.1.1",
79+
"picomatch": "4.0.7",
7980
"yaml": "2.9.1",
8081
"zod": "4.6.5"
8182
},
@@ -89,6 +90,7 @@
8990
"@octokit/openapi-webhooks": "12.1.0",
9091
"@types/bun": "1.4.2",
9192
"@types/node": "26.6.4",
93+
"@types/picomatch": "4.0.3",
9294
"action-docs": "2.5.1",
9395
"ajv": "8.20.0",
9496
"ajv-formats": "3.0.1",

‎src/discovery/discover.ts‎

Lines changed: 117 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,8 @@
55
*/
66

77
import type { components } from "@octokit/openapi-types";
8-
import { err, ok, ResultAsync } from "neverthrow";
8+
import { err, ok, type Result, ResultAsync } from "neverthrow";
9+
import picomatch from "picomatch/posix";
910
import type { GitHubClient } from "../github/api.js";
1011
import { classifyApiError } from "../github/api-error.js";
1112
import { paginate } from "../github/paginate.js";
@@ -64,7 +65,7 @@ export interface DiscoveryFilters {
6465
forks: (typeof FORKS_FILTERS)[number];
6566
affiliation: string[];
6667
topics: string[];
67-
exclude: string[];
68+
exclude: ExcludePattern[];
6869
}
6970

7071
export const DEFAULT_DISCOVERY_FILTERS: DiscoveryFilters = {
@@ -76,14 +77,120 @@ export const DEFAULT_DISCOVERY_FILTERS: DiscoveryFilters = {
7677
exclude: [],
7778
};
7879

79-
function compileExcludePattern(pattern: string): RegExp {
80-
const escaped = pattern.replace(/[.+^${}()|[\]\\?]/g, "\\$&").replace(/\*/g, ".*");
81-
return new RegExp(`^${escaped}$`, "i");
80+
/**
81+
* One `exclude` entry, compiled once at the input boundary. Plain data on purpose: a config built from argv must
82+
* equal one built from the environment, so the matcher is picomatch's RegExp, not a closure over it.
83+
*/
84+
export interface ExcludePattern {
85+
/** The pattern as the operator wrote it. */
86+
readonly written: string;
87+
/** The pattern as matched: `written` after the heals; the heal notice names both when they differ. */
88+
readonly pattern: string;
89+
/** "slug" when the pattern has a "/" and is compared against owner/name, "name" when against the name alone. */
90+
readonly scope: "slug" | "name";
91+
readonly regex: RegExp;
92+
}
93+
94+
/**
95+
* `nocase`: repository names compare case-insensitively on GitHub. `dot`: a leading "." is an ordinary character
96+
* in a repository name (".github"), not the hidden-file marker picomatch skips by default. `posix`: "[!ab]" is a
97+
* negated class, as in a shell, instead of a class holding "!". `noglobstar`: "**" is "*", since a slug has one
98+
* "/" and nothing to span; it also keeps picomatch's "***" parse from dropping the escape of a later dot.
99+
* `fastpaths: false`: the shortcut regexes picomatch mints for star-led patterns demand a character after a
100+
* literal dot, so "*.*" would miss "archive."; the full parser keeps "*" any run of characters. `debug`: an
101+
* invalid class like "[z-a]" throws instead of compiling to a never-matching regex. The posix entry point keeps
102+
* the matcher platform-independent, since a slug is a GitHub identifier, not a path.
103+
*/
104+
const EXCLUDE_GLOB_OPTIONS = {
105+
nocase: true,
106+
dot: true,
107+
posix: true,
108+
noglobstar: true,
109+
fastpaths: false,
110+
debug: true,
111+
} as const;
112+
113+
/** A character no owner or repository name holds and no glob operator here uses. */
114+
const EXCLUDE_STRAY = /[^\w.\-/*?[\]!^]/;
115+
116+
/** One side of the "/": name characters, "*", "?", and classes "[abc]" or "[!abc]" over name characters. */
117+
const EXCLUDE_SEGMENT = /^(?:[\w.\-*?]|\[[!^]?[\w.-]+\])+$/;
118+
119+
/** Malformed input with one unambiguous repair is repaired and reported, not refused. */
120+
function healExcludePattern(written: string): string {
121+
const trimmed = written.trim();
122+
const negated = trimmed.startsWith("!");
123+
const body = (negated ? trimmed.slice(1) : trimmed)
124+
.replace(/^(?:\.\/)+/, "")
125+
.replace(/\*{2,}/g, "*");
126+
return negated ? `!${body}` : body;
127+
}
128+
129+
/**
130+
* The grammar is owned here, not left to picomatch, because picomatch accepts far more than a repository name can
131+
* hold ("(a)+" a quantifier, "\d" a digit class, "{az..bz}" a range) and compiles several such inputs to a wrong
132+
* or never-matching regex with no option to refuse them. Each flaw names its fix.
133+
*/
134+
function excludePatternFlaw(pattern: string): string | null {
135+
const stray = EXCLUDE_STRAY.exec(pattern);
136+
if (stray) {
137+
return `it holds "${stray[0]}", which no owner or repository name contains`;
138+
}
139+
const negated = pattern.startsWith("!");
140+
const body = negated ? pattern.slice(1) : pattern;
141+
if (body === "") {
142+
return negated ? `"!" has nothing to negate` : "it is empty";
143+
}
144+
const segments = body.split("/");
145+
if (segments.length > 2 || segments.includes("")) {
146+
return `it takes at most one "/", with a non-empty glob on each side of it`;
147+
}
148+
if (segments.some((segment) => segment === "." || segment === "..")) {
149+
return `"." and ".." are never names`;
150+
}
151+
if (!segments.every((segment) => EXCLUDE_SEGMENT.test(segment))) {
152+
return (
153+
`"!" negates only at the start or as the first character of a class, "^" only as the first character ` +
154+
`of a class, and a class is "[", name characters, "]" on one side of the "/"`
155+
);
156+
}
157+
return null;
158+
}
159+
160+
/**
161+
* The scope split mirrors the repos-dir `<name>.yml` vs `<owner>/<name>.yml` pairing; picomatch's own basename
162+
* mode cannot express it (a slashed pattern stops matching the whole slug under it).
163+
*/
164+
export function compileExcludePattern(
165+
written: string,
166+
): Result<ExcludePattern, ProblemOf<"input-exclude-pattern-invalid">> {
167+
const pattern = healExcludePattern(written);
168+
const flaw = excludePatternFlaw(pattern);
169+
if (flaw !== null) {
170+
return err({ code: "input-exclude-pattern-invalid", pattern: written, reason: flaw });
171+
}
172+
const scope = pattern.includes("/") ? "slug" : "name";
173+
try {
174+
return ok({ written, pattern, scope, regex: picomatch.makeRe(pattern, EXCLUDE_GLOB_OPTIONS) });
175+
} catch (error) {
176+
return err({
177+
code: "input-exclude-pattern-invalid",
178+
pattern: written,
179+
reason: error instanceof Error ? error.message : String(error),
180+
});
181+
}
182+
}
183+
184+
/** The run-output line for a healed pattern, naming it as written and as matched; null when nothing was healed. */
185+
export function formatHealNotice(exclude: ExcludePattern): string | null {
186+
return exclude.written === exclude.pattern
187+
? null
188+
: `repos: "*" discovery healed the exclude pattern "${exclude.written}" to "${exclude.pattern}"`;
82189
}
83190

84-
export function excludeMatches(pattern: string, slug: string): boolean {
85-
const candidate = pattern.includes("/") ? slug : (slug.split("/")[1] ?? slug);
86-
return compileExcludePattern(pattern).test(candidate);
191+
export function excludeMatches(exclude: ExcludePattern, slug: string): boolean {
192+
const candidate = exclude.scope === "slug" ? slug : slug.slice(slug.indexOf("/") + 1);
193+
return exclude.regex.test(candidate);
87194
}
88195

89196
export interface DiscoveryResult {
@@ -184,8 +291,8 @@ function applyFilters(
184291
return null;
185292
},
186293
(repo) => {
187-
const hit = filters.exclude.find((pattern) => excludeMatches(pattern, repo.full_name));
188-
return hit ? `exclude pattern "${hit}"` : null;
294+
const hit = filters.exclude.find((exclude) => excludeMatches(exclude, repo.full_name));
295+
return hit ? `exclude pattern "${hit.pattern}"` : null;
189296
},
190297
];
191298
const kept: DiscoveredRepoRef[] = [];

‎src/flows/input-readers.ts‎

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,10 @@ import { err, ok, type Result, safeTry } from "neverthrow";
77
import {
88
AFFILIATIONS,
99
ARCHIVED_FILTERS,
10+
compileExcludePattern,
1011
DEFAULT_DISCOVERY_FILTERS,
1112
type DiscoveryFilters,
13+
type ExcludePattern,
1214
FORKS_FILTERS,
1315
VISIBILITY_FILTERS,
1416
} from "../discovery/discover.js";
@@ -196,13 +198,9 @@ export function readDiscoveryFilters(
196198
allowed: AFFILIATIONS,
197199
});
198200
}
199-
const exclude = input.list("exclude");
200-
const unmatchable = exclude.find((pattern) => {
201-
const parts = pattern.split("/");
202-
return parts.length > 2 || (parts.length === 2 && (!parts[0] || !parts[1]));
203-
});
204-
if (unmatchable !== undefined) {
205-
return err({ code: "input-exclude-pattern-invalid", pattern: unmatchable });
201+
const exclude: ExcludePattern[] = [];
202+
for (const pattern of input.list("exclude")) {
203+
exclude.push(yield* compileExcludePattern(pattern));
206204
}
207205
const discoveryFilters: DiscoveryFilters = {
208206
visibility,

‎src/flows/inputs.ts‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -181,8 +181,14 @@ export const INPUT_DECLS = {
181181
},
182182
exclude: {
183183
description:
184-
'Comma- or newline-separated wildcard patterns removing repositories from repos: "*" ' +
185-
"discovery.",
184+
'Comma- or newline-separated glob patterns removing repositories from repos: "*" discovery, compared ' +
185+
'case-insensitively against owner/name when the pattern has a "/" and against the name alone ' +
186+
'otherwise. A star matches any run of characters, "?" one character, "[abc]" one of a set, "[!abc]" ' +
187+
'or "[^abc]" one outside it, and one leading "!" negates the pattern. A leading "./" and a run of ' +
188+
"stars are repaired and noted in the run output. Beyond those a pattern holds " +
189+
'only letters, digits, ".", "-", and "_", with at most one "/", nothing empty around it, and no side ' +
190+
'that is just "." or ".."; anything else, a regex token such as "(a)+" included, fails the run naming ' +
191+
"the fix.",
186192
default: "",
187193
list: true,
188194
},

0 commit comments

Comments
 (0)