ci: narrow the language matrix to the languages a PR can affect - #649
Draft
malaverdiere wants to merge 4 commits into
Draft
ci: narrow the language matrix to the languages a PR can affect#649malaverdiere wants to merge 4 commits into
malaverdiere wants to merge 4 commits into
Conversation
malaverdiere
force-pushed
the
marc-andre/lang-579-ci-matrix-setup-cost
branch
from
July 30, 2026 17:12
656eba5 to
fc5fdff
Compare
malaverdiere
force-pushed
the
marc-andre/lang-579-matrix-changed-languages
branch
from
July 30, 2026 17:13
344a687 to
39de6d8
Compare
malaverdiere
changed the base branch from
marc-andre/lang-579-ci-matrix-setup-cost
to
marc-andre/lang-579-goal-5-make-the-ci-tests-pass
July 30, 2026 17:38
malaverdiere
force-pushed
the
marc-andre/lang-579-matrix-changed-languages
branch
from
July 30, 2026 17:39
39de6d8 to
36bc636
Compare
malaverdiere
force-pushed
the
marc-andre/lang-579-matrix-changed-languages
branch
from
July 30, 2026 18:33
36bc636 to
dbaae2c
Compare
malaverdiere
marked this pull request as draft
July 30, 2026 18:34
Base automatically changed from
marc-andre/lang-579-goal-5-make-the-ci-tests-pass
to
main
July 30, 2026 19:49
Review suggestion on #646: only run a language if it changed. A typical grammar bump touches one grammar dir, so this takes the matrix from 47 jobs to 1-3. The naive version of this is unsafe, because grammars read each other. Bumping tree-sitter-c must also test cpp *and* solidity; tree-sitter-c-sharp must also test c-sharp-pro; tree-sitter-xml and tree-sitter-hacklang have no language of their own and are only reachable through html and hack. lang/scripts/ languages-for-paths therefore walks the reference graph transitively, reading the edges out of the grammars themselves ("tree-sitter-<x>" tokens in grammar.js and prep) so the graph cannot drift from the grammars it describes. Fail-safe in every direction: - any changed path outside the grammar dirs (core/, lang/Makefile, lang/scripts, .github/, ...) selects every language; - an empty or unreadable diff selects every language; - push to main and workflow_dispatch always select every language, so main is never partially verified; - over-selection is tolerated (a "tree-sitter-rust" mention in a comment costs one extra job), under-selection is what the graph walk exists to prevent. The required "Test languages" check now accepts a skipped matrix only when the selection was genuinely empty, so a matrix that is skipped for any other reason cannot silently pass branch protection. Co-Authored-By: Claude <noreply@anthropic.com>
'github.event_name == '"'"'pull_request'"'"' && 0 || 1' evaluates to 1: 0 is falsy in a GitHub Actions expression, so the ternary falls through to the fallback. The enumerate job therefore had a depth-1 clone and could not resolve the base: fatal: Invalid symmetric difference expression 656eba5...HEAD Use fetch-depth: 0 unconditionally, and treat an unusable base as "select everything" rather than a failure. The required check behaved correctly meanwhile: enumerate failing left the matrix skipped with an empty selection, which the gate rejected instead of passing. Co-Authored-By: Claude <noreply@anthropic.com>
CI caught the exact under-selection this design exists to prevent. The
reference graph is read out of the upstream grammars, so with submodules
uninitialised the edges simply are not there:
assert {'c', 'cpp'} >= {'c', 'cpp', 'solidity'}
tree-sitter-solidity is what records that solidity reads tree-sitter-c, so
without it checked out a tree-sitter-c bump would silently skip solidity.
- languages-for-paths now refuses to narrow when any tree-sitter-* dir has no
content, and says which one. Dotfiles do not count as content: a deinitialised
submodule can retain a .git file and would otherwise look populated.
- The enumerate job checks the grammars out so narrowing actually applies. Done
by hand rather than via checkout's submodules: input, because gosu and
requirements use SSH URLs that need the HTTPS rewrite first.
- Tests now assert both modes: the graph edges when submodules are present, and
the fall-back-to-everything behaviour when they are not.
Co-Authored-By: Claude <noreply@anthropic.com>
malaverdiere
force-pushed
the
marc-andre/lang-579-matrix-changed-languages
branch
from
July 30, 2026 20:06
dbaae2c to
ea5f10c
Compare
Scan only live .js/prep refs (not comments), resolve prep's tree-sitter-"$name", and alias go-mod/gomod so every wrapper stays reachable. Fail closed when a grammar-dir diff resolves to no languages, and lock the JS→TS / C→cpp invariants in tests. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the review suggestion on #646: "Is there a way to limit the matrix to only run a language if it is changed?"
A typical grammar bump touches one grammar dir, so this takes the matrix from 47 jobs to 1–3.
Why the naive version is unsafe
Grammars read each other, so "changed dir → that language" silently under-tests:
tree-sitter-ccpp,soliditytree-sitter-c-sharpc-sharp-protree-sitter-xmlhtml(nosemgrep-xmlexists)tree-sitter-hacklanghack(nosemgrep-hacklangexists)lang/scripts/languages-for-pathstherefore walks the reference graph transitively, reading edges out of the grammars themselves (tree-sitter-<x>tokens ingrammar.js/prep) so the graph cannot drift from the grammars it describes.Fail-safe in every direction
core/,lang/Makefile,lang/scripts/,.github/, …) selects every language.push: mainandworkflow_dispatchalways select every language, so main is never partially verified.tree-sitter-rustmention in a comment costs one extra job); under-selection is what the graph walk exists to prevent.This PR's own diff touches
.github/andlang/scripts/, so it selects all 47 — visible in theEnumerate languagesjob log.Branch protection
The required
Test languagescheck accepts a skipped matrix only when the selection was genuinely empty ([]). A matrix skipped for any other reason still fails the check, so it cannot silently pass.Tests
lang/test_languages_for_paths.py(6 cases) covers the cross-grammar edges, the no-language-of-its-own dirs, shared paths, mixed diffs, and the empty-diff fallback. Wired intomake test-python.