ci: group Dependabot example updates and verify only what changed - #844
Merged
bnusunny merged 27 commits intoSep 17, 2026
Merged
Conversation
There was no dependabot.yml, so Dependabot opened one pull request per advisory per manifest: 69 open at the time of writing, 13 of them against the single lockfile in examples/remix/remix-app and 8 against examples/remix-zip. Siblings in the same lockfile conflict as soon as one lands, which is why 13 of the 69 are already CONFLICTING. Each ecosystem present under examples/ now gets a group with `applies-to: security-updates`, so an example app is updated by one pull request instead of eight. `open-pull-requests-limit: 0` disables version updates and leaves security updates on, which preserves today's behavior: PRs for advisories only, not for every dependency that has drifted. The adapter's own Cargo.toml is deliberately absent — it ships as the lambda-adapter binary and its dependencies are reviewed by hand.
Verify Examples ran all 18 matrix entries for any change under examples/, so a lockfile bump in examples/remix/remix-app rebuilt and booted springboot, nextjs, deno-zip and the rest. At ~70 open Dependabot pull requests that is the dominant CI cost, and none of it is signal. A `select` job now diffs the pull request and emits one matrix per job kind, so that bump runs a single job. The example lists move to .github/example-matrix.json so the selector and the matrices share one source of truth. The selector fails safe — no base commit, a base commit that is not available locally, or a change to shared code (src/, layer/, Cargo.toml, the workflow itself, the matrix file) all verify everything. Pushes to main and manual runs are unaffected: they have no base commit and so verify everything. Adds `examples-verified`, one aggregate result for the whole workflow, treating `skipped` as a pass since that is what a filtered-out matrix means. Verified against real commits: a bump under examples/remix/remix-app selects only remix, the SnapStart merge (src/) selects everything, and a bump under an example with no matrix entry selects nothing.
Merges a Dependabot pull request once Verify Examples has gone green on it, if every file it changes is under examples/ and at least one build-and-boot job actually ran. Keyed off the completed workflow run rather than `gh pr merge --auto`: auto-merge is gated on the repository's *required* status checks, and Verify Examples is path-filtered to examples/**, so requiring its result would never report on a source-only pull request and would block it forever. The completed run is also tied to the head commit being merged, which addresses the stale verdict problem — PRs opened months ago still carry check results from the main of that day (#827 and older show validate:FAILURE for exactly that reason). Three guards, all necessary: * author is Dependabot; * every changed file is under examples/, checked against the PR's file list rather than its branch name, because grouped updates do not reliably encode the directory in the ref; * at least one test-* job succeeded, so an example with no matrix entry cannot ride in on a green run that only validated templates. Scope is deliberate: examples are demo apps, where a bad bump costs a broken sample. The adapter's own dependencies, the workflows, and the layer templates stay manual.
Records the two behaviors this config depends on, both now confirmed rather than assumed: Grouping is per directory. PRs #804 and #811 carry the identical update set (body-parser + express) and even the identical branch hash multi-be700a2db9, yet Dependabot raised them as two separate PRs, one per directory. Cross-directory batching requires `group-by: dependency-name`, which applies to version updates only. So this config yields one PR per example app per ecosystem. `open-pull-requests-limit: 0` stops version updates without stopping security updates: those are exempt from the limit and do not count toward it.
![aws-sam-tooling-bot[bot]](https://avatars.githubusercontent.com/b/1290?s=60&v=4){kind=small}
Three fixes from review on #844, all in the merge step. Pin the merge to the verified commit. The evidence came from a workflow run tied to workflow_run.head_sha, but `gh pr merge` merged whatever the head was when the API call ran. Dependabot force-pushes its branches on rebase and recreate, so the head can move while the run finishes, and the result would be an unverified commit squashed into main. The step now compares the current head to the verified one and skips if it moved, and passes --match-head-commit to close the remaining window. Make the scope guard fail closed. `outside=$(gh api ... | grep -v '^examples/' || true)` applied `|| true` to the whole pipeline, so a rate-limited or failed API call left `outside` empty and the pull request read as example-only — the one check keeping the adapter's Cargo.toml, the workflows, and the layer templates out of auto-merge. The API call is now separate from the filtering, with an explicit refusal on an empty list. Confirmed by simulation: the old form merges on an API failure, the new form does not. Gate on coverage of the changed set, not a job count. "At least one test-* job succeeded" was weaker than its comment claimed: select-examples.sh silently drops changed examples with no matrix entry, so a pull request touching one covered and one uncovered example passed while the second was never built or booted. Every changed example must now appear in .github/example-matrix.json, read at the verified commit. Not reachable with today's config — all 69 open Dependabot pull requests touch exactly one example, and grouping is per directory — but it becomes reachable the moment grouping spans directories, and the job count is the wrong thing to assert either way. All six guard paths exercised against a stubbed gh: merge, moved head, files outside examples/, uncovered example, covered-plus-uncovered, and no test jobs.
Three fixes from the second review on #844. Add `actions: read`. Declaring a permissions block sets every unnamed scope to none, and the coverage gate lists the triggering run's jobs, so the Actions API would have returned 403 and aborted the step on every Dependabot pull request. It fails closed, but the automation would never have merged anything. commitlint-comment.yaml declares the same scope for the same reason. Count job names instead of asking jq for a length. `--paginate` applies `-q` per page, so a run spanning two pages yields one count per line ("18\n4"), and `[[ "18\n4" -eq 0 ]]` is an arithmetic syntax error that evaluates false — skipping the refusal and merging. Verified in bash: the multi-line form errors with "syntax error in expression" and takes the else branch. Single-page today at 22 jobs, but the failure direction is fail-open, and the headroom is smaller than it looks. Add the bundler ecosystem for examples/sinatra/app/src/Gemfile, which the original sweep missed. Beyond grouping, this is what gives those PRs a conventional commit prefix: Commit Lint runs on every pull request with no path filter, and #799 shows what the default message costs — commit "bump com.fasterxml.jackson.core:jackson-databind", Lint Commit Messages red.
Three fixes from the third review on #844. Trigger Verify Examples on its own machinery. The workflow only ran for examples/**, so a pull request changing example-matrix.json or select-examples.sh — now the single source of truth for what gets verified — never ran it. This pull request is the proof: it rewires all three matrices and its checks are Commit Lint, CodeQL and pr.yaml only, no validate or test-* at all. The matrix file, the selector directory and this workflow are now triggers, so it verifies its own changes. src/, layer/ and Cargo.* are deliberately still not pull request triggers even though the selector treats them as shared: adding them would run all eighteen matrix entries on every source pull request. Adapter changes stay verified against the examples on push to main, and the shared-path rule still applies to a pull request touching both. Gate on the whole check rollup, not just this one run. Commit Lint runs on every pull request with no path filter and does go red on Dependabot PRs (#799), yet the merge consulted only the Verify Examples run, and any check added later would have been ignored too. Anything not SUCCESS/SKIPPED/NEUTRAL — including still running, since a workflow_run job cannot wait — now refuses the merge; Dependabot rebases these branches often and any later run re-evaluates. Verified against live data: the query is empty for #842 (all green, CodeQL NEUTRAL) and names the failure on #844. This workflow's own run is excluded defensively, since an in-progress self-check would deadlock every merge if workflow_run runs ever joined the rollup. Add the github-actions ecosystem for the commit prefix, for the same reason bundler was added: every workflow pins actions, so an advisory would open a PR with no conventional type and a red Commit Lint. With the limit at 0 it adds no pull requests. Pre-emptive — there are no open actions alerts today (the 705 open alerts are npm, pip, go, rubygems, rust and maven).
Two fixes from the fourth review on #844. Both are cases where a condition the code intends to handle deliberately instead killed the step with no explanation. Read the example matrix from the default branch, not from the verified head. A Dependabot branch cut before this file existed 404s at its own head, and gh's non-zero exit aborted the whole step under set -e with a bare "Not Found" rather than the refusal every other unresolvable condition here gets. Reachable immediately: confirmed a 404 against #842's current head, and all 69 open Dependabot PRs branch from a main that predates the file. The default branch is also the right source of truth — it is repository config, the examples-only guard means the pull request cannot have changed it, and a pull_request workflow runs the merge-ref copy, so it is the matrix the run actually used. A failed read or a malformed file now refuses explicitly. Stop the selector aborting when nothing under examples/ changed. `grep -oE` exits 1 on no match and pipefail turned that into a red Verify Examples with no diagnostic, contradicting the fail-safe contract documented at the top of the script. Reproduced with an empty diff, which is reachable for a stale pull request whose change already landed through a duplicate. It now emits empty matrices and exits 0, which the `if: ... != '[]'` guards and the auto-merge job's job-count check already handle. grep is kept out of the pipeline so `|| true` tolerates only its no-match status, and the empty case is explicit rather than relying on `jq -R .` turning an empty string into [""].
Two fixes from the fifth review on #844. The per-directory grouping premise was wrong, and my earlier confirmation of it was bad evidence. #804 and #811 do show Dependabot splitting an identical update set per directory — but those PRs were produced with no dependabot.yml at all, so they describe ungrouped security updates, not what `directories` plus a group does. The search API finds 307,565 pull requests titled "Bump the <group> group across N directories with M updates", including "across 17 directories with 100 updates". Grouping does span directories. That breaks two things this PR asserted. The blast-radius rationale inverts: a bad bump would block the security fixes for every example in the group rather than one. And auto-merge would essentially never fire, because the coverage gate refuses any pull request touching an example that example-matrix.json does not build and boot, and a cross-directory npm or pip pull request would nearly always include one. So the config is now one entry per example — 47 of them, plus github-actions — each scoped with `directories: ["/examples/<name>", "/examples/<name>/**"]`. One pull request per example app is now a property of the config rather than an assumption about Dependabot's behavior, which is what the review asked for. The cost is a 579-line config file; the alternative was keeping eight short entries and an auto-merge workflow that never merges anything. Classify merge failures instead of going red on all of them. `gh pr merge` exits non-zero when the pull request is not mergeable, and under set -e that turned an expected outcome into a red run — on the one workflow whose colour signals whether the automation is healthy. The reachable case is the sibling race this PR exists to fix: several pull requests touch one lockfile, the first merge conflicts the rest, and GitHub has not necessarily recomputed mergeability yet. A moved head or a non-mergeable state now exits 0 with the reason; anything unexplained still fails loudly, so a real misconfiguration (squash merges disabled, a missing permission) is not swallowed. Four paths exercised against a stubbed gh.
Three fixes from the sixth review on #844. Give the merge gate a second entry point. It was evaluated exactly once, when a Verify Examples run completed, so a pull request whose other checks were still running at that instant was skipped and never reconsidered — nothing else re-triggers the workflow, and Dependabot pushes a branch only when it rebases or recreates it, so an idle pull request could wait indefinitely. The review's point that this PR makes the race more likely is right: with the selector, Verify Examples runs build-layer plus one example instead of eighteen, so it stops being reliably the slowest check (measured on #842: Commit Lint 18s, Verify Examples 3m50s at eighteen jobs). The decision now lives in .github/scripts/dependabot-automerge.sh, driven by both a completed run and an hourly sweep, and it resolves the run itself from the pull request's current head rather than trusting an event payload — so both paths behave identically and the "head moved" special case disappears. Skips are recorded in the job summary, so a stalled pull request is visible rather than silent. The sweep pre-filters the queue in one request (23 of the 69 open pull requests are candidates today) to keep an hourly job off the token's rate limit. Coverage proof now comes from the run's own job names rather than only from the matrix file, which also closes a gap nobody raised: a run predating a matrix change could otherwise be credited with verifying an example it never launched. The matrix is still consulted, to tell "no entry, add one" apart from "entry exists, not run". Stop emit_all swallowing jq's exit status. `echo "$kind=$(jq ...)"` returns echo's 0 even when jq dies, so a missing or malformed example-matrix.json wrote `image=` and reported success — and an empty value is worse than a failure, because `!= '[]'` is true for it, so the test jobs ran and died in fromJSON('') pointing at nothing. Reproduced: three jq errors, exit 0, three empty outputs. Now exit 2 with jq's own diagnostic. Add an entry for the adapter's own Cargo.toml, for the commit message only. One correction to the review: this is pre-emptive, not live — all six open rust alerts are in examples/rust-actix-web-zip and examples/rust-axum-zip, which the per-example entries already cover, and none are against the root manifest. Fifteen scenarios exercised locally: the selector's four selection paths, and the merge script's eleven guard paths from happy-path merge through each refusal to the one case that still fails loudly.
The config was rejected: "Update configs must have a unique combination of 'package-ecosystem', 'directory', and 'target-branch'. Dependabot cannot determine if 'npm' has overlapping directories." Globs are the problem — with several entries per ecosystem, Dependabot will not merely assume that /examples/remix/** and /examples/remix-zip/** are disjoint, it refuses the file. So each entry now lists the manifest directories inside its example, derived from `git ls-files examples`: 53 directories across 47 example entries. Three examples have more than one manifest and keep grouping across them, which is the point — datadog alone has five. Key the concurrency group per pull request. Only one run per group may be pending and a new arrival cancels the pending one, so a single group meant a burst of Verify Examples runs finishing together — the normal case, since Dependabot opens security pull requests in batches — would cancel each other's queued fast-path runs until only the last survived. The overlap this allows between a sweep and a fast path on one pull request is already handled by --match-head-commit and the mergeStateStatus branch, which turn the loser into a skip rather than a red run. Handle a failed API call the way the header says the script handles everything else. Five gh lookups would abort the script bare under set -e, with no summary line and rc=1 propagating out of the sweep loop — the opposite of the care the merge block takes to keep this workflow's red/green meaningful. They now skip with a reason. Author, head SHA and the check rollup also come from one `gh pr view` call rather than three, which both simplifies that handling and cuts the sweep's per-pull-request cost. Read the matrix from disk rather than the API. The workflow already checks out the default branch, and the API call read that same content, so it was a network round trip and two failure branches for nothing. Moving the coverage check ahead of the run and job lookups matters more than it looks: an example with no matrix entry can never merge, and most examples have none, so those pull requests now cost two API calls per sweep instead of four, forever. Fifteen guard paths exercised, including the four new API-failure paths: each exits 0 with a summary line, and the only loud failure is still a merge that fails for no discoverable reason.
Null check rollup no longer aborts the merge script. `gh pr view --json statusCheckRollup` emits null, not [], when the head commit has no check runs yet — a real few-second window every time Dependabot force-pushes a rebase — and `.statusCheckRollup[]` over null makes jq exit, taking the script with it under set -e. The sweep's pre-filter already used `[]?`, which is what made the combination reachable: it treats such a pull request as all-green and hands it straight to the script. Both now use `[]?`. The sweep's `gh pr list` is fail-soft like every lookup in the script. It was the one call left that would abort the step, and on an hourly cron an unexplained red run is worse than a quiet skip, since the next sweep picks up whatever was missed. The selector derives the diff base from the merge ref rather than the payload. `github.event.pull_request.base.sha` can be older than the tip refs/pull/N/merge was recomputed against, in which case merge-base returns the payload SHA and the diff also contains everything that landed on main in between — measured on a real merge commit here, one file becomes five, and a single intervening commit under src/ would then verify all eighteen examples. HEAD^1 is the base the merge was actually computed against, so HEAD^1..HEAD is exactly the pull request's contribution. The payload path remains as the fallback for a non-merge checkout, and its "base not fetched" branch moved with it. Add .github/scripts/check-example-config.sh, run from the validate job. A manifest under examples/ with no dependabot.yml entry does not just lose grouping: it reverts to one pull request per advisory with a commit header Commit Lint rejects — the exact state the config exists to prevent — and that is invisible until the pull requests appear weeks later. With 47 entries covering 53 manifest directories, drift is a matter of when. The check also catches a stale entry pointing at a directory with no manifest, and an example-matrix.json name that is not a directory under examples/, which would otherwise hand out coverage for an example that no longer exists. dependabot.yml joins the workflow's trigger paths so editing it runs the check. Twenty-two paths exercised: the selector's six base-resolution and selection paths, and the merge script's sixteen guards including the null rollup and all four API failures. The drift guard was exercised against four deliberate drifts — a new unclaimed manifest, a ghost matrix name, a stale entry, and the clean tree.
Three fixes from the eighth review on #844. The drift guard recognised only requirements.txt for pip, which broke it in both directions. An example shipping only a pyproject.toml — the mainstream Python layout — would not appear in the found set, so the unclaimed-manifest assertion would pass while Dependabot opened one ungrouped pull request per advisory for it; and a maintainer who added the correct entry for such a directory would have been told the entry was stale, failing validate on a correct config. Both reproduced against a scratch example. pip now lists pyproject.toml, Pipfile and setup.py, nuget covers the other project types and packages.config, bundler covers gemspecs, and gradle and composer are listed although no example uses them yet, so the next one that does is caught. docker stays out deliberately, with the reason recorded: 24 Dockerfiles are under examples/, but Dependabot alerts do not cover base images, so an entry for them would not exist and the check would fail for its absence. Require the pull request to still be open. A closed one passes every other gate — its rollup is still green, its files are unchanged, and --match-head-commit matches a head that never moved — so it reached `gh pr merge` and the outcome rested on whatever mergeStateStatus reports for a merged pull request; anything outside the expected set would have exited 1 and turned the sweep red for a benign race. The overlap that produces this is one the concurrency keys deliberately allow. The state comes from the request already being made, and is re-checked in the failure branch for a merge that lands mid-run. Fix the dependabot.yml comment claiming "Verify Examples does not check for that", which the check-example-config.sh added in this same branch contradicts. It now points at the script, so a maintainer who hits the check knows where it comes from. Nineteen merge-script paths re-exercised after the change, including both new closed-pull-request windows and a failed re-read, plus the drift guard against a pyproject-only example in both the missing-entry and correct-entry states.
…h paths Four fixes from the ninth review on #844. The first is a fail-open hole. Grant checks: read and statuses: read. statusCheckRollup is the gate both the workflow and the merge script depend on, and it is built from Checks resources and commit statuses — neither of which a permissions block naming only contents, pull-requests and actions grants. Either GraphQL errors, making the workflow a permanent no-op that looks healthy, or it returns the field empty, which `[]?` would have swallowed as "all checks green" and merged past a red Commit Lint. That is the exact check #799 shows going red on a Dependabot pull request. Belt and braces on the same hole: the script now refuses outright when statusCheckRollup is not an array, so neither an absent permission nor a head with no checks yet can read as success. It used to merge in that case; it now skips and lets the sweep look again. The drift guard read only the plural `directories`. The singular `directory` is equally valid and is the canonical form for one directory, so an entry using it contributed nothing and its manifest was reported as unclaimed — validate failing on correct config, with a message telling the author to add an entry already in the file. Both spellings are read now. The drift guard also asserts the two keys the grouping actually rests on. `applies-to: security-updates` is load-bearing because plain groups batch version updates only, and those are off via the limit; a 48th entry copy-pasted without it would pass every check here while its advisories reverted to one pull request each. A missing or non-zero open-pull-requests-limit is the mirror image. With 47 near-identical entries these are exactly the copy-paste omissions worth machine checking. Widen the push trigger to every input the selector calls shared. It listed only src/**, so layer/**, Cargo.toml and Cargo.lock were verified against the examples on no event at all — and layer/bootstrap is the code path all eight zip examples boot, since build-layer copies it into the artifact each of them injects. The comment claiming those changes are covered on push to main is now true. Pull request triggers are unchanged, so source pull requests still do not fan out to eighteen jobs. Twenty merge-script paths re-run with expected exit codes, zero mismatches, plus the drift guard against a singular-directory entry, a missing applies-to, a non-zero limit, and the clean tree.
Two fixes from the tenth review on #844. Every group is now named `security` rather than after its example and ecosystem. Dependabot builds the commit header from the group name and the directory, and commitlint caps the header at 120 characters, which three of the 55 directories already exceeded: 137 chore: bump the fastapi-backend-only-response-streaming-pip group in ... 127 chore: bump the springboot-response-streaming-zip-maven group in ... 121 chore: bump the fasthtml-response-streaming-zip-pip group in ... The consequence is the one this branch exists to remove: Commit Lint red, dependabot-automerge.sh refusing on "checks are not all green", and a hand amend needed for exactly the examples with the longest names. The name only has to be unique within its own entry, and the directory in the header is what distinguishes the pull requests, so encoding the example in it bought nothing. Worst header is now 103 characters, none over the limit. check-example-config.sh asserts on the group's applies-to value rather than its name, so it needed no change. The selector's shared-path rule named .github/scripts/ as a directory, which since this branch also holds dependabot-automerge.sh and check-example-config.sh — neither of which any example is built against. Combined with .github/scripts/** in the workflow's pull request paths, a one-line fix to the auto-merge script rebuilt and booted all eighteen entries: precisely the cost this branch exists to remove, reintroduced through a glob. The rule now names verify-http.sh and select-examples.sh, the two scripts every test job actually runs, and the trigger names the three scripts this workflow uses at all. While there: build-layer only runs when `select` chose something. The artifact exists for the test jobs, so building it for an empty matrix was several minutes of runner time for nobody.
Two fixes from the eleventh review on #844, plus one found while auditing. git ls-files output is now NUL-separated. `.stdout.split()` splits on any whitespace, and git prints a path containing a space verbatim, so "examples/x/my app/package.json" was torn into fragments and the tail one derived a directory of "/app" — reported as an unclaimed manifest that no entry could ever claim. Reproduced: the guard failed naming `npm: /app`; it now names `npm: /examples/space test/my app`, which is the truth. -z also disables git's C-style quoting of non-ASCII paths, which would corrupt the derived directory the same way. Latent today, but these are contributed demo apps and validate now runs on every examples pull request, so a false failure there would block all of them. Glob directory values are matched rather than compared as strings. A glob is a supported form of the key, and while this config avoids them because Dependabot refuses several entries per ecosystem it cannot prove are non-overlapping, that reasoning does not extend to a single-entry ecosystem — so a maintainer may well write one. Reproduced with `/examples/sinatra/*/src`: the covered manifest was reported as unclaimed *and* the pattern as stale, two contradictory errors on a config Dependabot accepts. Matching is segment-aware rather than fnmatch over the whole path: `*` and `?` stay inside a segment and `**` spans several, so a single `*` cannot silently span two directory levels and pass drift this check exists to catch. Both directions now use it: a manifest is unclaimed when no pattern for its ecosystem covers it, and a pattern is stale when it covers no manifest. Also refreshed the selector's header comment, which still described only the verify-everything fail-safes and predated both the select-nothing path and the merge-ref base resolution. Nine drift-guard states exercised: clean tree, a spaced path, a glob that covers, a glob that covers nothing, a `**` glob, a single `*` that must not span two segments, the singular `directory` key, a non-zero limit, and restored. Plus the merge script's twenty paths, zero mismatches, and the selector's three selection paths.
main is governed by a ruleset, not classic branch protection — which is why my earlier check of branches/main/protection returned 404 and I wrongly concluded the branch was unprotected. The ruleset has required_approving_review_count: 1, require_code_owner_review: true and zero bypass actors, and .github/CODEOWNERS assigns `*` to @aws/aws-lambda-tooling. Every Dependabot pull request is therefore BLOCKED with reviewDecision=REVIEW_REQUIRED: #842 BLOCKED REVIEW_REQUIRED MERGEABLE #841 BLOCKED REVIEW_REQUIRED MERGEABLE #839 BLOCKED REVIEW_REQUIRED MERGEABLE So the workflow could never have merged anything, and worse, it hid that: BLOCKED was lumped in with conflicts and reported as "most likely a sibling update landed first", so the hourly sweep would have skipped every pull request forever with a diagnosis that was simply wrong. Three changes, no governance change. No bypass actor is added and no approval is forged: a bot approval cannot satisfy a code-owner requirement anyway, and whether CI should be allowed to merge without review is the code owners' call, not this branch's. * reviewDecision is now a gate, checked last so that reaching it means the pull request is example-only, verified at its current head, and green. A pull request waiting on review is reported as exactly that, which turns the sweep's job summary into a worklist of "verified, waiting only on you". * BLOCKED is classified separately from DIRTY/BEHIND/DRAFT/UNKNOWN and names the ruleset as the cause. * The workflow is renamed Dependabot Merge and its header, the script's header and the pull request description say merge-after-approval rather than auto-merge. The script's own rollup exclusion is updated to match the new name. Twelve paths exercised: the four reviewDecision states, the three post-merge failure classifications under an approval, and the five earlier gates still firing ahead of the review check.
The approval gate added in b1670c3 was the wrong shape. It read reviewDecision and refused unless APPROVED, which is correct only while main's ruleset lists no bypass actors — the moment this workflow's identity becomes one, the gate would keep refusing merges the ruleset would have allowed, and the fix would be to remember to delete it. The merge is now attempted unconditionally and the outcome classified, so one script is right in both worlds: unattended where a bypass actor exists, and "blocked awaiting a code-owner approval (@aws/aws-lambda-tooling), or a ruleset bypass actor for this workflow" where none does. Nothing to keep in sync with a repository setting the script cannot see. BLOCKED is still separated from DIRTY/BEHIND/DRAFT/UNKNOWN, and now splits by reviewDecision so the three cases read differently: awaiting approval, approved but some other rule unsatisfied, and changes requested. It is no longer reported as "a sibling update landed first", which was never true for it. Seven paths exercised: unapproved without bypass, approved without bypass, unapproved with bypass, approved-but-blocked, changes-requested, a sibling conflict, and an unexplained failure still exiting 1.
bnusunny
force-pushed
the
ci/dependabot-grouping-and-targeted-example-verification
branch
from
September 14, 2026 22:17
346ca4a to
f049b3c
Compare
The comments and the skip message offered "or a ruleset bypass actor for this workflow" as though the decision were still open. It is not: main's ruleset keeps its code-owner approval requirement and gains no bypass actor, because unattended merging would mean granting some identity the right to bypass code-owner review on a public repository, which is not a trade worth making for dependency bumps in demo applications. A bot approval could not satisfy a CODEOWNERS requirement anyway. Behaviour is unchanged — the merge is still attempted and the outcome classified, which is simpler than mirroring the ruleset's configuration in the script and cannot drift from it. The skip message for the ordinary case now reads as a worklist entry: "ready to merge, awaiting a code-owner approval (@aws/aws-lambda-tooling)". What the workflow is worth without unattended merging: approve once and the merge happens within the hour, but only when the verification covers the exact commit that lands, so a stale approval cannot merge an unverified head. The hourly sweep is what makes approving a batch and walking away work. Six paths re-run, all unchanged.
bnusunny
changed the title
Deletes .github/workflows/dependabot-automerge.yaml and .github/scripts/dependabot-automerge.sh, 372 lines and an hourly cron. Unattended merging was never available: main's ruleset requires one code-owner approval with zero bypass actors, and granting some identity the right to bypass code-owner review on a public repository is not a trade worth making for dependency bumps in demo applications. What was left after that decision merged an approved pull request without a second visit — real, but not worth 372 lines and a workflow holding contents: write, when a code owner is already in the loop and one click away. So Verify Examples does what its name says and nothing more: it fails a Dependabot pull request when the examples it touches do not build and boot, and a code owner merges. Everything that carried the weight stays — the grouped config that turns 69 per-advisory pull requests into one per example app, the selector that verifies only the examples a pull request touches, the drift guard that keeps both hand-maintained lists honest, and examples-verified as the single result a reviewer reads before merging. Also cleans up the twelve comments across four files that described the merge gate as the reason for a rule, since it no longer exists: the selector's "select nothing" contract now rests on examples-verified treating a skipped job as a pass, and the drift guard's matrix-name check on the selector choosing an example whose directory is gone. Suites re-run after the removal: the drift guard's clean tree plus a covering glob, a missing applies-to, and an unclaimed spaced manifest; the selector's no-base, empty-diff and single-example paths.
bnusunny
changed the title
…ified Three fixes from review on fb69e5d. emit_all wrote `null` for a missing key. `jq -c '.stream'` prints the literal null and exits 0, so a renamed top-level key in example-matrix.json wrote `stream=null`, which `!= '[]'` reads as truthy — test-stream would start and die in fromJSON('null') pointing at the workflow rather than the matrix file. The asymmetry is what made it easy to miss: the selection loop already fails loudly on the same input, because `.$kind[]` over null is a jq error, so only the verify-everything path — every push to main — degraded silently. Confirmed: emit_all exited 0 having written `stream=null`, the selection loop exits 5. It now asserts the key with `has` and exits 5 too. The drift guard now asserts commit-message.prefix, the third load-bearing key and the one already paid for: without it Dependabot writes "bump <dep> from x to y", which has no conventional type, and Commit Lint runs on every pull request with no path filter (#799). The accepted types are parsed out of commitlint.config.js rather than copied, so the check cannot disagree with the linter that actually runs; an unparseable file downgrades the assertion to "a prefix is set" instead of inventing a list. Reading the list matters here because the github-actions entry deliberately uses `ci`, which an equality check against `chore` would have rejected. examples-verified now says which of two very different things its green means. The matrix covers 18 of the ~46 examples dependabot.yml claims, so for most Dependabot pull requests every matrix filters to empty, the test jobs skip, and the aggregate goes green having built and booted nothing. Failing instead would block those examples permanently, so it is surfaced instead: the selector emits a ::warning:: naming the changed examples no matrix entry covers, and the aggregate logs either "Built and booted: <names>" or "No example was built or booted", with the same line in the job summary. A reviewer reading one green check had no way to tell the difference, which is the same property that justifies the job existing — the per-example job names vanish when the matrix is filtered. Exercised: emit_all on a renamed key exits 5 writing no bad value downstream; the prefix assertion catches a missing prefix and a bogus one while accepting `ci`; the aggregate reports both cases and still exits 1 on a real failure.
…amples Two fixes from review on 0316c21. The matrix moved out of the workflow, where the per-kind key requirements were at least documented in the surrounding comments, into a JSON file the guard checked one field of. It now asserts the exact key set each kind interpolates: image needs name, path and expect_body; zip adds port; stream swaps in kind. Unknown top-level kinds are reported too. Exact sets rather than minimums, because a typo is worse than an omission here. verify-http.sh skips the body assertion entirely when the expectation is empty (`[ -z "$EXPECT_BODY" ] ||`), so `expect_bodY` would have left the job green while checking only the status code — a silent weakening. A missing `port` is merely noisy by comparison: test-zip interpolates it into PORT=, the app does not listen where the verify step looks, and the run burns its 90-second deadline to fail with "expectation not met", which reads as a broken example. Only `stream` failed clearly, through the `*)` arm on its `kind`. The three key assertions also ran over the two entries the file itself describes as not examples. Everything else in the script is scoped to examples/ — `found` comes from `git ls-files examples`, the stale check filters on the prefix — so applying the grouping policy repo-wide turned an examples-drift guard into a policy lock: enabling version updates for the adapter's own crates is a normal thing to want, has nothing to do with example grouping, and failed validate with a message that did not hint at editing this script. `applies-to: security-updates` and the limit are now asserted for example entries only. commit-message.prefix stays unconditional, since Commit Lint runs on every pull request regardless of which directory the bump came from. Nine states exercised: clean tree; a zip entry missing port; a typo'd expect_body; a renamed top-level kind; a stream entry missing kind; the root cargo entry with a non-zero limit, which now passes; an example entry with a non-zero limit and one missing applies-to, which still fail; and github-actions with no prefix, which still fails.
…keys Five fixes from a code review of the branch. git diff had rename detection on, which prints only the destination. `git mv examples/fasthtml/app/main.py examples/fasthtml-zip/app/main.py` therefore reported the destination alone: fasthtml was never selected, its matrix entry never ran, and the aggregate went green while the example had lost its app file. Reproduced in a scratch repository — default output one path, --no-renames output both. Now --no-renames. The example-path pattern is anchored on a trailing slash. Without it a file sitting directly under examples/ matched and became a phantom example name, yielding empty matrices and a spurious "no matrix entry builds or boots: README.md" warning. Latent — there is no such file today. The pull request selection loop now shares emit_all's has() assertion. A renamed top-level key failed there with jq's bare "Cannot iterate over null", naming neither the file nor the key, while the push path said which key was missing. Both paths now report `example-matrix.json has no "stream" key`. The drift guard asserts the last two properties dependabot.yml documents as load-bearing. `patterns: ["*"]`, because a group with patterns: ["lodash"] satisfies the applies-to assertion while leaving every other advisory for that example ungrouped. And the generated commit header's length, because the file records that a group named after its example produced a 137-character header against commitlint's 120 — nothing checked it, and the longest directory configured today leaves 11 characters of headroom. Both limits are parsed from commitlint.config.js rather than copied. That parse needed two guards of its own, learned by getting it wrong: the rule is [severity, applicability, value], so a lazy match returns the severity 2 and every header looks 118 characters over budget — it failed the clean tree until the regex took the last number. A floor of 40 now rejects an implausible parse rather than crying wolf. Also records why the push trigger's path list is deliberately shorter than the selector's shared-input set: this workflow, verify-http.sh and the matrix file are pull request triggers, so a change to them already fans out to all eighteen entries before it lands, and repeating them on push would only re-run what the pull request just ran. The two lists answer different questions. Twelve drift-guard states and five selector paths exercised.
…s change Two fixes from review on 3a96664. `configured` is a set, so a directory claimed twice for one ecosystem collapsed into one member and every assertion built on it still passed — the one drift mode this guard could not see, and the likeliest one in a file of 47 near-identical entries where the copy that forgets to change the directory is as plausible as the copy that forgets a key. It now reports the entry by name. One correction to the report: such a config would not have landed green. Dependabot's own validation rejects it — "Update configs must have a unique combination of 'package-ecosystem', 'directory', and 'target-branch'" is the same check that caught the glob mistake earlier in this branch — so the .github/dependabot.yml check goes red on the pull request. What this adds is a message that names the ecosystem and directory instead of the ecosystem alone, at the same place as the other assertions. The consequence if it did slip through is as described: a rejected config means none of it applies, not that one example degrades. commitlint.config.js joins the trigger paths. The guard parses type-enum and header-max-length out of it and asserts every entry's generated header against them, but the workflow did not run when that file changed, so lowering the cap would have passed on the pull request that made the change — pr.yaml and commitlint.yaml run, neither invokes the guard — and surfaced weeks later as a red Commit Lint on a Dependabot pull request nobody wrote. Headroom is ~12 characters. It costs nothing: the selector does not treat it as a shared input, so such a pull request selects no examples and only validate does work. The duplicate check also caught a mistake of its own making: it referenced `where` before assignment, which raised NameError for a duplicate inside the first entry while appearing to work across entries, where the name had leaked from the previous iteration of the loop. `where` is now assigned above its first use, and all three shapes are exercised — twice in one entry, across two entries, and the root cargo entry colliding with an example.
…late spellings Three fixes from review on 5b7e8b6. The kind list was hardcoded in three loops with nothing tying it to the jobs. Adding a fourth job with its matrix key and its MATRIX_KEYS entry while missing one of those loops wrote no output line for it, and `!= '[]'` is true for the empty string — so that job would start and die in fromJSON(''), the exact failure emit_all's comment describes. The loops now come from `jq -r 'keys_unsorted[]'` over the matrix file, and refuse to emit anything if that yields nothing. The guard's kind check became bidirectional to close the other direction: a kind a job reads but the file does not have is now reported, not just a key no job consumes. `directories` values are normalized when collected. Dependabot resolves "/examples/fastapi/app/" and "/examples/fastapi/app" to the same manifest, but comparing verbatim reported the entry as pointing at no manifest *and* the manifest as having no entry — two contradictory problems for a config that works, the failure already fixed here for the singular `directory` key and for globs. Stripping only slashes leaves glob patterns alone, and the duplicate check now also sees `/examples/x` against `/examples/x/` as the collision Dependabot rejects the file over. The validate job lints template.yml as well as template.yaml. examples/sinatra and examples/go-http-zip use the .yml spelling, so 43 of 45 templates were linted and those two were not — while examples-verified claimed "template validation only" for them. Both have Dependabot entries and no matrix entry, so that claim was the only thing their pull requests were going to get, and it was false. Exercised: a trailing-slash entry now passes; a slashed and unslashed pair of the same directory is reported as a duplicate; a matrix missing a kind a job reads, and a matrix with a kind no job reads, both fail; a fourth kind in the file gets its output line; an unreadable matrix refuses rather than emitting an empty selection; and the twelve-state guard suite plus three selector paths still behave.
| # that the bump they are approving was never launched, because the per-example job names | ||
| # disappear when the matrix is filtered. | ||
| uncovered="$(jq -r --argjson names "$names" \ | ||
| '([.image, .zip, .stream] | flatten | map(.name)) as $covered |
There was a problem hiding this comment.
[GENERAL] The uncovered warning is the one remaining hardcoded kind list in this file, and it is the place where a wrong answer is most visible to a reviewer.
uncovered="$(jq -r --argjson names "$names" \
'([.image, .zip, .stream] | flatten | map(.name)) as $covered
| [$names[] | select(IN($covered[]) | not)] | join(", ")' "$MATRIX")"KINDS is derived from the matrix file (line 36) precisely so that adding a job kind cannot silently skip a code path — the header comment says as much: "The kinds come from the matrix file, not from three hardcoded lists." $covered does not follow that rule. Add a fourth kind (say arm) with its matrix key and its job, update MATRIX_KEYS in check-example-config.sh (which the guard forces), and this expression still only looks at .image, .zip, .stream. Every example covered only by the new kind is then reported as uncovered:
::warning::No matrix entry builds or boots: javalin-zip — this run verifies templates only for them.
...while test-arm is in fact building and booting it. That warning and the matching $GITHUB_STEP_SUMMARY line exist because the per-example job names disappear when the matrix is filtered, so this is the reviewer's only signal — a false negative claim here is worse than no message. Deriving the set from the file removes the drift:
uncovered="$(jq -r --argjson names "$names" \
'[.[][] | .name] as $covered
| [$names[] | select(IN($covered[]) | not)] | join(", ")' "$MATRIX")"The same hardcoding appears in examples.yaml's examples-verified step (the three --argjson image/zip/stream bindings feeding the "Built and booted" list), where a new kind's examples would be omitted from the summary for the same reason. That one is more defensible since the job list is per-kind anyway, but it is worth fixing together.
Contributor
Author
There was a problem hiding this comment.
the uncovered warning now derives from the file, and the gate reads one kind-agnostic selected output — a fourth kind no longer produces a false "not built or booted" claim. Fixed in the latest commit.
…t schedule Three fixes from review on b13d801. The uncovered warning still named the kinds, which is the one place a stale list misleads a person rather than a job: adding a fourth kind would have had it report an example as "not built or booted" while the new job was building and booting it — a false claim in the only signal a reviewer has, since the per-example job names vanish when the matrix is filtered. It now derives the covered set from the file, and the selector publishes one kind-agnostic `selected` output that examples-verified consumes instead of binding image/zip/stream a third time. The simulated commit header now uses the entry's own prefix, and stays deliberately pessimistic: it assumes the `(deps)` scope and a three-digit update count. That is why it reads longer than the 137 the comment cites, which was measured without the scope. The report argued for dropping the scope as too strict; I kept it, because being wrong in the strict direction costs a shortened group name while being wrong in the lax direction costs a red Commit Lint on a pull request nobody wrote — which is the whole reason the assertion exists — and every Dependabot pull request in this repository is titled `chore(deps): ...`, so the scope is what Dependabot infers here. It still leaves 9 characters of headroom at the longest directory configured today, so the pessimism costs nothing. schedule.interval is now asserted. It is required for an updates entry and its absence is the worst copy-paste failure available: Dependabot rejects the whole file, so all 47 groups stop applying at once. Asserted outside the is_example gate, since an invalid root cargo or github-actions entry invalidates the file just the same. Exercised: `selected` on all four selector paths; a fourth matrix kind honoured by both the warning and `selected`; the gate reporting from the single output and still exiting 1 on a failure; a missing schedule on an example and on github-actions.
licjun
approved these changes
Sep 17, 2026
vicheey
approved these changes
Sep 17, 2026
| # jobs would run and die in fromJSON('') with an error unrelated to the real cause. | ||
| emit_all() { | ||
| local kind matrix | ||
| echo "selected=$(jq -r '[.[][] | .name] | sort | join(", ")' "$MATRIX")" >>"$GITHUB_OUTPUT" |
There was a problem hiding this comment.
[BUG] The one remaining echo "x=$(jq ...)" in this file is the very pattern the comment six lines above forbids:
emit_all() {
local kind matrix
echo "selected=$(jq -r '[.[][] | .name] | sort | join(", ")' "$MATRIX")" >>"$GITHUB_OUTPUT"The per-kind matrices were fixed to assign first, but selected was not. A jq failure here (a kind whose value is a scalar or object rather than an array makes .[][] error) becomes an argument to echo, which exits 0, so set -e never fires and selected= is written. Meanwhile the loop below uses .[$kind], which does not iterate and so still succeeds — the matrices come out populated, the test jobs run, and examples-verified reads an empty SELECTED and prints "No example was built or booted; this run verified templates only" for a run that built and booted everything. That is a false statement in the single check a reviewer reads, which is the reason this output exists.
Note that local selected="$(jq ...)" masks the status the same way — local returns its own exit code — so the declaration has to be separate, as it already is for kind and matrix:
emit_all() {
local kind matrix selected
selected="$(jq -r '[.[][] | .name] | sort | join(", ")' "$MATRIX")"
echo "selected=$selected" >>"$GITHUB_OUTPUT"| echo "Built and booted: $SELECTED" | ||
| echo "**Verified:** $SELECTED" >>"$GITHUB_STEP_SUMMARY" | ||
| else | ||
| echo "No example was built or booted; this run verified templates only." |
There was a problem hiding this comment.
[GENERAL] "this run verified templates only" is not true for four of the examples that now receive grouped Dependabot pull requests, and the same wording is in the ::warning:: at select-examples.sh:152.
The claim rests on validate finding a template, but that step only looks at examples/*/template.y{a,}ml (-maxdepth 2). Checked against the tree: 45 templates exist, and these four examples have none at any depth —
examples/datadogandexamples/datadog-zip— CDK (cdk/app/cdk.ts), no SAM templateexamples/sls—nestjs/serverless.yml, not a SAM template and outside the findexamples/fastapi-backend-only-response-streaming— onlyREADME.mdandapp/
All four have dependabot.yml entries (npm, pip) and no example-matrix.json entry, so their pull requests get no build, no boot, and no lint — every matrix empty, build-layer skipped, aggregate green — while both messages assert templates were validated. That is the same defect as the template.yml spelling fixed earlier in this branch: the aggregate check states a verification that did not happen. Since the message is the only signal here, it should distinguish "no matrix entry, but the template was linted" from "nothing ran at all", e.g. by testing for the template alongside the matrix lookup in select-examples.sh and naming those examples separately.
| # Named individually rather than as .github/scripts/, which also holds | ||
| # check-example-config.sh — which no example is built against, so matching the whole | ||
| # directory would rebuild and boot all eighteen entries for a change to it. | ||
| if grep -qE '^(src/|layer/|Cargo\.toml$|Cargo\.lock$|\.github/workflows/examples\.yaml$|\.github/scripts/verify-http\.sh$|\.github/scripts/select-examples\.sh$|\.github/example-matrix\.json$)' <<<"$changed"; then |
There was a problem hiding this comment.
[GENERAL] The shared-input list is presented as complete ("Every input select-examples.sh treats as shared" in the workflow, "Shared inputs every example is built against" here), but .cargo/config.toml is in neither it nor the push trigger:
if grep -qE '^(src/|layer/|Cargo\.toml$|Cargo\.lock$|\.github/workflows/examples\.yaml$|...)' <<<"$changed"; then.cargo/config.toml sets the musl linkers that build-layer's cargo lambda build --target x86_64-unknown-linux-musl depends on:
[target.x86_64-unknown-linux-musl]
linker = "x86_64-unknown-linux-musl-gcc"Every image and zip example runs the artifact that command produces, so it is a shared input by this file's own definition. Because it is also absent from the pull_request paths and the push paths (examples.yaml:37-40), a pull request touching only that file never runs Verify Examples at all, and the regression surfaces later as a build-layer failure on an unrelated examples pull request — the exact gap that motivated adding layer/** to the push list and commitlint.config.js to the pull request list. Adding it to both lists costs nothing, since the file changes rarely.
| build-layer: | ||
| needs: [select] | ||
| if: >- | ||
| needs.select.outputs.image != '[]' || |
There was a problem hiding this comment.
[GENERAL] build-layer's gate re-binds the kind list that KINDS, the uncovered warning, and SELECTED were all made data-driven to avoid:
if: >-
needs.select.outputs.image != '[]' ||
needs.select.outputs.zip != '[]' ||
needs.select.outputs.stream != '[]'A fourth kind added to example-matrix.json with its own job gets an output line (the loops derive from the file) and passes the guard's bidirectional kind check, but not this condition: with only the new kind selected, all three named outputs are [], build-layer is skipped, and the new test job is skipped with it, because a job whose needs are skipped is skipped unless it opts into always(). examples-verified counts both as passes, so the aggregate goes green having built nothing — while SELECTED is non-empty and reports "Built and booted: <example>".
The selected output already answers this question without naming kinds — it is non-empty exactly when at least one matrix is non-empty, on all three code paths:
if: needs.select.outputs.selected != ''| # Deliberately absent: docker. There are 24 Dockerfiles under examples/, but Dependabot | ||
| # alerts do not cover base images, so there is nothing for a security-updates entry to | ||
| # group — including it here would fail this check for entries that should not exist. | ||
| MANIFESTS = { |
There was a problem hiding this comment.
[GENERAL] The stale assertion fails any /examples/ entry whose ecosystem is absent from MANIFESTS, so the guard rejects valid config rather than drift. found only ever contains pairs for the nine ecosystems listed here, and stale requires a matching pair for every configured example entry, so an entry for a supported ecosystem this dict does not know about — docker for the 24 Dockerfiles, or a lockfile ecosystem for the bun examples — is reported as "points at directories with no matching manifest" and turns validate red.
The docstring reasons about only one direction of this (adding docker to MANIFESTS would demand entries that should not exist); the opposite direction, adding the entry, fails too, and the message points at the maintainer's entry with no hint that the remedy is editing this script — the same defect already fixed for the open-pull-requests-limit policy lock, and against this file's own standard that "a guard that cries wolf is worse than no guard." Scoping the reverse check to ecosystems the dict can actually speak to keeps the drift detection while dropping the false positive:
stale = sorted(
(ecosystem, pattern)
for ecosystem, pattern in configured
if pattern.startswith("/examples/")
and ecosystem in MANIFESTS
and not any(...)
)
bnusunny
deleted the
ci/dependabot-grouping-and-targeted-example-verification
branch
September 17, 2026 18:01
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cuts the handling cost of the Dependabot example PRs two ways: far fewer PRs, and verification that only runs the examples a PR touches. A code owner still reviews and merges each one — Verify Examples' job is to fail the PR when the examples it touches do not build and boot.
Starting point: 69 open Dependabot PRs, 13 already
CONFLICTING, each triggering all 18 Verify Examples matrix entries, and nodependabot.ymlat all.1. Group the updates —
.github/dependabot.ymlWithout a config, Dependabot opens one PR per advisory per manifest: 13 against
examples/remix/remix-app's single lockfile, 8 againstexamples/remix-zip. Siblings in one lockfile conflict as soon as any of them lands, which is where the 13CONFLICTINGcome from.One
updatesentry per example — 47 of them, plus the adapter's owncargoandgithub-actionsfor their commit headers — each listing the manifest directories inside that example. Two things are deliberate and both were learned the hard way:directorieswith a glob plus a group produces a single PR spanning every matching directory: theBump the npm group across 17 directories with 100 updatesform, a title shape the search API finds on over 300,000 pull requests. One bad bump would then block the security fixes for every other example in the group, and a PR spanning a dozen examples with one red matrix job is unreviewable — you cannot tell which bump broke which app without reading logs.Every group is named
securityrather than after its example: Dependabot builds the commit header from the group name and the directory, and commitlint caps it at 120 characters, which three directories already exceeded (worst: 137).open-pull-requests-limit: 0keeps version updates off while leaving security updates on, andcommit-message.prefixis what keeps Commit Lint green — #799 shows what the default header costs.2. Verify only what changed —
selectjob +.github/example-matrix.jsonA lockfile bump in
examples/remix/remix-apptoday rebuilds and bootsspringboot,nextjs,deno-zipand the rest. At ~70 open PRs that cost dominates CI and produces no signal.selectdiffs the PR and emits one matrix per job kind; the example lists move to.github/example-matrix.jsonso the selector and the matrices share one source of truth.build-layeronly runs when something was selected.The diff base is the merge ref's first parent, not
github.event.pull_request.base.sha— that payload value can be older than the tip the merge ref was recomputed against, and on a real merge commit here one file became five, which would trip the shared-path rule and verify all 18. Shared inputs (src/,layer/,Cargo.*, this workflow,verify-http.sh,select-examples.sh, the matrix) still verify everything; anything unresolvable also verifies everything.Verified against real history:
3a21552examples/remix/remix-app/package-lock.jsonimage=[remix], zip/stream empty4c38d8fsrc/**34d3a29examples/nextjs-response-streaming/**The workflow now also triggers on its own machinery, so a change to the matrix or the selector is verified by the thing it configures.
layer/**andCargo.*joined the push trigger, closing a hole:layer/bootstrapis the code path all eight zip examples boot, and it was verified against the examples on no event at all.examples-verifiedaggregates the run into one result, treating a skipped job as a pass since that is what a filtered-out matrix means. It is what a code owner reads before merging — the matrix jobs' names embed their parameters and disappear whenselectfilters them out, so there is no stable per-example check name to look for.3. Guard the two hand-maintained lists —
.github/scripts/check-example-config.shRun from
validate. A manifest with nodependabot.ymlentry does not merely lose grouping: it reverts to one PR per advisory with a header Commit Lint rejects — the exact state this config exists to prevent — and stays invisible until the PRs appear weeks later. With 47 entries covering 53 manifest directories, drift is a matter of when.It asserts that every manifest under
examples/is claimed for its own ecosystem, that no entry points at a directory with no manifest, that every entry hasapplies-to: security-updatesandopen-pull-requests-limit: 0(both load-bearing and both easy to omit in a copy-paste), and that everyexample-matrix.jsonname is a real directory underexamples/.What is and is not boot-tested
The matrix covers 18 examples, which is what "verified" means for a Dependabot PR. The rest get template validation only — unchanged from today, since they were never boot-tested; they were merely sitting in runs that booted other examples.
remix13,remix-zip8,nextjs5,fastapi-background-tasks4,gin2,fastapi1,fastapi-zip1,expressjs1)datadog7,nextjs-response-streaming5,sveltekit-ssr-zip4,datadog-zip4,nextjs-zip3, others)sam validate --lintonlyThe obvious follow-up is matrix entries for the busiest uncovered examples —
datadog,nextjs-response-streaming,nextjs-zip,sveltekit-ssr-zip— which is additive and touches none of this machinery.Not included, deliberately
An earlier revision added a workflow that merged verified example-only PRs. It is gone. Unattended merging was never available —
main's ruleset requires one code-owner approval (.github/CODEOWNERSassigns*to@aws/aws-lambda-tooling) with zero bypass actors, and granting some identity the right to bypass code-owner review on a public repository is not a trade worth making for dependency bumps in demo apps. What remained after that merged an already-approved PR without a second visit, which is not worth 372 lines and a workflow holdingcontents: writewhen a code owner is already one click away.Draining the 69
Land this, then
@dependabot closethe per-advisory PRs and let the grouped ones regenerate against currentmain. One fresh, verified PR per example app beats rebasing 13 competing patches against one lockfile.Validation
dependabot.ymlparse;bash -nclean on both scripts; Dependabot's own config check passes ("All changes look good").**glob, and a missing policy key.select-examples.sh, so every push fans out to all 18 entries.