[aashishpanthi]

Summary

Extends the translation tracker with:

1. Auto-close – When a translation PR is merged, find referenced tracker issues, verify the expected file was added/modified (not deleted/renamed), strike that language, remove its lang-* label, and close the issue when nothing remains.
2. Translation coverage – Publish a public coverage report at public/translation-status/coverage.json (served as /translation-status/coverage.json).

Addresses #1404.

Auto-close issue

How an issue is found

Relying on Resolves #123 did not worked well with real contributors: most translation PRs either use no linking keyword or mention the issue somewhere other than the body. So any #123 is treated as a candidate, and it is looked for in:

• the PR title and body
• review summaries (maintainers can/do write "this handles #123" during review)
• issues that cross-reference the PR from their own thread
• pasted https://github.com/<owner>/<repo>/issues/123 links

Also, comments written by bots are ignored, so the tracker's own automation cannot feed itself.

How false positives are avoided

Matching every #123 is deliberately greedy, so candidates are filtered:

• fenced code blocks and inline code spans are stripped first, which removes most hex colors and code samples
• owner/repo#123 cross-repo references and #123abc are not matched; numbers are capped at five digits because color codes are six digits and we are in four digits, we'll reach six digits in years (needs attention)
• the PR's own number is skipped
• each remaining candidate must exist, must not be a pull request, and must carry the needs translation label

How an issue is updated

Which languages a PR translated is derived from its changed file paths, for example src/content/examples/es/... means Spanish. For each such language, the action removes the lang-<code> label and strikes that language's line in the issue body. The issue is closed only when no lang-* labels are left.

One tracker issue often covers several languages for the same English file, which creates a wrinkle: GitHub closes an issue natively the instant a PR saying Fixes #123 is merged, even when only one of four languages was done. The action detects this and reopens the issue, leaving a comment naming the languages still working.

Translation coverage

• Workflow: translation-coverage.yml — runs on push to main for content changes, or via Actions → Translation Coverage → Run workflow.
• Script: .github/actions/translation-tracker/coverage.js — filesystem-only scan (present / missing / stub). Andd excludes reference.
• Commits coverage.json as a public file for the visualization.
• Manual inputs: languages, content_types which can be dispatched from web.

Test plan

• Merge a fork PR that updates the correct translation file and references a tracker issue → label removed, line struck, issue closed/reopened correctly. See here
• Run Translation Coverage from the Actions UI with defaults → coverage.json updates quickly. See here
• Confirm /translation-status/coverage.json is publicly reachable after deploy.

[aashishpanthi]

Note: This branches off the stub-file-generation PR. So, #1473 should be merged first, then this.

[Divyansh013]

Hi @aashishpanthi , I see one blocker here for which I'm not very sure.

For any PR opened from a fork (which is how virtually all external translation contributors submit), GitHub downgrades GITHUB_TOKEN to read-only regardless of the permissions: block. This workflow needs issues: write to remove labels, edit the body, comment, and close/reopen, all of those API calls (removeLabel, issues.update, createComment) will return 403 for fork PRs.

The whole motivation of the PR ("real contributors" not using linking keywords) implies fork-based PRs, so as written the feature might work only for maintainers pushing branches inside the repo.

Can we test this on a fork?

cc: @ksen0 do you have any views here?

[aashishpanthi]

Sorry for the very delayed response, but I created a new GitHub account just to try the new changes. As you can see, aap009#15

Here, the write on issues is working. In the workflow file, I changed the workflow trigger from pull_request to pull_request_target. Exactly this.

Let me know if any problems arise.

[ksen0]

Thanks @Divyansh013 for the excellent point, this is an important concern to deal with. @aashishpanthi you're right that technically pull_request_target would be able to access the elevated permissions afaik and I believe this is reasonable in this case.

We have many GHA workflows across the p5 projects so maintenance can be an issue. This is the most specialized one. Ideally, it should be as least-privileged as possible at every step. In this case, no fork-controlled code is executed without a manual merge; the code that is checked out and run is just the website main branch. If this is compromised, then that is a big problem overall. As long as merging is subject to manual review, this is ok.

(If merging becomes automated / skips human review, or if reviewer misses something on a non-translation PR that changes the files in this rather complex workflow and creates higher risk, that could cause a severe problem. But this is generally true, that's why it takes so long to review/merge things - because there's high potential risk for more complex PRs)

I think this is well-constructed as is but introduces a nontrivial supply chain surface to add to the security handbook, draft here: processing/p5.js#9318, eg, you could propose something like:

• T-x Malicious PR from a fork. Can link to https://docs.github.com/en/actions/reference/security/secure-use#good-practices and note the mitigation that untrusted code must never be run or influence execution. So even if pull_request_target is used, any subsequent execution must be conditional on human review.
• please feel free to comment on Perminder's PR with any other additions that help cover the GHA of translation. He's worked on auditing actions in the main p5.js repo but this is more complex than anything there

I think it's also worth adding a warning in the GHA itself as a comment, after line 26 that actions must never checkout untrusted code (eg from an unmerged fork), never use ${{ github.event.pull_request.title }} (or any untrusted text: commit message, body, branch name) in a run:, and never delete this comment. Such a warning also helps reviewer flag risky workflow changes.

[Divyansh013]

This function returns every file including status: "removed"/"renamed", and neither identifyLanguagesFromFiles nor the verifiedLanguages filter checks status:

const verifiedLanguages = languages.filter((lang) => {
const paths = expectedPaths.get(lang);
if (!paths || paths.length === 0) return false;
return paths.some((p) => changedFileNames.includes(p)); // matches removed files too
});

A PR that deletes src/content/examples/es/foo.mdx would strike Spanish, remove lang-es, and potentially close the tracker issue - the opposite of the truth. Filter to file.status === 'added' || 'modified' (or exclude removed) before deriving languages and verifying paths.

[aashishpanthi]

I resolved it, now we check if the file was updated, added, or deleted.

[aashishpanthi]

Also, latest update:

I have added the translation coverage to this PR. A detailed description is found above in PR description. But it can be dispatched from GitHub Actions. See one here.

This is what the dispatch looks like:

[ksen0]

Hi @aashishpanthi ! Sorry for the wait. For this PR:

1. I think it should be merged into v1, when it's ready. In its current state, it cannot be merged to v2 easily.
2. For getting this on v2, I made this demo: Tracker v2 site [demo - do not merge] #1632 where I included the cherry-picks from last year and this year. This is just a demo please feel free to either do it yourself (that way you are the main author of all the commits, instead of me also being there) or let me know that you'd like me to maintain and merge the v2 PR. If there are updates to this branch it should of course be also updated there. There's a test failure there right now I have not investigated, but I'll take a look at your comments above first!

[ksen0]

Many thanks for this excellent work! Some hardening comments as well, I can rereview promptly when done

[ksen0]

Needs package.json and lockfile then npm ci not npm install because this is a live supply chain surface in a privileged context, and then is not reproducible.

[ksen0]

In general, SHA-pin rather than tag-pin actions, eg https://github.com/processing/p5.js/tree/main/.github/workflows

[ksen0]

cc @perminder-17 @Vaivaswat2244 Please flag anything I might be missing for GHA hygiene? Also if this inspires any edits to Perminder's handbook PR, please do comment there too: processing/p5.js#9318 (Note: this will ultimately target main but due to branch divergence and for review sanity it's targeting v1 for now.)

[ksen0]

I think probably a good idea to do

concurrency:
  group: translation-auto-close
  cancel-in-progress: false

to queue them up. In practice some stewards merge multiple items in a row. May also be worth testing concurrent merges of different languages for same file? So they should result in sensible issue body update

[ksen0]

Same as above: Needs package.json and lockfile then npm ci not npm install because this is a live supply chain surface in a privileged context, and then is not reproducible.