Skip to content

Claim new flyteplugins PyPI names at merge time, not release time - #1538

Merged
cosmicBboy merged 2 commits into
mainfrom
nielsb/claim-plugin-names
Sep 4, 2026
Merged

Claim new flyteplugins PyPI names at merge time, not release time#1538
cosmicBboy merged 2 commits into
mainfrom
nielsb/claim-plugin-names

Conversation

@cosmicBboy

Copy link
Copy Markdown
Collaborator

Why

PyPI limits each account to creating 4 new projects per 24 hours (pypi/support#10572). Uploads to existing projects are unlimited — the limit only bites on the first-ever publish of a name. This broke the v2.7.0 release (run): flyteplugins-github and flyteplugins-llamacpp were new names and got 429 Too Many Requests after earlier new names in the same run consumed the budget.

Separately, a flyteplugins-* name that sits declared in the tree but unpublished is squattable by anyone — flyteplugins-redis, -echo, -nsight, and -agento11y were lost exactly that way and are now quarantined on PyPI (PEP 541 requests pending).

What

New claim-plugin-names.yml workflow:

  • pull_request (touching plugins/**/pyproject.toml): fails early if a single PR adds more than 4 new flyteplugins-* distributions, telling the author to split the PR and why (the PyPI new-project rate limit).
  • push to main: immediately publishes a 0.0.0a0 claim wheel for every still-unregistered name, so the PyPI project is registered the day the plugin merges instead of at the next release. If more than 4 names are somehow unregistered on a push (PR check bypassed), it fails loudly before publishing anything.
  • daily schedule / workflow_dispatch: drains any backlog at 4/day (e.g. names PyPI later frees from quarantine become 404 and get claimed automatically).

0.0.0a0 is a pre-release, so default pip resolution never picks the claim wheel over a real release.

Also adds a cross-reference comment in publish.yml's discover-plugin-packages job: by release time every discovered name should already exist, so releases only upload to existing projects and never hit the creation limit.

Validation

Dry-ran both scripts against the live repo/PyPI:

  • name-diff logic: v2.6.10→HEAD correctly reports 6 new distributions; HEAD→HEAD reports 0
  • discovery over all 44 in-tree names: exactly 2 currently unclaimed (flyteplugins-github, flyteplugins-llamacpp); the 4 quarantined names correctly read as already-registered (HTTP 200) and are skipped
  • both workflow files parse as valid YAML

First run of this workflow will claim flyteplugins-github and flyteplugins-llamacpp (2 ≤ 4 budget).

Notes for reviewers

  • Consider making the check-pr-plugin-count job a required status check on main so oversized plugin PRs can't merge.
  • The claim job is gated to github.repository == 'flyteorg/flyte-sdk' and uses the same PYPI_USERNAME/PYPI_PASSWORD secrets as publish.yml.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NwQixBcyR5va6BC75jaQx3

cosmicBboy and others added 2 commits September 4, 2026 13:03
PyPI limits each account to creating 4 new projects per 24h
(pypi/support#10572), which broke the v2.7.0 release for the new
flyteplugins-github and flyteplugins-llamacpp packages (429), and an
unpublished name is squattable by anyone -- four flyteplugins names
were lost exactly that way and are now quarantined.

Add a claim-plugin-names workflow that:
- fails a PR early (with the rate-limit rationale) when it adds more
  than 4 new flyteplugins distributions, so it gets split before merge;
- publishes a 0.0.0a0 claim wheel for every still-unregistered name as
  soon as a plugin lands on main;
- drains any backlog at 4/day on a daily schedule (e.g. names PyPI
  frees from quarantine later).

Release runs of publish.yml then only ever upload to existing projects
and never hit the new-project creation limit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NwQixBcyR5va6BC75jaQx3
@cosmicBboy
cosmicBboy marked this pull request as ready for review September 4, 2026 18:57
@cosmicBboy
cosmicBboy enabled auto-merge (squash) September 4, 2026 19:15
@cosmicBboy
cosmicBboy merged commit 156d378 into main Sep 4, 2026
175 checks passed
@cosmicBboy
cosmicBboy deleted the nielsb/claim-plugin-names branch September 4, 2026 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants