You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit bc609c8
Browse filesBrowse the repository at this point in the historyBrowse files
authored
ci: check new crates are reserved on crates.io before merge (#3185) (#3218)
* ci: check new crates are reserved on crates.io before merge
Trusted publishing can't create a crate, and the release workflow publishes
the whole workspace in one `cargo publish --workspace` run, which cargo
aborts if any crate already exists at the release version. So the existing
"publish the new crate by hand after the workflow fails" instructions no
longer work.
Add a lint step that fails if a publishable crate is missing from
crates.io, and rewrite the release guide appendix around reserving the
crate with a placeholder version when it is added.
* docs: reserve the crate name with a standalone placeholder crate
Overriding the version inside the workspace fails because iceberg depends
on the new crate at the workspace version. Verified with cargo publish
--dry-run.
* ci: match the existing publishable-crate filter; document partial publish recovery
* docs: clarify that reserving a crate needs a token with the publish-new scope
Owning the existing crates does not imply the token in ~/.cargo/credentials.toml
has publish-new, and crates.io reports a missing scope as a generic
authentication failure. Say so, and point at where to create the token.
* docs: fix stale first-publish note, empty placeholder, and owner invitations
- The release steps still said the publish workflow is expected to fail on
new crates and linked to a heading that no longer exists.
- `cargo new --lib` generates a non-empty lib.rs; use mkdir and touch.
- `cargo owner --add <user>` sends an invitation that must be accepted.
* ci: fail the crates.io check if the crate list is empty
A failed `cargo metadata` inside `for pkg in $(...)` never trips
`bash -e`, so the loop ran zero times and the step passed without
checking anything. Capture the list first and fail if it is empty.
---------
(cherry picked from commit cb912c9)
Co-authored-by: Kevin Liu <kevinjqliu@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: website/src/release.md
+76-25Lines changed: 76 additions & 25 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -531,9 +531,8 @@ The creation of the final release tag triggers the publish workflow for crates.
531
531
Python packages are manually triggered later.
532
532
Please verify that the triggered workflows for the crates succeeded.
533
533
534
-
Note, this workflow for crates is expected to fail if new crates are being published for the first time.
535
-
In this instance, a committer must manually publish the crate in order to continue.
536
-
See [publishing a crate for the first time](#publishing-a-crate-for-the-first-time) for what steps to take.
534
+
New crates are reserved on crates.io when they are added, so the workflow should not fail on a new crate.
535
+
If it does, see [if the Publish workflow fails on a missing crate](#if-the-publish-workflow-fails-on-a-missing-crate).
537
536
Once all the crates are published, Python publishing should start.
538
537
539
538
Python publishing is performed by a GitHub workflow, however the trigger is manual.
@@ -614,45 +613,97 @@ however it is in the community's best interest to publish it soon after.
614
613
615
614
## Appendix
616
615
617
-
### Publishing a crate for the first time
616
+
### Adding a new crate
618
617
619
-
Publishing the Iceberg crates is automated using GitHub Actions,
620
-
which authenticates with crates.io using trusted publishing.
618
+
Every publishable crate must exist on crates.io before the pull request that adds it is merged.
619
+
CI enforces this with a check that fails for any publishable crate missing from crates.io.
621
620
622
-
Trusted publishing must first be configured for each crate.
623
-
If the crate has never been published, crates.io rejects attempts to publish.
621
+
The release workflow publishes crates with [trusted publishing](https://crates.io/docs/trusted-publishing), which cannot create a crate.
622
+
crates.io requires the first version of a crate to be published manually with an API token ([announcement](https://blog.rust-lang.org/2025/07/11/crates-io-development-update-2025-07/)).
623
+
Publishing the crate by hand after the release workflow fails does not work either: the workflow runs `cargo publish --workspace`, which refuses to run if any crate already exists at the release version.
624
624
625
-
When a release workflow fails due to the presence of a new crate, a committer must perform steps enumerated below.
626
-
Once complete, future versions can be published automatically using GitHub Actions.
625
+
A committer therefore reserves the crate when it is added, in three steps: publish a placeholder version, configure owners, and configure trusted publishing.
627
626
628
-
#### Initial crate publish
627
+
#### Prerequisites
629
628
630
-
Manually publish the crate using the following command.
631
-
You **must** have the source code checked out matching the pushed Git tag for the release.
629
+
Any crates.io account can publish an unclaimed name, and the first publisher becomes the crate's only owner.
630
+
Reserve the crate promptly once the pull request is open, and have a committer do it, since the reserver must hand ownership to the project.
631
+
632
+
The committer needs:
633
+
634
+
- A crates.io account linked to their GitHub account.
635
+
- Membership in the `apache/iceberg-private` GitHub team, with the `read:org` permission granted to crates.io, in order to add that team as an owner.
636
+
- A crates.io API token with the `publish-new` and `change-owners` scopes, used with `cargo login`.
637
+
Existing tokens usually lack `publish-new`; create one at [crates.io/settings/tokens](https://crates.io/settings/tokens).
638
+
639
+
Committers who already own the existing Iceberg crates meet the first two.
640
+
List them with `cargo owner --list iceberg`.
641
+
642
+
#### Step 1: Reserve the crate name
643
+
644
+
Publish an empty placeholder crate as version `0.0.0`, so it never collides with a real release.
645
+
The placeholder is created outside the repository; overriding the version inside the workspace does not work because other workspace crates depend on the new crate at the workspace version.
632
646
633
647
```shell
634
-
cargo publish --package <package-name>
648
+
mkdir -p /tmp/<package-name>/src
649
+
cd /tmp/<package-name>
650
+
touch src/lib.rs
651
+
cat > Cargo.toml <<'EOF'
652
+
[package]
653
+
name = "<package-name>"
654
+
version = "0.0.0"
655
+
edition = "2024"
656
+
license = "Apache-2.0"
657
+
description = "Placeholder that reserves the crate name for Apache Iceberg Rust. See the repository for the real crate."
Additionally, add two PMC members (excluding yourself) as owners of the crate.
648
-
A GitHub team cannot manage permissions for the crate, so it is important that individuals have ownership to continue being able to manage the crate should a PMC member become inactive.
649
-
See the [Cargo documentation for `cargo owner`](https://doc.rust-lang.org/cargo/reference/publishing.html#cargo-owner) for reference.
674
+
Then add two PMC members other than yourself as individual owners.
650
675
651
676
```shell
652
-
cargo owner --add <github-handle>
677
+
cargo owner --add <github-handle><package-name>
653
678
```
654
679
655
-
#### Configure trusted publishing
680
+
This sends an invitation; ask each invitee to accept it on crates.io.
681
+
682
+
Individual owners are required because a team owner can publish and yank versions but cannot add or remove owners.
683
+
Without them, the crate could become unmanageable if a PMC member becomes inactive.
684
+
See the [Cargo documentation for `cargo owner`](https://doc.rust-lang.org/cargo/reference/publishing.html#cargo-owner).
685
+
686
+
#### Step 3: Configure trusted publishing
687
+
688
+
As a crate owner, open the crate's settings on the crates.io website and add a trusted publisher, following the [crates.io trusted publishing documentation](https://crates.io/docs/trusted-publishing).
689
+
Use an existing Iceberg crate as the reference:
656
690
657
-
Review the [crates.io trusted publishing documentation](https://crates.io/docs/trusted-publishing) for the latest instructions on how to configure it.
658
-
You should also review another already-existing crate for reference.
691
+
- Repository: `apache/iceberg-rust`
692
+
- Workflow: `publish.yml`
693
+
- Environment: `publish`
694
+
695
+
Then re-run CI on the pull request.
696
+
The next release publishes the crate's first real version along with the rest of the workspace.
697
+
698
+
#### If the Publish workflow fails on a missing crate
699
+
700
+
Reserve the crate as above, using the `0.0.0` placeholder and never the release version.
701
+
702
+
Then check the failed run's log for `Uploaded` lines to see which crates were already published, since cargo uploads crates one at a time in dependency order.
703
+
704
+
- If nothing was uploaded, re-run the failed workflow run from the GitHub Actions UI. Re-running keeps the original tag as the workflow ref.
705
+
- If some crates were uploaded, publish the rest from a clean checkout of the release tag, excluding each crate that was already published:
0 commit comments