Repository navigation
Merge 0.48.2-RC2 back into master - #3871
Merged
Merged
Conversation
Version/0 48 1 rc4
* Update version.txt * 0.48.2-RC1: #3832, #3858, #3857 on top of 0.48.1-RC4 (#3863) * Speed up file installation on Windows (#3832) * Resolve link paths once per tree instead of once per file smartlink.Link resolved both src and dest via ResolveUniquePath for every entry it recursed into. On Windows ResolveUniquePath calls GetLongPathName, which is a syscall, so installing a runtime with many files performed two extra syscalls per file - a major contributor to slow Windows installs. The tree root is already resolved by LinkContents (and now by the public Link), and descendant paths are built by joining an already-resolved parent with real (long) entry names, so they need no further resolution. Split the recursion into an unexported link() worker that assumes resolved inputs and does not re-resolve, keeping resolution only for the rare symlink-to-file case where the pre-existing behavior links to the dereferenced target. Also treat an "already exists" error from linkFile the same as the existing TargetExists skip, so concurrent deploys of a shared file degrade to the documented "first one wins" behavior rather than failing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Deploy linked artifacts concurrently instead of serializing them DeployViaLink held the depot-wide fsMutex for the entire link operation, so even though the install worker pool runs several artifacts at once, the actual file linking was serialized to one artifact at a time. On Windows, where each link is an individual (relatively slow) syscall, this made the install phase far slower than it needed to be. Linking touches no shared in-memory depot state (Exists guards its own map via mapMutex and depotPath is immutable after construction), directory creation is idempotent, and smartlink now treats an already-existing target as a skip, so it is safe to run without the lock. Drop it from DeployViaLink so linking parallelizes across the worker pool. DeployViaCopy keeps the lock: two artifacts copying the same file would race on os.Create and could corrupt the destination. That path is only used for artifacts needing file transforms, when hard links are unsupported, or in portable mode, so it remains the less common case. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Hold a read lock while linking so copies can't write through links Linking without the depot lock let a concurrent copy deployment write through a link created moments earlier: copyFiles checks the destination is free, a link to another artifact's depot file appears there, and os.Create then truncates and overwrites that depot file, corrupting the shared cache. TestConcurrentCopyAndLinkKeepDepotIntact reproduces it in every run without the lock. Make fsMutex an RWMutex: DeployViaLink holds it for reading, so links still run concurrently, while copies, Put and Undeploy hold it for writing. Also fix the non-constant format string in DeployViaCopy, which made go test's vet step fail the whole package. Add smartlink tests for an existing target, concurrent links to the same destination, and the Link refactor's handling of directories and symlinks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Skip per-file metadata calls when linking link() checked each file with IsDir, IsSymlink, MkdirUnlessExists on its parent and TargetExists before linking it, plus another IsDir inside the Windows linkFile. Each is a file open on Windows. Instead: - take the entry type from the directory listing (os.ReadDir already has it), so only symlinks need a Stat; - rely on the parent directory existing, since the recursion creates it (Link creates it for its single entry); - drop the TargetExists pre-check and treat the link call's "already exists" error as the skip, which already covered concurrent links; - check for a directory only on the symlink-to-directory path, which is the only one that can pass one to linkFile. A regular file now costs one syscall: the link itself. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 8f473b3) * Remediate CVE-2026-63209 in klauspost/compress (#3858) Bump github.com/klauspost/compress (indirect, via mholt/archives) from v1.18.0 to v1.18.7, the first patched release for GHSA-259r-337f-4rfw (integer overflow and out-of-bounds write in s2). Re-vendored. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit b09f7ee) * Support dependencies for encrypted private ingredients (#3857) * Support dependencies for encrypted private ingredients `state publish --build` can now declare and edit the dependencies of an encrypted private ingredient: - Read [project].dependencies from pyproject.toml and publish them as runtime dependencies on language/python packages, with names normalized by the inventory API. Version ranges carry over (~= and ==X.* are expanded). OS and Python version markers become conditions (kernel/Windows|Linux|Darwin, language/python); unsupported markers are skipped with a notice. The wheel's METADATA records them as Requires-Dist. - Read the --meta file before building, so a meta file's name, version and description feed the wheel (flags > meta file > pyproject.toml). - Add `state export ingredient-meta <namespace>/<name>[@Version]`, which writes an existing ingredient's meta file for use with --edit --meta. Fix --edit along the way: keep the meta file's dependencies and authors instead of overwriting them with the previous revision's, keep inherited dependency types and conditions, add the private-builder dependency only once, and reject author/description changes on an existing version up front instead of failing in the API. InventoryRequirementsToString no longer panics on requirements without a version. Add a readme walking through the publish and edit workflow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Address review: keep inherited deps, edit the meta file's version - When every pyproject.toml dependency is skipped, keep the dependencies already in place (on --edit, those inherited from the previous revision) instead of replacing them with an empty list. - Let a meta file's version select the version to publish or edit, like --version (which still takes precedence). An exported meta file passed back with `state publish --edit --meta` without --build was editing the latest version instead of the exported one. Add tests for both, for the up-front author/description rejection on --edit, for InventoryRequirementsToString with requirements missing a version or comparator, and for inherited conditions that carry requirements. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 44e750f) * Add 0.48.2 changelog. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * check-format: don't fail when a PR changes no files. With an empty diff, grep -v exits 1 and set -e aborts the script with no output, so a PR that changes nothing (e.g. merging beta into a version branch) fails Check Format on every OS. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Update to Go 1.26.10 and golang.org/x/net v0.60.0 to remediate CVEs. The CI Trivy scan fails on 13 Go stdlib CVEs (fixed in 1.26.9/1.26.10) and 11 golang.org/x/net findings (fixed in v0.60.0) published since the 0.48.1 scan. Re-vendored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Compare RSA keys by value in keypair tests. The tests compared a generated keypair with the same key parsed back from PEM using deep struct equality, which also compares rsa.PrivateKey's precomputed internals. Under Go 1.26.10 those can differ (e.g. a leading zero byte in dP) even though the keys are identical, so the tests failed intermittently. Compare with rsa.PrivateKey.Equal instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Allow for slow sw_vers in the macOS OSVersion performance test. (#3866) TestPerformance/OSVersion averages ten calls against a 10ms budget, so the single uncached call bounds it at ~100ms. On macOS that call shells out to sw_vers, which took 115-716ms on macos-latest runners and failed 3 of 4 runs. Cached calls take nanoseconds. Keep 10ms elsewhere; allow 100ms on darwin. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: mitchell <mitchellb@activestate.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Update version.txt * Fix state publish meta data issues found testing 0.48.2-RC1 (#3867) * Fix state publish meta data issues found testing 0.48.2-RC1 - state export ingredient-meta writes plain yaml, unwrapped and without field help comments, so the file can be passed back with --meta. - Dependencies and conditions without a version range (a bare name in pyproject.toml, --depend without a version, or a condition inherited without requirements) get >=0. The platform rejected them with "Expecting an OBJECT with a field named 'versionRequirements'". - When --build can't determine the name or version, say which is missing, why pyproject.toml didn't provide it (no file, not in [project], or listed in [project].dynamic), and how to set it. - Answering 'n' at the publish confirmation offers to edit the meta data in the editor or save it to <name>-<version>.meta.yaml with the command to publish it, instead of only cancelling. With --build or --edit, a name, namespace or version change in the editor is saved to a file instead, as the package was already built and looked up with them. - Fields removed in the editor are removed from the request, and the publish API error is shown instead of <no value>. The platform requires the source with every publish, so --edit still needs --build or an archive; the readme now says so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Update echo to v4.16.1 and drop an unused publish error argument echo v4.15.4 has GHSA-cxf3-9h37-7rr8 (critical), GHSA-9969-6676-49h5, GHSA-f48p-c9v4-h7xv and GHSA-h7wr-284f-5qvc, which failed the Scan job. err_uploadingredient_publish is defined in en-us.yaml without a {{.V0}}, so the joined message passed to it was never shown. The wrapped error is already printed beneath it, so drop the argument and the changelog claim. * Keep the API error in the publish failure message err_uploadingredient_publish in en-us.yaml ends in "Error received: {{.V0}}" and overrides the fallback text, so without the argument the message reads "<no value>". The wrapped API error isn't user-facing, so it isn't printed beneath it either; it only reaches the log. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Fix TestOpen failing on Windows; correct the publish error comment TestOpen removed its temp file while the hail watcher could still be reading it for a later write event: the remove was deferred after the cancel, so it ran first. On Windows an open file can't be removed. Stop the watcher and drain its channel before removing the file. The publish error comment said the wrapped API error is never printed; a transport failure becomes a user-facing BuildPlannerError and is. Only an error reported in the publish result (an errs.New) relies on {{.V0}}. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Validate each confirm pass and test the publish fixes through their call paths Address review on #3867: - Run the namespace and checkEditKeepsIdentity checks at the top of each pass of the confirm loop, now confirmRequest, so a namespace cleared or a description added in the editor is rejected with a clear message instead of failing in MarshalJSON or the API. - Add tests that fail if a fix is reverted: confirmRequest with a fake prompter (>=0 filling, editor removing fields, cleared namespace, description on a revision, private-builder restored, save, cancel), using the test binary as $EDITOR so it also runs on Windows; publishFailedError keeping the API error; metaFromIngredient filling conditions; and writeIngredientMeta writing plain yaml that round-trips. FetchMeta's pure part moves into metaFromIngredient and the export's writing into writeIngredientMeta so they can be tested without the API. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * Fix CI for 0.48.2-RC2: x/net CVEs, publish --editor, and runner-specific test failures (#3869) * Update golang.org/x/net to v0.62.0 for CVE-2026-78661 and CVE-2026-97045 The Scan job failed on both: x/net v0.60.0 is affected and v0.62.0 fixes them. go get also raised x/crypto, x/sys, x/term, x/text, x/sync and x/tools to the versions x/net v0.62.0 requires. * Don't search every namespace when publishing a new ingredient without one state publish --editor outside a project has no namespace until the editor fills it in, so Run searched every namespace for the name derived from the file. Logged in, that search took long enough to time out ("Could not search for ingredient"), so the editor never opened. This is why the "New ingredient with editor flag" integration test has failed nightly on master since at least September. Its first match could also be any ingredient with that name. Skip the lookup for a new ingredient without a namespace; confirmRequest requires one and checkNamespaceEdit checks the version once it is filled in. --edit still searches, since it needs to find the ingredient. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Let state publish --editor continue when no editor can be opened Without EDITOR set and without xdg-open, as on many headless Linux machines and the ubuntu-24.04-arm runner, OpenEditor fails and publish exited with "Could not publish ingredient. Error received: <no value>", even though the notice just before it says the file can be edited by hand. Say that no editor could be opened and wait for the confirmation as usual instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Fix integration tests that fail only because of the runner or branch name - TestPublishBuildEncrypted installs into a Python project that state init creates without a Linux/arm64 platform, so skip it there, like the other platform-specific skips. Integration tests only run on ARM for labelled PRs. - PerformanceIntegrationTestSuite's TearDownSuite stopped state-svc even when its tests were skipped by tag. On macOS a long branch channel makes the socket path too long to dial, so every labelled PR from such a branch failed there. Only stop the service when the Performance tests ran, using a new tagsuite.RunsForTags that OnlyRunForTags is now built on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Don't open a real editor in the publish --editor integration test The test writes its edits to the file itself, but with no EDITOR set, macOS `open` hands the file to whatever app the runner has registered. On macos-latest, publish then read the file back without the test's edits and failed with "You have to supply the namespace". Point EDITOR at a command that doesn't exist, so publish takes the "edit the file by hand" path on every OS. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Fix the publish --editor integration test's flaky file path match editorFileRx matched "file:\s*?(.*?)\.\s", whose lazy \s*? left the space after "file:" in the captured path whenever the temp file name had 9 digits instead of 10 (about one run in four). The test then wrote its edits to " /tmp/NNN.ingredient.yaml", a different file, so publish read the original back and failed with "You have to supply the namespace". The last commit's macOS explanation was wrong; this was the cause on every OS. Reproduced by driving a dev build through termtest 12 times: three runs captured the leading space before this change and none after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * Make the state publish meta data clearer to review and edit (#3868) * Update golang.org/x/net to v0.62.0 for CVE-2026-78661 and CVE-2026-97045 The Scan job failed on both: x/net v0.60.0 is affected and v0.62.0 fixes them. go get also raised x/crypto, x/sys, x/term, x/text, x/sync and x/tools to the versions x/net v0.62.0 requires. * Don't search every namespace when publishing a new ingredient without one state publish --editor outside a project has no namespace until the editor fills it in, so Run searched every namespace for the name derived from the file. Logged in, that search took long enough to time out ("Could not search for ingredient"), so the editor never opened. This is why the "New ingredient with editor flag" integration test has failed nightly on master since at least September. Its first match could also be any ingredient with that name. Skip the lookup for a new ingredient without a namespace; confirmRequest requires one and checkNamespaceEdit checks the version once it is filled in. --edit still searches, since it needs to find the ingredient. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Let state publish --editor continue when no editor can be opened Without EDITOR set and without xdg-open, as on many headless Linux machines and the ubuntu-24.04-arm runner, OpenEditor fails and publish exited with "Could not publish ingredient. Error received: <no value>", even though the notice just before it says the file can be edited by hand. Say that no editor could be opened and wait for the confirmation as usual instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Fix integration tests that fail only because of the runner or branch name - TestPublishBuildEncrypted installs into a Python project that state init creates without a Linux/arm64 platform, so skip it there, like the other platform-specific skips. Integration tests only run on ARM for labelled PRs. - PerformanceIntegrationTestSuite's TearDownSuite stopped state-svc even when its tests were skipped by tag. On macOS a long branch channel makes the socket path too long to dial, so every labelled PR from such a branch failed there. Only stop the service when the Performance tests ran, using a new tagsuite.RunsForTags that OnlyRunForTags is now built on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Don't open a real editor in the publish --editor integration test The test writes its edits to the file itself, but with no EDITOR set, macOS `open` hands the file to whatever app the runner has registered. On macos-latest, publish then read the file back without the test's edits and failed with "You have to supply the namespace". Point EDITOR at a command that doesn't exist, so publish takes the "edit the file by hand" path on every OS. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Fix the publish --editor integration test's flaky file path match editorFileRx matched "file:\s*?(.*?)\.\s", whose lazy \s*? left the space after "file:" in the captured path whenever the temp file name had 9 digits instead of 10 (about one run in four). The test then wrote its edits to " /tmp/NNN.ingredient.yaml", a different file, so publish read the original back and failed with "You have to supply the namespace". The last commit's macOS explanation was wrong; this was the cause on every OS. Reproduced by driving a dev build through termtest 12 times: three runs captured the leading space before this change and none after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Make the state publish meta data clearer to review and edit Follow-up to #3867 from testing the 0.48.2-RC2 build: - Without --namespace, a --build publish from a pyproject.toml now defaults to private/<org>/python (otherwise private/<org>, as before). - A new ingredient's namespace can be changed in the editor. It is checked like --namespace and put back, with the reason, if it can't be used. - The confirmation and editor say up front which fields can't be changed and why, and the error for changing one names it. - The preview and editor show plain yaml without a help comment on every field; the fields are described in the publish guide instead. The yaml-comment module is no longer used and is removed. - state export ingredient-meta now includes the ingredient's description. state publish --edit no longer refuses a different description on an existing version: it keeps the current one and says the change is ignored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Brings the 0.48.2-RC2 fix-ups back to master: the state publish meta fixes (#3867, #3868), the CI fixes (#3869), Go 1.26.10 and the x/net v0.62.0 CVE bumps. version/0-48-2-RC2 was deleted after #3870 merged, so this merges beta at f6957d4, which is that branch's squash. Conflicts: master already had RC1's copy of each publish file, so the RC2 side is taken. tagsuite.go keeps master's linux/arm64 skip in OnlyRunForTags and RC2's RunsForTags split. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MDrakos
approved these changes
Nov 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merges the released 0.48.2-RC2 (now live on beta as
0.48.2-SHAf6957d4) back into master. RC1 was merged back in 904939d; this brings the RC2 fix-ups:state publishmeta data fixes (echo 4.16.1)--editortests, runner-specific test failures)version.txt→0.48.2-RC2, as the earlier merge-backs didversion/0-48-2-RC2was deleted after #3870 squash-merged, so this mergesbetaat f6957d4, the squash of that branch.Please merge with a merge commit, not a squash, so the next release's range starts after it.
Conflicts
internal/testhelpers/tagsuite/tagsuite.go: keeps master's linux/arm64 skip inOnlyRunForTagsand RC2'sRunsForTagssplit. The arm64 skip runs first.Checked locally
go build ./...andgo mod verifypass, and the integration tests compile.-vet=off, as CI runs them). The vet printf findings in deptree, secrethelper and e2e predate this; beta has them too.Part of PROD-240.
🤖 Generated with Claude Code