Merge v2 into dev to unblock release pipeline - #644
Closed
naveenku-jfrog wants to merge 36 commits into
Closed
Conversation
…tension (#612) Adds a GitHub Actions workflow + two ADO pipelines that automate the previously manual pre-release sanity check, and add the OIDC regression coverage that was missing when PR #608 introduced the OIDC bug (later fixed in #609). Files added: - .github/workflows/e2e-plugin-tests.yml 3-job workflow: build/install, sanity, OIDC. Triggered on the "safe to test" PR label, with a concurrency lock to prevent two PRs racing in the dev org. - .pipelines/ado-sanity-pipeline.yml Replaces the manual sanity test (ToolsInstaller, GenericArtifacts round-trip, Maven, BuildInfo, DiscardBuilds). - .pipelines/ado-oidc-regression-pipeline.yml 9 parallel stages covering every configure*Server() code path with both OIDC and non-OIDC service connections. - buildScripts/trigger-ado-pipeline.sh Bridge script: triggers an ADO pipeline via REST API and polls for completion. Files modified: - buildScripts/publish-private.sh Adds optional EXTENSION_VERSION_OVERRIDE so CI can publish a traceable version (0.<PR>.<run>) instead of random numbers. Why --- Before this change the project had no automated test that: * installed the built .vsix into a real Azure DevOps org * ran tasks as genuine Azure Pipelines steps * validated OIDC authentication for any task type The OIDC regression in PR #608 went undetected because TaskMockRunner cannot simulate the Azure Pipelines agent or the jf eot OIDC token exchange. The fix in #609 had no automated regression guard before this change. One-time setup required (per dev org) ------------------------------------- * Create Marketplace publisher (https://aka.ms/vsm-create-publisher) * Create the 8 service connections + 14 Artifactory repos referenced in pipeline parameter defaults * Import both .pipelines/*.yml into ADO and record their definition IDs * Add 6 GitHub secrets: ADO_E2E_ORG, ADO_E2E_PAT, ADO_SANITY_PROJECT, ADO_SANITY_PIPELINE_ID, ADO_OIDC_PROJECT, ADO_OIDC_PIPELINE_ID Per-PR usage ------------ A reviewer adds the "safe to test" label; the workflow then publishes a private .vsix tagged 0.<PR>.<run>, installs it into the dev org, and runs both ADO pipelines in parallel. Three new checks appear on the PR. Co-authored-by: Cursor <cursoragent@cursor.com>
The e2e workflow added in #612 fails before reaching Marketplace because the freshly-built .vsix is currently ~65MB, which trips the local 40MB guard in publish-private.sh. The bloat is a separate node_modules regression (a transitive dep grew); it does not exist in the existing cloudTests.yml flow because that workflow runs `npm t` (mock-based tests) and never builds a .vsix. This change: * Adds an opt-in SKIP_VSIX_SIZE_CHECK env var to publish-private.sh. Defaults to false, so all existing manual/local usage is unchanged. * Sets SKIP_VSIX_SIZE_CHECK=true in the e2e-plugin-tests workflow only. Marketplace still enforces its own size limit at upload time, so we are not hiding any real production failure - only removing a pre-flight that is currently stricter than upstream. A separate issue will track the node_modules bloat itself. Co-authored-by: Cursor <cursoragent@cursor.com>
…ifest (#614) Three failure modes have been observed running publish-private.sh in GitHub Actions: 1. After a successful first publish, every subsequent run fails with "The extension already exists" even after `tfx extension unpublish` returns 200 OK. The publisher UI shows zero extensions, but the publish API still rejects creation, suggesting Marketplace keeps a tombstoned record after unpublish that the API cannot overwrite within reasonable CI time. 2. tfx's --publisher flag is meant to override the manifest's publisher field, but in some versions the manifest value silently wins and the publish lands in the wrong publisher (or conflicts with a public publisher of the same id). 3. Marketplace's APIs are eventually consistent, so retries within a short window also fail. Changes: * Force the manifest's "publisher" field to ${PUBLISHER} in-place via sed before any tfx call, removing any ambiguity about where the extension lands. * Skip the unpublish step. Each run now simply publishes a new version (the version override is already unique per CI run: 0.<PR>.<run>). Old versions accumulate in the private publisher, which is harmless for a dev publisher. Set REPUBLISH_FROM_SCRATCH=true to restore the old behaviour for one-off manual cleanups. * Keep the publish retry (3 attempts, 45s backoff) as a safety net for transient Marketplace flakiness on the first run after a manual cleanup. * Tolerate failures from unshare/unpublish (|| true) so the optional REPUBLISH_FROM_SCRATCH path doesn't crash on a clean publisher. Co-authored-by: Cursor <cursoragent@cursor.com>
Even after PR #614 stopped calling `tfx extension unpublish`, the e2e workflow still fails at the publish step: Checking if this extension is already published It isn't, create a new extension. error: The extension already exists. The retry loop and the manifest publisher fix did not help. The publisher UI is empty, but Marketplace's API persistently rejects new extensions with the id "jfrog-azure-devops-extension" on this private publisher - presumably because Marketplace reserves a permanent record of every (publisher, id) pair it has ever seen, and the first successful publish (run #4) left a record that no amount of waiting, retrying or unpublishing can clear within reasonable CI time. Switch the CI build to a fresh extension id ("jfrog-azure-devops- extension-e2e") that Marketplace has never seen, so create succeeds the first time and version-bumps thereafter. Tasks inside the .vsix are referenced by their own GUIDs, not by the extension id, so the sanity and OIDC pipelines need no changes. The new EXTENSION_ID_OVERRIDE env var keeps the manifest's original id the default for local manual usage, and is opted into only by the e2e workflow. Co-authored-by: Cursor <cursoragent@cursor.com>
…nual UI upload (#616) Marketplace's eventual-consistency + (publisher, extension-id) tombstoning makes publish-from-CI unreliable: even with per-run unique ids, --no-wait-validation, and auto-uninstall of stale extensions, runs still hit "The extension already exists" intermittently and block every PR. Pivot to a manual + automated hybrid: 1. CI builds the .vsix and uploads it as a workflow artifact (still validates that the PR compiles + packages cleanly end-to-end). 2. Author manually uploads the .vsix via the Marketplace UI and installs it in the dev org — roughly 3 minutes, only required when actually testing PR code changes. Step-by-step instructions are written to the run summary and stdout, including direct deep-links to the publisher page. 3. The ADO sanity + OIDC pipelines still trigger automatically and run against whatever .vsix is currently installed in the dev org. Trade-off: 3 manual minutes per PR (only when re-installing) in exchange for 100% reliability. The publish step is exactly the kind of one-click flow the Marketplace UI handles well and the API doesn't. publish-private.sh is left untouched for local developer use; the workflow no longer invokes it. Marketplace scopes are no longer required on the CI PAT. Co-authored-by: Cursor <cursoragent@cursor.com>
… time (#618) Marketplace rejects uploads when the manifest publisher and the publisher you are uploading under don't match exactly: Publisher ID 'e2e-build' provided in the extension manifest should match the publisher ID 'naveenkuorg-private' under which you are trying to publish this extension. The previous workflow baked in a generic 'e2e-build' placeholder, which required every manual upload to first re-pack the .vsix. Use "${ADO_E2E_ORG}-private" instead so the artifact uploads cleanly straight from the run page. Fails loudly if the secret is missing. Co-authored-by: Cursor <cursoragent@cursor.com>
…vsix (#619) Marketplace rejects uploads when either: 1. the manifest publisher and the destination publisher don't match (Publisher ID 'e2e-build' provided in the extension manifest should match the publisher ID 'naveenkuorg-private' under which you are trying to publish this extension), OR 2. the (publisher, extension-id) pair has ever existed in that publisher before (The extension already exists) — Marketplace permanently tombstones the pair after the first create even if you later delete the extension and the publisher UI shows it as empty. Bake both correct values into the .vsix at build time: - publisher = "${ADO_E2E_ORG}-private" - extension id = "jfrog-azure-devops-extension-e2etest" (overridable via the ADO_E2E_EXTENSION_ID repo secret if this id ever gets tombstoned too — set it to any unused string and re-run). Tasks inside the .vsix are referenced by their own GUIDs, so a different extension id is transparent to the ADO pipelines that consume them. Install instructions in the run summary now also explain the New extension vs Update flow (first upload vs subsequent versions) so the manual step stays painless on repeat use. Co-authored-by: Cursor <cursoragent@cursor.com>
trigger-ado-pipeline.sh was using `curl -sf`, which silently swallows
the response body and the status code. When the trigger fails, the
workflow log reads only:
ERROR: curl request to trigger pipeline failed.
That's not actionable — the failure could be a 401 (bad PAT), a 403
(missing scope), a 404 (wrong project or pipeline id), a 5xx, or DNS.
Capture the HTTP status + response body separately, print both, and
hint at the most likely cause based on the status code so we can
diagnose at a glance.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(e2e): inline service-connection names in pipeline YAMLs
Both .pipelines/*.yml files declared their service-connection names as
pipeline `parameters` and referenced them via `${{ parameters.X }}`. ADO
rejects template expressions in a number of contexts (most importantly
`resources.repositories[].endpoint`, which is parsed before parameter
expansion):
/.pipelines/ado-sanity-pipeline.yml (Line: 74, Col: 17):
A template expression is not allowed in this context
The GitHub Actions trigger only sets pipeline `variables`, never
`parameters`, so the parameter defaults were the only values ever used.
Drop the `parameters:` block entirely and inline each connection name as
a literal string. Same treatment applied to both sanity and OIDC
pipelines for consistency.
To rename a connection later, edit the literal in-place.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(e2e): rewrite task GUIDs so private .vsix coexists with public extension
ADO rejects installing a second extension that shares any task GUID with
an already-installed extension:
A task definition with ID '32f70de9-...' name 'JFrogBuildPromotion'
has already been uploaded by extension 'jfrog-azure-devops-extension'.
To upload the task, provide a different task id.
This happens because both the public release and our per-PR private build
come from the same source tree, sharing all task GUIDs. In team orgs
(e.g. vigneshc0742/ecomatrix-test) that have the public extension
installed, we can never install the private build alongside it.
Add a "Rewrite task GUIDs for private build" step before tfx create that
replaces every task GUID with a deterministic UUID v5 derived from
(fixed-namespace, original-GUID). Properties:
- Stable: same source GUID always maps to the same private GUID, so
the installed extension's tasks are found by the ADO pipeline YAML
across runs.
- Unique: the private GUIDs are guaranteed not to match the public ones.
- Self-contained: pure Python stdlib (uuid module), no extra deps.
This lets the private test extension coexist with the public release in
the same ADO org, which is required when the org has compute slots we
rely on (like vigneshc0742).
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(e2e): inline service-connection names in pipeline YAMLs
Both .pipelines/*.yml files declared their service-connection names as
pipeline `parameters` and referenced them via `${{ parameters.X }}`. ADO
rejects template expressions in a number of contexts (most importantly
`resources.repositories[].endpoint`, which is parsed before parameter
expansion):
/.pipelines/ado-sanity-pipeline.yml (Line: 74, Col: 17):
A template expression is not allowed in this context
The GitHub Actions trigger only sets pipeline `variables`, never
`parameters`, so the parameter defaults were the only values ever used.
Drop the `parameters:` block entirely and inline each connection name as
a literal string. Same treatment applied to both sanity and OIDC
pipelines for consistency.
To rename a connection later, edit the literal in-place.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(e2e): rewrite task GUIDs so private .vsix coexists with public extension
ADO rejects installing a second extension that shares any task GUID with
an already-installed extension:
A task definition with ID '32f70de9-...' name 'JFrogBuildPromotion'
has already been uploaded by extension 'jfrog-azure-devops-extension'.
To upload the task, provide a different task id.
This happens because both the public release and our per-PR private build
come from the same source tree, sharing all task GUIDs. In team orgs
(e.g. vigneshc0742/ecomatrix-test) that have the public extension
installed, we can never install the private build alongside it.
Add a "Rewrite task GUIDs for private build" step before tfx create that
replaces every task GUID with a deterministic UUID v5 derived from
(fixed-namespace, original-GUID). Properties:
- Stable: same source GUID always maps to the same private GUID, so
the installed extension's tasks are found by the ADO pipeline YAML
across runs.
- Unique: the private GUIDs are guaranteed not to match the public ones.
- Self-contained: pure Python stdlib (uuid module), no extra deps.
This lets the private test extension coexist with the public release in
the same ADO org, which is required when the org has compute slots we
rely on (like vigneshc0742).
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(e2e): decouple Marketplace publisher org from ADO test-run org
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
…623) When both the public JFrog extension and our private test build are installed in the same ADO org, pipelines fail at queue time: "The task name JFrogToolsInstaller is ambiguous." Root cause: the private build already has different task GUIDs (to pass Marketplace publish validation) but identical task *names*, so ADO cannot tell which extension's task to invoke. Fix: extend the existing GUID-rewrite step to also append "E2E" to every task's `name` field (e.g. JFrogToolsInstaller → JFrogToolsInstallerE2E). Update both test pipeline YAMLs to reference the E2E-suffixed names. Result: public and private extensions coexist in the same org with no name collision. The .pipelines/ado-*.yml files always target the private test build; the public extension is untouched. Co-authored-by: Cursor <cursoragent@cursor.com>
#624) Co-authored-by: Cursor <cursoragent@cursor.com>
The sanity pipeline checked out jfrog/project-examples via a GitHub service connection, which fails with SAML SSO enforcement on the jfrog org. Replace the checkout + external pom.xml reference with a minimal inline pom.xml (same pattern used by the OIDC pipeline), removing the github-jfrog-public service connection requirement entirely. Co-authored-by: Cursor <cursoragent@cursor.com>
…efaults (#626) The $[coalesce(variables[...], default)] syntax in the variables block expands as a literal string when referenced via $(VAR) macros in script steps, causing bash syntax errors. Since GH_PR_NUMBER and GH_COMMIT_SHA are now marked 'Settable at queue time' in both pipelines, simple static defaults are sufficient — the trigger script overrides them at queue time. Co-authored-by: Cursor <cursoragent@cursor.com>
The single-line script: script: echo "##[section]OIDC PR #$(GH_PR_NUMBER) commit $(GH_COMMIT_SHA)" was causing bash to fail with 'unexpected EOF while looking for matching "'. The ##[section] ADO log command embedded inside a double-quoted string in a single-line YAML scalar causes the ADO agent to strip the log command from the script source, leaving an unclosed double-quote that bash cannot parse. Fix: switch to a |block scalar and use bash env var syntax ($GH_PR_NUMBER) so ADO does not macro-expand the variable before writing the script, avoiding the parse issue entirely. Co-authored-by: Cursor <cursoragent@cursor.com>
Fixes multiple test configuration issues discovered during e2e test runs against the vigneshc0742/ecomatrix-test ADO organisation. - S1: fix traceability script bash parse error (block scalar + $VAR syntax) - S2 ToolsInstaller Token: find jf in tool cache path for verify step; JFrogToolsInstaller never calls prependPath so jf is not on $PATH for script steps - S2 ToolsInstaller Extractors: use sc-artifactory-token for CLI download; OIDC token exchange requires the CLI to already be on disk, making OIDC service connections unusable for the CLI download itself - S5 + S6: bump JavaToolInstaller to versionSpec 17; Gradle 9.5.1 on ubuntu-22.04 agents requires JVM 17+ - S6 Gradle: fix wrong input names (targetResolveRepo/targetDeployRepo/ artifactoryDeployService are Maven task names); corrected to sourceRepo/targetRepo/artifactoryDeployerService - S6 Gradle: add workDir pointing to the inline project directory; Gradle 9 removed the -b <buildfile> flag so without workDir Gradle runs in the agent checkout directory and finds no build files - S7 Go: add go mod tidy before jf go build; Go 1.21+ requires go.sum to exist before build and the inline module had none Co-authored-by: Cursor <cursoragent@cursor.com>
…628) Fixes multiple test configuration issues discovered during e2e test runs against the vigneshc0742/ecomatrix-test ADO organisation. - S1: fix traceability script bash parse error (block scalar + $VAR syntax) - S2 ToolsInstaller Token: find jf in tool cache path for verify step; JFrogToolsInstaller never calls prependPath so jf is not on $PATH for script steps - S2 ToolsInstaller Extractors: use sc-artifactory-token for CLI download; OIDC token exchange requires the CLI to already be on disk, making OIDC service connections unusable for the CLI download itself - S5 + S6: bump JavaToolInstaller to versionSpec 17; Gradle 9.5.1 on ubuntu-22.04 agents requires JVM 17+ - S6 Gradle: fix wrong input names (targetResolveRepo/targetDeployRepo/ artifactoryDeployService are Maven task names); corrected to sourceRepo/targetRepo/artifactoryDeployerService - S6 Gradle: add workDir pointing to the inline project directory; Gradle 9 removed the -b <buildfile> flag so without workDir Gradle runs in the agent checkout directory and finds no build files - S7 Go: add go mod tidy before jf go build; Go 1.21+ requires go.sum to exist before build and the inline module had none Co-authored-by: Cursor <cursoragent@cursor.com>
…low (#629) ADO pipeline YAML fixes (ado-oidc-regression-pipeline.yml): - Fix wrong Gradle task input names (sourceRepo/targetRepo/artifactoryDeployerService) - Fix CLI verify step: find jf in tool cache, not bare PATH - Use sc-artifactory-token for JFrogToolsInstaller CLI download (OIDC can't be used before the CLI binary exists on disk) - Bump JavaToolInstaller to versionSpec 17 (Gradle 9.5.1 requires JVM 17+) - Add workDir for JFrogGradle so Gradle 9+ finds build files (dropped -b flag) - Run go mod tidy before jf go build (Go 1.21+ requires go.sum) - Fix S1 traceability script bash parse error GitHub Actions workflow fixes (e2e-plugin-tests.yml): - Auto-publish .vsix to Marketplace when ADO_E2E_MARKETPLACE_PAT secret is set - Wait 90s after publish for Marketplace async validation to propagate - Wait 7 min in Jobs 2 & 3 before triggering ADO pipelines to ensure the latest extension version is active in the ADO org Co-authored-by: Cursor <cursoragent@cursor.com>
…ace publish step Co-authored-by: Cursor <cursoragent@cursor.com>
…ons bug on first publish Co-authored-by: Cursor <cursoragent@cursor.com>
…line start Co-authored-by: Cursor <cursoragent@cursor.com>
… and 3 Co-authored-by: Cursor <cursoragent@cursor.com>
… jobs 2 and 3 Co-authored-by: Cursor <cursoragent@cursor.com>
…on check Co-authored-by: Cursor <cursoragent@cursor.com>
…wait Co-authored-by: Cursor <cursoragent@cursor.com>
…AML parse error Co-authored-by: Cursor <cursoragent@cursor.com>
…ons GET endpoint Co-authored-by: Cursor <cursoragent@cursor.com>
…ivate extension version query Co-authored-by: Cursor <cursoragent@cursor.com>
… to query private extension version Co-authored-by: Cursor <cursoragent@cursor.com>
… in Job 1 Co-authored-by: Cursor <cursoragent@cursor.com>
…version extraction Co-authored-by: Cursor <cursoragent@cursor.com>
… syntax error Co-authored-by: Cursor <cursoragent@cursor.com>
…remove from job 1 Co-authored-by: Cursor <cursoragent@cursor.com>
…ch expected build version Co-authored-by: Cursor <cursoragent@cursor.com>
Join v2 history into dev so the release step `git merge origin/dev` on v2 can apply product changes (#638, CLI 2.111.0, Node 22) without e2e add/add conflicts. Keep current dev e2e files (newer refinements). Co-authored-by: Cursor <cursoragent@cursor.com>
3 tasks
Contributor
Author
|
Superseded by a single-commit PR into v2. Squashing this v2->dev merge would drop v2 from dev's ancestry and re-introduce the release-time e2e conflict, so replacing it with one clean commit onto v2 instead. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Merges
v2intodevso the release pipeline can later take care ofdev→v2.The release pipeline (
.jfrog-pipelines/pipelines.release.yml) runsgit merge origin/devonv2. That was failing because e2e files evolved independently on both branches (add/add conflicts).This PR joins
v2history intodevwith no content change ondev— conflicts in:.github/workflows/e2e-plugin-tests.ymlbuildScripts/trigger-ado-pipeline.share resolved by keeping current
devversions (newer e2e refinements). Product work already ondev(#638, CLI 2.111.0, Node 22) is untouched.After merge
Release on
v2can rungit merge origin/devcleanly and will pick up:#638OIDC ToolsInstaller CLI download auth2.111.0/ plugin2.14.2devTest plan
devis empty (history-only merge)v2,git merge origin/devis conflict-freeNEXT_VERSIONsetMade with Cursor