Skip to content

test: package OpenHands Enterprise 0.84.0-rc.4 - #1441

Draft
ak684 wants to merge 6 commits into
mainfrom
alona/package-0.84.0-rc.1
Draft

ak684 wants to merge 6 commits into
mainfrom
alona/package-0.84.0-rc.1

Conversation

@ak684

@ak684 ak684 commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Description

Release candidate for a limited trial. Do not merge: it pins enterprise and Canvas images built for this candidate. Once their changes are in releases, this candidate should move to the release tags or be closed.

It packages main as of 08333cb (0.83.0 plus #1337, #1410, #1428, #1429, #1431, #1436 and #1443) with:

What it adds over enterprise 1.73.0:

  • Conversations load skills from registered repositories that have no marketplace manifest. This needs SDK 1.56.0 on both the enterprise server and the agent-server.
  • Event batches no longer lose their execution-status updates and callbacks when their saves race on the conversation's event index.
  • The managed LLM key ownership repair accepts member settings saved before SDK 1.51. Without that, SDK 1.56.0 rejects a stored row that still has enable_sub_agents, which settings saved by 0.81.0 and earlier can have.
  • The enterprise server is built against the same SDK as the agent-server it runs with (1.56.0).
  • Agent Canvas shows a new conversation's first reply without a reload, and a delivered first message no longer stays pending ([Bug]: On Cloud, the first message stays "Sending..." below the agent's reply and then turns "Failed to send" until a reload OpenHands#18717). Canvas 1.25.0 and later, including the 1.27.1 in 0.81.0, have this bug.

Since this was built, main has merged #1440 and #1432, so it pins Canvas 1.28.1 and agent-server 1.56.0-python too. It still pins enterprise 1.73.0, which was built against SDK 1.55.1, so scripts/check_agent_server_sync.py fails on main (cba0177). Conversations do work with that pairing; see the notes below.

Already in 1.73.0 and so in main: Azure DevOps repository listing per organization (OpenHands/enterprise#847), work item commenters resolved by email (OpenHands/enterprise#802), and Bearer access tokens for Bitbucket Data Center (OpenHands/enterprise#870).

Not included:

Rollback: settings saved on this candidate are stored at agent settings schema 9. SDK 1.55.1 (enterprise 1.73.0 and earlier) refuses them, so going back after users have saved settings needs those rows restored first.

Helm Chart Checklist

  • I have tested the chart upgrade path from the previous version
  • I have verified backwards compatibility with existing values.yaml configurations
  • I have updated the chart's README.md if there are any breaking changes or new required values

Notes on the checklist: only image tags and the chart version change, so existing values files are unaffected. The upgrade path was tested through KOTS from 0.81.0 and from 0.83.0, as described below. No README change is needed.

Replicated release and KOTS upgrade tests

rc.1 (this chart with Canvas 1.28.1) was released to Replicated as 0.84.0-rc.1 with make release on a customer-specific channel. The Replicated lint showed no errors. Two test installs were upgraded to it through KOTS by assigning their test customers to that channel:

  • 0.76.0 → Stable 0.81.0 → 0.84.0-rc.1 (an install with 97 member settings rows):
    • all 16 preflights passed, and the upgrade deployed in about 3½ minutes;
    • database 186 → 192;
    • sha-cc0ced6 on all 3 enterprise Deployments and 4 CronJobs, Canvas 1.28.1, Keycloak 26.8.0, and agent-server 1.56.0 for new sandboxes;
    • every member's stored settings loaded with the new settings store;
    • no app errors, and every job since the upgrade succeeded.
  • 0.83.0 → 0.84.0-rc.1 (an install with an Azure DevOps user in two organizations), as that user in the classic UI:
    • the agent settings page saved and reloaded;
    • repositories were listed from both organizations;
    • resolver Service Hooks installed in both organizations, then were removed again;
    • an Azure DevOps skills repository whose project name contains a space previewed its skill and saved;
    • github:dotnet/skills listed its 16 plugins in about 1 s, and with Auto-Load a conversation loaded the skills bundled in them;
    • a conversation on the second organization's repository checked it out at the branch head the app reports;
    • afterwards there were no app errors, and the budget jobs and a manual run of the daily maintenance job succeeded.
  • Agent Canvas 1.28.1 on the second install: conversations start and run. If the agent replies before the page's event socket connects (about 14 s after sending), the reply appears only after a reload. Canvas 1.27.1, the Canvas in 0.81.0, has the same code. rc.2 adds the fix.

rc.2 was released to the same channel as 0.84.0-rc.2 (Replicated release 1901). On the second install, KOTS upgraded rc.1 to rc.2 in about 2 minutes with agent-canvas:sha-658a098. The Canvas first-reply check then passed: with the race forced, the reply showed at 15 s, and nothing was left pending past 175 s or after a reload.

rc.3 was released to the same channel as 0.84.0-rc.3 (Replicated release 1903) and tested on both installs:

  • rc.1 → rc.3 on the first install (97 member settings rows):
    • deployed in about 2¼ minutes, with sha-40d51ea on all 7 enterprise workloads;
    • the database stays at revision 192, and every member's settings load;
    • a member row in the shape 0.81.0 saved ({"enable_sub_agents": true}) now passes the managed-key repair's settings step, which rejected it on sha-cc0ced6;
    • a manual run of the daily maintenance job succeeded.
  • rc.2 → rc.3 on the second install:
    • the Azure DevOps checks above passed again: settings, repositories from both organizations, hooks in both organizations, the space-in-name skills repository, dotnet/skills plugins, and a second-organization checkout at the branch head;
    • the Canvas first-reply check passed, with the reply at 15 s under a forced race.

rc.4 was released as 0.84.0-rc.4 (Replicated release 1911) to a separate test channel first. The first install was upgraded to it from rc.3 through KOTS by moving its test customer to that channel:

  • all 16 preflights passed, and the upgrade deployed in about 2 minutes. Only the automation Deployment changed (automation and migrate on 1.23.1), and the automation database stays at revision 033;
  • as an organization member in Agent Canvas, Save draft worked, Test started a run that finished successfully in 55 s, and the draft was then deleted. Through the API, the member could edit and delete their own draft and upload, got 403 for another user's draft and 404 for another user's upload, and still could not create or delete webhooks;
  • as the organization owner, an automation was created, run (it finished in 30 s), edited and deleted, and a webhook was created and deleted;
  • the login check passed, and a new Canvas conversation got the agent's reply. The browser checks' only transport errors (requests cancelled by the login redirect, and blocked third-party icons) also appear on this install before rc.4;
  • the app logged no errors. The automation service logged only failed trace uploads to an absent Datadog agent, which 1.22.0 logs too.

After these checks, the same release (1911) was promoted to the customer-specific channel as 0.84.0-rc.4.

Additional Notes

Live test of the pinned enterprise image on a 0.83.0 install, with the app Deployments on sha-cc0ced6 and sandboxes on agent-server 1.56.0-python. It ran as a member whose settings enterprise 1.72.1 had stored, and passed:

  • Settings load and save, and the saved value survives a reload.
  • An older client's sub-agents switch is accepted.
  • A registered repository without a marketplace manifest: 9 skills previewed, and the same 9 loaded in a conversation.
  • A conversation that answered a message sent from its page, with the sub-agent tool set offered.
  • Three more conversations whose events were searched while the agent worked: all finished, with no "Error in webhook" in the app log.

The install was restored to 0.83.0 afterwards, before the KOTS test above.

For comparison, main's pairing (enterprise 1.73.0 with agent-server 1.56.0-python) on the same install: three conversations started, ran their commands and replied. The only webhook errors were the event index race that OpenHands/enterprise#878 fixes. SDK 1.56.0 also accepts the conversation start requests SDK 1.55.1 builds, with and without sub-agents.

The agent-server pins match: sha-40d51ea's pyproject.toml pins SDK, agent-server and tools 1.56.0, as the chart's 1.56.0-python needs. CI runs scripts/check_agent_server_sync.py.

This PR conflicts with main by design, because it keeps the candidate's pins, so GitHub does not run its pull_request checks. On the rc.4 head, the Helm unit tests (8 charts, 273 tests), the chart template tests (29) and the Replicated lint (no errors, same findings as rc.3) pass locally with CI's Helm version.

Validation evidence 1

@github-actions github-actions Bot added the type: test Adding or fixing tests label Oct 19, 2026
@ak684

ak684 commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

@OpenHands /codereview-roasted

Review exactly commit b24d578. If the PR head changes, report that and do not approve another commit. Review only: do not push code, merge, deploy, or change attribution. Check correctness, regression coverage, and whether the PR validation claims match the evidence. Keep code comments short. Post the substantive review and link it in your completion message. End the substantive review with these plain lines (no code fence), choosing one verdict:

REVIEWED_HEAD: b24d578
REVIEW_REQUEST: b8c99955057047d492e048c00860bc10
VERDICT: APPROVED

Use VERDICT: CHANGES_REQUESTED instead when findings require changes. If unable to review, explain the blocker without an approval verdict.

@openhands-ai

openhands-ai Bot commented Oct 19, 2026

Copy link
Copy Markdown

I'm on it! ak684 can track my progress at all-hands.dev

@ak684 ak684 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exactly commit b24d578ff730968622486536e2ec8dd61040cce3 (PR head matches; no other commit was evaluated).

What changed — five version-only bumps across 5 files: charts/openhands/Chart.yaml (appVersion: sha-522c97e, version: 0.84.0-rc.1), the enterprise image.tag in the umbrella values, the agent-server tag (1.56.0-python) in all three pin locations (umbrella global.agentServerImage, the runtime-api subchart default, and image-loader), and agent-canvas 1.28.1.

Correctness — claims check out

  • All three agent-server pins were updated in lockstep; no stale 1.55.1 references remain in charts/, replicated/, or e2e_tests/.
  • uv run scripts/check_agent_server_sync.py passes against the real remote: sha-522c97e resolves to commit 522c97e in OpenHands/enterprise, whose pyproject.toml pins SDK 1.56.0, matching 1.56.0-python. (The sha- tag is handled explicitly by that script's _SHA_TAG_RE.)
  • The commit range v1.72.1 → 522c97e is 9 commits ahead and includes exactly the fixes the PR cites: OpenHands/enterprise#803, #847, #802, #870, #751 (plus two cosmetic fixes not itemized, and the SDK 1.56.0 bump).
  • DB migrations are identical to 1.72.1: 192 files in migrations/versions/, zero differences between the refs. This substantiates the "no forward migration on upgrade" and the plain-rollback side of the rollback statement. The residual rollback risk is the disclosed agent-settings schema 9 vs 1.72.1's schema 8, and the PR documents that honestly.
  • agent-canvas 1.28.1 is byte-identical to the change in #1440 (still open), matching the "same change" claim.
  • appVersion == image.tag (both sha-522c97e), so the appVersion-drift check is satisfied.

Regression coverage — CI on the head is fully green: lint-and-test (all four charts), helm-unittest, test-scripts (which exercises check_agent_server_sync.py), check-appversion-drift, and image scans for all three moved images. Note: the "Check agent-server sync" workflow job is skipped on this PR, so the sync invariant ran only via the test-scripts suite + a local manual run that passes.

Minor, non-blocking

  • charts/openhands/values.yaml documents "do not use sha-based tags" and image.tag: sha-522c97e contravenes that stated invariant. This is deliberate for a release candidate and fully disclosed in the PR body ("Do not merge… move to the release tag or be closed"), so it is acceptable here — but it should not reach main in this form.

Verified head b24d578ff730968622486536e2ec8dd61040cce3 matches the review target; no drift found.

REVIEWED_HEAD: b24d578
REVIEW_REQUEST: b8c99955057047d492e048c00860bc10
VERDICT: APPROVED

@openhands-ai

openhands-ai Bot commented Oct 19, 2026

Copy link
Copy Markdown

The review has been posted to PR #1441. Here's the summary.

Review complete

Commit reviewed: b24d578ff730968622486536e2ec8dd61040cce3 (verified as the current PR head both before and after posting; the head did not change during review, so no other commit was evaluated).

Posting note: I couldn't submit the verdict as an official GitHub APPROVE status — GitHub rejects self-approval for the PR author's token (Review Can not approve your own pull request) — so the substantive review was posted as a full commentary review with the verdict lines embedded. The six-part verdict block is intact at the end.

What I verified

Verdict: APPROVED (per the repository's default-approve review policy).

🔗 Review: #1441 (review)

@ak684

ak684 commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

@OpenHands /codereview-roasted

Review exactly commit 2924389. If the PR head changes, report that and do not approve another commit. Review only: do not push code, merge, deploy, or change attribution. Check correctness, regression coverage, and whether the PR validation claims match the evidence. Keep code comments short. Post the substantive review and link it in your completion message. End the substantive review with these plain lines (no code fence), choosing one verdict:

REVIEWED_HEAD: 2924389
REVIEW_REQUEST: 96e150bfabe74609bf4efb4f78379b0f
VERDICT: APPROVED

Use VERDICT: CHANGES_REQUESTED instead when findings require changes. If unable to review, explain the blocker without an approval verdict.

@openhands-ai

openhands-ai Bot commented Oct 19, 2026

Copy link
Copy Markdown

I'm on it! ak684 can track my progress at all-hands.dev

@ak684 ak684 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exactly commit 29243892141672b7b939024838d49c9eab4c70ea (current PR head matches the review target; no other commit was evaluated).

What changed

Five version-only bumps across 5 files (net diff vs main):

  • charts/openhands/Chart.yaml: version 0.83.0 → 0.84.0-rc.1, appVersion 1.73.0 → sha-cc0ced6
  • charts/openhands/values.yaml: enterprise image.tag 1.73.0 → sha-cc0ced6; global.agentServerImage.tag 1.55.1-python → 1.56.0-python
  • charts/openhands/charts/runtime-api/values.yaml: agentServerImage.tag 1.55.1-python → 1.56.0-python
  • charts/image-loader/values.yaml: image.tag 1.55.1-python → 1.56.0-python
  • charts/openhands/charts/agent-canvas/values.yaml: 1.28.0 → 1.28.1

Correctness — claims check out

  • Agent-server sync: uv run scripts/check_agent_server_sync.py passes against the real remote: sha-cc0ced6 resolves to commit cc0ced6 in OpenHands/enterprise, whose pyproject.toml pins openhands-agent-server==1.56.0 (plus openhands-sdk/openhands-tools, all moving together), matching 1.56.0-python. All three agent-server pin locations agree; a repo-wide search finds no stale 1.55.1 references.
  • Enterprise build: cc0ced6 is exactly one commit ahead of 1.73.0 (cacffeadee). That single commit carries the SDK 1.56.0 bump (the #876 change, openhands-agent-server==1.55.1 → 1.56.0 in pyproject.toml plus the settings/schema handling for retired tool switches) and the concurrent event-save fix (the #878 change — _invalidate_index race handling in the AWS/filesystem/GCS event services).
  • Migrations: both 1.73.0 and cc0ced6 have 192 files under migrations/versions/, and the RC commit touches none of them — so "database migrations end at the same revision as 1.73.0" is accurate, and the "no forward migration" side of the rollback story holds. The disclosed residual risk (agent-settings schema 9 vs 8 — SDK 1.55.1 refuses rows saved on this candidate) is documented honestly.
  • agent-canvas: 1.28.1 is byte-identical to the change in #1440 (1.28.0 → 1.28.1, same file), matching the PR's "same change" claim.
  • appVersion drift: appVersion == image.tag == sha-cc0ced6; the check-appversion-drift check passes.

Regression evidence

All 24 checks on the head are green: lint-and-test for all four charts, helm-unittest, test-scripts (exercises check_agent_server_sync.py), check-appversion-drift, check-secret-checksum, and image scans for enterprise-server:sha-cc0ced6, agent-server:1.56.0-python, runtime-api:0.11.1. The dedicated "Check agent-server sync" workflow job is skipped on this PR by design (it only runs for release-please--* branches), so that invariant was covered by test-scripts plus my live manual run, which passes.

Minor, non-blocking

  • charts/openhands/values.yaml documents "You must use a stable, semver tag. Do not use sha-based tags", and image.tag: sha-cc0ced6 contravenes it. This is deliberate for this disclosed, do-not-merge RC (the PR states it should "move to the release tag or be closed"), and the check script's _SHA_TAG_RE handles it — but this exact form should not land on main.

No drift: the reviewed head 29243892141672b7b939024838d49c9eab4c70ea matches the review target and did not change during review.

REVIEWED_HEAD: 2924389
REVIEW_REQUEST: 96e150bfabe74609bf4efb4f78379b0f
VERDICT: APPROVED

@openhands-ai

openhands-ai Bot commented Oct 19, 2026

Copy link
Copy Markdown

Review complete for PR #1441, posted as a full commentary review.

Review link: #1441 (review)

What I verified

The review targeted exactly commit 29243892141672b7b939024838d49c9eab4c70ea, which I confirmed is the current PR head both before and after posting (it did not change during review, so no other commit was evaluated).

Scope: 5 files, version-only bumps — Chart.yaml (0.83.0 → 0.84.0-rc.1, appVersion: sha-cc0ced6), enterprise image.tag → sha-cc0ced6, agent-server 1.56.0-python in all three pin locations (umbrella, runtime-api subchart, image-loader), agent-canvas 1.28.1.

Correctness — claims check out:

  • uv run scripts/check_agent_server_sync.py passes against the live remote: sha-cc0ced6 → commit cc0ced6 in OpenHands/enterprise → SDK pins 1.56.0, matching 1.56.0-python. No stale 1.55.1 references anywhere in the repo.
  • cc0ced6 is exactly 1 commit ahead of 1.73.0; that commit carries precisely the feat(openhands): bump image tag to cloud-1.47.0 #876 (SDK 1.56.0 bump + settings/schema migration) and chore: add CODEOWNERS #878 (concurrent event-save race fix) changes the PR describes.
  • Migrations: 192 files in both 1.73.0 and cc0ced6, none touched by the RC commit — "same DB revision as 1.73.0" holds, substantiating the rollback story; the disclosed agent-settings schema 9 risk is documented honestly.
  • agent-canvas 1.28.1 is byte-identical to feat(agent-canvas): bump image tag to 1.28.1 #1440's change.
  • appVersion == image.tag → the drift check passes.

Regression evidence: all 24 CI checks green on the head (lint-and-test ×4, helm-unittest, test-scripts, check-appversion-drift, check-secret-checksum, image scans for all three moved images). The dedicated "Check agent-server sync" job is skipped on this PR by design (it only runs on release-please--* branches); that invariant was covered by test-scripts plus my live manual run.

Minor, non-blocking: the sha-cc0ced6 tag contravenes the chart's stated "no sha-based tags" invariant — acceptable and disclosed for this do-not-merge RC, but it shouldn't land on main as-is.

Verdict: APPROVED (per the repository's default-approve policy — no important-level findings).

@ak684 ak684 changed the title test: package OpenHands Enterprise 0.84.0-rc.1 test: package OpenHands Enterprise 0.84.0-rc.2 Oct 20, 2026
@ak684

ak684 commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

@OpenHands /codereview-roasted

Review exactly commit 61e69e3. If the PR head changes, report that and do not approve another commit. Review only: do not push code, merge, deploy, or change attribution. Check correctness, regression coverage, and whether the PR validation claims match the evidence. Keep code comments short. Post the substantive review and link it in your completion message. End the substantive review with these plain lines (no code fence), choosing one verdict:

REVIEWED_HEAD: 61e69e3
REVIEW_REQUEST: e817208c9ef24534a9d936baa14a8b47
VERDICT: APPROVED

Use VERDICT: CHANGES_REQUESTED instead when findings require changes. If unable to review, explain the blocker without an approval verdict.

@openhands-ai

openhands-ai Bot commented Oct 20, 2026

Copy link
Copy Markdown

I'm on it! ak684 can track my progress at all-hands.dev

@ak684 ak684 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exactly commit 61e69e369868598b32c263631ec4359dfa6325cc (confirmed as the current PR head both before and after review; no other commit was evaluated).

What this commit changes (rc.2 delta)

Two version-string edits on top of rc.1 — exactly matching the PR's claim that "rc.2 differs from rc.1 only in the Canvas image":

  • charts/openhands/Chart.yaml: version 0.84.0-rc.1 → 0.84.0-rc.2 (appVersion unchanged at sha-cc0ced6)
  • charts/openhands/charts/agent-canvas/values.yaml: image.tag "1.28.1" → "sha-658a098"

The rest of the PR (cumulative vs main, 5 files) is the same version-only bump set already reviewed on rc.1: enterprise image.tag 1.73.0 → sha-cc0ced6, agent-server 1.55.1-python → 1.56.0-python in all three pin sites, and the Canvas tag.

Correctness — claims check out

  • Canvas image: sha-658a098 resolves to commit 658a09808fe3a090ed7a23ffa4190c5be344d6a2, the head of OpenHands/OpenHands#18725, described as "v1.28.1 tag plus only the change from #18719" — the two Canvas fixes the PR attributes to rc.2 (#18717 first-reply / pending-message fix, and the 1.28.1 release). The manifest exists on ghcr (agent-canvas:sha-658a098 → HTTP 200).
  • Agent-server sync (live run): uv run scripts/check_agent_server_sync.py passes — sha-cc0ced6 resolves to commit cc0ced6 in OpenHands/enterprise, whose pyproject.toml pins openhands-agent-server==1.56.0, matching 1.56.0-python. All three agent-server pin locations agree; no stale 1.55.1 remains.
  • Enterprise build: cc0ced6e47fab0e63a937b6a98a25b0d1d8131d5 is the head of OpenHands/enterprise#880 ("release candidate 0.84.0-rc.1 build (do not merge)"), exactly as described.
  • Migrations / rollback: unchanged from the earlier reviews of this series (migrations end at the same revision as 1.73.0); this commit touches none of them. The disclosed residual risk — agent-settings schema 9 saved on this candidate is refused by SDK 1.55.1 if rolled back — is documented honestly in the PR body.
  • appVersion drift guard: appVersion == image.tag == sha-cc0ced6.
  • Pinned images all exist: agent-canvas:sha-658a098, enterprise-server:sha-cc0ced6, agent-server:1.56.0-python all return HTTP 200 from ghcr.

Regression coverage

Helm unit tests, test-scripts (which exercises check_agent_server_sync.py), Replicated lint, appVersion-drift and secret-checksum checks are all green on the immediately preceding head 292438921. On this exact head only the pr-title checks have completed so far — the heavier workflows are still (re)running after the push. Given the change is confined to two version strings whose referenced images all resolve on ghcr, this does not block approval; worth confirming the full suite goes green before considering a non-RC ship.

Minor, non-blocking

  • charts/openhands/values.yaml documents "do not use sha-based tags", and image.tag: sha-cc0ced6 (and now the Canvas sha-658a098 tag) contravene it. This is deliberate and fully disclosed for this do-not-merge RC, and the check scripts' _SHA_TAG_RE handle it — but this exact form must not land on main.

No drift: the reviewed head 61e69e369868598b32c263631ec4359dfa6325cc matches the review target and did not change during review.

REVIEWED_HEAD: 61e69e3
REVIEW_REQUEST: e817208c9ef24534a9d936baa14a8b47
VERDICT: APPROVED

@openhands-ai

openhands-ai Bot commented Oct 20, 2026

Copy link
Copy Markdown

The review is complete and posted. Here's the summary.

Review complete

Review posted: #1441 (review) (submitted as a full commentary review — the token belongs to ak684, who authored the PR, so GitHub rejects an official APPROVE status as self-approval, consistent with the earlier reviews in this thread).

Commit reviewed: 61e69e369868598b32c263631ec4359dfa6325cc, verified as the current PR head both before and after posting. The head did not change during review, so no other commit was evaluated.

What I verified

  • Scope: The commit is two version-string edits matching the PR claim — Chart.yaml 0.84.0-rc.1 → 0.84.0-rc.2, and agent-canvas image.tag 1.28.1 → sha-658a098. Cumulative PR diff is 5 version-only bumps.
  • Canvas image: sha-658a098 resolves to the head of chore: build a 1.28.1 Canvas image with the #18719 history fix (do not merge) OpenHands#18725 ("v1.28.1 tag plus only the change from #18719"); the manifest exists on ghcr.
  • Agent-server sync: I ran uv run scripts/check_agent_server_sync.py live — passes. sha-cc0ced6 resolves in OpenHands/enterprise, which pins SDK 1.56.0, matching 1.56.0-python; all three pin sites agree.
  • Enterprise build: cc0ced6 is exactly the head of chore: release candidate 0.84.0-rc.3 build (do not merge) enterprise#880, as described.
  • Invariants: appVersion == image.tag == sha-cc0ced6 (appVersion-drift guard satisfied); all three pinned images return HTTP 200.
  • Regression coverage: Full suite (helm-unittest, test-scripts, Replicated lint, checks) is green on the preceding head; only pr-title has completed on this exact head so far — noted as non-blocking since the delta is version strings referencing resolvable images.
  • Minor/non-blocking: the sha-based image.tag contravenes the values.yaml comment about semver tags — deliberate and disclosed for this do-not-merge RC, but should not land on main.

Verdict included per the requested format: APPROVED.

@ak684 ak684 changed the title test: package OpenHands Enterprise 0.84.0-rc.2 test: package OpenHands Enterprise 0.84.0-rc.3 Oct 20, 2026
@ak684

ak684 commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

@OpenHands /codereview-roasted

Review exactly commit 6048e5b. If the PR head changes, report that and do not approve another commit. Review only: do not push code, merge, deploy, or change attribution. Check correctness, regression coverage, and whether the PR validation claims match the evidence. Keep code comments short. Post the substantive review and link it in your completion message. End the substantive review with these plain lines (no code fence), choosing one verdict:

REVIEWED_HEAD: 6048e5b
REVIEW_REQUEST: 3ba0b71dbf41468ea2f85f80aaaf3b0c
VERDICT: APPROVED

Use VERDICT: CHANGES_REQUESTED instead when findings require changes. If unable to review, explain the blocker without an approval verdict.

@openhands-ai

openhands-ai Bot commented Oct 20, 2026

Copy link
Copy Markdown

I'm on it! ak684 can track my progress at all-hands.dev

@ak684 ak684 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exactly commit 6048e5b1fe79cbca974a81c127b9a7f62eb72635 (verified as the current PR head via the API before posting; no other commit was evaluated).

What this commit changes (rc.3 delta on top of rc.2)

Five version-only edits across 5 files, matching the PR's claim that rc.3 differs from rc.2 only in the enterprise image:

  • charts/openhands/Chart.yaml: version 0.84.0-rc.2 → 0.84.0-rc.3, appVersion sha-cc0ced6 → sha-40d51ea
  • charts/openhands/values.yaml: enterprise image.tag sha-cc0ced6 → sha-40d51ea (agent-server pins unchanged from rc.2)
  • Three agent-server pins remain 1.56.0-python in lockstep (umbrella global.agentServerImage, runtime-api subchart, image-loader); canvas stays sha-658a098

Correctness — claims check out

  • Agent-server / SDK sync passes. uv run scripts/check_agent_server_sync.py succeeds on this branch: sha-40d51ea resolves to commit 40d51ea in OpenHands/enterprise, whose pyproject.toml pins openhands-agent-server==1.56.0, openhands-sdk==1.56.0, openhands-tools==1.56.0 — matching the chart's 1.56.0-python pins. The sha- tag is handled explicitly by that script's _SHA_TAG_RE.
  • rc.3 enterprise delta is exactly one commit. cc0ced6 → 40d51ea is 1 commit ahead, 0 behind. It touches only managed_llm_key_ownership_processor.py (+6), its unit tests (+22), and two i18n files; no migration files. This substantiates "rc.3 differs from rc.2 only in the enterprise image, which adds the two PRs' last review follow-ups" and "DB migrations end at the same revision as 1.73.0."
  • Canvas pin resolves. sha-658a098 is the head commit of OpenHands/OpenHands#18725 (the 1.28.1 Canvas build with the #18719 history fix), as described. It is unchanged from rc.2.
  • appVersion == image.tag (both sha-40d51ea), so the appVersion-drift check is satisfied.
  • No stale 1.55.1 references remain anywhere in charts/, replicated/, or the Makefile; the Replicated overlay inherits the chart's agent-server tag rather than re-pinning it.

Regression coverage

The moved pins are the same shapes that passed on the previously reviewed heads (b24d578, 292438921, 61e69e36 had the full CI suite green: Helm Unit Tests, Validate Replicated, Pull Request Checks, Secret Checksum, Scan Docker Images). I re-ran the checksum scripts locally against this head: check_secret_checksum.py (47 fields) and check_litellm_checksum.py (27) both pass. Note: on this exact head only the pr title-lint run has completed so far — the broader pull_request suite (the head was pushed minutes before this review) had not appeared yet, so I could not confirm green status for 6048e5b1 itself. The delta vs the previously-green rc.2 head is only the two image/version strings, so risk is low, but worth confirming before shipping.

Non-blocking

  • charts/openhands/values.yaml documents "You must use a stable, semver tag. Do not use sha-based tags", and image.tag: sha-40d51ea contravenes it; the agent-canvas values header likewise says the default should track a published release. Both are deliberate for this release candidate and fully disclosed ("Do not merge… move to the release tag or be closed"), consistent with prior approved reviews — but this form should not reach main.

Verified head 6048e5b1fe79cbca974a81c127b9a7f62eb72635 matches the review target; no drift found.

REVIEWED_HEAD: 6048e5b
REVIEW_REQUEST: 3ba0b71dbf41468ea2f85f80aaaf3b0c
VERDICT: APPROVED

@openhands-ai

openhands-ai Bot commented Oct 20, 2026

Copy link
Copy Markdown

The review is complete and the PR head is unchanged. Here's the summary.

Review complete

Review posted: #1441 (review) (submitted as a full commentary review — the token belongs to ak684, who authored the PR, so GitHub rejects an official APPROVE status as self-approval; this is consistent with the three prior reviews in this thread).

Commit reviewed: 6048e5b1fe79cbca974a81c127b9a7f62eb72635, verified as the current PR head both before and after posting (it did not change during review, so no other commit was evaluated).

What I verified

The commit is the rc.3 delta (five version-only edits across 5 files), and the PR's validation claims match the evidence:

  • Agent-server / SDK sync — ran uv run scripts/check_agent_server_sync.py locally on the branch: sha-40d51ea resolves to commit 40d51ea in OpenHands/enterprise, whose pyproject.toml pins openhands-agent-server==1.56.0, openhands-sdk==1.56.0, openhands-tools==1.56.0, matching all three chart pins at 1.56.0-python.
  • rc.3 = rc.2 + one enterprise commit — cc0ced6 → 40d51ea is exactly 1 commit ahead touching only managed_llm_key_ownership_processor.py, its tests, and two i18n files; no migration files, confirming "DB migrations end at the same revision as 1.73.0."
  • Canvas pin sha-658a098 resolves to the head of chore: build a 1.28.1 Canvas image with the #18719 history fix (do not merge) OpenHands#18725 (1.28.1 build with #18719), unchanged from rc.2.
  • appVersion == image.tag (both sha-40d51ea), so the appVersion-drift check is satisfied.
  • No stale 1.55.1 refs anywhere in charts/, replicated/, or the Makefile; checksum scripts (check_secret_checksum.py — 47 fields, check_litellm_checksum.py — 27) both pass locally.

One note carried in the review: on this exact head only the title-lint workflow had completed when I checked — the broader pull_request CI suite (Helm Unit Tests, Validate Replicated, Pull Request Checks, scans) hadn't appeared for 6048e5b1 yet (head was pushed ~30 min before the review). The delta vs the previously-green rc.2 head is only the two image/version strings, so risk is low, but worth confirming CI is green before shipping.

Verdict: APPROVED (with the same non-blocking note as prior reviews that the sha- image tags contravene the values-file semver invariant, acceptable for this disclosed "do not merge" candidate but should not reach main).

@ak684 ak684 changed the title test: package OpenHands Enterprise 0.84.0-rc.3 test: package OpenHands Enterprise 0.84.0-rc.4 Oct 20, 2026
@ak684

ak684 commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

@OpenHands /codereview-roasted

Review exactly commit 6f843f1. If the PR head changes, report that and do not approve another commit. Review only: do not push code, merge, deploy, or change attribution. Check correctness, regression coverage, and whether the PR validation claims match the evidence. Keep code comments short. Post the substantive review and link it in your completion message. End the substantive review with these plain lines (no code fence), choosing one verdict:

REVIEWED_HEAD: 6f843f1
REVIEW_REQUEST: b94f073f459f4ea392edb0fe490f77aa
VERDICT: APPROVED

Use VERDICT: CHANGES_REQUESTED instead when findings require changes. If unable to review, explain the blocker without an approval verdict.

@openhands-ai

openhands-ai Bot commented Oct 20, 2026

Copy link
Copy Markdown

I'm on it! ak684 can track my progress at all-hands.dev

@ak684 ak684 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exactly commit 6f843f1f310f7120b84157ae2941e943b1ae999f (verified as the current PR head via the API both before and after review; no other commit was evaluated).

What this commit changes (rc.4 delta on top of rc.3)

Two version-only edits, exactly matching the PR's claim that "rc.4 differs from rc.3 only in the automation image":

  • charts/openhands/Chart.yaml: version 0.84.0-rc.3 → 0.84.0-rc.4 (appVersion unchanged at sha-40d51ea)
  • charts/openhands/charts/automation/values.yaml: image.tag 1.22.0 → 1.23.1

The rest of the PR (cumulative vs main base 08333cb, 6 files) is the same image/version pin set as the previously approved rc.1–rc.3 heads.

Correctness — claims check out

  • Enterprise / SDK / agent-server sync intact. Re-ran uv run scripts/check_agent_server_sync.py against this head: it resolves sha-40d51ea to commit 40d51ea in OpenHands/enterprise, whose pyproject.toml pins openhands-agent-server==1.56.0 / sdk / tools, matching the chart's three 1.56.0-python agent-server pins. Exit 0, "pins are in sync." (sha- tags are handled by the script's _SHA_TAG_RE; the round-trip also fails if the pinned commit were force-pushed away.)
  • The delta itself is low-risk. The rc.3→rc.4 diff touches only the chart version and the automation image.tag. The automation subchart is embedded/unversioned, and its templates render {{ .Values.image.repository }}:{{ .Values.image.tag }} on all three Deployments, so the bump flows through with no other wiring. Unit-test files under automation/tests/ assert no image tag and stay valid.
  • Automation 1.23.1 tag and release exist (refs/tags/1.23.1, release v1.23.1 published non-prerelease), satisfying the chart comment's "stable, semver tag" rule for this component.
  • appVersion == image.tag (both sha-40d51ea), satisfying the appVersion-drift check.
  • No stale references to the old tags found anywhere in charts/, replicated/, or the Makefile.

Regression coverage

The moved pins are the same shapes that passed on the previously reviewed and green heads (b24d578, 292438921, 61e69e36, 6048e5b1 ran Helm Unit Tests, Validate Replicated, Pull Request Checks, Secret Checksum, Scan Docker Images green; the rc.1–rc.3 KOTS upgrade + live Canvas/automation checks are documented in the PR body and were approved previously). As with rc.3, on this exact head only the pr title-lint run has completed — the broad pull_request CI suite had not (re)appeared for 6f843f1 at review time. The delta vs the previously-green rc.3 head is two version strings and a production semver automation tag, so risk is low, but worth confirming the suite is green on this head before it reaches a wide channel.

Non-blocking

  • Same deliberate-release-candidate caveat as prior approvals: image.tag: sha-40d51ea contravenes the "use a stable, semver tag" comment in values.yaml, and agent-canvas tracks sha-658a098 instead of a published tag. Both are fully disclosed in the PR body ("Do not merge… move to release tags or close") and consistent with the previously approved reviews — this form should not reach main.

Verified head 6f843f1f310f7120b84157ae2941e943b1ae999f matches the review target; no drift found.

REVIEWED_HEAD: 6f843f1
REVIEW_REQUEST: b94f073f459f4ea392edb0fe490f77aa
VERDICT: APPROVED

@openhands-ai

openhands-ai Bot commented Oct 20, 2026

Copy link
Copy Markdown

The head remains unchanged. The review is complete and posted.

Review complete

Review link: #1441 (review)

Commit reviewed: 6f843f1f310f7120b84157ae2941e943b1ae999f — verified as the current PR head both before and after posting. The head did not change during review, so no other commit was evaluated.

Posting note: Submitted as a commentary review (state COMMENTED) because the token belongs to ak684, who authored the PR — GitHub rejects an official APPROVE status as self-approval, consistent with the four earlier reviews in this thread. The verdict is recorded in the review body.

What I verified

  • Scope matches the claim. The rc.4 delta is exactly two version-string edits: chart 0.84.0-rc.3 → rc.4 and automation image.tag 1.22.0 → 1.23.1, confirming "rc.4 differs from rc.3 only in the automation image."
  • Agent-server/enterprise SDK sync passes. Ran uv run scripts/check_agent_server_sync.py: sha-40d51ea resolves to commit 40d51ea in OpenHands/enterprise, whose pyproject.toml pins openhands-agent-server==1.56.0/sdk/tools — matching the chart's three 1.56.0-python pins.
  • Automation 1.23.1 exists (refs/tags/1.23.1, non-prerelease release v1.23.1), and the tag bump flows through the embedded subchart's {{ .Values.image.tag }} with no other wiring; the automation unit tests assert no tag.
  • appVersion == image.tag (sha-40d51ea); no stale old-tag references anywhere.
  • CI note (non-blocking): consistent with the rc.3 review, only the pr title-lint suite had completed for this exact head at review time; the broader pull_request CI suite (Helm unit tests, etc.) had not reappeared, so worth confirming green before this ships wider.
REVIEWED_HEAD: 6f843f1f310f7120b84157ae2941e943b1ae999f
REVIEW_REQUEST: b94f073f459f4ea392edb0fe490f77aa
VERDICT: APPROVED

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: test Adding or fixing tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant