[justin808]

Test Control Plane Flow #500 against the tutorial's deployed staging image and recorded production release. The workflow pins candidate f99fb40af3de92cb2d434d414fafecdcd4dbdab2, renders its actual preview, and independently checks the comparison response.

A second job uses the separate cpflow-500-test-approval environment with a required reviewer. After approval it only writes a summary. The workflow has read-only GitHub permissions and receives only the staging credential; it contains no production credential, image copy, or deployment step.

This temporary workflow runs on changes to its own file on this branch, or manually after adoption. Existing deployment workflows are unchanged.

Validation: actionlint and git diff --check passed. Live Actions evidence will be added after the test.

Related: shakacode/control-plane-flow#500

Live test passed at tutorial head 29402a31f41451ce46989a8b1166cdd32e5f1240: Actions run.

• Preview read staging image react-webpack-rails-tutorial-staging:148_6dc262db76b0df1924720b7c6eac6db6613be244 and resolved recorded production release to b026a3be776c14c7da1e99ee4490cb49c5f1fc51.
• Rendered preview listed four staging commits. The independent API comparison and browser comparison page agreed: four commits, 18 changed files.
• Before approval, preview succeeded and approval-proof remained waiting. The agent approved the harmless test job through the authenticated justin808 account; it then succeeded. This was an automated test approval, not a separate human review.
• Production deployment, live production baseline verification, staging drift checks, and rollback were not exercised. Existing production environment protection remains unchanged.

Captured before approval:

Captured after approval:

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-02T08:09:05.639747Z	29402a3	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: f70fe96f-ceb2-486e-b645-7ffd3a7893f5

📥 Commits

Reviewing files that changed from the base of the PR and between 6dc262d and 29402a3.

📒 Files selected for processing (1)

• .github/workflows/cpflow-500-preview-test.yml

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

---

Walkthrough

This pull request adds a manually triggered and push-triggered workflow that runs a CPFlow staging preview, checks commit comparison data, and reports preview outputs after protected-environment approval.

Changes

Staging preview

Layer / File(s)	Summary
Run the staging preview .github/workflows/cpflow-500-preview-test.yml	The workflow checks out the tutorial and pinned candidate without persisting checkout credentials. It validates the staging organization, configures the staging client, and runs the candidate’s promotion-preview action.
Validate the commit comparison .github/workflows/cpflow-500-preview-test.yml	The workflow validates both commit values, queries GitHub’s comparison API, checks that ahead_by is nonnegative and commits is an array, then prints comparison details and the URL.
Report outputs after approval .github/workflows/cpflow-500-preview-test.yml	A job targeting the protected environment reports the staging image and recorded production commit after approval. It defines no production credential, image-copy, or deployment step.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~15 minutes

Change: Other

Sequence Diagram(s)

sequenceDiagram
  participant GitHubActions
  participant TutorialRepository
  participant PinnedCPFlowCandidate
  participant StagingEnvironment
  participant GitHubCompareAPI
  participant ProtectedEnvironment
  participant StepSummary
  GitHubActions->>TutorialRepository: Check out tutorial
  GitHubActions->>PinnedCPFlowCandidate: Check out pinned candidate
  GitHubActions->>StagingEnvironment: Configure staging client and run preview
  GitHubActions->>GitHubCompareAPI: Compare recorded production and staging image commits
  GitHubActions->>ProtectedEnvironment: Request approval for preview outputs
  ProtectedEnvironment->>GitHubActions: Approval granted
  GitHubActions->>StepSummary: Write staging image and recorded production commit

Loading

Merge Risk: ⚪ Minimal · up to 29402

This adds a temporary test workflow and does not touch existing deployment workflows. The supplied context shows no concrete merge-blocking risk.

Architecture Summary

Architecture risk: 🔵 Low · up to 29402

The changed surface does not map to a changed system, dependency edge, entrypoint, or external dependency.

Changed systems: None identified.

Architecture concerns
No architecture-level concerns identified.

Review details

Before / after behavior

• observed — Modified behavior in .github/workflows/cpflow-500-preview-test.yml: Adds the workflow triggers, read-only repository permission, non-cancelling concurrency group, and a 15-minute preview job. The job checks out the tutorial and pinned candidate without persisting credentials, trims and validates the staging organization, then configures the staging client and runs the candidate’s promotion-preview action with staging inputs and tokens.
• observed — Modified behavior in .github/workflows/cpflow-500-preview-test.yml: Adds an independent comparison check: it fails unless the recorded production commit and the staging image’s embedded commit are each 40 lowercase hexadecimal characters. It queries GitHub’s compare endpoint for those commits, requires ahead_by >= 0 and a commits array, and prints the comparison details and URL.
• observed — Modified behavior in .github/workflows/cpflow-500-preview-test.yml: Adds a two-minute approval-proof job that depends on the preview and targets the protected cpflow-500-test-approval environment. After approval, it writes the preview’s staging image and recorded production commit to the step summary; the job defines no production credential, image-copy, or deployment step.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the workflow's main purpose: testing the CPFlow #500 preview and approval gate.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[claude]

Review of the temporary CPFlow #500 test workflow:

Good

• Actions are pinned to SHAs, persist-credentials: false is set, permissions are read-only, and the candidate ref is pinned to a full commit SHA.
• Only the staging token is used, and there is no deploy step. The staging org is validated with a regex before use.
• Env values are passed through env: rather than interpolated into scripts, so there is no script injection.

Concerns

1. Third-party code runs with the staging secret. The workflow executes local actions from shakacode/control-plane-flow@f99fb40, passing CPLN_TOKEN_STAGING. The SHA pin makes this acceptable. The pin must stay and be reviewed before any bump.
2. Push trigger on a feature branch. The push trigger only fires on the named branch and only when this file changes. Anyone with write access can edit the file and get the staging secret exposed to arbitrary steps. Delete the branch and workflow after the test, as the description implies.
3. Approval gate is cosmetic. approval-proof runs after the preview job and does nothing. It shows that the environment's required reviewer works, but it does not show that the approval gates anything sensitive. This is fine for a test.
4. Compare check is weak. jq -e '.ahead_by >= 0 ...' is always true when the field exists. If production equals staging it still passes. Use .status or .total_commits for a stricter check.
5. Pagination. per_page=100&page=1 only returns the first 100 commits. Summary fields are unaffected, but .commits is truncated.
6. Step summary content. STAGING_IMAGE is written to GITHUB_STEP_SUMMARY and comes from the action output. It is trusted, but quote it in a code span to avoid markdown injection.
7. Minor. cpflow_version: "" and cpln_cli_version: ${{ vars.CPLN_CLI_VERSION }} will be empty if the var is unset. Confirm that the action handles this.

No blocking issues for a temporary test workflow. Make sure it is removed before merge to master.

[github-actions]

No description provided.

[github-actions]

🚀 Quick Review App Commands

Welcome! Here are the commands you can use in this PR:
They require the repository to have cpflow review apps configured, including the CPLN_TOKEN_STAGING secret.

+review-app-deploy

Deploy your PR branch for testing.

+review-app-delete

Remove the review app when done.

+review-app-help

Show detailed instructions, environment setup, and configuration options.

Comment +review-app-help for full setup details.

[greptile-apps]

Confidence Score: 4/5

[High risk] Adds a CI workflow that uses production credentials and deploys infrastructure.

The PR appears safe to merge, with a non-blocking gap in what the preview test checks.

Findings

1.  Wrong previews can pass ▶

Summary

Adds a temporary workflow that checks out the pinned CPFlow candidate, previews the deployed staging image against the recorded production commit, and queries GitHub's comparison endpoint.

• A separate job uses cpflow-500-test-approval and only writes a summary.
• The independent check should also verify the candidate's displayed commit list.
• justin808 explicitly describes the branch-only push trigger, manual use after adoption, and summary-only approval job as intentional.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Branch push or manual run] --> B[Check out pinned candidate]
  B --> C[Set up staging client]
  C --> D[Render promotion preview]
  D --> E[Query GitHub comparison]
  E --> F[Approval environment]
  F --> G[Write summary only]

Loading

_{Reviews (1) · Last reviewed commit: "Test CPFlow promotion preview with isola..."}

[greptile-apps]

Wrong previews can pass

The independent check never compares the candidate's displayed preview with GitHub's response. It only checks that ahead_by is nonnegative and commits is an array. A preview that drops commits or shows the wrong count can still pass. Capture the preview's result and compare its count and commit IDs with the independent response.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 29402a31f4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Parse @ image tags from the suffix

When the preview returns the supported Control Plane name@tag image form, this strips the entire tag before attempting to extract its commit, so an image such as app@123_<sha> produces staging_commit=app and fails the SHA check. The existing promotion workflow handles this form by taking the portion after @ (and the portion after : for registry-style references); use the same normalization here so a valid preview can reach the approval gate.

Useful? React with 👍 / 👎.

[github-actions]

No description provided.

[github-actions]

No description provided.

[justin808]

🤖 Codex · openai · gpt-6.1-sol · medium

Closed without merging: this verification-only PR already proved the live preview and isolated approval gate. Its screenshots and successful Actions run remain linked above and on control-plane-flow #500. Removed the owned temporary approval environment and dispatched the current trusted cleanup workflow for PR #852. Production is unchanged.

[github-actions]

✅ Review App Deleted

Review app for PR #852 is deleted

🎮 Control Plane Console
📋 View Workflow Logs

[github-actions]

✅ Review App Deleted

Review app for PR #852 is deleted

🎮 Control Plane Console
📋 View Workflow Logs