Skip to content

Commit eb4eff2

Browse files
cardoeclaude
cardoe
and
claude
committed
ci: diff the manifests ArgoCD renders on chart bumps
Renovate's OpenStack Helm pull requests are a single changed chartVersion, which says nothing about what lands in the cluster. Add a tool that renders both sides of the change the way ArgoCD's repo-server does and diffs the result, plus a workflow that runs it on pull requests and comments the summary. The render inputs are not hardcoded. The script runs helm template on charts/argocd-understack, reads the resulting Application resources, and takes the chart, repository, version, release name, destination namespace and the ordered valueFiles list from them, so it cannot drift from what ArgoCD actually does. Each run records the exact helm template command line it used for both sides. Both sides come from a git worktree, so a change to a components value file is compared as accurately as a chartVersion bump. The deploy repo's value files are skipped by default, matching ignoreMissingValueFiles on these Applications, and can be supplied with --deploy-repo; components/openstack-secrets.tpl.yaml is filled in with placeholders so the endpoints tree is present on both sides. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U1c2Qsc5agM715tVe96sBx
1 parent 442136d commit eb4eff2

4 files changed

Lines changed: 1502 additions & 0 deletions

File tree

‎.github/workflows/openstack-helm-chart-diff.yaml‎

Lines changed: 167 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,167 @@
1+
---
2+
name: OpenStack Helm chart diff
3+
4+
# Renders the manifests ArgoCD would produce on both sides of a pull request
5+
# and posts the difference. The point is renovate's chart bumps: the pull
6+
# request itself only moves a `chartVersion`, which says nothing about what
7+
# lands in the cluster.
8+
#
9+
# This is informational and never fails the pull request. What it reports is
10+
# input to a human deciding whether a bump is safe to merge.
11+
on:
12+
pull_request:
13+
paths:
14+
# a chartVersion, or the Application definition that carries one
15+
- "charts/argocd-understack/values.yaml"
16+
- "charts/argocd-understack/templates/application-*.yaml"
17+
- "charts/argocd-understack/templates/_helpers.tpl"
18+
# the value files those Applications feed to the charts
19+
- "components/**/values.yaml"
20+
- "components/*-values.yaml"
21+
- "components/images-openstack.yaml"
22+
- "components/openstack-secrets.tpl.yaml"
23+
- "scripts/osh-chart-diff.py"
24+
- ".github/workflows/openstack-helm-chart-diff.yaml"
25+
workflow_dispatch:
26+
inputs:
27+
base:
28+
description: "Git ref to compare against"
29+
required: false
30+
default: "main"
31+
type: string
32+
app:
33+
description: "Single Application to render (e.g. neutron); blank for all changed"
34+
required: false
35+
type: string
36+
render_all:
37+
description: "Render every app, not only the ones that changed"
38+
required: false
39+
default: false
40+
type: boolean
41+
42+
permissions: {}
43+
44+
concurrency:
45+
group: ${{ github.workflow }}-${{ github.ref }}
46+
cancel-in-progress: true
47+
48+
jobs:
49+
chart-diff:
50+
name: Diff rendered manifests
51+
runs-on: ubuntu-latest
52+
permissions:
53+
contents: read
54+
# Needed to post the summary comment. Pull requests from forks get a
55+
# read-only token, so the comment step tolerates failure and the report
56+
# is always written to the run summary as well.
57+
pull-requests: write
58+
steps:
59+
- name: Checkout
60+
uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6
61+
with:
62+
# The script compares two git worktrees, so it needs real history.
63+
fetch-depth: 0
64+
65+
- name: Set up Helm
66+
uses: azure/setup-helm@9bc31f4ebc9c6b171d7bfbaa5d006ae7abdb4310 # v5
67+
with:
68+
version: v3.18.4
69+
70+
- name: Set up Python
71+
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7
72+
with:
73+
python-version: "3.12"
74+
check-latest: true
75+
76+
- name: Install dependencies
77+
run: pip install pyyaml
78+
79+
- name: Determine base ref
80+
id: base
81+
env:
82+
EVENT_BASE: ${{ github.event.pull_request.base.ref }}
83+
INPUT_BASE: ${{ inputs.base }}
84+
run: |
85+
set -euo pipefail
86+
base="${EVENT_BASE:-${INPUT_BASE:-main}}"
87+
# actions/checkout leaves the base branch as a remote ref for
88+
# pull_request events, but fetch it explicitly so workflow_dispatch
89+
# and shallow-ish checkouts behave the same way.
90+
git fetch --no-tags --quiet origin "+refs/heads/${base}:refs/remotes/origin/${base}"
91+
echo "ref=origin/${base}" >> "$GITHUB_OUTPUT"
92+
93+
- name: Render and diff
94+
id: diff
95+
env:
96+
BASE_REF: ${{ steps.base.outputs.ref }}
97+
ONLY_APP: ${{ inputs.app }}
98+
RENDER_ALL: ${{ inputs.render_all }}
99+
run: |
100+
set -euo pipefail
101+
out="${RUNNER_TEMP}/osh-chart-diff"
102+
mkdir -p "$out"
103+
104+
args=(--base "$BASE_REF" --out "$out" --summary "${out}/comment.md")
105+
# GitHub rejects comment bodies over 65536 characters.
106+
args+=(--summary-limit 60000)
107+
if [ -n "$ONLY_APP" ]; then
108+
args+=(--app "$ONLY_APP")
109+
fi
110+
if [ "${RENDER_ALL}" = "true" ]; then
111+
args+=(--all)
112+
fi
113+
114+
python3 scripts/osh-chart-diff.py "${args[@]}"
115+
echo "out=${out}" >> "$GITHUB_OUTPUT"
116+
echo "changed=$(cat "${out}/changed")" >> "$GITHUB_OUTPUT"
117+
118+
- name: Add to run summary
119+
if: always() && steps.diff.outputs.out != ''
120+
env:
121+
OUT: ${{ steps.diff.outputs.out }}
122+
run: cat "${OUT}/summary.md" >> "$GITHUB_STEP_SUMMARY"
123+
124+
- name: Upload full diffs
125+
if: always() && steps.diff.outputs.out != ''
126+
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
127+
with:
128+
name: openstack-helm-chart-diff
129+
path: ${{ steps.diff.outputs.out }}
130+
retention-days: 14
131+
132+
- name: Comment on the pull request
133+
if: github.event_name == 'pull_request' && steps.diff.outputs.out != ''
134+
continue-on-error: true
135+
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
136+
env:
137+
OUT: ${{ steps.diff.outputs.out }}
138+
CHANGED: ${{ steps.diff.outputs.changed }}
139+
with:
140+
script: |
141+
const fs = require('fs');
142+
const marker = '<!-- osh-chart-diff -->';
143+
const body = marker + '\n' +
144+
fs.readFileSync(`${process.env.OUT}/comment.md`, 'utf8');
145+
146+
const { owner, repo } = context.repo;
147+
const issue_number = context.issue.number;
148+
149+
const existing = await github.paginate(
150+
github.rest.issues.listComments,
151+
{ owner, repo, issue_number, per_page: 100 },
152+
);
153+
const mine = existing.find((c) => c.body.includes(marker));
154+
155+
if (mine) {
156+
// Keep an existing comment current even when the newest push
157+
// renders identically, so it never sits there stale.
158+
await github.rest.issues.updateComment({
159+
owner, repo, comment_id: mine.id, body,
160+
});
161+
} else if (process.env.CHANGED === 'true') {
162+
await github.rest.issues.createComment({
163+
owner, repo, issue_number, body,
164+
});
165+
} else {
166+
core.info('Nothing changed in the rendered manifests, not commenting.');
167+
}

‎docs/operator-guide/openstack-helm-chart-diff.md‎

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
# Diffing OpenStack Helm chart bumps
2+
3+
Renovate opens a pull request for every new [OpenStack Helm][OSH] chart
4+
release. The pull request diff is one line:
5+
6+
``` diff
7+
- chartVersion: 2026.1.40+abc1234
8+
+ chartVersion: 2026.1.41+7c81bea7c
9+
```
10+
11+
That says nothing about what actually changes in the cluster. Upstream may
12+
have retimed a probe, renamed a config key, added a `PodDisruptionBudget`,
13+
changed a default we override, or changed nothing that we render at all.
14+
`scripts/osh-chart-diff.py` answers that question by rendering the manifests
15+
on both sides of the pull request and diffing the result.
16+
17+
## Why the render matches ArgoCD
18+
19+
None of the render inputs are hardcoded in the script. It runs `helm template`
20+
on `charts/argocd-understack` to produce the real `Application` resources, then
21+
reads each one back to find:
22+
23+
* `sources[0].chart`, `.repoURL` and `.targetRevision` — which chart, from
24+
where, at which version
25+
* `sources[0].helm.releaseName` — so `.Release.Name` matches, which OpenStack
26+
Helm uses in resource names and labels
27+
* `spec.destination.namespace` — so `.Release.Namespace` matches
28+
* `sources[0].helm.valueFiles` — the value files, **in the order ArgoCD
29+
applies them**, which is what decides who wins on a conflicting key
30+
31+
It then reproduces the invocation ArgoCD's repo-server makes:
32+
33+
``` bash
34+
helm template <releaseName> <chart> --include-crds \
35+
--namespace <destination.namespace> --kube-version <version> \
36+
--values <each valueFile, in order>
37+
```
38+
39+
Because the inputs come from the chart, adding an app or reordering a value
40+
file list is picked up with no change to the script. Every run writes the
41+
exact command line it used for each side to `<app>/render-inputs.txt` so you
42+
can confirm this yourself rather than taking it on faith.
43+
44+
Both sides come from a `git worktree`, one at the merge base and one at your
45+
tree, so a pull request that changes `components/<app>/values.yaml` or
46+
`components/images-openstack.yaml` is compared correctly too, not just a
47+
`chartVersion` bump.
48+
49+
## What it does not have
50+
51+
The `Application` also pulls value files from the deploy repository:
52+
53+
``` yaml
54+
- $deploy/<site>/secret-openstack.yaml
55+
- $deploy/<site>/images-openstack.yaml
56+
- $deploy/<site>/<app>/values.yaml
57+
```
58+
59+
Those are site-specific and live outside this repository, so by default they
60+
are skipped — which is legitimate, because these `Application`s set
61+
`ignoreMissingValueFiles: true` and ArgoCD skips them too when they are
62+
absent. `components/openstack-secrets.tpl.yaml` is the exception: the script
63+
fills it in with placeholder values so the `endpoints:` tree, which drives a
64+
lot of what OpenStack Helm emits, is present in the render. The same
65+
placeholders go into both sides, so nothing they touch appears as a
66+
difference.
67+
68+
For the highest fidelity, point the script at a checkout of your deploy
69+
repository and it will use the real values instead:
70+
71+
``` bash
72+
./scripts/osh-chart-diff.py --deploy-repo ~/src/undercloud-deploy \
73+
--site-name uc-iad3-prod
74+
```
75+
76+
The other gap is cluster capabilities. ArgoCD forwards the destination
77+
cluster's Kubernetes version and API versions to `helm template`, so a chart
78+
that gates a manifest on `.Capabilities` can render differently against a real
79+
cluster. The script pins `--kube-version` (see `--help` for the default) to
80+
keep runs reproducible; pass your site's version to match it.
81+
82+
## Running it
83+
84+
Against the current branch, comparing to `origin/main`:
85+
86+
``` bash
87+
./scripts/osh-chart-diff.py
88+
```
89+
90+
By default only the apps whose render inputs changed are rendered, which for a
91+
renovate pull request is exactly the one chart that moved. Useful options:
92+
93+
| Option | Purpose |
94+
| --- | --- |
95+
| `--base <ref>` | What to compare against, default `origin/main` |
96+
| `--head <ref>` | Compare a ref instead of your working tree |
97+
| `--app <name>` | Just one app, e.g. `--app neutron` |
98+
| `--all` | Render every app, not only the changed ones |
99+
| `--deploy-repo <path>` | Use the deploy repo's value files |
100+
| `--kube-version <ver>` | Match your cluster's Kubernetes version |
101+
| `--out <dir>` | Where to write the full diffs |
102+
| `--fail-on-change` | Exit non-zero when any manifest differs |
103+
104+
Each app gets a directory under `--out`:
105+
106+
| File | Contents |
107+
| --- | --- |
108+
| `manifests.diff` | The rendered manifest diff. This is the one to read. |
109+
| `chart-values.diff` | Upstream's default `values.yaml`, before and after |
110+
| `chart-source.diff` | The chart's own templates, before and after |
111+
| `render-inputs.txt` | The exact `helm template` command used for each side |
112+
| `before.yaml`, `after.yaml` | The full renders |
113+
114+
Manifests are split one resource per document and keyed by kind and name
115+
before diffing, so a template being reordered upstream does not show up as
116+
every resource having moved. Multi-line values keep block form, so the
117+
contents of a `ConfigMap` — the `neutron.conf` that OpenStack Helm assembles,
118+
for instance — still diff line by line.
119+
120+
## In CI
121+
122+
The `OpenStack Helm chart diff` workflow runs this on every pull request that
123+
touches a `chartVersion` or one of the value files those `Application`s
124+
consume, and posts the summary as a comment. The full diffs are attached to
125+
the workflow run as an artifact, since a large one will not fit in a comment.
126+
127+
The workflow is informational and never fails the pull request. Reading it is
128+
the point: it tells you whether a bump is a no-op, a config change worth
129+
reviewing, or something that needs a matching change in
130+
`components/<app>/values.yaml` before it can merge.
131+
132+
[OSH]: <https://docs.openstack.org/openstack-helm/latest/>

‎properdocs.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -263,6 +263,7 @@ nav:
263263
- operator-guide/server-firmware-update.md
264264
- 'Scripts and Tools':
265265
- operator-guide/scripts.md
266+
- operator-guide/openstack-helm-chart-diff.md
266267
- operator-guide/understackctl.md
267268
# Keep this list newest-series-first and maintain it by hand. Do not switch it
268269
# to include_dir_to_nav: that plugin only does an ascending ASCII sort, so

0 commit comments

Comments
 (0)