Skip to content

Commit f0fd8b3

Browse files
authored
Add GHCR package retention workflow (#124)
Replace ad hoc image cleanup with a tested retention policy that inventories all published GHCR packages, generates dry-run reports, and supports manual deletion with protected version pins. Document the retention rules and link them from release verification.
1 parent f37f13a commit f0fd8b3

6 files changed

Lines changed: 425 additions & 20 deletions

File tree

.github/workflows/cleanup.yml

Lines changed: 44 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -12,35 +12,59 @@ on:
1212
- arc-runner
1313
- ubuntu-latest
1414
default: arc-runner
15+
delete_packages:
16+
description: 'Delete eligible versions (review a dry-run report first)'
17+
type: boolean
18+
default: false
19+
protected_versions:
20+
description: 'Deployed/rollback versions to preserve, comma-separated (in addition to policy pins)'
21+
type: string
22+
default: ''
1523

1624
permissions:
17-
packages: write
18-
issues: write
19-
pull-requests: write
25+
contents: read
2026

2127
jobs:
22-
maintenance:
23-
name: Cleanup & Stale
28+
package-retention:
29+
name: Package retention
2430
runs-on: ${{ inputs.runner || 'arc-runner' }}
31+
concurrency:
32+
group: package-retention
33+
cancel-in-progress: false
34+
permissions:
35+
contents: read
36+
packages: write
2537
steps:
26-
- name: Cleanup backend images
27-
uses: actions/delete-package-versions@v5
28-
continue-on-error: true
38+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
39+
- uses: actions/setup-node@v4
2940
with:
30-
package-name: teslasync
31-
package-type: container
32-
min-versions-to-keep: 10
33-
delete-only-untagged-versions: true
34-
35-
- name: Cleanup web images
36-
uses: actions/delete-package-versions@v5
37-
continue-on-error: true
41+
node-version: '22'
42+
- name: Test retention policy
43+
run: node --test scripts/package-retention.test.mjs
44+
- name: Inventory and apply retention policy
45+
env:
46+
GH_TOKEN: ${{ github.token }}
47+
DELETE_PACKAGES: ${{ github.event_name == 'workflow_dispatch' && inputs.delete_packages == true }}
48+
PROTECTED_VERSIONS: ${{ inputs.protected_versions }}
49+
run: node scripts/package-retention.mjs
50+
- name: Upload retention report
51+
if: always()
52+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
3853
with:
39-
package-name: teslasync-web
40-
package-type: container
41-
min-versions-to-keep: 10
42-
delete-only-untagged-versions: true
54+
name: package-retention-report
55+
path: |
56+
package-retention-plan.json
57+
package-retention-results.jsonl
58+
retention-days: 90
59+
if-no-files-found: warn
4360

61+
maintenance:
62+
name: Stale issues and PRs
63+
runs-on: ${{ inputs.runner || 'arc-runner' }}
64+
permissions:
65+
issues: write
66+
pull-requests: write
67+
steps:
4468
- name: Mark stale issues and PRs
4569
uses: actions/stale@v9
4670
with:
Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
# Package retention
2+
3+
The Maintenance workflow inventories all seven GHCR packages listed in
4+
`ops/release/package-retention.json`, including `charts/teslasync`.
5+
It applies to existing versions as well as future publications.
6+
7+
Stable releases retain the newest 10 releases **or** anything published/updated
8+
within 90 days. Prereleases retain the newest 5 **or** anything within 14 days.
9+
Newest means the most recent package creation/update across the release's
10+
components, not semantic version order. A version must fall outside both
11+
protections before it becomes a deletion candidate.
12+
The count floor also applies to each individual package; a less frequently
13+
published package cannot lose its rollback history to newer releases of other
14+
components. Those retained versions protect matching components everywhere.
15+
16+
## Review and enable deletion
17+
18+
1. Add deployed and required rollback versions to `protectedVersions` in the
19+
policy. The workflow cannot discover installations in your homelab.
20+
2. Run Maintenance with `delete_packages` left false. Scheduled weekly runs
21+
are also always dry-runs.
22+
3. Review the job summary and the `package-retention-report` artifact. Its JSON
23+
lists every package version, tag, keep/delete decision, and reason.
24+
4. To delete, manually run Maintenance with `delete_packages` true. Optional
25+
`protected_versions` adds comma-separated pins for that run; use the policy
26+
file for durable pins. The workflow recomputes the plan against live data,
27+
rather than executing the previous report.
28+
29+
The workflow token needs package admin access for deletion on **each** package.
30+
Grant this repository access in the package's Actions access settings as needed.
31+
An inaccessible, missing, empty, or malformed inventory aborts cleanup before
32+
deletion. HTTP failures are reported, not ignored.
33+
34+
## Safety boundaries
35+
36+
Release decisions are shared across all packages. An alias such as `latest`
37+
protects every release tag on its digest, across the entire release. Pins and
38+
shared release digests propagate protection as well. Versions with unknown tags
39+
are retained.
40+
41+
**Untagged manifests, signatures, SBOMs, attestations, and other auxiliary
42+
artifacts are deliberately retained.** The GitHub package-version API alone
43+
does not establish an OCI reference graph; this policy does not attempt orphan
44+
garbage collection. Consequently, retention is not a strict cap on the total
45+
number of package versions or total storage.
46+
47+
The script inventories everything again before deletion and checks candidate
48+
digests/tags. Do not retag or publish packages during a deletion run: GitHub
49+
does not support conditional deletes, so these checks cannot eliminate races
50+
with external publishers. Release tags must remain immutable.
51+
52+
Deletion across packages is not atomic. On the first failure the run stops;
53+
the results JSONL records successful deletions and the failure. Inspect the
54+
partial result and fix permissions or the reported error before retrying.
55+
Remaining versions are re-inventoried on retry.
56+
57+
GitHub releases and Git tags are never deleted. Their historical package links
58+
may stop working after expiration. Existing running containers are not stopped,
59+
but deleted images cannot be pulled for recreation or rollback. Retaining an
60+
image does not make rollback safe across incompatible database migrations.

docs/operations/release-verification.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,9 @@ Policy: `ops/release/supply-chain.yaml`
77
Gate: `go run ./cmd/ops-gate -check supply-chain`
88
Producer: `.github/workflows/release.yml`
99

10+
Published package lifetime and protected rollback versions are covered by the
11+
[package retention policy](./package-retention.md).
12+
1013
## What every published image carries
1114

1215
| Artifact | Produced by | Proves |

ops/release/package-retention.json

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"packages": [
3+
"teslasync-api",
4+
"teslasync-web",
5+
"teslasync-notification-worker",
6+
"teslasync-export-worker",
7+
"teslasync-automation-worker",
8+
"teslasync-fleet-telemetry",
9+
"charts/teslasync"
10+
],
11+
"stable": { "keep": 10, "days": 90 },
12+
"prerelease": { "keep": 5, "days": 14 },
13+
"protectedVersions": []
14+
}

scripts/package-retention.mjs

Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
import { readFile, writeFile, appendFile } from 'node:fs/promises';
2+
import { pathToFileURL } from 'node:url';
3+
4+
const releaseTag = /^(?:v)?(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(-[0-9A-Za-z.-]+)?$/;
5+
const normalize = (tag) => tag.replace(/^v/, '');
6+
7+
export function planRetention(policy, inventory, now = Date.now(), protectedVersions = []) {
8+
if (!Array.isArray(policy.packages) || policy.packages.length === 0 ||
9+
new Set(policy.packages).size !== policy.packages.length) {
10+
throw new Error('Policy must contain unique package names');
11+
}
12+
for (const tier of ['stable', 'prerelease']) {
13+
if (!Number.isInteger(policy[tier]?.keep) || policy[tier].keep < 1 ||
14+
!Number.isInteger(policy[tier]?.days) || policy[tier].days < 1) {
15+
throw new Error(`Invalid ${tier} retention policy`);
16+
}
17+
}
18+
const protectedTags = [...policy.protectedVersions, ...protectedVersions];
19+
if (protectedTags.some((tag) => !releaseTag.test(tag))) {
20+
throw new Error('Protected versions must be release version tags');
21+
}
22+
const keep = new Set(protectedTags.map(normalize));
23+
const releases = new Map();
24+
const rows = [];
25+
for (const pkg of policy.packages) {
26+
if (!Array.isArray(inventory[pkg]) || inventory[pkg].length === 0) {
27+
throw new Error(`Missing or empty package inventory: ${pkg}`);
28+
}
29+
for (const version of inventory[pkg]) {
30+
const tags = version.metadata?.container?.tags;
31+
const created = Date.parse(version.created_at);
32+
const updated = Date.parse(version.updated_at ?? version.created_at);
33+
if (!Array.isArray(tags) || tags.some((tag) => typeof tag !== 'string') ||
34+
!Number.isSafeInteger(version.id) || !Number.isFinite(created) || !Number.isFinite(updated)) {
35+
throw new Error(`Invalid version metadata in ${pkg}`);
36+
}
37+
const names = tags.filter((tag) => releaseTag.test(tag)).map(normalize);
38+
const row = { package: pkg, id: version.id, digest: version.name, tags, releases: names };
39+
rows.push(row);
40+
for (const name of names) {
41+
// A recently republished component protects the entire release.
42+
releases.set(name, Math.max(releases.get(name) ?? 0, created, updated));
43+
if (tags.some((tag) => !releaseTag.test(tag))) keep.add(name);
44+
}
45+
}
46+
}
47+
for (const tier of ['stable', 'prerelease']) {
48+
const ordered = [...releases].filter(([name]) => name.includes('-') === (tier === 'prerelease'))
49+
.sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0]));
50+
ordered.forEach(([name, date], index) => {
51+
if (index < policy[tier].keep || now - date <= policy[tier].days * 86400000) keep.add(name);
52+
});
53+
// Older or less frequently published packages still need rollback coverage.
54+
for (const pkg of policy.packages) {
55+
const available = new Set(rows.filter((row) => row.package === pkg).flatMap((row) => row.releases));
56+
ordered.filter(([name]) => available.has(name)).slice(0, policy[tier].keep)
57+
.forEach(([name]) => keep.add(name));
58+
}
59+
}
60+
// Shared digests cannot be removed without removing every tag on them.
61+
let changed;
62+
do {
63+
changed = false;
64+
for (const row of rows) {
65+
if (row.releases.some((name) => keep.has(name))) {
66+
for (const name of row.releases) {
67+
if (!keep.has(name)) { keep.add(name); changed = true; }
68+
}
69+
}
70+
}
71+
} while (changed);
72+
return rows.map((row) => ({
73+
...row,
74+
action: row.releases.length > 0 && row.tags.every((tag) => releaseTag.test(tag)) &&
75+
row.releases.every((name) => !keep.has(name)) ? 'delete' : 'keep',
76+
reason: row.releases.length === 0 ? 'untagged or auxiliary artifact: preserved' :
77+
row.tags.some((tag) => !releaseTag.test(tag)) ? 'alias or auxiliary tag: preserved' :
78+
row.releases.some((name) => keep.has(name)) ? 'retention window, pin, or shared digest' :
79+
'outside both count and age protections',
80+
}));
81+
}
82+
83+
export async function inventoryPackages(policy, request, owner) {
84+
const inventory = {};
85+
for (const pkg of policy.packages) {
86+
const versions = [];
87+
for (let page = 1; ; page++) {
88+
const batch = await request(`/orgs/${owner}/packages/container/${encodeURIComponent(pkg)}/versions?per_page=100&page=${page}`);
89+
if (!Array.isArray(batch)) throw new Error(`Invalid inventory response: ${pkg}`);
90+
versions.push(...batch);
91+
if (batch.length < 100) break;
92+
}
93+
inventory[pkg] = versions;
94+
}
95+
return inventory;
96+
}
97+
98+
export async function deletePlanned(plan, request, owner, report) {
99+
// Preflight ALL candidates before the first DELETE.
100+
for (const row of plan.filter((entry) => entry.action === 'delete')) {
101+
const path = `/orgs/${owner}/packages/container/${encodeURIComponent(row.package)}/versions/${row.id}`;
102+
const current = await request(path);
103+
if (current.name !== row.digest ||
104+
JSON.stringify([...current.metadata.container.tags].sort()) !== JSON.stringify([...row.tags].sort())) {
105+
throw new Error(`Version changed since inventory: ${row.package}/${row.id}`);
106+
}
107+
}
108+
for (const row of plan.filter((entry) => entry.action === 'delete')) {
109+
try {
110+
await request(`/orgs/${owner}/packages/container/${encodeURIComponent(row.package)}/versions/${row.id}`, 'DELETE');
111+
await report({ package: row.package, id: row.id, status: 'deleted' });
112+
} catch (error) {
113+
await report({ package: row.package, id: row.id, status: 'failed', error: error.message });
114+
throw error;
115+
}
116+
}
117+
}
118+
119+
async function main() {
120+
const policy = JSON.parse(await readFile(new URL('../ops/release/package-retention.json', import.meta.url), 'utf8'));
121+
const owner = process.env.GITHUB_REPOSITORY_OWNER;
122+
const token = process.env.GH_TOKEN;
123+
if (!owner || !token) throw new Error('GITHUB_REPOSITORY_OWNER and GH_TOKEN are required');
124+
const request = async (path, method = 'GET') => {
125+
const response = await fetch(`https://api.github.com${path}`, {
126+
method,
127+
headers: {
128+
Authorization: `Bearer ${token}`,
129+
Accept: 'application/vnd.github+json',
130+
'X-GitHub-Api-Version': '2022-11-28',
131+
},
132+
signal: AbortSignal.timeout(30000),
133+
});
134+
if (!response.ok) throw new Error(`${method} ${path}: HTTP ${response.status}`);
135+
return response.status === 204 ? null : response.json();
136+
};
137+
const pins = (process.env.PROTECTED_VERSIONS ?? '').split(/[\s,]+/).filter(Boolean);
138+
const now = Date.now();
139+
const inventory = await inventoryPackages(policy, request, owner);
140+
const plan = planRetention(policy, inventory, now, pins);
141+
await writeFile('package-retention-plan.json', JSON.stringify({ generatedAt: new Date(now), plan }, null, 2));
142+
const deleting = process.env.DELETE_PACKAGES === 'true';
143+
const summary = [
144+
`## Package retention (${deleting ? 'DELETE ENABLED' : 'dry-run'})`,
145+
'',
146+
'| Package | Keep | Delete candidates |',
147+
'| --- | ---: | ---: |',
148+
...policy.packages.map((pkg) =>
149+
`| ${pkg} | ${plan.filter((r) => r.package === pkg && r.action === 'keep').length} | ${plan.filter((r) => r.package === pkg && r.action === 'delete').length} |`),
150+
'',
151+
'Full version IDs, tags, and reasons are in package-retention-plan.json.',
152+
'Untagged manifests and auxiliary artifacts are deliberately preserved.',
153+
'',
154+
].join('\n');
155+
console.log(summary);
156+
if (process.env.GITHUB_STEP_SUMMARY) await appendFile(process.env.GITHUB_STEP_SUMMARY, summary);
157+
if (deleting) {
158+
if (process.env.GITHUB_EVENT_NAME !== 'workflow_dispatch') throw new Error('Deletion requires a manual workflow dispatch');
159+
const refreshed = planRetention(policy, await inventoryPackages(policy, request, owner), now, pins);
160+
if (JSON.stringify(refreshed) !== JSON.stringify(plan)) throw new Error('Inventory changed; rerun dry-run before deleting');
161+
await deletePlanned(plan, request, owner, async (entry) => {
162+
console.log(JSON.stringify(entry));
163+
await appendFile('package-retention-results.jsonl', `${JSON.stringify(entry)}\n`);
164+
});
165+
}
166+
}
167+
168+
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
169+
main().catch((error) => { console.error(error.message); process.exitCode = 1; });
170+
}

0 commit comments

Comments
 (0)