Skip to content

Commit 8a8d6b1

Browse files
fix(tmachine): make K3s HA staging rerunnable and wait for PostgreSQL
Apply the OpenShell namespace instead of creating it so the staging play tolerates an existing namespace, and wait for the PostgreSQL rollout so gateway Pods do not start crash-looping against a missing database. Reflow and pluralize the K3s lane notes in CI.md. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
1 parent 31a92f1 commit 8a8d6b1

2 files changed

Lines changed: 26 additions & 8 deletions

File tree

‎CI.md‎

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -144,19 +144,18 @@ push-triggered workflow jobs themselves.
144144

145145
### K3s conformance version baseline
146146

147-
The tmachine `ubuntu-k3s` conformance lane pins Agent Sandbox v0.5.0 as the
148-
compatibility baseline for the v1beta1 Sandbox API. It does not track the local
147+
The tmachine `ubuntu-k3s` conformance lanes pin Agent Sandbox v0.5.0 as the
148+
compatibility baseline for the v1beta1 Sandbox API. They do not track the local
149149
K3s development default, currently v1.0.3. OpenShell also supports v0.4.6 through
150150
its v1alpha1 fallback, so v0.5.0 is not the overall minimum supported version.
151151

152152
The `ubuntu-k3s` lanes register the gateway's ClusterIP Service directly, so
153153
recreating the Service requires reprovisioning the fixture. The `k3s` lane runs
154154
one plaintext gateway. The `k3s-ha` lane runs the supported HA configuration:
155155
three gateway replicas sharing PostgreSQL, serving TLS, with the CLI connecting
156-
over mTLS. Before conformance,
157-
every installer waits up to five minutes for a connected gateway; interactive
158-
shells skip the wait. K3s failures upload diagnostics as `tmachine-diagnostics-*`
159-
artifacts.
156+
over mTLS. Before conformance, every installer waits up to five minutes for a
157+
connected gateway; interactive shells skip the wait. K3s failures upload
158+
diagnostics as `tmachine-diagnostics-*` artifacts.
160159

161160
### Run only the policy advisor conformance tests
162161

‎tests/ansible/playbooks/openshell-k3s-ha.yaml‎

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,14 @@
1111
ansible.builtin.wait_for_connection:
1212

1313
# The install layer boots fresh, so the K3s API may still be starting.
14-
- name: Create OpenShell namespace
14+
- name: Apply OpenShell namespace
1515
ansible.builtin.command:
16-
argv: [/usr/local/bin/k3s, kubectl, create, namespace, openshell]
16+
argv: [/usr/local/bin/k3s, kubectl, apply, --filename, "-"]
17+
stdin: |
18+
apiVersion: v1
19+
kind: Namespace
20+
metadata:
21+
name: openshell
1722
register: k3s_namespace
1823
until: k3s_namespace.rc == 0
1924
retries: 30
@@ -24,6 +29,20 @@
2429
argv: [/usr/local/bin/k3s, kubectl, --namespace, openshell, apply, --filename, "-"]
2530
stdin: "{{ lookup('ansible.builtin.file', openshell_postgres_fixture) }}"
2631

32+
# Gateway Pods that start before the database crash-loop with growing backoff.
33+
- name: Wait for PostgreSQL
34+
ansible.builtin.command:
35+
argv:
36+
- /usr/local/bin/k3s
37+
- kubectl
38+
- --namespace
39+
- openshell
40+
- rollout
41+
- status
42+
- deployment/openshell-e2e-postgres
43+
- --timeout=300s
44+
changed_when: false
45+
2746
- import_playbook: openshell-k3s.yaml
2847
vars:
2948
openshell_k3s_helm_values_overlay: |

0 commit comments

Comments
 (0)