Looking for the operator overview — architecture diagrams, quick start, images, testing? That is
README.md. This file is the design of record.
Status: approved; landing one phase per PR. This document describes how distributed Ansible execution and control-plane HA are added to this repository without changing the existing controller image or its behaviour. It remains the design of record — the checklist in §7 tracks what has actually shipped, and the phases still unticked are plan, not code.
Add a second execution path to the existing Ansible controller:
- Direct (existing):
make run— the controller executes Ansible itself. Unchanged. - Distributed (new): work is dispatched to execution nodes grouped into pools per zone. The controller becomes an orchestrator; execution happens on the nodes, not on the controller.
The mesh differentiates local targets (a directly-reachable zone) from network targets (segmented zones reachable only through a node), and provides HA at two independent layers: execution-node pools and the control plane.
Built on Receptor + Ansible Runner + mTLS only. No web UI, REST API, database, message broker, or scheduler service — see §11 Non-goals.
Documentation map (which page owns what — update the owner when behavior changes): README.md is the entry point and first-deployment guide; RUNBOOK.md owns day-2 procedures (enrollment, rotation, eviction, upgrade, recovery); this document owns the why — architecture, trust boundaries, failure semantics, tradeoffs — and the test matrix that keeps the other two honest.
These are load-bearing. The existing controller must be provably unchanged until an operator opts in.
| # | Guarantee | Enforced by |
|---|---|---|
| 1 | Root docker-compose.yml, Makefile direct targets, and the controller image stay identical |
docker/Dockerfile is untouched; mesh images build from a separate docker/mesh/Dockerfile FROM the controller and never modify it; controller proven byte/scan-identical (see §9.1) |
| 2 | make up / make down / make run behave exactly as today |
Sidecar and nodes are Compose-profile-gated (--profile mesh); default up never starts them |
| 3 | The controller build context is unchanged | mesh/ is excluded via .dockerignore |
| 4 | The release pipeline (#54–#57: cosign, semver, Trivy gate) is untouched | Node/relay images are not added to docker-publish.yml until a deliberate later phase |
| 5 | The CVE posture stays clean | Mesh images build FROM the controller, inheriting its pip-vendor patch + pebble removal automatically; every mesh image is scanned by the same gate before it can publish |
Rule: any mesh PR whose CI shows a change to the controller image scan (still 0 CRITICAL/HIGH) or a failing controller smoke test is rejected, not merged.
DIRECT (existing, unchanged)
Controller ───SSH──► Target
DISTRIBUTED (new)
Controller (orchestrator)
│ ansible-runner transmit
▼
Controller Receptor (sidecar)
│ Receptor mTLS
▼
Execution node (receptor + ansible-runner + ansible-core)
│ SSH / WinRM
▼
Target
│ job events / rc
▼ (back up the same path)
Controller ──► CLI output + artifacts
HA is provided at two layers that fail and recover independently:
- Execution-node HA — multiple nodes per zone; dispatch fails over between them.
- Control-plane HA — see §3. Tier 1 (ingress redundancy) is native to Receptor; Tier 2 (active/active orchestrators) is designed-for but built later.
Operators / CLI
│
VIP or DNS round-robin
┌──────┴──────┐
▼ ▼
┌──────────────┐ ┌──────────────┐
│ CONTROL-A │ │ CONTROL-B │ active/active, on-demand
│ orchestrator │ │ orchestrator │ (no scheduler → no leader
│ + receptor │ │ + receptor │ election needed)
└──────┬───────┘ └───────┬──────┘
│ shared state │ playbooks (git), PKI pub,
└────────┬────────┘ zones/pools, per-job meta.json
│
┌───────────┴───────────┐
│ RECEPTOR MESH │ both A & B are ingress peers;
│ (routes around a │ nodes: tcp-peers=[A,B] redial
│ dead ingress) │
└───────────┬───────────┘
┌─────────────────┼─────────────────┐
▼ ▼ ▼
pool: local pool: net20 pool: net30
(N exec nodes) (N exec nodes, HA) (N exec nodes)
│ │ ▲ │
SSH/WinRM SSH/WinRM │ SSH/WinRM
▼ ▼ failover ▼
local targets net20 targets net30 targets
CA private key: OFFLINE only — never on A or B. Not a runtime SPOF.
mesh-run ZONE=net20 PLAYBOOK=patch.yml LIMIT=web01
│
├─(1) classify: ZONE=net20 → pool = [exec-net20-a, exec-net20-b]
├─(2) receptorctl status → healthy = reachable subset of the pool
├─(3) select (round-robin) → exec-net20-a
├─(4) build UNIQUE private_data_dir jobs/<uuid>/
├─(5) write per-job jobs/<uuid>/meta.json {job-id, node, unit-id, status}
│ (one file per job — no shared-file race between concurrent runs)
├─(6) ansible-runner transmit → receptorctl work submit --node exec-net20-a
│ │
│ ├─ submit ACKed — OR any lost/ambiguous response ──► ⛔ NON-RETRYABLE
│ │ the unit may be running; on failure → REPORT, never retry
│ │ elsewhere. A lost ACK is not proof that nothing ran.
│ │
│ └─ DEFINITIVELY pre-submit only (node absent from receptorctl
│ status, or connection refused before the request was sent)
│ └─ failover → exec-net20-b ✅ safe (submit never left)
├─(7) stream events → ansible-runner process ; update meta.json status
├─(8) artifacts → logs/runner/<job-id>/ (stdout, rc, job_events)
└─(9) return Ansible rc
The failover boundary is the most important correctness rule in this design.
Failover is allowed only for submissions that definitively never left the
controller — the target node is absent from receptorctl status, or the
connection was refused before the request was sent. A submission that was ACKed —
or whose response was lost or ambiguous — is treated as possibly-running and is
never retried elsewhere; a lost ACK is not proof that nothing ran. Each job
carries a stable work-unit id, so its real state can be queried (receptorctl work list/status) instead of inferred from a failed call.
| Tier | What | Decision |
|---|---|---|
| 1 | ≥2 Receptor ingress sidecars on the control host (orchestrator fails its control socket over between them) + execution nodes multi-peer with redial | Build now. Native, cheap, real. Survives losing one sidecar/ingress; losing the orchestrator host is Tier 2. |
| 2 | Active/active orchestrators (VIP/DNS, shared config+PKI, git playbooks) | Design for it, build when a real 2-host target exists. Interim recovery = documented fast orchestrator restart (state is external, so seconds). |
| 2.5 | Recover an in-flight job’s live stream onto the surviving orchestrator | Don’t build the recovery logic. But write a per-job meta.json now (for status/history) so the logic is nearly free later. Meaningless until Tier 2 exists. |
| 3 | Full stateful HA (real state store) | Don’t. That store is a database; if ever truly required, evaluate adopting AWX instead of reimplementing it. |
In-flight failure posture (interim): if an orchestrator dies mid-job, the job keeps running on the execution node (Receptor is store-and-forward); the live stream is lost; recovery is re-query artifacts or re-run (playbooks are idempotent).
Hold these from the first mesh commit even in the single-orchestrator build:
| Invariant | Why | Cost now |
|---|---|---|
| Orchestrator holds no local-only state — all state on mountable/shared storage or git | One→two orchestrators becomes config, not a rewrite | discipline only |
Every job gets a stable UUID and a per-job jobs/<uuid>/meta.json (one file per job — concurrency-safe by construction) |
Status/history today; Tier 2.5 recovery later | small; wanted anyway |
| Execution nodes peer to a list of ingress addresses (even a list of one) | Tier 1 becomes "add controller-B to the list" | ~0 |
| CA private key never on a runtime container | CA is not a runtime SPOF; signing is offline | ~0 |
PKI material is never committed to git. The CA, certs, keys, and CSRs all live
under mesh/secrets/ on shared storage and are distributed by the signing workflow
(§6 Phase 6); public certs are shared the same way,
not tracked in the repo. "Shared config+PKI" above means shared storage, not the
repository.
Everything mesh-specific lives under mesh/. The existing docker/Dockerfile
is left unchanged — it stays the single-stage controller build, so a build with
no --target still produces today's controller. The mesh images build from a
separate docker/mesh/Dockerfile that starts FROM the finished controller, so
they inherit its exact runtime (and its CVE remediation) with zero dependency drift
and zero change to the controller:
docker/Dockerfile → controller # UNCHANGED; the PUBLISHED image and
# the default (no-target) build
docker/mesh/Dockerfile:
ARG BASE=<controller image>
FROM ${BASE} AS orchestrator # controller + ansible-runner + receptorctl
FROM orchestrator AS execution-node # + receptor (patched source build);
# entrypoint runs receptor, NOT sshd; the
# inherited port-22 HEALTHCHECK is overridden
# with a receptor-readiness check (inherited
# sshd is never started — a later slimming target)
Why not a shared ansible-runtime base stage inside docker/Dockerfile? Two
review findings ruled it out: (a) declaring execution-node as a later stage makes
it the default build target, so the existing pipeline — which sets no target: —
would publish a mesh image under the controller name; and (b) putting orchestration
packages in a shared base leaks them into the published controller, breaking the
byte-identical guarantee. Building the mesh images FROM the finished controller
sidesteps both: the controller build is untouched, and the mesh images are explicit,
never-default --target builds. Their build inputs (entrypoints) live under
docker/mesh/ — inside the build context — so the top-level mesh/ tree stays
.dockerignored and contributes nothing to any image.
ansible-controller/
├── docker/
│ ├── Dockerfile # UNCHANGED single-stage controller (default build)
│ └── mesh/ # mesh image builds — FROM the controller image
│ ├── Dockerfile # targets: orchestrator, execution-node
│ ├── orchestrator-entrypoint.sh
│ └── node-entrypoint.sh # renders receptor node config from env
├── docker-compose.yml # UNCHANGED — controller only
├── Makefile # existing targets untouched; mesh targets appended
│
└── mesh/ # ← the whole subsystem, isolated
├── README.md # operator overview (diagrams, quick start)
├── DESIGN.md # this document
├── compose.mesh.yml # overlay: sidecar + local exec-node, profile-gated
├── compose.node.yml # node-side deployment file: one node, env-driven
├── node.env.example # its settings (node id, ingress addresses, image)
├── config/
│ ├── zones.yml # inventory group / network → zone (local | net20…)
│ ├── pools.yml # zone → [node_id, node_id] (HA membership)
│ └── receptor/
│ ├── controller.yml
│ └── node.yml.template
├── pki/ # mesh-ca-init, controller-cert, node-csr, node-sign
├── bin/
│ ├── mesh-run # dispatcher (classify → select → submit → collect)
│ ├── mesh-status.sh
│ └── mesh-ping.sh
├── deploy/ # HA overlays: multi-ingress, VIP, shared-storage mounts
├── secrets/ # gitignored (keys never committed)
│ └── receptor/.gitkeep
└── tests/ # multi-network lab + mTLS/SSH negative tests
One PR per phase. Each is independently reversible. "Touches controller?" is the risk column; anything that touches it must pass §9.1.
| Ph | Goal | Touches controller image? | Guardrail before merge |
|---|---|---|---|
| 0 | SSH hardening: add managed known_hosts / strict path, label dev override; do not flip default |
Config only, opt-in | Existing direct run still works with current default |
| 1 | Add docker/mesh/Dockerfile scaffold (FROM controller); docker/Dockerfile untouched |
No — controller build unchanged | Controller smoke + Trivy = 0 unchanged, both arches (§9.1) |
| 2 | orchestrator target in docker/mesh/Dockerfile (FROM controller + ansible-runner + receptorctl) |
No — separate image, explicit --target |
Controller digest unchanged; orchestrator scan 0 |
| 3 | receptor-controller sidecar + shared socket volume, profile-gated |
No (opt-in) | make up unchanged |
| 4 | execution-node target in docker/mesh/Dockerfile (FROM controller + receptor + ansible-runner; entrypoint runs receptor; HEALTHCHECK → receptor readiness) + test lab, no TLS (dev only) |
No | Local build only; never published |
| 5 | Prove transmit → work submit → worker → process across the mesh |
No | Positive tests §9.2 |
| 6 | PKI scripts + mandatory mTLS + Tier-1 (2nd receptor sidecar, orchestrator control-socket failover, node multi-peer) | No | Negative mTLS + Tier-1 tests §9.3/§9.7 |
| 7 | Target SSH credential handling + artifacts + per-job meta.json |
No | SSH-negative test §9.4 |
| 8 | Pools + failover dispatcher (execution-node HA) + concurrency caps | No | HA + concurrency tests §9.5/§9.6 |
| 9 | Work signing (--signwork) — authorises the future third-party boundary |
No | Signed-work test; unsigned rejected |
| 10 | Makefile convenience targets + docs/runbook | Appends only | Existing targets still work |
| 11 | (optional, later) publish + cosign + scan node image | Release pipeline | Extend the matrix, don’t fork it |
| — | Deferred: Tier 2 active/active; fan-out sharding; auto-routing engine | — | Built only when a real driver exists |
- Add a managed
known_hostsmechanism +StrictHostKeyChecking=yespath (configs/ansible.cfg documents/configs/known_hosts+ strict/accept-new) - Clearly label the existing lax settings as dev-only override
- Document; do not change the current default (flip only at a major bump)
- Add
docker/mesh/DockerfilewithARG BASE;orchestratorbuildsFROM ${BASE}andexecution-nodedeliberately buildsFROM orchestrator(the Phase 4 deviation recorded below) - Leave
docker/Dockerfileunchanged; confirm the default (no-target) build is still the controller - Prove controller image smoke + Trivy = 0 on amd64 and arm64 (unchanged)
-
orchestratortarget =FROM ${BASE}+pip install ansible-runner receptorctl - Build only via explicit
--target orchestrator; the published controller is never rebuilt -
orchestratorscan 0; controller digest unchanged
-
mesh/config/receptor/controller.yml(Unix control socket, no TCP control) — receptor 1.6.7 takes a YAML list of action maps; it rejects the mapping/version: 2style withcannot unmarshal !!map into []interface {} -
mesh/compose.mesh.ymlwithreceptor-controllerunderprofiles: [mesh] - Shared named volume
receptor-runtime:/run/receptor - Verify
make upstill starts controller only - Sidecar drops to the unprivileged
receptoruid; control socket is 0600
-
execution-nodetarget +receptor+ansible-runner(Phase 5 submitsansible-runner workerhere) — builtFROMthe orchestrator stage rather than${BASE}so the pinned pip closure and its build-time assertions exist in exactly one place - receptor rebuilt from the v1.6.7 tag commit with patched Go modules (x/crypto 0.53.0, x/net 0.56.0, x/text 0.39.0) — upstream's binary carries 11 fixable HIGH CVEs and no fixed release exists yet; drop the source stage when upstream ships one
- Override the inherited port-22
HEALTHCHECKwith a Receptor-readiness check (node runs receptor, not sshd; runs as uid 1000) -
docker/mesh/node-entrypoint.sh(render node config from env; peers is a LIST — Tier 1 ready) -
mesh/tests/multi-network e2e integration environment (controller cannot reach targets) — regression suite (e2e-check.sh; 20 properties as of Phase 6), run by Mesh CI on every mesh change - No-TLS peering confined to the e2e environment (never in a production compose file)
-
mesh/bin/mesh-run(minimal: transmit → submit → worker → process) — per-job UUID +jobs/<uuid>/meta.jsonlifecycle from the first commit (day-one invariants, §4); success requires the artifacts' ownrcfile, because a broken results stream leavesansible-runner processexiting 0 - Node-side
mesh-workerwrapper keeps the results stream protocol-pure — receptor merges the work command's stderr into the unit stdout, and one non-JSON line (OpenSSL greets stderr at every python start) makes the controller-side Processor abort at its first read - Positive test: controller can’t SSH target directly;
mesh-runsucceeds (e2e check 9) - "Prove-on-node" playbook shows node id, not controller (e2e check 10, both directions)
- Real ansible rc + artifacts (stdout, rc, job_events) return to the controller (checks 11–13, including nonzero rc propagation)
-
mesh/pki/:mesh-ca-init.sh,controller-cert.sh,node-csr.sh,node-sign.sh— receptor's own cert tooling (nodeid OID in SAN) run in the pinned image; signing is gated on the CSR carrying EXACTLY the authorised node id - Idempotent CA init (STOP if CA exists unless
--force); CA key never in any runtime mount — onlyissued/bundles reach containers - Node-id ⇄ cert identity checking on;
insecureskipverify/skipreceptornameschecknever set (e2e asserts their absence as active keys) - Multi-ingress peer list (Tier 1): nodes peer to A and B; the dispatcher fails its control socket over between the two sidecars
- All negative mTLS tests pass — the full §9.3 matrix (e2e 15–19): certless node, unknown client CA, valid-cert-wrong-identity, EXPIRED cert, and (reversed) a node refusing a controller whose cert comes from an unknown CA + Tier-1 failover proven (e2e 20: sidecar A stopped, dispatch through B succeeds, A re-peers)
- mTLS is MANDATORY: the node entrypoint refuses to start without cert,
key, and CA.
RECEPTOR_INSECURE_DEV=1is a loud, dev-only escape: no production file sets it, and the e2e suite uses it in exactly one place — as the ATTACKER in negative check 15, proving a plaintext node is rejected by the mesh
-
env/ssh_keyin the transmit payload; never logged/echoed - Node runtime Python deps (pip-requirements.txt equivalents, e.g. boto3
for cloud inventory plugins): document the site-image extension pattern
(
FROM execution-node+pip install -r) — per-job staging of compiled packages is the wrong layer (README "Node runtime Python dependencies") - Artifacts to
logs/runner/<job-id>/(stdout, rc, job_events) — e2e #22 - Write a per-job
jobs/<uuid>/meta.json(one file per job — concurrency-safe, RFC 3339updatedon every transition) - Secure cleanup of transient key copies — controller copy shredded via EXIT trap (e2e #21); node copy dies with the released work unit
-
mesh/config/zones.yml+pools.yml(templates; mounted at/etc/mesh/) - Dispatcher: classify → healthy-node select → submit → dispatch-only failover — e2e #23
- Per-node concurrency cap via an atomic
flockreservation on a per-node slot file (not awork listcheck) — e2e #24 - Unique PDD + artifact dir + work-unit id per concurrent job (held since Phase 5; re-proven under contention by #24)
- Controller work-signing private key (offline-issued via
mesh/pki/work-sign-init.sh); nodes get only the verification pubkey (RECEPTOR_WORK_PUBKEY) -
verifysignature: trueon the node worktype (mandatory outsideRECEPTOR_INSECURE_DEV);mesh-runsubmits--signworkby default - Test: unsigned work rejected — e2e #25
-
mesh-up,mesh-down,mesh-status,mesh-ping,mesh-run— the dispatcher is baked into the orchestrator image; pools/zones configs mount live-editable at/etc/mesh/ - Runbook: create/enroll/sign/install/verify/run/rotate/revoke/troubleshoot
—
mesh/RUNBOOK.md - Confirm every pre-existing Make target still works (unchanged; verified
by
make -nacross the full target list) -
mesh/compose.node.yml+node.env.example— the packaged node-side deployment (the RUNBOOK'sdocker runas a file); e2e #26 starts a node from it, unmodified, and dispatches work to it
| UC | Scenario | Expected outcome |
|---|---|---|
| UC1 | Local scan via colocated local execution node | Runs on exec-local-*, not on the controller |
| UC2 | Network scan into segmented net20 |
Runs on exec-net20-*; controller has no route to the target |
| UC3 | Execution-node failover — exec-net20-a down at dispatch |
exec-net20-b serves; job succeeds |
| UC4 | Concurrency — many jobs across zones at once | Isolated PDDs/artifacts; one meta.json per UUID (no shared-file race); per-node cap enforced by an atomic flock reservation (not a check-then-act race) |
| UC5 | Control-plane ingress failover (Tier 1) — one receptor sidecar down | Orchestrator fails its control socket over to the 2nd sidecar; nodes reconnect via it; new dispatch works. (Losing the orchestrator host is Tier 2.) |
| UC6 | Third-party node (future) — external team runs exec-partner-* |
mTLS admits it (your CA signed it); work-signing means it executes but cannot submit |
| UC7 | Direct legacy mode — make run PLAYBOOK=ping.yml |
Unchanged; does not traverse Receptor |
| UC8 | Unauthorised node — cert from unknown CA / wrong node id | Rejected by the controller |
| UC9 | Target SSH failure — bad/removed job SSH key | Receptor + Runner reach the node; SSH to target fails; job returns FAILED |
| UC10 | Certificate rotation — a node’s cert expires/rotates | Re-CSR → sign → install → reconnect, no CA-key exposure |
The controller must be provably untouched:
# root controller files unchanged by a mesh PR
git diff --stat origin/main -- docker/ docker-compose.yml Makefile configs/ | \
grep -q . && echo "controller touched — REVIEW" || echo "controller untouched"
# root compose starts the controller ONLY (the sidecar lives in the overlay, not root)
docker compose config --services # expect: ansible (no receptor-controller)
# the sidecar appears only when the overlay is loaded AND the profile is enabled:
docker compose -f docker-compose.yml -f mesh/compose.mesh.yml --profile mesh \
config --services # expect: ansible + receptor-controller
# controller image identical posture (existing CI already asserts these)
# - build + smoke test green (ansible --version, sshd running)
# - Trivy CRITICAL/HIGH == 0 on amd64 AND arm64 (#55 gate)1. Prove the controller CANNOT reach the target directly (no route / SSH fails).
2. make mesh-run ZONE=net20 PLAYBOOK=ping.yml → SUCCESS on the controller CLI.
3. "Prove-on-node" playbook returns the execution node's hostname/id, not the
controller's (§25 of the source spec).
4. Correct Ansible exit code and artifacts returned to the controller.
Each must be rejected (encryption alone is not acceptance — both sides authenticate):
- node with no certificate
- node cert signed by an unknown CA
- expired node certificate
- controller cert signed by an unknown CA → node rejects controller
- node id ≠ certificate Receptor identity
- assert insecureskipverify / skipreceptornamescheck are NOT set in prod config
Remove/replace the job SSH credential, then mesh-run:
Receptor connects ✓ Runner reaches node ✓ SSH to target FAILS ✓ job = FAILED
Proves Receptor identity and target SSH identity are separate.
- Pool net20 = [exec-net20-a, exec-net20-b]; stop exec-net20-a.
- mesh-run ZONE=net20 → dispatcher selects exec-net20-b → SUCCESS.
- Assert: a job that had already STARTED on a node is NOT retried elsewhere
(failover-boundary rule).
- Launch N concurrent mesh-run jobs across zones.
- Assert unique PDDs, unique artifact dirs, no work-unit id collision.
- Assert all N per-job meta.json files survive (one file per UUID; no shared-file
corruption or lost entries).
- Per-node cap is enforced by an ATOMIC reservation: select+submit hold a `flock`
on a per-node slot file (on shared storage), so the cap is not a check-then-act
race on `receptorctl work list`.
- Assert the cap holds under a burst where every dispatcher sees a "free" slot at
once — the surplus queues on the lock; they do not all submit to the same node.
- Control host runs TWO receptor sidecars (A, B); execution nodes peer to both.
- Orchestrator control socket targets A, with B configured as failover.
- Stop sidecar A → orchestrator fails over to B's control socket; nodes reconnect
to B; mesh-status healthy; a FRESH dispatch succeeds through B.
- Out of Tier-1 scope: losing the orchestrator host itself → that is Tier 2.
make runstill executes directly (regression proven).make mesh-run ZONE=net20 PLAYBOOK=ping.ymlruns on the node and returns rc + output.- Controller has no direct network path to the target (proven).
- mTLS mutual, node-id checking on,
insecureskipverifynever set. - No CA private key inside any runtime container.
- Execution-node HA: pool failover works (dispatch-only).
- Control-plane HA Tier 1: 2nd receptor sidecar + orchestrator control-socket failover works.
- Per-job
meta.jsonwritten per UUID — concurrency-safe (Tier 2.5 enablement). - Controller image scan == 0 CRITICAL/HIGH, both arches; existing pipeline untouched.
- No UI, API, database, broker, or scheduler introduced.
- Runbook documents: create / enroll / sign / install / start / verify / run / rotate / revoke / troubleshoot (mTLS, Runner, target SSH).
- Tier 2 active/active orchestrators — designed-for (stateless invariant), built when a real 2-host target exists.
- Tier 2.5 in-flight recovery logic — enabled by the per-job
meta.jsonfiles; logic built with Tier 2. - Tier 3 stateful HA — not built; if ever required, evaluate AWX.
- Fan-out sharding (one job split across nodes) — after single-job-to-pool is solid.
- Auto-routing engine (subnet → zone) — static
zones.ymlonly, for now. - Publishing the node image — until the flow is proven (Phase 11).
- No web UI, REST API, database, message broker, Kafka/RabbitMQ/ActiveMQ, Airflow, scheduler service, custom worker/enrollment/auth server.
- Local execution: ship a colocated local execution node (so even local
scans avoid the controller), or is legacy
make runthe local path and nodes are network-only initially? - Deployment target for Tier 2: two VMs, or an orchestrator (k8s)? Determines when/how Tier 2 becomes real.
- Command naming — RESOLVED. Distributed jobs use
make mesh-run ZONE=<zone> PLAYBOOK=<pb>, joining the existingmesh-*namespace (mesh-up,mesh-down,mesh-status,mesh-ping).make runstays as the direct/legacy path, unchanged.ZONEunifies local (ZONE=local) and network (ZONE=net20), so no separate verbs are needed; when auto-routing lands,ZONEbecomes optional and is classified fromzones.yml— no new verb. Rejected:control-run(the control plane does not execute) andrelay-run(mislabels an execution node as a pass-through). The source spec’srelay-run/RELAY=may be kept as back-compat aliases if desired.