Skip to content

Commit e295eb2

Browse files
committed
Spec alignment
1 parent 0db2d64 commit e295eb2

12 files changed

Lines changed: 817 additions & 51 deletions

File tree

‎PRODUCTION_REFACTOR_PROMPT.md‎

Lines changed: 287 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,287 @@
1+
# V1 Refactor Prompt: Package Harness Worker
2+
3+
You are working in `DAppNodePackage-package-harness`. Convert the current inbound job service into a small, reliable polling worker for tropibot.
4+
5+
This is intentionally a simple v1 for a trusted system with only a few workers. Do not introduce PostgreSQL, distributed consensus, fencing tokens, mTLS enrollment, worker capability scheduling, object storage, or a general workflow engine. Preserve clean interfaces so those can be added later.
6+
7+
The matching tropibot work is happening independently and in parallel. The HTTP protocol below is normative. Do not change endpoint paths, field names, casing, or response semantics without documenting the blocker.
8+
9+
Do not commit or push. Preserve all existing staged and unstaged user work.
10+
11+
## Responsibility boundary
12+
13+
The harness owns:
14+
15+
- Dappmanager access and destructive safety checks.
16+
- One-at-a-time baseline/candidate execution.
17+
- Stabilization, evidence capture, deterministic verdict, and cleanup.
18+
- Heuristic and Nexus analysis.
19+
- Redaction and size/time bounds.
20+
- Local restart recovery.
21+
- Polling tropibot and delivering results.
22+
23+
The harness does not own:
24+
25+
- GitHub credentials, comments, Checks, or Markdown rendering.
26+
- Global job persistence or scheduling.
27+
- PR-head freshness or result-publication policy.
28+
29+
Keep Nexus in the harness. Preserve the current bounded, redacted, no-tools request, heuristic fallback, and rule that Nexus cannot override deterministic failures.
30+
31+
## Simple v1 topology
32+
33+
```text
34+
Harness
35+
-> poll tropibot
36+
-> persist claimed job locally
37+
-> execute one job
38+
-> heartbeat phase/cancellation state
39+
-> clean up
40+
-> persist exact completion payload
41+
-> retry completion until acknowledged
42+
```
43+
44+
The worker needs outbound HTTPS access to tropibot. It should no longer require a publicly reachable mutation endpoint. Keep local health and readiness endpoints for supervision.
45+
46+
## Normative worker API v1
47+
48+
All JSON uses camelCase. Authenticate every request with one trusted shared secret:
49+
50+
```text
51+
Authorization: Bearer <PACKAGE_HARNESS_WORKER_TOKEN>
52+
Content-Type: application/json
53+
User-Agent: dappnode-package-harness/<version>
54+
```
55+
56+
Required configuration:
57+
58+
```text
59+
TROPIBOT_URL=https://tropibot.example
60+
PACKAGE_HARNESS_WORKER_ID=worker-01
61+
PACKAGE_HARNESS_WORKER_TOKEN=<shared trusted secret>
62+
PACKAGE_HARNESS_POLL_SECONDS=10
63+
PACKAGE_HARNESS_HEARTBEAT_SECONDS=20
64+
```
65+
66+
Validate values at startup and never log the token.
67+
68+
### Claim
69+
70+
`POST /v1/package-harness/jobs/claim`
71+
72+
```json
73+
{
74+
"schemaVersion": 1,
75+
"workerId": "worker-01"
76+
}
77+
```
78+
79+
Responses:
80+
81+
- `204 No Content`: no work; wait the configured poll interval.
82+
- `200 OK`: one job was atomically assigned to this worker.
83+
- `401/403`: stop polling and report not-ready until configuration changes.
84+
- `409`: this worker already has an unresolved job; reconcile local state.
85+
- `429/5xx/network error`: retry with a small capped exponential backoff and jitter.
86+
87+
Claim response:
88+
89+
```json
90+
{
91+
"schemaVersion": 1,
92+
"jobId": "gh-pr-42-0123456789ab-abcdef1234567890",
93+
"claimToken": "opaque-random-value",
94+
"source": {
95+
"repository": "dappnode/DAppNodePackage-example",
96+
"pullRequest": 42,
97+
"headSha": "0123456789abcdef0123456789abcdef01234567"
98+
},
99+
"package": {
100+
"dnpName": "example.dnp.dappnode.eth",
101+
"candidateRef": "/ipfs/QmCandidate",
102+
"baselineRef": null
103+
}
104+
}
105+
```
106+
107+
Persist the full claim before executing anything. Validate schema, job ID, package identity, references, and self/core-package restrictions before mutation.
108+
109+
### Heartbeat
110+
111+
`POST /v1/package-harness/jobs/{jobId}/heartbeat`
112+
113+
```json
114+
{
115+
"schemaVersion": 1,
116+
"workerId": "worker-01",
117+
"claimToken": "opaque-random-value",
118+
"phase": "candidate_stabilization",
119+
"cleanupRequired": true
120+
}
121+
```
122+
123+
Response:
124+
125+
```json
126+
{
127+
"schemaVersion": 1,
128+
"cancelRequested": false
129+
}
130+
```
131+
132+
Requirements:
133+
134+
- Heartbeat during long stabilization, analysis, and cleanup phases.
135+
- Cancellation is checked at safe phase boundaries and inside bounded polling loops.
136+
- Cancellation never skips required cleanup.
137+
- A transient heartbeat failure does not erase the local job or permit claiming another.
138+
- `404/409` means tropibot no longer recognizes the claim. Do not start another mutating phase; finish the current operation, reconcile and clean up, then retain the local record for operator inspection.
139+
140+
There is deliberately no automatic lease expiry or reassignment in v1. Tropibot may mark a silent worker as `worker_lost`, but a human or a later explicit retry decides what happens next.
141+
142+
### Complete
143+
144+
`POST /v1/package-harness/jobs/{jobId}/complete`
145+
146+
Normal result:
147+
148+
```json
149+
{
150+
"schemaVersion": 1,
151+
"workerId": "worker-01",
152+
"claimToken": "opaque-random-value",
153+
"outcome": {
154+
"type": "result",
155+
"result": {}
156+
}
157+
}
158+
```
159+
160+
`outcome.result` is the existing harness result schema v1 and its `runId` equals `jobId`.
161+
162+
If the worker cannot produce a normal result, send:
163+
164+
```json
165+
{
166+
"schemaVersion": 1,
167+
"workerId": "worker-01",
168+
"claimToken": "opaque-random-value",
169+
"outcome": {
170+
"type": "worker_error",
171+
"code": "interrupted",
172+
"summary": "bounded explanation",
173+
"cleanupStatus": "passed"
174+
}
175+
}
176+
```
177+
178+
Initial worker error codes are `interrupted`, `unsupported_job`, `cleanup_failed`, `local_persistence_failed`, and `unexpected_error`.
179+
180+
Completion responses:
181+
182+
- First valid payload: `200 { "schemaVersion": 1, "disposition": "recorded" }`.
183+
- Exact duplicate: `200 { "schemaVersion": 1, "disposition": "duplicate" }`.
184+
- Conflicting payload or invalid claim: `409`.
185+
186+
Persist the exact serialized completion body before sending it. Retry the identical body after transient failures and restarts. Do not claim another job until the completion is acknowledged. If cleanup failed, remain not-ready and retain the record for operator action.
187+
188+
## Required implementation
189+
190+
### 1. Replace inbound submission with polling
191+
192+
- Remove direct GitHub reporting, GitHub App credentials, JWT/token exchange, and Markdown rendering.
193+
- Remove `github_pr_comment` configuration and dependencies.
194+
- Remove the production use of `POST /v1/runs`; retain only read-only local diagnostics if useful.
195+
- Add a typed `TropibotClient` or `CoordinatorClient` implementing claim, heartbeat, and complete.
196+
- Keep protocol DTOs isolated in one module and add golden JSON fixtures.
197+
- Bound request timeouts, response sizes, retries, and error previews.
198+
199+
### 2. Reuse simple local persistence
200+
201+
Keep the existing atomic file-based store rather than introducing a database. Extend the persisted record or add a small worker envelope containing:
202+
203+
- Full claimed job and claim token.
204+
- Current phase and whether cleanup is required.
205+
- Final result or worker error.
206+
- Exact pending completion body.
207+
- Completion acknowledgement state.
208+
- A not-ready/manual-recovery reason when cleanup or claim reconciliation is unresolved.
209+
210+
Only one local active job is allowed. Persist before phase transitions that affect recovery.
211+
212+
Keep persistence behind the existing `RunStore` or a similarly narrow trait so SQLite or another implementation can be added later without changing the controller.
213+
214+
### 3. Integrate heartbeat and cancellation cleanly
215+
216+
- Keep the current `RunController`, `PackageManager`, analyzer, and comparison design where possible.
217+
- Add a narrow progress/cancellation port rather than putting HTTP calls inside execution logic.
218+
- Publish current phase to the heartbeat task.
219+
- Check cancellation between phases and during stabilization polling.
220+
- Once mutation may have happened, all cancellation/error paths must still clean up.
221+
- Do not allow heartbeat work to block cleanup.
222+
223+
### 4. Recover safely after restart
224+
225+
On startup:
226+
227+
1. If a completion body is pending, resend it first.
228+
2. If an interrupted job may have mutated the node, inspect Dappmanager state.
229+
3. Perform bounded cleanup and confirm target absence.
230+
4. Send a `worker_error` completion describing the interruption.
231+
5. Poll for new work only after completion is acknowledged and cleanup is confirmed.
232+
233+
Never automatically rerun an interrupted job.
234+
235+
### 5. Preserve Nexus analysis
236+
237+
- Keep heuristic-only operation when no Nexus key is configured.
238+
- Keep one bounded, redacted Nexus request with no tools.
239+
- Never persist or send the Nexus credential.
240+
- Keep Nexus failure advisory and non-blocking for cleanup/result delivery.
241+
- Keep tests proving malformed or unavailable Nexus falls back safely.
242+
243+
### 6. Update packaging and documentation
244+
245+
Update `.env.example`, Compose, setup wizard, package metadata, README, architecture docs, and result docs:
246+
247+
- Replace inbound API/callback/GitHub settings with tropibot URL, worker ID, shared token, poll interval, and heartbeat interval.
248+
- Explain that the token is appropriate for this small trusted v1 and can later become per-worker auth.
249+
- Explain restart and manual-recovery behavior.
250+
- Document that the Dappnode remains dedicated and disposable; cleanup is not a security boundary.
251+
- Fix existing documentation inconsistencies such as the pinned Rust version.
252+
253+
## Deliberately deferred
254+
255+
Do not implement these in v1, but keep interfaces open for them:
256+
257+
- Per-worker credentials or mTLS.
258+
- Leases, fencing tokens, and automatic failover.
259+
- Multiple concurrent jobs per worker.
260+
- Test-profile/capability scheduling.
261+
- Ephemeral VM provisioning or snapshot automation.
262+
- External evidence storage.
263+
264+
## Acceptance criteria
265+
266+
- Harness has no GitHub credentials or GitHub API code.
267+
- Harness obtains work only by polling tropibot.
268+
- Exactly one local job can execute.
269+
- Claims and completion bodies survive restart.
270+
- Completion delivery is idempotent and retried.
271+
- Worker loss never causes automatic duplicate destructive execution.
272+
- Cancellation still performs cleanup.
273+
- Nexus remains in the harness with its current safety properties.
274+
- Existing execution/verdict/cleanup tests remain meaningful.
275+
276+
Add tests for claim/no-job/auth failure, protocol fixtures, heartbeat cancellation, restart recovery, duplicate completion, conflicting completion, cleanup failure, and Nexus fallback.
277+
278+
Run:
279+
280+
```bash
281+
cargo fmt --check
282+
cargo clippy --all-targets --all-features -- -D warnings
283+
cargo test --all-features
284+
cargo build --release
285+
```
286+
287+
At completion, report changed files, tests run, remaining external work, and any protocol deviation. Do not commit or push.

‎README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -77,11 +77,11 @@ POST /v1/package-harness/jobs/{jobId}/complete
7777

7878
Every request carries `Authorization: Bearer <PACKAGE_HARNESS_WORKER_TOKEN>`, `Content-Type: application/json`, and `User-Agent: dappnode-package-harness/<version>`. Protocol DTOs and their golden fixtures live under [`src/coordinator`](src/coordinator) and [`tests/fixtures`](tests/fixtures).
7979

80-
Before mutating, the worker atomically persists the full claim and opaque claim token under `DATA_DIR`. It persists every recovery-relevant phase and, before network delivery, the exact serialized completion JSON. Transient coordinator failures retry with capped backoff and worker-specific jitter; the same bytes are retried after a restart. The worker never claims another job until completion is acknowledged as `recorded` or `duplicate`.
80+
Before mutating, the worker atomically persists the full claim and opaque claim token under `DATA_DIR`. It persists every recovery-relevant phase and, before network delivery, the exact serialized completion JSON. Transient failures and coordinator compatibility rejections retry the same bytes with capped backoff and worker-specific jitter, so a coordinator upgrade heals delivery without a worker restart. Successful delivery clears legacy delivery-recovery markers. The worker never claims another job until completion is acknowledged as `recorded` or `duplicate`.
8181

82-
On restart, a pending completion is retried first. An interrupted job is never rerun: if it may have mutated the target, the worker inspects Dappmanager, performs bounded cleanup, then sends an `interrupted` worker-error completion. Cleanup failure, a lost claim, or a conflicting completion leaves the local record in manual-recovery state and makes `/readyz` fail until an operator resolves it.
82+
On restart, a pending completion is retried first. An interrupted job is never rerun: if it may have mutated the target, the worker inspects Dappmanager, performs bounded cleanup, then sends an `interrupted` worker-error completion. A lost claim is released locally after cleanup succeeds; if Tropibot still holds unresolved coordinator state, the next claim receives a conflict and polling stops without inventing new local recovery work. Manual recovery is reserved for failed cleanup, ambiguous local records, or a conflicting completion that proves Tropibot already holds different state for the claim.
8383

84-
Heartbeats run independently of package execution. A cancellation request is observed at safe phase boundaries and inside stabilization polling. It prevents a new mutating phase but never bypasses required cleanup. If Tropibot no longer recognizes the claim, the worker finishes the current operation, reconciles cleanup, and stops for operator inspection.
84+
Heartbeats run independently of package execution. A cancellation request is observed at safe phase boundaries and inside stabilization polling. It prevents a new mutating phase but never bypasses required cleanup. If Tropibot no longer recognizes the claim, the worker finishes the current operation, reconciles cleanup, and releases the obsolete local claim when the target is safe.
8585

8686
## Dappmanager MCP
8787

‎docs/architecture.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,9 +16,9 @@ Tropibot <--------------------------------------------> Package harness
1616

1717
`PackageHarnessWorker` persists a full claim before execution, then starts a heartbeat task beside the controller. The controller depends only on the narrow `RunProgress` port: it publishes its already-persisted phase and reads cancellation/claim-loss signals. No coordinator HTTP call runs in the controller or can block cleanup.
1818

19-
`RunRecord.worker` holds the opaque claim token, mutation/cleanup flag, final worker error when no normal result exists, exact pending completion JSON, acknowledgement status, and a manual-recovery reason. The atomic file store writes a temporary file in `/data` and persists it atomically. Only one unacknowledged claimed record is allowed to proceed.
19+
`RunRecord.worker` holds the opaque claim token, mutation/cleanup flag, final worker error when no normal result exists, exact pending completion JSON, acknowledgement status, and a manual-recovery reason. The atomic file store writes a temporary file in `/data` and persists it atomically. Only one unacknowledged claimed record is allowed to proceed. Pending completion bytes take precedence over legacy delivery-related recovery markers during restart. Compatibility rejections retry automatically with capped backoff, and an acknowledgement clears any stale marker.
2020

21-
On startup the worker finds an unresolved record before it polls. It retries a pending completion first. If an active job might have changed Dappmanager, it checks the target, refuses the harness and core packages, and executes the persisted recovery plan: restore the exact pre-test or intentionally retained baseline, or remove a target that was absent before the run. It verifies the final inventory, then sends an `interrupted` worker error. It never reruns the interrupted test. A failed reconciliation, lost claim, or conflicting completion stays visible as a not-ready/manual-recovery condition.
21+
On startup the worker finds an unresolved record before it polls. It retries a pending completion first. If an active job might have changed Dappmanager, it checks the target, refuses the harness and core packages, and executes the persisted recovery plan: restore the exact pre-test or intentionally retained baseline, or remove a target that was absent before the run. It verifies the final inventory, then sends an `interrupted` worker error. It never reruns the interrupted test. Successfully reconciled claim loss releases the obsolete local claim and lets Tropibot decide whether another claim is allowed. Manual recovery remains for failed target reconciliation, ambiguous multiple-record state, and conflicting completions that prove coordinator state differs; ordinary delivery compatibility failures self-heal when Tropibot becomes compatible.
2222

2323
The controller’s baseline and candidate paths share stabilization and capture. An existing target is reused when no explicit baseline was requested; an explicit `baselineRef` is always applied, while the original installed version is retained as the restoration target. Stabilization requires a non-empty, all-running container list with the same sorted names across consecutive samples. Failed detail calls, empty lists, non-running states, and container-set changes reset the streak. Evidence history, log collection, redaction, and analysis input are bounded.
2424

‎src/analysis/nexus.rs‎

Lines changed: 1 addition & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -307,10 +307,5 @@ fn normalize_result(result: &mut LogAnalysisResult) {
307307
}
308308

309309
fn bound_text(input: &str, maximum_bytes: usize) -> String {
310-
const TRUNCATION_SUFFIX_BYTES: usize = "…[truncated]".len();
311-
if input.len() <= maximum_bytes {
312-
input.to_owned()
313-
} else {
314-
truncate_utf8(input, maximum_bytes.saturating_sub(TRUNCATION_SUFFIX_BYTES))
315-
}
310+
truncate_utf8(input, maximum_bytes)
316311
}

‎src/analysis/redaction.rs‎

Lines changed: 34 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ use regex::Regex;
22
use sha2::{Digest, Sha256};
33

44
const REDACTED: &str = "[REDACTED]";
5+
const TRUNCATION_SUFFIX: &str = "…[truncated]";
56

67
pub fn redact_and_bound(input: &str, maximum_bytes: usize) -> String {
78
let mut output = input.to_owned();
@@ -45,11 +46,16 @@ pub fn truncate_utf8(input: &str, maximum_bytes: usize) -> String {
4546
if input.len() <= maximum_bytes {
4647
return input.to_owned();
4748
}
48-
let mut boundary = maximum_bytes;
49+
let suffix = if TRUNCATION_SUFFIX.len() <= maximum_bytes {
50+
TRUNCATION_SUFFIX
51+
} else {
52+
""
53+
};
54+
let mut boundary = maximum_bytes.saturating_sub(suffix.len());
4955
while boundary > 0 && !input.is_char_boundary(boundary) {
5056
boundary -= 1;
5157
}
52-
format!("{}…[truncated]", &input[..boundary])
58+
format!("{}{suffix}", &input[..boundary])
5359
}
5460

5561
fn redact_long_tokens(input: &str) -> String {
@@ -77,7 +83,7 @@ fn redact_long_tokens(input: &str) -> String {
7783

7884
#[cfg(test)]
7985
mod tests {
80-
use super::redact_and_bound_single_line;
86+
use super::{TRUNCATION_SUFFIX, redact_and_bound_single_line, truncate_utf8};
8187

8288
#[test]
8389
fn log_values_are_redacted_and_kept_on_one_line() {
@@ -88,4 +94,29 @@ mod tests {
8894

8995
assert_eq!(value, "502 Bad Gateway Authorization: [REDACTED] try again");
9096
}
97+
98+
#[test]
99+
fn truncation_never_exceeds_its_byte_limit() {
100+
for maximum_bytes in 0..64 {
101+
for input in [
102+
"a".repeat(128),
103+
"é".repeat(128),
104+
"🦀".repeat(128),
105+
format!("prefix-{}", "🦀éa".repeat(64)),
106+
] {
107+
let bounded = truncate_utf8(&input, maximum_bytes);
108+
assert!(bounded.len() <= maximum_bytes);
109+
assert!(input.starts_with(bounded.trim_end_matches(TRUNCATION_SUFFIX)));
110+
assert!(bounded.is_char_boundary(bounded.len()));
111+
}
112+
}
113+
}
114+
115+
#[test]
116+
fn truncation_reserves_space_for_the_suffix() {
117+
let bounded = truncate_utf8(&"x".repeat(1_000), 500);
118+
119+
assert_eq!(bounded.len(), 500);
120+
assert!(bounded.ends_with(TRUNCATION_SUFFIX));
121+
}
91122
}

0 commit comments

Comments
 (0)