You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
| Broader formal model coverage | Finite configured TLC models are closed above, including bounded prepare branch-priority/try-witness coverage, finite 3-, 5-, and 7-replica Accept-Deps optimized-recovery evidence coverage, finite 3-, 5-, and 7-replica abstract TryPreAccept response branch-slice coverage, finite 3-, 5-, and 7-replica TryPreAccept message-path coverage, finite 3-, 5-, and 7-replica committed-conflict evidence-query guard/fail-closed coverage, one finite three-voter committed-conflict evidence-staleness request-scoping slice (`tla/EPaxosEvidenceStaleness.cfg` generated `6/6` states), finite 3-, 5-, and 7-replica uncommitted-conflict force/defer quorum coverage, finite configuration-barrier/add/remove/chain pinning coverage, one finite normal configuration-transition retry-timer slice (`tla/EPaxosConfigTransitionRetry.cfg` generated `8/8` states), one finite normal configuration-transition response de-duplication slice (`tla/EPaxosConfigTransitionDedup.cfg` generated `16/16` states), one finite durable configuration replay slice, finite config recovery-after-removal, recovery-after-addition, lost/duplicate response de-duplication, and recovery retry-timer slices (`tla/EPaxosConfigRecovery.cfg` generated `44/30` states, `tla/EPaxosConfigAddRecovery.cfg` generated `15/15` states, `tla/EPaxosConfigRecoveryDedup.cfg` generated `11/11` states, and `tla/EPaxosConfigRecoveryRetry.cfg` generated `8/8` states), a finite rollback-allocation next-instance/skip/apply-order check, and a finite `TOQClockDiscipline.tla` bounded-skew/bounded-delay contract. The TOQ operational-clock boundary is now documented in `EPAXOS.MD` and `MODEL_EQ_REPORT.MD`: the core consumes embedder-provided clock, one-way-delay, and sync-group values, but does not implement synchronization, measurement, drift monitoring, or target-environment validation. Remaining open: arbitrary/general recovery under configuration changes beyond the finite recovery slices, arbitrary membership histories, arbitrary durable histories, joint consensus, arbitrary message loss and retry/rebroadcast behavior, complete optimized-recovery branch parity, unbounded proofs, external target proof, synchronized-clock implementation, one-way-delay measurement, runtime drift enforcement, and operational clock-discipline proof. |
126
126
| Deployment manifest | Example systemd artifacts now exist (`deploy/systemd/kvnode@.service`, `deploy/systemd/kvnode.env.example`) plus `tests/kvnode_systemd_manifest_audit.sh`, which renders the example EnvironmentFile into the `ExecStart` contract, emits `release_claim=none-target-environment-deployment-manifest-still-required`, and keeps `systemd-analyze verify` opt-in via `KVNODE_SYSTEMD_ANALYZE=yes`; these artifacts are checked by `tests/operations_readiness_audit.sh`. A reviewed and exercised target deployment under systemd/container/orchestration remains open before this can be a production manifest claim, so target-environment deployment execution remains open. |
127
127
| Data lifecycle | Local destructive-storage remove/restore evidence exists, the KV example has exercised Pebble checkpoint/whole-directory restore plus offline and live-source checkpoint-backed repair tests for checksum-detected bit-level corruption, `examples/kv/cmd/kvcheckpoint` provides a maintained offline checkpoint/verify/verified-restore/repair helper, `TestRestoreRejectsCorruptCheckpointWithoutReplacingLiveData` verifies restore fails closed before replacement, `KVNODE_CHECKPOINT_REPORT=/path/report.env` lets successful helper operations write `status=example-operator-report` plus `release_claim=none-target-environment-data-lifecycle-drill-still-required`, `tests/kvnode_local_runner.go --mode data` stops one local loopback node and runs offline checkpoint/verify/restore/repair on a stopped local node before restart/catch-up verification, the runner uses distinct `checkpoint-report.env`, `verify-report.env`, `restore-report.env`, and `repair-report.env` paths under `data-lifecycle/*-report.env` and validates each report's `status`, `operation`, and `release_claim` before writing `data-lifecycle-summary.txt`, those reports record `operation=checkpoint`, `operation=verify`, `operation=restore`, `operation=repair`, and `result=success`, `data-lifecycle-summary.txt` records `data_lifecycle=offline-checkpoint-verify-restore-repair`, `reports=checkpoint-report.env,verify-report.env,restore-report.env,repair-report.env`, and `none-target-environment-data-lifecycle-drill-still-required`, and `docs/operations/kvnode-data-lifecycle-incident-runbook.md` documents checkpoint, verification, repair, restore, helper reports, checksum-mismatch, local data-lifecycle drill, and evidence-capture procedures. A reviewed operator backup/restore/disaster-recovery drill in the target environment remains open; target-environment backup/restore/disaster-recovery drill remains open. |
128
-
| Capacity envelope | `tests/kvnode_capacity_envelope.sh` is an opt-in bounded harness for throughput, latency, memory RSS, disk growth, queue depth, value size, scan limit, and peer-count samples; its `metadata.env` and `summary.md` emit `release_claim=none-target-environment-capacity-results-still-required` plus bounded single-line `environment_label` and `workload_label` provenance fields; `tests/kvnode_local_capacity_drill.sh` starts a disposable three-node loopback cluster and runs that harness against all three client/admin listeners with PIDs and data dirs while preserving the same release-claim non-claim and defaulting provenance to `environment_label=local-loopback` and `workload_label=local-capacity-drill`; `tests/kvnode_local_runner.go` is a custom Go runner that starts the same local-only three-node loopback shape and records bounded write/read/scan latency plus admin metric samples. `bash -n tests/kvnode_capacity_envelope.sh`, `bash tests/kvnode_capacity_envelope.sh --help`, `bash tests/kvnode_local_capacity_drill.sh --help`, `go run -tags kvnode_local_runner ./tests/kvnode_local_runner.go --help`, and `tests/operations_readiness_audit.sh` pass. Local loopback samples have passed, including the earlier single-node workstation sample, a three-node local wrapper sample with 5 ops per value-size phase, 64/1024-byte values, scan limits 1/8, a non-claim metadata sample with 1 op, 16-byte values, scan limit 1, peer_count=3, and `release_claim=none-target-environment-capacity-results-still-required`, and a custom Go runner sample with `KVNODE_GO_RUNNER_OPS_PER_PHASE=2`, `KVNODE_GO_RUNNER_VALUE_BYTES=16`, `KVNODE_GO_RUNNER_SCAN_LIMITS=1`, and `status=local-go-runner-only`. This is workstation harness evidence only; measured target-environment capacity results remain open because target-environment capacity measurement remains open. |
128
+
| Capacity envelope | `tests/kvnode_capacity_envelope.sh` is an opt-in bounded harness for throughput, latency, memory RSS, disk growth, queue depth, value size, scan limit, and peer-count samples; its `metadata.env` and `summary.md` emit `release_claim=none-target-environment-capacity-results-still-required` plus bounded single-line `environment_label` and `workload_label` provenance fields; `tests/kvnode_local_capacity_drill.sh` starts a disposable three-node loopback cluster and runs that harness against all three client/admin listeners with PIDs and data dirs while preserving the same release-claim non-claim and defaulting provenance to `environment_label=local-loopback` and `workload_label=local-capacity-drill`; `tests/kvnode_local_runner.go` is a custom Go runner that starts the same local-only three-node loopback shape, records bounded write/read/scan latency plus admin metric samples, accepts `KVNODE_GO_RUNNER_ENVIRONMENT_LABEL` and `KVNODE_GO_RUNNER_WORKLOAD_LABEL`, defaults them to `environment_label=local-loopback` and `workload_label=local-go-runner`, validates both as non-empty single-line values without `=` and with maximum length 128, and writes the labels to `metadata.env`, `capacity-summary.txt`, and final `summary.txt` when capacity mode runs. These checks state that the custom Go runner capacity labels document `KVNODE_GO_RUNNER_ENVIRONMENT_LABEL` and `KVNODE_GO_RUNNER_WORKLOAD_LABEL`; the defaulting custom Go runner provenance to `environment_label=local-loopback` and `workload_label=local-go-runner` behavior remains explicit; custom Go runner validates label values as non-empty, single-line, without `=`, and at most 128 characters; custom Go runner writes `environment_label` and `workload_label` to `metadata.env`, `capacity-summary.txt`, and capacity `summary.txt` when `capacity_ran=true`. `bash -n tests/kvnode_capacity_envelope.sh`, `bash tests/kvnode_capacity_envelope.sh --help`, `bash tests/kvnode_local_capacity_drill.sh --help`, `go run -tags kvnode_local_runner ./tests/kvnode_local_runner.go --help`, and `tests/operations_readiness_audit.sh` pass. Local loopback samples have passed, including the earlier single-node workstation sample, a three-node local wrapper sample with 5 ops per value-size phase, 64/1024-byte values, scan limits 1/8, a non-claim metadata sample with 1 op, 16-byte values, scan limit 1, peer_count=3, `release_claim=none-target-environment-capacity-results-still-required`, `environment_label=local-loopback`, and `workload_label=local-capacity-drill`, plus a custom Go runner sample with `KVNODE_GO_RUNNER_OPS_PER_PHASE=2`, `KVNODE_GO_RUNNER_VALUE_BYTES=16`, `KVNODE_GO_RUNNER_SCAN_LIMITS=1`, `status=local-go-runner-only`, and a custom Go runner provenance sample with `environment_label=local-loopback`, `workload_label=local-go-runner-capacity`, and `latency_rows=3`. This is workstation harness evidence only; measured target-environment capacity results remain open because target-environment capacity measurement remains open. |
129
129
| Incident readiness |`docs/operations/kvnode-data-lifecycle-incident-runbook.md` now covers storage failure, network partition, peer compromise, replay/checksum suspicion, and recovery stalls, with evidence-capture steps and non-claims; `tests/kvnode_incident_tabletop_drill.sh` locally rehearses the storage-failure and network-partition test-fault branches on a disposable loopback cluster and writes `release_claim=none-target-environment-operator-review-still-required` into raw tabletop evidence; `tests/kvnode_local_runner.go` also locally exercised `/faults/storage`, `/faults/transport`, `/readyz`, `/metrics`, and post-clear canaries with `status=local-go-runner-only`; `tests/operations_readiness_audit.sh` checks those artifacts. Operator-reviewed target-environment tabletop or live drill evidence remains open, so target-environment incident-response operator review remains open. |
130
130
131
131
TryPreAccept response branch-slice note: `tla/EPaxosTryPreAcceptBranches.tla` is only a finite abstract scenario/stage model for stale restart, committed evidence ignore/fail-closed, direct/forced accept, one uncommitted deferral with duplicate suppression, and OK slow-quorum accept. Its only quorum detail is `okVotes >= SlowQuorum`, it runs for 3/5/7, and complete optimized-recovery branch parity beyond this finite slice remains open.
0 commit comments