Skip to content

Commit bffb27a

Browse files
committed
Add TryConflict force decision model
1 parent 15199d2 commit bffb27a

11 files changed

Lines changed: 199 additions & 1 deletion

EPAXOS_IMPLEMENTATION_PROOF.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -458,6 +458,7 @@ Evidence: `examples/kv/cmd/kvnode/main.go`, `examples/kv/cmd/kvnode/main_test.go
458458
| `tla/EPaxosOptimizedRecovery.tla` | Focused 3/5/7 Accept-Deps stale-dependency optimized-recovery evidence slices. | The concrete `MsgEvidence` exchange, every technical-report branch, and unbounded proof. |
459459
| `tla/EPaxosTryPreAcceptBranches.tla` | Focused 3/5/7 finite abstract TryPreAccept scenario/stage response-branch slice: stale restart, committed evidence ignore/fail-closed, direct/forced accept, one uncommitted deferral with duplicate suppression, and OK slow-quorum accept with `okVotes >= SlowQuorum` as the only quorum detail in this model. | TryPreAccept message paths, complete optimized-recovery branch parity, unbounded recovery trees, arbitrary message loss, and recovery under reconfiguration. |
460460
| `tla/EPaxosTryPreAcceptMessagePath.tla` | Focused 3/5/7 finite TryPreAccept request/response message paths: follower commit-only, stale/conflict reject, duplicate matching re-ack without durable rewrite, fresh durable ack; coordinator stale restart, committed evidence/direct accept, uncommitted forced/deferred handling, older-ballot ignore, duplicate OK ignore, first OK below quorum, pre-seeded quorum immediate accept, and OK slow-quorum accept. | Full network histories, evidence-query internals, complete optimized-recovery branch parity, unbounded recovery trees, arbitrary message loss, and recovery under reconfiguration. |
461+
| `tla/EPaxosTryConflictForce.tla` | Focused 3/5/7 finite quorum-arithmetic check for `tryConflictForcesSlowAccept`: required conflict leader forces slow Accept only without an existing dependency, required deferred-cycle leader forces slow Accept, optional leaders defer, and non-force cases start blocker recovery. | Full recovery histories, TryPreAccept message paths, evidence-query internals, complete optimized-recovery branch parity, and unbounded proof. |
461462
| `tla/EPaxosEvidenceQuery.tla` | Focused 3/5/7 committed-conflict evidence-query slice: guard-gated `MsgEvidence`, read-only responses, duplicate/mismatched drops, sender-preserving evidence validation, stale rejection restart, and fail-closed fallback. | Every technical-report branch, arbitrary membership/reconfiguration recovery, and unbounded proof. |
462463
| `tla/EPaxosRecovery.tla` | Stopped-owner dependency recovery and no-op unblocking for finite configs. | Arbitrary recovery under reconfiguration. |
463464
| `tla/EPaxosRollbackAllocation.tla` | Rollback allocation: a restored local checkpoint learns a later own committed instance from quorum, advances `nextInstance`, skips a known future local ref under a defensive stale-next state, allocates a fresh local ref, and preserves learned-before-fresh apply order. | Full EPaxos recovery, unbounded rollback histories, storage checksums, message loss, or arbitrary multi-replica rollback. |

MODEL_EQ_REPORT.MD

Lines changed: 6 additions & 0 deletions
Large diffs are not rendered by default.

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@
2424
- [EPAXOS_IMPLEMENTATION_PROOF.md](EPAXOS_IMPLEMENTATION_PROOF.md) explains the implemented algorithm, property claims, failure-count boundaries, proof rationale, and evidence/non-claims in detail.
2525
- [MODEL_EQ_REPORT.MD](MODEL_EQ_REPORT.MD) describes the current TLA+ model correspondence and implementation verification scope.
2626
- [RELEASE_SCOPE.md](RELEASE_SCOPE.md) is the self-contained release-scope lock for closed items, open items, and non-claims.
27-
- [tla/EPaxos.tla](tla/EPaxos.tla), [tla/EPaxosResponses.tla](tla/EPaxosResponses.tla), [tla/EPaxosRecovery.tla](tla/EPaxosRecovery.tla), [tla/EPaxosOptimizedRecovery.tla](tla/EPaxosOptimizedRecovery.tla), [tla/EPaxosTryPreAcceptBranches.tla](tla/EPaxosTryPreAcceptBranches.tla), [tla/EPaxosTryPreAcceptMessagePath.tla](tla/EPaxosTryPreAcceptMessagePath.tla), [tla/EPaxosEvidenceQuery.tla](tla/EPaxosEvidenceQuery.tla), [tla/EPaxosConfigBarrier.tla](tla/EPaxosConfigBarrier.tla), [tla/EPaxosConfigTransition.tla](tla/EPaxosConfigTransition.tla), [tla/EPaxosConfigRemoveTransition.tla](tla/EPaxosConfigRemoveTransition.tla), [tla/EPaxosConfigChainTransition.tla](tla/EPaxosConfigChainTransition.tla), [tla/EPaxosRevisited.tla](tla/EPaxosRevisited.tla), [tla/TOQClockDiscipline.tla](tla/TOQClockDiscipline.tla), [tla/ReadyAdvance.tla](tla/ReadyAdvance.tla), [tla/Quorum.tla](tla/Quorum.tla), [tla/KVTimestampStaleness.tla](tla/KVTimestampStaleness.tla), and [tla/KVOmissionRecovery.tla](tla/KVOmissionRecovery.tla) contain the finite executable formal models checked by CI.
27+
- [tla/EPaxos.tla](tla/EPaxos.tla), [tla/EPaxosResponses.tla](tla/EPaxosResponses.tla), [tla/EPaxosRecovery.tla](tla/EPaxosRecovery.tla), [tla/EPaxosOptimizedRecovery.tla](tla/EPaxosOptimizedRecovery.tla), [tla/EPaxosTryPreAcceptBranches.tla](tla/EPaxosTryPreAcceptBranches.tla), [tla/EPaxosTryPreAcceptMessagePath.tla](tla/EPaxosTryPreAcceptMessagePath.tla), [tla/EPaxosTryConflictForce.tla](tla/EPaxosTryConflictForce.tla), [tla/EPaxosEvidenceQuery.tla](tla/EPaxosEvidenceQuery.tla), [tla/EPaxosConfigBarrier.tla](tla/EPaxosConfigBarrier.tla), [tla/EPaxosConfigTransition.tla](tla/EPaxosConfigTransition.tla), [tla/EPaxosConfigRemoveTransition.tla](tla/EPaxosConfigRemoveTransition.tla), [tla/EPaxosConfigChainTransition.tla](tla/EPaxosConfigChainTransition.tla), [tla/EPaxosRevisited.tla](tla/EPaxosRevisited.tla), [tla/TOQClockDiscipline.tla](tla/TOQClockDiscipline.tla), [tla/ReadyAdvance.tla](tla/ReadyAdvance.tla), [tla/Quorum.tla](tla/Quorum.tla), [tla/KVTimestampStaleness.tla](tla/KVTimestampStaleness.tla), and [tla/KVOmissionRecovery.tla](tla/KVOmissionRecovery.tla) contain the finite executable formal models checked by CI.
2828
- [examples/kv](examples/kv) contains the Pebble/MyRocks-style key-value example.
2929
- [jepsen](jepsen) contains the Jepsen workload harness for external validation.
3030
- [tests](tests) contains the repository verification scripts used by CI.

RELEASE_SCOPE.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,8 @@ This matrix is the current simulation/local-loopback fault-tolerance envelope. A
8585
| Evidence bundle and go/no-go workflow | `release/EPAXOS_READINESS_EVIDENCE.md`; `tests/go_no_go_workflow.sh`; `tests/ci.sh`; `bash tests/go_no_go_workflow.sh` returns the current `No-go.` decision and lists open release items; `bash tests/release_scope_audit.sh` checks the evidence/workflow paths. |
8686

8787
TryPreAccept message-path coverage note: `tests/tla_model_check.sh` now runs `tla/EPaxosTryPreAcceptMessagePath.cfg`, `tla/EPaxosTryPreAcceptMessagePathFive.cfg`, and `tla/EPaxosTryPreAcceptMessagePathSeven.cfg`; each finite 3/5/7 config covers follower `MsgTryPreAccept` commit-only, stale/conflict reject, duplicate matching re-ack without durable rewrite, fresh durable ack, and coordinator `MsgTryPreAcceptResp` stale restart, older/duplicate OK ignore, first OK below quorum, pre-seeded quorum immediate accept, and OK slow-quorum accept.
88+
TryConflict force/defer quorum note: `tests/tla_model_check.sh` now runs `tla/EPaxosTryConflictForce.cfg`, `tla/EPaxosTryConflictForceFive.cfg`, and `tla/EPaxosTryConflictForceSeven.cfg`; the finite 3/5/7 model checks `leaderMustBeInCandidateFastQuorum` arithmetic for conflict-leader force, deferred-cycle force, optional-leader deferral, existing-dependency guard, and blocker-recovery deferral.
89+
8890

8991
## Open release items
9092

@@ -99,6 +101,8 @@ TryPreAccept message-path coverage note: `tests/tla_model_check.sh` now runs `tl
99101
TryPreAccept response branch-slice note: `tla/EPaxosTryPreAcceptBranches.tla` is only a finite abstract scenario/stage model for stale restart, committed evidence ignore/fail-closed, direct/forced accept, one uncommitted deferral with duplicate suppression, and OK slow-quorum accept. Its only quorum detail is `okVotes >= SlowQuorum`, it runs for 3/5/7, and complete optimized-recovery branch parity beyond this finite slice remains open.
100102

101103
TryPreAccept message-path limit: `tla/EPaxosTryPreAcceptMessagePath.tla` is finite 3/5/7 request/response path coverage, not a full network, evidence-query-internal, complete optimized-recovery branch-parity, or unbounded recovery proof.
104+
TryConflict force/defer limit: `tla/EPaxosTryConflictForce.tla` is finite quorum-arithmetic coverage for one uncommitted-conflict force/defer decision, not arbitrary recovery history, full message delivery, complete optimized-recovery branch parity, or unbounded proof.
105+
102106

103107
## Review baseline
104108

release/EPAXOS_READINESS_EVIDENCE.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,7 @@ Status: no-go evidence bundle for the active EPaxos production-readiness goal. T
8181
- `JAVA_BIN=/opt/homebrew/opt/openjdk/bin/java bash tests/tla_model_check.sh` passed after wiring `tla/EPaxosConfigChainTransition.cfg`; `EPaxosConfigChainTransition.cfg` generated 299 states and 299 distinct states with no TLC error.
8282
- `JAVA_BIN=/opt/homebrew/opt/openjdk/bin/java bash tests/tla_model_check.sh` passed after wiring `tla/EPaxosTryPreAcceptBranches.cfg`, `tla/EPaxosTryPreAcceptBranchesFive.cfg`, and `tla/EPaxosTryPreAcceptBranchesSeven.cfg`; each abstract branch-slice config generated 17 states and 17 distinct states with no TLC error.
8383
- `JAVA_BIN=/opt/homebrew/opt/openjdk/bin/java bash tests/tla_model_check.sh` passed after wiring `tla/EPaxosTryPreAcceptMessagePath.cfg`, `tla/EPaxosTryPreAcceptMessagePathFive.cfg`, and `tla/EPaxosTryPreAcceptMessagePathSeven.cfg`; each finite message-path config generated 32 states and 32 distinct states with no TLC error.
84+
- `JAVA_BIN=/opt/homebrew/opt/openjdk/bin/java bash tests/tla_model_check.sh` passed after wiring `tla/EPaxosTryConflictForce.cfg`, `tla/EPaxosTryConflictForceFive.cfg`, and `tla/EPaxosTryConflictForceSeven.cfg`; the finite force/defer configs generated 104/104, 168/168, and 232/232 states respectively with no TLC error.
8485

8586
### Fault-tolerance envelope proof summary
8687

@@ -164,6 +165,8 @@ Non-claims remain explicit: No target-environment remote claim, no in-place Pebb
164165
- Add finite 3-, 5-, and 7-replica abstract scenario/stage TryPreAccept response-branch coverage: stale restart, committed evidence ignore/fail-closed, direct/forced accept, one uncommitted deferral with duplicate suppression, and OK slow-quorum accept. The only quorum detail modeled inside this slice is `okVotes >= SlowQuorum`; this is not a TryPreAccept message-path model, complete optimized-recovery branch parity, or unbounded recovery proof.
165166
- `tla/EPaxosTryPreAcceptMessagePath.tla`, `tla/EPaxosTryPreAcceptMessagePath.cfg`, `tla/EPaxosTryPreAcceptMessagePathFive.cfg`, and `tla/EPaxosTryPreAcceptMessagePathSeven.cfg`
166167
- Add finite 3-, 5-, and 7-replica TryPreAccept request/response message-path coverage for follower `MsgTryPreAccept` commit-only, stale/conflict reject, duplicate matching re-ack without durable rewrite, fresh durable ack, and coordinator `MsgTryPreAcceptResp` stale restart, committed evidence/direct accept, uncommitted forced/deferred handling, older-ballot ignore, duplicate OK ignore, first OK below quorum, pre-seeded quorum immediate accept, and OK slow-quorum accept. This is not a full-network, evidence-query-internal, complete optimized-recovery branch-parity, or unbounded recovery proof.
168+
- `tla/EPaxosTryConflictForce.tla`, `tla/EPaxosTryConflictForce.cfg`, `tla/EPaxosTryConflictForceFive.cfg`, and `tla/EPaxosTryConflictForceSeven.cfg`
169+
- Add finite 3-, 5-, and 7-replica quorum-arithmetic coverage for `tryConflictForcesSlowAccept`: a conflict leader forces slow Accept only when the candidate lacks that dependency and every possible candidate fast quorum must include the leader, a deferred-cycle leader can independently force slow Accept when required, and optional/non-required leaders defer with blocker recovery. This is not a full recovery, network, or unbounded proof.
167170
- `tla/EPaxosEvidenceQuery.tla`, `tla/EPaxosEvidenceQuery.cfg`, `tla/EPaxosEvidenceQueryFive.cfg`, and `tla/EPaxosEvidenceQuerySeven.cfg`
168171
- Add finite 3-, 5-, and 7-replica committed-conflict evidence-query coverage: candidate-dependency/same-config guards before `MsgEvidence`, read-only response handling, duplicate/mismatched response drops, sender-preserving `AcceptEvidence` validation, stale TryPreAccept rejection restart, and fail-closed slow accept on missing, legacy-only, malformed, contradictory, or insufficient evidence.
169172
- `tla/EPaxosConfigBarrier.tla` and `tla/EPaxosConfigBarrier.cfg`
@@ -212,6 +215,7 @@ The following blockers are still listed in `RELEASE_SCOPE.md` and prevent a go d
212215
- Broader formal model coverage remains open beyond the finite configured TLC suite; `tla/EPaxosResponses.tla` adds bounded prepare branch-priority/try-witness checks, `tla/EPaxosOptimizedRecovery.tla` adds finite 3-, 5-, and 7-replica Accept-Deps optimized-recovery evidence checks, `tla/EPaxosEvidenceQuery.tla` adds finite 3-, 5-, and 7-replica committed-conflict evidence-query guard/fail-closed checks, `tla/EPaxosConfigBarrier.tla` adds finite local config-barrier checks, `tla/EPaxosConfigTransition.tla` adds one finite add-voter config-transition pinning check, `tla/EPaxosConfigRemoveTransition.tla` adds one finite remove-voter config-transition pinning check, `tla/EPaxosConfigChainTransition.tla` adds one finite add-then-remove configuration-chain pinning check, `tla/EPaxosRollbackAllocation.tla` adds one finite rollback-allocation next-instance/skip/apply-order check, and `tla/TOQClockDiscipline.tla` adds a finite bounded-skew/bounded-delay `ProcessAt` contract check, but operational synchronized-clock/OWD-measurement implementation proof for TOQ deployments, unbounded proof, arbitrary membership-change proof, arbitrary multi-step reconfiguration proof, recovery under configuration changes, complete optimized-recovery branch parity, full rollback-history proof, and arbitrary application/state-machine semantics remain open.
213216
- `tla/EPaxosTryPreAcceptBranches.tla` is a finite abstract scenario/stage branch slice only; it covers stale restart, committed evidence ignore/fail-closed, direct/forced accept, one uncommitted deferral with duplicate suppression, and OK slow-quorum accept for 3/5/7 with `okVotes >= SlowQuorum` as the only quorum detail. Complete optimized-recovery branch parity beyond that slice remains open.
214217
- `tla/EPaxosTryPreAcceptMessagePath.tla` adds finite 3/5/7 concrete TryPreAccept request/response message-path coverage, including pre-seeded quorum immediate accept, but complete optimized-recovery branch parity beyond the finite branch/message-path/evidence-query slices remains open.
218+
- `tla/EPaxosTryConflictForce.tla` adds finite 3/5/7 quorum-arithmetic coverage for uncommitted-conflict force/defer decisions, but arbitrary recovery histories and complete optimized-recovery branch parity remain open.
215219
- Deployment manifest artifacts are example/operator material only; `tests/kvnode_systemd_manifest_audit.sh` renders and audits the example `ExecStart` contract, but reviewed execution under a target system manager, container, or orchestration environment remains open.
216220
- Data lifecycle now has a maintained local offline `kvcheckpoint` helper plus runbook/audit evidence, but a reviewed operator backup/restore/disaster-recovery drill in the target environment remains open. The mixed-version drill's binary rollback keeps current data and does not exercise checkpoint restore.
217221
- Target-environment capacity-envelope measurements remain open; local evidence now includes a single-node workstation sample plus a three-node loopback wrapper sample, but neither is a measured target-environment capacity result.

tests/release_scope_audit.sh

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,9 @@ require_text "TryPreAccept response branch-slice"
114114
require_text "tla/EPaxosTryPreAcceptMessagePath.cfg"
115115
require_text "tla/EPaxosTryPreAcceptMessagePathSeven.cfg"
116116
require_text "TryPreAccept message-path coverage"
117+
require_text "tla/EPaxosTryConflictForce.cfg"
118+
require_text "tla/EPaxosTryConflictForceSeven.cfg"
119+
require_text "force/defer quorum"
117120
require_text "tla/EPaxosEvidenceQuery.tla"
118121
require_text "tla/EPaxosEvidenceQuerySeven.cfg"
119122
require_text 'unique TLC `-metadir`'
@@ -207,6 +210,10 @@ require_path "tla/EPaxosTryPreAcceptMessagePath.tla"
207210
require_path "tla/EPaxosTryPreAcceptMessagePath.cfg"
208211
require_path "tla/EPaxosTryPreAcceptMessagePathFive.cfg"
209212
require_path "tla/EPaxosTryPreAcceptMessagePathSeven.cfg"
213+
require_path "tla/EPaxosTryConflictForce.tla"
214+
require_path "tla/EPaxosTryConflictForce.cfg"
215+
require_path "tla/EPaxosTryConflictForceFive.cfg"
216+
require_path "tla/EPaxosTryConflictForceSeven.cfg"
210217
require_path "tla/EPaxosEvidenceQuery.tla"
211218
require_path "tla/EPaxosEvidenceQuerySeven.cfg"
212219
require_path "tla/EPaxosConfigTransition.tla"

tests/tla_model_check.sh

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,10 @@ for cfg in tla/EPaxosTryPreAcceptMessagePath.cfg tla/EPaxosTryPreAcceptMessagePa
7777
run_tlc tla/EPaxosTryPreAcceptMessagePath.tla "$cfg"
7878
done
7979

80+
for cfg in tla/EPaxosTryConflictForce.cfg tla/EPaxosTryConflictForceFive.cfg tla/EPaxosTryConflictForceSeven.cfg; do
81+
run_tlc tla/EPaxosTryConflictForce.tla "$cfg"
82+
done
83+
8084
for cfg in tla/EPaxosEvidenceQuery.cfg tla/EPaxosEvidenceQueryFive.cfg tla/EPaxosEvidenceQuerySeven.cfg; do
8185
run_tlc tla/EPaxosEvidenceQuery.tla "$cfg"
8286
done

tla/EPaxosTryConflictForce.cfg

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
SPECIFICATION Spec
2+
INVARIANT TypeOK
3+
INVARIANT Safety
4+
PROPERTY EventuallyCoversForceDecision
5+
CHECK_DEADLOCK FALSE
6+
CONSTANTS
7+
ReplicaCount = 3

0 commit comments

Comments
 (0)