Skip to content

offline: let a replay report that it was never delivered - #354

Merged
Yaraslaut merged 2 commits into
masterfrom
feature/343-replay-outcome
Aug 31, 2026
Merged

offline: let a replay report that it was never delivered#354
Yaraslaut merged 2 commits into
masterfrom
feature/343-replay-outcome

Conversation

@Yaraslaut

Copy link
Copy Markdown
Member

Closes #343.

The gap

SyncWorker::ReplayFunction returns bool. Its only two outcomes are "remove it" and "charge one attempt", so there was no way for a caller to say "this never reached the server" — a transport failure and a server-side rejection were charged to the same 5-attempt budget, and that budget is durable (written back through IOfflineQueue::setAttempts()).

Five reconnect flaps therefore exhausted the budget of every queued item and dropped them all, through the same DeadLetterSink call and the same user-facing "N changes could not be synced" state a genuine rejection produces. Work the server never saw was reported as work that could not be applied, and the payload was gone unless the host's sink persisted it.

Reproduced on master, exactly the shape the issue asked this finding to carry before triage:

queued 3 items
flap 1: successful=0 failed=3 deadLettered=0  queueDepth=3
flap 2: successful=0 failed=3 deadLettered=0  queueDepth=3
flap 3: successful=0 failed=3 deadLettered=0  queueDepth=3
flap 4: successful=0 failed=3 deadLettered=0  queueDepth=3
flap 5: successful=0 failed=0 deadLettered=3  queueDepth=0

dead-lettered: card-move-0/1/2 (attempts=5 each)

Two shipped conditions make this reachable rather than theoretical: ReconnectCoordinator::onOnline() holds its mutex for the whole retry loop, so a flap back offline cannot preempt an in-progress replay; and nothing in the framework wires a NetworkMonitor transition to SyncWorker::stop().

The change

ReplayOutcome { Succeeded, Rejected, Undelivered } plus a DetailedReplayFunction constructor overload. Only Rejected spends an attempt. Undelivered touches neither the in-memory count nor setAttempts() — advancing a durable budget on one side alone would still walk the item towards the sink across a restart. SyncResult gains undelivered.

This is the distinction the count is spent on, not a knob on the count. The hard-coded cap of 5 is a documented design decision in offline.md and is unchanged; the issue explicitly does not dispute it, and neither does this PR.

Compatibility — the boolean form is untouched

false maps to Rejected, deliberately not to Undelivered. Every existing caller written against bool means "this attempt failed", and silently re-reading that as undelivered would retry a genuinely poison payload forever — the exact failure the cap exists to bound. A throw is charged for the same reason: it reports failure but says nothing about delivery, and "I don't know" has to be charged.

The boolean overload adapts into the detailed one, so run() implements a single contract rather than two. The overloads are unambiguous: ReplayOutcome is a scoped enum, so neither return type implicitly converts to the other.

The framework side chose the seam over the alternative the issue offered — "document an obligation on the host to gate run() on liveness". That pushes a correctness requirement onto every application, and the rung that motivated the finding shows how that goes: kanban never wires NetworkMonitor to stop(). A host obligation the flagship consumer did not discover is a trap, not a seam.

Tests

Six cases in tests/test_sync_worker.cpp: the five-flap reproduction; that Undelivered leaves the durable count untouched (via a queue recording every setAttempts write); that Rejected still spends the budget and dead-letters on the 5th; that undelivered flaps interleaved between rejections do not shorten the budget; that the boolean form keeps its exact previous meaning; and that a throwing detailed replay is still charged.

Mutation-tested, per AGENTS.md's "would this still pass if the feature did nothing?" — these tests reference a new enum, so compiling is not evidence. Neutering the Undelivered branch so the feature does nothing:

test cases:  6 |  3 passed |  3 failed
assertions: 69 | 38 passed | 31 failed

Exactly the three Undelivered cases fail; the Rejected, boolean and throw cases correctly still pass, since the mutant does not change those paths.

Verification

  • Full suite green under Clang: 1361/1361.
  • Rebuilt and re-run under GCC: green (61/61 in the offline subset).
  • check_spec_citations.sh, check_test_type_names.sh, check_deprecated_markers.sh: pass.
  • docs/spec/offline/offline.md updated: SyncResult table, the SyncWorker API table, and the retry/dead-letter rules.

Deliberately not in this PR

examples/kanban/gui_lib/board_qml_bridge.cpp's replay lambda still returns false on any onError and so still cannot distinguish the two cases. Adopting DetailedReplayFunction there is a rung change, not a framework one, and folding it in would mix two things — the framework seam it needs now exists.

`SyncWorker::ReplayFunction` returns `bool`, whose only two outcomes are
"remove it" and "charge one attempt". There was no way to say *"this never
reached the server"*, so a transport failure and a server-side rejection were
charged to the same 5-attempt budget — and that budget is durable, written
back through `IOfflineQueue::setAttempts()`.

Five reconnect flaps therefore exhausted the budget of every queued item and
dropped them all, through the same `DeadLetterSink` call and the same
user-facing "N changes could not be synced" state a genuine rejection
produces. Work the server never saw was reported to the user as work that
could not be applied, and the payload was gone unless the host's sink
persisted it. Two shipped conditions make that reachable rather than
theoretical: `ReconnectCoordinator::onOnline()` holds its mutex for the whole
retry loop, so a flap back offline cannot preempt an in-progress replay; and
nothing in the framework wires a `NetworkMonitor` transition to
`SyncWorker::stop()`.

Adds `ReplayOutcome` (`Succeeded`/`Rejected`/`Undelivered`) and a
`DetailedReplayFunction` constructor overload taking it. Only `Rejected`
spends an attempt; `Undelivered` touches neither the in-memory count nor
`setAttempts()`, since advancing a durable budget on one side alone would
still walk the item towards the sink across a restart. `SyncResult` gains
`undelivered`.

This is the distinction the count is spent on, not a knob on the count: the
hard-coded cap of 5 is a documented design decision and is unchanged.

The boolean `ReplayFunction` keeps its exact previous meaning — `false` maps
to `Rejected`, deliberately not to `Undelivered`, because every existing
caller means "this attempt failed" and re-reading that as undelivered would
retry a poison payload forever. A throw is charged for the same reason: it
reports failure but says nothing about delivery. The boolean overload adapts
into the detailed one, so `run()` implements a single contract. The two
overloads are unambiguous — `ReplayOutcome` is a scoped enum, so neither
return type implicitly converts to the other.

Closes #343

Signed-off-by: Yaraslau Tamashevich <yaraslau.tamashevich@gmail.com>
@codecov

codecov Bot commented Aug 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

The whole-tree clang-format gate flagged the #343 workflow tests. No
behaviour change; all 25 sync cases still pass.

Signed-off-by: Yaraslau Tamashevich <yaraslau.tamashevich@gmail.com>
@Yaraslaut
Yaraslaut merged commit 8e15fc3 into master Aug 31, 2026
40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SyncWorker's replay budget cannot tell an undelivered replay from a rejected one, so reconnect flaps dead-letter work the server never saw

1 participant