Skip to content

Commit b3c9397

Browse files
OneTwo3DOneTwo3Dclaude
authored
WP2 S3: route Xero producers through producerDisposition (o3d-zjsb5.26) (#764)
* WP2 S3 (wip): Xero producer seam, shadow store, HELD_SHADOW standing Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb * WP2 S3 (wip): seam tests, census seam checks, reader tweaks, docs Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb * WP2 S3 (wip): shadow display cause, tests, operator docs Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb * WP2 S3: Xero producer seam (final wording of the sync log help) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb * WP2 S3: lint (no assignment to the variable named module), drop an unused class in the db test Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb * WP2 S3: claim-boundary gate in the Xero processor, re-queue gating, distinct SHADOW_NOT_SENT_BY_IMS standing Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb * WP2 S3: strengthen the manual-retry structural arm Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb * WP2 S3: instruction-site census, retention fragment for empty ids, delete-guard wording Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb * WP2 S3 round 2: hold at the follow-up revival, durable shadow evidence for destructive readers, switch-off daily reset restored 1. enqueueFollowUpSyncLog: a revival of a FAILED row asks xeroProducerSeamVerdict before the PENDING update and the outbox schedule; a held revival leaves the FAILED row untouched and falls to the create path, which records the shadow. New PENDING-transition census test (every sync-row PENDING write declared with whether it consults the seam; fails on an unlisted one). The manual-retry no-candidates fenced branch now asks the seam too. 2. Retention deletes a shadow's sync row but outbound_shadow_writes persists: findDurableShadowRecords (new) is consulted by findSalesOrderDeleteBlocker (order, shipments, staged batch refs), adjustmentJournalMayHaveReachedLedger and countSalesInvoiceRowsThatMayHavePosted; the ledger-standing reader census gains a durable-shadow section (SHADOW_READER_DECLARATIONS). Payment-receipt retract cannot consult it yet (filed). 3. resetFailedDailyBatchLogs: with enforcement not on for Xero the original status-wide statements run unchanged; the id-set path is only for enforcement ON. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb * WP2 S3 round 2: type the new test fixtures Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb * WP2 S3 round 2: retention note outside the generated docs block Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb --------- Co-authored-by: OneTwo3D <onetwo3dcouk@gmail.com> Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
1 parent 0a1bed1 commit b3c9397

66 files changed

Lines changed: 3663 additions & 92 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.env.example‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -291,11 +291,14 @@ MINTSOFT_WEBHOOK_SWEEPER_PAGE_SIZE=250
291291
# MINTSOFT_WRITE_ALLOWED=
292292
# XERO_WRITE_ALLOWED_TENANT=
293293
#
294-
# PRODUCER-SIDE HOLD (NOT YET ENFORCED): the instant from which a decision module would let IMS
295-
# produce LIVE work for a destination (one UTC instant in ISO-8601 with an explicit Z). In this
296-
# version nothing in IMS reads these variables and no producer behaves differently because of them;
297-
# a grant alone permits the existing outbound writes through the transport. Producers will consult
298-
# the decision in later changes. Leave them unset. See docs/installation.md ("Producer-side hold").
294+
# PRODUCER-SIDE HOLD (ENFORCED ONLY FOR THE DESTINATIONS NAMED BELOW; UNSET BY DEFAULT): the instant from which
295+
# IMS may produce LIVE work for a destination (one UTC instant in ISO-8601 with an explicit Z), and the list of
296+
# destinations whose producers obey the decision at all. Unset, every producer queues work exactly as before and a
297+
# grant alone permits the outbound writes through the transport. Today only the Xero producers consult the
298+
# decision; naming mintsoft or woocommerce changes nothing for them yet. With xero named and no grant and no
299+
# live-from instant set, every new Xero write becomes a recorded shadow and the Xero queue receives nothing new.
300+
# Leave all of these unset unless you mean to hold Xero. See docs/installation.md ("Producer-side hold").
301+
# PRODUCER_HOLD_ENFORCED_DESTINATIONS=
299302
# XERO_WRITES_LIVE_FROM=
300303
# MINTSOFT_WRITES_LIVE_FROM=
301304
# WC_WRITES_LIVE_FROM=

‎app/actions/settings.ts‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -679,6 +679,11 @@ export async function generateMissingXeroTaxRates(
679679
reportTypeOverrides?: Record<string, string>,
680680
): Promise<MissingTaxRateGenerateResult> {
681681
await requirePermission('settings.company')
682+
// THE PRODUCER-SIDE HOLD, before anything is sent to Xero: this action writes straight to Xero with no queue row, so
683+
// it asks here. A SHADOW is recorded and nothing is created (lib/domain/accounting/xero-direct-write-seam.ts).
684+
const { xeroTaxRateWriteShadow } = await import('@/lib/domain/accounting/xero-direct-write-seam')
685+
const shadow = await xeroTaxRateWriteShadow({ taxRateIds, reportTypeOverrides })
686+
if (shadow) return { success: false, created: 0, failed: [], externalRatesCount: 0, supported: true, error: shadow.notice }
682687
try {
683688
const { getXeroTaxRates } = await import('@/lib/connectors/xero/accounts')
684689
const { putXeroTaxRate } = await import('@/lib/connectors/xero/tax-rates')

‎app/actions/xero-sync.ts‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
'use server'
22

3+
import { xeroProducerSeamVerdict } from '@/lib/domain/accounting/xero-producer-seam'
34
import { freshAuthFailureResult, requireFreshPermission, requirePermission, requireRole } from '@/lib/auth/server'
45

56
import { revalidatePath } from 'next/cache'
@@ -746,6 +747,13 @@ export async function retryFailedXeroSync(
746747
// o3d-0m56 filter. The fence is run precisely to produce the reason: its CAS cannot match, and
747748
// it reads the row to say whether the STATUS or the ATTEMPT moved.
748749
if (entryId === undefined || expectedAttemptRevision === undefined) return { success: true, reset: 0 }
750+
// THE PRODUCER-SIDE HOLD: this fence CAS can still land (the row may have failed since the candidate read above), and it
751+
// writes PENDING, so the decision is asked on the row's own type and payload BEFORE the write. Held: the row stays FAILED.
752+
const lateRow = await db.accountingSyncLog.findUnique({ where: { id: entryId }, select: { type: true, payload: true } })
753+
if (lateRow) {
754+
const lateHold = xeroProducerSeamVerdict({ connector: 'xero', type: String(lateRow.type), payload: lateRow.payload })
755+
if (lateHold.kind === 'shadow') return { success: false, reset: 0, error: lateHold.notice }
756+
}
749757
const outcome = await applyFencedAttemptDecision(db, {
750758
id: entryId,
751759
expectedAttemptRevision,
@@ -857,6 +865,13 @@ export async function retryFailedXeroSync(
857865
const allowed: typeof scopeCandidates = []
858866
const refused: Array<{ id: string; reason: string; tokenCount: number }> = []
859867
for (const candidate of scopeCandidates) {
868+
// THE PRODUCER-SIDE HOLD AT THE RE-QUEUE: a row whose decision is SHADOW is not made claimable again. (The claim
869+
// boundary in the processor would refuse to send it anyway; this stops the row cycling back to PENDING.)
870+
const hold = xeroProducerSeamVerdict({ connector: 'xero', type: String(candidate.type), payload: candidate.payload })
871+
if (hold.kind === 'shadow') {
872+
refused.push({ id: candidate.id, reason: hold.notice, tokenCount: 1 })
873+
continue
874+
}
860875
const plan = planManualRetry({
861876
type: candidate.type,
862877
reference: `${candidate.referenceType} ${candidate.referenceId}`,

‎docs/installation.md‎

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5094,23 +5094,35 @@ Runbook. If a feed is reported stale, first find out which side stopped: look at
50945094
### Producer-side hold: LIVE or SHADOW for each unit of work
50955095

50965096
<!-- producer-disposition:overview -->
5097-
THE PRODUCER-SIDE HOLD IS NOT YET ENFORCED. In this version it is a decision module only: nothing in IMS calls it, `npm run outbound:status` does not report on it, and every producer still queues work exactly as before. The only barrier is the outbound-write hold above, which reads a grant on its own: a destination whose grant is set permits the existing outbound writes through the transport whether or not the live-from variable below is set. The places that will consult the decision arrive in later changes, and the paragraphs below describe what the decision returns, not what IMS does today.
5097+
THE PRODUCER-SIDE HOLD IS ENFORCED ONLY FOR THE DESTINATIONS NAMED IN `PRODUCER_HOLD_ENFORCED_DESTINATIONS`, WHICH IS UNSET BY DEFAULT. Unset, every producer queues work exactly as it did before the decision existed, and the only barrier is the outbound-write hold above, which reads a grant on its own: a destination whose grant is set permits the outbound writes through the transport whether or not the live-from variable below is set. Today only the Xero producers consult the decision (every place IMS creates an accounting sync log row, and the direct tax-rate creation from Settings; the one sync log type whose destination is WooCommerce, the invoice note, follows the WooCommerce setting); the WooCommerce and Mintsoft producers do not yet, so naming them in the variable changes nothing for them in this version, and `npm run outbound:status` does not report on the decision.
50985098

5099-
For each destination and operation the decision is LIVE or SHADOW. A SHADOW is the record of what IMS would have written, which a later change will keep instead of queuing the work; it is never to be delivered later. The decision reads the environment only: no database and no network call. It is LIVE only when every one of these holds, and SHADOW otherwise: the outbound-write grant for the destination is readable; the live-from variable for the destination is set and readable; the current time is at or after it; the ownership map says IMS owns that operation in the installation phase that follows from the two; and the time of the business event is at or after the live-from instant (an operation whose ownership-map row requires that time is SHADOW when the producer does not supply it). A value that is absent, unreadable or inconsistent is never LIVE. There is no phase setting: an installation is in its live phase for a destination exactly when the grant and the live-from instant are both in force, so a restored backup, a clone or a new checkout never inherits it.
5099+
For each destination and operation the decision is LIVE or SHADOW. It is LIVE only when every one of these holds, and SHADOW otherwise: the outbound-write grant for the destination is readable; the live-from variable for the destination is set and readable; the current time is at or after it; the ownership map says IMS owns that operation in the installation phase that follows from the two; and the time of the business event is at or after the live-from instant (an operation whose ownership-map row requires that time is SHADOW when the producer does not supply it). A value that is absent, unreadable or inconsistent is never LIVE. There is no phase setting: an installation is in its live phase for a destination exactly when the grant and the live-from instant are both in force, so a restored backup, a clone or a new checkout never inherits it. The decision itself reads the environment only: no database and no network call.
51005100

5101-
A grant without a live-from instant, or a live-from instant without a grant, is inconsistent: the decision is SHADOW, while today the transport still allows the writes of a granted destination. A later change will report the inconsistency in `npm run outbound:status`. Never move a live-from instant earlier once work has been produced after it: that is the only change that lets work from before the move reach the destination.
5101+
A SHADOW is the record of what IMS would have written, kept instead of queuing the work; it is never delivered later. For Xero a shadow is a row in the accounting sync log that is CANCELLED with the settlement basis `HELD_SHADOW`, plus a row in `outbound_shadow_writes` (one per distinct piece of work, counting repeats). A shadow has no outbox job, is never given an accounting event, and has its own ledger standing (shadow, not sent by IMS) that is NOT proof the document is absent from Xero: it says nothing about whether the owner of the operation wrote it, so deleting, reversing or refunding over it is refused as unproven. A destination whose enforcement is on and whose grant and live-from instant are both unset therefore queues nothing new: every Xero write IMS would have produced becomes a shadow.
5102+
5103+
A grant without a live-from instant, or a live-from instant without a grant, is inconsistent: the decision is SHADOW, while the transport still allows the writes of a granted destination. A later change will report the inconsistency in `npm run outbound:status`. Never move a live-from instant earlier once work has been produced after it: that is the only change that lets work from before the move reach the destination. Rows already queued when enforcement is switched on are not changed, but the Xero queue processor asks the same decision immediately before it sends: a row whose decision is not LIVE is handed back unsent with the operator text (no retry is spent) and is never posted, and a manual re-queue or the daily-batch reset leaves such a row unclaimable. A later change will also refuse to deliver a row created before the live-from instant and retire the backlog.
51025104
<!-- /producer-disposition:overview -->
51035105

5106+
Retention and the shadow record: a shadow's sync-log row expires with the sync-log retention window (it holds no ledger claim), but the `outbound_shadow_writes` record does not, and the readers that would destroy the local record of a document (the sales-order hard delete, an edit of a posted stock movement, a stored invoice-number correction) go on refusing from that record after the row is gone; the payment-receipt retract is the one known reader that cannot yet do so (tracked). With enforcement OFF the daily-batch re-queue runs its original statement unchanged.
5107+
51045108
<!-- producer-disposition:cutoffs -->
51055109
| Variable | Destination | Value |
51065110
|---|---|---|
51075111
| `WC_WRITES_LIVE_FROM` | WooCommerce | one UTC instant in ISO-8601 with an explicit Z, for example 2026-12-01T00:00:00Z |
51085112
| `MINTSOFT_WRITES_LIVE_FROM` | Mintsoft | one UTC instant in ISO-8601 with an explicit Z, for example 2026-12-01T00:00:00Z |
51095113
| `XERO_WRITES_LIVE_FROM` | Xero | one UTC instant in ISO-8601 with an explicit Z, for example 2026-12-01T00:00:00Z |
51105114

5111-
Each variable names one instant. A date without a time, a time without the Z, an offset such as +01:00, precision finer than a millisecond, surrounding whitespace, a list or any other shape is unreadable, and the decision for that destination is SHADOW. Setting a variable changes nothing in this version, because nothing consults it yet; in later changes, once the producers consult the decision, each destination will need its outbound-write grant as well, and the ownership map will decide which operations IMS may produce.
5115+
Each variable names one instant. A date without a time, a time without the Z, an offset such as +01:00, precision finer than a millisecond, surrounding whitespace, a list or any other shape is unreadable, and the decision for that destination is SHADOW. A destination needs its outbound-write grant as well as its live-from instant before the decision can be LIVE, and the ownership map decides which operations IMS may produce. A variable has an effect only for a destination named in `PRODUCER_HOLD_ENFORCED_DESTINATIONS`.
51125116
<!-- /producer-disposition:cutoffs -->
51135117

5118+
<!-- producer-disposition:enforcement -->
5119+
| Variable | Value |
5120+
|---|---|
5121+
| `PRODUCER_HOLD_ENFORCED_DESTINATIONS` | a comma-separated list of destinations from xero, mintsoft and woocommerce, or all; unset or empty enforces nothing |
5122+
5123+
Names the destinations whose producers obey the decision. A destination that is named obeys it for every operation; one that is not named is untouched. A value that is not a list of known destinations or `all` (a misspelling, a destination that does not exist) enforces EVERY destination, because a typo must not leave a hold off. Switching it on for Xero on an installation with no grant and no live-from instant makes every new Xero write a shadow and stops the Xero queue from receiving new work; switching it off again does not recover shadows, which are records and not queued work. Do not switch it on for WooCommerce or Mintsoft in this version: those producers do not consult the decision yet.
5124+
<!-- /producer-disposition:enforcement -->
5125+
51145126

51155127
### Connection pooling in front of `DATABASE_URL` is not supported
51165128

‎help-docs/xero-sync.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1987,6 +1987,7 @@ The sync log at **Integrations → Xero** shows queued transactions for the **cu
19871987
- **Failed** — Failed after 5 retries (shows error message)
19881988

19891989
- **Cancelled** (shown as **Retired**) — the row was taken out of the queue rather than completed. Usually nothing was posted (a sales order cancelled before its invoice went out, an orphaned queue row swept up). **Sometimes something was**: when the repair sweep retires a row because the sale is not live, the Xero document it names is real and is kept on the row on purpose. No sweep re-queues a retired row.
1990+
- **Cancelled, with the reason "Not sent by IMS: writes to Xero are held on this installation (…)"** — a *shadow*. The producer-side hold recorded what IMS would have queued instead of queuing it (see "Producer-side hold" in the installation guide). It has no Xero document id, was never claimable, has no outbox job and no accounting event, and carries a **shadow** badge on this page whose tooltip says IMS did not send it. It says nothing about whether the owner it names (Xeroom, Qoblex's own Xero integration, an operator working by hand) wrote the document to Xero, so it counts as **unproven**, not as absent: deleting an order, deleting a payment receipt, reversing or re-posting over a shadow is refused until Xero itself has been checked. A queued entry that already existed when the hold was switched on is not changed, but it is handed back unsent (activity `xero_sync_producer_held_before_post`) instead of being posted while the hold says it may not be. Shadows exist only on an installation that names Xero in `PRODUCER_HOLD_ENFORCED_DESTINATIONS`; the same piece of work produced again is counted on one shadow record rather than added as another row, and Retry and Retry All never touch a shadow (they only touch Failed rows). A Failed entry that the hold would now shadow is not retried either (Retry and Retry All leave it Failed and name the hold), and neither the automatic follow-up revival nor the daily-batch reset turns it back into queued work. **The shadow row expires with the sync-log retention period, the shadow record does not.** Once the row has been deleted, deleting an order, correcting a stored invoice number and editing a posted stock adjustment are still refused, from the persistent shadow record, with the same unproven wording. One gap is known and tracked: deleting a payment receipt after the shadow row has expired is not yet refused on the shadow record, because that record does not yet carry the payment it was for.
19901991

19911992
Failed entries can be investigated via the error message and retried by resetting their status in the database.
19921993

‎lib/accounting.ts‎

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -329,12 +329,20 @@ export type ConnectorEnqueueOutcome = {
329329
* still going to post, and rolling back an empty transaction while telling the operator "nothing
330330
* was sent" is the one message that guarantees nobody goes looking for it.
331331
*/
332-
reason?: 'not-configured' | 'refused' | 'already-queued' | 'handled-by-hand' | 'hand-post-deferred'
332+
reason?: 'not-configured' | 'refused' | 'already-queued' | 'handled-by-hand' | 'hand-post-deferred' | 'shadowed'
333333
/*
334334
* `handled-by-hand` (o3d-j625 r7) — `queued: true` WITHOUT A WRITE, like `already-queued`: someone
335335
* marked this exact posting handled after posting it BY HAND, so IMS did not post it and never will
336336
* (posting-suppression.ts). A counterpart exists in the ledger; it is not owed; it was not IMS's.
337337
*/
338+
/*
339+
* `shadowed` (producer-side hold) — `queued: true` WITHOUT A QUEUED POSTING: the producer-side hold said SHADOW for this
340+
* posting (lib/security/producer-disposition.ts), so IMS recorded a shadow and created no outbox job and no accounting
341+
* event. `queued: true` is the answer the business flows need - the purchase is received, the payment is registered,
342+
* the batch ran - and `postingIsOwed` is false: IMS owes the ledger nothing here, because in the current phase another
343+
* writer owns the operation (or none does). It is NOT a statement that the document exists in the ledger, and no caller
344+
* may read it as one; the ledger standing of the shadow row is SHADOW_NOT_SENT_BY_IMS: IMS did not send it, and that is not proof of absence.
345+
*/
338346
/*
339347
* `hand-post-deferred` (o3d-j625 r18, Codex round 17 HIGH 1) — `queued: FALSE`, and the difference from
340348
* `handled-by-hand` is the whole of round 17's HIGH 1. An operator holds the hand-posting CLAIM on this
@@ -1661,6 +1669,10 @@ export async function queueAccountingSyncTx(
16611669
if (created.suppressed === 'handled_by_hand') return answer({ queued: true, reason: 'handled-by-hand' }, context.connector)
16621670
return answer({ queued: false, reason: 'hand-post-deferred' }, context.connector)
16631671
}
1672+
// THE PRODUCER-SIDE HOLD SAID SHADOW: the posting is recorded as a shadow and nothing is queued - no outbox job and no
1673+
// accounting event (a shadow must not count as an IMS ledger fact). `queued: true`, with its own reason, because the
1674+
// posting is recorded and nothing is owed to IMS's queue; see 'shadowed' on ConnectorEnqueueOutcome.
1675+
if (created.shadowed) return answer({ queued: true, reason: 'shadowed' }, context.connector)
16641676
const log = created.row
16651677
if (context.connector === 'xero') {
16661678
const { scheduleXeroAccountingOutbox } = await import('@/lib/connectors/xero/outbox')

0 commit comments

Comments
 (0)