Repository navigation
Commit e2c55ed
fix(driver-turso): the remote canonical backfill never writes a julian misreading over a stored value (#19719)
Fixes #6009
**Clause-②: no**
The sequel to PR #19688, which fixed the `driver-sql` (Knex) half of
this card at squash `40626bdca6`. The REMOTE path is a separate module
that builds its own statements and did not inherit the guard.
## The defect
`RemoteTransport.mapFieldTypeToSQL` declares every temporal column
`TEXT`, so a `Field.datetime` / `Field.time` column can hold a
digits-only string such as `'2026'`, `'86400'`, `'12'` — or the keyword
`'now'`. SQLite's time-value grammar accepts a bare number as a JULIAN
DAY and `now` as the wall clock, so for those two shapes `strftime`
answers confidently rather than returning NULL, and the
`coalesce(strftime(...), col)` that is supposed to preserve unparseable
values never fires. The convergence `UPDATE` in
`backfillRemoteCanonicalColumn` wrote that answer **over the stored
bytes**, and no later run can get them back.
Measured here on better-sqlite3 13.0.3 / SQLite 3.53.4, column declared
`TEXT`:
```text
'2026' -> -4707-06-11T12:00:00.000Z 'now' -> the wall clock, now
'86400' -> -4476-06-15T12:00:00.000Z '12' -> -4713-12-06T12:00:00.000Z
```
In a `Field.time` column the same `'2026'` becomes `'12:00:00'` — a wall
clock nobody wrote. Both are pinned.
## What changed
All of it inside `packages/drivers/driver-turso/src/`. **No
`packages/drivers/driver-sql/` file is touched**, and neither
`sqliteCanonicalDatetimeSql` nor `sqliteCanonicalTimeSql` moves — the
2026-08-03 cloud#1005 ruling that refused a heuristic in the shared read
expression is untouched, and #19688's pins that those expressions still
misread a bare number stay green.
- **`TursoDriver` hands its own `sqliteNonTemporalTextSql` across the
module boundary**, beside the canonical expression it already hands
over, as `{ canonical, nonTemporalText }`. The predicate is `protected`
on `SqlDriver` and stays that way; nothing is copied, so remote and
local cannot drift about which bytes are safe to overwrite.
- **The convergence `UPDATE` gains `and not (guard)`** — the local
twin's exclusion, same construction.
- **Withheld rows block the canonical mark.** This is the half that
keeps every query answer identical, and here it costs no second gate: a
withheld row is by construction `col IS NOT canonical`, so it is already
inside the probe's `residual`, and the mark already requires `residual
=== 0`. Withholding alone would not do it — a marked column drops the
read-side repair, and the raw `'2026'` would then compare as TEXT
instead of as the instant the repair reads it as.
- **The probe promises exactly what an apply rewrites.** `buildProbeSql`
gains a fourth count, `non_temporal_text` — the migration's own `WHERE`
with the guard ANDed on instead of negated — and `residual -
nonTemporalText` is what decides whether the convergence phase runs. The
remote side has no other preview surface:
`SqlDriver.previewDeferredSchemaWork` reads `deferredSchemaObjects`,
which only the Knex sync path fills, so in remote mode it answers `[]`
and this probe is the whole plan. `residual` itself is deliberately
**not** netted down, because that is what holds the mark.
- **A new per-column report field, `nonTemporalTextRowsWithheld`**, plus
an operator warning naming the consequence and the remedy. It is `null`
— NOT MEASURED — when no guard was supplied, never `0`.
- **No new gate**, no new `check:*` script, and no recommendation for
one.
### The corrected sentence
The module's contract said rows outside the epoch-recovery band "do not
block the canonical mark, because they are fixpoints of the shared
repair". That is true only **above** SQLite's julian-day ceiling, where
`strftime` returns NULL and `coalesce` hands the value back. **Below
it** — `'12'`, `'2026'`, `'86400'` — the repair answers, the row is not
a fixpoint, and dropping the repair changes what it matches. Both halves
are now written with the measurement behind them, and the field's own
contract says which of the two a given row is in. The count was **not**
filed under `unresolvedEpochTextRows`, whose documented meaning is
*recorded and harmless to the mark*; these rows are the opposite, so
they got their own name.
### Why the entry points are not breaking
The third position of `probeRemoteCanonicalColumns`,
`backfillRemoteCanonicalColumn` and `backfillRemoteCanonicalColumns` now
accepts `RemoteBackfillSqlRules` **or** the pre-existing bare
`CanonicalSqlFor`. The bare arm is **fail-closed, not a fallback**: with
no guard to withhold by, the convergence phase is refused, the column
reports `error` and stays unmarked, and reads stay correct on the
repair. So an external caller compiled against 17.4.0 keeps compiling
and still cannot write a misreading. A required parameter was considered
and rejected: this repo forbids a `major` changeset during the launch
window, and a `minor` carrying a BREAKING banner would owe an ADR-0087
disposition for which no honest category exists here
(`runtime-interface-only` needs zero in-code references,
`type-surface-only` needs an erased base-side type,
`no-migration-prescription` is closed by the FROM/TO the changeset
owes). The union keeps the fix out of that dead end.
## The three questions
**1. Does the julian misread reach the epoch-text `UPDATE`?** No, and
for two independent reasons, either sufficient.
*Structural:* its SET expression is `canonicalSqlFor(kind, 'cast(col as
real)')`. `typeof(cast(X as real))` is always `'real'`, so the canonical
expression takes its integer/real limb — `strftime('%Y-%m-%dT%H:%M:%fZ',
v/1000.0, 'unixepoch')`. The julian limb lives only in the `else` arm's
`strftime(..., col)`, which that statement can never reach, whatever the
row holds.
*Set-theoretic:* its `WHERE` selects digits-only TEXT in `1e12 <= v <
4.1024448e12`, and a bare number is read as a julian day only below
`5373484.5`. Disjoint (see Q3). ANDing the guard on there would withhold
nothing and would falsely imply the two sets meet, so the statement is
left alone and the reading is recorded beside it, with a pin.
**2. After this change, can a column still be marked canonical while
holding a row the shared repair reads as a BC date?** No — and the path
is named rather than assumed absent. `canonical` is `after.residual ===
0 && after.epochInBand === 0`. A julian-readable digits-only cell is not
equal to what the repair yields for it (`'2026'` vs
`'-4707-06-11T12:00:00.000Z'`), so `col IS NOT canonical` holds and the
row is counted in `residual`; it is withheld from the `UPDATE`, so it
stays counted; therefore `residual >= 1` and the mark cannot be granted.
The early fast-path return is gated on the same `probe.residual === 0`,
so it is not a second door.
*Control:* the same instrument reports `residual = 0` for a digits-only
value **above** the julian ceiling (`'1753660800'`, `'999999999999'`,
`REMOTE_BACKFILL_EPOCH_MS_MAX + 1`) — so the non-zero residual is about
the value, not a stuck counter. The `'huge'` test asserts that column is
still marked, and asserts `julianday()` is NULL for it, taken from
SQLite rather than claimed.
**3. Where does the epoch band sit relative to the julian range?** There
is a **gap of nearly six orders of magnitude**; the bands neither
overlap nor abut.
| | lower | upper |
|---|---|---|
| julian-day reading of a bare number | `0` | `5373484.5` (exclusive) |
| `REMOTE_BACKFILL_EPOCH_MS_MIN` .. `_MAX` | `1000000000000` |
`4102444800000` (exclusive) |
Measured, not derived: `julianday('5373484.4')` answers `5373484.4`,
`julianday('5373484.5')` is NULL, and `julianday()` is NULL at both ends
of the epoch band. The gap `5373484.5 <= v < 1e12` is digits-only text
that neither limb touches — genuine `coalesce` fixpoints, and that is
where `'1753660800'` (epoch **seconds**) lands. Consequence: the guard
cannot withhold a single row the 后果 B recovery would convert, which is
pinned end to end by a column holding both an in-band epoch-text row and
a withheld julian row.
## Verification
Run on the merged tree (`origin/main` merged, then the branch squashed
onto it with the byte-identical tree; final head as pushed).
- `pnpm --filter @objectstack/driver-turso test` — **55 files, 1291
tests, 0 failed** (1289 before; the 9 new tests, minus the rewritten
one, account for it).
- `pnpm --filter @objectstack/driver-turso typecheck` — exit 0.
- `pnpm --filter '...@objectstack/driver-turso' typecheck` — downstream
consumers, **scope 21 of 82 workspace projects**, exit 0. No in-repo
consumer of the three entry points exists outside this package
(grepped); the sweep proves the widened third position breaks none of
them.
- `pnpm exec turbo run build --filter='./packages/*'
--filter='./packages/*/*'` — 72/72, exit 0.
- `pnpm lint` (repo-wide `eslint . --no-inline-config`) — exit 0. Not
narrowed; the whole scan ran.
- **Gate families**: `node scripts/pm/dispatch-gates.mjs --commands
--repo objectstack-ai/objectstack` derived **61** commands. All 61 run
on the merged, built tree, each exit code captured before any pipe; **61
run, 0 NOT-MEASURED, 0 UNRUN**, reconciled with `--ran` carrying every
exit code (a derived zero, not a claimed one). Three families first
answered **exit 3 PREREQUISITE NOT MET** (`check:dual-build-cjs-loads`,
`check:lean-entry-closure`, `check:type-check-debt`) and were discharged
by building the closure they named, never by declaring them
inapplicable.
- `check:doc-authoring` caught a real finding on the first pass — the
refusal string carried a tracker id an operator cannot resolve. The id
moved to an adjacent comment and the gate is green.
### Ablation
One anchor, `resolveBackfillSql`'s `nonTemporalText:
sql.nonTemporalText`, replaced with a predicate that matches nothing —
the single funnel both the probe count and the `UPDATE`'s `WHERE` read,
so the mutated module behaves exactly as the pre-fix one. Driven through
`scripts/ablation-replace.mjs`, which proves the write against the disk:
```text
anchor "nonTemporalText: sql.nonTemporalText" x1 -> x0
replace "nonTemporalText: () => '(0)'" x0 -> x1
blob dcbb875 -> 1dd0ca3cfab510d38ace81d4de30bea0975e1995
```
Result: **5 failed | 24 passed (29)**. The subject is resolved from
source by a relative in-package import, so no build sits between the
mutation and the run.
Red, each for the right reason: the datetime row became
`-4707-06-11T12:00:00.000Z`; the time row became `12:00:00`; `'now'`
became the run's own wall clock; the plan's `residual` read 3 instead of
4; and the julian row in the mixed column was rewritten.
Deliberate positive controls that stayed **green**, so the suite cannot
pass vacuously: the above-the-ceiling fixpoint row is still marked
canonical; the not-over-broad case (naive, offset, canonical and junk
rows) still converges and marks; the band-disjointness measurement is
unmoved; both fail-closed cases are unmoved (they take the bare arm,
which the mutation does not reach); and every #5770 batching,
resumability, failure and identifier-safety test is unmoved.
Restore proven, not assumed: blob back to `dcbb875a6bfb`, `git diff
HEAD` empty, and whole-tree `git status --porcelain` empty.
## Changeset
`.changeset/6009-turso-remote-julian-guard.md`,
`@objectstack/driver-turso: minor`. Measured rather than assumed: the
package is not private, its `files[]` ships `dist`, and the changed
bytes reach it — `nonTemporalTextRowsWithheld` appears 7x in
`dist/index.js`, 7x in `dist/index.mjs` and 2x in `dist/index.d.ts`,
`RemoteBackfillSqlRules` 4x in the `.d.ts`, and the refusal string once
in `dist/index.js`, against a negative control that matches nothing.
`minor` rather than `patch` because new exported types and a new report
field land; no BREAKING banner, because nothing breaks.
## Acceptance notes
- `unresolvedEpochTextRows` and `nonTemporalTextRowsWithheld`
deliberately **overlap** and neither is a subset of the other: the guard
also matches shapes the digits-only count never sees (a leading `+` or
space, `'2026e0'`, a digits-only BLOB — all measured). Subtracting one
from the other to make the older field's sentence uniformly true would
have produced a count that can go negative; the fields' contracts say
which rows are in which half instead.
- The convergence phase still issues one statement that matches zero
rows when every residual row is withheld, exactly as the local twin
does. Noted, not filed: one wasted statement on a column that cannot
converge, on a path that already costs a probe.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01NcPSwnmJHczmTu6FG7NMjE)_
Co-authored-by: Claude <noreply@anthropic.com>1 parent 99c19b3 commit e2c55ed
6 files changed
Lines changed: 745 additions & 112 deletions
File tree
- .changeset
- packages/drivers/driver-turso/src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
55 | 55 | | |
56 | 56 | | |
57 | 57 | | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
58 | 61 | | |
59 | 62 | | |
60 | 63 | | |
| |||
0 commit comments