Repository navigation
Commit 5924e50
fix(driver-mongodb): two records with a null unique field are both admitted, as on driver-sql and driver-memory (#29339)
<!-- claude-projects-attribution: {"github_login":"os-steve"} -->
_Requested by **Steve** · [project
thread](https://claude.ai/code/project/chan_01FdL1XeUyvVNxLreh3F4Qi9?thread=cmsg_01FdL1XeUyvVNxLreh3F4Qi9JDHkRvsqpBwWHeNYZk2JHR)_
Fixes #29278
Clause-②: yes (widening)
Metadata-protocol: none
## 维护者速读
- **改了什么**:MongoDB 驱动的唯一索引不再把 `null` 当成一个值:两条唯一字段为空(`null` 或缺失)的记录都能保存,和
SQL、内存驱动一致;真正的重复值(含 `''`)照旧报 `E11000`。
- **为什么改**:旧索引是 `sparse`,只跳过“没有这个字段”的文档,显式 `null`
仍进索引,所以第二条清空唯一字段的记录被拒。已用真实 mongod 8.3.7 在主干上复现。
- **风险与回滚**:已部署的集合在下次 schema 同步时会把旧的 sparse
唯一索引删掉并按新形态重建(新索引约束更少,必然建成功);中间有毫秒级无唯一约束窗口,若期间写入重复值则同步失败并报错,不会静默。每个唯一索引多一个普通读索引,写入成本略增。回滚
= revert 本 PR(按源码推断、未实测:旧代码遇到同名的新形索引会在同步时报 86,需先手工删掉该索引)。
- **先看**:`packages/drivers/driver-mongodb/src/mongodb-schema.ts` 的
`nullDistinctUniqueOps` 与
`rebuildLegacySparseUnique`;`src/mongodb-unique-null-distinct.test.ts`。
- **两分钟验证**:`pnpm --filter @objectstack/driver-mongodb exec vitest run
src/mongodb-unique-null-distinct.test.ts`,默认 7 个 DDL 测试绿、8 个真实 mongod
测试跳过;加 `OS_TEST_MONGODB_MEMORY_SERVER_ENABLED=1` 且能拿到 mongod 时 15
个全绿(主干上 8 红)。
- **需要你做的**:无,席位落地。
## What changes
- **The unique index is NULL-distinct.** `syncCollectionSchema` built a
field-level `unique` and a declared `indexes[]` entry carrying `unique`
as `{ unique: true, sparse: true }`. Both are now `{ unique: true,
partialFilterExpression: { FIELD: { $type: [every non-null BSON type] }
} }`, one entry per key part, with no `sparse` (MongoDB refuses `sparse`
beside a partial filter). A document with a `null` or missing value in
any key part is not constrained; every other value is constrained
exactly as before.
- Why a type list: MongoDB refuses `$ne` / `$not` inside a partial
filter (`{ $exists: true, $ne: null }` → `CannotCreateIndex`, code 67,
measured).
- Why the full list and not the field's declared type: a stored value of
an unexpected type stays constrained, as under the sparse index.
- **A plain read index beside each unique index.** The planner cannot
use a partial index for an equality lookup (measured: `find({ code: 'X'
})` → `COLLSCAN` on the partial index, `EXPRESS_IXSCAN` on the old
sparse one; a single `$type: 'string'` filter is not used either). So
each unique index also gets `idx_FIELDS`, a plain index over the same
keys (the name a declared plain index over those columns already gets,
so they converge). A plain index already present under another name (a
lookup's `idx_FIELD_lookup`) answers `IndexOptionsConflict` (85), which
the loop already skips. An author-named unique index that already takes
the `idx_FIELDS` name gets no read index.
- **Declared `indexes[]` unique entries are in this fix, not left on the
old shape.** A single-column declaration and a field-level `unique`
converge on one index name (`idx_FIELD_unique`, pinned in
`mongodb-schema-declared-indexes.test.ts`); leaving the declared route
sparse would ask for one name with two option sets. Multi-column: a
`null` in ANY key part exempts the row, the rule
`memory-unique-constraint.ts` states for driver-sql and driver-memory.
- **Existing sparse indexes on deployed collections are rebuilt at the
next schema sync (measured).** Asking for the new options under the old
name is answered `IndexKeySpecsConflict` (86) on 8.3.7, and `collMod`
cannot change `sparse` / `partialFilterExpression`. On that conflict the
loop reads the collection's indexes; when the same-name index is exactly
the legacy shape this driver built (`unique: true`, `sparse: true`, same
keys in the same order, no partial filter) it is dropped and created
again under its own name. Measured on a real server: a collection seeded
with the legacy index, a `null` row and an `'X'` row refuses a second
`null` (`E11000`) before the sync; after `syncSchema` the index carries
the partial filter and no `sparse`, a second `null` is admitted, a
second `'X'` is still refused, and a second sync is a no-op.
- The rebuild always succeeds on existing data: the new index constrains
a subset of what the old one did.
- Between the drop and the create there is a moment with no unique
index; a non-null duplicate written in that moment makes the create fail
`E11000`, which is thrown (the sync fails loudly, the field is not left
unconstrained). The read index is created first, so lookups stay
indexed. A concurrent sync's drop of an already-dropped index
(`IndexNotFound`, 27) is ignored.
- Any other same-name index (an option this driver never wrote, or other
keys) is left alone and answers as before: 85 skipped, 86 thrown.
- **What the driver writes is unchanged.** Not changed: driver-sql,
driver-memory, `packages/spec`, any error code. A non-null duplicate is
refused with the server's own `E11000 duplicate key error` (code 11000),
as before.
## Tests
No existing test was edited. One new file,
`packages/drivers/driver-mongodb/src/mongodb-unique-null-distinct.test.ts`,
two suites:
- **Recording fake `Db` (7 tests, runs on every lane):** the field-level
index options (partial filter, no `sparse`); the read index created
before the unique one; the declared two-column filter over both parts;
the author-named collision; the legacy rebuild sequence `createIndex →
dropIndex → createIndex`; and two controls where a same-name index the
driver did not build (extra `collation`; other keys) is not dropped and
the `86` still throws.
- **Real mongod (8 tests, opt-in
`OS_TEST_MONGODB_MEMORY_SERVER_ENABLED=1`, #5517):** two `null` creates
admitted; two updates clearing to `null` admitted; two missing admitted;
a non-null duplicate refused `11000` with message starting `E11000
duplicate key error`; two `''` refused; declared `(region, tag)`: `(eu,
null)` twice admitted, `(eu, t)` twice refused; a lookup by the unique
field is not a `COLLSCAN`; the legacy-index migration.
<details><summary>Red on main, green on head (real mongod
8.3.7)</summary>
`mongodb-schema.ts` restored to `276ca307` in the tree only, the new
file run, then restored to HEAD (`git diff HEAD` empty, status clean).
The suites import `src/` directly (no `dist/` in the path).
| Reading | Result |
|---|---|
| Base `276ca307` | `Tests 8 failed, 7 passed (15)`. `admits two records
whose unique field is null`: `expected 11000 to be 'ok'`; the migration
test: `expected true to be undefined` (`sparse` still set). The 7 that
pass on base are the ones that must: missing ×2 admitted, non-null and
`''` duplicates refused, the lookup index, the author-named case, the
two leave-alone controls. |
| Head `84fb196d` | `Tests 15 passed (15)`, no suite skipped |
Premise probe on base, raw client against the same server (`{ f: 1 }`
index): sparse unique: two `null` → second `E11000`; two missing → both
`ok`; two `''`, two `'x'`, `1` and `1.0`, two equal dates, two `true` →
second `E11000`. The partial index answers the same row for every case
except two `null` (both `ok`).
</details>
<details><summary>Ablations of the negative pins</summary>
Each mutation was made with `scripts/ablation-replace.mjs` (anchor 1 →
0, blob changed, then restored with blob equal to HEAD and `git diff
HEAD` empty). No build leg: the suite reads `src/`.
| Ablation | Red |
|---|---|
| A1: the unique index built `unique: false` | 6 failed / 15, incl.
`still refuses a non-null duplicate`: `expected undefined to be 11000`,
`''` refused, `(eu, t)` refused, the migration's `'X'` refusal |
| A2: any same-name index read as the legacy shape | 2 failed / 15: both
leave-alone controls |
| A3: no read index | 3 failed / 15, incl. `serves a lookup … not a
collection scan`: `expected '{"isCached":false,"stage":"COLLSCAN",…' not
to contain 'COLLSCAN'` |
</details>
## Acceptance notes
- **The same case on all three drivers, measured** (head `84fb196d`,
built `dist`, each driver's own `syncSchema` + `create`, field `code`
`unique: 'global'`, declared `(region, tag)` unique):
| case | memory | sql (SQLite) | mongo |
|---|---|---|---|
| `code` null ×2 | ok, ok | ok, ok | ok, ok |
| `code` missing ×2 | ok, ok | ok, ok | ok, ok |
| `code` `''` ×2 | ok, refused | ok, refused | ok, refused |
| `code` `'X'` ×2 | ok, refused | ok, refused | ok, refused |
| `(eu, null)` ×2 | ok, ok | ok, ok | ok, ok |
| `(eu, t)` ×2 | ok, refused | ok, refused | ok, refused |
On base, mongo answered `ok, refused` for both `null` rows.
- **No shared conformance case-set was added.** The case-sets live in
`packages/spec/src/data/*-conformance.ts`, which this lane does not
edit. The table above is the cross-driver reading; a shared `unique`
case-set is an open question for the spec seat.
`check:driver-conformance`: 66 covered, 0 DEBT, 0 exempt, on base and
head (unchanged, as expected with no new case-set).
- **The real-server suite runs on no CI lane** (opt-in since #5517). The
7 DDL tests are the ones CI runs. This PR's real-server readings were
taken with a MongoDB 8.3.7 `mongod` from conda-forge
(`mongodb-8.3.7-hc064ac0_0.conda`, sha256 matching conda-forge's index)
supplied to `mongodb-memory-server` through `MONGOMS_SYSTEM_BINARY`,
because `fastdl.mongodb.org` is refused in the card container.
- **Write cost.** Each unique index now has a plain read index beside
it, so a write to a unique field updates two indexes instead of one.
- **`''` still collides**, as on driver-sql and driver-memory: `''` is a
value. PR #29271 (open) makes the engine store a cleared optional unique
text field as `null`; with this PR, Mongo admits those too.
## Verification
<details><summary>Suites, typecheck, lint, gates</summary>
| Run | Head | Result |
|---|---|---|
| `pnpm --filter @objectstack/driver-mongodb test` with the real mongod
| `84fb196d` | Test Files 38 passed (38) · Tests 911 passed (911), no
skip |
| the same, default lane (no opt-in) | `84fb196d` | Test Files 33
passed, 5 skipped (38) · Tests 721 passed, 190 skipped (911) |
| `pnpm --filter @objectstack/driver-mongodb typecheck` | `e38ab227` |
exit 0; `check:test-typecheck` 0 debt |
| `pnpm exec eslint --no-inline-config packages/drivers/driver-mongodb`
| `e38ab227` | exit 0 |
| `turbo run build --filter='./packages/**'` (gate prerequisite) |
`e38ab227` | VERDICT command-exit 0 |
| `dispatch-gates.mjs --commands` (60 families), each run | `e38ab227` |
60 × exit 0; `--ran`: 60 derived, 60 run, 0 NOT-MEASURED, 0 UNRUN |
One family first exited 3 (prerequisite, nothing measured):
`check-plugin-teardown-shape --self-test` needed its pinned fixture
commit fetched; after `git fetch --depth=1 origin 621a487…` it exited
0. `origin/main` moved 27 commits past the base `276ca307` while this
ran; none touch `packages/drivers` or `packages/spec/src/data`, so the
branch was not merged.
Not run locally: the full dogfood battery (no `!` in the title, no
`(narrowing)` arm).
</details>
---
_Generated by [Claude
Code](https://claude.ai/code/session_01ADHkmRc2jLWPKqPfVHStXw)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent eef116e commit 5924e50
3 files changed
Lines changed: 449 additions & 12 deletions
File tree
- .changeset
- packages/drivers/driver-mongodb/src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
10 | | - | |
| 10 | + | |
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
14 | 149 | | |
15 | 150 | | |
16 | 151 | | |
| |||
264 | 399 | | |
265 | 400 | | |
266 | 401 | | |
267 | | - | |
268 | | - | |
269 | | - | |
270 | | - | |
| 402 | + | |
| 403 | + | |
271 | 404 | | |
272 | 405 | | |
273 | 406 | | |
| |||
315 | 448 | | |
316 | 449 | | |
317 | 450 | | |
| 451 | + | |
| 452 | + | |
| 453 | + | |
| 454 | + | |
| 455 | + | |
| 456 | + | |
| 457 | + | |
| 458 | + | |
318 | 459 | | |
319 | 460 | | |
320 | | - | |
321 | | - | |
322 | | - | |
323 | | - | |
324 | | - | |
325 | | - | |
326 | | - | |
| 461 | + | |
327 | 462 | | |
328 | 463 | | |
329 | 464 | | |
330 | 465 | | |
331 | 466 | | |
332 | 467 | | |
333 | 468 | | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
| 473 | + | |
| 474 | + | |
| 475 | + | |
334 | 476 | | |
335 | 477 | | |
336 | 478 | | |
| |||
0 commit comments