From 0a3a7a585116d9db13ef799697e96c4071ebf920 Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 19:40:51 +0900 Subject: [PATCH 01/26] docs: add v2 SQLite migration design spec Design for migrating y-durableobjects from the legacy key-value Durable Object storage backend to SQLite, bundled with the bug fixes that the forced breaking change makes it possible to ship at once. Key decisions: - Single `updates` table with a `kind` column; snapshots and incremental updates are no longer distinguished. This removes the 128KiB document ceiling, the 128-key delete limit, and the key-ordering hazard by construction. - Compaction fires on a row-count threshold and on last disconnect only; no alarms. - awareness clientID ownership is persisted via serializeAttachment so it survives hibernation. - On persistence failure, close every connection and discard in-memory state; CRDT clients re-send on reconnect, so the fault self-heals. - No ORM. Drizzle's async API risks breaking implicit-transaction atomicity; Kysely compiles synchronously but earns nothing against three static queries. SQL generation is isolated in queries.ts so Kysely can be swapped in later. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015mdNeCzNPZysRHzmkFp9Bj --- .../2026-08-14-sqlite-migration-design.md | 389 ++++++++++++++++++ 1 file changed, 389 insertions(+) create mode 100644 docs/superpowers/specs/2026-08-14-sqlite-migration-design.md diff --git a/docs/superpowers/specs/2026-08-14-sqlite-migration-design.md b/docs/superpowers/specs/2026-08-14-sqlite-migration-design.md new file mode 100644 index 0000000..813af49 --- /dev/null +++ b/docs/superpowers/specs/2026-08-14-sqlite-migration-design.md @@ -0,0 +1,389 @@ +# y-durableobjects v2: SQLite バックエンドへの完全移行 + +- 日付: 2026-08-14 +- 対象バージョン: v2.0.0（破壊的変更を含むメジャーリリース） +- 現行バージョン: v1.0.5 + +## 1. 背景 + +`y-durableobjects` は Cloudflare Durable Objects の **key-value バックエンド**（`new_classes`）を前提に設計されている。Cloudflare は現在、既存の KV バックエンド namespace を持たないアカウントに対して KV バックエンドの新規作成を許可しておらず、Free プランでは SQLite バックエンドのみが利用可能である。したがって現行の v1 は **新規ユーザーがそもそもデプロイできない**状態にある。 + +加えて、KV バックエンドの制約に起因する複数の重大な不具合が存在する。 + +| ID | 内容 | 根拠 | +| --- | --- | --- | +| C-1 | `new_classes`（KV バックエンド）を前提としており、新規アカウントで利用不可 | `wrangler.toml:12`、README | +| C-2 | スナップショット全体を 1 キーに `put` するため、ドキュメントが 128KiB を超えると保存に失敗する | `src/yjs/storage/index.ts:105`、KV 値上限 128KiB | +| C-3 | コンパクション時の `delete(keys)` が 128 キー制限を超えて例外になる（既定 `maxUpdates` は 500） | `src/yjs/storage/index.ts:102` | +| C-4 | WebSocket が 1 本切断されると、部屋全体の awareness state が消える | `src/yjs/index.ts:132-134` | +| C-5 | 永続化が floating promise であり、完了保証・エラー処理・順序保証のいずれも無い | `src/yjs/index.ts:56-58` | +| H-1 | syncStep2 の応答が要求元だけでなく全接続にブロードキャストされる | `src/yjs/remote/ws-shared-doc.ts:56-58` | +| H-2 | 送信者自身に更新がエコーバックされる（origin を除外していない） | `src/yjs/remote/ws-shared-doc.ts:99-103` | +| H-3 | `getYDoc()` は生の Yjs update を返すが、`updateYDoc()` はプロトコル framing 済みメッセージを要求する。README のサンプルは動作しない | `src/yjs/index.ts:91-97`、`src/e2e/helper.ts:16` | +| H-4 | `webSocketMessage` に例外境界が無く、不正なバイナリ 1 通で DO がリセットされ全接続が落ちる | `src/yjs/index.ts:99-107` | +| H-5 | ストレージキーがゼロパディングされておらず、`list()` の復元順が更新順と一致しない | `src/yjs/storage/storage-key/index.ts:12` | +| H-7 | `queryAwareness`(3) と `auth`(2) が未実装で、`update()` の switch に `default` が無く未知の型を無言で捨てる | `src/yjs/message-type/index.ts:3-6` | + +ストレージ種別は namespace 作成後に変更できない（`storage_type_mismatch`）ため、SQLite への移行はどのみち破壊的変更になる。この一度きりの機会に上記の不具合をすべて解消する。 + +## 2. スコープ + +### 含むもの + +- KV バックエンドから SQLite バックエンドへの完全移行（KV 実装は削除する） +- C-1 〜 C-5、H-1 〜 H-5、H-7 の修正 +- WebSocket Hibernation の衛生（`setWebSocketAutoResponse`、`serializeAttachment` の実用化） +- `destroy()` RPC の追加 +- 既存ユーザー向けの移行手順の文書化と、KV バックエンド検出時の明示的エラー + +### 含まないもの + +- 認証フック（`onBeforeConnect` 等の API 追加）。v2.1 以降に回す +- KV バックエンド向けの互換実装・移行ヘルパーの同梱 +- R2 版履歴、Point-in-Time Recovery、Analytics Engine、Vectorize 連携などの新機能 +- alarm を用いた時間ベースのコンパクションやアイドル部屋のアーカイブ + +## 3. 決定事項 + +| 論点 | 決定 | 理由 | +| --- | --- | --- | +| リリーススコープ | SQLite 移行とバグ修正を v2.0.0 に一括で入れる | ストレージ種別変更が破壊的変更を強制するため、同じリリースにまとめる | +| 既存ユーザーの移行 | ライブラリは移行コードを持たない。KV バックエンドを検出したら手順書 URL 付きで throw し、README に移行レシピを載せる | H-3 修正により v1 の `getYDoc()` → v2 の `updateYDoc()` が成立し、専用コードが不要になる | +| ストレージスキーマ | 単一 `updates` テーブル。スナップショットと増分更新を区別しない | C-2 / C-3 / H-5 が設計から消える。2MB 超のチャンク分割が構造的に自然に扱える | +| コンパクション発火条件 | 行数しきい値（既定 2000 行）と全接続切断時の 2 つのみ。alarm による時間ベースの発火は使わない | 実装が単純でテストしやすい。rows read は rows written の 1/1000 の単価なので、しきい値を高く取るのが最適 | +| awareness の所有権 | `serializeAttachment` に clientID を永続化する | hibernation を跨いで所有権が保たれる。メモリ上の Map のみでは復帰後に幽霊カーソルが残る | +| 永続化失敗時の挙動 | 全接続を閉じ、メモリ上の状態も破棄する | CRDT ではクライアントが完全な状態を保持しており、再接続時に失われた更新が自己修復する | +| ORM / クエリビルダ | 採用しない。SQL は `storage/queries.ts` に隔離する | クエリが 3 本ですべて静的。Drizzle は API が非同期のため暗黙トランザクションの atomicity を壊すリスクがある | +| スキーマ版管理 | `PRAGMA user_version` ベースの自前マイグレーションランナー | 完全に同期実行でき、依存ゼロで 10 行程度に収まる | + +### ORM を採用しない判断の詳細 + +**Drizzle ORM** は `drizzle-orm/durable-sqlite` で Durable Objects を正式サポートしているが、API が Promise ベースである。本設計はコンパクション時の `DELETE` と複数の `INSERT` が `await` を挟まずに実行されることで暗黙トランザクションの atomicity を得ているため、非同期 API はこの前提を壊すリスクがある。加えて unpacked size が 10MB あり、published library の依存としては重い。 + +**Kysely** は `compile(): CompiledQuery` が同期であり、`{ sql, parameters }` を取り出して自前の `sql.exec()` に渡す「コンパイラとしてのみ使う」構成が可能なため、atomicity の問題は発生しない。unpacked size も 1.7MB と Drizzle の 1/6 である。しかし本設計のクエリは 3 本ですべて静的であり、クエリビルダの価値が発生しない。テーブルが 1 つなので、型安全性は `sql.exec()` に渡す行型を 1 つ手書きすれば実質的に同等に得られる。なお Durable Objects 向けの公式 dialect は `kysely-do@0.0.1-rc.1` のみで、published library の依存としては採用できない（上記の compile-only 構成では dialect 自体が不要）。 + +将来スキーマが増えた場合に備え、SQL 生成は `storage/queries.ts` に隔離する。Kysely への差し替えが必要になった場合、このファイルの内部だけの変更で完結する。 + +## 4. アーキテクチャ + +### ファイル構成 + +``` +src/yjs/ + index.ts YDurableObjects（SQLite 専用） + internal.ts テスト用の内部インターフェース + session/index.ts 新規: SessionRegistry（ws ↔ clientID、attachment 永続化） + storage/ + index.ts YSqliteStorage + type.ts YStorage インターフェース + schema.ts 新規: user_version マイグレーションランナー + queries.ts 新規: SQL 文字列の隔離 + storage-key/ 削除 + remote/ws-shared-doc.ts origin 対応に改修 + message-type/index.ts auth / queryAwareness を追加 + hono/index.ts 変更なし + client/setup.ts 変更なし +src/middleware/index.ts 変更なし +src/index.ts yRoute（getByName への変更のみ） +``` + +`SessionRegistry` を独立させることで、awareness の所有権管理・attachment の読み書き・hibernation 復帰時の再構築を 1 箇所に閉じ込め、`YDurableObjects` 本体の肥大化を防ぐ。単体でテスト可能な単位とする。 + +## 5. ストレージ層 + +### スキーマ + +```sql +CREATE TABLE IF NOT EXISTS updates ( + seq INTEGER PRIMARY KEY AUTOINCREMENT, + kind INTEGER NOT NULL, -- 0 = 独立した update / 1 = 直前行から続くバイト断片 + data BLOB NOT NULL +); +``` + +`kind` カラムが必要な理由: Yjs の update はバイト列として単純に分割・連結できない。`Y.mergeUpdates([a, b])` は正しく動作するが、1 本の update を機械的に分割した断片は単体では有効な update ではなく、`mergeUpdates` では復元できない。したがってコンパクション結果が SQLite の BLOB 上限を超える場合はバイト断片として分割し、読み出し時に連結してから 1 本の update として扱う必要がある。 + +### インターフェース + +```ts +export interface YStorage { + getUpdate(): Promise; + storeUpdate(update: Uint8Array): Promise; + commit(): Promise; + destroy(): Promise; +} +``` + +`getYDoc(): Promise` は廃止し、`getUpdate(): Promise` にする。呼び出し側は `applyUpdate(this.doc, update)` するだけでよく、v1 の「起動時に Doc を 2 つ構築する」無駄が解消される。 + +### 読み出し + +`SELECT kind, data FROM updates ORDER BY seq` を走査し、`kind = 1` の行は直前のバッファに連結、`kind = 0` で新しい update を開始する。最後に `Y.mergeUpdates([...])` で 1 本にまとめて返す。**Doc を一切構築しない。** + +### 書き込み + +`INSERT INTO updates (kind, data) VALUES (0, ?)` の 1 文のみ。行数はメモリ上のカウンタで保持し、起動時に `SELECT COUNT(*)` で 1 回だけ復元する。v1 の `bytes` / `count` キーへの 2 回の追加 `put` が不要になり、1 更新あたりの書き込みが 3 回から 1 回に減る。 + +### コンパクション + +`maxRows`（既定 2000）に到達したら実行する。 + +1. 全行を `SELECT` し `Y.mergeUpdates` で 1 本化する +2. `DELETE FROM updates` +3. マージ結果を `maxChunkBytes`（1MB）ごとに分割し、先頭を `kind = 0`、以降を `kind = 1` として `INSERT` +4. メモリ上の行数カウンタを分割後のチャンク数で更新する + +`sql.exec` は同期であるため、この一連の書き込みの間に `await` を挟まなければ暗黙のトランザクションとして atomic に適用される。Cloudflare のドキュメントは "Any series of write operations with no intervening `await` will automatically be submitted atomically" と明記している。したがって v1 の `TransactionStorage` 抽象および `storage.transaction()` の利用は不要となり、削除する。 + +加えて、全接続が切断されたとき（`SessionRegistry` が空になったとき）にも `commit()` を実行する。これは v1 の `cleanup()` と同じ発想を維持する。時間ベースの発火（alarm）は導入しない。 + +### コンストラクタオプション + +v1 の `{ maxBytes?, maxUpdates? }` を廃止し、`{ maxRows?, maxChunkBytes? }` に置き換える。 + +| オプション | 既定値 | 意味 | +| --- | --- | --- | +| `maxRows` | 2000 | この行数に到達したらコンパクションを実行する | +| `maxChunkBytes` | 1MB | コンパクション結果を分割する単位。SQLite の 2MB 上限に対する安全マージン | + +`maxBytes` はバイト数ベースの制御であり、KV の 128KiB 値上限に合わせるために存在していた。SQLite では行サイズがコストに影響しないため、この概念は廃止する。 + +### サイズ上限 + +SQLite の BLOB / 行の上限は 2MB のため、チャンクサイズは安全マージンを取って 1MB とする。Durable Object あたりのストレージは 10GB だが、`mergeUpdates` の結果と Yjs の `Doc` をメモリに載せる必要があるため、**実質的な上限は Durable Object の 128MB メモリ**である。この制約は README に明記する。 + +### スキーマ版管理 + +```ts +// src/yjs/storage/schema.ts +const MIGRATIONS: readonly string[] = [ + `CREATE TABLE updates ( + seq INTEGER PRIMARY KEY AUTOINCREMENT, + kind INTEGER NOT NULL, + data BLOB NOT NULL + )`, +]; + +export const migrate = (sql: SqlStorage): void => { + const version = sql.exec<{ user_version: number }>("PRAGMA user_version").one().user_version; + for (let i = version; i < MIGRATIONS.length; i++) sql.exec(MIGRATIONS[i]); + sql.exec(`PRAGMA user_version = ${MIGRATIONS.length}`); +}; +``` + +すべて同期実行のため `blockConcurrencyWhile` の中で atomic に完了する。将来スキーマを変更する場合は `MIGRATIONS` 配列に `ALTER TABLE` を追記するだけでよい。Durable Object の SQLite はインスタンスごとに独立した DB であるため、各インスタンスがそれぞれ初回起動時に自分のペースでマイグレートする。 + +## 6. Durable Object 本体 + +### 6-1. broadcast への origin の伝播（H-1 / H-2） + +`WSSharedDoc` の listener を origin 付きに変更する。 + +```ts +class WSSharedDoc extends Doc { + #listeners = new Map(); // origin(WebSocket) → listener + + notify(origin: object, listener: Listener): Unsubscribe; + + update(message: Uint8Array, origin: object): void { + // sync: readSyncMessage(decoder, encoder, this, origin) + // length(encoder) > 1 なら origin にのみ送信 + // awareness: applyAwarenessUpdate(this.awareness, payload, origin) + } +} +``` + +Yjs は `doc.on("update", (update, origin) => …)` および `awareness.on("update", (changes, origin) => …)` の第 2 引数に transaction origin を渡す。これを利用して、更新のブロードキャストから origin を除外する。 + +- syncStep2 の応答は要求元の接続にのみ返す（H-1） +- 送信者自身にはエコーバックしない（H-2） + +RPC の `updateYDoc()` 経由の更新は origin を持たないため全接続にブロードキャストされる。これは意図した挙動である。 + +### 6-2. awareness の所有権（C-4） + +`awarenessClients: Set` を廃止し、`SessionRegistry` に置き換える。 + +```ts +export type SessionAttachment = { + roomId: string; + connectedAt: number; + clientIds: number[]; +}; +``` + +clientID の特定には 6-1 で伝播させた origin をそのまま使う。 + +```ts +this.doc.awareness.on("update", ({ added, updated }, origin) => { + if (origin instanceof WebSocket) registry.track(origin, [...added, ...updated]); +}); +``` + +`applyAwarenessUpdate(awareness, payload, ws)` に渡した origin が awareness の update イベントまで伝播するため、awareness メッセージをデコードして clientID を抽出する処理は不要である。 + +切断時は該当接続の clientIds のみを `removeAwarenessStates` に渡す。 + +```ts +async webSocketClose(ws: WebSocket) { + removeAwarenessStates(this.doc.awareness, registry.clientIdsOf(ws), null); + registry.remove(ws); + await this.maybeCommit(); +} +``` + +hibernation からの復帰時は `onStart` で `state.getWebSockets()` を走査し、各 WebSocket の `deserializeAttachment()` から `SessionRegistry` を再構築する。v1 で書き込まれていながら一度も読まれていなかった `WebSocketAttachment` が、ここで実際に機能する。 + +### 6-3. 永続化の直列化（C-5） + +```ts +#persist: Promise = Promise.resolve(); + +#schedulePersist(update: Uint8Array): void { + this.#persist = this.#persist + .then(() => this.storage.storeUpdate(update)) + .catch((e) => this.#onPersistFailure(e)); + this.state.waitUntil(this.#persist); +} +``` + +`doc.on("update")` からこれを呼び、`webSocketMessage` の末尾で `await this.#persist` する。宙に浮いた Promise が無くなり、行数カウンタの競合も直列化によって解消する。 + +### 6-4. 永続化失敗時の挙動 + +`#onPersistFailure` は以下を行う。 + +1. エラーをログに出力する +2. `state.getWebSockets()` の全接続を close code `1011`（Internal Error）で閉じる +3. メモリ上の Doc の状態を破棄する + +3 が必要な理由: 書き込み失敗時点でメモリ上の `this.doc` はストレージより進んでおり、DO 自体は生存し続ける。接続を閉じるだけでは、その後に接続したクライアントが「正常に見える」メモリ上の状態を受け取り、DO が evict された時点で差分が無言で失われる。`state.abort()` によって Durable Object をリセットし、次回起動時にストレージから読み直させる方針とする（`state.abort()` の実挙動は実装時に workerd 上で検証する）。 + +この選択が妥当な理由: CRDT ではクライアント側が完全な状態を保持している。`1011` での切断後、y-websocket クライアントは自動再接続し、syncStep1 / syncStep2 の過程で失われた更新を再送するため、障害が自己修復する。 + +### 6-5. 例外境界（H-4） + +```ts +async webSocketMessage(ws: WebSocket, message: string | ArrayBuffer) { + if (!(message instanceof ArrayBuffer)) return; + try { + this.doc.update(new Uint8Array(message), ws); + } catch (e) { + console.error("[y-durableobjects] invalid message", e); + ws.close(1003, "invalid message"); + return; + } + await this.#persist; +} +``` + +不正なバイナリを送った接続のみを閉じ、Durable Object と他の接続は影響を受けない。 + +### 6-6. Hibernation の衛生 + +```ts +this.state.setWebSocketAutoResponse(new WebSocketRequestResponsePair("ping", "pong")); +``` + +文字列メッセージは `webSocketMessage` の先頭で無視されているため、既存の挙動と衝突しない。README に「クライアントが `"ping"` を定期送信すれば Durable Object を起こさずに keepalive できる」ことを明記する。duration 課金への影響が最も大きい項目である。 + +`acceptWebSocket()` のタグは付けない。clientID は接続確立時点では未確定であり、タグは `acceptWebSocket` 呼び出し時に確定している必要があるため相性が悪い。 + +## 7. 公開 API と移行 + +### RPC API + +```ts +getYDoc(): Promise // 生の Yjs update（v1 から変更なし） +updateYDoc(update: Uint8Array): Promise // 生の Yjs update を受け取る（破壊的変更） +destroy(): Promise // 新規 +``` + +`updateYDoc` からプロトコル framing の要求を外し、内部で `applyUpdate(this.doc, update)` する（H-3）。これにより `getYDoc()` の出力をそのまま `updateYDoc()` に渡せるようになり、README のサンプルが実際に動作する。 + +`destroy()` は全接続を close code `1001` で閉じ、`DELETE FROM updates` を実行する。テーブル定義と `user_version` は維持する。 + +### エクスポートされる型の変更 + +`src/index.ts` から公開している型のうち、以下が破壊的に変更される。 + +| v1 | v2 | 備考 | +| --- | --- | --- | +| `YTransactionStorage` | `YStorage` | `getYDoc(): Promise` が `getUpdate(): Promise` になり、`destroy()` が追加される | +| `WSSharedDoc` | `WSSharedDoc` | `notify(cb)` が `notify(origin, cb)` に、`update(message)` が `update(message, origin)` になる | +| `RemoteDoc` | `RemoteDoc` | 変更なし | + +`TransactionStorage`（`storage/type.ts`）は非公開だったため、削除しても公開 API には影響しない。 + +### KV バックエンドの検出 + +コンストラクタの `blockConcurrencyWhile` 内で SQLite バックエンドかを確認し、そうでなければ移行手順の URL を含むエラーを throw する。`"sql" in storage` による判定は確実でない可能性があるため、`sql.exec("SELECT 1")` を実際に試行する方式とし、workerd 上の実挙動は実装時に検証する。 + +### 移行レシピ（README に掲載） + +v1 と v2 を別バインディングに共存させ、RPC で内容をコピーする。 + +```ts +app.post("/migrate/:id", async (c) => { + const id = c.req.param("id"); + const legacy = c.env.Y_LEGACY.getByName(id); // v1 / KV バックエンド + const next = c.env.Y_DURABLE_OBJECTS.getByName(id); // v2 / SQLite バックエンド + await next.updateYDoc(await legacy.getYDoc()); + return c.json({ ok: true }); +}); +``` + +v1 の `getYDoc()` は元から生の update を返しており、v2 の `updateYDoc()` は生の update を受け取るため、この組み合わせが成立する。ライブラリ側に移行専用コードを持つ必要がない。 + +### 設定ファイル + +- `wrangler.toml` および README の migration を `new_sqlite_classes` に変更する +- `compatibility_date` を更新する +- README では新しい `exports` 形式にも言及する + +### メッセージ型（H-7） + +`message-type/index.ts` に `auth`(2) と `queryAwareness`(3) を追加し、`WSSharedDoc.update()` の switch に `default` 節を設けて未知の型を無言で捨てないようにする。 + +### `yRoute` + +`obj.get(obj.idFromName(id))` を `obj.getByName(id)` に置き換える。それ以外の挙動は変更しない。 + +## 8. テスト戦略 + +実装は TDD で進める。既知の不具合については、先に失敗するテストを書いてから修正する。 + +`wrangler.toml` のテスト環境も `new_sqlite_classes` に切り替える。現行の `src/yjs/storage/storage.test.ts` は `TransactionStorage` をモックする方式だが、v2 ではテスト対象が SQLite の挙動そのものになるため、`runInDurableObject` 経由で実物の `ctx.storage.sql` を使う統合テストに書き換える。 + +| # | 検証内容 | 対応 ID | +| --- | --- | --- | +| 1 | 2MB を超えるドキュメントがチャンク分割され、round-trip で復元できる | C-2 | +| 2 | しきい値到達でコンパクションが走り、行数が減って内容が保たれる | C-3 | +| 3 | 3 接続のうち 1 つを切断しても、他 2 接続の awareness state が残る | C-4 | +| 4 | 永続化失敗時に全接続が閉じ、メモリ上の状態が破棄される | 6-4 | +| 5 | syncStep2 が要求元にのみ返り、他の接続には届かない | H-1 | +| 6 | 送信者に自分の更新がエコーバックされない | H-2 | +| 7 | `getYDoc()` → `updateYDoc()` の round-trip が成立する | H-3 | +| 8 | 不正なバイナリでその接続のみ閉じ、DO と他接続は生存する | H-4 | +| 9 | 1000 件の更新が `seq` 順に復元される | H-5 | +| 10 | `deserializeAttachment` から awareness の所有権が復元される | 6-2 | +| 11 | `user_version` マイグレーションが冪等である | 5 | + +### テスト上の制約 + +`@cloudflare/vitest-pool-workers` には hibernation を強制的に発生させる API が存在しない。したがって #10 は「新しい `SessionRegistry` を生成し、既存の WebSocket の attachment から所有権を再構築できること」をユニットレベルで検証する形になり、実際の hibernation 往復は E2E では検証できない。 + +## 9. 既知の限界 + +- ドキュメントサイズの実質的な上限は Durable Object の 128MB メモリである。SQLite の 10GB は活用しきれない +- 実際の hibernation 往復を自動テストで検証できない（8 節参照） +- KV バックエンドで稼働中の既存ユーザーは、手動でのデータ移行が必要である +- `state.abort()` の実挙動は未検証であり、実装時に確認が必要である +- `sql.exec("SELECT 1")` による KV バックエンド検出の確実性は未検証であり、実装時に確認が必要である + +## 10. 参考 + +- [Durable Objects Pricing](https://developers.cloudflare.com/durable-objects/platform/pricing/) +- [Durable Objects Limits](https://developers.cloudflare.com/durable-objects/platform/limits/) +- [Durable Object Storage API](https://developers.cloudflare.com/durable-objects/api/storage-api/) +- [Durable Objects Migrations](https://developers.cloudflare.com/durable-objects/reference/durable-objects-migrations/) From 1a86aae72728873598108a8de9c861df5c5fe84b Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 19:45:18 +0900 Subject: [PATCH 02/26] docs: correct spec after probing DO SQLite on real workerd PRAGMA is rejected on Durable Object SQLite (SQLITE_AUTH), so the user_version migration runner in the spec would not have worked. Replaced it with a schema_version table. Also recorded the other probed behaviours: BLOB columns come back as ArrayBuffer (conversion required on read), both Uint8Array and ArrayBuffer bind fine, AUTOINCREMENT is allowed, and all 41 existing tests pass unchanged on the SQLite backend because the KV API is transparently backed by __cf_kv. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015mdNeCzNPZysRHzmkFp9Bj --- .../2026-08-14-sqlite-migration-design.md | 36 ++++++++++++++++--- 1 file changed, 31 insertions(+), 5 deletions(-) diff --git a/docs/superpowers/specs/2026-08-14-sqlite-migration-design.md b/docs/superpowers/specs/2026-08-14-sqlite-migration-design.md index 813af49..6544853 100644 --- a/docs/superpowers/specs/2026-08-14-sqlite-migration-design.md +++ b/docs/superpowers/specs/2026-08-14-sqlite-migration-design.md @@ -54,7 +54,7 @@ | awareness の所有権 | `serializeAttachment` に clientID を永続化する | hibernation を跨いで所有権が保たれる。メモリ上の Map のみでは復帰後に幽霊カーソルが残る | | 永続化失敗時の挙動 | 全接続を閉じ、メモリ上の状態も破棄する | CRDT ではクライアントが完全な状態を保持しており、再接続時に失われた更新が自己修復する | | ORM / クエリビルダ | 採用しない。SQL は `storage/queries.ts` に隔離する | クエリが 3 本ですべて静的。Drizzle は API が非同期のため暗黙トランザクションの atomicity を壊すリスクがある | -| スキーマ版管理 | `PRAGMA user_version` ベースの自前マイグレーションランナー | 完全に同期実行でき、依存ゼロで 10 行程度に収まる | +| スキーマ版管理 | `schema_version` テーブルベースの自前マイグレーションランナー | 完全に同期実行でき、依存ゼロで済む。`PRAGMA` は Durable Objects の SQLite では拒否される（下記の検証結果を参照） | ### ORM を採用しない判断の詳細 @@ -120,6 +120,8 @@ export interface YStorage { `SELECT kind, data FROM updates ORDER BY seq` を走査し、`kind = 1` の行は直前のバッファに連結、`kind = 0` で新しい update を開始する。最後に `Y.mergeUpdates([...])` で 1 本にまとめて返す。**Doc を一切構築しない。** +BLOB カラムは `ArrayBuffer` として返るため、各行で `new Uint8Array(row.data)` への変換が必要である。 + ### 書き込み `INSERT INTO updates (kind, data) VALUES (0, ?)` の 1 文のみ。行数はメモリ上のカウンタで保持し、起動時に `SELECT COUNT(*)` で 1 回だけ復元する。v1 の `bytes` / `count` キーへの 2 回の追加 `put` が不要になり、1 更新あたりの書き込みが 3 回から 1 回に減る。 @@ -154,6 +156,8 @@ SQLite の BLOB / 行の上限は 2MB のため、チャンクサイズは安全 ### スキーマ版管理 +版番号は `schema_version` テーブルに保持する。`PRAGMA` は使えない（後述の検証結果を参照）。 + ```ts // src/yjs/storage/schema.ts const MIGRATIONS: readonly string[] = [ @@ -165,14 +169,36 @@ const MIGRATIONS: readonly string[] = [ ]; export const migrate = (sql: SqlStorage): void => { - const version = sql.exec<{ user_version: number }>("PRAGMA user_version").one().user_version; - for (let i = version; i < MIGRATIONS.length; i++) sql.exec(MIGRATIONS[i]); - sql.exec(`PRAGMA user_version = ${MIGRATIONS.length}`); + sql.exec("CREATE TABLE IF NOT EXISTS schema_version (version INTEGER NOT NULL)"); + const row = sql.exec<{ version: number }>("SELECT version FROM schema_version").toArray().at(0); + const current = row?.version ?? 0; + for (let i = current; i < MIGRATIONS.length; i++) sql.exec(MIGRATIONS[i]); + if (row === undefined) { + sql.exec("INSERT INTO schema_version (version) VALUES (?)", MIGRATIONS.length); + } else { + sql.exec("UPDATE schema_version SET version = ?", MIGRATIONS.length); + } }; ``` すべて同期実行のため `blockConcurrencyWhile` の中で atomic に完了する。将来スキーマを変更する場合は `MIGRATIONS` 配列に `ALTER TABLE` を追記するだけでよい。Durable Object の SQLite はインスタンスごとに独立した DB であるため、各インスタンスがそれぞれ初回起動時に自分のペースでマイグレートする。 +### 実機で検証済みの挙動 + +設計の前提となる以下の項目を、本リポジトリの `@cloudflare/vitest-pool-workers` 環境（`new_sqlite_classes` に切り替えた状態）で実測した。 + +| 項目 | 結果 | +| --- | --- | +| `PRAGMA user_version` の読み書き | **不可**。`Error: not authorized: SQLITE_AUTH` で throw する | +| `CREATE TABLE ... INTEGER PRIMARY KEY AUTOINCREMENT` | 可 | +| `sql.exec` への `Uint8Array` バインド | 可 | +| `sql.exec` への `ArrayBuffer` バインド | 可 | +| BLOB カラムの戻り値の型 | **`ArrayBuffer`**。`Uint8Array` ではないため読み出し時に変換が必要 | +| `sqlite_master` の SELECT | 可 | +| 既存テスト 41 件を SQLite バックエンドで実行 | 全件 pass。KV API が `__cf_kv` 経由で透過的に動作するため、バックエンド切り替え単体ではコード変更を要しない | + +BLOB が `ArrayBuffer` で返る点は重要である。読み出し時は `new Uint8Array(row.data)` による変換が必須となる。逆に書き込み時は、lib0 のエンコーダが返す `Uint8Array` が大きなバッファへのビューであることが多いため、`u8.buffer` をそのまま渡してはならない。`Uint8Array` を直接バインドできることが確認できたので、変換せずそのまま渡す方針とする。 + ## 6. Durable Object 本体 ### 6-1. broadcast への origin の伝播（H-1 / H-2） @@ -367,7 +393,7 @@ v1 の `getYDoc()` は元から生の update を返しており、v2 の `update | 8 | 不正なバイナリでその接続のみ閉じ、DO と他接続は生存する | H-4 | | 9 | 1000 件の更新が `seq` 順に復元される | H-5 | | 10 | `deserializeAttachment` から awareness の所有権が復元される | 6-2 | -| 11 | `user_version` マイグレーションが冪等である | 5 | +| 11 | `schema_version` マイグレーションが冪等である（2 回実行しても壊れない） | 5 | ### テスト上の制約 From b8e918c5e227a99f07732bdbc51a7e720c590f12 Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 19:52:54 +0900 Subject: [PATCH 03/26] docs: add v2 SQLite migration implementation plan Ten tasks, seventy TDD steps, each ending on a green suite and a commit. Ordered so the backend switch lands first (verified to pass unchanged because the KV API is backed by __cf_kv), then the new storage layer is built alongside the old one, then the Durable Object swaps over and the key-value implementation is deleted. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015mdNeCzNPZysRHzmkFp9Bj --- .../plans/2026-08-14-sqlite-migration-v2.md | 2024 +++++++++++++++++ 1 file changed, 2024 insertions(+) create mode 100644 docs/superpowers/plans/2026-08-14-sqlite-migration-v2.md diff --git a/docs/superpowers/plans/2026-08-14-sqlite-migration-v2.md b/docs/superpowers/plans/2026-08-14-sqlite-migration-v2.md new file mode 100644 index 0000000..0387f39 --- /dev/null +++ b/docs/superpowers/plans/2026-08-14-sqlite-migration-v2.md @@ -0,0 +1,2024 @@ +# y-durableobjects v2 SQLite Migration Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** `y-durableobjects` を Durable Objects の legacy key-value ストレージバックエンドから SQLite バックエンドへ完全移行し、その破壊的変更に便乗して既知の重大な不具合（C-1〜C-5 / H-1〜H-5 / H-7）をすべて解消する。 + +**Architecture:** 単一の `updates` テーブルにすべてを保存する。スナップショットと増分更新を区別せず、コンパクションは「全行を `Y.mergeUpdates` で 1 本にして書き戻す」だけの操作にする。1 本化した結果が SQLite の BLOB 上限を超える場合はバイト断片に分割し、`kind` カラムで断片であることを示す。Yjs の transaction origin を WebSocket まで通すことで、ブロードキャストの除外と awareness 所有権の追跡を同一の仕組みで解決する。 + +**Tech Stack:** TypeScript (strict), Cloudflare Durable Objects (SQLite backend), Yjs / y-protocols / lib0, Hono, Vitest + `@cloudflare/vitest-pool-workers`, tsup, ESLint + Prettier, changesets + +**Spec:** `docs/superpowers/specs/2026-08-14-sqlite-migration-design.md` + +## Global Constraints + +- ブランチは `feat/sqlite-migration-v2`。spec は既にこのブランチにコミット済み。 +- テスト実行は `pnpm exec vitest run `。`pnpm test` は全件実行。`mise` 管理のため `pnpm` が見つからない場合は `export PATH="$HOME/.local/share/mise/shims:$PATH"` を先に実行する。 +- コミット前に必ず `pnpm fmt`（Prettier + ESLint --fix）と `pnpm typecheck` を通す。 +- ESLint: `no-console` は `error`。ログ出力が必要な箇所は既存コードと同様に `// eslint-disable-next-line no-console` を直前に置く。`newline-before-return` が `error` なので `return` の前に空行を入れる。 +- TypeScript strict。`any` は使用しない。`ws.deserializeAttachment()` は `any` を返すので `const raw: unknown = ws.deserializeAttachment();` のように `unknown` で受ける。 +- ファイル名は kebab-case。import 順序は ESLint が強制する（`pnpm fmt` で自動整列）。 +- **`PRAGMA` は Durable Objects の SQLite では使用不可**（`Error: not authorized: SQLITE_AUTH`）。スキーマ版管理は `schema_version` テーブルで行う。 +- **BLOB カラムは `ArrayBuffer` として返る。** 読み出し時は `new Uint8Array(row.data)` で変換する。書き込み時は `Uint8Array` をそのままバインドできる。 +- `sql.exec()` は `BEGIN TRANSACTION` / `SAVEPOINT` を実行できない。`await` を挟まない連続した書き込みが暗黙のトランザクションとして atomic に適用されることを利用する。**コンパクションの `DELETE` と `INSERT` の間に `await` を入れてはならない。** +- 確認済みの型定義（`worker-configuration.d.ts`）: + - `DurableObjectState.abort(reason?: string): void` + - `DurableObjectState.setWebSocketAutoResponse(maybeReqResp?: WebSocketRequestResponsePair): void` + - `new WebSocketRequestResponsePair(request: string, response: string)` + - `SqlStorage.exec>(query: string, ...bindings: any[]): SqlStorageCursor` + - `type SqlStorageValue = ArrayBuffer | string | number | null` + - カーソルは `toArray()` / `one()` / `next()` / `raw()` を持つ + +--- + +### Task 1: SQLite バックエンドへの切り替えとスキーマ管理 + +KV バックエンドから SQLite バックエンドへ設定を切り替え、`schema_version` テーブルによるマイグレーションランナーを追加する。KV API は SQLite バックエンド上でも隠しテーブル `__cf_kv` 経由で透過的に動作するため、既存コードは無変更で全テストが通る（検証済み）。 + +**Files:** +- Modify: `wrangler.toml:3`（`compatibility_date`）, `wrangler.toml:12`（`new_classes` → `new_sqlite_classes`） +- Create: `src/yjs/storage/schema.ts` +- Test: `src/yjs/storage/schema.test.ts` + +**Interfaces:** +- Consumes: なし +- Produces: `migrate(sql: SqlStorage): void` — `updates` テーブルと `schema_version` テーブルを作成し、冪等に実行できる + +- [ ] **Step 1: wrangler.toml を SQLite バックエンドに切り替える** + +`wrangler.toml` を以下の内容にする。 + +```toml +name = "yjs-workers" +main = "src/e2e/index.ts" +compatibility_date = "2025-04-01" +compatibility_flags=["nodejs_compat"] + +[[durable_objects.bindings]] +name = "Y_DURABLE_OBJECTS" +class_name = "YDurableObjects" + +[[migrations]] +tag = "v1" +new_sqlite_classes = ["YDurableObjects"] +``` + +- [ ] **Step 2: 既存テストが SQLite バックエンドでも全件通ることを確認する** + +Run: `pnpm exec vitest run` +Expected: PASS（41 tests）。`create-app.test.ts` の stderr に `Error: Service Error` が出るが、これは意図的なエラー系テストの出力であり失敗ではない。 + +- [ ] **Step 3: schema.test.ts に失敗するテストを書く** + +```ts +// src/yjs/storage/schema.test.ts +import { env, runInDurableObject } from "cloudflare:test"; +import { describe, expect, it } from "vitest"; + +import { migrate } from "./schema"; + +const withSql = async (fn: (sql: SqlStorage) => void): Promise => { + const stub = env.Y_DURABLE_OBJECTS.get(env.Y_DURABLE_OBJECTS.newUniqueId()); + await runInDurableObject(stub, (_instance, state) => { + fn(state.storage.sql); + }); +}; + +describe("migrate", () => { + it("creates the updates table", async () => { + await withSql((sql) => { + migrate(sql); + + const tables = sql + .exec<{ name: string }>("SELECT name FROM sqlite_master WHERE type = 'table'") + .toArray() + .map((row) => row.name); + + expect(tables).toContain("updates"); + expect(tables).toContain("schema_version"); + }); + }); + + it("records the schema version", async () => { + await withSql((sql) => { + migrate(sql); + + const row = sql.exec<{ version: number }>("SELECT version FROM schema_version").one(); + + expect(row.version).toBe(1); + }); + }); + + it("is idempotent and preserves existing rows", async () => { + await withSql((sql) => { + migrate(sql); + sql.exec("INSERT INTO updates (kind, data) VALUES (?, ?)", 0, new Uint8Array([1, 2, 3])); + + migrate(sql); + + const count = sql.exec<{ count: number }>("SELECT COUNT(*) AS count FROM updates").one(); + const version = sql.exec<{ version: number }>("SELECT version FROM schema_version").one(); + + expect(count.count).toBe(1); + expect(version.version).toBe(1); + }); + }); +}); +``` + +- [ ] **Step 4: テストが失敗することを確認する** + +Run: `pnpm exec vitest run src/yjs/storage/schema.test.ts` +Expected: FAIL — `Failed to resolve import "./schema"` + +- [ ] **Step 5: schema.ts を実装する** + +```ts +// src/yjs/storage/schema.ts + +/** + * スキーマのマイグレーション定義。 + * 変更するときは既存の要素を書き換えず、末尾に ALTER TABLE を追記すること。 + * Durable Object の SQLite はインスタンスごとに独立した DB なので、 + * 各インスタンスが初回起動時に自分のペースでマイグレートする。 + */ +const MIGRATIONS: readonly string[] = [ + `CREATE TABLE updates ( + seq INTEGER PRIMARY KEY AUTOINCREMENT, + kind INTEGER NOT NULL, + data BLOB NOT NULL + )`, +]; + +/** + * 未適用のマイグレーションを適用する。すべて同期実行なので、 + * 呼び出し中に await を挟まなければ暗黙のトランザクションとして atomic に完了する。 + * + * PRAGMA user_version は Durable Objects の SQLite では SQLITE_AUTH で拒否されるため、 + * 版番号は schema_version テーブルに保持する。 + */ +export const migrate = (sql: SqlStorage): void => { + sql.exec("CREATE TABLE IF NOT EXISTS schema_version (version INTEGER NOT NULL)"); + + const current = sql.exec<{ version: number }>("SELECT version FROM schema_version").toArray().at(0); + const applied = current?.version ?? 0; + + for (let i = applied; i < MIGRATIONS.length; i++) { + sql.exec(MIGRATIONS[i]); + } + + if (current === undefined) { + sql.exec("INSERT INTO schema_version (version) VALUES (?)", MIGRATIONS.length); + } else if (applied !== MIGRATIONS.length) { + sql.exec("UPDATE schema_version SET version = ?", MIGRATIONS.length); + } +}; +``` + +- [ ] **Step 6: テストが通ることを確認する** + +Run: `pnpm exec vitest run src/yjs/storage/schema.test.ts` +Expected: PASS（3 tests） + +- [ ] **Step 7: 全体を検証してコミットする** + +```bash +pnpm fmt && pnpm typecheck && pnpm exec vitest run +git add wrangler.toml src/yjs/storage/schema.ts src/yjs/storage/schema.test.ts +git commit -m "feat: switch to SQLite storage backend and add schema migration runner" +``` + +--- + +### Task 2: SQL 定義と YSqliteStorage の読み書き + +`updates` テーブルへの読み書きを実装する。コンパクションは Task 3 で追加する。既存の KV 実装には触れないため、この時点では両方の実装が並存する。 + +**Files:** +- Create: `src/yjs/storage/queries.ts` +- Modify: `src/yjs/storage/type.ts`（`YStorage` と `UpdateKind` を追加。既存の `TransactionStorage` は Task 4 まで残す） +- Create: `src/yjs/storage/sqlite.ts` +- Test: `src/yjs/storage/sqlite.test.ts` + +**Interfaces:** +- Consumes: `migrate(sql: SqlStorage): void`（Task 1） +- Produces: + - `const UpdateKind: { readonly standalone: 0; readonly continuation: 1 }` + - `interface YStorage { getUpdate(): Promise; storeUpdate(update: Uint8Array): Promise; commit(): Promise; destroy(): Promise; }` + - `type YSqliteStorageOptions = { maxRows?: number; maxChunkBytes?: number }` + - `class YSqliteStorage implements YStorage`、コンストラクタは `(sql: SqlStorage, options?: YSqliteStorageOptions)` + +- [ ] **Step 1: sqlite.test.ts に失敗するテストを書く** + +```ts +// src/yjs/storage/sqlite.test.ts +import { env, runInDurableObject } from "cloudflare:test"; +import { describe, expect, it } from "vitest"; +import { Doc, applyUpdate, encodeStateAsUpdate } from "yjs"; + +import { YSqliteStorage } from "./sqlite"; + +import type { YSqliteStorageOptions } from "./sqlite"; + +const withStorage = async ( + fn: (storage: YSqliteStorage) => Promise, + options?: YSqliteStorageOptions, +): Promise => { + const stub = env.Y_DURABLE_OBJECTS.get(env.Y_DURABLE_OBJECTS.newUniqueId()); + await runInDurableObject(stub, async (_instance, state) => { + await fn(new YSqliteStorage(state.storage.sql, options)); + }); +}; + +const docWith = (text: string): Doc => { + const doc = new Doc(); + doc.getText("root").insert(0, text); + + return doc; +}; + +const textOf = (update: Uint8Array): string => { + const doc = new Doc(); + applyUpdate(doc, update); + + return doc.getText("root").toString(); +}; + +describe("YSqliteStorage", () => { + it("returns null when nothing has been stored", async () => { + await withStorage(async (storage) => { + expect(await storage.getUpdate()).toBeNull(); + }); + }); + + it("round-trips a single update", async () => { + await withStorage(async (storage) => { + await storage.storeUpdate(encodeStateAsUpdate(docWith("Hello World!"))); + + const update = await storage.getUpdate(); + + expect(update).not.toBeNull(); + expect(textOf(update!)).toBe("Hello World!"); + }); + }); + + it("applies stored updates in insertion order", async () => { + await withStorage(async (storage) => { + const doc = new Doc(); + const text = doc.getText("root"); + + // 1000 件の更新を個別に保存する。キーの辞書順ではなく seq 順で + // 復元されることを検証する（H-5 の回帰テスト）。 + const updates: Uint8Array[] = []; + doc.on("update", (update: Uint8Array) => updates.push(update)); + for (let i = 0; i < 1000; i++) { + text.insert(text.length, String(i % 10)); + } + for (const update of updates) { + await storage.storeUpdate(update); + } + + const restored = await storage.getUpdate(); + + expect(textOf(restored!)).toBe(text.toString()); + }); + }); + + it("clears all rows on destroy", async () => { + await withStorage(async (storage) => { + await storage.storeUpdate(encodeStateAsUpdate(docWith("gone"))); + await storage.destroy(); + + expect(await storage.getUpdate()).toBeNull(); + }); + }); +}); +``` + +- [ ] **Step 2: テストが失敗することを確認する** + +Run: `pnpm exec vitest run src/yjs/storage/sqlite.test.ts` +Expected: FAIL — `Failed to resolve import "./sqlite"` + +- [ ] **Step 3: queries.ts を作成する** + +```ts +// src/yjs/storage/queries.ts + +/** + * SQL 文字列をこのファイルに隔離する。 + * 将来スキーマが複雑になり Kysely 等のクエリビルダを導入する場合も、 + * 変更はこのファイル内で完結する。 + */ +export const SELECT_ALL_UPDATES = "SELECT kind, data FROM updates ORDER BY seq"; +export const INSERT_UPDATE = "INSERT INTO updates (kind, data) VALUES (?, ?)"; +export const DELETE_ALL_UPDATES = "DELETE FROM updates"; +export const COUNT_UPDATES = "SELECT COUNT(*) AS count FROM updates"; +``` + +- [ ] **Step 4: type.ts に YStorage と UpdateKind を追加する** + +`src/yjs/storage/type.ts` の既存の `TransactionStorage` はそのまま残し、末尾に以下を追記する。 + +```ts +/** + * updates テーブルの kind カラムの値。 + * + * Yjs の update はバイト列として単純に分割・連結できないため、 + * コンパクション結果が BLOB 上限を超える場合はバイト断片に分割して保存する。 + * continuation は「直前の行から続く断片」であることを示す。 + */ +export const UpdateKind = { + standalone: 0, + continuation: 1, +} as const; + +export interface YStorage { + /** 保存されているすべての update を 1 本にマージして返す。空なら null */ + getUpdate(): Promise; + /** 増分 update を 1 行追加する。しきい値を超えたらコンパクションする */ + storeUpdate(update: Uint8Array): Promise; + /** 明示的にコンパクションする */ + commit(): Promise; + /** すべての update を削除する。テーブル定義とスキーマ版は維持する */ + destroy(): Promise; +} +``` + +- [ ] **Step 5: sqlite.ts を実装する（コンパクションは Task 3 で追加）** + +```ts +// src/yjs/storage/sqlite.ts +import { mergeUpdates } from "yjs"; + +import { COUNT_UPDATES, DELETE_ALL_UPDATES, INSERT_UPDATE, SELECT_ALL_UPDATES } from "./queries"; +import { migrate } from "./schema"; +import { UpdateKind } from "./type"; + +import type { YStorage } from "./type"; + +type UpdateRow = { + kind: number; + data: ArrayBuffer; +}; + +export type YSqliteStorageOptions = { + /** + * この行数を超えたらコンパクションする。 + * @default 2000 + */ + maxRows?: number; + /** + * コンパクション結果を分割する単位（バイト）。 + * SQLite の BLOB 上限 2MB に対する安全マージンを取る。 + * @default 1024 * 1024 + */ + maxChunkBytes?: number; +}; + +const DEFAULT_MAX_ROWS = 2000; +const DEFAULT_MAX_CHUNK_BYTES = 1024 * 1024; +const SQLITE_BLOB_LIMIT = 2 * 1024 * 1024; + +const concat = (parts: readonly Uint8Array[]): Uint8Array => { + if (parts.length === 1) return parts[0]; + + const total = parts.reduce((sum, part) => sum + part.byteLength, 0); + const merged = new Uint8Array(total); + let offset = 0; + for (const part of parts) { + merged.set(part, offset); + offset += part.byteLength; + } + + return merged; +}; + +export class YSqliteStorage implements YStorage { + readonly #sql: SqlStorage; + readonly #maxRows: number; + readonly #maxChunkBytes: number; + #rowCount: number; + + constructor(sql: SqlStorage, options?: YSqliteStorageOptions) { + this.#maxChunkBytes = options?.maxChunkBytes ?? DEFAULT_MAX_CHUNK_BYTES; + if (this.#maxChunkBytes > SQLITE_BLOB_LIMIT) { + // https://developers.cloudflare.com/durable-objects/platform/limits/ + throw new Error("maxChunkBytes must not exceed 2MB"); + } + this.#maxRows = options?.maxRows ?? DEFAULT_MAX_ROWS; + + this.#sql = sql; + migrate(sql); + this.#rowCount = sql.exec<{ count: number }>(COUNT_UPDATES).one().count; + } + + async getUpdate(): Promise { + const updates = this.#readAll(); + if (updates.length === 0) return null; + + return mergeUpdates(updates); + } + + async storeUpdate(update: Uint8Array): Promise { + this.#sql.exec(INSERT_UPDATE, UpdateKind.standalone, update); + this.#rowCount += 1; + } + + async commit(): Promise { + // Task 3 で実装する + } + + async destroy(): Promise { + this.#sql.exec(DELETE_ALL_UPDATES); + this.#rowCount = 0; + } + + /** + * 全行を読み、continuation の断片を連結して独立した update の配列に戻す。 + * BLOB は ArrayBuffer で返るため Uint8Array へ変換する。 + */ + #readAll(): Uint8Array[] { + const rows = this.#sql.exec(SELECT_ALL_UPDATES).toArray(); + + const updates: Uint8Array[] = []; + let pending: Uint8Array[] = []; + for (const row of rows) { + const bytes = new Uint8Array(row.data); + if (row.kind === UpdateKind.continuation && pending.length > 0) { + pending.push(bytes); + continue; + } + if (pending.length > 0) updates.push(concat(pending)); + pending = [bytes]; + } + if (pending.length > 0) updates.push(concat(pending)); + + return updates; + } +} +``` + +- [ ] **Step 6: テストが通ることを確認する** + +Run: `pnpm exec vitest run src/yjs/storage/sqlite.test.ts` +Expected: PASS（4 tests） + +- [ ] **Step 7: 全体を検証してコミットする** + +```bash +pnpm fmt && pnpm typecheck && pnpm exec vitest run +git add src/yjs/storage/queries.ts src/yjs/storage/type.ts src/yjs/storage/sqlite.ts src/yjs/storage/sqlite.test.ts +git commit -m "feat: add SQLite-backed Yjs storage with ordered update log" +``` + +--- + +### Task 3: コンパクションとチャンク分割 + +行数しきい値に到達したら全行を 1 本にマージして書き戻す。マージ結果が `maxChunkBytes` を超える場合はバイト断片に分割する。これで C-2（128KiB 制限）と C-3（delete 128 キー制限）が構造的に解消する。 + +**Files:** +- Modify: `src/yjs/storage/sqlite.ts`（`commit()` と `storeUpdate()`、`#split()` を追加） +- Test: `src/yjs/storage/sqlite.test.ts`（テストを追記） + +**Interfaces:** +- Consumes: Task 2 の `YSqliteStorage` +- Produces: 変更なし（`commit()` の実装が入るのみ） + +- [ ] **Step 1: 失敗するテストを追記する** + +`src/yjs/storage/sqlite.test.ts` の `describe("YSqliteStorage", ...)` の中に以下を追記する。`countRows` ヘルパーはファイル冒頭のヘルパー群の隣に置く。 + +```ts +// ファイル冒頭のヘルパー群に追加 +const withStorageAndSql = async ( + fn: (storage: YSqliteStorage, sql: SqlStorage) => Promise, + options?: YSqliteStorageOptions, +): Promise => { + const stub = env.Y_DURABLE_OBJECTS.get(env.Y_DURABLE_OBJECTS.newUniqueId()); + await runInDurableObject(stub, async (_instance, state) => { + await fn(new YSqliteStorage(state.storage.sql, options), state.storage.sql); + }); +}; + +const countRows = (sql: SqlStorage): number => + sql.exec<{ count: number }>("SELECT COUNT(*) AS count FROM updates").one().count; +``` + +```ts +// describe("YSqliteStorage", ...) の中に追加 +it("compacts once the row threshold is exceeded", async () => { + await withStorageAndSql( + async (storage, sql) => { + const doc = new Doc(); + const text = doc.getText("root"); + const updates: Uint8Array[] = []; + doc.on("update", (update: Uint8Array) => updates.push(update)); + for (let i = 0; i < 50; i++) { + text.insert(text.length, "x"); + } + for (const update of updates) { + await storage.storeUpdate(update); + } + + // maxRows 10 に対し 50 件保存したので、行数は大幅に減っているはず + expect(countRows(sql)).toBeLessThanOrEqual(10); + + const restored = await storage.getUpdate(); + expect(textOf(restored!)).toBe(text.toString()); + }, + { maxRows: 10 }, + ); +}); + +it("splits a compacted update that exceeds maxChunkBytes and restores it", async () => { + await withStorageAndSql( + async (storage, sql) => { + const doc = new Doc(); + const text = doc.getText("root"); + const updates: Uint8Array[] = []; + doc.on("update", (update: Uint8Array) => updates.push(update)); + for (let i = 0; i < 20; i++) { + text.insert(text.length, "abcdefghij".repeat(50)); + } + for (const update of updates) { + await storage.storeUpdate(update); + } + await storage.commit(); + + // 512 バイトずつに分割されるので、複数行になっているはず + expect(countRows(sql)).toBeGreaterThan(1); + + const restored = await storage.getUpdate(); + expect(textOf(restored!)).toBe(text.toString()); + }, + { maxRows: 1000, maxChunkBytes: 512 }, + ); +}); + +it("stores a document larger than the legacy 128KiB key-value limit", async () => { + await withStorage(async (storage) => { + const doc = new Doc(); + // 300KB 相当。KV バックエンドでは 1 キーに収まらず保存に失敗していた（C-2） + doc.getText("root").insert(0, "y".repeat(300 * 1024)); + await storage.storeUpdate(encodeStateAsUpdate(doc)); + await storage.commit(); + + const restored = await storage.getUpdate(); + + expect(textOf(restored!).length).toBe(300 * 1024); + }); +}); + +it("is a no-op when there is at most one row to compact", async () => { + await withStorageAndSql(async (storage, sql) => { + await storage.storeUpdate(encodeStateAsUpdate(docWith("single"))); + await storage.commit(); + + expect(countRows(sql)).toBe(1); + expect(textOf((await storage.getUpdate())!)).toBe("single"); + }); +}); +``` + +- [ ] **Step 2: テストが失敗することを確認する** + +Run: `pnpm exec vitest run src/yjs/storage/sqlite.test.ts` +Expected: FAIL — 3 件が失敗する。`commit()` が未実装のため行数が減らず、`expect(countRows(sql)).toBeLessThanOrEqual(10)` などが失敗する。 + +- [ ] **Step 3: storeUpdate にしきい値判定を追加する** + +`src/yjs/storage/sqlite.ts` の `storeUpdate` を置き換える。 + +```ts + async storeUpdate(update: Uint8Array): Promise { + this.#sql.exec(INSERT_UPDATE, UpdateKind.standalone, update); + this.#rowCount += 1; + + if (this.#rowCount > this.#maxRows) { + await this.commit(); + } + } +``` + +- [ ] **Step 4: commit と #split を実装する** + +`src/yjs/storage/sqlite.ts` の `commit()` を置き換え、`#split()` を `#readAll()` の隣に追加する。 + +```ts + async commit(): Promise { + if (this.#rowCount <= 1) return; + + const updates = this.#readAll(); + if (updates.length === 0) return; + + const chunks = this.#split(mergeUpdates(updates)); + + // ここから下では await を挟まないこと。 + // 連続した同期書き込みが暗黙のトランザクションとして atomic に適用される。 + this.#sql.exec(DELETE_ALL_UPDATES); + for (const [index, chunk] of chunks.entries()) { + const kind = index === 0 ? UpdateKind.standalone : UpdateKind.continuation; + this.#sql.exec(INSERT_UPDATE, kind, chunk); + } + this.#rowCount = chunks.length; + } + + /** + * マージ済みの update を maxChunkBytes 以下のバイト断片に分割する。 + * subarray ではなく slice を使ってコピーを作る。ビューをそのまま + * バインドすると基底バッファ全体が書き込まれる可能性があるため。 + */ + #split(update: Uint8Array): Uint8Array[] { + if (update.byteLength <= this.#maxChunkBytes) return [update]; + + const chunks: Uint8Array[] = []; + for (let offset = 0; offset < update.byteLength; offset += this.#maxChunkBytes) { + chunks.push(update.slice(offset, Math.min(offset + this.#maxChunkBytes, update.byteLength))); + } + + return chunks; + } +``` + +- [ ] **Step 5: テストが通ることを確認する** + +Run: `pnpm exec vitest run src/yjs/storage/sqlite.test.ts` +Expected: PASS（8 tests） + +- [ ] **Step 6: 全体を検証してコミットする** + +```bash +pnpm fmt && pnpm typecheck && pnpm exec vitest run +git add src/yjs/storage/sqlite.ts src/yjs/storage/sqlite.test.ts +git commit -m "feat: compact update log with chunk splitting for large documents" +``` + +--- + +### Task 4: YDurableObjects を YSqliteStorage に載せ替え、KV 実装を削除 + +Durable Object 本体を新しいストレージに切り替え、KV バックエンド向けの実装をすべて削除する。起動時に Doc を 2 つ構築していた無駄も解消する。 + +**Files:** +- Modify: `src/yjs/index.ts:28-35`（`storage` フィールド）, `src/yjs/index.ts:39-46`（コンストラクタ）, `src/yjs/index.ts:48-50`（`onStart`） +- Modify: `src/yjs/storage/index.ts`（再エクスポートのみにする） +- Modify: `src/yjs/storage/type.ts`（`TransactionStorage` を削除） +- Modify: `src/yjs/internal.ts:2,7` +- Modify: `src/index.ts:40` +- Delete: `src/yjs/storage/storage-key/index.ts`, `src/yjs/storage/storage-key/storage-key.test.ts`, `src/yjs/storage/storage.test.ts` + +**Interfaces:** +- Consumes: `YSqliteStorage`, `YStorage`（Task 2） +- Produces: `YDurableObjects.storage` の型が `YSqliteStorage` になる。`src/index.ts` が `YStorage` をエクスポートする（旧 `YTransactionStorage` は削除） + +- [ ] **Step 1: 旧実装のテストを削除する** + +```bash +git rm src/yjs/storage/storage.test.ts src/yjs/storage/storage-key/storage-key.test.ts src/yjs/storage/storage-key/index.ts +``` + +- [ ] **Step 2: storage/index.ts を再エクスポートだけにする** + +`src/yjs/storage/index.ts` の内容を完全に置き換える。 + +```ts +export { YSqliteStorage } from "./sqlite"; +export { UpdateKind } from "./type"; + +export type { YSqliteStorageOptions } from "./sqlite"; +export type { YStorage } from "./type"; +``` + +- [ ] **Step 3: type.ts から TransactionStorage を削除する** + +`src/yjs/storage/type.ts` から `ListOptions` インターフェースと `TransactionStorage` インターフェースを削除し、Task 2 で追記した `UpdateKind` と `YStorage` のみを残す。 + +- [ ] **Step 4: YDurableObjects を新しいストレージに載せ替える** + +`src/yjs/index.ts` の該当箇所を書き換える。`storage` はフィールド初期化子ではなくコンストラクタ本体で代入する。フィールド初期化子は `useDefineForClassFields` の下でパラメータプロパティ（`public state`）の代入より先に走るため、初期化子から `this.state` を参照すると `undefined` になる。 + +```ts + protected app = createApp({ + createRoom: this.createRoom.bind(this), + }); + protected doc = new WSSharedDoc(); + protected storage: YSqliteStorage; + protected sessions = new Map void>(); + private awarenessClients = new Set(); + + constructor( + public state: DurableObjectState, + public env: T["Bindings"], + ) { + super(state, env); + + this.storage = new YSqliteStorage(state.storage.sql); + + void this.state.blockConcurrencyWhile(this.onStart.bind(this)); + } + + protected async onStart(): Promise { + const update = await this.storage.getUpdate(); + if (update !== null) { + applyUpdate(this.doc, update); + } + + for (const ws of this.state.getWebSockets()) { + this.registerWebSocket(ws); + } + + this.doc.on("update", async (update) => { + await this.storage.storeUpdate(update); + }); + this.doc.awareness.on( + "update", + async ({ added, removed, updated }: AwarenessChanges) => { + for (const client of [...added, ...updated]) { + this.awarenessClients.add(client); + } + for (const client of removed) { + this.awarenessClients.delete(client); + } + }, + ); + } +``` + +import 文も更新する。`encodeStateAsUpdate` は `getYDoc()` でまだ使うので残す。 + +```ts +import { applyUpdate, encodeStateAsUpdate } from "yjs"; +// ... +import { YSqliteStorage } from "./storage"; +``` + +- [ ] **Step 5: internal.ts と src/index.ts のエクスポートを更新する** + +`src/yjs/internal.ts` の 2 行目と 7 行目: + +```ts +import type { YSqliteStorage } from "./storage"; +``` + +```ts + storage: YSqliteStorage; +``` + +`src/index.ts:40` を置き換える: + +```ts +export type { YStorage, YSqliteStorageOptions } from "./yjs/storage"; +``` + +- [ ] **Step 6: 全テストが通ることを確認する** + +Run: `pnpm typecheck && pnpm exec vitest run` +Expected: PASS。`storage.test.ts` と `storage-key.test.ts` が消えた分テスト数は減る（41 → 34 前後）が、e2e テストは全件通る。 + +- [ ] **Step 7: コミットする** + +```bash +pnpm fmt && pnpm typecheck && pnpm exec vitest run +git add -A src/ +git commit -m "refactor!: replace key-value storage layer with SQLite implementation" +``` + +--- + +### Task 5: WSSharedDoc に origin を通し、メッセージ型を拡張する + +Yjs の transaction origin を WebSocket まで伝播させ、syncStep2 を要求元だけに返し（H-1）、送信者へのエコーバックを止める（H-2）。あわせて `queryAwareness` と `auth` を追加し、未知のメッセージ型を無言で捨てないようにする（H-7）。 + +**Files:** +- Modify: `src/yjs/message-type/index.ts` +- Modify: `src/yjs/remote/ws-shared-doc.ts`（全面改修） +- Modify: `src/yjs/index.ts:119-125`（`registerWebSocket`）, `src/yjs/index.ts:91-94`（`updateYDoc`） +- Modify: `src/yjs/remote/ws-shared-doc.test.ts`（`notify` / `update` の新シグネチャに追従） +- Modify: `src/yjs/message-type/messaeg-type.test.ts`（型が増えたことに追従） + +**Interfaces:** +- Consumes: なし +- Produces: + - `messageType: { sync: 0; awareness: 1; auth: 2; queryAwareness: 3 }` + - `WSSharedDoc.notify(origin: object, listener: (message: Uint8Array) => void): () => void` + - `WSSharedDoc.update(message: Uint8Array, origin: object): void` + - `const RPC_ORIGIN: object`（`src/yjs/index.ts` 内で定義。WebSocket 由来でない更新の origin として使う） + +- [ ] **Step 1: 失敗するテストを書く** + +`src/yjs/remote/ws-shared-doc.test.ts` に以下を追記する。 + +```ts +it("sends the sync step 2 reply only to the requesting origin", () => { + const doc = new WSSharedDoc(); + const requester = {}; + const bystander = {}; + const toRequester: Uint8Array[] = []; + const toBystander: Uint8Array[] = []; + doc.notify(requester, (message) => toRequester.push(message)); + doc.notify(bystander, (message) => toBystander.push(message)); + + doc.getText("root").insert(0, "seed"); + toRequester.length = 0; + toBystander.length = 0; + + const encoder = createEncoder(); + writeVarUint(encoder, messageType.sync); + writeSyncStep1(encoder, new Doc()); + doc.update(toUint8Array(encoder), requester); + + expect(toRequester.length).toBe(1); + expect(toBystander.length).toBe(0); +}); + +it("does not echo an update back to its origin", () => { + const doc = new WSSharedDoc(); + const sender = {}; + const receiver = {}; + const toSender: Uint8Array[] = []; + const toReceiver: Uint8Array[] = []; + doc.notify(sender, (message) => toSender.push(message)); + doc.notify(receiver, (message) => toReceiver.push(message)); + + const source = new Doc(); + source.getText("root").insert(0, "hello"); + const encoder = createEncoder(); + writeVarUint(encoder, messageType.sync); + writeUpdate(encoder, encodeStateAsUpdate(source)); + doc.update(toUint8Array(encoder), sender); + + expect(toSender.length).toBe(0); + expect(toReceiver.length).toBe(1); +}); + +it("throws on an unknown message type", () => { + const doc = new WSSharedDoc(); + const encoder = createEncoder(); + writeVarUint(encoder, 99); + + expect(() => doc.update(toUint8Array(encoder), {})).toThrow(); +}); +``` + +必要な import を先頭に追加する。 + +```ts +import { createEncoder, toUint8Array, writeVarUint } from "lib0/encoding"; +import { writeSyncStep1, writeUpdate } from "y-protocols/sync"; +import { Doc, encodeStateAsUpdate } from "yjs"; + +import { messageType } from "../message-type"; +``` + +- [ ] **Step 2: テストが失敗することを確認する** + +Run: `pnpm exec vitest run src/yjs/remote/ws-shared-doc.test.ts` +Expected: FAIL — `notify` が 2 引数を受け取らず、型エラーまたは実行時エラーになる + +- [ ] **Step 3: message-type/index.ts を拡張する** + +```ts +import { createEncoder, writeVarUint } from "lib0/encoding"; + +export const messageType = { + sync: 0, + awareness: 1, + auth: 2, + queryAwareness: 3, +} as const; + +export const isMessageType = ( + type: string, +): type is keyof typeof messageType => { + return Object.keys(messageType).includes(type); +}; + +export const createTypedEncoder = (type: keyof typeof messageType) => { + if (!isMessageType(type)) { + throw new Error(`Unsupported message type: ${type}`); + } + + const encoder = createEncoder(); + writeVarUint(encoder, messageType[type]); + + return encoder; +}; +``` + +- [ ] **Step 4: ws-shared-doc.ts を書き換える** + +```ts +import { createDecoder, readVarUint, readVarUint8Array } from "lib0/decoding"; +import { + createEncoder, + length, + toUint8Array, + writeVarUint, + writeVarUint8Array, +} from "lib0/encoding"; +import { + applyAwarenessUpdate, + Awareness, + encodeAwarenessUpdate, +} from "y-protocols/awareness"; +import { readSyncMessage, writeUpdate } from "y-protocols/sync"; +import { Doc } from "yjs"; + +import { createTypedEncoder, messageType } from "../message-type"; + +import type { AwarenessChanges, RemoteDoc } from "."; + +type Listener = (message: Uint8Array) => void; +type Unsubscribe = () => void; + +interface Notification extends RemoteDoc { + notify(origin: object, listener: Listener): Unsubscribe; +} + +export class WSSharedDoc extends Doc implements Notification { + /** origin（通常は WebSocket）をキーにした配信先 */ + private listeners = new Map(); + readonly awareness = new Awareness(this); + + constructor(gc = true) { + super({ gc }); + this.awareness.setLocalState(null); + + // カーソルなどの付加情報の更新通知 + this.awareness.on("update", (changes: AwarenessChanges, origin: unknown) => { + this.awarenessChangeHandler(changes, origin); + }); + // yDoc の更新通知 + this.on("update", (update: Uint8Array, origin: unknown) => { + this.syncMessageHandler(update, origin); + }); + } + + update(message: Uint8Array, origin: object) { + const encoder = createEncoder(); + const decoder = createDecoder(message); + const type = readVarUint(decoder); + + switch (type) { + case messageType.sync: { + writeVarUint(encoder, messageType.sync); + readSyncMessage(decoder, encoder, this, origin); + + // sync step 1 への応答は要求元にだけ返す + if (length(encoder) > 1) { + this.send(origin, toUint8Array(encoder)); + } + break; + } + case messageType.awareness: { + applyAwarenessUpdate(this.awareness, readVarUint8Array(decoder), origin); + break; + } + case messageType.queryAwareness: { + const states = this.awareness.getStates(); + if (states.size > 0) { + const reply = createTypedEncoder("awareness"); + writeVarUint8Array( + reply, + encodeAwarenessUpdate(this.awareness, Array.from(states.keys())), + ); + this.send(origin, toUint8Array(reply)); + } + break; + } + case messageType.auth: { + // auth はサーバからクライアントへの一方向のメッセージなので受信しても何もしない + break; + } + default: { + throw new Error(`Unsupported message type: ${type}`); + } + } + } + + notify(origin: object, listener: Listener) { + this.listeners.set(origin, listener); + + return () => { + this.listeners.delete(origin); + }; + } + + private syncMessageHandler(update: Uint8Array, origin: unknown) { + const encoder = createTypedEncoder("sync"); + writeUpdate(encoder, update); + + this.broadcast(toUint8Array(encoder), origin); + } + + private awarenessChangeHandler( + { added, updated, removed }: AwarenessChanges, + origin: unknown, + ) { + const changed = [...added, ...updated, ...removed]; + const encoder = createTypedEncoder("awareness"); + const update = encodeAwarenessUpdate( + this.awareness, + changed, + this.awareness.states, + ); + writeVarUint8Array(encoder, update); + + this.broadcast(toUint8Array(encoder), origin); + } + + private send(origin: object, message: Uint8Array) { + this.listeners.get(origin)?.(message); + } + + private broadcast(message: Uint8Array, exclude: unknown) { + for (const [origin, listener] of this.listeners) { + if (origin === exclude) continue; + listener(message); + } + } +} +``` + +- [ ] **Step 5: 呼び出し側を新シグネチャに合わせる** + +`src/yjs/index.ts` のファイル先頭付近（import の直後）に定数を追加する。 + +```ts +/** WebSocket 由来でない更新（JS RPC 経由）の origin */ +const RPC_ORIGIN: object = Object.freeze({ source: "rpc" }); +``` + +`registerWebSocket` を書き換える。 + +```ts + protected registerWebSocket(ws: WebSocket) { + setupWSConnection(ws, this.doc); + const s = this.doc.notify(ws, (message) => { + ws.send(message); + }); + this.sessions.set(ws, s); + } +``` + +`updateYDoc` と `webSocketMessage` を書き換える。 + +```ts + async updateYDoc(update: Uint8Array): Promise { + this.doc.update(update, RPC_ORIGIN); + await this.cleanup(); + } +``` + +```ts + async webSocketMessage( + ws: WebSocket, + message: string | ArrayBuffer, + ): Promise { + if (!(message instanceof ArrayBuffer)) return; + + this.doc.update(new Uint8Array(message), ws); + await this.cleanup(); + } +``` + +- [ ] **Step 6: message-type のテストを更新する** + +`src/yjs/message-type/messaeg-type.test.ts` に `auth` と `queryAwareness` のケースを追加し、既存のアサーションが新しいキー集合と矛盾しないよう修正する。 + +- [ ] **Step 7: テストが通ることを確認する** + +Run: `pnpm typecheck && pnpm exec vitest run` +Expected: PASS + +- [ ] **Step 8: コミットする** + +```bash +pnpm fmt && pnpm typecheck && pnpm exec vitest run +git add -A src/ +git commit -m "fix!: route sync replies to the requester and stop echoing to the sender" +``` + +--- + +### Task 6: SessionRegistry で awareness の所有権を追跡する + +接続ごとの awareness clientID を `serializeAttachment` に永続化し、切断時にその接続の clientID だけを削除する（C-4）。hibernation を跨いでも所有権が保たれる。 + +**Files:** +- Create: `src/yjs/session/index.ts` +- Test: `src/yjs/session/session.test.ts` +- Modify: `src/yjs/index.ts`（`sessions` を `SessionRegistry` に置換、`awarenessClients` を削除、`createRoom` / `onStart` / `registerWebSocket` / `unregisterWebSocket` / `cleanup`） +- Modify: `src/yjs/internal.ts` + +**Interfaces:** +- Consumes: `WSSharedDoc.notify(origin, listener)`（Task 5） +- Produces: + - `type SessionAttachment = { roomId: string; connectedAt: number; clientIds: number[] }` + - `class SessionRegistry` — `size: number`, `add(ws, dispose)`, `remove(ws)`, `has(ws)`, `sockets(): IterableIterator`, `clientIdsOf(ws): number[]`, `track(ws, clientIds)` + +- [ ] **Step 1: 失敗するテストを書く** + +```ts +// src/yjs/session/session.test.ts +import { describe, expect, it, vi } from "vitest"; + +import { SessionRegistry } from "."; + +import type { SessionAttachment } from "."; + +const fakeSocket = (attachment: SessionAttachment | null): WebSocket => { + let current = attachment; + + return { + serializeAttachment: (value: SessionAttachment) => { + current = value; + }, + deserializeAttachment: () => current, + } as unknown as WebSocket; +}; + +const attachment = (clientIds: number[]): SessionAttachment => ({ + roomId: "room1", + connectedAt: 0, + clientIds, +}); + +describe("SessionRegistry", () => { + it("tracks and disposes sockets", () => { + const registry = new SessionRegistry(); + const ws = fakeSocket(attachment([])); + const dispose = vi.fn(); + + registry.add(ws, dispose); + expect(registry.size).toBe(1); + expect(registry.has(ws)).toBe(true); + + registry.remove(ws); + expect(dispose).toHaveBeenCalledTimes(1); + expect(registry.size).toBe(0); + }); + + it("records client ids on the attachment", () => { + const registry = new SessionRegistry(); + const ws = fakeSocket(attachment([])); + registry.add(ws, () => {}); + + registry.track(ws, [7]); + + expect(registry.clientIdsOf(ws)).toEqual([7]); + }); + + it("does not duplicate client ids", () => { + const registry = new SessionRegistry(); + const ws = fakeSocket(attachment([7])); + registry.add(ws, () => {}); + + registry.track(ws, [7, 8]); + registry.track(ws, [8]); + + expect(registry.clientIdsOf(ws).sort()).toEqual([7, 8]); + }); + + it("restores ownership from an existing attachment after hibernation", () => { + // hibernation 復帰を模す。registry は空だが WebSocket の attachment は残っている + const ws = fakeSocket(attachment([42])); + const registry = new SessionRegistry(); + registry.add(ws, () => {}); + + expect(registry.clientIdsOf(ws)).toEqual([42]); + }); + + it("returns an empty list for a socket without a valid attachment", () => { + const registry = new SessionRegistry(); + const ws = fakeSocket(null); + registry.add(ws, () => {}); + + expect(registry.clientIdsOf(ws)).toEqual([]); + }); +}); +``` + +- [ ] **Step 2: テストが失敗することを確認する** + +Run: `pnpm exec vitest run src/yjs/session/session.test.ts` +Expected: FAIL — `Failed to resolve import "."` + +- [ ] **Step 3: SessionRegistry を実装する** + +```ts +// src/yjs/session/index.ts + +export type SessionAttachment = { + roomId: string; + /** epoch ミリ秒。attachment は structured clone されるが、数値の方が扱いが単純 */ + connectedAt: number; + /** この接続が所有する awareness の clientID */ + clientIds: number[]; +}; + +const isSessionAttachment = (value: unknown): value is SessionAttachment => { + if (typeof value !== "object" || value === null) return false; + const candidate = value as Record; + + return ( + typeof candidate.roomId === "string" && + typeof candidate.connectedAt === "number" && + Array.isArray(candidate.clientIds) && + candidate.clientIds.every((id) => typeof id === "number") + ); +}; + +/** + * WebSocket と awareness の所有権を管理する。 + * + * 所有権は WebSocket の attachment に永続化するため、Durable Object が + * hibernation から復帰してメモリ上の状態を失っても復元できる。 + */ +export class SessionRegistry { + private disposers = new Map void>(); + + get size(): number { + return this.disposers.size; + } + + add(ws: WebSocket, dispose: () => void): void { + this.disposers.set(ws, dispose); + } + + remove(ws: WebSocket): void { + this.disposers.get(ws)?.(); + this.disposers.delete(ws); + } + + has(ws: WebSocket): boolean { + return this.disposers.has(ws); + } + + sockets(): IterableIterator { + return this.disposers.keys(); + } + + clientIdsOf(ws: WebSocket): number[] { + return this.attachmentOf(ws)?.clientIds ?? []; + } + + /** + * この接続が所有する clientID を記録する。 + * 実際に増えたときだけ attachment を書き直すので、通常は接続あたり 1 回で済む。 + */ + track(ws: WebSocket, clientIds: readonly number[]): void { + const current = this.attachmentOf(ws); + if (current === null) return; + + const merged = new Set([...current.clientIds, ...clientIds]); + if (merged.size === current.clientIds.length) return; + + ws.serializeAttachment({ + ...current, + clientIds: Array.from(merged), + } satisfies SessionAttachment); + } + + private attachmentOf(ws: WebSocket): SessionAttachment | null { + const raw: unknown = ws.deserializeAttachment(); + + return isSessionAttachment(raw) ? raw : null; + } +} +``` + +- [ ] **Step 4: テストが通ることを確認する** + +Run: `pnpm exec vitest run src/yjs/session/session.test.ts` +Expected: PASS（5 tests） + +- [ ] **Step 5: YDurableObjects を SessionRegistry に置き換える** + +`src/yjs/index.ts` を以下のように変更する。`WebSocketAttachment` 型のエクスポートは `SessionAttachment` に置き換える。 + +```ts +// フィールド + protected sessions = new SessionRegistry(); + // private awarenessClients = new Set(); ← 削除 + +// onStart の awareness ハンドラを置き換える + this.doc.awareness.on("update", ({ added, updated }: AwarenessChanges, origin: unknown) => { + if (origin instanceof WebSocket) { + this.sessions.track(origin, [...added, ...updated]); + } + }); + +// createRoom + protected createRoom(roomId: string) { + const pair = new WebSocketPair(); + const client = pair[0]; + const server = pair[1]; + server.serializeAttachment({ + roomId, + connectedAt: Date.now(), + clientIds: [], + } satisfies SessionAttachment); + + this.state.acceptWebSocket(server); + this.registerWebSocket(server); + + return client; + } + +// registerWebSocket + protected registerWebSocket(ws: WebSocket) { + setupWSConnection(ws, this.doc); + const dispose = this.doc.notify(ws, (message) => { + ws.send(message); + }); + this.sessions.add(ws, dispose); + } + +// unregisterWebSocket + protected async unregisterWebSocket(ws: WebSocket) { + try { + // この接続が所有する clientID だけを削除する。 + // 部屋全体の clientID を削除すると他の参加者の presence まで消える。 + removeAwarenessStates(this.doc.awareness, this.sessions.clientIdsOf(ws), null); + this.sessions.remove(ws); + } catch (e) { + // eslint-disable-next-line no-console + console.error(e); + } + } + +// cleanup + protected async cleanup() { + if (this.sessions.size < 1) { + await this.storage.commit(); + } + } +``` + +import を追加する。 + +```ts +import { SessionRegistry } from "./session"; + +import type { SessionAttachment } from "./session"; +``` + +`export type WebSocketAttachment = { ... }` を削除し、代わりに再エクスポートする。 + +```ts +export type { SessionAttachment } from "./session"; +``` + +- [ ] **Step 6: internal.ts を更新する** + +```ts +import type { SessionRegistry } from "./session"; +``` + +```ts + sessions: SessionRegistry; + // awarenessClients: Set; ← 削除 +``` + +- [ ] **Step 7: e2e テストの awareness 回帰テストを追加する** + +`src/e2e/y-durableobjects.test.ts` に追記する。 + +```ts +it("keeps other clients' awareness when one connection closes", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject(stub, async (instance: InternalYDurableObject) => { + await instance.createRoom("room1"); + await instance.createRoom("room1"); + const [first, second] = Array.from(instance.sessions.sockets()); + + instance.sessions.track(first, [1]); + instance.sessions.track(second, [2]); + instance.doc.awareness.setLocalStateField("user", { name: "a" }); + + await instance.webSocketClose(first); + + // first の clientId だけが除去され、second の所有分は残る + expect(instance.sessions.clientIdsOf(second)).toEqual([2]); + expect(instance.sessions.size).toBe(1); + }); +}); +``` + +- [ ] **Step 8: テストが通ることを確認する** + +Run: `pnpm typecheck && pnpm exec vitest run` +Expected: PASS。既存の `instance.sessions.size` を参照するテストは `SessionRegistry.size` でそのまま動く。`Array.from(instance.sessions.entries())` を使っている既存テスト（`y-durableobjects.test.ts:95,110`）は `Array.from(instance.sessions.sockets())` に書き換える。 + +- [ ] **Step 9: コミットする** + +```bash +pnpm fmt && pnpm typecheck && pnpm exec vitest run +git add -A src/ +git commit -m "fix!: scope awareness removal to the disconnecting connection" +``` + +--- + +### Task 7: 永続化を直列化し、失敗時に安全側へ倒す + +宙に浮いていた永続化 Promise を直列化し（C-5）、失敗時は全接続を閉じてメモリ状態を破棄する。あわせて不正メッセージの例外境界を設ける（H-4）。 + +**Files:** +- Modify: `src/yjs/index.ts`（`onStart` の update ハンドラ、`webSocketMessage`、`updateYDoc`、新規プライベートメソッド 2 つ） +- Test: `src/e2e/y-durableobjects.test.ts`（追記） + +**Interfaces:** +- Consumes: `YSqliteStorage.storeUpdate`（Task 2）、`WSSharedDoc.update(message, origin)`（Task 5） +- Produces: なし（内部実装の変更のみ） + +- [ ] **Step 1: 失敗するテストを書く** + +`src/e2e/y-durableobjects.test.ts` に追記する。 + +```ts +it("closes only the offending connection on a malformed message", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject(stub, async (instance: InternalYDurableObject) => { + await instance.createRoom("room1"); + await instance.createRoom("room1"); + const [first] = Array.from(instance.sessions.sockets()); + + // 未知のメッセージ型。例外が外に漏れると DO 全体がリセットされる + const malformed = new Uint8Array([99]).buffer; + await expect(instance.webSocketMessage(first, malformed)).resolves.toBeUndefined(); + + // DO は生存し、他の接続も維持されている + expect(instance.sessions.size).toBe(2); + }); +}); + +it("persists an update before webSocketMessage resolves", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject(stub, async (instance: InternalYDurableObject) => { + const client = await instance.createRoom("room1"); + const message = createSyncMessage(createYDocMessage("persisted")); + + await instance.webSocketMessage(client, message.slice(0).buffer); + + // ストレージから読み直しても内容が入っている + const stored = await instance.storage.getUpdate(); + expect(stored).not.toBeNull(); + }); +}); +``` + +- [ ] **Step 2: テストが失敗することを確認する** + +Run: `pnpm exec vitest run src/e2e/y-durableobjects.test.ts` +Expected: FAIL — 未知のメッセージ型で `webSocketMessage` が reject する + +- [ ] **Step 3: 永続化キューと失敗ハンドラを実装する** + +`src/yjs/index.ts` にフィールドとプライベートメソッドを追加する。 + +```ts + /** 永続化を直列化するためのキュー。Yjs の update イベントは同期的に発火するため必要 */ + private persist: Promise = Promise.resolve(); +``` + +```ts + private schedulePersist(update: Uint8Array): void { + this.persist = this.persist + .then(() => this.storage.storeUpdate(update)) + .catch((error: unknown) => { + this.onPersistFailure(error); + }); + this.state.waitUntil(this.persist); + } + + /** + * 永続化に失敗したら全接続を閉じ、Durable Object をリセットする。 + * + * 接続を閉じるだけではメモリ上の Doc がストレージより進んだまま残り、 + * 後続の接続が「正常に見える」状態を受け取ったあと、eviction 時に + * 差分が無言で失われる。abort() でストレージから読み直させる。 + * + * CRDT ではクライアント側が完全な状態を保持しているため、再接続時の + * sync step 1 / 2 で失われた更新が再送され、障害は自己修復する。 + */ + private onPersistFailure(error: unknown): void { + // eslint-disable-next-line no-console + console.error("[y-durableobjects] failed to persist update", error); + + for (const ws of this.state.getWebSockets()) { + ws.close(1011, "storage failure"); + } + this.state.abort("failed to persist a Yjs update"); + } +``` + +- [ ] **Step 4: onStart の update ハンドラを差し替える** + +```ts + this.doc.on("update", (update: Uint8Array) => { + this.schedulePersist(update); + }); +``` + +- [ ] **Step 5: webSocketMessage に例外境界を設け、永続化を待つ** + +```ts + async webSocketMessage( + ws: WebSocket, + message: string | ArrayBuffer, + ): Promise { + if (!(message instanceof ArrayBuffer)) return; + + try { + this.doc.update(new Uint8Array(message), ws); + } catch (error) { + // eslint-disable-next-line no-console + console.error("[y-durableobjects] invalid message", error); + ws.close(1003, "invalid message"); + + return; + } + + await this.persist; + await this.cleanup(); + } +``` + +`updateYDoc` も同様に永続化を待つ。 + +```ts + async updateYDoc(update: Uint8Array): Promise { + this.doc.update(update, RPC_ORIGIN); + await this.persist; + await this.cleanup(); + } +``` + +- [ ] **Step 6: テストが通ることを確認する** + +Run: `pnpm typecheck && pnpm exec vitest run` +Expected: PASS + +- [ ] **Step 7: コミットする** + +```bash +pnpm fmt && pnpm typecheck && pnpm exec vitest run +git add -A src/ +git commit -m "fix!: serialize persistence and fail closed when storage writes fail" +``` + +--- + +### Task 8: hibernation の衛生と KV バックエンドの検出、destroy API + +ping/pong を自動応答にして hibernation 中の起床を防ぎ、KV バックエンドで起動された場合は移行手順を示して即座に失敗させ、部屋の削除 API を追加する。 + +**Files:** +- Modify: `src/yjs/index.ts`（コンストラクタ、`destroy()` の追加） +- Modify: `src/yjs/internal.ts` +- Test: `src/e2e/y-durableobjects.test.ts`（追記） + +**Interfaces:** +- Consumes: `YSqliteStorage.destroy()`（Task 2） +- Produces: `YDurableObjects.destroy(): Promise` + +- [ ] **Step 1: 失敗するテストを書く** + +```ts +it("configures a ping/pong auto response", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject(stub, async (_instance, state) => { + const pair = state.getWebSocketAutoResponse(); + + expect(pair?.request).toBe("ping"); + expect(pair?.response).toBe("pong"); + }); +}); + +it("clears the document and closes connections on destroy", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject(stub, async (instance: InternalYDurableObject) => { + await instance.createRoom("room1"); + await instance.updateYDoc(createSyncMessage(createYDocMessage("bye")).slice(0)); + + await instance.destroy(); + + expect(await instance.storage.getUpdate()).toBeNull(); + }); +}); +``` + +- [ ] **Step 2: テストが失敗することを確認する** + +Run: `pnpm exec vitest run src/e2e/y-durableobjects.test.ts` +Expected: FAIL — auto response が未設定、`destroy` が未定義 + +- [ ] **Step 3: コンストラクタに SQLite 検出と auto response を追加する** + +`src/yjs/index.ts` のコンストラクタを書き換える。定数はファイル先頭に置く。 + +```ts +const MIGRATION_GUIDE_URL = + "https://github.com/napolab/y-durableobjects#migrating-from-v1-key-value-backend"; + +/** + * SQLite バックエンドで動作しているかを確認する。 + * KV バックエンドでは sql へのアクセスが失敗するため、 + * 原因不明のクラッシュではなく移行手順を示したエラーにする。 + */ +const assertSqliteBackend = (storage: DurableObjectStorage): void => { + try { + storage.sql.exec("SELECT 1"); + } catch (error) { + throw new Error( + `y-durableobjects v2 requires the SQLite storage backend. ` + + `Use "new_sqlite_classes" in your wrangler migrations. ` + + `Migration guide: ${MIGRATION_GUIDE_URL}`, + { cause: error }, + ); + } +}; +``` + +```ts + constructor( + public state: DurableObjectState, + public env: T["Bindings"], + ) { + super(state, env); + + assertSqliteBackend(state.storage); + this.storage = new YSqliteStorage(state.storage.sql); + + // ping を自動応答にすることで、keepalive で Durable Object を + // 起こさずに済む。duration 課金に最も効く設定。 + state.setWebSocketAutoResponse(new WebSocketRequestResponsePair("ping", "pong")); + + void this.state.blockConcurrencyWhile(this.onStart.bind(this)); + } +``` + +- [ ] **Step 4: destroy を実装する** + +`getYDoc` / `updateYDoc` の隣に追加する。 + +```ts + /** 部屋のデータを削除し、すべての接続を閉じる */ + async destroy(): Promise { + for (const ws of this.state.getWebSockets()) { + ws.close(1001, "room destroyed"); + } + await this.storage.destroy(); + } +``` + +`src/yjs/internal.ts` の public api セクションに追記する。 + +```ts + destroy(): Promise; +``` + +- [ ] **Step 5: テストが通ることを確認する** + +Run: `pnpm typecheck && pnpm exec vitest run` +Expected: PASS + +- [ ] **Step 6: コミットする** + +```bash +pnpm fmt && pnpm typecheck && pnpm exec vitest run +git add -A src/ +git commit -m "feat!: add hibernation auto-response, SQLite backend assertion, and destroy API" +``` + +--- + +### Task 9: 公開 API を整える + +`updateYDoc` が生の Yjs update を受け取るようにして `getYDoc()` との往復を成立させ（H-3）、`yRoute` を `getByName` に切り替える。 + +**Files:** +- Modify: `src/yjs/index.ts`（`updateYDoc`） +- Modify: `src/index.ts:16`（`idFromName` → `getByName`） +- Modify: `src/e2e/index.ts:25-26,45-46,55-56`（同上） +- Test: `src/e2e/y-durableobjects.test.ts`（`updateYDoc` のテストを書き換え） + +**Interfaces:** +- Consumes: なし +- Produces: `YDurableObjects.updateYDoc(update: Uint8Array): Promise` が **生の Yjs update** を受け取る（v1 はプロトコル framing 済みメッセージを要求していた） + +- [ ] **Step 1: 往復テストを書く** + +`src/e2e/y-durableobjects.test.ts` の `"updates YDoc correctly"` を以下に置き換える。 + +```ts +it("round-trips between getYDoc and updateYDoc", async () => { + const source = env.Y_DURABLE_OBJECTS.get(env.Y_DURABLE_OBJECTS.newUniqueId()); + const target = env.Y_DURABLE_OBJECTS.get(env.Y_DURABLE_OBJECTS.newUniqueId()); + + const message = createYDocMessage("Hello World!"); + await runInDurableObject(source, async (instance: InternalYDurableObject) => { + await instance.updateYDoc(message.slice(0)); + }); + + const exported = await runInDurableObject( + source, + (instance: InternalYDurableObject) => instance.getYDoc(), + ); + await runInDurableObject(target, async (instance: InternalYDurableObject) => { + // getYDoc の出力をそのまま updateYDoc に渡せる + await instance.updateYDoc(exported); + }); + + const copied = await runInDurableObject( + target, + (instance: InternalYDurableObject) => instance.getYDoc(), + ); + + const doc = new Doc(); + applyUpdate(doc, copied); + expect(doc.getText("root").toString()).toBe("Hello World!"); +}); +``` + +ファイル先頭に import を追加する。 + +```ts +import { Doc, applyUpdate } from "yjs"; +``` + +- [ ] **Step 2: テストが失敗することを確認する** + +Run: `pnpm exec vitest run src/e2e/y-durableobjects.test.ts -t "round-trips"` +Expected: FAIL — `updateYDoc` が生の update をプロトコルメッセージとして解釈しようとして失敗する + +- [ ] **Step 3: updateYDoc を生の update を受け取るように変更する** + +```ts + /** + * 生の Yjs update を適用する。 + * WebSocket 経路と違い、プロトコルのフレーミングは不要。 + * getYDoc() の戻り値をそのまま渡せる。 + */ + async updateYDoc(update: Uint8Array): Promise { + applyUpdate(this.doc, update, RPC_ORIGIN); + await this.persist; + await this.cleanup(); + } +``` + +- [ ] **Step 4: 他のテストの updateYDoc 呼び出しを修正する** + +`createSyncMessage` でラップして `updateYDoc` に渡している箇所（Task 7・Task 8 で追加したテストを含む）を、生の update を渡すように修正する。`webSocketMessage` に渡す箇所は引き続き `createSyncMessage` が必要である。 + +- [ ] **Step 5: getByName に切り替える** + +`src/index.ts:15-16` を置き換える。 + +```ts + const obj = selector(c.env as E["Bindings"]); + const stub = obj.getByName(c.req.param("id")); +``` + +`src/e2e/index.ts` の 3 箇所（`:25-26`, `:45-46`, `:55-56`）も同様に `const stub = c.env.Y_DURABLE_OBJECTS.getByName(roomId);` に置き換える。 + +- [ ] **Step 6: テストが通ることを確認する** + +Run: `pnpm typecheck && pnpm exec vitest run` +Expected: PASS + +- [ ] **Step 7: コミットする** + +```bash +pnpm fmt && pnpm typecheck && pnpm exec vitest run +git add -A src/ +git commit -m "fix!: make updateYDoc accept a raw Yjs update so it round-trips with getYDoc" +``` + +--- + +### Task 10: ドキュメントとリリース準備 + +README を SQLite 前提に更新し、移行手順を掲載し、changeset を追加する。 + +**Files:** +- Modify: `README.md` +- Modify: `CLAUDE.md`（アーキテクチャ節のストレージ記述） +- Create: `.changeset/sqlite-migration.md` + +**Interfaces:** +- Consumes: Task 1〜9 のすべて +- Produces: なし + +- [ ] **Step 1: README の wrangler 設定を更新する** + +`README.md` の "Configuration for Durable Objects" 節の TOML を置き換える。 + +```toml +name = "your-worker-name" +main = "src/index.ts" +compatibility_date = "2025-04-01" + +account_id = "your-account-id" +workers_dev = true + +# Durable Objects binding +[durable_objects] +bindings = [ + { name = "Y_DURABLE_OBJECTS", class_name = "YDurableObjects" } +] + +# Durable Objects migrations +# v2 requires the SQLite storage backend. +[[migrations]] +tag = "v1" +new_sqlite_classes = ["YDurableObjects"] +``` + +- [ ] **Step 2: README に移行手順の節を追加する** + +`## Usage` の直前に以下を挿入する。アンカーは `assertSqliteBackend` が案内する URL と一致させること（`#migrating-from-v1-key-value-backend`）。 + +```markdown +## Migrating from v1 (key-value backend) + +v2 requires the SQLite storage backend. A Durable Object namespace's storage +type is immutable, so an existing v1 namespace cannot be converted in place — +Cloudflare rejects it with `storage_type_mismatch`. + +Run both versions side by side and copy each room across: + +1. Keep your existing v1 binding (for example `Y_LEGACY`) pinned to + `y-durableobjects@1`. +2. Add a new binding backed by v2 with `new_sqlite_classes`. +3. Copy each room over. `getYDoc()` returns a raw Yjs update and v2's + `updateYDoc()` accepts one, so a single round trip is enough: + +```typescript +app.post("/migrate/:id", async (c) => { + const id = c.req.param("id"); + const legacy = c.env.Y_LEGACY.getByName(id); + const next = c.env.Y_DURABLE_OBJECTS.getByName(id); + + await next.updateYDoc(await legacy.getYDoc()); + + return c.json({ ok: true }); +}); +``` + +4. Once every room is copied, remove the v1 binding. + +### Document size + +Durable Objects give each instance 10GB of SQLite storage, but the whole +document must fit in the instance's 128MB of memory. That memory limit — not +storage — is the practical ceiling on document size. + +### Keepalive and hibernation + +v2 registers a `"ping"` / `"pong"` auto-response. If your client sends `"ping"` +as a keepalive, the Durable Object answers without waking from hibernation, +which is the single biggest lever on duration billing. +``` + +- [ ] **Step 3: README の updateYDoc の例を修正する** + +`#### updateYDoc` 節の説明に、生の Yjs update を渡すことを明記する。既存のサンプルはリクエストボディをそのまま渡しているため、v2 の挙動と一致するようになる。以下の一文を追加する。 + +```markdown +`updateYDoc` takes a raw Yjs update — the same format `getYDoc` returns and +`Y.encodeStateAsUpdate(doc)` produces. It is not a sync-protocol message. +``` + +- [ ] **Step 4: CLAUDE.md のアーキテクチャ記述を更新する** + +`3. **YTransactionStorage** (`src/yjs/storage/index.ts`)` の項目を置き換える。 + +```markdown +3. **YSqliteStorage** (`src/yjs/storage/sqlite.ts`) + + - Persistence layer backed by the Durable Objects SQLite storage backend + - Single `updates` table; snapshots and incremental updates are not distinguished + - Compacts with `Y.mergeUpdates` on a row-count threshold, splitting the + result into chunks when it exceeds the SQLite BLOB limit + - `PRAGMA` is unavailable on Durable Objects SQLite; schema versioning uses a + `schema_version` table (`src/yjs/storage/schema.ts`) +``` + +同ファイルの `Development Constraints` に以下を追記する。 + +```markdown +4. **SQLite Storage Backend** + + - Requires `new_sqlite_classes` in wrangler migrations + - BLOB columns are returned as `ArrayBuffer`; convert with `new Uint8Array(value)` + - Consecutive synchronous `sql.exec` calls with no intervening `await` form an + implicit transaction — do not await inside a compaction +``` + +- [ ] **Step 5: changeset を追加する** + +```markdown + +--- +"y-durableobjects": major +--- + +Migrate to the Durable Objects SQLite storage backend and fix the defects the +key-value backend had forced. + +**Breaking changes** + +- Requires `new_sqlite_classes` in your wrangler migrations. A v1 namespace + cannot be converted in place — see "Migrating from v1" in the README. +- `updateYDoc()` now takes a raw Yjs update instead of a sync-protocol message, + so it round-trips with `getYDoc()`. +- The exported `YTransactionStorage` type is replaced by `YStorage`. +- `WSSharedDoc.notify(listener)` is now `notify(origin, listener)` and + `WSSharedDoc.update(message)` is now `update(message, origin)`. +- `WebSocketAttachment` is replaced by `SessionAttachment`, which carries the + connection's awareness client ids. + +**Fixes** + +- Documents are no longer capped at 128KiB. +- Compaction no longer exceeds the 128-key limit of `delete()`. +- Closing one connection no longer clears every participant's awareness state. +- Updates are persisted in order and awaited rather than left as floating promises. +- Sync step 2 replies go only to the requesting client instead of the whole room. +- Updates are no longer echoed back to their sender. +- A malformed binary message closes only that connection instead of resetting + the Durable Object. +- Stored updates are restored in insertion order. + +**Additions** + +- `destroy()` deletes a room's data and closes its connections. +- A `"ping"` / `"pong"` auto-response keeps keepalives from waking the Durable + Object from hibernation. +``` + +- [ ] **Step 6: 最終確認とコミット** + +```bash +pnpm fmt && pnpm typecheck && pnpm lint && pnpm exec vitest run && pnpm build +git add -A +git commit -m "docs: document the SQLite migration and add the v2 changeset" +``` + +--- + +## Self-Review + +**Spec coverage:** + +| Spec の要求 | 対応タスク | +| --- | --- | +| C-1 SQLite バックエンドへの移行 | Task 1, 4 | +| C-2 128KiB 制限の解消 | Task 3（300KB ドキュメントのテスト） | +| C-3 delete 128 キー制限の解消 | Task 3（`DELETE FROM updates` 1 文） | +| C-4 awareness の所有権 | Task 6 | +| C-5 永続化の直列化 | Task 7 | +| H-1 syncStep2 を要求元にのみ | Task 5 | +| H-2 エコーバックの停止 | Task 5 | +| H-3 `updateYDoc` の往復 | Task 9 | +| H-4 例外境界 | Task 7 | +| H-5 更新順序 | Task 2（1000 件の順序テスト） | +| H-7 メッセージ型の拡張 | Task 5 | +| `schema_version` マイグレーション | Task 1 | +| チャンク分割 | Task 3 | +| `maxRows` / `maxChunkBytes` | Task 2, 3 | +| hibernation 衛生 | Task 8 | +| KV バックエンド検出 | Task 8 | +| `destroy()` | Task 8 | +| `getByName` | Task 9 | +| 移行レシピの文書化 | Task 10 | +| 公開型の変更 | Task 4（`YStorage`）、Task 5（`WSSharedDoc`）、Task 6（`SessionAttachment`） | + +すべての spec 要求にタスクが対応している。 + +**型の整合性:** + +- `YStorage` は Task 2 で定義し、Task 4 でエクスポートする。メソッド名は全タスクで `getUpdate` / `storeUpdate` / `commit` / `destroy` に統一されている。 +- `SessionRegistry` のメソッド名は Task 6 の定義（`add` / `remove` / `has` / `sockets` / `clientIdsOf` / `track` / `size`）で、Task 7・8 のテストからも同じ名前で参照している。 +- `UpdateKind.standalone` / `UpdateKind.continuation` は Task 2 で定義し Task 3 で使用。 +- `RPC_ORIGIN` は Task 5 で定義し Task 9 で再利用。 +- `WSSharedDoc.notify(origin, listener)` の 2 引数シグネチャは Task 5 で導入し、Task 6 の `registerWebSocket` でも同じ順序で呼んでいる。 + +**残る不確実性:** + +`assertSqliteBackend` が KV バックエンドを実際に検出できるかは、KV バックエンドの Durable Object を用意できないため自動テストでは検証できない。Task 8 では SQLite バックエンド上で例外を投げないことのみを確認する。KV バックエンド上での挙動は、v2 リリース前に手動で 1 度確認することが望ましい。 From c0d42e720304cffd837b8de5bf15a3a935c89cd6 Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 20:11:56 +0900 Subject: [PATCH 04/26] feat: switch to SQLite storage backend and add schema migration runner Also register SqlStorage as an ESLint global (eslint.config.js), mirroring the existing DurableObjectState entry, since it is an ambient type from worker-configuration.d.ts used without import in schema.ts. --- eslint.config.js | 1 + src/yjs/storage/schema.test.ts | 64 ++++++++++++++++++++++++++++++++++ src/yjs/storage/schema.ts | 45 ++++++++++++++++++++++++ wrangler.toml | 4 +-- 4 files changed, 112 insertions(+), 2 deletions(-) create mode 100644 src/yjs/storage/schema.test.ts create mode 100644 src/yjs/storage/schema.ts diff --git a/eslint.config.js b/eslint.config.js index 7990ab0..6b148e6 100644 --- a/eslint.config.js +++ b/eslint.config.js @@ -77,6 +77,7 @@ const config = [ DurableObjectStub: "readonly", DurableObjectTransaction: "readonly", DurableObjectState: "readonly", + SqlStorage: "readonly", WebSocketPair: "readonly", RequestInfo: "readonly", }, diff --git a/src/yjs/storage/schema.test.ts b/src/yjs/storage/schema.test.ts new file mode 100644 index 0000000..40a8922 --- /dev/null +++ b/src/yjs/storage/schema.test.ts @@ -0,0 +1,64 @@ +import { env, runInDurableObject } from "cloudflare:test"; +import { describe, expect, it } from "vitest"; + +import { migrate } from "./schema"; + +const withSql = async (fn: (sql: SqlStorage) => void): Promise => { + const stub = env.Y_DURABLE_OBJECTS.get(env.Y_DURABLE_OBJECTS.newUniqueId()); + await runInDurableObject(stub, (_instance, state) => { + fn(state.storage.sql); + }); +}; + +describe("migrate", () => { + it("creates the updates table", async () => { + await withSql((sql) => { + migrate(sql); + + const tables = sql + .exec<{ name: string }>( + "SELECT name FROM sqlite_master WHERE type = 'table'", + ) + .toArray() + .map((row) => row.name); + + expect(tables).toContain("updates"); + expect(tables).toContain("schema_version"); + }); + }); + + it("records the schema version", async () => { + await withSql((sql) => { + migrate(sql); + + const row = sql + .exec<{ version: number }>("SELECT version FROM schema_version") + .one(); + + expect(row.version).toBe(1); + }); + }); + + it("is idempotent and preserves existing rows", async () => { + await withSql((sql) => { + migrate(sql); + sql.exec( + "INSERT INTO updates (kind, data) VALUES (?, ?)", + 0, + new Uint8Array([1, 2, 3]), + ); + + migrate(sql); + + const count = sql + .exec<{ count: number }>("SELECT COUNT(*) AS count FROM updates") + .one(); + const version = sql + .exec<{ version: number }>("SELECT version FROM schema_version") + .one(); + + expect(count.count).toBe(1); + expect(version.version).toBe(1); + }); + }); +}); diff --git a/src/yjs/storage/schema.ts b/src/yjs/storage/schema.ts new file mode 100644 index 0000000..037d12f --- /dev/null +++ b/src/yjs/storage/schema.ts @@ -0,0 +1,45 @@ +/** + * スキーマのマイグレーション定義。 + * 変更するときは既存の要素を書き換えず、末尾に ALTER TABLE を追記すること。 + * Durable Object の SQLite はインスタンスごとに独立した DB なので、 + * 各インスタンスが初回起動時に自分のペースでマイグレートする。 + */ +const MIGRATIONS: readonly string[] = [ + `CREATE TABLE updates ( + seq INTEGER PRIMARY KEY AUTOINCREMENT, + kind INTEGER NOT NULL, + data BLOB NOT NULL + )`, +]; + +/** + * 未適用のマイグレーションを適用する。すべて同期実行なので、 + * 呼び出し中に await を挟まなければ暗黙のトランザクションとして atomic に完了する。 + * + * PRAGMA user_version は Durable Objects の SQLite では SQLITE_AUTH で拒否されるため、 + * 版番号は schema_version テーブルに保持する。 + */ +export const migrate = (sql: SqlStorage): void => { + sql.exec( + "CREATE TABLE IF NOT EXISTS schema_version (version INTEGER NOT NULL)", + ); + + const current = sql + .exec<{ version: number }>("SELECT version FROM schema_version") + .toArray() + .at(0); + const applied = current?.version ?? 0; + + for (let i = applied; i < MIGRATIONS.length; i++) { + sql.exec(MIGRATIONS[i]); + } + + if (current === undefined) { + sql.exec( + "INSERT INTO schema_version (version) VALUES (?)", + MIGRATIONS.length, + ); + } else if (applied !== MIGRATIONS.length) { + sql.exec("UPDATE schema_version SET version = ?", MIGRATIONS.length); + } +}; diff --git a/wrangler.toml b/wrangler.toml index 14c9cf4..50492b9 100644 --- a/wrangler.toml +++ b/wrangler.toml @@ -1,6 +1,6 @@ name = "yjs-workers" main = "src/e2e/index.ts" -compatibility_date = "2024-04-05" +compatibility_date = "2025-04-01" compatibility_flags=["nodejs_compat"] [[durable_objects.bindings]] @@ -9,5 +9,5 @@ class_name = "YDurableObjects" [[migrations]] tag = "v1" -new_classes = ["YDurableObjects"] +new_sqlite_classes = ["YDurableObjects"] From b1adb939a19bce1fd7eb8935aaff78f21e005f70 Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 20:17:21 +0900 Subject: [PATCH 05/26] feat: add SQLite-backed Yjs storage with ordered update log Adds YSqliteStorage implementing the new YStorage interface (getUpdate, storeUpdate, commit, destroy) backed by the `updates` table from Task 1's schema migration. Reads restore updates in seq order and merge continuation fragments; writes insert standalone rows. commit() is a placeholder for Task 3's compaction. The legacy KV-backed TransactionStorage is untouched and both implementations coexist until Task 4 swaps the Durable Object over. Also ignores docs/superpowers and .superpowers in Prettier so `pnpm fmt` stops reformatting the plan/spec markdown on every task. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015mdNeCzNPZysRHzmkFp9Bj --- .prettierignore | 2 + src/yjs/storage/queries.ts | 9 +++ src/yjs/storage/sqlite.test.ts | 81 +++++++++++++++++++++++ src/yjs/storage/sqlite.ts | 115 +++++++++++++++++++++++++++++++++ src/yjs/storage/type.ts | 23 +++++++ 5 files changed, 230 insertions(+) create mode 100644 src/yjs/storage/queries.ts create mode 100644 src/yjs/storage/sqlite.test.ts create mode 100644 src/yjs/storage/sqlite.ts diff --git a/.prettierignore b/.prettierignore index 5dbb561..05e258e 100644 --- a/.prettierignore +++ b/.prettierignore @@ -4,3 +4,5 @@ node_modules pnpm-lock.yaml dist worker-configuration.d.ts +docs/superpowers +.superpowers diff --git a/src/yjs/storage/queries.ts b/src/yjs/storage/queries.ts new file mode 100644 index 0000000..0b8ce76 --- /dev/null +++ b/src/yjs/storage/queries.ts @@ -0,0 +1,9 @@ +/** + * SQL 文字列をこのファイルに隔離する。 + * 将来スキーマが複雑になり Kysely 等のクエリビルダを導入する場合も、 + * 変更はこのファイル内で完結する。 + */ +export const SELECT_ALL_UPDATES = "SELECT kind, data FROM updates ORDER BY seq"; +export const INSERT_UPDATE = "INSERT INTO updates (kind, data) VALUES (?, ?)"; +export const DELETE_ALL_UPDATES = "DELETE FROM updates"; +export const COUNT_UPDATES = "SELECT COUNT(*) AS count FROM updates"; diff --git a/src/yjs/storage/sqlite.test.ts b/src/yjs/storage/sqlite.test.ts new file mode 100644 index 0000000..c3dbd39 --- /dev/null +++ b/src/yjs/storage/sqlite.test.ts @@ -0,0 +1,81 @@ +import { env, runInDurableObject } from "cloudflare:test"; +import { describe, expect, it } from "vitest"; +import { Doc, applyUpdate, encodeStateAsUpdate } from "yjs"; + +import { YSqliteStorage } from "./sqlite"; + +import type { YSqliteStorageOptions } from "./sqlite"; + +const withStorage = async ( + fn: (storage: YSqliteStorage) => Promise, + options?: YSqliteStorageOptions, +): Promise => { + const stub = env.Y_DURABLE_OBJECTS.get(env.Y_DURABLE_OBJECTS.newUniqueId()); + await runInDurableObject(stub, async (_instance, state) => { + await fn(new YSqliteStorage(state.storage.sql, options)); + }); +}; + +const docWith = (text: string): Doc => { + const doc = new Doc(); + doc.getText("root").insert(0, text); + + return doc; +}; + +const textOf = (update: Uint8Array): string => { + const doc = new Doc(); + applyUpdate(doc, update); + + return doc.getText("root").toString(); +}; + +describe("YSqliteStorage", () => { + it("returns null when nothing has been stored", async () => { + await withStorage(async (storage) => { + expect(await storage.getUpdate()).toBeNull(); + }); + }); + + it("round-trips a single update", async () => { + await withStorage(async (storage) => { + await storage.storeUpdate(encodeStateAsUpdate(docWith("Hello World!"))); + + const update = await storage.getUpdate(); + + expect(update).not.toBeNull(); + expect(textOf(update!)).toBe("Hello World!"); + }); + }); + + it("applies stored updates in insertion order", async () => { + await withStorage(async (storage) => { + const doc = new Doc(); + const text = doc.getText("root"); + + // 1000 件の更新を個別に保存する。キーの辞書順ではなく seq 順で + // 復元されることを検証する（H-5 の回帰テスト）。 + const updates: Uint8Array[] = []; + doc.on("update", (update: Uint8Array) => updates.push(update)); + for (let i = 0; i < 1000; i++) { + text.insert(text.length, String(i % 10)); + } + for (const update of updates) { + await storage.storeUpdate(update); + } + + const restored = await storage.getUpdate(); + + expect(textOf(restored!)).toBe(text.toString()); + }); + }); + + it("clears all rows on destroy", async () => { + await withStorage(async (storage) => { + await storage.storeUpdate(encodeStateAsUpdate(docWith("gone"))); + await storage.destroy(); + + expect(await storage.getUpdate()).toBeNull(); + }); + }); +}); diff --git a/src/yjs/storage/sqlite.ts b/src/yjs/storage/sqlite.ts new file mode 100644 index 0000000..fb99cf6 --- /dev/null +++ b/src/yjs/storage/sqlite.ts @@ -0,0 +1,115 @@ +import { mergeUpdates } from "yjs"; + +import { + COUNT_UPDATES, + DELETE_ALL_UPDATES, + INSERT_UPDATE, + SELECT_ALL_UPDATES, +} from "./queries"; +import { migrate } from "./schema"; +import { UpdateKind } from "./type"; + +import type { YStorage } from "./type"; + +type UpdateRow = { + kind: number; + data: ArrayBuffer; +}; + +export type YSqliteStorageOptions = { + /** + * この行数を超えたらコンパクションする。 + * @default 2000 + */ + maxRows?: number; + /** + * コンパクション結果を分割する単位（バイト）。 + * SQLite の BLOB 上限 2MB に対する安全マージンを取る。 + * @default 1024 * 1024 + */ + maxChunkBytes?: number; +}; + +const DEFAULT_MAX_ROWS = 2000; +const DEFAULT_MAX_CHUNK_BYTES = 1024 * 1024; +const SQLITE_BLOB_LIMIT = 2 * 1024 * 1024; + +const concat = (parts: readonly Uint8Array[]): Uint8Array => { + if (parts.length === 1) return parts[0]; + + const total = parts.reduce((sum, part) => sum + part.byteLength, 0); + const merged = new Uint8Array(total); + let offset = 0; + for (const part of parts) { + merged.set(part, offset); + offset += part.byteLength; + } + + return merged; +}; + +export class YSqliteStorage implements YStorage { + readonly #sql: SqlStorage; + // eslint-disable-next-line no-unused-private-class-members -- read by commit() in Task 3 + readonly #maxRows: number; + readonly #maxChunkBytes: number; + // eslint-disable-next-line no-unused-private-class-members -- read by commit() in Task 3 + #rowCount: number; + + constructor(sql: SqlStorage, options?: YSqliteStorageOptions) { + this.#maxChunkBytes = options?.maxChunkBytes ?? DEFAULT_MAX_CHUNK_BYTES; + if (this.#maxChunkBytes > SQLITE_BLOB_LIMIT) { + // https://developers.cloudflare.com/durable-objects/platform/limits/ + throw new Error("maxChunkBytes must not exceed 2MB"); + } + this.#maxRows = options?.maxRows ?? DEFAULT_MAX_ROWS; + + this.#sql = sql; + migrate(sql); + this.#rowCount = sql.exec<{ count: number }>(COUNT_UPDATES).one().count; + } + + async getUpdate(): Promise { + const updates = this.#readAll(); + if (updates.length === 0) return null; + + return mergeUpdates(updates); + } + + async storeUpdate(update: Uint8Array): Promise { + this.#sql.exec(INSERT_UPDATE, UpdateKind.standalone, update); + this.#rowCount += 1; + } + + async commit(): Promise { + // Task 3 で実装する + } + + async destroy(): Promise { + this.#sql.exec(DELETE_ALL_UPDATES); + this.#rowCount = 0; + } + + /** + * 全行を読み、continuation の断片を連結して独立した update の配列に戻す。 + * BLOB は ArrayBuffer で返るため Uint8Array へ変換する。 + */ + #readAll(): Uint8Array[] { + const rows = this.#sql.exec(SELECT_ALL_UPDATES).toArray(); + + const updates: Uint8Array[] = []; + let pending: Uint8Array[] = []; + for (const row of rows) { + const bytes = new Uint8Array(row.data); + if (row.kind === UpdateKind.continuation && pending.length > 0) { + pending.push(bytes); + continue; + } + if (pending.length > 0) updates.push(concat(pending)); + pending = [bytes]; + } + if (pending.length > 0) updates.push(concat(pending)); + + return updates; + } +} diff --git a/src/yjs/storage/type.ts b/src/yjs/storage/type.ts index 4bc86af..574f0bc 100644 --- a/src/yjs/storage/type.ts +++ b/src/yjs/storage/type.ts @@ -16,3 +16,26 @@ export interface TransactionStorage { closure: (txn: Omit) => Promise, ): Promise; } + +/** + * updates テーブルの kind カラムの値。 + * + * Yjs の update はバイト列として単純に分割・連結できないため、 + * コンパクション結果が BLOB 上限を超える場合はバイト断片に分割して保存する。 + * continuation は「直前の行から続く断片」であることを示す。 + */ +export const UpdateKind = { + standalone: 0, + continuation: 1, +} as const; + +export interface YStorage { + /** 保存されているすべての update を 1 本にマージして返す。空なら null */ + getUpdate(): Promise; + /** 増分 update を 1 行追加する。しきい値を超えたらコンパクションする */ + storeUpdate(update: Uint8Array): Promise; + /** 明示的にコンパクションする */ + commit(): Promise; + /** すべての update を削除する。テーブル定義とスキーマ版は維持する */ + destroy(): Promise; +} From 711ddfb6f09430be9b7232c4b3b1c7ab04a6cd44 Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 20:23:15 +0900 Subject: [PATCH 06/26] fix: surface orphaned continuation rows instead of silent corruption Review fix round 1 for Task 2. #readAll previously fell through to treating a kind=1 continuation row with no preceding row as a standalone update, feeding raw fragment bytes to Y.mergeUpdates. It now throws an explicit error naming the offending seq. SELECT_ALL_UPDATES and UpdateRow gained the seq column needed to report it, and a regression test inserts a malformed row directly to cover the corrupt-table path. Also rewrites the misleading comment on the 1000-update round-trip test: it never proved seq-ordered restoration (Y.mergeUpdates converges regardless of update arrival order), so the comment and test name now describe what the test actually covers. Genuine ordering coverage lands in Task 3. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015mdNeCzNPZysRHzmkFp9Bj --- src/yjs/storage/queries.ts | 3 ++- src/yjs/storage/sqlite.test.ts | 28 +++++++++++++++++++++++++--- src/yjs/storage/sqlite.ts | 16 +++++++++++++++- 3 files changed, 42 insertions(+), 5 deletions(-) diff --git a/src/yjs/storage/queries.ts b/src/yjs/storage/queries.ts index 0b8ce76..9243da8 100644 --- a/src/yjs/storage/queries.ts +++ b/src/yjs/storage/queries.ts @@ -3,7 +3,8 @@ * 将来スキーマが複雑になり Kysely 等のクエリビルダを導入する場合も、 * 変更はこのファイル内で完結する。 */ -export const SELECT_ALL_UPDATES = "SELECT kind, data FROM updates ORDER BY seq"; +export const SELECT_ALL_UPDATES = + "SELECT seq, kind, data FROM updates ORDER BY seq"; export const INSERT_UPDATE = "INSERT INTO updates (kind, data) VALUES (?, ?)"; export const DELETE_ALL_UPDATES = "DELETE FROM updates"; export const COUNT_UPDATES = "SELECT COUNT(*) AS count FROM updates"; diff --git a/src/yjs/storage/sqlite.test.ts b/src/yjs/storage/sqlite.test.ts index c3dbd39..182d30d 100644 --- a/src/yjs/storage/sqlite.test.ts +++ b/src/yjs/storage/sqlite.test.ts @@ -48,13 +48,17 @@ describe("YSqliteStorage", () => { }); }); - it("applies stored updates in insertion order", async () => { + it("round-trips a large number of individually stored updates", async () => { await withStorage(async (storage) => { const doc = new Doc(); const text = doc.getText("root"); - // 1000 件の更新を個別に保存する。キーの辞書順ではなく seq 順で - // 復元されることを検証する（H-5 の回帰テスト）。 + // 1000 件の更新を個別に保存し、まとめて復元しても元のドキュメントと + // 一致することを検証する（大量行のバルクラウンドトリップ）。 + // 注意: Y.mergeUpdates は因果的に依存する struct を並べ替えて解決する + // ため、この結果だけでは seq 順に復元されていることは証明されない。 + // ORDER BY seq を外しても本テストは通ってしまう。順序に依存する + // 検証（バイト断片の再構成）は Task 3 で追加する。 const updates: Uint8Array[] = []; doc.on("update", (update: Uint8Array) => updates.push(update)); for (let i = 0; i < 1000; i++) { @@ -78,4 +82,22 @@ describe("YSqliteStorage", () => { expect(await storage.getUpdate()).toBeNull(); }); }); + + it("throws when a continuation row has no preceding row to attach to", async () => { + const stub = env.Y_DURABLE_OBJECTS.get(env.Y_DURABLE_OBJECTS.newUniqueId()); + await runInDurableObject(stub, async (_instance, state) => { + const storage = new YSqliteStorage(state.storage.sql); + + // 破損した状態を直接作る: 先行する standalone 行なしに continuation + // (kind = 1) 行だけを挿入する。 + state.storage.sql.exec( + "INSERT INTO updates (kind, data) VALUES (1, ?)", + new Uint8Array([1, 2, 3]), + ); + + await expect(storage.getUpdate()).rejects.toThrow( + /orphaned continuation row at seq=\d+/, + ); + }); + }); }); diff --git a/src/yjs/storage/sqlite.ts b/src/yjs/storage/sqlite.ts index fb99cf6..7dd3b66 100644 --- a/src/yjs/storage/sqlite.ts +++ b/src/yjs/storage/sqlite.ts @@ -12,6 +12,7 @@ import { UpdateKind } from "./type"; import type { YStorage } from "./type"; type UpdateRow = { + seq: number; kind: number; data: ArrayBuffer; }; @@ -101,7 +102,20 @@ export class YSqliteStorage implements YStorage { let pending: Uint8Array[] = []; for (const row of rows) { const bytes = new Uint8Array(row.data); - if (row.kind === UpdateKind.continuation && pending.length > 0) { + if (row.kind === UpdateKind.continuation) { + if (pending.length === 0) { + // A continuation row with nothing preceding it to attach to means + // the updates table itself is corrupt (truncated compaction, + // manual tampering, a bug elsewhere). Silently reinterpreting the + // fragment as a standalone update would hand raw fragment bytes + // to Y.mergeUpdates and either corrupt the document without any + // signal or fail later with a confusing decode error far from the + // real cause. A storage library must surface its own corruption + // loudly instead of guessing, so we fail here and name the row. + throw new Error( + `YSqliteStorage: orphaned continuation row at seq=${row.seq} has no preceding row to attach to`, + ); + } pending.push(bytes); continue; } From 83177b809a79d07a936e7181294523c8a064b304 Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 20:27:15 +0900 Subject: [PATCH 07/26] feat: compact update log with chunk splitting for large documents Implements commit() to merge accumulated update rows once maxRows is exceeded and split the merged result into maxChunkBytes fragments, removing the legacy 128KiB key-value ceiling. Adds a byte-equality regression test for chunk reassembly ordering, since ordering is the one place concatenating fragments out of seq order silently corrupts data. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015mdNeCzNPZysRHzmkFp9Bj --- src/yjs/storage/sqlite.test.ts | 102 ++++++++++++++++++++++++++++++++- src/yjs/storage/sqlite.ts | 48 +++++++++++++++- 2 files changed, 146 insertions(+), 4 deletions(-) diff --git a/src/yjs/storage/sqlite.test.ts b/src/yjs/storage/sqlite.test.ts index 182d30d..c3a9733 100644 --- a/src/yjs/storage/sqlite.test.ts +++ b/src/yjs/storage/sqlite.test.ts @@ -1,6 +1,6 @@ import { env, runInDurableObject } from "cloudflare:test"; import { describe, expect, it } from "vitest"; -import { Doc, applyUpdate, encodeStateAsUpdate } from "yjs"; +import { Doc, applyUpdate, encodeStateAsUpdate, mergeUpdates } from "yjs"; import { YSqliteStorage } from "./sqlite"; @@ -16,6 +16,20 @@ const withStorage = async ( }); }; +const withStorageAndSql = async ( + fn: (storage: YSqliteStorage, sql: SqlStorage) => Promise, + options?: YSqliteStorageOptions, +): Promise => { + const stub = env.Y_DURABLE_OBJECTS.get(env.Y_DURABLE_OBJECTS.newUniqueId()); + await runInDurableObject(stub, async (_instance, state) => { + await fn(new YSqliteStorage(state.storage.sql, options), state.storage.sql); + }); +}; + +const countRows = (sql: SqlStorage): number => + sql.exec<{ count: number }>("SELECT COUNT(*) AS count FROM updates").one() + .count; + const docWith = (text: string): Doc => { const doc = new Doc(); doc.getText("root").insert(0, text); @@ -83,6 +97,92 @@ describe("YSqliteStorage", () => { }); }); + it("compacts once the row threshold is exceeded", async () => { + await withStorageAndSql( + async (storage, sql) => { + const doc = new Doc(); + const text = doc.getText("root"); + const updates: Uint8Array[] = []; + doc.on("update", (update: Uint8Array) => updates.push(update)); + for (let i = 0; i < 50; i++) { + text.insert(text.length, "x"); + } + for (const update of updates) { + await storage.storeUpdate(update); + } + + // maxRows 10 に対し 50 件保存したので、行数は大幅に減っているはず + expect(countRows(sql)).toBeLessThanOrEqual(10); + + const restored = await storage.getUpdate(); + expect(textOf(restored!)).toBe(text.toString()); + }, + { maxRows: 10 }, + ); + }); + + it("splits a compacted update that exceeds maxChunkBytes and restores it", async () => { + await withStorageAndSql( + async (storage, sql) => { + const doc = new Doc(); + const text = doc.getText("root"); + const updates: Uint8Array[] = []; + doc.on("update", (update: Uint8Array) => updates.push(update)); + for (let i = 0; i < 20; i++) { + text.insert(text.length, "abcdefghij".repeat(50)); + } + for (const update of updates) { + await storage.storeUpdate(update); + } + + // This is the ordering regression test for the compaction/chunk-split + // path. Byte-fragment reassembly is the one place in this library + // where ordering is genuinely load-bearing: concatenating chunks out + // of `seq` order produces garbage bytes that no amount of CRDT + // convergence in Y.mergeUpdates repairs. Asserting on the restored + // *text* only proves the document round-trips (mergeUpdates could + // in principle mask a reordering that a stricter comparison would + // catch), so we additionally capture the merged bytes before + // compaction and require getUpdate() to reproduce them exactly. + const expectedBytes = mergeUpdates(updates); + + await storage.commit(); + + // 512 バイトずつに分割されるので、複数行になっているはず + expect(countRows(sql)).toBeGreaterThan(1); + + const restored = await storage.getUpdate(); + expect(textOf(restored!)).toBe(text.toString()); + expect(restored!).toEqual(expectedBytes); + }, + { maxRows: 1000, maxChunkBytes: 512 }, + ); + }); + + it("stores a document larger than the legacy 128KiB key-value limit", async () => { + await withStorage(async (storage) => { + const doc = new Doc(); + // 300KB 相当。KV バックエンドでは 1 キーに収まらず保存に失敗していた（C-2） + doc.getText("root").insert(0, "y".repeat(300 * 1024)); + await storage.storeUpdate(encodeStateAsUpdate(doc)); + await storage.commit(); + + const restored = await storage.getUpdate(); + + expect(textOf(restored!).length).toBe(300 * 1024); + }); + }); + + it("is a no-op when there is at most one row to compact", async () => { + await withStorageAndSql(async (storage, sql) => { + await storage.storeUpdate(encodeStateAsUpdate(docWith("single"))); + await storage.commit(); + + expect(countRows(sql)).toBe(1); + expect(textOf((await storage.getUpdate())!)).toBe("single"); + }); + }); + it("throws when a continuation row has no preceding row to attach to", async () => { const stub = env.Y_DURABLE_OBJECTS.get(env.Y_DURABLE_OBJECTS.newUniqueId()); await runInDurableObject(stub, async (_instance, state) => { diff --git a/src/yjs/storage/sqlite.ts b/src/yjs/storage/sqlite.ts index 7dd3b66..3c103f1 100644 --- a/src/yjs/storage/sqlite.ts +++ b/src/yjs/storage/sqlite.ts @@ -51,10 +51,8 @@ const concat = (parts: readonly Uint8Array[]): Uint8Array => { export class YSqliteStorage implements YStorage { readonly #sql: SqlStorage; - // eslint-disable-next-line no-unused-private-class-members -- read by commit() in Task 3 readonly #maxRows: number; readonly #maxChunkBytes: number; - // eslint-disable-next-line no-unused-private-class-members -- read by commit() in Task 3 #rowCount: number; constructor(sql: SqlStorage, options?: YSqliteStorageOptions) { @@ -80,10 +78,29 @@ export class YSqliteStorage implements YStorage { async storeUpdate(update: Uint8Array): Promise { this.#sql.exec(INSERT_UPDATE, UpdateKind.standalone, update); this.#rowCount += 1; + + if (this.#rowCount > this.#maxRows) { + await this.commit(); + } } async commit(): Promise { - // Task 3 で実装する + if (this.#rowCount <= 1) return; + + const updates = this.#readAll(); + if (updates.length === 0) return; + + const chunks = this.#split(mergeUpdates(updates)); + + // ここから下では await を挟まないこと。 + // 連続した同期書き込みが暗黙のトランザクションとして atomic に適用される。 + this.#sql.exec(DELETE_ALL_UPDATES); + for (const [index, chunk] of chunks.entries()) { + const kind = + index === 0 ? UpdateKind.standalone : UpdateKind.continuation; + this.#sql.exec(INSERT_UPDATE, kind, chunk); + } + this.#rowCount = chunks.length; } async destroy(): Promise { @@ -126,4 +143,29 @@ export class YSqliteStorage implements YStorage { return updates; } + + /** + * マージ済みの update を maxChunkBytes 以下のバイト断片に分割する。 + * subarray ではなく slice を使ってコピーを作る。ビューをそのまま + * バインドすると基底バッファ全体が書き込まれる可能性があるため。 + */ + #split(update: Uint8Array): Uint8Array[] { + if (update.byteLength <= this.#maxChunkBytes) return [update]; + + const chunks: Uint8Array[] = []; + for ( + let offset = 0; + offset < update.byteLength; + offset += this.#maxChunkBytes + ) { + chunks.push( + update.slice( + offset, + Math.min(offset + this.#maxChunkBytes, update.byteLength), + ), + ); + } + + return chunks; + } } From 832aad37fadcdef7bfc2ffca7ef3940cf2788ec4 Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 20:33:27 +0900 Subject: [PATCH 08/26] refactor!: replace key-value storage layer with SQLite implementation Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015mdNeCzNPZysRHzmkFp9Bj --- src/index.ts | 2 +- src/yjs/index.ts | 19 +- src/yjs/internal.ts | 4 +- src/yjs/storage/index.ts | 118 +--------- src/yjs/storage/storage-key/index.ts | 13 -- .../storage/storage-key/storage-key.test.ts | 15 -- src/yjs/storage/storage.test.ts | 217 ------------------ src/yjs/storage/type.ts | 19 -- 8 files changed, 15 insertions(+), 392 deletions(-) delete mode 100644 src/yjs/storage/storage-key/index.ts delete mode 100644 src/yjs/storage/storage-key/storage-key.test.ts delete mode 100644 src/yjs/storage/storage.test.ts diff --git a/src/index.ts b/src/index.ts index d039852..6e8b97a 100644 --- a/src/index.ts +++ b/src/index.ts @@ -37,5 +37,5 @@ export const yRoute = (selector: Selector) => { export { YDurableObjects, type YDurableObjectsAppType } from "./yjs"; export type YRoute = ReturnType; -export type { YTransactionStorage } from "./yjs/storage"; +export type { YStorage, YSqliteStorageOptions } from "./yjs/storage"; export { type RemoteDoc, WSSharedDoc } from "./yjs/remote"; diff --git a/src/yjs/index.ts b/src/yjs/index.ts index 684f0bf..d31ed3b 100644 --- a/src/yjs/index.ts +++ b/src/yjs/index.ts @@ -6,7 +6,7 @@ import { WSSharedDoc } from "../yjs/remote"; import { setupWSConnection } from "./client/setup"; import { createApp } from "./hono"; -import { YTransactionStorageImpl } from "./storage"; +import { YSqliteStorage } from "./storage"; import type { AwarenessChanges } from "../yjs/remote"; import type { Env } from "hono"; @@ -25,14 +25,7 @@ export class YDurableObjects extends DurableObject< createRoom: this.createRoom.bind(this), }); protected doc = new WSSharedDoc(); - protected storage = new YTransactionStorageImpl({ - get: (key) => this.state.storage.get(key), - list: (options) => this.state.storage.list(options), - put: (key, value) => this.state.storage.put(key, value), - delete: async (key) => - this.state.storage.delete(Array.isArray(key) ? key : [key]), - transaction: (closure) => this.state.storage.transaction(closure), - }); + protected storage: YSqliteStorage; protected sessions = new Map void>(); private awarenessClients = new Set(); @@ -42,12 +35,16 @@ export class YDurableObjects extends DurableObject< ) { super(state, env); + this.storage = new YSqliteStorage(state.storage.sql); + void this.state.blockConcurrencyWhile(this.onStart.bind(this)); } protected async onStart(): Promise { - const doc = await this.storage.getYDoc(); - applyUpdate(this.doc, encodeStateAsUpdate(doc)); + const update = await this.storage.getUpdate(); + if (update !== null) { + applyUpdate(this.doc, update); + } for (const ws of this.state.getWebSockets()) { this.registerWebSocket(ws); diff --git a/src/yjs/internal.ts b/src/yjs/internal.ts index b2fea57..17d3db6 100644 --- a/src/yjs/internal.ts +++ b/src/yjs/internal.ts @@ -1,10 +1,10 @@ import type { WSSharedDoc } from "./remote"; -import type { YTransactionStorageImpl } from "./storage"; +import type { YSqliteStorage } from "./storage"; export interface InternalYDurableObject { // private state doc: WSSharedDoc; - storage: YTransactionStorageImpl; + storage: YSqliteStorage; sessions: Map void>; awarenessClients: Set; diff --git a/src/yjs/storage/index.ts b/src/yjs/storage/index.ts index df6c086..e3d54f2 100644 --- a/src/yjs/storage/index.ts +++ b/src/yjs/storage/index.ts @@ -1,115 +1,5 @@ -import { Doc, applyUpdate, encodeStateAsUpdate } from "yjs"; +export { YSqliteStorage } from "./sqlite"; +export { UpdateKind } from "./type"; -import { storageKey } from "./storage-key"; - -import type { TransactionStorage } from "./type"; - -export interface YTransactionStorage { - getYDoc(): Promise; - storeUpdate(update: Uint8Array): Promise; - commit(): Promise; -} - -type Options = { - /** - * @description default is 10KB - * @default 10 * 1024 * 1 - */ - maxBytes?: number; - /** - * @description default is 500 snapshot - * @default 500 - */ - maxUpdates?: number; -}; - -export class YTransactionStorageImpl implements YTransactionStorage { - private readonly MAX_BYTES: number; - private readonly MAX_UPDATES: number; - - constructor( - private readonly storage: TransactionStorage, - options?: Options, - ) { - this.MAX_BYTES = options?.maxBytes ?? 10 * 1024; - if (this.MAX_BYTES > 128 * 1024) { - // https://developers.cloudflare.com/durable-objects/platform/limits/ - throw new Error("maxBytes must be less than 128KB"); - } - - this.MAX_UPDATES = options?.maxUpdates ?? 500; - } - - async getYDoc(): Promise { - const snapshot = await this.storage.get( - storageKey({ type: "state", name: "doc" }), - ); - const data = await this.storage.list({ - prefix: storageKey({ type: "update" }), - }); - - const updates: Uint8Array[] = Array.from(data.values()); - const doc = new Doc(); - - doc.transact(() => { - if (snapshot) { - applyUpdate(doc, snapshot); - } - for (const update of updates) { - applyUpdate(doc, update); - } - }); - - return doc; - } - - storeUpdate(update: Uint8Array): Promise { - return this.storage.transaction(async (tx) => { - const bytes = - (await tx.get(storageKey({ type: "state", name: "bytes" }))) ?? - 0; - const count = - (await tx.get(storageKey({ type: "state", name: "count" }))) ?? - 0; - - const updateBytes = bytes + update.byteLength; - const updateCount = count + 1; - - if (updateBytes > this.MAX_BYTES || updateCount > this.MAX_UPDATES) { - const doc = await this.getYDoc(); - applyUpdate(doc, update); - - await this._commit(doc, tx); - } else { - await tx.put(storageKey({ type: "state", name: "bytes" }), updateBytes); - await tx.put(storageKey({ type: "state", name: "count" }), updateCount); - await tx.put(storageKey({ type: "update", name: updateCount }), update); - } - }); - } - - private async _commit(doc: Doc, tx: Omit) { - const data = await tx.list({ - prefix: storageKey({ type: "update" }), - }); - - for (const update of data.values()) { - applyUpdate(doc, update); - } - - const update = encodeStateAsUpdate(doc); - - await tx.delete(Array.from(data.keys())); - await tx.put(storageKey({ type: "state", name: "bytes" }), 0); - await tx.put(storageKey({ type: "state", name: "count" }), 0); - await tx.put(storageKey({ type: "state", name: "doc" }), update); - } - - async commit(): Promise { - const doc = await this.getYDoc(); - - return this.storage.transaction(async (tx) => { - await this._commit(doc, tx); - }); - } -} +export type { YSqliteStorageOptions } from "./sqlite"; +export type { YStorage } from "./type"; diff --git a/src/yjs/storage/storage-key/index.ts b/src/yjs/storage/storage-key/index.ts deleted file mode 100644 index 55484a6..0000000 --- a/src/yjs/storage/storage-key/index.ts +++ /dev/null @@ -1,13 +0,0 @@ -export type Key = - | { - type: "update"; - name?: number; - } - | { - type: "state"; - name: "bytes" | "doc" | "count"; - }; - -export const storageKey = (key: Key) => { - return `ydoc:${key.type}:${key.name ?? ""}`; -}; diff --git a/src/yjs/storage/storage-key/storage-key.test.ts b/src/yjs/storage/storage-key/storage-key.test.ts deleted file mode 100644 index 0b8f2a9..0000000 --- a/src/yjs/storage/storage-key/storage-key.test.ts +++ /dev/null @@ -1,15 +0,0 @@ -import { storageKey } from "."; - -import type { Key } from "."; - -describe("storageKey functionality", () => { - it.each([ - [{ type: "update" }, "ydoc:update:"], // nameが省略された場合 - [{ type: "update", name: 1 }, "ydoc:update:1"], // nameが数値で提供された場合 - [{ type: "state", name: "bytes" }, "ydoc:state:bytes"], // typeがstateでnameがbytesの場合 - [{ type: "state", name: "doc" }, "ydoc:state:doc"], // typeがstateでnameがdocの場合 - [{ type: "state", name: "count" }, "ydoc:state:count"], // typeがstateでnameが新しく追加されたcountの場合 - ])("correctly generates storage key for key: %o", (key, expected) => { - expect(storageKey(key as Key)).toEqual(expected); - }); -}); diff --git a/src/yjs/storage/storage.test.ts b/src/yjs/storage/storage.test.ts deleted file mode 100644 index c44a30d..0000000 --- a/src/yjs/storage/storage.test.ts +++ /dev/null @@ -1,217 +0,0 @@ -import { vi } from "vitest"; -import { Doc, encodeStateAsUpdate } from "yjs"; - -import { storageKey } from "./storage-key"; - -import { YTransactionStorageImpl } from "."; - -import type { TransactionStorage } from "./type"; -import type { Mocked } from "vitest"; - -describe("YTransactionStorageImpl", () => { - let storage: Mocked; - - beforeEach(() => { - storage = { - get: vi.fn(), - list: vi.fn(), - put: vi.fn(), - delete: vi.fn(), - transaction: vi.fn(async (closure) => closure(storage)), - } as Mocked; - }); - - afterEach(() => { - vi.clearAllMocks(); - vi.resetAllMocks(); - }); - - describe("Document Retrieval", () => { - it("returns an empty YDoc when there are no updates", async () => { - storage.get.mockResolvedValueOnce(undefined); - storage.list.mockResolvedValueOnce(new Map()); - - const yStorage = new YTransactionStorageImpl(storage); - const doc = await yStorage.getYDoc(); - expect(encodeStateAsUpdate(doc)).toEqual(encodeStateAsUpdate(new Doc())); - }); - - it("reconstructs the correct YDoc state from updates", async () => { - const stored = new Doc(); - stored.getText("root").insert(0, "Hello World"); - const update = encodeStateAsUpdate(stored); - - storage.get.mockResolvedValueOnce(undefined); - storage.list.mockResolvedValueOnce(new Map([["ydoc:update:1", update]])); - - const yStorage = new YTransactionStorageImpl(storage); - const doc = await yStorage.getYDoc(); - expect(encodeStateAsUpdate(doc)).toEqual(encodeStateAsUpdate(stored)); - }); - }); - - describe("Update Storage", () => { - it("stores updates correctly under unique keys", async () => { - const update = new Uint8Array([1, 2, 3]); - storage.get.mockResolvedValueOnce(0).mockResolvedValueOnce(0); - - const yStorage = new YTransactionStorageImpl(storage); - await yStorage.storeUpdate(update); - expect(storage.put).toHaveBeenCalledWith("ydoc:state:bytes", 3); - expect(storage.put).toHaveBeenCalledWith("ydoc:state:count", 1); - expect(storage.put).toHaveBeenCalledWith("ydoc:update:1", update); - }); - - it("increments the update count and update bytes on subsequent updates", async () => { - storage.get.mockResolvedValueOnce(3).mockResolvedValueOnce(1); - - const yStorage = new YTransactionStorageImpl(storage); - const update = new Uint8Array([4, 5, 6]); - await yStorage.storeUpdate(update); - expect(storage.put).toHaveBeenCalledWith( - "ydoc:state:bytes", - 3 + update.byteLength, - ); - expect(storage.put).toHaveBeenCalledWith("ydoc:state:count", 2); - expect(storage.put).toHaveBeenCalledWith("ydoc:update:2", update); - }); - }); - - describe("Handling Exceeded Limits", () => { - it.each([ - [2048 * 1024 * 2 + 1, 10], // Exceeded maxBytes - [10, 501], // Exceeded maxUpdates - ])( - "resets counts and bytes and stores combined state doc when limits are exceeded (%p bytes, %p updates)", - async (exceededBytes, exceededUpdates) => { - const doc = new Doc(); - doc.getText("root").insert(0, "Hello World"); - const update = encodeStateAsUpdate(doc); - - storage.get.mockImplementation((key) => { - switch (key) { - case storageKey({ type: "state", name: "bytes" }): - return Promise.resolve(exceededBytes); - case storageKey({ type: "state", name: "count" }): - return Promise.resolve(exceededUpdates); - default: - return Promise.resolve(undefined); - } - }); - storage.list.mockResolvedValue( - new Map( - Array(exceededUpdates) - .fill(0) - .map((_, i) => [`ydoc:update:${i + 1}`, update]), - ), - ); - - const yStorage = new YTransactionStorageImpl(storage); - await yStorage.storeUpdate(update); - - const newDoc = await yStorage.getYDoc(); - const expectedState = encodeStateAsUpdate(newDoc); - - expect(storage.delete).toHaveBeenCalledWith( - Array(exceededUpdates) - .fill(0) - .map((_, i) => `ydoc:update:${i + 1}`), - ); - expect(storage.put).toHaveBeenCalledWith("ydoc:state:bytes", 0); - expect(storage.put).toHaveBeenCalledWith("ydoc:state:count", 0); - expect(storage.put).toHaveBeenCalledWith( - "ydoc:state:doc", - expect.any(Uint8Array), - ); - expect(expectedState).toEqual(encodeStateAsUpdate(newDoc)); - }, - ); - }); - - describe("Configuration Options", () => { - it("handles options to modify maxBytes and maxUpdates", () => { - const yStorage = new YTransactionStorageImpl(storage, { - maxBytes: 1024, - maxUpdates: 100, - }); - expect(yStorage).toHaveProperty("MAX_BYTES", 1024); - expect(yStorage).toHaveProperty("MAX_UPDATES", 100); - }); - - it("throws an error when maxBytes exceeds 128KB", () => { - expect(() => { - return new YTransactionStorageImpl(storage, { - maxBytes: 128 * 1024 + 1, - maxUpdates: 100, - }); - }).toThrow("maxBytes must be less than 128KB"); - }); - }); - - describe("commit method", () => { - it("commits all updates and clears all related storage keys", async () => { - const yStorage = new YTransactionStorageImpl(storage); - - const doc = new Doc(); - doc.getText("root").insert(0, "Hello World"); - const update = encodeStateAsUpdate(doc); - storage.get.mockImplementation((key) => { - switch (key) { - case storageKey({ type: "state", name: "bytes" }): - return Promise.resolve(update.byteLength); - case storageKey({ type: "state", name: "count" }): - return Promise.resolve(1); - case storageKey({ type: "state", name: "doc" }): - return Promise.resolve(undefined); - default: { - throw new Error("Unexpected key"); - } - } - }); - storage.list.mockResolvedValue( - new Map( - Array(1) - .fill(0) - .map((_, i) => [`ydoc:update:${i + 1}`, update]), - ), - ); - - await yStorage.commit(); - - expect(storage.delete).toHaveBeenCalledWith(expect.any(Array)); - expect(storage.put).toHaveBeenCalledWith( - storageKey({ type: "state", name: "bytes" }), - 0, - ); - expect(storage.put).toHaveBeenCalledWith( - storageKey({ type: "state", name: "count" }), - 0, - ); - expect(storage.put).toHaveBeenCalledWith( - storageKey({ type: "state", name: "doc" }), - expect.any(Uint8Array), - ); - }); - - it("does not throw errors when there are no updates to commit", async () => { - const yStorage = new YTransactionStorageImpl(storage); - storage.get.mockImplementation((key) => { - switch (key) { - case storageKey({ type: "state", name: "bytes" }): - return Promise.resolve(undefined); - case storageKey({ type: "state", name: "count" }): - return Promise.resolve(undefined); - case storageKey({ type: "state", name: "doc" }): - return Promise.resolve(undefined); - default: { - throw new Error("Unexpected key"); - } - } - }); - - storage.list.mockResolvedValue(new Map()); - - await expect(yStorage.commit()).resolves.not.toThrow(); - }); - }); -}); diff --git a/src/yjs/storage/type.ts b/src/yjs/storage/type.ts index 574f0bc..49af110 100644 --- a/src/yjs/storage/type.ts +++ b/src/yjs/storage/type.ts @@ -1,22 +1,3 @@ -interface ListOptions { - start?: string; - startAfter?: string; - end?: string; - prefix?: string; - reverse?: boolean; - limit?: number; -} - -export interface TransactionStorage { - get(key: string): Promise; - list(options?: ListOptions): Promise>; - put(key: string, value: T): Promise; - delete(key: string | string[]): Promise; - transaction( - closure: (txn: Omit) => Promise, - ): Promise; -} - /** * updates テーブルの kind カラムの値。 * From 1175fb8293be7447d33b8ce6aa06dfd45f538d66 Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 20:38:23 +0900 Subject: [PATCH 09/26] test: assert onStart rehydrates the document from SQLite storage on cold start Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015mdNeCzNPZysRHzmkFp9Bj --- src/e2e/y-durableobjects.test.ts | 38 ++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/src/e2e/y-durableobjects.test.ts b/src/e2e/y-durableobjects.test.ts index 2ca7713..1ef1683 100644 --- a/src/e2e/y-durableobjects.test.ts +++ b/src/e2e/y-durableobjects.test.ts @@ -1,8 +1,10 @@ import { env, runInDurableObject } from "cloudflare:test"; import { hc } from "hono/client"; import { expect, describe, it } from "vitest"; +import { Doc, encodeStateAsUpdate } from "yjs"; import { YDurableObjects } from "../yjs"; +import { YSqliteStorage } from "../yjs/storage"; import { createSyncMessage, createYDocMessage } from "./helper"; @@ -21,6 +23,42 @@ describe("YDurableObjects", () => { }); }); + it("rehydrates the document from SQLite storage on startup", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject( + stub, + async (instance: InternalYDurableObject, state) => { + // Write directly into this instance's SQLite storage, bypassing + // instance.doc entirely, so that the only way for the content to + // reach instance.doc is through onStart()'s getUpdate()/applyUpdate + // wiring — the exact seam this task changed. + const storage = new YSqliteStorage(state.storage.sql); + const seed = new Doc(); + seed.getText("root").insert(0, "Hello World!"); + await storage.storeUpdate(encodeStateAsUpdate(seed)); + + // vitest-pool-workers keeps this Durable Object instance alive for + // the whole worker lifetime, so there is no way to force a genuinely + // fresh construction here. Invoking onStart() directly is the + // workable substitute for a cold start: it re-runs the exact + // rehydration logic construction would have run. + // + // Known wart surfaced by this: onStart() unconditionally + // re-registers the doc "update" and awareness "update" listeners, + // so after this second call the instance carries duplicate + // listeners. That does not affect this assertion (rehydration reads + // storage once, before any listener fires), but it is a real + // pre-existing issue in onStart() worth flagging for Task 7, which + // rewrites this listener wiring. + await instance.onStart(); + + expect(instance.doc.getText("root").toString()).toBe("Hello World!"); + }, + ); + }); + it("create a room from request", async () => { const id = env.Y_DURABLE_OBJECTS.newUniqueId(); const stub = env.Y_DURABLE_OBJECTS.get(id); From fe706389187d58e05cbd55c57ffe1e4de14316ca Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 20:43:25 +0900 Subject: [PATCH 10/26] fix!: route sync replies to the requester and stop echoing to the sender Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015mdNeCzNPZysRHzmkFp9Bj --- src/yjs/index.ts | 11 ++- src/yjs/message-type/index.ts | 4 +- src/yjs/message-type/messaeg-type.test.ts | 2 + src/yjs/remote/ws-shared-doc.test.ts | 67 +++++++++++++++-- src/yjs/remote/ws-shared-doc.ts | 91 +++++++++++++++-------- 5 files changed, 135 insertions(+), 40 deletions(-) diff --git a/src/yjs/index.ts b/src/yjs/index.ts index d31ed3b..2195f6e 100644 --- a/src/yjs/index.ts +++ b/src/yjs/index.ts @@ -11,6 +11,9 @@ import { YSqliteStorage } from "./storage"; import type { AwarenessChanges } from "../yjs/remote"; import type { Env } from "hono"; +/** WebSocket 由来でない更新（JS RPC 経由）の origin */ +const RPC_ORIGIN: object = Object.freeze({ source: "rpc" }); + export type WebSocketAttachment = { roomId: string; connectedAt: Date; @@ -86,7 +89,7 @@ export class YDurableObjects extends DurableObject< } async updateYDoc(update: Uint8Array): Promise { - this.doc.update(update); + this.doc.update(update, RPC_ORIGIN); await this.cleanup(); } async getYDoc(): Promise { @@ -99,8 +102,8 @@ export class YDurableObjects extends DurableObject< ): Promise { if (!(message instanceof ArrayBuffer)) return; - const update = new Uint8Array(message); - await this.updateYDoc(update); + this.doc.update(new Uint8Array(message), ws); + await this.cleanup(); } async webSocketError(ws: WebSocket): Promise { @@ -115,7 +118,7 @@ export class YDurableObjects extends DurableObject< protected registerWebSocket(ws: WebSocket) { setupWSConnection(ws, this.doc); - const s = this.doc.notify((message) => { + const s = this.doc.notify(ws, (message) => { ws.send(message); }); this.sessions.set(ws, s); diff --git a/src/yjs/message-type/index.ts b/src/yjs/message-type/index.ts index a50b03f..052fae9 100644 --- a/src/yjs/message-type/index.ts +++ b/src/yjs/message-type/index.ts @@ -3,7 +3,9 @@ import { createEncoder, writeVarUint } from "lib0/encoding"; export const messageType = { sync: 0, awareness: 1, -}; + auth: 2, + queryAwareness: 3, +} as const; export const isMessageType = ( type: string, diff --git a/src/yjs/message-type/messaeg-type.test.ts b/src/yjs/message-type/messaeg-type.test.ts index 47dcd21..7db9c83 100644 --- a/src/yjs/message-type/messaeg-type.test.ts +++ b/src/yjs/message-type/messaeg-type.test.ts @@ -9,6 +9,8 @@ describe("createTypedEncoder", () => { const cases = [ ["sync", messageType.sync], ["awareness", messageType.awareness], + ["auth", messageType.auth], + ["queryAwareness", messageType.queryAwareness], ] as const; it.each(cases)( diff --git a/src/yjs/remote/ws-shared-doc.test.ts b/src/yjs/remote/ws-shared-doc.test.ts index 309ce04..2892857 100644 --- a/src/yjs/remote/ws-shared-doc.test.ts +++ b/src/yjs/remote/ws-shared-doc.test.ts @@ -1,7 +1,7 @@ import { createDecoder, readVarUint } from "lib0/decoding"; import { createEncoder, toUint8Array, writeVarUint } from "lib0/encoding"; import { vi, describe, it, expect, beforeEach, afterEach } from "vitest"; -import { readSyncMessage, writeUpdate } from "y-protocols/sync"; +import { readSyncMessage, writeSyncStep1, writeUpdate } from "y-protocols/sync"; import { Doc, encodeStateAsUpdate } from "yjs"; import { messageType } from "../message-type"; @@ -39,12 +39,14 @@ const applyMessage = (message: Uint8Array) => { describe("WSSharedDoc", () => { let doc: WSSharedDoc; + let origin: object; let mockListener: Mock; beforeEach(() => { doc = new WSSharedDoc(); + origin = {}; mockListener = vi.fn(); - doc.notify(mockListener); + doc.notify(origin, mockListener); }); afterEach(() => { @@ -56,7 +58,7 @@ describe("WSSharedDoc", () => { const update = createYDocMessage("Hello, world!"); const message = createSyncMessage(update); - doc.update(message); + doc.update(message, {}); expect(mockListener).toHaveBeenCalledWith(expect.any(Uint8Array)); expect(mockListener.mock.calls[0][0]).toEqual(message); @@ -68,21 +70,74 @@ describe("WSSharedDoc", () => { describe("Event Notification", () => { it("should add and remove listeners correctly", () => { + const anotherOrigin = {}; const anotherListener = vi.fn(); - const unsubscribe = doc.notify(anotherListener); + const unsubscribe = doc.notify(anotherOrigin, anotherListener); const message1 = createSyncMessage(createYDocMessage("text1")); const message2 = createSyncMessage(createYDocMessage("text2")); - doc.update(message1); + doc.update(message1, {}); expect(mockListener).toHaveBeenCalledTimes(1); expect(anotherListener).toHaveBeenCalledTimes(1); unsubscribe(); - doc.update(message2); + doc.update(message2, {}); expect(mockListener).toHaveBeenCalledTimes(2); expect(anotherListener).toHaveBeenCalledTimes(1); }); }); + + describe("Origin-aware routing", () => { + it("sends the sync step 2 reply only to the requesting origin", () => { + const doc = new WSSharedDoc(); + const requester = {}; + const bystander = {}; + const toRequester: Uint8Array[] = []; + const toBystander: Uint8Array[] = []; + doc.notify(requester, (message) => toRequester.push(message)); + doc.notify(bystander, (message) => toBystander.push(message)); + + doc.getText("root").insert(0, "seed"); + toRequester.length = 0; + toBystander.length = 0; + + const encoder = createEncoder(); + writeVarUint(encoder, messageType.sync); + writeSyncStep1(encoder, new Doc()); + doc.update(toUint8Array(encoder), requester); + + expect(toRequester.length).toBe(1); + expect(toBystander.length).toBe(0); + }); + + it("does not echo an update back to its origin", () => { + const doc = new WSSharedDoc(); + const sender = {}; + const receiver = {}; + const toSender: Uint8Array[] = []; + const toReceiver: Uint8Array[] = []; + doc.notify(sender, (message) => toSender.push(message)); + doc.notify(receiver, (message) => toReceiver.push(message)); + + const source = new Doc(); + source.getText("root").insert(0, "hello"); + const encoder = createEncoder(); + writeVarUint(encoder, messageType.sync); + writeUpdate(encoder, encodeStateAsUpdate(source)); + doc.update(toUint8Array(encoder), sender); + + expect(toSender.length).toBe(0); + expect(toReceiver.length).toBe(1); + }); + + it("throws on an unknown message type", () => { + const doc = new WSSharedDoc(); + const encoder = createEncoder(); + writeVarUint(encoder, 99); + + expect(() => doc.update(toUint8Array(encoder), {})).toThrow(); + }); + }); }); diff --git a/src/yjs/remote/ws-shared-doc.ts b/src/yjs/remote/ws-shared-doc.ts index 9eb022b..567b206 100644 --- a/src/yjs/remote/ws-shared-doc.ts +++ b/src/yjs/remote/ws-shared-doc.ts @@ -18,14 +18,16 @@ import { createTypedEncoder, messageType } from "../message-type"; import type { AwarenessChanges, RemoteDoc } from "."; -type Listener = (message: T) => void; +type Listener = (message: Uint8Array) => void; type Unsubscribe = () => void; -interface Notification extends RemoteDoc { - notify(cb: Listener): Unsubscribe; + +interface Notification extends RemoteDoc { + notify(origin: object, listener: Listener): Unsubscribe; } -export class WSSharedDoc extends Doc implements Notification { - private listeners = new Set>(); +export class WSSharedDoc extends Doc implements Notification { + /** origin（通常は WebSocket）をキーにした配信先 */ + private listeners = new Map(); readonly awareness = new Awareness(this); constructor(gc = true) { @@ -33,16 +35,19 @@ export class WSSharedDoc extends Doc implements Notification { this.awareness.setLocalState(null); // カーソルなどの付加情報の更新通知 - this.awareness.on("update", (changes: AwarenessChanges) => { - this.awarenessChangeHandler(changes); - }); + this.awareness.on( + "update", + (changes: AwarenessChanges, origin: unknown) => { + this.awarenessChangeHandler(changes, origin); + }, + ); // yDoc の更新通知 - this.on("update", (update: Uint8Array) => { - this.syncMessageHandler(update); + this.on("update", (update: Uint8Array, origin: unknown) => { + this.syncMessageHandler(update, origin); }); } - update(message: Uint8Array) { + update(message: Uint8Array, origin: object) { const encoder = createEncoder(); const decoder = createDecoder(message); const type = readVarUint(decoder); @@ -50,40 +55,63 @@ export class WSSharedDoc extends Doc implements Notification { switch (type) { case messageType.sync: { writeVarUint(encoder, messageType.sync); - readSyncMessage(decoder, encoder, this, null); + readSyncMessage(decoder, encoder, this, origin); - // changed remote doc + // sync step 1 への応答は要求元にだけ返す if (length(encoder) > 1) { - this._notify(toUint8Array(encoder)); + this.send(origin, toUint8Array(encoder)); } break; } case messageType.awareness: { - applyAwarenessUpdate(this.awareness, readVarUint8Array(decoder), null); + applyAwarenessUpdate( + this.awareness, + readVarUint8Array(decoder), + origin, + ); + break; + } + case messageType.queryAwareness: { + const states = this.awareness.getStates(); + if (states.size > 0) { + const reply = createTypedEncoder("awareness"); + writeVarUint8Array( + reply, + encodeAwarenessUpdate(this.awareness, Array.from(states.keys())), + ); + this.send(origin, toUint8Array(reply)); + } break; } + case messageType.auth: { + // auth はサーバからクライアントへの一方向のメッセージなので受信しても何もしない + break; + } + default: { + throw new Error(`Unsupported message type: ${type}`); + } } } - notify(listener: Listener) { - this.listeners.add(listener); + notify(origin: object, listener: Listener) { + this.listeners.set(origin, listener); return () => { - this.listeners.delete(listener); + this.listeners.delete(origin); }; } - private syncMessageHandler(update: Uint8Array) { + private syncMessageHandler(update: Uint8Array, origin: unknown) { const encoder = createTypedEncoder("sync"); writeUpdate(encoder, update); - this._notify(toUint8Array(encoder)); + this.broadcast(toUint8Array(encoder), origin); } - private awarenessChangeHandler({ - added, - updated, - removed, - }: AwarenessChanges) { + + private awarenessChangeHandler( + { added, updated, removed }: AwarenessChanges, + origin: unknown, + ) { const changed = [...added, ...updated, ...removed]; const encoder = createTypedEncoder("awareness"); const update = encodeAwarenessUpdate( @@ -93,12 +121,17 @@ export class WSSharedDoc extends Doc implements Notification { ); writeVarUint8Array(encoder, update); - this._notify(toUint8Array(encoder)); + this.broadcast(toUint8Array(encoder), origin); + } + + private send(origin: object, message: Uint8Array) { + this.listeners.get(origin)?.(message); } - private _notify(message: Uint8Array) { - for (const subscriber of this.listeners) { - subscriber(message); + private broadcast(message: Uint8Array, exclude: unknown) { + for (const [origin, listener] of this.listeners) { + if (origin === exclude) continue; + listener(message); } } } From 061380edffd6d837ad44eac52b42c019f30a4288 Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 20:48:47 +0900 Subject: [PATCH 11/26] fix: unsubscribe by listener identity, not by origin key Prevents an earlier unsubscribe() call from deleting a newer listener that notify() registered for the same origin after the first one, which would otherwise silently stop message delivery to that origin. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015mdNeCzNPZysRHzmkFp9Bj --- src/yjs/remote/ws-shared-doc.test.ts | 18 ++++++++++++++++++ src/yjs/remote/ws-shared-doc.ts | 7 ++++++- 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/src/yjs/remote/ws-shared-doc.test.ts b/src/yjs/remote/ws-shared-doc.test.ts index 2892857..44f813d 100644 --- a/src/yjs/remote/ws-shared-doc.test.ts +++ b/src/yjs/remote/ws-shared-doc.test.ts @@ -87,6 +87,24 @@ describe("WSSharedDoc", () => { expect(mockListener).toHaveBeenCalledTimes(2); expect(anotherListener).toHaveBeenCalledTimes(1); }); + + it("does not let an earlier unsubscribe for an origin remove a later listener registered for that same origin", () => { + const sharedOrigin = {}; + const firstListener = vi.fn(); + const secondListener = vi.fn(); + + const unsubscribeFirst = doc.notify(sharedOrigin, firstListener); + // A second notify() for the same origin overwrites the first entry. + doc.notify(sharedOrigin, secondListener); + + // This should be a no-op: it registered `firstListener`, which is no + // longer the listener stored for `sharedOrigin`. + unsubscribeFirst(); + + doc.update(createSyncMessage(createYDocMessage("text3")), {}); + + expect(secondListener).toHaveBeenCalledTimes(1); + }); }); describe("Origin-aware routing", () => { diff --git a/src/yjs/remote/ws-shared-doc.ts b/src/yjs/remote/ws-shared-doc.ts index 567b206..a11095f 100644 --- a/src/yjs/remote/ws-shared-doc.ts +++ b/src/yjs/remote/ws-shared-doc.ts @@ -97,7 +97,12 @@ export class WSSharedDoc extends Doc implements Notification { this.listeners.set(origin, listener); return () => { - this.listeners.delete(origin); + // A later notify() for the same origin overwrites this entry; only + // remove it if it's still the listener *this* call registered, so an + // earlier unsubscribe can't delete a newer listener for the same origin. + if (this.listeners.get(origin) === listener) { + this.listeners.delete(origin); + } }; } From d52014d54d43e65b690044b45bb76de300a5ab67 Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 20:56:07 +0900 Subject: [PATCH 12/26] fix!: scope awareness removal to the disconnecting connection Track each connection's owned awareness clientIDs in a SessionRegistry, persisted into the WebSocket's attachment so ownership survives Durable Object hibernation. unregisterWebSocket() now removes only the disconnecting connection's clientIDs instead of the whole room's, fixing the bug where any one client leaving wiped every participant's cursor/presence state. --- src/e2e/y-durableobjects.test.ts | 81 +++++++++++++++++++++++++++++++- src/yjs/index.ts | 42 ++++++++--------- src/yjs/internal.ts | 4 +- src/yjs/session/index.ts | 77 ++++++++++++++++++++++++++++++ src/yjs/session/session.test.ts | 76 ++++++++++++++++++++++++++++++ 5 files changed, 254 insertions(+), 26 deletions(-) create mode 100644 src/yjs/session/index.ts create mode 100644 src/yjs/session/session.test.ts diff --git a/src/e2e/y-durableobjects.test.ts b/src/e2e/y-durableobjects.test.ts index 1ef1683..c67be8d 100644 --- a/src/e2e/y-durableobjects.test.ts +++ b/src/e2e/y-durableobjects.test.ts @@ -1,9 +1,17 @@ import { env, runInDurableObject } from "cloudflare:test"; import { hc } from "hono/client"; +import { + createEncoder, + toUint8Array, + writeVarUint, + writeVarUint8Array, +} from "lib0/encoding"; import { expect, describe, it } from "vitest"; +import { Awareness, encodeAwarenessUpdate } from "y-protocols/awareness"; import { Doc, encodeStateAsUpdate } from "yjs"; import { YDurableObjects } from "../yjs"; +import { messageType } from "../yjs/message-type"; import { YSqliteStorage } from "../yjs/storage"; import { createSyncMessage, createYDocMessage } from "./helper"; @@ -11,6 +19,23 @@ import { createSyncMessage, createYDocMessage } from "./helper"; import type { YDurableObjectsAppType } from "../yjs"; import type { InternalYDurableObject } from "../yjs/internal"; +// Encodes a real awareness protocol message, the way an actual client would. +// Sending this through webSocketMessage() is the only way to exercise the +// production path that decides ownership: WSSharedDoc.update() passes the +// receiving WebSocket as `origin` into applyAwarenessUpdate(), which is what +// the awareness "update" handler in onStart() branches on with +// `origin instanceof WebSocket`. +const createAwarenessMessage = (awareness: Awareness) => { + const encoder = createEncoder(); + writeVarUint(encoder, messageType.awareness); + writeVarUint8Array( + encoder, + encodeAwarenessUpdate(awareness, [awareness.clientID]), + ); + + return toUint8Array(encoder); +}; + describe("YDurableObjects", () => { it("initializes correctly", async () => { const id = env.Y_DURABLE_OBJECTS.newUniqueId(); @@ -130,7 +155,7 @@ describe("YDurableObjects", () => { await runInDurableObject(stub, async (instance: InternalYDurableObject) => { const roomId = "room1"; await instance.createRoom(roomId); - const [server] = Array.from(instance.sessions.entries()).at(0)!; + const [server] = Array.from(instance.sessions.sockets()); await instance.webSocketError(server); @@ -145,11 +170,63 @@ describe("YDurableObjects", () => { await runInDurableObject(stub, async (instance: InternalYDurableObject) => { const roomId = "room1"; await instance.createRoom(roomId); - const [server] = Array.from(instance.sessions.entries()).at(0)!; + const [server] = Array.from(instance.sessions.sockets()); await instance.webSocketClose(server); expect(instance.sessions.size).toBe(0); }); }); + + it("keeps other clients' awareness when one connection closes", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject(stub, async (instance: InternalYDurableObject) => { + await instance.createRoom("room1"); + await instance.createRoom("room1"); + const [first, second] = Array.from(instance.sessions.sockets()); + + instance.sessions.track(first, [1]); + instance.sessions.track(second, [2]); + instance.doc.awareness.setLocalStateField("user", { name: "a" }); + + await instance.webSocketClose(first); + + // first の clientId だけが除去され、second の所有分は残る + expect(instance.sessions.clientIdsOf(second)).toEqual([2]); + expect(instance.sessions.size).toBe(1); + }); + }); + + it("derives awareness ownership from the WebSocket origin observed at runtime", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject(stub, async (instance: InternalYDurableObject) => { + await instance.createRoom("room1"); + const [server] = Array.from(instance.sessions.sockets()); + + // Simulate a remote peer announcing its presence, exactly as a real + // Yjs client would over the wire. + const remoteDoc = new Doc(); + const remoteAwareness = new Awareness(remoteDoc); + remoteAwareness.setLocalStateField("user", { name: "remote" }); + + const message = createAwarenessMessage(remoteAwareness); + await instance.webSocketMessage(server, message.slice(0).buffer); + + // sessions.track() is only ever called from the awareness "update" + // handler wired in onStart(), and only when `origin instanceof + // WebSocket` is true. This test never calls sessions.track() itself, + // so if that instanceof check silently failed to match (e.g. because + // WebSocketPair sockets, or the sockets returned by + // state.getWebSockets() after hibernation, are not real WebSocket + // instances in this runtime), clientIdsOf(server) would stay empty + // here and the assertion below would fail. + expect(instance.sessions.clientIdsOf(server)).toEqual([ + remoteAwareness.clientID, + ]); + }); + }); }); diff --git a/src/yjs/index.ts b/src/yjs/index.ts index 2195f6e..c3d1c7f 100644 --- a/src/yjs/index.ts +++ b/src/yjs/index.ts @@ -6,18 +6,17 @@ import { WSSharedDoc } from "../yjs/remote"; import { setupWSConnection } from "./client/setup"; import { createApp } from "./hono"; +import { SessionRegistry } from "./session"; import { YSqliteStorage } from "./storage"; +import type { SessionAttachment } from "./session"; import type { AwarenessChanges } from "../yjs/remote"; import type { Env } from "hono"; /** WebSocket 由来でない更新（JS RPC 経由）の origin */ const RPC_ORIGIN: object = Object.freeze({ source: "rpc" }); -export type WebSocketAttachment = { - roomId: string; - connectedAt: Date; -}; +export type { SessionAttachment } from "./session"; export type YDurableObjectsAppType = ReturnType; @@ -29,8 +28,7 @@ export class YDurableObjects extends DurableObject< }); protected doc = new WSSharedDoc(); protected storage: YSqliteStorage; - protected sessions = new Map void>(); - private awarenessClients = new Set(); + protected sessions = new SessionRegistry(); constructor( public state: DurableObjectState, @@ -58,12 +56,9 @@ export class YDurableObjects extends DurableObject< }); this.doc.awareness.on( "update", - async ({ added, removed, updated }: AwarenessChanges) => { - for (const client of [...added, ...updated]) { - this.awarenessClients.add(client); - } - for (const client of removed) { - this.awarenessClients.delete(client); + ({ added, updated }: AwarenessChanges, origin: unknown) => { + if (origin instanceof WebSocket) { + this.sessions.track(origin, [...added, ...updated]); } }, ); @@ -75,8 +70,9 @@ export class YDurableObjects extends DurableObject< const server = pair[1]; server.serializeAttachment({ roomId, - connectedAt: new Date(), - } satisfies WebSocketAttachment); + connectedAt: Date.now(), + clientIds: [], + } satisfies SessionAttachment); this.state.acceptWebSocket(server); this.registerWebSocket(server); @@ -118,20 +114,22 @@ export class YDurableObjects extends DurableObject< protected registerWebSocket(ws: WebSocket) { setupWSConnection(ws, this.doc); - const s = this.doc.notify(ws, (message) => { + const dispose = this.doc.notify(ws, (message) => { ws.send(message); }); - this.sessions.set(ws, s); + this.sessions.add(ws, dispose); } protected async unregisterWebSocket(ws: WebSocket) { try { - const dispose = this.sessions.get(ws); - dispose?.(); - this.sessions.delete(ws); - const clientIds = this.awarenessClients; - - removeAwarenessStates(this.doc.awareness, Array.from(clientIds), null); + // この接続が所有する clientID だけを削除する。 + // 部屋全体の clientID を削除すると他の参加者の presence まで消える。 + removeAwarenessStates( + this.doc.awareness, + this.sessions.clientIdsOf(ws), + null, + ); + this.sessions.remove(ws); } catch (e) { // eslint-disable-next-line no-console console.error(e); diff --git a/src/yjs/internal.ts b/src/yjs/internal.ts index 17d3db6..fe5b603 100644 --- a/src/yjs/internal.ts +++ b/src/yjs/internal.ts @@ -1,12 +1,12 @@ import type { WSSharedDoc } from "./remote"; +import type { SessionRegistry } from "./session"; import type { YSqliteStorage } from "./storage"; export interface InternalYDurableObject { // private state doc: WSSharedDoc; storage: YSqliteStorage; - sessions: Map void>; - awarenessClients: Set; + sessions: SessionRegistry; // private api diff --git a/src/yjs/session/index.ts b/src/yjs/session/index.ts new file mode 100644 index 0000000..1366276 --- /dev/null +++ b/src/yjs/session/index.ts @@ -0,0 +1,77 @@ +export type SessionAttachment = { + roomId: string; + /** epoch ミリ秒。attachment は structured clone されるが、数値の方が扱いが単純 */ + connectedAt: number; + /** この接続が所有する awareness の clientID */ + clientIds: number[]; +}; + +const isSessionAttachment = (value: unknown): value is SessionAttachment => { + if (typeof value !== "object" || value === null) return false; + const candidate = value as Record; + + return ( + typeof candidate.roomId === "string" && + typeof candidate.connectedAt === "number" && + Array.isArray(candidate.clientIds) && + candidate.clientIds.every((id) => typeof id === "number") + ); +}; + +/** + * WebSocket と awareness の所有権を管理する。 + * + * 所有権は WebSocket の attachment に永続化するため、Durable Object が + * hibernation から復帰してメモリ上の状態を失っても復元できる。 + */ +export class SessionRegistry { + private disposers = new Map void>(); + + get size(): number { + return this.disposers.size; + } + + add(ws: WebSocket, dispose: () => void): void { + this.disposers.set(ws, dispose); + } + + remove(ws: WebSocket): void { + this.disposers.get(ws)?.(); + this.disposers.delete(ws); + } + + has(ws: WebSocket): boolean { + return this.disposers.has(ws); + } + + sockets(): IterableIterator { + return this.disposers.keys(); + } + + clientIdsOf(ws: WebSocket): number[] { + return this.attachmentOf(ws)?.clientIds ?? []; + } + + /** + * この接続が所有する clientID を記録する。 + * 実際に増えたときだけ attachment を書き直すので、通常は接続あたり 1 回で済む。 + */ + track(ws: WebSocket, clientIds: readonly number[]): void { + const current = this.attachmentOf(ws); + if (current === null) return; + + const merged = new Set([...current.clientIds, ...clientIds]); + if (merged.size === current.clientIds.length) return; + + ws.serializeAttachment({ + ...current, + clientIds: Array.from(merged), + } satisfies SessionAttachment); + } + + private attachmentOf(ws: WebSocket): SessionAttachment | null { + const raw: unknown = ws.deserializeAttachment(); + + return isSessionAttachment(raw) ? raw : null; + } +} diff --git a/src/yjs/session/session.test.ts b/src/yjs/session/session.test.ts new file mode 100644 index 0000000..ce6658d --- /dev/null +++ b/src/yjs/session/session.test.ts @@ -0,0 +1,76 @@ +import { describe, expect, it, vi } from "vitest"; + +import { SessionRegistry } from "."; + +import type { SessionAttachment } from "."; + +const fakeSocket = (attachment: SessionAttachment | null): WebSocket => { + let current = attachment; + + return { + serializeAttachment: (value: SessionAttachment) => { + current = value; + }, + deserializeAttachment: () => current, + } as unknown as WebSocket; +}; + +const attachment = (clientIds: number[]): SessionAttachment => ({ + roomId: "room1", + connectedAt: 0, + clientIds, +}); + +describe("SessionRegistry", () => { + it("tracks and disposes sockets", () => { + const registry = new SessionRegistry(); + const ws = fakeSocket(attachment([])); + const dispose = vi.fn(); + + registry.add(ws, dispose); + expect(registry.size).toBe(1); + expect(registry.has(ws)).toBe(true); + + registry.remove(ws); + expect(dispose).toHaveBeenCalledTimes(1); + expect(registry.size).toBe(0); + }); + + it("records client ids on the attachment", () => { + const registry = new SessionRegistry(); + const ws = fakeSocket(attachment([])); + registry.add(ws, () => {}); + + registry.track(ws, [7]); + + expect(registry.clientIdsOf(ws)).toEqual([7]); + }); + + it("does not duplicate client ids", () => { + const registry = new SessionRegistry(); + const ws = fakeSocket(attachment([7])); + registry.add(ws, () => {}); + + registry.track(ws, [7, 8]); + registry.track(ws, [8]); + + expect(registry.clientIdsOf(ws).sort()).toEqual([7, 8]); + }); + + it("restores ownership from an existing attachment after hibernation", () => { + // hibernation 復帰を模す。registry は空だが WebSocket の attachment は残っている + const ws = fakeSocket(attachment([42])); + const registry = new SessionRegistry(); + registry.add(ws, () => {}); + + expect(registry.clientIdsOf(ws)).toEqual([42]); + }); + + it("returns an empty list for a socket without a valid attachment", () => { + const registry = new SessionRegistry(); + const ws = fakeSocket(null); + registry.add(ws, () => {}); + + expect(registry.clientIdsOf(ws)).toEqual([]); + }); +}); From f492015b096272994676a8698b6c8626eea7f724 Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 21:04:28 +0900 Subject: [PATCH 13/26] fix: unsubscribe the WSSharedDoc listener before removing awareness states MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit unregisterWebSocket() called removeAwarenessStates() while the departing socket was still a registered WSSharedDoc listener. removeAwarenessStates synchronously broadcasts to all listeners including the departing one, and send() on an already-closed Workers WebSocket throws — which the outer catch swallowed before sessions.remove(ws) could run, leaking the session and its listener for the rest of the Durable Object's life and breaking delivery to every other listener reached after it. Reorder unregisterWebSocket to unsubscribe first. Also harden WSSharedDoc.broadcast with a per-listener try/catch so one dead connection can never abort delivery to the rest of the room, and replace the awareness-on-close regression test with one that drives a real awareness-protocol message through both connections and closes one via webSocketClose, since the previous version never actually exercised the broadcast path that broke. --- src/e2e/y-durableobjects.test.ts | 65 ++++++++++++++++++++++++++++---- src/yjs/index.ts | 14 ++++--- src/yjs/remote/ws-shared-doc.ts | 10 ++++- 3 files changed, 76 insertions(+), 13 deletions(-) diff --git a/src/e2e/y-durableobjects.test.ts b/src/e2e/y-durableobjects.test.ts index c67be8d..0ba6415 100644 --- a/src/e2e/y-durableobjects.test.ts +++ b/src/e2e/y-durableobjects.test.ts @@ -187,15 +187,66 @@ describe("YDurableObjects", () => { await instance.createRoom("room1"); const [first, second] = Array.from(instance.sessions.sockets()); - instance.sessions.track(first, [1]); - instance.sessions.track(second, [2]); - instance.doc.awareness.setLocalStateField("user", { name: "a" }); - - await instance.webSocketClose(first); + // Each connection publishes real awareness state through an actual + // awareness-protocol message, the same way "derives awareness + // ownership from the WebSocket origin observed at runtime" does. This + // is essential: driving state through the real onStart() awareness + // handler is what makes removeAwarenessStates() below have anything + // to remove, which is the exact path that broke when + // unregisterWebSocket() removed awareness states while the departing + // socket was still a registered WSSharedDoc listener (broadcast() + // would call send() on an already-closed socket and throw). A version + // of this test that sets clientIds by calling sessions.track() + // directly, or by calling awareness.setLocalStateField() on the + // shared doc (which fires under the *local* client id, not either + // socket's), would never exercise that path and would pass even + // against the broken ordering. + const firstAwareness = new Awareness(new Doc()); + firstAwareness.setLocalStateField("user", { name: "a" }); + await instance.webSocketMessage( + first, + createAwarenessMessage(firstAwareness).slice(0).buffer, + ); + + const secondAwareness = new Awareness(new Doc()); + secondAwareness.setLocalStateField("user", { name: "b" }); + await instance.webSocketMessage( + second, + createAwarenessMessage(secondAwareness).slice(0).buffer, + ); + + expect(instance.sessions.clientIdsOf(first)).toEqual([ + firstAwareness.clientID, + ]); + expect(instance.sessions.clientIdsOf(second)).toEqual([ + secondAwareness.clientID, + ]); - // first の clientId だけが除去され、second の所有分は残る - expect(instance.sessions.clientIdsOf(second)).toEqual([2]); + // Close `first` the way a real disconnect would leave it: the + // underlying socket is already closed by the time webSocketClose() + // fires, so any send() on it throws. This reproduces the exact + // condition the ordering bug hit — a throwing listener still + // registered in WSSharedDoc when removeAwarenessStates() runs. + first.close(); + + // (a) the call completes without throwing, even though `first` is a + // dead socket and, until the fix, awareness removal still tried to + // broadcast to it. + await expect(instance.webSocketClose(first)).resolves.toBeUndefined(); + + // (b) the departing socket is fully unregistered — no leaked session + // or listener. + expect(instance.sessions.has(first)).toBe(false); expect(instance.sessions.size).toBe(1); + expect(instance.sessions.clientIdsOf(second)).toEqual([ + secondAwareness.clientID, + ]); + + // (c) the remaining connection's awareness state survives; only the + // departing connection's clientId was removed from the room. + const states = instance.doc.awareness.getStates(); + expect(states.has(secondAwareness.clientID)).toBe(true); + expect(states.has(firstAwareness.clientID)).toBe(false); }); }); diff --git a/src/yjs/index.ts b/src/yjs/index.ts index c3d1c7f..71d0fb1 100644 --- a/src/yjs/index.ts +++ b/src/yjs/index.ts @@ -124,12 +124,16 @@ export class YDurableObjects extends DurableObject< try { // この接続が所有する clientID だけを削除する。 // 部屋全体の clientID を削除すると他の参加者の presence まで消える。 - removeAwarenessStates( - this.doc.awareness, - this.sessions.clientIdsOf(ws), - null, - ); + const clientIds = this.sessions.clientIdsOf(ws); + + // 先にリスナーを解除してから removeAwarenessStates を呼ぶこと。 + // removeAwarenessStates は awareness の "update" を同期的に発火させ、 + // WSSharedDoc.broadcast がまだ登録されたままの ws.send() を呼んでしまう。 + // 切断直後のソケットへの send() は例外を投げるため、その場合に + // sessions.remove(ws) が実行されずリスナーがリークし、以降このルームの + // 配信が全滅する。 this.sessions.remove(ws); + removeAwarenessStates(this.doc.awareness, clientIds, null); } catch (e) { // eslint-disable-next-line no-console console.error(e); diff --git a/src/yjs/remote/ws-shared-doc.ts b/src/yjs/remote/ws-shared-doc.ts index a11095f..b8f8840 100644 --- a/src/yjs/remote/ws-shared-doc.ts +++ b/src/yjs/remote/ws-shared-doc.ts @@ -136,7 +136,15 @@ export class WSSharedDoc extends Doc implements Notification { private broadcast(message: Uint8Array, exclude: unknown) { for (const [origin, listener] of this.listeners) { if (origin === exclude) continue; - listener(message); + + try { + listener(message); + } catch (e) { + // 1 つの接続が切断済み・エラー状態で listener(=ws.send) が例外を + // 投げても、それ以降の健全な接続への配信を止めてはいけない。 + // eslint-disable-next-line no-console + console.error(e); + } } } } From 50e506a1b52099734fb98387261868441675a4e5 Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 21:08:22 +0900 Subject: [PATCH 14/26] test: assert unregisterWebSocket's ordering invariant directly The prior fix commit (f492015) reordered unregisterWebSocket so the departing socket's WSSharedDoc listener is unsubscribed before its awareness states are removed, but nothing discriminated that ordering on its own: with the broadcast guard in place, re-reversing the two lines still passed every existing test. Add a test that subscribes to the awareness "update" event and records whether the departing socket is still registered in sessions at the moment the event fires, then asserts it was not -- proving the unsubscribe ran first, independent of whether ws.send() throws. Verified by toggling: re-reversing the two lines (with the broadcast guard untouched) makes this test fail while the pre-existing awareness-on-close test still passes, reproducing exactly the gap this closes. --- src/e2e/y-durableobjects.test.ts | 49 ++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/src/e2e/y-durableobjects.test.ts b/src/e2e/y-durableobjects.test.ts index 0ba6415..2d26fd5 100644 --- a/src/e2e/y-durableobjects.test.ts +++ b/src/e2e/y-durableobjects.test.ts @@ -250,6 +250,55 @@ describe("YDurableObjects", () => { }); }); + it("unsubscribes the departing connection before broadcasting its awareness removal", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject(stub, async (instance: InternalYDurableObject) => { + await instance.createRoom("room1"); + const [server] = Array.from(instance.sessions.sockets()); + + // The departing socket must genuinely own awareness state, or + // removeAwarenessStates() in unregisterWebSocket() has nothing to + // remove and never emits "update" — in which case the handler below + // would never run and the observation would never be recorded, + // making the test vacuously pass no matter the ordering. + const awareness = new Awareness(new Doc()); + awareness.setLocalStateField("user", { name: "a" }); + await instance.webSocketMessage( + server, + createAwarenessMessage(awareness).slice(0).buffer, + ); + expect(instance.sessions.clientIdsOf(server)).toEqual([ + awareness.clientID, + ]); + + // Assert the ordering invariant behaviourally, without relying on + // ws.send() throwing (the broadcast guard added for Finding 2 means a + // throw there no longer surfaces as a test failure on its own — see + // the toggle-off note in the report for why that guard alone doesn't + // prove the ordering is right). Capture the observation inside the + // handler rather than asserting inside it, so a failure surfaces as a + // normal assertion in the test body instead of an exception thrown + // from deep inside Awareness's emit() and swallowed by + // unregisterWebSocket()'s own try/catch. + let serverStillRegisteredWhenAwarenessUpdateFired: boolean | undefined; + instance.doc.awareness.on("update", () => { + serverStillRegisteredWhenAwarenessUpdateFired = + instance.sessions.has(server); + }); + + await instance.webSocketClose(server); + + // Guard against a vacuous pass: the handler must actually have run. + expect(serverStillRegisteredWhenAwarenessUpdateFired).not.toBeUndefined(); + // The session must already be gone by the time the awareness removal + // broadcasts — i.e. sessions.remove(ws) ran before + // removeAwarenessStates(...) in unregisterWebSocket(). + expect(serverStillRegisteredWhenAwarenessUpdateFired).toBe(false); + }); + }); + it("derives awareness ownership from the WebSocket origin observed at runtime", async () => { const id = env.Y_DURABLE_OBJECTS.newUniqueId(); const stub = env.Y_DURABLE_OBJECTS.get(id); From acf18ec1c9c91da069600a6dd405f729357100b6 Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 21:23:38 +0900 Subject: [PATCH 15/26] fix!: serialize persistence and fail closed when storage writes fail Persistence used to happen from an unawaited floating promise in the doc "update" listener, so a caller could observe a write that hadn't actually landed in storage yet, and a rejected storeUpdate() was silently dropped. Route every update through a serialized persist queue that webSocketMessage and updateYDoc both await, and if a write fails, close every connection with 1011 and abort() the Durable Object so the next start rehydrates from storage instead of serving state that has silently drifted ahead of it. Also wrap webSocketMessage's doc.update() in a try/catch so a malformed message closes only the offending socket instead of crashing the whole Durable Object. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015mdNeCzNPZysRHzmkFp9Bj --- src/e2e/y-durableobjects.test.ts | 110 +++++++++++++++++++++++++++++++ src/yjs/index.ts | 79 ++++++++++++++++++---- 2 files changed, 177 insertions(+), 12 deletions(-) diff --git a/src/e2e/y-durableobjects.test.ts b/src/e2e/y-durableobjects.test.ts index 2d26fd5..e2995a9 100644 --- a/src/e2e/y-durableobjects.test.ts +++ b/src/e2e/y-durableobjects.test.ts @@ -329,4 +329,114 @@ describe("YDurableObjects", () => { ]); }); }); + + it("closes only the offending connection on a malformed message", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject(stub, async (instance: InternalYDurableObject) => { + await instance.createRoom("room1"); + await instance.createRoom("room1"); + const [first, other] = Array.from(instance.sessions.sockets()); + + // 未知のメッセージ型。例外が外に漏れると DO 全体がリセットされる + const malformed = new Uint8Array([99]).buffer; + await expect( + instance.webSocketMessage(first, malformed), + ).resolves.toBeUndefined(); + + // DO は生存し、他の（無関係な）接続も維持されている。webSocketClose + // が `first` に対して呼ばれるかどうかはランタイムの詳細であり、この + // テストが依存すべき性質ではない。ここで確かめるべきは「DO がリセット + // されず、正常な接続が生き残ること」だけ。 + expect(instance.sessions.has(other)).toBe(true); + }); + }); + + it("persists an update before webSocketMessage resolves", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject(stub, async (instance: InternalYDurableObject) => { + const client = await instance.createRoom("room1"); + + // storeUpdate に人為的な遅延を挟む。SQLite への実書き込みは同期 API + // なので、遅延を入れずに `await this.persist` を webSocketMessage から + // 消してみても、たまたまマイクロタスクの実行順序だけで書き込みが先に + // 終わってしまい、「直列化されている」ことを何も検証しない空振りの + // テストになる（実際に確認した。トグルオフ検証は報告書を参照）。 + let persisted = false; + const originalStoreUpdate = instance.storage.storeUpdate.bind( + instance.storage, + ); + instance.storage.storeUpdate = async (update: Uint8Array) => { + await new Promise((resolve) => setTimeout(resolve, 10)); + await originalStoreUpdate(update); + persisted = true; + }; + + const message = createSyncMessage(createYDocMessage("persisted")); + await instance.webSocketMessage(client, message.slice(0).buffer); + + // webSocketMessage が解決した時点で、遅延込みの永続化が完了している + expect(persisted).toBe(true); + + // ストレージから読み直しても内容が入っている + const stored = await instance.storage.getUpdate(); + expect(stored).not.toBeNull(); + }); + }); + + it("closes every connection and asks the runtime to reset when persistence fails", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject( + stub, + async (instance: InternalYDurableObject, state: DurableObjectState) => { + const first = await instance.createRoom("room1"); + await instance.createRoom("room1"); + const sockets = Array.from(instance.sessions.sockets()); + + // state.abort() の"本物"の実装は、この Durable Object の + // io-context（≒このテストの runInDurableObject 呼び出しそのもの） + // を丸ごと破棄する。実際に呼ばせると、それを待っている + // runInDurableObject() 自身の Promise が二度と解決/reject されず、 + // このファイルどころか単一ランタイム上の以降の全テストごと + // ハングすることを確認済み（トグルオフ検証時に実測、報告書に記載）。 + // ここで検証したいのは「失敗ハンドラが state.abort() を正しい理由 + // 付きで呼び、全ソケットを 1011 で閉じたか」という自分のコードの + // 振る舞いであり、workerd 自身の abort 実装の正しさではないため、 + // abort() をスパイに差し替えて実行だけ観測する。 + let abortCalled = false; + let abortReason: string | undefined; + state.abort = (reason?: string) => { + abortCalled = true; + abortReason = reason; + }; + + instance.storage.storeUpdate = async () => { + throw new Error("simulated storage failure"); + }; + + const message = createSyncMessage( + createYDocMessage("will not persist"), + ); + + // webSocketMessage 自体は例外を投げずに解決する + // (Step 5 の try/catch は doc.update() の同期例外用。永続化の失敗は + // schedulePersist の中で catch され、webSocketMessage はそれを + // 待つだけなので reject しない)。 + await expect( + instance.webSocketMessage(first, message.slice(0).buffer), + ).resolves.toBeUndefined(); + + for (const ws of sockets) { + expect(ws.readyState).not.toBe(WebSocket.READY_STATE_OPEN); + } + expect(abortCalled).toBe(true); + expect(abortReason).toBe("failed to persist a Yjs update"); + }, + ); + }); }); diff --git a/src/yjs/index.ts b/src/yjs/index.ts index 71d0fb1..77160e6 100644 --- a/src/yjs/index.ts +++ b/src/yjs/index.ts @@ -30,6 +30,11 @@ export class YDurableObjects extends DurableObject< protected storage: YSqliteStorage; protected sessions = new SessionRegistry(); + /** 永続化を直列化するためのキュー。Yjs の update イベントは同期的に発火するため必要 */ + private persist: Promise = Promise.resolve(); + /** onStart() が複数回呼ばれても doc/awareness のリスナーを二重登録しないためのガード */ + private listenersRegistered = false; + constructor( public state: DurableObjectState, public env: T["Bindings"], @@ -51,17 +56,27 @@ export class YDurableObjects extends DurableObject< this.registerWebSocket(ws); } - this.doc.on("update", async (update) => { - await this.storage.storeUpdate(update); - }); - this.doc.awareness.on( - "update", - ({ added, updated }: AwarenessChanges, origin: unknown) => { - if (origin instanceof WebSocket) { - this.sessions.track(origin, [...added, ...updated]); - } - }, - ); + // 本番では onStart はコンストラクタの blockConcurrencyWhile からしか + // 呼ばれず、二重登録は起こらない。ただし一部のテストは冷起動の代わりに + // onStart() を直接呼び直して再水和ロジックを検証するため、リスナー登録 + // だけは冪等にしておく。ここを冪等にしないと、update リスナーが二重に + // 登録され、以降の update ごとに schedulePersist が二重発火して永続化が + // 重複する。 + if (!this.listenersRegistered) { + this.listenersRegistered = true; + + this.doc.on("update", (update: Uint8Array) => { + this.schedulePersist(update); + }); + this.doc.awareness.on( + "update", + ({ added, updated }: AwarenessChanges, origin: unknown) => { + if (origin instanceof WebSocket) { + this.sessions.track(origin, [...added, ...updated]); + } + }, + ); + } } protected createRoom(roomId: string) { @@ -86,6 +101,7 @@ export class YDurableObjects extends DurableObject< async updateYDoc(update: Uint8Array): Promise { this.doc.update(update, RPC_ORIGIN); + await this.persist; await this.cleanup(); } async getYDoc(): Promise { @@ -98,7 +114,17 @@ export class YDurableObjects extends DurableObject< ): Promise { if (!(message instanceof ArrayBuffer)) return; - this.doc.update(new Uint8Array(message), ws); + try { + this.doc.update(new Uint8Array(message), ws); + } catch (error) { + // eslint-disable-next-line no-console + console.error("[y-durableobjects] invalid message", error); + ws.close(1003, "invalid message"); + + return; + } + + await this.persist; await this.cleanup(); } @@ -145,4 +171,33 @@ export class YDurableObjects extends DurableObject< await this.storage.commit(); } } + + private schedulePersist(update: Uint8Array): void { + this.persist = this.persist + .then(() => this.storage.storeUpdate(update)) + .catch((error: unknown) => { + this.onPersistFailure(error); + }); + this.state.waitUntil(this.persist); + } + + /** + * 永続化に失敗したら全接続を閉じ、Durable Object をリセットする。 + * + * 接続を閉じるだけではメモリ上の Doc がストレージより進んだまま残り、 + * 後続の接続が「正常に見える」状態を受け取ったあと、eviction 時に + * 差分が無言で失われる。abort() でストレージから読み直させる。 + * + * CRDT ではクライアント側が完全な状態を保持しているため、再接続時の + * sync step 1 / 2 で失われた更新が再送され、障害は自己修復する。 + */ + private onPersistFailure(error: unknown): void { + // eslint-disable-next-line no-console + console.error("[y-durableobjects] failed to persist update", error); + + for (const ws of this.state.getWebSockets()) { + ws.close(1011, "storage failure"); + } + this.state.abort("failed to persist a Yjs update"); + } } From ccd3c871bdf8bde0accedf35829c1e7b2b3afdc2 Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 21:31:48 +0900 Subject: [PATCH 16/26] fix: contain close() failures in onPersistFailure and tighten test coverage A throwing ws.close() (e.g. on a socket already closed by the exception boundary, or the socket whose own error caused the failure) used to escape the close loop in onPersistFailure and abort() unwrapped, which meant state.abort() was never reached and the failure policy silently degraded to "some sockets stay open, in-memory doc still ahead of storage." Wrap the per-socket close() in try/catch, matching the same pattern already used in WSSharedDoc.broadcast(). Also tightens the malformed-message test to assert the offending socket was actually closed, adds a regression test proving a throwing close() no longer blocks abort() or the remaining sockets, and corrects a rehydration-test comment made stale by the earlier idempotency guard. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015mdNeCzNPZysRHzmkFp9Bj --- src/e2e/y-durableobjects.test.ts | 67 ++++++++++++++++++++++++++++---- src/yjs/index.ts | 13 ++++++- 2 files changed, 72 insertions(+), 8 deletions(-) diff --git a/src/e2e/y-durableobjects.test.ts b/src/e2e/y-durableobjects.test.ts index e2995a9..eabc11f 100644 --- a/src/e2e/y-durableobjects.test.ts +++ b/src/e2e/y-durableobjects.test.ts @@ -70,13 +70,12 @@ describe("YDurableObjects", () => { // workable substitute for a cold start: it re-runs the exact // rehydration logic construction would have run. // - // Known wart surfaced by this: onStart() unconditionally - // re-registers the doc "update" and awareness "update" listeners, - // so after this second call the instance carries duplicate - // listeners. That does not affect this assertion (rehydration reads - // storage once, before any listener fires), but it is a real - // pre-existing issue in onStart() worth flagging for Task 7, which - // rewrites this listener wiring. + // onStart() guards its doc "update" and awareness "update" listener + // registration with a `listenersRegistered` flag (Task 7), so this + // second call re-runs rehydration without re-registering either + // listener. That guard doesn't affect this assertion either way + // (rehydration reads storage once, before any listener fires) — it's + // just why calling onStart() twice here is safe to do at all. await instance.onStart(); expect(instance.doc.getText("root").toString()).toBe("Hello World!"); @@ -350,6 +349,11 @@ describe("YDurableObjects", () => { // テストが依存すべき性質ではない。ここで確かめるべきは「DO がリセット // されず、正常な接続が生き残ること」だけ。 expect(instance.sessions.has(other)).toBe(true); + + // 実際に不正なメッセージを送った接続自体は 1003 で閉じられている。 + // これがないと、例外境界が ws.close(1003, ...) を一切呼ばない + // (単に catch { return; } するだけの) 実装でもこのテストは通ってしまう。 + expect(first.readyState).not.toBe(WebSocket.READY_STATE_OPEN); }); }); @@ -439,4 +443,53 @@ describe("YDurableObjects", () => { }, ); }); + + it("still closes the other sockets and reaches abort() when one socket's close() throws", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject( + stub, + async (instance: InternalYDurableObject, state: DurableObjectState) => { + await instance.createRoom("room1"); + await instance.createRoom("room1"); + const [troublesome, healthy] = Array.from(instance.sessions.sockets()); + + // workerd throws if close() is called on a socket that is already + // closed or errored — exactly what state.getWebSockets() can hand + // the failure handler: the socket the exception boundary just + // closed with 1003, or the socket whose own error caused this + // failure in the first place. Simulate that here. + troublesome.close = () => { + throw new Error("already closed"); + }; + + let abortCalled = false; + state.abort = () => { + abortCalled = true; + }; + + instance.storage.storeUpdate = async () => { + throw new Error("simulated storage failure"); + }; + + const message = createSyncMessage( + createYDocMessage("will not persist"), + ); + + // webSocketMessage still resolves normally: the per-socket + // try/catch in onPersistFailure must contain the throwing close(), + // not let it escape and reject the persist chain. + await expect( + instance.webSocketMessage(troublesome, message.slice(0).buffer), + ).resolves.toBeUndefined(); + + // The other socket is not left open just because one close() threw. + expect(healthy.readyState).not.toBe(WebSocket.READY_STATE_OPEN); + // And critically, abort() — the actual point of this failure + // policy — is still reached despite the throw. + expect(abortCalled).toBe(true); + }, + ); + }); }); diff --git a/src/yjs/index.ts b/src/yjs/index.ts index 77160e6..922a2f6 100644 --- a/src/yjs/index.ts +++ b/src/yjs/index.ts @@ -196,7 +196,18 @@ export class YDurableObjects extends DurableObject< console.error("[y-durableobjects] failed to persist update", error); for (const ws of this.state.getWebSockets()) { - ws.close(1011, "storage failure"); + // すでに閉じている・エラー状態のソケットに close() を呼ぶと workerd は + // 例外を投げる（例えば、この例外境界が 1003 で閉じた直後のソケット、 + // あるいは今回の失敗の原因になったソケット自身）。1 つのソケットを + // 閉じられないことが他のソケットを閉じ損ねたり、下の abort() の + // 呼び出しを妨げたりしてはいけない。abort() こそがこのポリシーの + // 本体であり、閉じ損ねたソケットが 1 つあっても必ず到達させる。 + try { + ws.close(1011, "storage failure"); + } catch (e) { + // eslint-disable-next-line no-console + console.error(e); + } } this.state.abort("failed to persist a Yjs update"); } From 3452034d206b9dc4a325ddeac22b0de16087b613 Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 21:38:45 +0900 Subject: [PATCH 17/26] feat!: add hibernation auto-response, SQLite backend assertion, and destroy API Registers a ping/pong WebSocketAutoResponse pair so client keepalives no longer wake the Durable Object from hibernation, asserts the SQLite storage backend is in use (failing loudly with a migration pointer otherwise), and adds a destroy() API that closes all connections and deletes the room's storage. destroy()'s socket-close loop guards each close() individually (same shape as onPersistFailure) so one already-closed/errored socket can't prevent storage.destroy() from running. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015mdNeCzNPZysRHzmkFp9Bj --- eslint.config.js | 2 ++ src/e2e/y-durableobjects.test.ts | 55 ++++++++++++++++++++++++++++++++ src/yjs/index.ts | 46 ++++++++++++++++++++++++++ src/yjs/internal.ts | 1 + 4 files changed, 104 insertions(+) diff --git a/eslint.config.js b/eslint.config.js index 6b148e6..590903a 100644 --- a/eslint.config.js +++ b/eslint.config.js @@ -77,8 +77,10 @@ const config = [ DurableObjectStub: "readonly", DurableObjectTransaction: "readonly", DurableObjectState: "readonly", + DurableObjectStorage: "readonly", SqlStorage: "readonly", WebSocketPair: "readonly", + WebSocketRequestResponsePair: "readonly", RequestInfo: "readonly", }, }, diff --git a/src/e2e/y-durableobjects.test.ts b/src/e2e/y-durableobjects.test.ts index eabc11f..fe926e6 100644 --- a/src/e2e/y-durableobjects.test.ts +++ b/src/e2e/y-durableobjects.test.ts @@ -492,4 +492,59 @@ describe("YDurableObjects", () => { }, ); }); + + it("configures a ping/pong auto response", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject(stub, async (_instance, state) => { + const pair = state.getWebSocketAutoResponse(); + + expect(pair?.request).toBe("ping"); + expect(pair?.response).toBe("pong"); + }); + }); + + it("clears the document and closes connections on destroy", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject(stub, async (instance: InternalYDurableObject) => { + await instance.createRoom("room1"); + await instance.updateYDoc( + createSyncMessage(createYDocMessage("bye")).slice(0), + ); + + await instance.destroy(); + + expect(await instance.storage.getUpdate()).toBeNull(); + }); + }); + + it("still destroys storage when one socket's close() throws", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject(stub, async (instance: InternalYDurableObject) => { + await instance.createRoom("room1"); + await instance.createRoom("room1"); + const [troublesome] = Array.from(instance.sessions.sockets()); + + // Same workerd hazard as onPersistFailure: close() on an + // already-closed/errored socket throws. destroy()'s per-socket + // try/catch must contain it so storage.destroy() is still reached — + // otherwise the caller is told the room was destroyed while its data + // is still on disk. + troublesome.close = () => { + throw new Error("already closed"); + }; + + await instance.updateYDoc( + createSyncMessage(createYDocMessage("bye")).slice(0), + ); + + await expect(instance.destroy()).resolves.toBeUndefined(); + expect(await instance.storage.getUpdate()).toBeNull(); + }); + }); }); diff --git a/src/yjs/index.ts b/src/yjs/index.ts index 922a2f6..c53d28d 100644 --- a/src/yjs/index.ts +++ b/src/yjs/index.ts @@ -16,6 +16,27 @@ import type { Env } from "hono"; /** WebSocket 由来でない更新（JS RPC 経由）の origin */ const RPC_ORIGIN: object = Object.freeze({ source: "rpc" }); +const MIGRATION_GUIDE_URL = + "https://github.com/napolab/y-durableobjects#migrating-from-v1-key-value-backend"; + +/** + * SQLite バックエンドで動作しているかを確認する。 + * KV バックエンドでは sql へのアクセスが失敗するため、 + * 原因不明のクラッシュではなく移行手順を示したエラーにする。 + */ +const assertSqliteBackend = (storage: DurableObjectStorage): void => { + try { + storage.sql.exec("SELECT 1"); + } catch (error) { + throw new Error( + `y-durableobjects v2 requires the SQLite storage backend. ` + + `Use "new_sqlite_classes" in your wrangler migrations. ` + + `Migration guide: ${MIGRATION_GUIDE_URL}`, + { cause: error }, + ); + } +}; + export type { SessionAttachment } from "./session"; export type YDurableObjectsAppType = ReturnType; @@ -41,8 +62,15 @@ export class YDurableObjects extends DurableObject< ) { super(state, env); + assertSqliteBackend(state.storage); this.storage = new YSqliteStorage(state.storage.sql); + // ping を自動応答にすることで、keepalive で Durable Object を + // 起こさずに済む。duration 課金に最も効く設定。 + state.setWebSocketAutoResponse( + new WebSocketRequestResponsePair("ping", "pong"), + ); + void this.state.blockConcurrencyWhile(this.onStart.bind(this)); } @@ -108,6 +136,24 @@ export class YDurableObjects extends DurableObject< return encodeStateAsUpdate(this.doc); } + /** 部屋のデータを削除し、すべての接続を閉じる */ + async destroy(): Promise { + for (const ws of this.state.getWebSockets()) { + // すでに閉じている・エラー状態のソケットに close() を呼ぶと workerd は + // 例外を投げることがある。1 つのソケットを閉じられないことが他の + // ソケットを閉じ損ねたり、下の storage.destroy() の呼び出しを妨げたり + // してはいけない。destroy() の存在意義はデータを消すことそのものなので、 + // 閉じ損ねたソケットが 1 つあってもデータ削除は必ず到達させる。 + try { + ws.close(1001, "room destroyed"); + } catch (e) { + // eslint-disable-next-line no-console + console.error(e); + } + } + await this.storage.destroy(); + } + async webSocketMessage( ws: WebSocket, message: string | ArrayBuffer, diff --git a/src/yjs/internal.ts b/src/yjs/internal.ts index fe5b603..c586168 100644 --- a/src/yjs/internal.ts +++ b/src/yjs/internal.ts @@ -25,4 +25,5 @@ export interface InternalYDurableObject { getYDoc(): Promise; updateYDoc(update: Uint8Array): Promise; + destroy(): Promise; } From 70c1b0daf204ca09f9d52b6db9ccfbb2284fef4a Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 21:45:32 +0900 Subject: [PATCH 18/26] fix!: make updateYDoc accept a raw Yjs update so it round-trips with getYDoc updateYDoc previously ran its argument through WSSharedDoc.update(), which expects a sync-protocol-framed message, while getYDoc() returns a raw encodeStateAsUpdate() result. That made it impossible to feed getYDoc()'s output back into updateYDoc(), silently breaking the most natural export/ import use case and the README's own example, and blocking the v1-to-v2 migration recipe (v1's getYDoc() already returned a raw update). updateYDoc now calls applyUpdate(this.doc, update, RPC_ORIGIN) directly, so it takes a raw Yjs update. Updated every updateYDoc call site in tests that was wrapping its payload with createSyncMessage (including the POST /rooms/:id/update e2e test, which forwards its body straight to updateYDoc), while leaving webSocketMessage's framed-message call sites untouched. Added a round-trip test that exports from one room via getYDoc and imports into another via updateYDoc. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015mdNeCzNPZysRHzmkFp9Bj --- src/e2e/e2e.test.ts | 8 ++--- src/e2e/y-durableobjects.test.ts | 54 ++++++++++++++++++++++---------- src/yjs/index.ts | 7 ++++- 3 files changed, 46 insertions(+), 23 deletions(-) diff --git a/src/e2e/e2e.test.ts b/src/e2e/e2e.test.ts index 4604350..f846bb7 100644 --- a/src/e2e/e2e.test.ts +++ b/src/e2e/e2e.test.ts @@ -2,7 +2,7 @@ import { SELF, env, runInDurableObject } from "cloudflare:test"; import { hc } from "hono/client"; import { fromUint8Array } from "js-base64"; -import { createSyncMessage, createYDocMessage } from "./helper"; +import { createYDocMessage } from "./helper"; import type { AppType } from "."; import type { InternalYDurableObject } from "../yjs/internal"; @@ -56,10 +56,9 @@ describe("endpoint request", () => { const id = env.Y_DURABLE_OBJECTS.idFromName(roomId); const stub = env.Y_DURABLE_OBJECTS.get(id); const message = createYDocMessage("get state"); - const update = createSyncMessage(message); await runInDurableObject(stub, async (instance: InternalYDurableObject) => { - await instance.updateYDoc(update.slice(0)); + await instance.updateYDoc(message.slice(0)); }); const res = await SELF.fetch(`http://localhost/rooms/${roomId}/state`); @@ -71,7 +70,6 @@ describe("endpoint request", () => { it("should update the YDoc state", async () => { const message = createYDocMessage("get state"); - const update = createSyncMessage(message); const roomId = "1"; const id = env.Y_DURABLE_OBJECTS.idFromName(roomId); @@ -79,7 +77,7 @@ describe("endpoint request", () => { const res = await SELF.fetch(`http://localhost/rooms/${roomId}/update`, { method: "POST", - body: update.slice(0).buffer, + body: message.slice(0).buffer, }); expect(res.status).toBe(200); diff --git a/src/e2e/y-durableobjects.test.ts b/src/e2e/y-durableobjects.test.ts index fe926e6..f675138 100644 --- a/src/e2e/y-durableobjects.test.ts +++ b/src/e2e/y-durableobjects.test.ts @@ -8,7 +8,7 @@ import { } from "lib0/encoding"; import { expect, describe, it } from "vitest"; import { Awareness, encodeAwarenessUpdate } from "y-protocols/awareness"; -import { Doc, encodeStateAsUpdate } from "yjs"; +import { Doc, applyUpdate, encodeStateAsUpdate } from "yjs"; import { YDurableObjects } from "../yjs"; import { messageType } from "../yjs/message-type"; @@ -116,18 +116,42 @@ describe("YDurableObjects", () => { }); }); - it("updates YDoc correctly", async () => { - const id = env.Y_DURABLE_OBJECTS.newUniqueId(); - const stub = env.Y_DURABLE_OBJECTS.get(id); + it("round-trips between getYDoc and updateYDoc", async () => { + const source = env.Y_DURABLE_OBJECTS.get( + env.Y_DURABLE_OBJECTS.newUniqueId(), + ); + const target = env.Y_DURABLE_OBJECTS.get( + env.Y_DURABLE_OBJECTS.newUniqueId(), + ); - await runInDurableObject(stub, async (instance: InternalYDurableObject) => { - const message = createYDocMessage(); - const update = createSyncMessage(message); - await instance.updateYDoc(update.slice(0)); + const message = createYDocMessage("Hello World!"); + await runInDurableObject( + source, + async (instance: InternalYDurableObject) => { + await instance.updateYDoc(message.slice(0)); + }, + ); - const docState = await instance.getYDoc(); - expect(docState).toEqual(message); - }); + const exported = await runInDurableObject( + source, + (instance: InternalYDurableObject) => instance.getYDoc(), + ); + await runInDurableObject( + target, + async (instance: InternalYDurableObject) => { + // getYDoc の出力をそのまま updateYDoc に渡せる + await instance.updateYDoc(exported); + }, + ); + + const copied = await runInDurableObject( + target, + (instance: InternalYDurableObject) => instance.getYDoc(), + ); + + const doc = new Doc(); + applyUpdate(doc, copied); + expect(doc.getText("root").toString()).toBe("Hello World!"); }); it("handles WebSocket messages correctly", async () => { @@ -511,9 +535,7 @@ describe("YDurableObjects", () => { await runInDurableObject(stub, async (instance: InternalYDurableObject) => { await instance.createRoom("room1"); - await instance.updateYDoc( - createSyncMessage(createYDocMessage("bye")).slice(0), - ); + await instance.updateYDoc(createYDocMessage("bye").slice(0)); await instance.destroy(); @@ -539,9 +561,7 @@ describe("YDurableObjects", () => { throw new Error("already closed"); }; - await instance.updateYDoc( - createSyncMessage(createYDocMessage("bye")).slice(0), - ); + await instance.updateYDoc(createYDocMessage("bye").slice(0)); await expect(instance.destroy()).resolves.toBeUndefined(); expect(await instance.storage.getUpdate()).toBeNull(); diff --git a/src/yjs/index.ts b/src/yjs/index.ts index c53d28d..d3c7254 100644 --- a/src/yjs/index.ts +++ b/src/yjs/index.ts @@ -127,8 +127,13 @@ export class YDurableObjects extends DurableObject< return this.app.request(request, undefined, this.env); } + /** + * 生の Yjs update を適用する。 + * WebSocket 経路と違い、プロトコルのフレーミングは不要。 + * getYDoc() の戻り値をそのまま渡せる。 + */ async updateYDoc(update: Uint8Array): Promise { - this.doc.update(update, RPC_ORIGIN); + applyUpdate(this.doc, update, RPC_ORIGIN); await this.persist; await this.cleanup(); } From b95b18ffe565e5c840cb2a39679dcd78f1a50622 Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 21:52:37 +0900 Subject: [PATCH 19/26] test: cover updateYDoc broadcasting to connected WebSocket clients MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fix round 1 on Task 9's updateYDoc change: nothing proved an update applied via updateYDoc actually reaches connected WebSocket clients, only that it round-trips through storage between two Durable Objects. Adds a test that connects a client, calls updateYDoc with a raw update, and decodes the broadcast the socket receives (past the outer sync-message wrapper, via y-protocols/sync's readSyncMessage) to assert its content matches — not just that some send happened. Verified the test discriminates: temporarily made updateYDoc pass the room's own connected socket as the update origin (a live WSSharedDoc broadcast-listener key, standing in for RPC_ORIGIN colliding with one), confirmed the test fails with "expected spy to be called at least once", then reverted with no production diff. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015mdNeCzNPZysRHzmkFp9Bj --- src/e2e/y-durableobjects.test.ts | 53 +++++++++++++++++++++++++++++++- 1 file changed, 52 insertions(+), 1 deletion(-) diff --git a/src/e2e/y-durableobjects.test.ts b/src/e2e/y-durableobjects.test.ts index f675138..1b538d7 100644 --- a/src/e2e/y-durableobjects.test.ts +++ b/src/e2e/y-durableobjects.test.ts @@ -1,13 +1,15 @@ import { env, runInDurableObject } from "cloudflare:test"; import { hc } from "hono/client"; +import { createDecoder, readVarUint } from "lib0/decoding"; import { createEncoder, toUint8Array, writeVarUint, writeVarUint8Array, } from "lib0/encoding"; -import { expect, describe, it } from "vitest"; +import { expect, describe, it, vi } from "vitest"; import { Awareness, encodeAwarenessUpdate } from "y-protocols/awareness"; +import { readSyncMessage } from "y-protocols/sync"; import { Doc, applyUpdate, encodeStateAsUpdate } from "yjs"; import { YDurableObjects } from "../yjs"; @@ -154,6 +156,55 @@ describe("YDurableObjects", () => { expect(doc.getText("root").toString()).toBe("Hello World!"); }); + it("broadcasts an updateYDoc change to connected WebSocket clients", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject(stub, async (instance: InternalYDurableObject) => { + await instance.createRoom("room1"); + const [server] = Array.from(instance.sessions.sockets()); + + // registerWebSocket() already sent an initial sync/awareness pair + // synchronously when the room was created above; only care about + // what gets sent in response to updateYDoc, so install the spy + // after that initial handshake has already happened. + const sent = vi.fn(); + server.send = sent; + + const message = createYDocMessage("via rpc"); + await instance.updateYDoc(message.slice(0)); + + // applyUpdate(this.doc, update, RPC_ORIGIN) must fire WSSharedDoc's + // "update" handler, which calls broadcast(msg, RPC_ORIGIN) — and + // broadcast only excludes an origin that is itself a registered + // listener key. RPC_ORIGIN is never registered (only WebSockets are, + // via notify()), so every connected socket, including this one, + // must receive the broadcast. + expect(sent).toHaveBeenCalled(); + + const syncMessage = sent.mock.calls + .map((call: unknown[]) => call[0]) + .find((raw): raw is Uint8Array => { + if (!(raw instanceof Uint8Array)) return false; + const decoder = createDecoder(raw); + + return readVarUint(decoder) === messageType.sync; + }); + expect(syncMessage).toBeDefined(); + + // Decode past the outer messageType.sync wrapper and apply the inner + // sync-protocol payload to a fresh Doc exactly as a real client would + // on receipt. This proves the *content* of the RPC update actually + // reached the socket, not merely that some send() happened to fire. + const decoder = createDecoder(syncMessage as Uint8Array); + readVarUint(decoder); // messageType.sync, already checked above + const received = new Doc(); + readSyncMessage(decoder, createEncoder(), received, null); + + expect(received.getText("root").toString()).toBe("via rpc"); + }); + }); + it("handles WebSocket messages correctly", async () => { const id = env.Y_DURABLE_OBJECTS.newUniqueId(); const stub = env.Y_DURABLE_OBJECTS.get(id); From d74f7954cbe3629f0eef06c24476a8599b4c91e2 Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 21:58:24 +0900 Subject: [PATCH 20/26] docs: document the SQLite migration and add the v2 changeset Update README and CLAUDE.md for the SQLite storage backend, add a migration-from-v1 recipe (using the two-step get/idFromName form, since getByName isn't declared in this repo's worker-configuration.d.ts), and correct the same getByName inaccuracy in the design spec. Add the v2 changeset. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015mdNeCzNPZysRHzmkFp9Bj --- .changeset/sqlite-migration.md | 36 ++++++++++++ CLAUDE.md | 21 +++++-- README.md | 56 ++++++++++++++++++- .../2026-08-14-sqlite-migration-design.md | 20 +++++-- 4 files changed, 122 insertions(+), 11 deletions(-) create mode 100644 .changeset/sqlite-migration.md diff --git a/.changeset/sqlite-migration.md b/.changeset/sqlite-migration.md new file mode 100644 index 0000000..dc161ca --- /dev/null +++ b/.changeset/sqlite-migration.md @@ -0,0 +1,36 @@ +--- +"y-durableobjects": major +--- + +Migrate to the Durable Objects SQLite storage backend and fix the defects the +key-value backend had forced. + +**Breaking changes** + +- Requires `new_sqlite_classes` in your wrangler migrations. A v1 namespace + cannot be converted in place — see "Migrating from v1" in the README. +- `updateYDoc()` now takes a raw Yjs update instead of a sync-protocol message, + so it round-trips with `getYDoc()`. +- The exported `YTransactionStorage` type is replaced by `YStorage`. +- `WSSharedDoc.notify(listener)` is now `notify(origin, listener)` and + `WSSharedDoc.update(message)` is now `update(message, origin)`. +- `WebSocketAttachment` is replaced by `SessionAttachment`, which carries the + connection's awareness client ids. + +**Fixes** + +- Documents are no longer capped at 128KiB. +- Compaction no longer exceeds the 128-key limit of `delete()`. +- Closing one connection no longer clears every participant's awareness state. +- Updates are persisted in order and awaited rather than left as floating promises. +- Sync step 2 replies go only to the requesting client instead of the whole room. +- Updates are no longer echoed back to their sender. +- A malformed binary message closes only that connection instead of resetting + the Durable Object. +- Stored updates are restored in insertion order. + +**Additions** + +- `destroy()` deletes a room's data and closes its connections. +- A `"ping"` / `"pong"` auto-response keeps keepalives from waking the Durable + Object from hibernation. diff --git a/CLAUDE.md b/CLAUDE.md index 755f0ff..ae85f65 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -34,7 +34,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co - Main Durable Object class extending Cloudflare's DurableObject - Manages WebSocket connections and Yjs document synchronization - - Handles persistence through YTransactionStorage + - Handles persistence through YSqliteStorage - Provides JS RPC methods: `getYDoc()` and `updateYDoc()` 2. **WSSharedDoc** (`src/yjs/remote/ws-shared-doc.ts`) @@ -43,11 +43,14 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co - Manages awareness protocol for collaborative features - Handles document updates and broadcasts -3. **YTransactionStorage** (`src/yjs/storage/index.ts`) +3. **YSqliteStorage** (`src/yjs/storage/sqlite.ts`) - - Persistence layer for Yjs updates - - Uses Durable Object storage with transaction support - - Implements incremental update storage with periodic compaction + - Persistence layer backed by the Durable Objects SQLite storage backend + - Single `updates` table; snapshots and incremental updates are not distinguished + - Compacts with `Y.mergeUpdates` on a row-count threshold, splitting the + result into chunks when it exceeds the SQLite BLOB limit + - `PRAGMA` is unavailable on Durable Objects SQLite; schema versioning uses a + `schema_version` table (`src/yjs/storage/schema.ts`) 4. **Hono Integration** (`src/index.ts`) - Provides `yRoute()` helper for easy Hono app integration @@ -74,11 +77,19 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co - Consistent type imports/exports required 3. **Code Style** + - Kebab-case for filenames - No console.log in production code - Import ordering enforced by ESLint - Prettier formatting required +4. **SQLite Storage Backend** + + - Requires `new_sqlite_classes` in wrangler migrations + - BLOB columns are returned as `ArrayBuffer`; convert with `new Uint8Array(value)` + - Consecutive synchronous `sql.exec` calls with no intervening `await` form an + implicit transaction — do not await inside a compaction + ### Testing Approach Tests follow these patterns: diff --git a/README.md b/README.md index 897cb97..dd1a98d 100644 --- a/README.md +++ b/README.md @@ -41,7 +41,7 @@ This configuration ensures that your Cloudflare Worker can correctly instantiate ```toml name = "your-worker-name" main = "src/index.ts" -compatibility_date = "2024-04-05" +compatibility_date = "2025-04-01" account_id = "your-account-id" workers_dev = true @@ -53,11 +53,60 @@ bindings = [ ] # Durable Objects migrations +# v2 requires the SQLite storage backend. [[migrations]] tag = "v1" -new_classes = ["YDurableObjects"] +new_sqlite_classes = ["YDurableObjects"] ``` +## Migrating from v1 (key-value backend) + +v2 requires the SQLite storage backend. A Durable Object namespace's storage +type is immutable, so an existing v1 namespace cannot be converted in place — +Cloudflare rejects it with `storage_type_mismatch`. + +Run both versions side by side and copy each room across: + +1. Keep your existing v1 binding (for example `Y_LEGACY`) pinned to + `y-durableobjects@1`. +2. Add a new binding backed by v2 with `new_sqlite_classes`. +3. Copy each room over. `getYDoc()` returns a raw Yjs update and v2's + `updateYDoc()` accepts one, so a single round trip is enough: + +```typescript +app.post("/migrate/:id", async (c) => { + const id = c.req.param("id"); + const legacy = c.env.Y_LEGACY.get(c.env.Y_LEGACY.idFromName(id)); + const next = c.env.Y_DURABLE_OBJECTS.get( + c.env.Y_DURABLE_OBJECTS.idFromName(id), + ); + + await next.updateYDoc(await legacy.getYDoc()); + + return c.json({ ok: true }); +}); +``` + +4. Once every room is copied, remove the v1 binding. + +### Document size + +Durable Objects give each instance 10GB of SQLite storage, but the whole +document must fit in the instance's 128MB of memory. That memory limit — not +storage — is the practical ceiling on document size. + +### Keepalive and hibernation + +v2 registers a `"ping"` / `"pong"` auto-response via +`setWebSocketAutoResponse`. When a client sends `"ping"` as a keepalive, the +Workers runtime answers with `"pong"` directly — the Durable Object is never +woken from hibernation to run `webSocketMessage`, because the auto-response +is matched before that handler would be invoked at all. This interception is +the entire point of the hibernation optimization, and it's the single biggest +lever on duration billing. As a secondary safety net, `webSocketMessage` +ignores non-binary (string) messages, so even if a `"ping"` ever did reach a +woken instance it would be a no-op. + ## Usage ### With Hono shorthand @@ -174,6 +223,9 @@ export { YDurableObjects }; This API updates the state of the YDoc within a Durable Object. +`updateYDoc` takes a raw Yjs update — the same format `getYDoc` returns and +`Y.encodeStateAsUpdate(doc)` produces. It is not a sync-protocol message. + Example usage in Hono: ```typescript diff --git a/docs/superpowers/specs/2026-08-14-sqlite-migration-design.md b/docs/superpowers/specs/2026-08-14-sqlite-migration-design.md index 6544853..9bfae21 100644 --- a/docs/superpowers/specs/2026-08-14-sqlite-migration-design.md +++ b/docs/superpowers/specs/2026-08-14-sqlite-migration-design.md @@ -84,7 +84,7 @@ src/yjs/ hono/index.ts 変更なし client/setup.ts 変更なし src/middleware/index.ts 変更なし -src/index.ts yRoute（getByName への変更のみ） +src/index.ts 変更なし（`obj.get(obj.idFromName(id))` の二段形式を維持） ``` `SessionRegistry` を独立させることで、awareness の所有権管理・attachment の読み書き・hibernation 復帰時の再構築を 1 箇所に閉じ込め、`YDurableObjects` 本体の肥大化を防ぐ。単体でテスト可能な単位とする。 @@ -352,8 +352,10 @@ v1 と v2 を別バインディングに共存させ、RPC で内容をコピー ```ts app.post("/migrate/:id", async (c) => { const id = c.req.param("id"); - const legacy = c.env.Y_LEGACY.getByName(id); // v1 / KV バックエンド - const next = c.env.Y_DURABLE_OBJECTS.getByName(id); // v2 / SQLite バックエンド + const legacy = c.env.Y_LEGACY.get(c.env.Y_LEGACY.idFromName(id)); // v1 / KV バックエンド + const next = c.env.Y_DURABLE_OBJECTS.get( + c.env.Y_DURABLE_OBJECTS.idFromName(id), + ); // v2 / SQLite バックエンド await next.updateYDoc(await legacy.getYDoc()); return c.json({ ok: true }); }); @@ -361,6 +363,14 @@ app.post("/migrate/:id", async (c) => { v1 の `getYDoc()` は元から生の update を返しており、v2 の `updateYDoc()` は生の update を受け取るため、この組み合わせが成立する。ライブラリ側に移行専用コードを持つ必要がない。 +> **`getByName` を採用しなかった理由**: 本リポジトリにコミットされている +> `worker-configuration.d.ts` の `DurableObjectNamespace` 宣言には +> `newUniqueId` / `idFromName` / `idFromString` / `get` / `jurisdiction` しか +> 存在せず、`getByName` は宣言されていない。そのため `getByName` を使うコード +> は本リポジトリの型定義ではコンパイルできない。実装（`src/index.ts` の +> `yRoute`）およびこの移行レシピは、いずれも従来どおり +> `obj.get(obj.idFromName(id))` の二段形式を採用する。 + ### 設定ファイル - `wrangler.toml` および README の migration を `new_sqlite_classes` に変更する @@ -373,7 +383,9 @@ v1 の `getYDoc()` は元から生の update を返しており、v2 の `update ### `yRoute` -`obj.get(obj.idFromName(id))` を `obj.getByName(id)` に置き換える。それ以外の挙動は変更しない。 +`obj.get(obj.idFromName(id))` の二段形式を維持する。`getByName` への置き換えは行わない +— 本リポジトリの `worker-configuration.d.ts` の `DurableObjectNamespace` 宣言に +`getByName` が存在しないため、置き換えると型チェックが通らない。挙動は変更しない。 ## 8. テスト戦略 From 719d13c43c32bd0731f292b2dacef3a98edd5cd8 Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 22:03:58 +0900 Subject: [PATCH 21/26] docs: address review findings on changeset and doc accuracy - Changeset now documents the persistence-failure mass-disconnect behavior (close 1011 + state.abort()) and why it's safe. - CLAUDE.md's Testing Approach no longer claims storage tests use in-memory implementations; they run against the real SQLite backend via runInDurableObject. - Plan document's Task 9 section and self-review table no longer instruct switching to getByName, which isn't declared in this repo's worker-configuration.d.ts and wouldn't typecheck. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015mdNeCzNPZysRHzmkFp9Bj --- .changeset/sqlite-migration.md | 5 ++++ CLAUDE.md | 4 +++- .../plans/2026-08-14-sqlite-migration-v2.md | 24 +++++++++---------- 3 files changed, 20 insertions(+), 13 deletions(-) diff --git a/.changeset/sqlite-migration.md b/.changeset/sqlite-migration.md index dc161ca..466ba3e 100644 --- a/.changeset/sqlite-migration.md +++ b/.changeset/sqlite-migration.md @@ -23,6 +23,11 @@ key-value backend had forced. - Compaction no longer exceeds the 128-key limit of `delete()`. - Closing one connection no longer clears every participant's awareness state. - Updates are persisted in order and awaited rather than left as floating promises. +- If a storage write fails, the Durable Object now closes every connection + (`1011`) and aborts itself instead of continuing to serve in-memory state + that storage doesn't have. This is a deliberate mass disconnect, not an + outage: Yjs clients hold the full document, so they reconnect and re-sync + automatically. - Sync step 2 replies go only to the requesting client instead of the whole room. - Updates are no longer echoed back to their sender. - A malformed binary message closes only that connection instead of resetting diff --git a/CLAUDE.md b/CLAUDE.md index ae85f65..71f3de7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -97,4 +97,6 @@ Tests follow these patterns: - Unit tests for individual components - Integration tests using Cloudflare Workers test environment - WebSocket connection tests with mock implementations -- Storage tests with in-memory implementations +- Storage tests run against the real Durable Objects SQLite backend via + `runInDurableObject` (`@cloudflare/vitest-pool-workers`), not an in-memory + fake diff --git a/docs/superpowers/plans/2026-08-14-sqlite-migration-v2.md b/docs/superpowers/plans/2026-08-14-sqlite-migration-v2.md index 0387f39..29fe7a1 100644 --- a/docs/superpowers/plans/2026-08-14-sqlite-migration-v2.md +++ b/docs/superpowers/plans/2026-08-14-sqlite-migration-v2.md @@ -1709,12 +1709,12 @@ git commit -m "feat!: add hibernation auto-response, SQLite backend assertion, a ### Task 9: 公開 API を整える -`updateYDoc` が生の Yjs update を受け取るようにして `getYDoc()` との往復を成立させ（H-3）、`yRoute` を `getByName` に切り替える。 +`updateYDoc` が生の Yjs update を受け取るようにして `getYDoc()` との往復を成立させる（H-3）。`yRoute` の `obj.get(obj.idFromName(id))` はそのまま維持する（`getByName` へは切り替えない — 理由は Step 5 を参照）。 **Files:** - Modify: `src/yjs/index.ts`（`updateYDoc`） -- Modify: `src/index.ts:16`（`idFromName` → `getByName`） -- Modify: `src/e2e/index.ts:25-26,45-46,55-56`（同上） +- Modify なし: `src/index.ts`（`idFromName` の二段形式を維持。Step 5 参照） +- Modify なし: `src/e2e/index.ts`（同上） - Test: `src/e2e/y-durableobjects.test.ts`（`updateYDoc` のテストを書き換え） **Interfaces:** @@ -1785,16 +1785,16 @@ Expected: FAIL — `updateYDoc` が生の update をプロトコルメッセー `createSyncMessage` でラップして `updateYDoc` に渡している箇所（Task 7・Task 8 で追加したテストを含む）を、生の update を渡すように修正する。`webSocketMessage` に渡す箇所は引き続き `createSyncMessage` が必要である。 -- [ ] **Step 5: getByName に切り替える** +- [ ] **Step 5: `getByName` は採用しない（変更なし）** -`src/index.ts:15-16` を置き換える。 +`src/index.ts:15-16` および `src/e2e/index.ts` の該当箇所は、引き続き +`obj.get(obj.idFromName(id))` の二段形式を使う。`getByName` には切り替えない。 -```ts - const obj = selector(c.env as E["Bindings"]); - const stub = obj.getByName(c.req.param("id")); -``` - -`src/e2e/index.ts` の 3 箇所（`:25-26`, `:45-46`, `:55-56`）も同様に `const stub = c.env.Y_DURABLE_OBJECTS.getByName(roomId);` に置き換える。 +> **理由**: 本リポジトリにコミットされている `worker-configuration.d.ts` の +> `DurableObjectNamespace` 宣言には `newUniqueId` / `idFromName` / +> `idFromString` / `get` / `jurisdiction` しか存在せず、`getByName` は +> 宣言されていない。そのため `getByName` を使うコードは本リポジトリの型定義 +> ではコンパイルできず、`pnpm typecheck` が失敗する。 - [ ] **Step 6: テストが通ることを確認する** @@ -2005,7 +2005,7 @@ git commit -m "docs: document the SQLite migration and add the v2 changeset" | hibernation 衛生 | Task 8 | | KV バックエンド検出 | Task 8 | | `destroy()` | Task 8 | -| `getByName` | Task 9 | +| `getByName` は不採用（二段形式を維持） | Task 9 | | 移行レシピの文書化 | Task 10 | | 公開型の変更 | Task 4（`YStorage`）、Task 5（`WSSharedDoc`）、Task 6（`SessionAttachment`） | From 90bc41a57bc7f3d2ed4daf291a213ac9b30cf2ba Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 22:05:00 +0900 Subject: [PATCH 22/26] docs: fix last getByName sample in plan's Task 10 migration recipe The embedded README migration-recipe snippet in Task 10 of the plan still used getByName, which isn't declared in this repo's worker-configuration.d.ts. Switch it to the same two-step get(idFromName(id)) form used everywhere else. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015mdNeCzNPZysRHzmkFp9Bj --- docs/superpowers/plans/2026-08-14-sqlite-migration-v2.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/plans/2026-08-14-sqlite-migration-v2.md b/docs/superpowers/plans/2026-08-14-sqlite-migration-v2.md index 29fe7a1..c042bd7 100644 --- a/docs/superpowers/plans/2026-08-14-sqlite-migration-v2.md +++ b/docs/superpowers/plans/2026-08-14-sqlite-migration-v2.md @@ -1871,8 +1871,10 @@ Run both versions side by side and copy each room across: ```typescript app.post("/migrate/:id", async (c) => { const id = c.req.param("id"); - const legacy = c.env.Y_LEGACY.getByName(id); - const next = c.env.Y_DURABLE_OBJECTS.getByName(id); + const legacy = c.env.Y_LEGACY.get(c.env.Y_LEGACY.idFromName(id)); + const next = c.env.Y_DURABLE_OBJECTS.get( + c.env.Y_DURABLE_OBJECTS.idFromName(id), + ); await next.updateYDoc(await legacy.getYDoc()); From e5dce725624770266490f2203d4e0609c913f4c4 Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 22:30:57 +0900 Subject: [PATCH 23/26] fix: apply final code-review fix wave for SQLite migration v2 Addresses the last-gate review of feat/sqlite-migration-v2: chunk oversized updates in storeUpdate() (not just commit()) so a single update over Cloudflare's 2MB row limit no longer wedges a room permanently, floor the compaction threshold so a small maxRows can't thrash, guard the remaining unguarded socket paths (onStart's hibernation-wake loop, WSSharedDoc.send), make server-initiated closes actually unregister the session instead of relying on webSocketClose firing, have destroy() drain the persist queue and discard the in-memory Doc before deleting, fix the now-broken README RPC extension example, and clean up several smaller export/type/lint inconsistencies flagged in review. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015mdNeCzNPZysRHzmkFp9Bj --- README.md | 30 +++-- eslint.config.js | 5 + src/e2e/y-durableobjects.test.ts | 170 +++++++++++++++++++++++++++ src/index.ts | 2 + src/yjs/index.ts | 80 +++++++++++-- src/yjs/internal.ts | 10 +- src/yjs/remote/ws-shared-doc.test.ts | 17 +++ src/yjs/remote/ws-shared-doc.ts | 11 +- src/yjs/session/session.test.ts | 12 +- src/yjs/storage/sqlite.test.ts | 73 ++++++++++++ src/yjs/storage/sqlite.ts | 34 +++++- 11 files changed, 413 insertions(+), 31 deletions(-) diff --git a/README.md b/README.md index dd1a98d..bb275b9 100644 --- a/README.md +++ b/README.md @@ -263,24 +263,40 @@ export { YDurableObjects }; By supporting JS RPC, `y-durableobjects` allows for advanced operations through extensions. You can manipulate the protected fields for custom functionality: +`this.doc` is a `WSSharedDoc`, and its `update(message, origin)` method expects a +**sync-protocol-framed message** — the same bytes a WebSocket client sends over +the wire — not a raw Yjs update. `origin` identifies the source of the change; +it must not be a value already registered as a WebSocket listener (see +`notify()`), so a fresh object works. Frame a raw update before passing it in: + Example: ```typescript -import { applyUpdate, encodeStateAsUpdate } from "yjs"; +import { createEncoder, toUint8Array, writeVarUint } from "lib0/encoding"; +import { writeUpdate } from "y-protocols/sync"; import { YDurableObjects } from "y-durableobjects"; export class CustomDurableObject extends YDurableObjects { - async customMethod() { - // Access and manipulate the YDoc state - const update = new Uint8Array([ - /* some update data */ - ]); - this.doc.update(update); + async customMethod(update: Uint8Array) { + // Wrap the raw update as a sync-protocol message (type 0 = sync) so + // this.doc.update() can dispatch it the same way it dispatches an + // incoming WebSocket message. + const encoder = createEncoder(); + writeVarUint(encoder, 0 /* sync */); + writeUpdate(encoder, update); + + this.doc.update(toUint8Array(encoder), {}); await this.cleanup(); } } ``` +If you already have a raw Yjs update and don't need protocol-level dispatch +(sync step replies, etc.), applying it directly with `applyUpdate(this.doc, update)` +from `yjs` — the same way the built-in `updateYDoc()` RPC method does — is +simpler and broadcasts to connected clients just the same, since `WSSharedDoc` +listens for its own `Doc` "update" event either way. + ### Hono RPC support for ClientSide - Utilizes Hono's WebSocket Helper, making the `$ws` method available in `hono/client` for WebSocket communications. diff --git a/eslint.config.js b/eslint.config.js index 590903a..d2c0cb8 100644 --- a/eslint.config.js +++ b/eslint.config.js @@ -14,6 +14,11 @@ const standard = fixupConfigRules(compat.config({ extends: ["standard"] })); /** @type {import("eslint").Linter.Config[]} */ const config = [ + { + linterOptions: { + reportUnusedDisableDirectives: "error", + }, + }, eslint.configs.recommended, ...standard, { diff --git a/src/e2e/y-durableobjects.test.ts b/src/e2e/y-durableobjects.test.ts index 1b538d7..e903623 100644 --- a/src/e2e/y-durableobjects.test.ts +++ b/src/e2e/y-durableobjects.test.ts @@ -432,6 +432,43 @@ describe("YDurableObjects", () => { }); }); + it("unregisters the session immediately when the exception boundary closes a socket, without relying on webSocketClose", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject(stub, async (instance: InternalYDurableObject) => { + await instance.createRoom("room1"); + const [server] = Array.from(instance.sessions.sockets()); + + // Give it real awareness state to remove, so the assertions below + // actually exercise removeAwarenessStates() rather than a no-op. + const awareness = new Awareness(new Doc()); + awareness.setLocalStateField("user", { name: "a" }); + await instance.webSocketMessage( + server, + createAwarenessMessage(awareness).slice(0).buffer, + ); + expect(instance.sessions.clientIdsOf(server)).toEqual([ + awareness.clientID, + ]); + + // 未知のメッセージ型。例外境界がこのソケットを 1003 で閉じる。 + const malformed = new Uint8Array([99]).buffer; + await instance.webSocketMessage(server, malformed); + + // This DO closed the socket itself; webSocketClose is not guaranteed + // to fire for a close the DO initiated. Without the exception + // boundary's own explicit unregisterWebSocket() call, the session and + // this socket's awareness state would leak indefinitely (or until + // webSocketClose happens to fire on its own), instead of being gone + // by the time webSocketMessage() resolves. + expect(instance.sessions.has(server)).toBe(false); + expect(instance.doc.awareness.getStates().has(awareness.clientID)).toBe( + false, + ); + }); + }); + it("persists an update before webSocketMessage resolves", async () => { const id = env.Y_DURABLE_OBJECTS.newUniqueId(); const stub = env.Y_DURABLE_OBJECTS.get(id); @@ -586,14 +623,147 @@ describe("YDurableObjects", () => { await runInDurableObject(stub, async (instance: InternalYDurableObject) => { await instance.createRoom("room1"); + const [server] = Array.from(instance.sessions.sockets()); await instance.updateYDoc(createYDocMessage("bye").slice(0)); await instance.destroy(); + // Storage is cleared. + expect(await instance.storage.getUpdate()).toBeNull(); + + // The connection is actually closed, not just left registered. + expect(server.readyState).not.toBe(WebSocket.READY_STATE_OPEN); + expect(instance.sessions.size).toBe(0); + + // The in-memory Doc is discarded too, not just storage. Otherwise a + // client connecting to this still-live instance after destroy() would + // receive the old content via sync step 1/2, and subsequent deltas + // would reference struct parents that no longer exist anywhere. + expect(instance.doc.getText("root").toString()).toBe(""); + }); + }); + + it("drains the persistence queue before deleting so a slow write cannot resurrect data after destroy", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject(stub, async (instance: InternalYDurableObject) => { + const client = await instance.createRoom("room1"); + + // Delay storeUpdate so it is still in flight (queued on `this.persist`) + // when destroy() starts running -- the same hazard a real slow SQLite + // write could hit, made deterministic here. + const originalStoreUpdate = instance.storage.storeUpdate.bind( + instance.storage, + ); + instance.storage.storeUpdate = async (update: Uint8Array) => { + await new Promise((resolve) => setTimeout(resolve, 10)); + await originalStoreUpdate(update); + }; + + const message = createSyncMessage(createYDocMessage("late")); + // Deliberately not awaited: doc.update() runs synchronously inside + // this call and enqueues the delayed write onto `this.persist` before + // this line returns, but the write itself is still pending. + const pending = instance.webSocketMessage( + client, + message.slice(0).buffer, + ); + + // destroy() must await the same persist queue before deleting, or the + // delayed write above would land after DELETE FROM updates and + // resurrect an orphan row. + await instance.destroy(); + await pending; + expect(await instance.storage.getUpdate()).toBeNull(); }); }); + it("keeps registering the remaining sockets when an earlier one throws on the hibernation-wake path", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject( + stub, + async (instance: InternalYDurableObject, state: DurableObjectState) => { + // Simulate what onStart() sees on a real hibernation wake: sockets + // already accepted by the runtime (state.getWebSockets()), with the + // in-memory SessionRegistry empty because this process never + // registered them. Accept two: the loop must not let the first + // socket's failure stop it from reaching the second. + const brokenPair = new WebSocketPair(); + const brokenServer = brokenPair[1]; + brokenServer.serializeAttachment({ + roomId: "room1", + connectedAt: 0, + clientIds: [], + }); + state.acceptWebSocket(brokenServer); + + const healthyPair = new WebSocketPair(); + const healthyServer = healthyPair[1]; + healthyServer.serializeAttachment({ + roomId: "room1", + connectedAt: 0, + clientIds: [], + }); + state.acceptWebSocket(healthyServer); + + // registerWebSocket() -> setupWSConnection() -> ws.send(). Simulate + // the socket workerd hands back already dead, the way it can on + // hibernation wake. + brokenServer.send = () => { + throw new Error("simulated dead socket on hibernation wake"); + }; + + expect(instance.sessions.has(healthyServer)).toBe(false); + + // onStart() is what a real cold start / hibernation wake runs. + await expect(instance.onStart()).resolves.toBeUndefined(); + + // The broken socket's registration failure must not have stopped + // the loop before it reached the socket after it. + expect(instance.sessions.has(healthyServer)).toBe(true); + }, + ); + }); + + it("restores session ownership from a socket's attachment when onStart() re-registers it", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject( + stub, + async (instance: InternalYDurableObject, state: DurableObjectState) => { + // Simulate what a real hibernation wake leaves behind: a WebSocket + // that state.getWebSockets() already returns, carrying a + // SessionAttachment with clientIds recorded by a prior connection, + // but with the in-memory SessionRegistry never having seen it in + // this process (it never called sessions.add() for it before now). + const pair = new WebSocketPair(); + const server = pair[1]; + server.serializeAttachment({ + roomId: "room1", + connectedAt: 0, + clientIds: [42], + }); + state.acceptWebSocket(server); + + expect(instance.sessions.has(server)).toBe(false); + + // onStart() is the actual reconstruction path a cold start / + // hibernation wake runs -- not a stand-in for it. Real hibernation + // cannot be forced in this test environment, so this drives the + // genuine code path instead of faking eviction/restart. + await instance.onStart(); + + expect(instance.sessions.has(server)).toBe(true); + expect(instance.sessions.clientIdsOf(server)).toEqual([42]); + }, + ); + }); + it("still destroys storage when one socket's close() throws", async () => { const id = env.Y_DURABLE_OBJECTS.newUniqueId(); const stub = env.Y_DURABLE_OBJECTS.get(id); diff --git a/src/index.ts b/src/index.ts index 6e8b97a..f6006c7 100644 --- a/src/index.ts +++ b/src/index.ts @@ -37,5 +37,7 @@ export const yRoute = (selector: Selector) => { export { YDurableObjects, type YDurableObjectsAppType } from "./yjs"; export type YRoute = ReturnType; +export type { SessionAttachment } from "./yjs"; +export { YSqliteStorage } from "./yjs/storage"; export type { YStorage, YSqliteStorageOptions } from "./yjs/storage"; export { type RemoteDoc, WSSharedDoc } from "./yjs/remote"; diff --git a/src/yjs/index.ts b/src/yjs/index.ts index d3c7254..fa3583a 100644 --- a/src/yjs/index.ts +++ b/src/yjs/index.ts @@ -81,7 +81,22 @@ export class YDurableObjects extends DurableObject< } for (const ws of this.state.getWebSockets()) { - this.registerWebSocket(ws); + // ここはハイバネーションからの復帰パス。state.getWebSockets() が返す + // ソケットのうち 1 つがすでに死んでいる（閉じている・エラー状態）こと + // があり得る。registerWebSocket() は setupWSConnection() 経由で + // ws.send() を呼ぶため、それだけで例外を投げうる。ここでガードせずに + // 例外を外へ伝播させると、この for ループがそこで止まり、以降の + // ソケットが二度と登録されないまま静かに配信を受けなくなる。さらに + // 例外はコンストラクタの blockConcurrencyWhile まで伝播し、Durable + // Object 全体がリセットされる — 例外境界を設けた意味そのものが + // 失われる。1 つのソケットの復帰失敗が他のソケットの復帰を妨げては + // いけない。 + try { + this.registerWebSocket(ws); + } catch (e) { + // eslint-disable-next-line no-console + console.error(e); + } } // 本番では onStart はコンストラクタの blockConcurrencyWhile からしか @@ -92,21 +107,29 @@ export class YDurableObjects extends DurableObject< // 重複する。 if (!this.listenersRegistered) { this.listenersRegistered = true; - - this.doc.on("update", (update: Uint8Array) => { - this.schedulePersist(update); - }); - this.doc.awareness.on( - "update", - ({ added, updated }: AwarenessChanges, origin: unknown) => { - if (origin instanceof WebSocket) { - this.sessions.track(origin, [...added, ...updated]); - } - }, - ); + this.wireDocListeners(); } } + /** + * this.doc に対して update / awareness update のリスナーを配線する。 + * onStart() から一度だけ呼ばれるほか、destroy() が this.doc を新しい + * WSSharedDoc に差し替えたときにも呼び直す。 + */ + private wireDocListeners(): void { + this.doc.on("update", (update: Uint8Array) => { + this.schedulePersist(update); + }); + this.doc.awareness.on( + "update", + ({ added, updated }: AwarenessChanges, origin: unknown) => { + if (origin instanceof WebSocket) { + this.sessions.track(origin, [...added, ...updated]); + } + }, + ); + } + protected createRoom(roomId: string) { const pair = new WebSocketPair(); const client = pair[0]; @@ -155,8 +178,32 @@ export class YDurableObjects extends DurableObject< // eslint-disable-next-line no-console console.error(e); } + + // この DO が自分から閉じたソケットは webSocketClose が確実に発火する + // 保証がない。発火しなければ WSSharedDoc のリスナーがリークして以降 + // 毎回の broadcast が失敗するし、このソケットの awareness state が + // 亡霊カーソルとしてルームに残り続け、sessions.size が 0 に落ちないため + // cleanup() の「全員退出でコンパクション」条件も二度と成立しなくなる。 + // unregisterWebSocket は冪等なので、webSocketClose が別途発火しても + // 問題ない。 + await this.unregisterWebSocket(ws); } + + // destroy() の時点で永続化キューにまだ書き込みが残っていることがある。 + // 先に drain してから DELETE しないと、キューに残っていた storeUpdate が + // DELETE FROM updates の後に着地し、孤児行として復活してしまう。 + await this.persist; await this.storage.destroy(); + + // メモリ上の Doc をストレージより進んだまま残すと、まだ生きているこの + // インスタンスに接続してきたクライアントが sync step 1/2 で消したはずの + // 内容を受け取ってしまい、以降の差分は実在しない親 struct を参照する + // ようになる。onPersistFailure が state.abort() でインスタンスごと + // 破棄するのと同じ理由だが、destroy() は RPC でありインスタンスを + // 生かしたまま応答する必要があるため、abort() の代わりに Doc を + // 作り直して同じ効果を得る。 + this.doc = new WSSharedDoc(); + this.wireDocListeners(); } async webSocketMessage( @@ -172,6 +219,13 @@ export class YDurableObjects extends DurableObject< console.error("[y-durableobjects] invalid message", error); ws.close(1003, "invalid message"); + // この DO 自身が閉じたソケットに対して webSocketClose が確実に発火する + // 保証はない。発火しなければ unregisterWebSocket が一生呼ばれず、 + // WSSharedDoc のリスナーがリークして以降の broadcast がずっと失敗し、 + // このソケットの awareness state もルームに残り続ける。冪等なので + // webSocketClose が別途発火しても問題ない。 + await this.unregisterWebSocket(ws); + return; } diff --git a/src/yjs/internal.ts b/src/yjs/internal.ts index c586168..19322f8 100644 --- a/src/yjs/internal.ts +++ b/src/yjs/internal.ts @@ -14,14 +14,14 @@ export interface InternalYDurableObject { createRoom(roomId: string): WebSocket; registerWebSocket(ws: WebSocket): void; - unregisterWebSocket(ws: WebSocket): void; - cleanup(): void; + unregisterWebSocket(ws: WebSocket): Promise; + cleanup(): Promise; // public api fetch(request: Request): Promise; - webSocketMessage(ws: WebSocket, message: string | ArrayBuffer): void; - webSocketError(ws: WebSocket): void; - webSocketClose(ws: WebSocket): void; + webSocketMessage(ws: WebSocket, message: string | ArrayBuffer): Promise; + webSocketError(ws: WebSocket): Promise; + webSocketClose(ws: WebSocket): Promise; getYDoc(): Promise; updateYDoc(update: Uint8Array): Promise; diff --git a/src/yjs/remote/ws-shared-doc.test.ts b/src/yjs/remote/ws-shared-doc.test.ts index 44f813d..35cba6c 100644 --- a/src/yjs/remote/ws-shared-doc.test.ts +++ b/src/yjs/remote/ws-shared-doc.test.ts @@ -150,6 +150,23 @@ describe("WSSharedDoc", () => { expect(toReceiver.length).toBe(1); }); + it("does not let a throwing recipient's send escape update()", () => { + const doc = new WSSharedDoc(); + const requester = {}; + doc.notify(requester, () => { + throw new Error("dead socket"); + }); + + // A sync step 1 reply goes only to the requester via the private + // send() path (this.send(origin, ...)), not broadcast() -- so this + // exercises send()'s own guard, not broadcast()'s. + const encoder = createEncoder(); + writeVarUint(encoder, messageType.sync); + writeSyncStep1(encoder, new Doc()); + + expect(() => doc.update(toUint8Array(encoder), requester)).not.toThrow(); + }); + it("throws on an unknown message type", () => { const doc = new WSSharedDoc(); const encoder = createEncoder(); diff --git a/src/yjs/remote/ws-shared-doc.ts b/src/yjs/remote/ws-shared-doc.ts index b8f8840..c476863 100644 --- a/src/yjs/remote/ws-shared-doc.ts +++ b/src/yjs/remote/ws-shared-doc.ts @@ -130,7 +130,16 @@ export class WSSharedDoc extends Doc implements Notification { } private send(origin: object, message: Uint8Array) { - this.listeners.get(origin)?.(message); + try { + this.listeners.get(origin)?.(message); + } catch (e) { + // A dead socket's send() throwing here would otherwise escape + // update() and be misreported by callers (e.g. webSocketMessage's + // exception boundary) as an "invalid message", when the real cause + // was a disconnected recipient. Same shape as broadcast()'s guard. + // eslint-disable-next-line no-console + console.error(e); + } } private broadcast(message: Uint8Array, exclude: unknown) { diff --git a/src/yjs/session/session.test.ts b/src/yjs/session/session.test.ts index ce6658d..0448be4 100644 --- a/src/yjs/session/session.test.ts +++ b/src/yjs/session/session.test.ts @@ -57,8 +57,16 @@ describe("SessionRegistry", () => { expect(registry.clientIdsOf(ws).sort()).toEqual([7, 8]); }); - it("restores ownership from an existing attachment after hibernation", () => { - // hibernation 復帰を模す。registry は空だが WebSocket の attachment は残っている + it("reads clientIds from the attachment already present when add() is called", () => { + // This only exercises SessionRegistry in isolation: add() with a socket + // whose attachment already carries clientIds, on a registry that never + // called track() itself. It does NOT drive state.getWebSockets() or + // onStart()'s reconstruction loop, so it does not cover hibernation + // wake-up -- see "restores session ownership from a socket's attachment + // when onStart() re-registers it" in src/e2e/y-durableobjects.test.ts + // for that path. Real hibernation cannot be forced in this test + // environment, so that test drives the actual onStart() code path + // instead of faking eviction/restart. const ws = fakeSocket(attachment([42])); const registry = new SessionRegistry(); registry.add(ws, () => {}); diff --git a/src/yjs/storage/sqlite.test.ts b/src/yjs/storage/sqlite.test.ts index c3a9733..039aef7 100644 --- a/src/yjs/storage/sqlite.test.ts +++ b/src/yjs/storage/sqlite.test.ts @@ -183,6 +183,79 @@ describe("YSqliteStorage", () => { }); }); + it("splits a single update larger than maxChunkBytes without needing commit()", async () => { + await withStorageAndSql( + async (storage, sql) => { + const doc = new Doc(); + doc.getText("root").insert(0, "z".repeat(5000)); + const update = encodeStateAsUpdate(doc); + expect(update.byteLength).toBeGreaterThan(200); + + // storeUpdate() alone, with no commit() call, must split an + // oversized update into multiple rows. A bare INSERT of the whole + // update would eventually exceed Cloudflare's 2MB SQLite BLOB limit + // and throw inside storeUpdate(), wedging the room forever (every + // reconnect resends the same oversized update and hits the same + // error). This uses a small maxChunkBytes instead of an actual >2MB + // payload so the test stays fast, but exercises the same code path. + await storage.storeUpdate(update); + + expect(countRows(sql)).toBeGreaterThan(1); + + const restored = await storage.getUpdate(); + expect(textOf(restored!)).toBe("z".repeat(5000)); + }, + { maxChunkBytes: 200, maxRows: 1000 }, + ); + }); + + it("does not recompact on every storeUpdate once maxRows is below the data's minimum chunk count", async () => { + await withStorageAndSql( + async (storage, sql) => { + let commitCalls = 0; + const originalCommit = storage.commit.bind(storage); + storage.commit = async () => { + commitCalls += 1; + await originalCommit(); + }; + + const doc = new Doc(); + const text = doc.getText("root"); + const updates: Uint8Array[] = []; + doc.on("update", (update: Uint8Array) => updates.push(update)); + // Each individual insert's update must stay under maxChunkBytes on + // its own (so a single storeUpdate() call never needs to split by + // itself) -- only the *cumulative* merged document, once compacted, + // should need more than maxRows chunks. + for (let i = 0; i < 30; i++) { + text.insert(text.length, "ab"); + } + + for (const update of updates) { + expect(update.byteLength).toBeLessThan(300); + await storage.storeUpdate(update); + } + + // maxRows (1) is far below the number of rows the merged content + // actually needs at maxChunkBytes (300), so compaction can never + // bring the row count back under maxRows. Without flooring the + // threshold at (roughly) the row count the last compaction actually + // produced, every single storeUpdate() call after the first + // compaction re-triggers a full commit() (confirmed empirically: + // 29 of 30 calls recompact without the floor). The floor at least + // halves that (observed: 15 of 30) by skipping compaction until the + // row count has grown past what the last compaction produced, not + // just past the configured maxRows. + expect(commitCalls).toBeGreaterThan(0); + expect(commitCalls).toBeLessThan(updates.length * 0.7); + + const restored = await storage.getUpdate(); + expect(textOf(restored!)).toBe(text.toString()); + }, + { maxRows: 1, maxChunkBytes: 300 }, + ); + }); + it("throws when a continuation row has no preceding row to attach to", async () => { const stub = env.Y_DURABLE_OBJECTS.get(env.Y_DURABLE_OBJECTS.newUniqueId()); await runInDurableObject(stub, async (_instance, state) => { diff --git a/src/yjs/storage/sqlite.ts b/src/yjs/storage/sqlite.ts index 3c103f1..5ece881 100644 --- a/src/yjs/storage/sqlite.ts +++ b/src/yjs/storage/sqlite.ts @@ -54,6 +54,14 @@ export class YSqliteStorage implements YStorage { readonly #maxRows: number; readonly #maxChunkBytes: number; #rowCount: number; + /** + * commit() を呼ばずに済む行数の下限。通常は #maxRows と同じだが、 + * コンパクション結果が #maxRows を超える行数を必要とする場合は + * その行数（+1）まで引き上げる。そうしないと、以降の storeUpdate が + * 呼ばれるたびに毎回フルコンパクションを再実行してしまう + * （コンパクションしても #maxRows 以下にはならないため）。 + */ + #compactionFloor: number; constructor(sql: SqlStorage, options?: YSqliteStorageOptions) { this.#maxChunkBytes = options?.maxChunkBytes ?? DEFAULT_MAX_CHUNK_BYTES; @@ -62,6 +70,7 @@ export class YSqliteStorage implements YStorage { throw new Error("maxChunkBytes must not exceed 2MB"); } this.#maxRows = options?.maxRows ?? DEFAULT_MAX_ROWS; + this.#compactionFloor = this.#maxRows; this.#sql = sql; migrate(sql); @@ -76,10 +85,24 @@ export class YSqliteStorage implements YStorage { } async storeUpdate(update: Uint8Array): Promise { - this.#sql.exec(INSERT_UPDATE, UpdateKind.standalone, update); - this.#rowCount += 1; + // Cloudflare の SQLite は BLOB 1 行あたり 2MB までしか許さない。単一の + // update がその上限を超えることは実際に起こる（大きな貼り付け、埋め込み + // 画像、新規クライアントの sync step 2 が運ぶ大きなドキュメント全体など）。 + // #split() で maxChunkBytes 以下の断片に割ってから複数行として書き込む。 + // commit() と同じ分割ロジックを再利用するので、#readAll() の復元側は + // 変更不要（continuation はすでに連結される）。 + const chunks = this.#split(update); + + // ここから下では await を挟まないこと。 + // 連続した同期書き込みが暗黙のトランザクションとして atomic に適用される。 + for (const [index, chunk] of chunks.entries()) { + const kind = + index === 0 ? UpdateKind.standalone : UpdateKind.continuation; + this.#sql.exec(INSERT_UPDATE, kind, chunk); + } + this.#rowCount += chunks.length; - if (this.#rowCount > this.#maxRows) { + if (this.#rowCount > this.#compactionFloor) { await this.commit(); } } @@ -101,6 +124,11 @@ export class YSqliteStorage implements YStorage { this.#sql.exec(INSERT_UPDATE, kind, chunk); } this.#rowCount = chunks.length; + // このコンパクションが生んだ実際の行数が #maxRows を超えるなら、 + // それより低いしきい値で次の storeUpdate を毎回コンパクションさせても + // 無駄な全件書き直しを繰り返すだけで行数は減らない。しきい値をこの + // 行数（+1）まで引き上げて、実際に増えたときだけ再コンパクションする。 + this.#compactionFloor = Math.max(this.#maxRows, chunks.length + 1); } async destroy(): Promise { From d6b17002ffe052f7ea67ac3a7737da994d63e1f7 Mon Sep 17 00:00:00 2001 From: napochaan Date: Fri, 14 Aug 2026 23:03:32 +0900 Subject: [PATCH 24/26] fix: guard ws.close() in webSocketMessage's exception boundary The fix wave's own unregisterWebSocket() call in webSocketMessage's catch block sat after a bare ws.close(1003, ...). workerd can throw closing an already-errored socket -- the same hazard already guarded in destroy() and onPersistFailure -- which would both skip the unregister (reinstating the IMPORTANT 3 leak) and reject webSocketMessage, resetting the DO on a single malformed frame. Wrap close() in the same try/catch/log pattern used elsewhere so neither depends on close() succeeding. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015mdNeCzNPZysRHzmkFp9Bj --- src/e2e/y-durableobjects.test.ts | 45 ++++++++++++++++++++++++++++++++ src/yjs/index.ts | 16 +++++++++++- 2 files changed, 60 insertions(+), 1 deletion(-) diff --git a/src/e2e/y-durableobjects.test.ts b/src/e2e/y-durableobjects.test.ts index e903623..7fbf932 100644 --- a/src/e2e/y-durableobjects.test.ts +++ b/src/e2e/y-durableobjects.test.ts @@ -469,6 +469,51 @@ describe("YDurableObjects", () => { }); }); + it("still unregisters the session and resolves when close() throws inside the malformed-message exception boundary", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject(stub, async (instance: InternalYDurableObject) => { + await instance.createRoom("room1"); + const [server] = Array.from(instance.sessions.sockets()); + + const awareness = new Awareness(new Doc()); + awareness.setLocalStateField("user", { name: "a" }); + await instance.webSocketMessage( + server, + createAwarenessMessage(awareness).slice(0).buffer, + ); + expect(instance.sessions.clientIdsOf(server)).toEqual([ + awareness.clientID, + ]); + + // workerd throws if close() is called on a socket that is already + // closed or errored -- exactly what could be true of the socket that + // just sent a malformed frame. Simulate that here. + server.close = () => { + throw new Error("already closed"); + }; + + const malformed = new Uint8Array([99]).buffer; + + // webSocketMessage must still resolve normally: the close() throwing + // must not escape the exception boundary and reject the call, or the + // whole room resets on a single malformed frame -- the exact outage + // H-4 exists to prevent. + await expect( + instance.webSocketMessage(server, malformed), + ).resolves.toBeUndefined(); + + // And unregisterWebSocket() must still have run despite close() + // throwing, or the session/awareness leak IMPORTANT 3 fixed comes + // back for any socket close() fails on. + expect(instance.sessions.has(server)).toBe(false); + expect(instance.doc.awareness.getStates().has(awareness.clientID)).toBe( + false, + ); + }); + }); + it("persists an update before webSocketMessage resolves", async () => { const id = env.Y_DURABLE_OBJECTS.newUniqueId(); const stub = env.Y_DURABLE_OBJECTS.get(id); diff --git a/src/yjs/index.ts b/src/yjs/index.ts index fa3583a..ef36437 100644 --- a/src/yjs/index.ts +++ b/src/yjs/index.ts @@ -217,7 +217,21 @@ export class YDurableObjects extends DurableObject< } catch (error) { // eslint-disable-next-line no-console console.error("[y-durableobjects] invalid message", error); - ws.close(1003, "invalid message"); + + // すでに閉じている・エラー状態のソケットに close() を呼ぶと workerd は + // 例外を投げることがある（destroy()/onPersistFailure と同じ危険）。 + // それがこの下の unregisterWebSocket() 呼び出しを妨げてはいけない。 + // 妨げると unregisterWebSocket が一生呼ばれず、WSSharedDoc のリスナーが + // リークして以降の broadcast がずっと失敗するばかりか、この catch + // ブロックの外へ例外が漏れて webSocketMessage 全体が reject し、DO が + // リセットされて部屋の全接続が落ちる — このガード自体が防ごうとして + // いた全断障害を、形を変えて再現してしまう。 + try { + ws.close(1003, "invalid message"); + } catch (e) { + // eslint-disable-next-line no-console + console.error(e); + } // この DO 自身が閉じたソケットに対して webSocketClose が確実に発火する // 保証はない。発火しなければ unregisterWebSocket が一生呼ばれず、 From 73594d10267d33fb1adf0a0b1a70ee87a83f36ab Mon Sep 17 00:00:00 2001 From: napochaan Date: Sat, 15 Aug 2026 00:21:12 +0900 Subject: [PATCH 25/26] fix: address code review findings on SQLite migration v2 - Document that wrangler.toml is only a fresh-environment test fixture, and that a deployed v1 Worker must append a migration rather than rewrite the "v1" tag; show the concrete two-binding wrangler config and npm-aliasing recipe for running v1 and v2 side by side in README. - Make awareness clientID ownership exclusive in SessionRegistry.track() so an overlapping reconnect can't leave both the old and new socket claiming the same ID, which previously caused a false departure broadcast when the old socket closed. - Destroy the old WSSharedDoc before replacing it in destroy(), so the Awareness instance it owns (and its repeating setInterval) is cleaned up instead of leaking for the rest of the Durable Object's lifetime. - Validate maxChunkBytes and maxRows as positive integers in YSqliteStorage, since a non-positive maxChunkBytes previously caused an infinite loop in #split(). compaction floor from the row count actually loaded (not reset to maxRows), so a rehydrated large document doesn't immediately recompact on its first write after waking. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015mdNeCzNPZysRHzmkFp9Bj --- README.md | 62 +++++++++++++++++++++++-- src/e2e/y-durableobjects.test.ts | 31 +++++++++++++ src/yjs/index.ts | 10 ++++ src/yjs/session/index.ts | 24 ++++++++++ src/yjs/session/session.test.ts | 39 ++++++++++++++++ src/yjs/storage/sqlite.test.ts | 78 ++++++++++++++++++++++++++++++++ src/yjs/storage/sqlite.ts | 28 +++++++++++- wrangler.toml | 13 ++++++ 8 files changed, 280 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index bb275b9..92adda4 100644 --- a/README.md +++ b/README.md @@ -67,9 +67,60 @@ Cloudflare rejects it with `storage_type_mismatch`. Run both versions side by side and copy each room across: -1. Keep your existing v1 binding (for example `Y_LEGACY`) pinned to - `y-durableobjects@1`. -2. Add a new binding backed by v2 with `new_sqlite_classes`. +1. Depend on both package versions at once, using npm aliasing to install + the old one under a different name: + + ```json + { + "dependencies": { + "y-durableobjects": "^2.0.0", + "y-durableobjects-v1": "npm:y-durableobjects@^1" + } + } + ``` + + Each installed version exports its own `YDurableObjects` class, so + re-export them from your Worker's entry script under distinct local + names — this is what lets the wrangler config below register them as two + separate Durable Object classes: + + ```typescript + export { YDurableObjects as YDurableObjectsLegacy } from "y-durableobjects-v1"; + export { YDurableObjects as YDurableObjectsSqlite } from "y-durableobjects"; + ``` + +2. In `wrangler.toml`, keep your existing v1 migration entry byte-for-byte — + migration history is append-only and a namespace's storage type is + immutable, so editing tag `"v1"` (renaming its class or switching it to + `new_sqlite_classes`) cannot convert the already-deployed key-value + namespace and will break it. Instead **append** a new migration with a + new tag that registers a distinct class name for the SQLite-backed + object, and add a second binding for it: + + ```toml + [[durable_objects.bindings]] + name = "Y_LEGACY" + class_name = "YDurableObjectsLegacy" + + [[durable_objects.bindings]] + name = "Y_DURABLE_OBJECTS" + class_name = "YDurableObjectsSqlite" + + [[migrations]] + tag = "v1" + new_classes = ["YDurableObjectsLegacy"] # unchanged from what's already deployed + + [[migrations]] + tag = "v2" + new_sqlite_classes = ["YDurableObjectsSqlite"] # appended, new tag, new class name + ``` + + The class name your v1 binding already points to in production may not + literally be `YDurableObjectsLegacy` — use whatever name is actually + recorded in your deployed `"v1"` migration entry (do not rename it), and + pick any unused name for the new SQLite class as long as it's different + from the legacy one. + 3. Copy each room over. `getYDoc()` returns a raw Yjs update and v2's `updateYDoc()` accepts one, so a single round trip is enough: @@ -87,7 +138,10 @@ app.post("/migrate/:id", async (c) => { }); ``` -4. Once every room is copied, remove the v1 binding. +4. Once every room is copied, remove the v1 binding, its export, and the + `y-durableobjects-v1` dependency. Leave the `"v1"` migration entry in + `wrangler.toml` in place — migration history is append-only, so old tags + must stay even after their class is no longer bound. ### Document size diff --git a/src/e2e/y-durableobjects.test.ts b/src/e2e/y-durableobjects.test.ts index 7fbf932..894226e 100644 --- a/src/e2e/y-durableobjects.test.ts +++ b/src/e2e/y-durableobjects.test.ts @@ -688,6 +688,37 @@ describe("YDurableObjects", () => { }); }); + it("destroys the replaced document instead of only discarding it", async () => { + const id = env.Y_DURABLE_OBJECTS.newUniqueId(); + const stub = env.Y_DURABLE_OBJECTS.get(id); + + await runInDurableObject(stub, async (instance: InternalYDurableObject) => { + const oldDoc = instance.doc; + + await instance.destroy(); + + // destroy() replaces this.doc with a fresh WSSharedDoc. WSSharedDoc's + // constructor builds a y-protocols Awareness, which installs a + // repeating setInterval and a `doc.on('destroy', ...)` listener that + // stay alive until the Yjs document itself is destroyed. If the old + // doc is merely dropped (`this.doc = new WSSharedDoc()`) instead of + // destroyed first, that interval and listener leak for the rest of + // this Durable Object's lifetime -- once per room destruction. + // + // Y.Doc.destroy() sets `isDestroyed = true` and emits 'destroy', + // which is what Awareness listens for to clear its own interval (see + // node_modules/.pnpm/y-protocols@*/node_modules/y-protocols/awareness.js, + // lines 78-88: `doc.on('destroy', () => this.destroy())`, and + // `destroy()` calls `clearInterval(this._checkInterval)`). isDestroyed + // is the only state destroy() leaves behind that a test can observe + // directly -- the timer/listener cleanup itself isn't inspectable + // from here. + expect(oldDoc.isDestroyed).toBe(true); + expect(instance.doc).not.toBe(oldDoc); + expect(instance.doc.isDestroyed).toBe(false); + }); + }); + it("drains the persistence queue before deleting so a slow write cannot resurrect data after destroy", async () => { const id = env.Y_DURABLE_OBJECTS.newUniqueId(); const stub = env.Y_DURABLE_OBJECTS.get(id); diff --git a/src/yjs/index.ts b/src/yjs/index.ts index ef36437..1ae6d94 100644 --- a/src/yjs/index.ts +++ b/src/yjs/index.ts @@ -202,6 +202,16 @@ export class YDurableObjects extends DurableObject< // 破棄するのと同じ理由だが、destroy() は RPC でありインスタンスを // 生かしたまま応答する必要があるため、abort() の代わりに Doc を // 作り直して同じ効果を得る。 + // + // 差し替える前に古い Doc を破棄すること。WSSharedDoc は y-protocols の + // Awareness を構築しており、Awareness は repeating setInterval と + // `doc.on('destroy', () => this.destroy())` を登録する。Y.Doc.destroy() + // を呼ばずに参照を捨てるだけだと、この interval と listener が Durable + // Object の残り寿命の間ずっとリークする（部屋を破棄するたびに 1 つずつ）。 + // Y.Doc.destroy() は 'destroy' イベントを発火して自身の全リスナーを + // 解除するので、Awareness の破棄はそれだけで連鎖する + // (node_modules の y-protocols/awareness.js で確認済み)。 + this.doc.destroy(); this.doc = new WSSharedDoc(); this.wireDocListeners(); } diff --git a/src/yjs/session/index.ts b/src/yjs/session/index.ts index 1366276..0a3e9e9 100644 --- a/src/yjs/session/index.ts +++ b/src/yjs/session/index.ts @@ -55,11 +55,35 @@ export class SessionRegistry { /** * この接続が所有する clientID を記録する。 * 実際に増えたときだけ attachment を書き直すので、通常は接続あたり 1 回で済む。 + * + * 所有権は排他的にする。overlapping reconnect（Yjs の Doc がソケットの + * 再接続を跨いで生き残るケース）では、同じ awareness clientID が旧ソケット + * と新ソケットの両方から送られてくることがある。ここで他のソケットから + * 剥奪しておかないと、旧ソケットが後で閉じたときに + * unregisterWebSocket() が新ソケットも使っている awareness state を + * 消してしまい、他の参加者には再接続したユーザーが消えたように見える。 */ track(ws: WebSocket, clientIds: readonly number[]): void { const current = this.attachmentOf(ws); if (current === null) return; + for (const other of this.disposers.keys()) { + if (other === ws) continue; + + const otherAttachment = this.attachmentOf(other); + if (otherAttachment === null) continue; + + const remaining = otherAttachment.clientIds.filter( + (id) => !clientIds.includes(id), + ); + if (remaining.length === otherAttachment.clientIds.length) continue; + + other.serializeAttachment({ + ...otherAttachment, + clientIds: remaining, + } satisfies SessionAttachment); + } + const merged = new Set([...current.clientIds, ...clientIds]); if (merged.size === current.clientIds.length) return; diff --git a/src/yjs/session/session.test.ts b/src/yjs/session/session.test.ts index 0448be4..34c37ed 100644 --- a/src/yjs/session/session.test.ts +++ b/src/yjs/session/session.test.ts @@ -81,4 +81,43 @@ describe("SessionRegistry", () => { expect(registry.clientIdsOf(ws)).toEqual([]); }); + + it("makes ownership of a reused client id exclusive between sockets", () => { + // Simulates an overlapping reconnect: the same awareness clientID (the + // Yjs Doc survives socket reconnects, so this can genuinely happen) is + // published by an old socket and then by its replacement. Ownership + // must move to the new socket, not be shared by both -- otherwise + // closing the old socket later removes awareness state the new socket + // is still using. + const registry = new SessionRegistry(); + const oldWs = fakeSocket(attachment([])); + const newWs = fakeSocket(attachment([])); + registry.add(oldWs, () => {}); + registry.add(newWs, () => {}); + + registry.track(oldWs, [7]); + expect(registry.clientIdsOf(oldWs)).toEqual([7]); + + registry.track(newWs, [7]); + + expect(registry.clientIdsOf(oldWs)).toEqual([]); + expect(registry.clientIdsOf(newWs)).toEqual([7]); + }); + + it("does not remove a reclaimed client id when its former socket closes", () => { + const registry = new SessionRegistry(); + const oldWs = fakeSocket(attachment([])); + const newWs = fakeSocket(attachment([])); + const oldDispose = vi.fn(); + registry.add(oldWs, oldDispose); + registry.add(newWs, () => {}); + + registry.track(oldWs, [7]); + registry.track(newWs, [7]); + + registry.remove(oldWs); + + expect(oldDispose).toHaveBeenCalledTimes(1); + expect(registry.clientIdsOf(newWs)).toEqual([7]); + }); }); diff --git a/src/yjs/storage/sqlite.test.ts b/src/yjs/storage/sqlite.test.ts index 039aef7..6ae8412 100644 --- a/src/yjs/storage/sqlite.test.ts +++ b/src/yjs/storage/sqlite.test.ts @@ -256,6 +256,84 @@ describe("YSqliteStorage", () => { ); }); + it("rejects a non-positive maxChunkBytes", async () => { + const stub = env.Y_DURABLE_OBJECTS.get(env.Y_DURABLE_OBJECTS.newUniqueId()); + await runInDurableObject(stub, async (_instance, state) => { + // maxChunkBytes <= 0 makes #split()'s offset never advance, looping + // forever on any non-empty update. This must be rejected eagerly at + // construction time instead of hanging the first storeUpdate() call. + expect( + () => new YSqliteStorage(state.storage.sql, { maxChunkBytes: 0 }), + ).toThrow(/maxChunkBytes/); + expect( + () => new YSqliteStorage(state.storage.sql, { maxChunkBytes: -1 }), + ).toThrow(/maxChunkBytes/); + expect( + () => new YSqliteStorage(state.storage.sql, { maxChunkBytes: 1.5 }), + ).toThrow(/maxChunkBytes/); + }); + }); + + it("rejects a non-positive maxRows", async () => { + const stub = env.Y_DURABLE_OBJECTS.get(env.Y_DURABLE_OBJECTS.newUniqueId()); + await runInDurableObject(stub, async (_instance, state) => { + expect( + () => new YSqliteStorage(state.storage.sql, { maxRows: 0 }), + ).toThrow(/maxRows/); + expect( + () => new YSqliteStorage(state.storage.sql, { maxRows: -5 }), + ).toThrow(/maxRows/); + expect( + () => new YSqliteStorage(state.storage.sql, { maxRows: 2.5 }), + ).toThrow(/maxRows/); + }); + }); + + it("does not immediately recompact a rehydrated document whose row count already exceeds maxRows", async () => { + const stub = env.Y_DURABLE_OBJECTS.get(env.Y_DURABLE_OBJECTS.newUniqueId()); + await runInDurableObject(stub, async (_instance, state) => { + const sql = state.storage.sql; + + // Build up a document whose compacted row count (5) exceeds the + // maxRows we'll later reopen it with (1), the same way a real + // compaction under a small maxRows would. + const first = new YSqliteStorage(sql, { maxRows: 1, maxChunkBytes: 50 }); + const doc = new Doc(); + const text = doc.getText("root"); + text.insert(0, "x".repeat(400)); + await first.storeUpdate(encodeStateAsUpdate(doc)); + await first.commit(); + + const rowsAfterCompaction = countRows(sql); + expect(rowsAfterCompaction).toBeGreaterThan(1); + + // Simulate a Durable Object waking up: construct a fresh storage + // instance over the same (already-populated) sql handle, still + // configured with maxRows = 1 -- far below the row count just loaded. + let commitCalls = 0; + const second = new YSqliteStorage(sql, { maxRows: 1, maxChunkBytes: 50 }); + const originalCommit = second.commit.bind(second); + second.commit = async () => { + commitCalls += 1; + await originalCommit(); + }; + + // A single small follow-up update must not trigger a full + // merge/delete/reinsert -- the compaction floor must have been + // initialised from the row count actually loaded, not reset to + // maxRows. + const updates: Uint8Array[] = []; + doc.on("update", (update: Uint8Array) => updates.push(update)); + text.insert(text.length, "y"); + await second.storeUpdate(updates[0]); + + expect(commitCalls).toBe(0); + + const restored = await second.getUpdate(); + expect(textOf(restored!)).toBe(text.toString()); + }); + }); + it("throws when a continuation row has no preceding row to attach to", async () => { const stub = env.Y_DURABLE_OBJECTS.get(env.Y_DURABLE_OBJECTS.newUniqueId()); await runInDurableObject(stub, async (_instance, state) => { diff --git a/src/yjs/storage/sqlite.ts b/src/yjs/storage/sqlite.ts index 5ece881..2f3698a 100644 --- a/src/yjs/storage/sqlite.ts +++ b/src/yjs/storage/sqlite.ts @@ -35,6 +35,21 @@ const DEFAULT_MAX_ROWS = 2000; const DEFAULT_MAX_CHUNK_BYTES = 1024 * 1024; const SQLITE_BLOB_LIMIT = 2 * 1024 * 1024; +/** + * maxChunkBytes / maxRows は両方とも「この行数・バイト数に到達したら + * 前進する」しきい値として使われる。0 や負の値を渡すと #split() のオフ + * セットが一度も進まず無限ループになる（maxChunkBytes）か、あるいは + * 実際に減らせないしきい値で毎回フルコンパクションを引き起こし続ける + * だけになる（maxRows）。どちらも呼び出し側の設定ミスとして早期に + * throw し、Worker がタイムアウトやメモリ枯渇に至る前に検出できる + * ようにする。 + */ +const assertPositiveInteger = (value: number, name: string): void => { + if (!Number.isInteger(value) || value <= 0) { + throw new Error(`${name} must be a positive integer`); + } +}; + const concat = (parts: readonly Uint8Array[]): Uint8Array => { if (parts.length === 1) return parts[0]; @@ -65,16 +80,27 @@ export class YSqliteStorage implements YStorage { constructor(sql: SqlStorage, options?: YSqliteStorageOptions) { this.#maxChunkBytes = options?.maxChunkBytes ?? DEFAULT_MAX_CHUNK_BYTES; + assertPositiveInteger(this.#maxChunkBytes, "maxChunkBytes"); if (this.#maxChunkBytes > SQLITE_BLOB_LIMIT) { // https://developers.cloudflare.com/durable-objects/platform/limits/ throw new Error("maxChunkBytes must not exceed 2MB"); } this.#maxRows = options?.maxRows ?? DEFAULT_MAX_ROWS; - this.#compactionFloor = this.#maxRows; + assertPositiveInteger(this.#maxRows, "maxRows"); this.#sql = sql; migrate(sql); this.#rowCount = sql.exec<{ count: number }>(COUNT_UPDATES).one().count; + // コンストラクタは Durable Object が起きるたびに（hibernation からの + // 復帰も含めて）実行される。#compactionFloor を無条件に #maxRows へ + // リセットすると、すでに #maxRows を超える行数を抱えたドキュメントを + // 読み込んだ直後の最初の storeUpdate() が必ずコンパクションの条件を + // 満たしてしまい、フルマージ・削除・再挿入を毎回の起床のたびに + // 引き起こす — フロアが本来防ぐはずのスラッシングそのものが + // 再現してしまう。commit() が算出する "+1" の余白（直後の 1 回の + // storeUpdate では再トリガーしない）と同じ考え方で、実際に読み込んだ + // 行数から初期化する。 + this.#compactionFloor = Math.max(this.#maxRows, this.#rowCount + 1); } async getUpdate(): Promise { diff --git a/wrangler.toml b/wrangler.toml index 50492b9..c97969a 100644 --- a/wrangler.toml +++ b/wrangler.toml @@ -1,3 +1,16 @@ +# NOTE: This file is the test fixture for a fresh, never-deployed local +# environment only (it is exercised by `pnpm test`, not shipped to anyone). +# Rewriting the "v1" migration entry below to `new_sqlite_classes` is safe +# here purely because nothing has ever been deployed against it. +# +# If you already have a deployed v1 (key-value backend) Worker, do NOT copy +# this pattern: migration history in wrangler is append-only and a +# namespace's storage type is immutable, so rewriting an existing "v1" +# entry cannot convert its storage and will leave the binding pointed at a +# SQLite-only class instance backed by key-value storage, which fails +# `assertSqliteBackend`. APPEND a new migration with a new tag and a +# distinct class name instead — see the "Migrating from v1 (key-value +# backend)" section in README.md. name = "yjs-workers" main = "src/e2e/index.ts" compatibility_date = "2025-04-01" From 4ca5c95c421cf7b4783249ffafdfddddeec3d248 Mon Sep 17 00:00:00 2001 From: napochaan Date: Sat, 15 Aug 2026 00:32:53 +0900 Subject: [PATCH 26/26] fix: clear Awareness's repeating setInterval so Durable Objects can hibernate y-protocols' Awareness installs a repeating setInterval in its constructor (every 3s by default) purely to time out stale remote clients -- the "renew local clock" branch never fires here since WSSharedDoc immediately calls setLocalState(null). Any pending setInterval/setTimeout blocks Durable Object hibernation entirely, so every YDurableObjects instance has been staying awake, and billed for duration, for its whole lifetime, regardless of the ping/pong auto-response. Clear the interval right after Awareness is constructed, accepting the loss of the timeout GC (awareness is already cleaned up on disconnect and rebuilt from nothing on restart). Guard the fix with a runtime type check on the underscore-prefixed, non-public `_checkInterval` field so a future y-protocols internals change fails loudly at construction instead of silently regressing hibernation. Update README, the sqlite-migration changeset, and the design spec's 6-6 section to reflect that the interval -- not the ping/pong response -- was actually blocking hibernation. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015mdNeCzNPZysRHzmkFp9Bj --- .changeset/sqlite-migration.md | 6 + README.md | 29 +++-- .../2026-08-14-sqlite-migration-design.md | 4 +- src/yjs/remote/ws-shared-doc.test.ts | 110 +++++++++++++++++- src/yjs/remote/ws-shared-doc.ts | 61 ++++++++++ 5 files changed, 199 insertions(+), 11 deletions(-) diff --git a/.changeset/sqlite-migration.md b/.changeset/sqlite-migration.md index 466ba3e..912300b 100644 --- a/.changeset/sqlite-migration.md +++ b/.changeset/sqlite-migration.md @@ -39,3 +39,9 @@ key-value backend had forced. - `destroy()` deletes a room's data and closes its connections. - A `"ping"` / `"pong"` auto-response keeps keepalives from waking the Durable Object from hibernation. +- The repeating `setInterval` that `y-protocols`' `Awareness` installs in its + constructor is now cleared immediately. That interval previously kept + every Durable Object instance awake for its entire lifetime, so this is + what actually makes hibernation reachable. As a side effect, the server no + longer expires a stale awareness entry on a timer; each connection's + awareness state is still removed on disconnect. diff --git a/README.md b/README.md index 92adda4..b93fe49 100644 --- a/README.md +++ b/README.md @@ -151,15 +151,26 @@ storage — is the practical ceiling on document size. ### Keepalive and hibernation -v2 registers a `"ping"` / `"pong"` auto-response via -`setWebSocketAutoResponse`. When a client sends `"ping"` as a keepalive, the -Workers runtime answers with `"pong"` directly — the Durable Object is never -woken from hibernation to run `webSocketMessage`, because the auto-response -is matched before that handler would be invoked at all. This interception is -the entire point of the hibernation optimization, and it's the single biggest -lever on duration billing. As a secondary safety net, `webSocketMessage` -ignores non-binary (string) messages, so even if a `"ping"` ever did reach a -woken instance it would be a no-op. +`y-protocols`' `Awareness` class installs a repeating `setInterval` in its +constructor to time out stale remote clients. Any pending `setInterval` or +`setTimeout` prevents a Durable Object from hibernating at all, so previously +every `YDurableObjects` instance stayed awake — and billed for duration — for +its entire lifetime, regardless of any other keepalive handling. v2 now +clears that interval immediately after constructing `Awareness`, which is +what makes hibernation reachable in the first place. One consequence: the +server no longer expires a client's awareness state on a timer. This is +accepted because each connection's awareness ids are already removed on +disconnect, and awareness lives only in memory, so it is rebuilt from nothing +whenever an instance restarts anyway. + +With hibernation actually reachable, v2 also registers a `"ping"` / `"pong"` +auto-response via `setWebSocketAutoResponse`. When a client sends `"ping"` as +a keepalive, the Workers runtime answers with `"pong"` directly — the Durable +Object is never woken from hibernation to run `webSocketMessage`, because the +auto-response is matched before that handler would be invoked at all. As a +secondary safety net, `webSocketMessage` ignores non-binary (string) +messages, so even if a `"ping"` ever did reach a woken instance it would be a +no-op. ## Usage diff --git a/docs/superpowers/specs/2026-08-14-sqlite-migration-design.md b/docs/superpowers/specs/2026-08-14-sqlite-migration-design.md index 9bfae21..450571f 100644 --- a/docs/superpowers/specs/2026-08-14-sqlite-migration-design.md +++ b/docs/superpowers/specs/2026-08-14-sqlite-migration-design.md @@ -311,7 +311,9 @@ async webSocketMessage(ws: WebSocket, message: string | ArrayBuffer) { this.state.setWebSocketAutoResponse(new WebSocketRequestResponsePair("ping", "pong")); ``` -文字列メッセージは `webSocketMessage` の先頭で無視されているため、既存の挙動と衝突しない。README に「クライアントが `"ping"` を定期送信すれば Durable Object を起こさずに keepalive できる」ことを明記する。duration 課金への影響が最も大きい項目である。 +文字列メッセージは `webSocketMessage` の先頭で無視されているため、既存の挙動と衝突しない。README に「クライアントが `"ping"` を定期送信すれば Durable Object を起こさずに keepalive できる」ことを明記する。 + +> **訂正（実装後判明）**: 上記の「duration 課金への影響が最も大きい項目」という記述は誤りだった。`WSSharedDoc` が構築する y-protocols の `Awareness` はコンストラクタで repeating `setInterval`（既定 3 秒間隔）を張っており、これが生きている限り Durable Object は ping/pong 対応の有無に関わらずそもそもハイバネートしない。実際に duration 課金へ最も効くのはこの interval を `Awareness` 構築直後に `clearInterval` することであり、ping/pong 自動応答はハイバネーションに到達できるようになって初めて意味を持つ副次的な最適化である。詳細は `src/yjs/remote/ws-shared-doc.ts` の `clearAwarenessCheckInterval()` とその呼び出し箇所のコメントを参照。 `acceptWebSocket()` のタグは付けない。clientID は接続確立時点では未確定であり、タグは `acceptWebSocket` 呼び出し時に確定している必要があるため相性が悪い。 diff --git a/src/yjs/remote/ws-shared-doc.test.ts b/src/yjs/remote/ws-shared-doc.test.ts index 35cba6c..b74c409 100644 --- a/src/yjs/remote/ws-shared-doc.test.ts +++ b/src/yjs/remote/ws-shared-doc.test.ts @@ -1,6 +1,16 @@ import { createDecoder, readVarUint } from "lib0/decoding"; -import { createEncoder, toUint8Array, writeVarUint } from "lib0/encoding"; +import { + createEncoder, + toUint8Array, + writeVarUint, + writeVarUint8Array, +} from "lib0/encoding"; import { vi, describe, it, expect, beforeEach, afterEach } from "vitest"; +import { + Awareness, + encodeAwarenessUpdate, + removeAwarenessStates, +} from "y-protocols/awareness"; import { readSyncMessage, writeSyncStep1, writeUpdate } from "y-protocols/sync"; import { Doc, encodeStateAsUpdate } from "yjs"; @@ -27,6 +37,18 @@ const createSyncMessage = (update: Uint8Array) => { return toUint8Array(encoder); }; +// Encodes a real awareness protocol message, the way an actual client would. +const createAwarenessMessage = (awareness: Awareness) => { + const encoder = createEncoder(); + writeVarUint(encoder, messageType.awareness); + writeVarUint8Array( + encoder, + encodeAwarenessUpdate(awareness, [awareness.clientID]), + ); + + return toUint8Array(encoder); +}; + // Helper to apply a received message to a new document const applyMessage = (message: Uint8Array) => { const receivedDoc = new Doc(); @@ -175,4 +197,90 @@ describe("WSSharedDoc", () => { expect(() => doc.update(toUint8Array(encoder), {})).toThrow(); }); }); + + describe("Awareness hibernation guard", () => { + // Cloudflare Durable Objects cannot hibernate while any + // setInterval/setTimeout is pending. y-protocols' Awareness installs a + // repeating setInterval in its own constructor, so WSSharedDoc must + // clear it immediately or the Durable Object stays awake -- and billed + // -- for its entire lifetime. These tests verify that mechanism (the + // timer handle gets torn down, and the guard actually fires if + // y-protocols' internals change out from under it). Real hibernation + // itself can't be exercised under @cloudflare/vitest-pool-workers -- + // there is no API to force a Durable Object to hibernate and observe + // that it went dormant -- so the outcome (fewer billed milliseconds) + // is not, and cannot be, asserted here. + it("clears y-protocols' repeating awareness check interval right after construction", () => { + const clearIntervalSpy = vi.spyOn(globalThis, "clearInterval"); + + const localDoc = new WSSharedDoc(); + const handle: unknown = localDoc.awareness._checkInterval; + + // y-protocols/awareness.js sets `_checkInterval` to whatever + // `setInterval` returns; in this runtime that's a number (see + // worker-configuration.d.ts). If it were anything else, WSSharedDoc's + // own guard would already have thrown during construction above. + expect(typeof handle).toBe("number"); + expect(clearIntervalSpy).toHaveBeenCalledWith(handle); + }); + + it("throws loudly if y-protocols' Awareness stops exposing a numeric _checkInterval handle", () => { + // Simulates a future y-protocols release changing what `_checkInterval` + // holds (e.g. renaming/removing it, or moving to a non-numeric handle + // in some other runtime). WSSharedDoc must fail construction instead + // of silently leaving the real interval running. + vi.spyOn(globalThis, "setInterval").mockReturnValue( + undefined as unknown as ReturnType, + ); + + expect(() => new WSSharedDoc()).toThrow(/_checkInterval/); + }); + }); + + describe("Awareness protocol", () => { + it("applies a remote awareness update and broadcasts it to other listeners", () => { + const remoteAwareness = new Awareness(new Doc()); + remoteAwareness.setLocalStateField("user", { name: "a" }); + + const anotherOrigin = {}; + const anotherListener = vi.fn(); + doc.notify(anotherOrigin, anotherListener); + + // Sent with `origin` (already registered via notify() in beforeEach) + // as the sender, matching how WSSharedDoc#update() is driven from a + // real WebSocket message. + doc.update(createAwarenessMessage(remoteAwareness), origin); + + expect(doc.awareness.getStates().get(remoteAwareness.clientID)).toEqual({ + user: { name: "a" }, + }); + expect(anotherListener).toHaveBeenCalledTimes(1); + // The sender itself doesn't get its own update echoed back, same as + // sync updates. + expect(mockListener).not.toHaveBeenCalled(); + + remoteAwareness.destroy(); + }); + + it("removes an awareness state and broadcasts the removal, the same way disconnect cleanup does", () => { + const remoteAwareness = new Awareness(new Doc()); + remoteAwareness.setLocalStateField("user", { name: "a" }); + doc.update(createAwarenessMessage(remoteAwareness), {}); + expect(doc.awareness.getStates().has(remoteAwareness.clientID)).toBe( + true, + ); + + mockListener.mockClear(); + // This is exactly what YDurableObjects#unregisterWebSocket calls when + // a connection closes (see src/yjs/index.ts). + removeAwarenessStates(doc.awareness, [remoteAwareness.clientID], null); + + expect(doc.awareness.getStates().has(remoteAwareness.clientID)).toBe( + false, + ); + expect(mockListener).toHaveBeenCalledTimes(1); + + remoteAwareness.destroy(); + }); + }); }); diff --git a/src/yjs/remote/ws-shared-doc.ts b/src/yjs/remote/ws-shared-doc.ts index c476863..04b90e5 100644 --- a/src/yjs/remote/ws-shared-doc.ts +++ b/src/yjs/remote/ws-shared-doc.ts @@ -25,6 +25,62 @@ interface Notification extends RemoteDoc { notify(origin: object, listener: Listener): Unsubscribe; } +/** + * Narrows `_checkInterval` to the numeric interval-id shape that + * `setInterval` / `clearInterval` use in the Workers runtime (see + * `worker-configuration.d.ts`). y-protocols' own `.d.ts` types the field as + * `any`; funneling it through `unknown` here means we never trust that + * `any` directly -- we check its actual runtime shape before touching it. + */ +function isIntervalHandle(value: unknown): value is number { + return typeof value === "number"; +} + +/** + * y-protocols' `Awareness` constructor installs a repeating `setInterval` + * that fires every `floor(outdatedTimeout / 10)` ms (3s with the library + * default) -- see the installed y-protocols package's `awareness.js` + * (`node_modules/y-protocols/awareness.js`). + * Any pending `setInterval`/`setTimeout` prevents a Durable Object from + * hibernating, so leaving this running keeps every `YDurableObjects` + * instance awake -- and billed for duration -- for its entire lifetime. + * + * The timer does two things: (1) renew the local client's own clock, gated + * on `getLocalState() !== null`. `WSSharedDoc`'s constructor immediately + * calls `awareness.setLocalState(null)`, so that branch never fires here. + * (2) evict remote clients whose state hasn't been refreshed within + * `outdatedTimeout` (30s). Only (2) does anything on the server, and we + * accept losing it: each connection's awareness ids are torn down on + * disconnect (`unregisterWebSocket`), the Durable Objects runtime delivers + * `webSocketClose`/`webSocketError` for abnormal disconnects too, and + * awareness only ever lives in memory, so it's rebuilt from nothing on + * every restart regardless. The GC's remaining value is small; the + * hibernation cost of keeping the timer alive is not. + * + * `_checkInterval` is underscore-prefixed and not part of y-protocols' + * public API -- a future release could rename or drop it. If that happens + * this must fail loudly instead of silently leaving the interval running: + * a silent regression here goes back to costing real money, continuously + * and invisibly, on every running instance. + */ +function clearAwarenessCheckInterval(awareness: Awareness): void { + const handle: unknown = awareness._checkInterval; + + if (!isIntervalHandle(handle)) { + throw new Error( + "y-protocols Awareness no longer exposes its repeating check timer " + + `as a numeric \`_checkInterval\` handle (got ${typeof handle}). ` + + "Without clearing it, that interval keeps every YDurableObjects " + + "instance from ever hibernating, which is billed as continuous " + + "Durable Object duration. Update clearAwarenessCheckInterval() in " + + "ws-shared-doc.ts for the new y-protocols internals before " + + "releasing this.", + ); + } + + clearInterval(handle); +} + export class WSSharedDoc extends Doc implements Notification { /** origin（通常は WebSocket）をキーにした配信先 */ private listeners = new Map(); @@ -32,6 +88,11 @@ export class WSSharedDoc extends Doc implements Notification { constructor(gc = true) { super({ gc }); + + // Awareness のコンストラクタが張る repeating setInterval を即座に破棄する。 + // これが生き続ける限り Durable Object は絶対にハイバネートしない。 + // 詳細は clearAwarenessCheckInterval() のコメントを参照。 + clearAwarenessCheckInterval(this.awareness); this.awareness.setLocalState(null); // カーソルなどの付加情報の更新通知