|
| 1 | +# RFC 0007: Global Database |
| 2 | + |
| 3 | +- Issue: [#11104](https://github.com/thunderbird/thunderbird-android/issues/11104) |
| 4 | +- Technical design: [Global Database](../technical-designs/0003-global-database.md) |
| 5 | +- Related RFC: [UUIDv7 Identifier Migration](0009-uuidv7-identifier-migration.md) |
| 6 | +- Repository pattern: [ADR 0010 proposal](https://github.com/thunderbird/thunderbird-android/pull/11452) |
| 7 | +- Portable data format: [RFC 0008: Portable Profile Data Format](0008-portable-profile-data-format.md) |
| 8 | +- Status: **Proposed** |
| 9 | +- Backend decision: **Room 3 selected by the |
| 10 | + [completed database spike](https://github.com/thunderbird/thunderbird-android/issues/11195)** |
| 11 | + |
| 12 | +## Summary |
| 13 | + |
| 14 | +Global Database replaces the legacy per-account mail stores with one Room 3-backed mail database for the application. It |
| 15 | +preserves existing mail behavior and moves production mail access to the global database only after a complete migration |
| 16 | +succeeds. |
| 17 | + |
| 18 | +The migration imports durable mail data and downloaded attachments, validates the result, establishes the durable |
| 19 | +cutover state, and removes legacy storage. If migration fails before cutover, legacy storage remains authoritative. |
| 20 | + |
| 21 | +## Motivation |
| 22 | + |
| 23 | +Legacy mail storage uses one database and one attachment directory per account. This makes unified mail behavior more |
| 24 | +complex and exposes legacy storage types to mail code. |
| 25 | + |
| 26 | +One global database provides a single local source of truth while keeping callers independent of storage |
| 27 | +implementation. The migration must preserve durable mail data without exposing a partial or mixed store. |
| 28 | + |
| 29 | +## Proposal |
| 30 | + |
| 31 | +### Storage boundaries |
| 32 | + |
| 33 | +- `core:database` will provide the domain-neutral Room 3 backend. |
| 34 | +- `core:database` owns one coordinated schema and migration history, and composes feature schema contributions |
| 35 | + deterministically. |
| 36 | +- The internal mail database module owns mail schema and storage implementations. |
| 37 | +- Focused repository contracts are the boundary between mail code and storage. They follow the proposed ADR 0010 and |
| 38 | + do not expose persistence or legacy storage types. |
| 39 | +- Application composition selects the active repository implementation. |
| 40 | + |
| 41 | +### Global mail representation |
| 42 | + |
| 43 | +The global database keeps the legacy mail-store behavior and schema shape. It makes account scope explicit and adds |
| 44 | +global identifiers only where repository contracts require them. Refining the schema is deferred to a future iteration. |
| 45 | + |
| 46 | +Every durable legacy mail record, queued operation, and downloaded attachment is included. Derived data, such as search |
| 47 | +indexes, is rebuilt as part of the migration. The technical design owns the table-by-table migration inventory that |
| 48 | +records which legacy tables are copied, rebuilt, or excluded. |
| 49 | + |
| 50 | +### Global identifiers |
| 51 | + |
| 52 | +In this RFC, "global" means unique across the mail records of all accounts in one application profile. It does not mean |
| 53 | +an identifier shared across devices or provided by a mail server. In particular, IMAP UIDs, server folder IDs, and RFC |
| 54 | +5322 `Message-ID` headers retain their protocol-specific scopes and are not global database identifiers. |
| 55 | + |
| 56 | +The global store uses these identifiers at repository boundaries: |
| 57 | + |
| 58 | +- `AccountId` is the existing UUID-backed account identifier. The global mail database uses it to retain account scope |
| 59 | + and does not replace or regenerate it during cutover. |
| 60 | +- `FolderId` identifies one local folder record across all accounts, replacing the legacy account-local folder number. |
| 61 | +- `MessageId` identifies one local message record across all accounts. Copies of a message in different folders are |
| 62 | + separate local records and have separate identifiers. |
| 63 | +- `ThreadId` identifies a conversation within one account. A conversation can contain message records from multiple |
| 64 | + folders, such as Inbox and Sent. |
| 65 | +- `AttachmentId` is an opaque reference used to access an attachment. It is not a raw message-part database key. |
| 66 | + |
| 67 | +These domain identifiers are stable for the lifetime of their local records. Repository contracts do not expose legacy |
| 68 | +numeric IDs or persistence keys. The database may still use internal surrogate keys where required. The technical design |
| 69 | +defines identifier ownership and migration mappings. RFC 0009 separately defines UUID representation and generation |
| 70 | +policy. |
| 71 | + |
| 72 | +### Migration and cutover |
| 73 | + |
| 74 | +The migration has one authoritative mail store at a time: |
| 75 | + |
| 76 | +1. Calculate and display the required storage headroom. Do not start the migration when available space is insufficient. |
| 77 | +2. For every POP3 account, create and verify the portable mail archive defined by RFC 0008. A user may decline the |
| 78 | + archive after an explicit warning and continue at their own risk. For IMAP accounts, offer the same archive as an |
| 79 | + optional action. |
| 80 | +3. Read legacy storage without modifying it. Import durable data into an unpublished global database. |
| 81 | +4. Rebuild derived data, then validate the imported database, queued operations, attachments, and representative search |
| 82 | + queries. |
| 83 | +5. Set the durable cutover state and switch repository bindings to the global implementation. |
| 84 | +6. Remove all legacy database and attachment artifacts, including data left behind by previously deleted accounts. |
| 85 | + |
| 86 | +Before cutover, legacy storage remains authoritative. After cutover, global storage remains authoritative. There are no |
| 87 | +dual reads, dual writes, or fallback to legacy storage. |
| 88 | + |
| 89 | +The free-space check is a blocking pre-flight requirement, not a warning. Migration temporarily needs enough storage for |
| 90 | +the legacy and global stores to coexist, along with working space for database transactions and rebuilt data. If the |
| 91 | +check fails, the app tells the user how much additional space is required and asks them to free storage before retrying. |
| 92 | +The technical design defines the headroom calculation and runtime handling if available space changes during migration. |
| 93 | + |
| 94 | +Failures before cutover leave legacy storage authoritative and report an actionable error locally. The user can retry |
| 95 | +the migration without losing access to the legacy data. An attachment that fails to copy or validate is recorded in the |
| 96 | +migration result and fails the migration. Insufficient free space is the expected cause. |
| 97 | + |
| 98 | +After successful cutover, cleanup removes all legacy mail data, including artifacts left behind by previously deleted |
| 99 | +accounts. A cleanup failure does not block normal use of the global store or make legacy storage authoritative again. |
| 100 | +The technical design defines cleanup state, retries, and user feedback. The legacy storage implementation stays in the |
| 101 | +codebase, unbound and unused, until a later release removes it. |
| 102 | + |
| 103 | +### Migration gate and reporting |
| 104 | + |
| 105 | +The migration surfaces progress, failure feedback, retry, and local report export in a dedicated migration screen. That |
| 106 | +screen could reuse the existing Android database-migration activity or introduce a suitable replacement. Startup |
| 107 | +routing and background mail work respect the migration gate. No mail work bypasses it. |
| 108 | + |
| 109 | +Progress information, failure reports, and logs exclude personally identifiable data. They stay on the device and are |
| 110 | +never uploaded. |
| 111 | + |
| 112 | +### Platform support |
| 113 | + |
| 114 | +The global mail schema, repository contracts, and attachment access are KMP-ready for Android and JVM desktop. Legacy |
| 115 | +import is Android-only. |
| 116 | + |
| 117 | +### Not included |
| 118 | + |
| 119 | +This RFC does not: |
| 120 | + |
| 121 | +- redesign or normalize the legacy mail schema or queue behavior |
| 122 | +- move account settings or profile runtime storage into the global database |
| 123 | +- introduce profiles or synchronization |
| 124 | +- define portable-data or backup formats. RFC 0008 owns that work. |
| 125 | +- select or migrate UUID identifier formats. RFC 0009 owns that work |
| 126 | +- add remote synchronization, telemetry, or remote migration reporting |
| 127 | +- add iOS or Web persistence support |
| 128 | + |
| 129 | +## Alternatives Considered |
| 130 | + |
| 131 | +### Keep the legacy stores |
| 132 | + |
| 133 | +This retains account-specific storage and prevents establishing a single local source of truth for mail data. |
| 134 | + |
| 135 | +### Use SQLDelight |
| 136 | + |
| 137 | +The completed database spike compared Room 3 and SQLDelight against the same representative mail, queue, and full-text |
| 138 | +search slice. Both candidates met the architectural baseline, but Room 3 delivered stronger response-time and |
| 139 | +concurrent-write results. |
| 140 | + |
| 141 | +### Redesign the schema during migration |
| 142 | + |
| 143 | +Combining a schema redesign with migration would increase risk and make behavior parity harder to verify. |
| 144 | + |
| 145 | +### Retain legacy storage after cutover |
| 146 | + |
| 147 | +Keeping the legacy artifacts would preserve a local recovery path for a defect found after cutover. It would also store |
| 148 | +mail data twice for as long as it is retained, and no code path would read it, because cutover permits no fallback. |
| 149 | +Validation before cutover and the archive from step 1 cover that risk instead. |
| 150 | + |
| 151 | +## Risks & Drawbacks |
| 152 | + |
| 153 | +- Migration requires time and storage, especially for downloaded attachments. |
| 154 | +- The user or operating system can terminate the app during migration, leaving an incomplete global database that must |
| 155 | + never become authoritative. The migration requires interruption-safe state tracking and restart recovery. |
| 156 | +- Rebuilding the search index during migration adds time proportional to locally stored mail. |
| 157 | +- A device without enough free space cannot migrate until space is freed. |
| 158 | +- After cutover, locally stored mail exists only in the global database and in any archive the user created. |
| 159 | +- The legacy storage implementation stays in the codebase until a later release removes it. |
| 160 | +- A POP3 archive adds a user-visible prerequisite before migration. |
| 161 | +- Replacing legacy storage access is broad work. |
| 162 | +- The legacy schema retains existing constraints until a later change addresses them. |
| 163 | + |
| 164 | +## Open Questions |
| 165 | + |
| 166 | +- Should migration require a pre-flight free-space check with a stated minimum headroom, and should a failed check block |
| 167 | + migration or only warn? |
| 168 | +- Does declining the POP3 archive require a durable record of the user's acknowledgement? |
| 169 | + |
| 170 | +The technical design owns implementation questions, including driver configuration, the identifier mapping inventory, |
| 171 | +the migration screen, source-schema support, and the cleanup retry strategy. |
| 172 | + |
| 173 | +## Outcome |
| 174 | + |
| 175 | +Filled in when the RFC is accepted, rejected, or obsolete. |
| 176 | + |
| 177 | +Summarize the final decision and link follow-up work. |
0 commit comments