Skip to content

Commit b6ccb4c

Browse files
authored
docs(technical-design): global database (#11472)
2 parents 05392dd + a787446 commit b6ccb4c

5 files changed

Lines changed: 408 additions & 9 deletions

File tree

‎docs/SUMMARY.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ generator, in this case, **mdbook**. It defines the structure and navigation of
2929
- [User Flows](architecture/user-flows.md)
3030
- [Settings](architecture/settings.md)
3131
- [Feature Flags](architecture/feature-flags.md)
32+
- [Database](architecture/database.md)
3233
- [Legacy Module Integration](architecture/legacy-module-integration.md)
3334
- [Engineering](engineering/README.md)
3435
- [Delivery Planning](engineering/delivery-planning.md)
@@ -39,14 +40,14 @@ generator, in this case, **mdbook**. It defines the structure and navigation of
3940
- [RFCs](engineering/rfcs/README.md)
4041
- [Template](engineering/rfcs/0000-rfc-template.md)
4142
- [Proposed]()
42-
- [0007 - Global Database](engineering/rfcs/0007-global-database.md)
4343
- [Accepted]()
4444
- [0001 - Changelog System Replacement](engineering/rfcs/0001-changelog-system-replacement.md)
4545
- [0002 - Thundermail – Add an Account interim screen](engineering/rfcs/0002-thundermail-add-an-account-interim-screen.md)
4646
- [0003 - Render inline images in plain-text messages](engineering/rfcs/0003-render-inline-images-in-plain-text-messages.md)
4747
- [0004 - Add a Declarative Feature Flag Catalog](engineering/rfcs/0004-feature-flag-new-architecture.md)
4848
- [0005: Enable IMAP IDLE ("Push") for the Inbox by Default on Newly Added IMAP Accounts](engineering/rfcs/0005-imap-idle-push-on-new-account-inboxes.md)
4949
- [0006: Remote Feature Flags](engineering/rfcs/0006-remote-feature-flags.md)
50+
- [0007 - Global Database](engineering/rfcs/0007-global-database.md)
5051
- [Rejected]()
5152
- [Obsolete]()
5253
- [Architecture Decision Records](engineering/adr/README.md)
@@ -68,6 +69,8 @@ generator, in this case, **mdbook**. It defines the structure and navigation of
6869
- [Technical Designs](engineering/technical-designs/README.md)
6970
- [Template](engineering/technical-designs/0000-technical-design-template.md)
7071
- [Proposed]()
72+
- [0003 - Global Database](engineering/technical-designs/0003-global-database.md)
73+
- [Legacy Table Inventory](engineering/technical-designs/0003-global-database/legacy-table-inventory.md)
7174
- [Accepted]()
7275
- [0001 - Changelog System Replacement](engineering/technical-designs/0001-changelog-system-replacement.md)
7376
- [0002: Declarative Feature Flag Catalog](engineering/technical-designs/0002-feature-flag-declarative-catalog.md)

‎docs/architecture/database.md‎

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
# Database Architecture
2+
3+
> **Status:** Proposed direction. This describes the target architecture introduced by Global Database. It is not a
4+
> description of the current legacy implementation.
5+
6+
## Purpose
7+
8+
The database architecture provides one local source of truth for mail data in the application, while keeping
9+
mail domain code independent of storage technology and legacy storage types.
10+
It establishes the boundaries that Global Database implements. The [RFC](../engineering/rfcs/0007-global-database.md)
11+
and [technical design](../engineering/technical-designs/0003-global-database.md) remain the authoritative records
12+
for its scope and implementation.
13+
14+
## Current and target state
15+
16+
Today, legacy mail storage uses one SQLite database and one attachment directory per account.
17+
18+
Global Database introduces one Room 3-backed mail database and one file-backed attachment directory for the
19+
application.
20+
21+
## Boundaries and ownership
22+
23+
```mermaid
24+
flowchart LR
25+
Caller[Mail domain callers] --> Contract[Focused mail repository contracts]
26+
Contract --> Implementation[Repository implementations]
27+
Implementation --> GlobalSource[Global local data sources]
28+
Implementation --> LegacySource[Legacy local data sources during migration]
29+
GlobalSource --> MailDatabase[Global mail database]
30+
GlobalSource --> AttachmentFiles[Global attachment files]
31+
MailDatabase --> CoreDatabase[core:database]
32+
LegacySource --> LegacyDatabase[Per-account legacy databases]
33+
LegacySource --> LegacyFiles[Per-account attachment files]
34+
AppComposition[Application composition] --> Implementation
35+
```
36+
37+
- **Mail repository contracts** are the only storage boundary visible to mail-domain callers. They follow the
38+
[Repository pattern ADR](../engineering/adr/0010-adopt-project-wide-repository-pattern.md): they are focused,
39+
explicitly account-scoped, and do not expose Room, SQL, cursors, files, or legacy `LocalStore` types.
40+
- **Repository implementations and local data sources** belong to the mail feature's internal persistence
41+
implementation. They own mappings between domain data and both global and legacy representations.
42+
- **The global mail database** owns the legacy-compatible mail schema, migration metadata, and derived data.
43+
It is one physical database for the application, not one database per account.
44+
- **The attachment store** remains file-backed for content that legacy already held on disk, which is every body part
45+
above a size threshold. Parts at or below it stay `message_parts` BLOBs, as in legacy. Database records and attachment
46+
files are associated through internal persistence mappings.
47+
- **`core:database`** implements Room 3 as one domain-neutral backend and provides lifecycle, transaction, migration,
48+
and Android/JVM desktop driver support needed by the mail implementation. It composes feature schema contributions
49+
deterministically and owns one coordinated schema and migration history. It must not own mail schema, mail entities
50+
or DAOs, repository contracts, or mail domain types.
51+
- **Application composition** binds repository contracts to one implementation. It is the only place that switches from
52+
legacy to global storage after successful migration.
53+
54+
## Data and migration model
55+
56+
The global schema begins as a compatible representation of the legacy mail-store schema and behavior.
57+
Account-local numeric IDs are preserved through account-qualified legacy keys. Global identifiers are added only where
58+
repository contracts require them.
59+
The [legacy table inventory](../engineering/technical-designs/0003-global-database/legacy-table-inventory.md)
60+
defines the exact source-to-target mapping, including external attachment files.
61+
62+
Migration has one authoritative storage representation at a time:
63+
64+
| Migration phase | Authoritative storage | Normal mail access |
65+
|------------------------------------|-------------------------------------------|-----------------------------------------------------------------------------------|
66+
| Before cutover | Legacy per-account storage | Legacy repository implementation |
67+
| Import and validation | Legacy per-account storage | Migration gate holds UI and background mail work. Global data remains unpublished |
68+
| After cutover state is established | Global mail database and attachment store | Global repository implementation |
69+
70+
No normal caller may read from or write to both representations.
71+
Legacy database and attachment artifacts are deleted only after validation and the durable cutover state is established.
72+
The legacy storage implementation stays in the codebase, unbound and unused, and a later release removes it.
73+
74+
## Constraints
75+
76+
- Global Database does not redesign or normalize the legacy mail schema.
77+
- Android and JVM desktop drivers are in scope. The global attachment-store contract is KMP-safe on both targets.
78+
Legacy import is Android-only.
79+
- On Android, a dedicated migration screen owns the migration UI. It could reuse the existing database-migration
80+
activity or introduce a suitable replacement. It shows non-sensitive progress and a completion or failure state with
81+
retry and local report export. The shared migration gate blocks normal mail access and background sync until cutover
82+
or failure.
83+
- Database implementation types remain internal and follow the API/internal module boundary.
84+
- Migration reports are local and user-exportable, but exclude personally identifiable data and are never uploaded.
85+
86+
## Related documentation
87+
88+
- [RFC 0007: Global Database](../engineering/rfcs/0007-global-database.md)
89+
- [Technical Design 0003: Global Database](../engineering/technical-designs/0003-global-database.md)
90+
- [Legacy Table Inventory](../engineering/technical-designs/0003-global-database/legacy-table-inventory.md)
91+

‎docs/engineering/rfcs/0007-global-database.md‎

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
- Related RFC: [UUIDv7 Identifier Migration](0009-uuidv7-identifier-migration.md)
66
- Repository pattern: [ADR 0010 proposal](https://github.com/thunderbird/thunderbird-android/pull/11452)
77
- Portable data format: [RFC 0008: Portable Profile Data Format](0008-portable-profile-data-format.md)
8-
- Status: **Proposed**
8+
- Status: **Accepted**
99
- Backend decision: **Room 3 selected by the
1010
[completed database spike](https://github.com/thunderbird/thunderbird-android/issues/11195)**
1111

@@ -102,12 +102,14 @@ codebase, unbound and unused, until a later release removes it.
102102

103103
### Migration gate and reporting
104104

105-
The migration surfaces progress, failure feedback, retry, and local report export in a dedicated migration screen. That
106-
screen could reuse the existing Android database-migration activity or introduce a suitable replacement. Startup
107-
routing and background mail work respect the migration gate. No mail work bypasses it.
105+
The migration surfaces progress, failure feedback, retry, and local report export in a dedicated migration screen. On
106+
Android, a user-initiated data synchronization foreground service continues the migration when the app leaves the
107+
foreground and provides progress and completion through a persistent notification. The in-app screen and notification
108+
show the same migration state.
108109

109-
Progress information, failure reports, and logs exclude personally identifiable data. They stay on the device and are
110-
never uploaded.
110+
Startup routing and background mail work respect the migration gate. No mail work bypasses it. Progress information,
111+
failure reports, notifications, and logs exclude personally identifiable data. They stay on the device and are never
112+
uploaded.
111113

112114
### Platform support
113115

@@ -153,6 +155,8 @@ Validation before cutover and the archive from step 1 cover that risk instead.
153155
- Migration requires time and storage, especially for downloaded attachments.
154156
- The user or operating system can terminate the app during migration, leaving an incomplete global database that must
155157
never become authoritative. The migration requires interruption-safe state tracking and restart recovery.
158+
- Android can limit foreground service execution, and a user can explicitly stop the app. Long migrations therefore
159+
cannot rely on the service alone and must remain safe to retry after interruption.
156160
- Rebuilding the search index during migration adds time proportional to locally stored mail.
157161
- A device without enough free space cannot migrate until space is freed.
158162
- After cutover, locally stored mail exists only in the global database and in any archive the user created.
@@ -163,8 +167,6 @@ Validation before cutover and the archive from step 1 cover that risk instead.
163167

164168
## Open Questions
165169

166-
- Should migration require a pre-flight free-space check with a stated minimum headroom, and should a failed check block
167-
migration or only warn?
168170
- Does declining the POP3 archive require a durable record of the user's acknowledgement?
169171

170172
The technical design owns implementation questions, including driver configuration, the identifier mapping inventory,

0 commit comments

Comments
 (0)