Rust implementation of Pimdir standard
This project is composed of 3 feature-gated layers:
- Low-level I/O-free core: no_std-compatible schema, statements and model-to-column encodings, reusable by any implementation
- Mid-level std client:
PimdirStore,PimdirProducerandPimdirReader, the owner, producer and reader handles running the statements against SQLite and the blob files, servicing the io-replica storage seam - High-level CLI: the
pimdirbinary, the operator front-end over a store (requires theclifeature)
- Portable store: a single SQLite index plus a content-addressed blob directory, readable by any conformant pimdir implementation.
- Deduplicated bodies: each body is stored once by content hash, so a message filed in two mailboxes costs one copy.
- Offline-first: keeps the shared item and a per-source base, the raw material a sync engine reconciles against.
- Short public ids: one small, store-global id per message, shared across every collection and never reused.
- Crash-safe writes: one transaction per batch, bodies durable before the rows that reference them, and blobs garbage collected inside it.
- Retention: a removal retires an item instead of destroying it, hidden from every read and from the sync, until an explicit purge reclaims it.
- Action queue: processes that do not own the store request mutations by appending actions the owner applies exactly once, with parked failures queryable and collection generations carrying the handle-space epoch to readers.
- Three roles, three handles: one owner that writes, any number of producers that enqueue, and any number of readers that take no lock and carry no write at all, so a frontend cannot drain or sweep a store it only reads.
- Read-your-writes: a reader folds the queue over the committed items on request, so a staged flag, removal, move or copy shows before the owner applies it, while a queued creation is reported apart, having no public id yet.
- Operator CLI: inspect a store while a sync is running, read the trash, restore or purge an item, prune the queue, check consistency, collect what nothing references and dump the whole store (requires the
clifeature). - no_std core: the schema, statements and encodings need no allocator beyond
allocand pull SQLite in only behind theclientfeature.
Tip
io-pimdir is written in Rust and uses cargo features to gate each layer. The default feature set is declared in Cargo.toml or on docs.rs.
io-pimdir implements the pimdir on-disk store specification: a SQLite database plus a content-addressed blob directory, with a canonical schema and forward-only migrations. The spec is the cross-implementation contract, so a store written here is readable by any other conformant implementation (a native Android SQLite store, for example). The sync model it services (a shared item, a per-source base, detail levels, conflicts) lives in io-replica.
The CLI binary pimdir has not been officially released yet. Install it from crates.io with cargo:
cargo install io-pimdir --locked --features cliTo use io-pimdir as a library, add it to your Cargo.toml: the cli feature is not part of the defaults, so a library consumer never compiles the binary or its terminal dependencies.
The pimdir binary is to a store what sqlite3 is to a database: an operator and debugging tool, not an end-user client. It is kind-agnostic and never interprets item content, so it prints ids, flags, levels and the raw meta, and exports raw bytes; rendering a message or a contact belongs to himalaya and cardamum. Reads open the store read-only, so inspecting a store mid-sync is always safe. A few real-world invocations:
pimdir -s ~/mail store info
pimdir -s ~/mail collection list
pimdir -s ~/mail item list INBOX --retained
pimdir -s ~/mail item restore 42
pimdir -s ~/mail item purge --older-than 90d
pimdir -s ~/mail queue list --parked
pimdir -s ~/mail checkRun pimdir --help for the full command tree and flags, and add --json to any command for machine-readable output. The CLI contract (the roles it opens a store with, its verb surface, its confirmation rules) lives in cairn/spec/cli.md. See documentation at docs.rs.
See complete examples at ./tests.
This project is licensed under either of:
- Chat on Matrix
- News on Mastodon or RSS
- Mail at pimalaya.org@posteo.net
Special thanks to the NLnet foundation and the European Commission that have been financially supporting the project for years:
- 2022 → 2023: NGI Assure
- 2023 → 2024: NGI Zero Entrust
- 2024 → 2026: NGI Zero Core
- 2026 → 2027: NGI Zero Commons Fund
This program is part of Pimalaya, free software funded entirely by grants and donations. If you find it useful, consider sponsoring its development:
