All notable changes to this project are documented in this file. The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
0.4.0 - 2026-08-31
-
Added
bindings.shared_object, what a source last agreed with the hub on.A binding recorded one base, the state last agreed with its own remote, and the engine measured two different questions against it: only a sync moves that base, so a body the source itself folded in and has not pushed yet leaves the same gap another source folding in leaves. A second offline edit was therefore filed as a cross-source conflict in a store with one source and no second source anywhere, and so was the edit resolving a conflict, whose merged body then never became the item's. io-replica gave the second axis a base of its own; this is where it survives a restart, which is the only place it matters, the absorb that files a conflict and the edit that settles it being different runs.
Round-tripped through
ReplicaSourceBindingand meaningful on every binding, conflicted or not, unlike the conflict pair beside it: gated on the flag it would be erased at the moment a resolving edit needs it. It names an object and pins none, taking noREFERENCES, no index and no refcount, being only ever compared for equality and never read as bytes. Folded into schema version 1 under the draft allowance (spec §6) and backfilled from the item's own body, so an upgraded store opens in agreement rather than conflicting once for every item whose push is pending.item showprints it beside the base it is not. -
Added
bindings.conflict_object, the diverging body an unresolved conflict is waiting on.A binding recorded the remote revision it diverged at and nothing else, so whoever resolved the conflict had to fetch the body that revision named, which meant credentials in the resolving tool. The column holds that body, round-tripped through
ReplicaSourceBindingand meaningful only whileconflictedis set, on the same terms as the revision beside it: base, local and remote are all readable from the store alone.It is a reference like any other, counted in the refcount and therefore kept out of the collector for as long as the binding stays conflicted, which is the interval between the run that found the divergence and the day a person sits down to it; resolving is an ordinary edit and releases the pin. The item-level
items.conflictedanditems.conflict_object, the cross-source axis, is untouched. Folded into schema version 1 under the draft allowance (spec §6), so a store written before the column is reconciled on open rather than refused.item showprints it beside the revision. -
Added
PimdirReader::list_conflicts, what is waiting for a decision.The conflict flag was written and read back with its row and never filtered on, so counting the outstanding conflicts of a store meant paging every collection, on every run. One statement now answers it across an account's collections, naming each binding by its collection, link id, source and handle and carrying the three hashes the divergence is between, and it rides a partial index that holds only the outstanding rows and is empty at rest.
-
Added
PimdirReader, the read role (spec §8).The format names three roles and the crate shipped two handles:
open_read_onlyreturned aPimdirStore, the type that also drains the queue, sweeps the objects and purges the trash, so a consumer that only reads held a handle that could destroy the store and "it never calls those" was the only thing keeping it from doing so. The reads move to a handle that takes no lock and carries no write at all;PimdirStoredereferences to one, so the projection stays a single implementation whichever role reads it. -
Added the pending-action overlay (spec §15.4).
A reader built with
with_pendingfolds the queue's pending actions over the committed items, so a producer sees what it staged before the owner applies it:set-flagsandupdaterestate an item,removeandmovetake it out of a collection, andmoveandcopybring it into the target. All of them address an existing item, whoseseqfollows its link id store-wide (spec §9.1), so nothing invents an identifier. A queued create has nosequntil the owner applies it and is reported apart, bypending_createsandcount_pending_creates. Parked rows never overlay, their error saying they will not be applied without an operator.The choice is made when the reader is built, never per call, so one handle cannot answer two ways about one collection. A page keeps its meaning: the fold reads past the limit by the number of pending removals and cuts back afterwards, so a page comes back short only where the collection ends and a scan written to page until a short page does not end early.
-
Added
PimdirStore::cancel_action, cancelling one queue row as the owner while holding that role only for the length of the call.Cancelling is an owner write (spec §15.5) and the only retraction a queued create has, the kinds addressing an existing item being retracted by their inverse instead. A consumer that is otherwise a reader and a producer needed the whole owner handle to reach it, which is the handle it must not hold.
-
Added
PimdirBlobs::path, where a body under a hash lives.The sharded layout is normative (spec §5) and the format invites a consumer to stream a body straight to that path and index it with a byteless
StoreObjectafterwards (spec §14), which until now meant deriving the sharding a second time. -
Added
conventions, the per-kind derivations (spec Annex A).One
derive(kind, body)returning thelink_id, themetasummary and thesort_keyformessage/rfc822,text/vcardandtext/calendar, in the I/O-free core so a consumer running its own SQLite driver reaches it. Annex A is informative, so nothing reported the divergence three consumers writing it separately had already produced: one wrote a whole calendar summary and a resolved key where another wrote{"v":1,"etag":…}and an empty key for every item. The fallback ids are fixed here too (alt:for a message with noMessage-ID,hash:for a card or a resource with noUID), because two writers disagreeing about one item's id link it twice and store one body twice. Checked against the format's own vectors, fixtures read as bytes and structures compared rather than JSON text. -
Added
pimdir gc, the collector.It drops the object rows nothing references, unlinks their bodies and the orphan blob files a crash left, and reports what it freed. It takes the owner role, so it never runs beside a sync, and reports a producer mid-append rather than waiting on one. A store that wants its unreferenced objects bounded schedules the verb.
-
Added the single-owner rule, enforced (spec §8).
Every owning handle takes an exclusive advisory lock on the store directory and holds it for its lifetime; a second owner gets
PimdirError::Ownedimmediately, naming the store, rather than the 30-secondbusy_timeoutwait that used to end in a stall with no signal. The rule is about processes, so a two-sided sync or a multi-account owner opening several handles still holds one lock. Readers take none. Producers take a shared one, so several append at once while none of them keeps the owner out, and a body is never between the blob tree and the queue row that pins it while a collector runs. The kernel releases the lock with the process, so a crash leaves nothing to recover. -
Added
PimdirReader::item_bindingsand the bindingspimdir item showprints.A binding is where a source's own view of an item lives: the handle it is addressed by, the base the last sync agreed on, and the marker saying it diverged. None of it was readable, so the question a duplicated identity raises, which resource does each copy come from, could only be answered by going back to the server or opening the database by hand.
item shownow prints one block per source under each placement,conflicted at revisionappearing only when it applies.item listis untouched: its rows are one query for a whole page, and a binding lookup per row is not what a listing should cost. -
Added the minted-key count
pimdir checkreports per collection, the second copies of an identity a source hands over twice (spec §9).Not a defect and nothing to repair, so it is counted apart from the problems: what it is worth saying is that a collection whose count climbs every sync is a source renaming the same duplicate on every run.
-
BREAKING Dropped the source parameter from
PimdirStore::openandopen_read_only, and moved the source-bound operations toPimdirSourceStore, yielded byfor_source.PimdirSourceStorecarries the storage seam, the rekeyed write and the queue drain, and dereferences to the store for everything else. Purge, queue cancellation and every read consulted no source but had to be handed one anyway, so the CLI invented"pimdir"for its readers and scannedbindingsfor a name it then discarded. Both are gone, and a store an operator reads and sweeps now records no source at all. -
BREAKING Deduplicated the crate against itself, with no behaviour attached.
Fourteen reads that each wrote out prepare-map-push-return share one
rowshelper; the statements and thesql::ALLindex are declared by one macro, so the two tests that re-read this crate's own source to keep them in step are gone;PimdirRetainedItemis folded intoPimdirItemas an optionalretention, read by one row mapper instead of two; the operator tool's second read-only connection (PimdirDb) is folded onto the store as a diagnostics block;parkbecomes a call tofail_action. Breaking beyond the fold:PimdirPlacementcarries the same typed columns as every other read (ReplicaLinkId,ReplicaFlags,ReplicaLevel), andPimdirError::Versionsplits intoVersion(a schema this crate does not service) andUncreated(no schema yet, which only an owner creates). -
BREAKING Bumped the
io-replicadependency to 0.5, taking the engine's new API with it.loadcarries a scope,DropPlacementcarries a reason, andReplicaCollectionis gone. The git patch this crate carried until the engine release that mints a key for a duplicated identity is dropped. -
BREAKING Made a purge report the rows it retired, not the bytes it reclaimed.
It releases the references a retained item held; the bodies are freed by the collector, which is what can report them.
PimdirPurgeReportloses itsbytesfield anditem purgeits byte count. -
Moved the blob write out of the database's writer lock.
Bodies land in the blob store before the transaction that indexes them opens, which spec §14 asks for in as many words: inside it, the same write held SQLite's single writer lock across a file write, two
fsyncs and a rename, serialising every other writer behind an I/O path that touches no database page. The queue drain, which builds its ops inside the transaction that claims its row, keeps writing inside it; the blob write is idempotent, so that costs one existence check. -
Stopped the collector from holding every hash in memory.
It read the whole
objectsindex into a set to diff the blob tree against, which is hundreds of thousands of names at the scale spec §1 promises, to answer a question that is always about one file. It asks per file on the primary key now (OBJECT_EXISTS, new in the format's statements). -
Made a purge release the pins its own delete reported.
PURGE_ITEMandPURGE_RETAINED_BEFOREreturn each removed row'sobject_hashandconflict_object, so the pins are settled from the statement that took them rather than from a read that visits every swept row a second time.RETAINED_ITEM_BY_SEQandRETAINED_BEFOREare retired. -
Checked the format's own SQL, and its one MUST vector.
The fidelity suite prepares every canonical statement against the canonical schema rather than only checking that each is inlined here by name: this crate holds the only toolchain that ever loads those files.
tests/objects.rschecks object naming againstvectors/objects.json(spec §16), both algorithms, whole and streamed, shard paths included. Both suites, andconventions, skip silently without the sibling spec checkout, so the new CI in this repository and in pimdir asserts they ran. -
Made
checkdiagnose and--fixrepair, neither reclaiming.--fixused to delete orphan blob files behind a--gracewindow and a confirmation prompt, while every write swept with neither. Both flags are gone: it now recomputes the drifted refcounts from the pointers that justify them and clears the bindings whose item is gone, which destroys nothing and needs no guard. Orphan files are reported, andpimdir gctakes them. -
Made
item restorequeue rather than fail when the store is owned.It appends its action first and takes the owner role only to apply it, so a restore issued while a sync runs reports
queuedand is drained by the owner that holds the store, which is what the queue is for. Purge andqueue cancelstill need the role and still say so. -
Scoped a write to the rows its batch names.
Folding a batch into a collection loaded, cloned and diffed the whole collection, so the cost of one flag on one message was the size of the mailbox: measured 3.5 ms at a thousand items, 13 ms at four thousand, 59 ms at sixteen thousand, cleanly linear, against a promise of hundreds of thousands. The read is now scoped to the link ids the batch carries, with each dropped handle resolved through the new
bindings_by_handleindex, and the same measurement is flat at 150 to 175 µs across that range. -
Keyed the residual rather than listing it.
A first sync probes a whole collection before linking any of it, so the list grew to the collection size while every insertion, drop and lookup searched it linearly.
-
Made the drain answer its point questions with point reads.
The
Addcollision check, the handle lookup and the mutation it drives each loaded the whole collection, once per drained action. -
Made
release_pinsone statement rather than one per hash.A purge of fifty thousand retained items was a hundred thousand point updates inside one transaction.
-
Made the object sweep test
refcount <= 0, so a count a double release drove negative is collected rather than leaking for ever with nothing reporting it. -
Added the indexes
objects_garbage(partial, so the sweep stops scanning the whole table on every write transaction),items_by_seq_global(the store-global public id the format promises had no store-global index),bindings_by_handle,items_by_conflict_objectandqueue_by_object.
- Deprecated
open_read_onlyin favour ofPimdirReader.
-
Fixed a queued action the draining source could not place being parked, destroying it for the source that could.
Staging an existing item's action resolves that item's binding for the draining source, and no binding was answered with a park:
errorset, filtered out by every later drain, cleared by nothing in the crate. The first source to reach an action it could not place therefore made it unappliable for the source that held the item. On a consumer draining once per source, in name order, over one store, that is every action a frontend queues against a collection whose source does not sort first. The spec already said an action the owner cannot apply here is skipped and left pending, never parked, so that is now what happens: the row is left exactly as found, itsattemptsuntouched, and counted asskipped. Rows already parked stay parked,queue cancelremaining the only exit. -
Fixed a trash page sorting the whole trash to return fifty rows.
list_retained_pagemoved onto the publicseq(spec §14.1) anditems_retaineddid not move with it, so the read could not ride its own index. An existing store is repaired on open: an index whose columns moved is now dropped and recreated, which the ensure batch could not do,CREATE INDEX IF NOT EXISTSkeying on the name and leaving the old shape in place silently. -
Fixed a process refusing itself the store it had just released.
The owner lock is the process's and shared across handles, but a strong count reaches zero before the file description it named is closed, so a handle taken in between opened a second description and
flockrefused it against this process's own:PimdirError::Ownednaming a store nobody else held, on no schedule, which nothing above can act on. The registry owns the description now and closes it as the last handle goes, inside the same critical section the next acquisition takes. -
Fixed a body stored without a placement being destroyed at the end of the batch.
Every write swept the object rows at refcount zero and unlinked their blobs, so a consumer that streamed bodies into the blob tree and attached them in a later batch, which spec §14 invites and which
STORE_OBJECTinserting at refcount zero exists for, lost them silently, bytes included, before the batch that would have referenced them ran. No write collects any more;pimdir gcdoes. -
Fixed a body lookup crossing accounts.
lookup_objectsresolved a link id against every collection in the store, so two accounts holding the same vCardUID, which spec §9.2 names as a thing unrelated servers do, handed each other's bodies across: the receiving sync then believed the item was hydrated and never fetched the real one. It is scoped to the caller's own account now, which is the axis a link id is trustworthy on; across collections it still answers, which is what the read exists for. -
Fixed a base of no revision, no body and unread markers round-tripping as no base at all, so an agreed placement read as never-agreed and the sync re-derived the same push on every run.
bindings.base_presentrecords the fact its three value columns cannot express; those columns stay a witness for rows written before it. -
Fixed an identity a source held twice being stored once, the write that lost the other silent.
A binding pins one handle, so a source holding one link id twice (a double delivery, a retried append, a restore, a migration, two DAV resources sharing a
UID) had nowhere to put the second copy: the write repointed the binding at it, and no layer above could afterwards tell the source held the identity twice, while deleting the bound copy propagated a delete that removed the only copy on a source nobody touched. The second copy is an item of its own now, under the key the engine mints for it (spec §9), with its ownseq, its own binding and its own body; a write that still resolves a stored binding to another handle is refused withPimdirError::Rebindrather than applied, and nothing is recorded in its place. The one licensed rebind stays licensed: a handle-space rebuild supersedes the handle it replaces, per handle, so a renumbered collection carries over and a rebuild holding a genuine second copy is still refused for that one. -
Fixed a write carrying a new sort key silently discarding it.
The diff that decides whether a row needs an
UPDATEcompared every column the statement writes exceptsort_key, so a key that changed and nothing else reported the row unchanged and no statement was issued. A key is derived rather than given, and a connector fixing its derivation, a tzdb update moving a zoned start, or the second source of a two-source sync all restate one; the item stayed where the first derivation put it, for good. The suite missed it because it only covered the other half of the invariant, that a write carrying no key must leave the stored one alone. -
Fixed a descending page hiding every item sorting above its first cursor.
"No cursor" was expressed as a key no real one could outrank, but a sort key is arbitrary text a writer derives, so no value is reserved and the sentinel was outranked by two of the same character. Such an item was invisible to every descending page, permanently, while the count still reported it. The statement now says what it means, a
NULLcursor, and keeps the same keyset comparison and the same index. -
Fixed two owners draining one collection applying every action twice.
The pending rows are read outside any transaction, and the row was deleted at the end of the applying transaction, so a second owner holding the same list re-applied all of it;
addandcopyare not idempotent, and the operator CLI opens a second owner handle routinely. The delete is now the first statement of the transaction (CLAIM_ACTION, aDELETE ... RETURNING id) and a claim that deletes nothing skips the row: exactly-once is a property of the statement rather than a convention about who runs the drain. -
Fixed a blob rename never being made durable.
The body was written,
fsynced and renamed, and the directory entry that carries the name was not synced, while the SQLite commit is. A power loss could leave a committed row pointing at a body that never arrived, which is the one asymmetry the write order exists to prevent. -
Fixed a flag set the store could not decode reading as a known-empty one.
That is an authoritative "this item carries no markers", which the merge took as one side's opinion: it cleared every marker the other side reported and persisted the result, turning a read failure into permanent loss. It now decodes as unknown, which holds no opinion.
-
Fixed
created_atholding epoch milliseconds where the column is declared to hold an RFC 3339 timestamp, and the empty string when the clock predated the epoch.It is written by SQLite itself now, in the form the retirement clock already uses, which also keeps the crate free of a clock.
0.3.0 - 2026-08-24
-
The store owns the content hash its objects are named by.
PimdirHashAlgoimplements both algorithms the format admits (blake3, recommended, andsha256-128) with the encoding spec §5 fixes, lowercase base32 (RFC 4648, no padding), and a store, a producer and the algorithm itself hand it out whole (hash) or incremental (hasher, for a body streamed into the blob store).Until now this crate stamped
store_meta.hash_algowithblake3and hashed nothing, while every Rust consumer carried its own 128-bit FNV-1a rendered as hex and the Android app computedsha256-128as base32. The recorded algorithm was therefore false, the digest was not cryptographic (spec §2), the encoding was not the one a blob path is specified to use, and the two implementations of one store named the same body differently: no dedup, no blob found, and nothing erroring while it happened.open_with_hashdeclares the algorithm a store is created with,openadopts whatever an existing store records, and an open declaring a different one is refused withPimdirError::HashAlgo.Breaking:
PimdirBlobs::opentakes the algorithm too, since the blob directory is what names files by it;PimdirStore::blobshands out a handle already bound to its store, which is how a consumer avoids picking one. -
A store created before the rename cascades is refused on open, with
PimdirError::Unreconcilablenaming the table. Every foreign key onto a renamable parent carriesON UPDATE CASCADE(spec §14), and noALTER TABLEcan add one, so the draft reconciliation cannot reach it; spec §6's other branch is to refuse the store and have the operator recreate it, which costs a resync of what the format calls a derived cache.Opened anyway, such a store works until something renames a collection, and then SQLite refuses the rename one dependent row down: a server-side rename or an account rename could never be applied, and nothing said so until one was attempted. Every store created by 0.2.0 is in this state.
-
An unknown flag set is written as
NULL. Spec §13 keeps two absences apart:NULLmeans nothing has read an item's markers,'[]'means it is known to carry none. io-replica had no unknown state, so this crate wrote at least'[]'and a probed placement claimed to carry no markers.flags_to_jsonnow returnsNoneforReplicaFlags::Unknownandflags_from_jsondecodes aNULLcolumn back to it, so the column means what the specification says it means.In a queue payload (§15.3) an unknown set encodes as
nullrather than[]: an action states an intent, so its set is known in every payload the format defines, and a nonsensical one stays legible instead of reading as a deliberate clearing of every flag. -
The two schema stamps are checked against each other on open. Spec §4.2 has
PRAGMA user_versionandstore_meta.versionmirror one another and calls a store where they disagree corrupt; this crate read only the pragma, so a half-applied schema change opened as a store at whichever version the pragma happened to hold. Both the owner and the read-only open now refuse it withPimdirError::VersionMismatch. A store whosestore_metarow is absent is left alone, since refusing there would make a missing stamp unrepairable. -
A collection can be paged in its kind's own order.
itemsgained asort_keycolumn and anitems_by_sortindex, andlist_items_page_asc/list_items_page_descreturn a keyset page ordered by it: newest first for mail, A to Z for contacts, a date range for calendars. Until now the only orderings a store could serve were bylink_idorseq, neither of which means anything to a reader, so every consumer had to scan a whole collection into memory to show fifty rows.The cursor is the
(sort_key, seq)pair rather than the key alone, because a key is not unique: two messages share a timestamp, two contacts share a name.seqbreaks the tie, which is what stops a page boundary that lands inside a tie from skipping an item or serving it twice. The first page takes no cursor.An empty key means unknown and is the default, so an item is orderable before it has been summarised: it sorts to the end of a newest-first listing and to the head of an A-to-Z one.
set_sort_keyrestates one item's key, for a store written before its kind had a convention or a consumer whose sync engine does not carry the key inline yet. An ordinary write never resets a key it does not carry. -
A collection can declare which account it belongs to.
collectionsgained a nullableaccountcolumn and the partialcollections_by_accountindex (spec §9.2), a handle speaks for one account (PimdirStore::open(dir, source).for_account("work"), and the same builder onPimdirProducer), andlist_accounts,list_collections_by_account,collection_accountandset_collection_accountread and restate it. Without the column, a merged view, whose defining operation is "everything except this account", had to reverse-engineer the owner's naming convention with aLIKE 'work/%'over collection ids. Folded into version 1, since the format is still a draft, and reconciled on open, so an earlier-draft store heals rather than failing on a missing column.The account partitions nothing, which is the substance of the change. Link ids, object hashes and
seqs keep their store-wide meaning, so two accounts holding oneMessage-IDshare aseqand one body reaching both is one object refcounted twice: scoping identity per account would compile a mail-shaped policy into a kind-agnostic store, and would leave one link id carrying two short forms. What reports the multiplicity instead islink_placementson the identity axis andobject_placementson the dedup axis, each returning every live placement (PimdirPlacement) with the collection and account it sits in, and resolving nothing: a mail view lists them, because two receipts of a newsletter have two read states, while a contact view may offer to merge them.There is no
accountstable: the store records which account a collection belongs to and nothing else, so credentials, endpoints and display names stay with whatever configures the owner.list_accountsis therefore what the collections say rather than a configured roster, and an account with no collection yet does not appear in it. -
rename_collection, which gives a collection a new id and carries its items, bindings, sources, queue rows and child collections with it. Every foreign key ontocollections(id)is nowON UPDATE CASCADE, as isbindings(collection, link_id), which is a parent one level down and refuses the cascade without it.This is the only safe way to change an id, and it matters because the obvious alternative is destructive: deleting a collection and recreating it under a new id cascades every item and binding away, turning a rename into a full re-download and discarding staged local changes. A server renaming a folder and an owner renaming an account both land here.
-
A spec-fidelity test suite comparing the inlined
sqlmodule against the canonical pimdir specification checked out beside it: the schema semantically (columns, defaults, foreign-key actions and indexes, through SQLite's pragmas rather than by text), the presence of every canonical statement by name, and that every inlined statement prepares against the inlined schema. Statement text is deliberately not compared, since the specification permits an equivalent substitution; the three this crate uses are listed explicitly instead. Skips when the specification is not checked out beside this crate.
-
A store created by 0.2.0 was unreadable. The draft-shape reconciliation on open (spec §6) did not carry
items.sort_keyor theitems_by_sortindex, although both were folded into version 1 after 0.2.0 shipped, so every paged read of such a store failed withno such column: sort_key. The column and the index are now reconciled with the rest, and a regression test derives an earlier-draft store from the current schema by dropping each folded-in column, so the next fold is covered without rewriting it.Still not reconcilable: a 0.2.0 store carries no
ON UPDATE CASCADE, whichALTER TABLEcannot add, sorename_collectionon one is refused by SQLite rather than silently orphaning its rows. -
The inlined schema had drifted from the specification.
sql::MIGRATION_0001carried neither thesort_keycolumn nor anyON UPDATE CASCADE, so this crate was creating stores that did not match the format it implements, and nothing detected it. The point ofsqlis to be the canonical copy a consumer runs on its own SQLite driver, so a silent disagreement is the worst failure it has; the fidelity test above exists so it cannot recur.
-
ItemRow.flagsisOption<Vec<String>>,nullin the JSON output while nothing has read the markers. Breaking for anything parsingpimdir item list --output json. -
PimdirItemandPimdirRetainedItemgained asort_keyfield. Breaking for anyone constructing them. -
The sort key io-replica now carries on a placement is bound on write.
loadreturns it, insert and update write it. This reverses the arrangement above, where the key was preserved by the update never naming it: that held only while nothing upstream carried a key, and aloadthat drops it now hands every save an unknown key, which the update would write back, blanking on every sync what the previous one derived. Both halves have to carry the key or neither can.
0.2.0 - 2026-08-07
-
A
pimdiroperator CLI, shipped from this crate behind theclifeature (a[[bin]]withrequired-features, so a library consumer never compiles clap or any terminal dependency). It is to a store whatsqlite3is to a database:collection list,item list(live or--retained, keyset-paged),item show,item export,item restore,item purge(oneseq,--older-than <DURATION>or--all),queue list [--parked],queue cancel,store info,check [--fix]andexport, pluscompletionsandmanuals, each rendering as JSON under--jsonwith logs on stderr.It never interprets item content: a store is kind-agnostic, so the tool prints ids, flags, levels, object hashes and the raw meta, and exports bodies byte for byte. Rendering a message or a vCard belongs to himalaya and cardamum.
Reads open the store read-only, so inspecting a store while a sync runs is always safe.
item restoregoes through the queue as a producer and then drains that collection as the owner, reading the item back to report applied rather than trusting collection-wide drain counters; when a sync holds the lock the action stays queued and applies at the next drain. Purge, queue cancel and the orphan sweep have no action kind, so they take the owner role directly, andPimdirError::Busyreports as "another writer holds the store lock (a sync is running?)". Destructive verbs confirm (with counts and bytes when the store can price them) unless--yes, and refuse to prompt into a pipe or under--json. Listings never truncate silently.checkcloses two gaps the format leaves open: orphan blob files (a crash may leave one and nothing cleaned them) and refcount or reference drift. Only orphan files are reclaimable (--fix, guarded by a--gracewindow because a body is written before the row referencing it); drift and dangling rows are reported, never repaired. -
The store retains items instead of deleting them. An item whose last source binding vanishes is now soft-deleted rather than removed:
itemsgainedretained_at(RFC 3339, stamped by SQLite so the crate stays clock-free) andretained_by, plus a partialitems_retainedindex. The row keeps itsobject_hash, so its body keeps a reference and survives garbage collection. A remote expunge therefore never destroys the local copy, which is what makes a store usable as a backup of a source it does not control. Retention is unconditional: whether a removal is terminal must read identically to every process that opens the store, so it is not configurable. How long to keep, and when to sweep, is the owner's schedule.LOAD_ITEMShides retained rows from the sync seam. That is the condition of correctness, not an optimisation: io-replica's storage spec states that the merge reconciles only whatloadreturns, so a hidden row is never re-derived, on a delta sync or on a full one. io-replica itself needed no change. -
Purge, the only true delete.
purge(collection, seq)takes one retained item,purge_retained_before(cutoff)sweeps every item retired strictly before an RFC 3339 instant the caller computes from its own retention policy. Both release the row's object pin and let the ordinary refcount sweep unlink the body, and both refuse to touch a live item.list_retained,count_retainedandretained_bytesare the trash view beside the live reads;retained_bytesis an upper bound on what a purge would reclaim, since a body a live item also points at survives. -
A reappearing link id revives its retained row (clearing
deleted,retained_atandretained_by, adopting the new content, keeping the message'sseq) instead of colliding on the primary key. One branch serves a source-side resurrection and a client restore alike, so restoring an item is an ordinaryAddover the values retention preserved: no new action kind, no network. -
An owner now skips the queue actions it cannot apply. An unrecognised action kind decodes as
PimdirAction::Unknown { kind, payload, object_hash }, payload verbatim and body still pinned, and the drain leaves the row pending (counted inPimdirDrainReport.skipped) instead of parking it, without blocking the actions behind it. Parking claims an action is permanently unappliable, which is wrong for an intent another owner can perform: this is what lets one queue carry store mutations any owner applies beside capability-bound intents such as a mail submission. Malformed payloads still park. -
drop_action(id)removes one queue row, pending or parked, releasing its object pin in the same transaction: one verb for cancelling a queued action and for acknowledging an intent performed out of band.fail_action(id, error)records a failed attempt, bumpingattemptsfor a transient failure or parking with the reason for a permanent one. -
The store persists a per-source content conflict.
bindingsgainedconflictedandconflict_revision, round-tripped throughReplicaSourceBinding, so the sync layer's memory of "this source and its own remote diverged, unresolved" survives a restart. Without it the merge re-derived the push its remote had already rejected on every run, never converging, and a client could not tell which items needed a human. Distinct from the item-levelconflicted/conflict_object, which is the cross-source divergence; the two are persisted independently. Carried onReplicaSourceBindingas of io-replica 0.3.0.The revision is meaningful only while conflicted (spec §11), so a resolved binding cannot hand a stale one to the next sync.
-
A store written by an earlier draft of schema version 1 is now reconciled on open. The columns above were folded into version 1 rather than added as version 2, the pimdir spec being still
draft, soPRAGMA user_versionstays1, which means an older store is not detectably out of date and would otherwise fail on a query much later.init_schemanow adds any folded-in column it finds missing (and any index over one), guarded byPRAGMA table_infoso it is a no-op for every store after the first open (spec §6's draft allowance). This machinery lapses when the spec freezes its first version.
- BREAKING: bumped io-replica to
0.3, whoseReplicaSourceBindingcarries the per-source conflict this release persists. PimdirAction::kind()returns&strrather than&'static str, since an owner-defined kind is carried as data.
-
PimdirActionError::UnknownKind: an unrecognised action kind is no longer an error, so nothing can construct it. -
sql::DELETE_ITEM: a per-item hard delete has no caller and no counterpart in the format spec's canonical queries any more. An item no source holds is retained (sql::RETAIN_ITEM), and the only true deletes aresql::PURGE_ITEMandsql::PURGE_RETAINED_BEFORE.
0.1.0 - 2026-08-06
- Initial pimdir store: a SQLite index plus a content-addressed, two-level-sharded blob directory, implementing io-replica's storage seam (load, lookup_objects, write) for one source.
- no_std core reusable without the SQLite client: the canonical schema and statements (sql) and the model-to-column encodings (codec).
- Store-global public ids (seq): one per message, shared across every collection it is filed in, monotonic and never reused.
- Streaming blob ingest and read, so a large body is never held whole; a byteless object write indexes a body already streamed to its content-addressed path.
- Incremental, cross-collection-correct reference counting with blob garbage collection inside the write transaction; a crash leaves at worst an orphan blob, never a row without its body.
- Single-writer serialisation via BEGIN IMMEDIATE and a generous busy timeout, so several same-source handles overlap network while their writes serialise.
- An availability-aware, paginated client read surface (list_items, get_item, count_items, distinct_sources, seq_for_link) projecting the store as a local backend.
- The action queue table and collections.generation as part of the draft v1 schema, with user_version and store_meta.version kept in agreement and a store stamped with a higher schema version refused on open (the spec is a draft: draft stores are recreated, never migrated).
- The action queue (spec §14): PimdirProducer (the single enqueue transaction any non-owner process may run, pinning a pre-written body against garbage collection) and the owner's drain (drain_collection applies each action and deletes its row in one transaction, parking permanently failing actions), plus queued_collections, pending_actions (the read-your-writes overlay) and parked_actions.
- The action payload codec in the no_std core: PimdirAction (add, set-flags, remove, move, copy, update, addressing items by public seq) with a strict, versioned JSON round-trip.
- Collection generations (spec §15): the handle-space epoch on PimdirCollection and generation(), bumped atomically with a rebuild batch by write_rekeyed().
- Read-only store open (open_read_only): opens an existing store with SQLITE_OPEN_READ_ONLY, never creates anything, refuses any other schema version, and exposes the full read surface for frontend processes that must be unable to write.