Commit 562c200
authored
test: isolate the keychain from the test suite (#214)
Rebased onto `main` @ 9d67765 (after #201, #216, #213, #212, #203,
#219).
#207 already fixed the root cause for the two crates it touched, so this
PR is
now the **audit**, the **consolidation**, and the **docs** - the three
halves it
did not cover.
## Root cause (recap, for context)
`keyring` 4.x's `Entry::new` installs the PLATFORM-NATIVE store as the
process
default on its **first** call, overwriting whatever default is already
set
(`keyring-4.1.5/src/v1.rs`, the `SET_CREDENTIAL_STORE` latch). So
installing
`keyring-core`'s mock *before* the first real `Entry` is silently
undone, and
every "mock" write lands in the real OS keychain. On macOS each rebuild
is a new
binary identity, so the OS re-prompts on every single rebuild and the
run blocks
on the modal dialog.
Confirmed against the maintainer's login keychain before touching
anything - it
held the tests' own hard-coded account ids:
```
driven.google.refresh_token / acct-with-token (created 20260729180400Z)
driven.google.client_creds / acct-byo
driven.s3.credentials / acct-s3-round-trip (created 20260729181820Z)
```
## 1. The audit: are there any other paths?
**Verified answer: no. Every test that reaches a real `keyring::Entry`
is now
behind the shared helper - there are no unisolated paths left.**
Done empirically rather than by inspection. A temporary probe panicked
at all
**four** `Entry` construction sites in the workspace -
`driven-crypto/src/keystore.rs:63`,
`driven-drive/src/google/token_store.rs:102`
and `:157`, `driven-s3/src/config.rs:221` - and
`cargo test --workspace --no-fail-fast` ran every test binary on this
exact
tree. Exactly **13** tests reach an entry, across 5 binaries:
| Crate | Tests reaching an entry | Services touched |
|---|---|---|
| `driven-backend` | 3 (pre-existing) | `driven.google.refresh_token`,
`driven.google.client_creds` |
| `driven-s3` | 4 (pre-existing) | `driven.s3.credentials` |
| `driven-crypto` | 3 (new here) | `dev.maxhogan.driven` |
| `driven-drive` | 2 (new here) | `driven.google.refresh_token`,
`driven.google.client_creds` |
| `src-tauri` | 1 (new here) | `dev.maxhogan.driven` |
All 13 go through `driven_test_fixtures::keychain::isolated()`. The
probe hit
list and the isolated-test list match exactly, with no remainder.
Also checked and clear: `driven-cli` (its integration tests only
exercise
`--help` / missing-argument paths, so the spawned binary never opens an
entry),
every integration test under `crates/*/tests` and `src-tauri/tests`, and
the two
new crates from this week - `driven-localfs` and `driven-rclone`
reference
neither `keyring` nor any of the credential-store types.
## 2. Consolidation: one helper, not three
`driven_test_fixtures::keychain`
(`crates/driven-test-fixtures/src/keychain.rs`)
is now the single implementation. Both of #207's local copies are
repointed at
it and `keyring-core` is dropped from both crates' dev-deps:
- `crates/driven-backend/src/lib.rs` - 60-line local helper -> one-line
delegate
- `crates/driven-s3/src/config.rs` - same
It keeps #207's load-bearing ordering (burn the latch with a throwaway
`Entry`,
*then* install the mock) and adds one thing #207's version lacks:
**An I/O-free precondition ahead of the proof write.** The installed
default
store must report `CredentialPersistence::ProcessOnly`, which by
definition
means its credentials cannot outlive the process; a real OS keychain
reports
`UntilDelete`/`UntilReboot`. So a defeated mock is caught **before
anything is
written**. Under #207's version the sentinel write is itself the first
thing
that would leak into a real keychain when the mock fails. The sentinel
round
trip still runs, after the precondition passes, as the functional check.
Constructing the throwaway `Entry` is safe: `build` in
`apple-native-keyring-store-1.0.1/src/keychain.rs` is pure struct
construction
with no keychain I/O, so it raises no prompt and creates nothing.
**Drift protection:** `the_test_suite_is_isolated_from_the_os_keychain`
in every
crate that owns a keychain call site - `driven-crypto`, `driven-drive`,
`driven-s3`, `driven-backend`, `src-tauri`. These *assert* rather than
skip, so a
future `keyring` upgrade that breaks the mechanism fails loudly instead
of the
suite quietly resuming real writes. All five crates now have the dev-dep
wired,
so isolating a new test is a one-line change with no `Cargo.toml` work.
Production is untouched: no shipped code path changed, and
`keyring-core` is a
direct dependency only of `driven-test-fixtures`, which is `publish =
false` and
only ever a `[dev-dependencies]` entry.
## 3. The docs half
### README - new "macOS re-prompts for keychain access after every
update"
Placed in the existing macOS caveats, between the APFS locked-file
section and
the auto-updater caveat. Verified, not restated:
- Driven's macOS build carries **no Developer ID signature**. It is only
ad-hoc
(linker) signed, so it has no stable
[designated
requirement](https://developer.apple.com/library/archive/technotes/tn2206/_index.html)
and its cdhash changes with every build. (`tauri.conf.json` sets no
`signingIdentity`; `release.yml` runs no `codesign`.)
- macOS pins keychain ACLs to the identity of the binary that was
granted
access. Apple states the rule directly:
["This dialog appears if you recently updated your system software or
the app, or if the app has been
modified"](https://support.apple.com/guide/keychain-access/if-a-trusted-app-asks-for-keychain-access-kyca1331/mac).
So "Always Allow" does not carry across an update, for any of the four
services Driven uses (`dev.maxhogan.driven`,
`driven.google.refresh_token`, `driven.google.client_creds`,
`driven.s3.credentials`).
- Denying it is safe by construction - an encrypted source whose master
key
cannot be read fails closed (`crypto.key_missing`) rather than uploading
plaintext - but it stalls encrypted sources until the user
re-authorizes.
- The only fix is a Developer ID signature. Stated without overstating:
no
entitlement or partial workaround makes an ad-hoc-signed build's grants
survive an update.
**Aligned with #216, not contradicting it.** #216's
`fdaBanner.unsignedNote`
already covers the TCC/Full Disk Access half ("macOS ties this
permission to the
app's signature, and Driven is not signed yet ... remove Driven from the
Full
Disk Access list and add it back"). The README's FDA bullet gives the
same
mechanism and the same remove-and-re-add remedy, and now explicitly says
the
in-app banner says the same thing. The genuinely new material is the
**keychain**
half, which nothing documented.
### DESIGN
- 3.6 gains "macOS, second cost: permission grants do not survive an
update"
next to the existing "no Apple Developer ID" material. It defers the TCC
half
to 5.3.3 rather than restating it, and documents the keychain half.
- 5.3.3 (#216's FDA onboarding section, which already explains the
cdhash
binding for TCC) gains a short pointer noting the same binding governs
keychain ACLs, with a cross-reference to 3.6.
### CONTRIBUTING
A local-gates subsection: the suite is keychain-isolated, how to isolate
a new
test, why macOS makes it load-bearing, and cleanup commands for anyone
who ran
the suite while the flawed helper was on `main`.
## Bonus: coverage that was previously impossible
The old module docs in `driven-crypto/src/keystore.rs` and
`driven-drive/src/google/token_store.rs` claimed the mock store was
unusable and
steered contributors away from testing these paths. Both are corrected,
and the
paths they excluded are now covered for real against the in-memory
store:
- `Keystore` store/load/delete master key, `NotFound` on a wiped
keychain,
idempotent delete, per-account scoping
- `KeyringTokenStore` and `ClientCredsStore` round trips, incl. the
empty-secret
PKCE case and per-account isolation
- `KeystoreCryptoProvider` with an encrypted source that HAS a wrapped
key but
no master key - the only path there that actually opens the keystore,
which no
existing test reached (every existing one short-circuits on a missing
wrapped
key). This is the GA-critical fail-closed rule.
## Verification: zero keychain prompts
Prompts are interactive and the machine is locked, so this is proved,
not
asserted. Headless throughout - no GUI automation, no app launch.
1. **Complete audit.** The 4-site panic probe above enumerated every
test in the
workspace that constructs a keychain entry; all 13 are isolated.
2. **The five guard tests pass**, so the mock really is the effective
default
store in each of those binaries - the exact thing that was silently
false.
3. **mdat unchanged.** After a full `cargo test --workspace`, the S3
item's
`mdat` is still `20260729181820Z` - unchanged across multiple full runs
and a
targeted `cargo test -p driven-s3`, whose
`credentials_round_trip_through_the_keychain` stores to that exact
account.
Nothing wrote.
4. **Nothing created.** The two Google items were cleaned up
mid-session, so
runs since then started from an empty state for those services. After
the
final full run, `dev.maxhogan.driven`, `driven.google.refresh_token`,
`driven.google.client_creds`, the helper's own
`driven.test.keyring-latch` /
`driven.test.keyring-sentinel` and the test round-trip service are **all
absent** - despite the suite performing real store/load/delete round
trips
against every production service.
5. **No dialog.** Every run executed non-interactively in the background
and
exited on its own, so nothing blocked on a modal prompt.
## Gates (all re-run after the rebase onto 9d67765)
- `SQLX_OFFLINE=true cargo test --workspace` - pass, 0 failures
- `SQLX_OFFLINE=true cargo clippy --workspace --all-targets -- -D
warnings` - clean
- `cargo fmt --all -- --check` - clean
- `git diff --check` - clean
- `cargo deny check` - `advisories ok, bans ok, licenses ok, sources ok`
- No em/en dashes in the diff (checked)
- UI suite not run: no UI file is touched by this PR.
Note: `test:` is a hidden changelog type here, so the user-facing
README/DESIGN
macOS caveat will not appear in the release notes. Flagging deliberately
- a
follow-up `docs:` commit is your call.1 parent 7d36ef0 commit 562c200
17 files changed
Lines changed: 658 additions & 154 deletions
File tree
- crates
- driven-backend
- src
- driven-crypto
- src
- driven-drive
- src/google
- driven-s3
- src
- driven-test-fixtures
- src
- design
- src-tauri
- src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
105 | 105 | | |
106 | 106 | | |
107 | 107 | | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
108 | 163 | | |
109 | 164 | | |
110 | 165 | | |
| |||
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
272 | 272 | | |
273 | 273 | | |
274 | 274 | | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
275 | 310 | | |
276 | 311 | | |
277 | 312 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
27 | | - | |
28 | | - | |
29 | | - | |
30 | | - | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
35 | | - | |
36 | | - | |
37 | | - | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
38 | 35 | | |
39 | 36 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
420 | 420 | | |
421 | 421 | | |
422 | 422 | | |
423 | | - | |
424 | 423 | | |
425 | | - | |
426 | | - | |
427 | | - | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
428 | 430 | | |
429 | | - | |
430 | | - | |
431 | | - | |
432 | | - | |
433 | | - | |
434 | | - | |
435 | | - | |
436 | | - | |
437 | | - | |
438 | | - | |
439 | | - | |
440 | | - | |
441 | | - | |
442 | | - | |
443 | | - | |
444 | | - | |
445 | | - | |
446 | | - | |
447 | | - | |
448 | | - | |
449 | | - | |
450 | | - | |
451 | | - | |
452 | | - | |
453 | | - | |
454 | | - | |
455 | | - | |
456 | | - | |
457 | | - | |
458 | | - | |
459 | | - | |
460 | | - | |
461 | | - | |
462 | | - | |
463 | | - | |
464 | | - | |
465 | | - | |
466 | | - | |
467 | | - | |
468 | | - | |
469 | | - | |
470 | | - | |
471 | | - | |
472 | | - | |
473 | | - | |
474 | | - | |
475 | | - | |
476 | | - | |
477 | | - | |
478 | | - | |
479 | | - | |
| 431 | + | |
| 432 | + | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
480 | 436 | | |
481 | 437 | | |
482 | 438 | | |
| |||
492 | 448 | | |
493 | 449 | | |
494 | 450 | | |
| 451 | + | |
| 452 | + | |
| 453 | + | |
| 454 | + | |
| 455 | + | |
| 456 | + | |
| 457 | + | |
| 458 | + | |
| 459 | + | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
| 464 | + | |
495 | 465 | | |
496 | 466 | | |
497 | 467 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
0 commit comments