Skip to content

Publish RBAC changes once durable metadata is written, even if staging fails - #79

Merged
VibeSWE merged 2 commits into
mainfrom
fix/put-metadata-staging-consistency
Oct 25, 2026
Merged

VibeSWE merged 2 commits into
mainfrom
fix/put-metadata-staging-consistency

Conversation

@VibeSWE

@VibeSWE VibeSWE commented Oct 25, 2026 •

Copy link
Copy Markdown

Summary

Follow-up from the #67 review (pre-existing, Medium): a staging-write failure in put_metadata left this node's live authorization behind durable state.

  • put_metadata writes RBAC metadata to object storage, then to local staging.
  • If the remote write succeeded but staging failed, it returned Err. Callers then skipped updating the in-memory maps and sessions.
  • So, for example, a role removal could be persisted while the user kept the role on this node until restart.

Fix:

  • Remote metadata is the authoritative copy. Staging is used for startup reconciliation, deployment checks and migrations.
  • So once the remote write succeeds, a staging failure is logged with tracing::error! (with tenant context) and put_metadata returns Ok, letting callers publish the change.
  • Remote failures still propagate.
  • The role and OIDC writers, which had their own copies of the sequence, now use the shared helper.

I audited all 19 callers; none handles the staging error separately.

Tests

The remote-then-staging sequence is factored so it can be tested with injected writers. Committed red first:

  • a staging PermissionDenied after a successful remote write returns Ok;
  • a remote failure still returns Err and skips staging;
  • the success path writes both.

Verification

  • cargo fmt --check ✅, cargo clippy -- -D warnings ✅
  • cargo test --lib: 1024 passed, 0 failed, 2 ignored

Remaining risk

A failed staging write can leave a stale or partial staging file, and startup parsing can still fail on a corrupt one. Atomic staging writes are separate work.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

VibeSWE and others added 2 commits October 24, 2026 22:34
Extract the existing durable-then-staging sequence behind injected writers without changing its behavior. Regress the staging failure that incorrectly prevents callers from publishing a successful durable update, and cover durable failure and successful ordering.

Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Remote metadata is authoritative at runtime and startup. Log a failed local staging refresh with tenant context and return success once the durable write commits, so callers continue their live map and session updates. Keep durable write errors fatal and route the standalone role and OIDC writers through the shared helper.

Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>

@VibeSWE VibeSWE left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial review of PR #79 (at ef0b3084)

One Opus 5.5 and one Sol 6.1 reviewer. Neither found anything blocking.

Remote metadata is authoritative on every path:

  • Startup: resolve_parseable_metadata only uses staging to check that the file exists and that its deployment_id matches. It loads RBAC from remote and rewrites staging from remote.
  • Migrations: they handle remote and staging separately, and the stale staging copy is then overwritten.
  • Runtime: get_metadata never falls back to staging.
  • No overwrite of remote: the reviewers found no path where stale staging can overwrite remote.

All 19 callers were traced; none reads staging back after put_metadata. The role.rs and oidc.rs copies that were removed were byte-for-byte the same as the shared helper.

Severity Finding Outcome
Low A partly written staging file (ENOSPC mid-write) still panics the next startup, and now the operator only sees an error! log, not a 500 Pre-existing; noted in the PR body. Atomic staging writes are follow-up work
Low The staging_dir == storage.root guard compares paths as strings (./data vs /abs/data) Pre-existing misconfiguration edge case; follow-up
Nit Ingestor sync handlers still drop staging errors with let _ Being addressed in the ingestor sync PRs
Nit The log line could include the file path Left as is: I/O errors usually carry the path

@VibeSWE
VibeSWE merged commit 2a3f258 into main Oct 25, 2026
9 checks passed
@VibeSWE
VibeSWE deleted the fix/put-metadata-staging-consistency branch October 25, 2026 04:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant