Repository navigation
fix(imageupgrade): serialise upgrade calls and sync the ESP directory - #376
Merged
Merged
Conversation
StageImage, RollbackImage and ActivateImage took no lock, so two upgrade calls at the same time could share BOOTX64.NEW.EFI and leave a half-written boot image. A shared mutex on the node's image upgrader now refuses a call made while another of the three is already running with FailedPrecondition, rather than queuing it or racing it. The rename that swaps the active image in is also not durable on its own: vfat needs the directory entry fsynced, which a file's own fsync does not cover. Stage and Rollback now fsync the ESP's mounted directory right after that rename, so a power loss right after leaves the old or the new image, never neither. Signed-off-by: Bugs5382 <12115015+Bugs5382@users.noreply.github.com>
Bugs5382
marked this pull request as ready for review
October 14, 2026 15:15
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
Confirmed in code first, as the issue asked:
StageImage,RollbackImageandActivateImagetook no lock anywhere (internal/grpc,internal/init, andthe
realESPMounterthat mounts the ESP per call). Two upgrade calls at thesame time could both write
EFI/BOOT/BOOTX64.NEW.EFIand leave ahalf-written boot image, so this PR covers both halves of the issue.
Serialisation.
nodeImageUpgrader(thegrpc.ImageUpgraderthe nodeserves) now holds one
sync.Mutex, taken withTryLockat the top ofStage,RollbackandActivate. A call made while another of the three isalready running is refused immediately with a new sentinel,
grpc.ErrUpgradeInProgress, which the three gRPC handlers map toFailedPrecondition.TryLockrather thanLock: a concurrent caller istold outright rather than queued or left racing the one in flight.
ESP directory sync. The activating rename (
BOOTX64.NEW.EFI->BOOTX64.EFI) is a vfat directory-entry update; a file's ownfsync(already done in
WriteFile) doesn't make that entry durable. TheFSinterface in
internal/imageupgradegainsSync() error;StageandRollbackcall it immediately after their renames, and the real ESPimplementation (
dirFSininternal/init) opens the mounted root andfsyncs it.
No docs change: nothing here is a new user-facing behavior, only a
correctness fix to an unlocked race and an unsynced rename that were never
documented as safe.
Refs #373
Verification
How this was verified
call order / wrong error), then made to pass:
assert the directory Sync call lands immediately after the activating/restoring rename.
with a blocking Mount hook: the first holds the lock mid-call, the second gets
ErrUpgradeInProgress immediately, then the first completes cleanly once released.
ActivateRefusesWhileAnUpgradeIsInProgress hold the lock directly and assert each of the three
entry points refuses rather than mounting the ESP.