Skip to content

Add cross-platform locking primitives - #2492

Open
mjcheetham wants to merge 9 commits into
git-ecosystem:mainfrom
mjcheetham:xplat-mutex
Open

mjcheetham wants to merge 9 commits into
git-ecosystem:mainfrom
mjcheetham:xplat-mutex

Conversation

@mjcheetham

@mjcheetham mjcheetham commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Parallel Git operations, such as fetching submodules, can start several GCM processes that all need to authenticate the same account. Concurrent requests have become more prevalent with AI agent-based workflows, which often have multiple subagents and worktrees with Git operations occurring at the same time. Visual Studio Code's 'auto fetch' setting similarly can trigger many concurrent git fetch requests that may trigger auth prompts.

When silent authentication cannot satisfy those requests, each process can start its own interactive sign-in flow. This creates duplicate prompts and can cause contention between authentication attempts.

Coordinating work across independent GCM processes requires a shared locking primitive. This PR provides that foundation on Windows, macOS, and Linux.

This PR deliberately stops at the locking infrastructure: it does not integrate locks with any host provider, change authentication behaviour, or introduce authentication settings. Using these primitives to coordinate authentication is left for a follow-up PR.

Locking model and diagnostics

The series introduces injectable ILock and ILockFactory abstractions through the command context, with a file-backed current-user implementation. Locks use an exclusively opened local file beneath the user's data directory. Ownership follows the open file handle, not the existence of the pathname or the PID recorded in it.
Unlocked leftover files can therefore be reclaimed without guessing whether their recorded owner is still alive.

Callers can attempt acquisition without waiting, specify a finite timeout, or wait indefinitely. Asynchronous acquisition supports cancellation, and contention retries use jitter to avoid synchronised retry storms.

Acquisition is asynchronous, release can occur on another thread, and the timeout budget covers both same-instance queuing and file contention. Only current-user locks are implemented; process and machine scopes remain out of scope. The implementation requires a local filesystem with .NET file locking enabled.

git-credential-manager diagnose gains a locking diagnostic that inspects lock metadata and reports potentially orphaned or corrupt files without deleting them automatically. Classic trace output also includes the process ID to help correlate concurrent GCM instances.

Example usage

Given an ICommandContext context, a caller's cancellation token ct, and an asynchronous operation DoWorkAsync, a caller can bound the wait for a named current-user lock:

using System;
using GitCredentialManager.Concurrency;

using (ILock operationLock = context.LockFactory.CreateLock(
    LockScope.CurrentUser, "example-shared-operation"))
{
    await operationLock.AcquireAsync(TimeSpan.FromSeconds(10), ct);
    await DoWorkAsync(ct);
}

Other GCM processes running as the same user and using the same lock name coordinate through the same file. The name must be a single filename component, not a path.

The using scope releases and disposes the lock even if the protected work throws. The lock can be held across await points without thread affinity. The ten-second timeout applies only to acquisition, not to DoWorkAsync: AcquireAsync throws TimeoutException if it cannot acquire the lock in time, or OperationCanceledException if the caller cancels acquisition.

For optional work that should be skipped rather than queued, use a zero-timeout attempt:

using (ILock operationLock = context.LockFactory.CreateLock(
    LockScope.CurrentUser, "example-shared-operation"))
{
    if (await operationLock.TryAcquireAsync(TimeSpan.Zero, ct))
    {
        await DoWorkAsync(ct);
    }
}

TryAcquireAsync returns false on timeout; cancellation and other errors still throw. Alternatively, AcquireAsync(ct) waits indefinitely with cancellation, and TryAcquire() makes a synchronous no-wait attempt. Callers can also explicitly Release() a held lock and acquire it again before disposing the instance.

Test coverage

The series adds coverage for contention, timeout and cancellation behaviour, disposal, release from another thread, and recovery of unlocked leftover files. A real-filesystem test exercises contention between multiple lock instances. Additional tests cover factory path validation, lock extension methods, and timeout jitter.

Emit the PID of our process to the classic tracing mechanism, alongside
the existing runtime environment info, to help debugging.

Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com>
Windows APIs can return HRESULTs which is a 32-bit number that encodes
Win32 error codes in the lower 16-bits (when some other, higher bits are
set). We want to be able to convert HRESULT into Win32 error codes, so
let's make a helper for that.

Assisted-by: GPT-6.1 Sol
Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com>
Add an extension method for the Stopwatch that can be used to quickly
determine how long is left on a running stopwatch compared to a total
budget.

We will use this in future commits to easily keep track of a remaining
timeout budget during a multi-step lock acquisition process.

Assisted-by: GPT-6.1 Sol
Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com>
Timeouts that have static values can cause thundering-herd issues[1].
Introduce an easy way to add a jitter[2] to `TimeSpan`s; add a random
amount of time to avoid timeouts from all occurring at the exact same
time.

[1]: https://en.wikipedia.org/wiki/Thundering_herd_problem
[2]: https://en.wikipedia.org/wiki/Jitter

Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com>
Add a method to the IFileSystem interface to create a new file stream
with custom FileOptions, such as DeleteOnClose.

Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com>
Add a new interface (currently with no implementations) of a lock object
that can be acquired and released.

In addition, provide a factory, reachable from the command context for
creating lock objects for various scopes (machine-wide, current user,
current process).

This allows consumers to mock and test locking, and components that use
the lock factory.

Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com>
Concurrent interactive authentication must serialize across independent
GCM processes, including processes that do not share managed state. An
exclusively opened local file supplies that coordination without relying
on managed finalizers or PID-based ownership guesses.

Serialize same-instance callers separately from file contention so async
waiting and release from another thread remain safe. The file contains a
PID for diagnostics only; OS ownership determines whether a leftover
pathname can be reclaimed.

Expose the implementation through current-user locks beneath the user
data directory. Other scopes remain deliberately unimplemented.

Assisted-by: GPT-6.1 Sol
Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com>
@mjcheetham
mjcheetham requested a review from a team as a code owner October 29, 2026 14:32
@becm

becm commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

Still not really clear to me; is the intent to (also) support locking for the concept of splitting get and store?
Which would not be mappable to "open" file status, as an atomic operation spans 2 process invocations.

Add a new diagnostic for inspecting possibly orphaned file locks. To
allow for read-only inspection of the lock file (that contains the
owning PID as the contents), we need to change the sharing mode on
Windows, and bypass the flock-call on Unix.

Since Unix file locks are advisory we can just bypass the exclusive lock
(LOCK_EX) by directly calling open() without flock(). .NET FileStream
with a file path contructor will try and call flock() so we avoid that
and manually open the file descriptor, and manually read from the file.

On Windows, where file locks are mandatory, we need to adjust how we
take ownership of the lock file in the first place, to permit other
processes to open it again with read-only permissions.

Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com>
@mjcheetham

Copy link
Copy Markdown
Contributor Author

support locking for the concept of splitting get and store?

As you mentioned, no helper can effectively coordinate safely across the get/store (or erase) boundary. Even with state[] attributes. There'd need to be a fundamental redesign of the auth story in Git itself, which if you've looked deeply at this (as I have) would first require some major changes to how Git handles HTTP connections (it's an absolute mess and very fragile!).

The intent here would be for providers to be able to coordinate across interactive scenarios, so only within their own internal logic of a single get request. E.g., serialising Entra interactive authentication requests for the same authority.

I think trying to enforce something at a higher level (GCM wide) is not sensible without Git credential protocol coordination and first-class conceptual support in Git for 'concurrent Git operations on the same remote/repo etc'.

@mjcheetham

Copy link
Copy Markdown
Contributor Author

There'd need to be a fundamental redesign of the auth story in Git itself, which if you've looked deeply at this (as I have) would first require some major changes to how Git handles HTTP connections (it's an absolute mess and very fragile!).

Aside, to mention more on this point, I would like to see the following changes in Git itself for a modern auth protocol:

  1. A real, instance-based, HTTP client API. Currently it's just a global/static mess of interleaving functions in http.c (and not to mention that there are other parts of the codebase that reach directly into various curl handles etc and do not use the http.c function(!)

  2. An 'authenticator' API that delegates all authentication challenges to a 3rd party entity, via a more high bandwidth channel (probably based on Git's existing Simple IPC infrastructure that is already used for FSMonitor). Consumers like HTTP transport, or FTP, or SMTP/sendmail should listen to the appropriate events (401, 403, proxy auth) and raise these over the IPC channel to a configured authenticator. The authenticator should be able to respond and modify the operation in any way that's reasonable, and decide how to resubmit a request - the request flow should be controllable by an authenticator.

  3. Appropriately strong unit and integration testing around existing HTTP/FTP/SMTP behaviour. This is really a pre-requisite for any changes to ensure externally facing behaviour of Git does it degrade. There is a lot of untested or undocumented, and often unintended behaviour.

  4. Provide a built-in authenticator implementation that can map to old-style credential helpers, for compatibility or when no other authenticator is configured.

@brianelivalenciavelaquez-boop

This comment was marked as spam.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants