caddytls: reused certificate cache races with renewal workers during provisioning

[potatosips]

Issue Details

Issue

On Caddy master at 9738f49b952a337cdaf42175a0e92b01974c0e3c, the race-enabled caddytls tests detect a data race when TLS.Provision reuses the process-wide CertMagic cache while a certificate-management worker is reading its options.

Reproduction

Environment: Ubuntu 24.04.4 LTS, ARM64, Go 1.25.1, CertMagic v0.25.3.

go test -race ./modules/caddytls -run '^TestAvoidDuplicateAutomation$' -count=50

This exits 1 and reproduced reliably during the audit.

Race report

WARNING: DATA RACE
Write at ...:
  github.com/caddyserver/certmagic.(*Cache).SetOptions()
      cache.go:135
  github.com/caddyserver/caddy/v2/modules/caddytls.(*TLS).Provision()
      tls.go:218

Previous read at ...:
  github.com/caddyserver/certmagic.(*Config).certNeedsRenewal()
      certificates.go:108
  github.com/caddyserver/certmagic.Certificate.NeedsRenewal()
      certificates.go:80

Root cause

Caddy calls certCache.SetOptions(cacheOpts) from TLS.Provision when the global cache already exists. CertMagic’s SetOptions writes certCache.options while holding optionsMu.Lock, but certificates.go reads cfg.certCache.options.RenewCheckInterval without optionsMu.RLock.

The same unlocked reads are present on CertMagic master at 38cdd6254bf2fc9c5e456aae41a93214e0e64514, around certificates.go lines 108, 142, and 179. The test provisions successive TLS configurations while an asynchronous management job is active, so a live config reload may exercise the same race.

Implemented fix

A focused CertMagic patch now takes optionsMu.RLock once near the start of Config.certNeedsRenewal, snapshots RenewCheckInterval, unlocks, and uses that snapshot at all three sites. This removes the race while keeping one coherent interval for the whole renewal decision.

The patch also adds TestCertificateNeedsRenewalConcurrentCacheOptions. It starts two goroutines together and performs 10,000 concurrent Cache.SetOptions and Certificate.NeedsRenewal calls. The regression fails immediately under -race before the source fix and passes afterward.

Verification after the fix

• New CertMagic regression: passes under -race, 20 repetitions.
• go test -v -short -race ./... in CertMagic: passes.
• go vet ./... in CertMagic: passes.
• Caddy TestAvoidDuplicateAutomation using the locally patched CertMagic: passes under -race, 50 repetitions.
• Patch applies cleanly to the pinned CertMagic master commit above.

The Caddy integration command was:

go mod edit -replace github.com/caddyserver/certmagic=/home/ubuntu/certmagic-race-audit
go test -race ./modules/caddytls -run '^TestAvoidDuplicateAutomation$' -count=50

The temporary module replacement was removed after verification. The source fix belongs in caddyserver/certmagic; this Caddy issue records the user-visible caddytls trigger and complete verification.

Additional context

This was found while verifying an unrelated resource-leak patch for #7833. None of that patch’s changed files are in the caddytls or CertMagic paths.

Assistance Disclosure

AI not used

If AI was used, describe the extent to which it was used.

used chatgpt to format the issue

[steadytao]

This looks correctly isolated to CertMagic rather than Caddy. Could you properly link the CertMagic PR containing the locking change and regression test?