Skip to content

authserver: JWKS endpoint publishes only the signing key, making documented key rotation a hard cutover #6451

Description

@jhrozek

keys.Config documents a three-step, zero-downtime signing-key rotation built on FallbackKeyFiles (pkg/authserver/server/keys/config.go:31-39):

//  1. Add the new key to FallbackKeyFiles and roll out to all replicas.
//  2. Promote it to SigningKeyFile, move the old key to FallbackKeyFiles, roll out.
//  3. Remove the old key from FallbackKeyFiles after its tokens have expired.

Step 1 has no effect. /.well-known/jwks.json publishes exactly one key regardless of how many fallbacks are configured, so the overlap window the procedure depends on never opens and step 2 becomes an atomic cutover that invalidates every outstanding JWT.

Where it breaks

FileProvider loads the fallbacks into allKeys:

// pkg/authserver/server/keys/provider.go:75
for _, filename := range cfg.FallbackKeyFiles {
    key, err := loadKeyFromFile(keyPath)
    allKeys = append(allKeys, key)
}

allKeys is only ever surfaced through PublicKeys(ctx) — and PublicKeys is never called in non-test code. Its sole mention outside the interface declaration is a doc comment at keys/provider.go:47 asserting the behaviour that does not happen. Server construction reads only the primary:

// pkg/authserver/server_impl.go:132
signingKey, err := cfg.KeyProvider.SigningKey(ctx)

which becomes three scalars (server_impl.go:144-146) and then a fixed one-element set:

// pkg/authserver/server/provider.go:325
SigningJWKS: &jose.JSONWebKeySet{Keys: []jose.JSONWebKey{jwk}},

PublicJWKS() (provider.go:405) — what JWKSHandler actually serves — just maps .Public() over that slice, so it inherits the single-key limit.

Impact

  • Signing-key rotation cannot be done without a token-invalidation window. With Cache-Control: public, max-age=3600 on the JWKS response (handlers/discovery.go:88), consumers holding a cached document extend that window to up to an hour.
  • Fails silently: step 1 reports success while changing nothing observable.
  • Asymmetric behaviour: a resource server that resolves keys by kid against a locally-configured full key set will accept a token that any consumer reading the published JWKS rejects.

Suggested fix

Populate SigningJWKS from KeyProvider.PublicKeys(ctx) rather than SigningKey(ctx), keeping the primary first. SigningKey(ctx) stays the source for signing.

Worth checking whether this overlaps with the MIRRORS-OSS divergence tracked in #5773.


Found during a post-merge review of an enterprise-side change that made these keys durable; observed against v0.45.0.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingneeds-triageIssue needs initial triage by a maintainer

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions