[9.5](backport #7712) feat: add synchronous enrollment write strategy to prevent ghost agents

[mergify]

What is the problem this PR solves?

When an enrollment write is committed to fleet-server's in-memory bulk queue but the HTTP response is lost before it reaches the agent (network drop, EOF), the agent retries enrollment. If the retry lands on a different pod, that pod's FindAgent search finds nothing — the write hasn't been flushed to Elasticsearch yet — and creates a second agent document. These duplicate documents ("ghost agents") accumulate and do not resolve on their own.

The batched pre-refresh dedup approach in #7662 cannot prevent this: cross-pod retries are never in the same flush batch, so in-batch dedup never fires. Evidence from the 30k serverless checkin scale test: 35 ghost agents with enrolled_at timestamps clustered in a 21-second window (11:53:42–11:54:03 UTC), zero ErrEnrollDuplicate 429s — confirming in-batch dedup never triggered for any of the 35 retries.

How does this PR solve the problem?

A new feature flag, inputs[].server.feature_flags._sync_enrollment_write (default: false), selects the enrollment write strategy.

When false (default), the existing async bulk queue path is unchanged.

When true, createFleetAgent writes the agent document directly to Elasticsearch using op_type=create&refresh=wait_for. ES blocks until the document is committed and visible to search on all shards before returning. The enrollment response is not sent until this write completes, so any retry on any pod finds the existing document immediately. A 409 from op_type=create is treated as success.

The _ prefix on the flag name signals that it is internal and temporary. The intent is to enable it in Staging via serverless-gitops, validate with the 30k checkin scale test, and then remove the flag entirely — making the sync path the permanent enrollment write path.

How to test this PR locally

Enable the flag and run the 30k Fleet Server checkin scale test against Staging. Confirm ghost agent count is 0 after enrollment completes.

Design Checklist

• I have ensured my design is stateless and will work when multiple fleet-server instances are behind a load balancer.
• I have or intend to scale test my changes, ensuring it will work reliably with 100K+ agents connected.
• I have included fail safe mechanisms to limit the load on fleet-server: rate limiting, circuit breakers, caching, load shedding, etc.

Checklist

• I have added an entry in ./changelog/fragments using the changelog tool

Related issues

• Relates feat: batch enrollment FindAgent searches with pre-refresh dedup #7662
• Relates elastic/ingest-dev#9167

  ---

  This is an automatic backport of pull request feat: add synchronous enrollment write strategy to prevent ghost agents #7712 done by Mergify.

[mergify]

Cherry-pick of d6324c7 has failed:

On branch mergify/bp/9.5/pr-7712
Your branch is up to date with 'origin/9.5'.

You are currently cherry-picking commit d6324c7.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   internal/pkg/api/handleEnroll.go
	modified:   internal/pkg/config/input.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   internal/pkg/api/handleEnroll_test.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

[ebeahan]

@ycombinator if we want this in the next 9.5.x patch, we need to merge this backport ASAP.