Skip to content

feat(otlptrace): add SyncClient and SyncExporter with arena reuse - #8892

Open
marwan562 wants to merge 1 commit into
open-telemetry:mainfrom
marwan562:feat/8860-sync-exporter
Open

feat(otlptrace): add SyncClient and SyncExporter with arena reuse#8892
marwan562 wants to merge 1 commit into
open-telemetry:mainfrom
marwan562:feat/8860-sync-exporter

Conversation

@marwan562

Copy link
Copy Markdown

Fixes #8860

Problem

The current Exporter/Client contract does not guarantee that telemetry passed to Client.UploadTraces is fully consumed when the method returns. A client may retain or process it asynchronously, which prevents safe reuse of arena-backed protobuf memory. PR #8425 demonstrated the allocation/CPU opportunity but reuse was incompatible with the existing contract.

Solution

Introduce a synchronous ownership boundary:

  • SyncClient in go.opentelemetry.io/otel/exporters/otlp/otlptrace extends Client with UploadTracesSync. Contract: MUST fully consume protoSpans before returning and MUST NOT retain/reference/mutate after return.
  • SyncExporter (NewSync/NewSyncUnstarted) uses UploadTracesSync exclusively and reuses an arena via a bounded sync.Pool. Arena is internal; no arena-backed object escapes the synchronous operation.
  • Options (WithInitialBatchSize, WithMaxRetainedBatchSize) allow tuning of preallocation and bounded retention (large arenas are discarded).
  • internal/tracetransform adds Arena with chunkedStorage for KeyValue/AnyValue plus SpansWithArena/KeyValuesWithArena etc. Existing Spans path remains heap-allocated for backward compatibility.
  • HTTP and gRPC clients implement SyncClient (both now assert var _ SyncClient = (*client)(nil)) by factoring UploadTraces into uploadTraces helper so both sync and async paths share the synchronous transport logic (eager marshal for HTTP, retained pbRequest only for the duration of requestFunc for gRPC).

Arena details

  • NewArena(size) preallocates size*8 KeyValue/AnyValue slots plus slices for string/bool/int/double wrappers.
  • Reset clears with clear() and Reset() on protobuf messages; Exceeds(maxSpans) implements bounded retention.
  • SyncExporter.ExportSpans gets arena from pool, calls SpansWithArena, invokes UploadTracesSync, then resets and conditionally returns to pool.

Testing

  • Added sync_test.go for SyncExporter: sync dispatch, error wrapping, MarshalLog filtering, arena reuse, and Option handling.
  • Added arena_sync_test.go verifying SpansWithArena matches heap Spans for attributes/events/links, arena reuse, and Exceeds.
  • Existing tests pass: go test ./exporters/otlp/otlptrace/... and .../otlptracehttp and .../otlptracegrpc (including race).

Prior Art

Follows the proposal in #8860 and reuses logic from #8425 but confines pooling to SyncExporter where the contract makes it safe. Existing Client semantics are unchanged.

Closes #8860

Introduce SyncClient interface extending Client with UploadTracesSync
which guarantees synchronous consumption and no retention after return.
Add SyncExporter (NewSync/NewSyncUnstarted) that uses arena-backed
tracetransform via SpansWithArena and safely reuses arenas through a
bounded sync.Pool. Provide Option tuning for initial batch size and
max retained size to bound memory.

Implement UploadTracesSync for otlptracehttp and otlptracegrpc clients
by reusing existing transport logic via internal uploadTraces helper.

Add internal arena implementation with chunkedStorage for KeyValue and
AnyValue, plus ValueWithArena/KeyValueWithArena helpers. Arena reuse
is only safe under the SyncClient contract; existing Exporter remains
heap-allocated.

Update doc.go and CHANGELOG.md for open-telemetry#8860.

Closes open-telemetry#8860

Signed-off-by: marwan562 <mixing.gamer546@gmail.com>
@linux-foundation-easycla

linux-foundation-easycla Bot commented Aug 28, 2026

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

  • ✅ login: marwan562 / name: marwan562 (d645558)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[otlptrace] add SyncExporter

1 participant