feat(otlptrace): add SyncClient and SyncExporter with arena reuse - #8892
Open
marwan562 wants to merge 1 commit into
Open
feat(otlptrace): add SyncClient and SyncExporter with arena reuse#8892marwan562 wants to merge 1 commit into
marwan562 wants to merge 1 commit into
Conversation
Introduce SyncClient interface extending Client with UploadTracesSync which guarantees synchronous consumption and no retention after return. Add SyncExporter (NewSync/NewSyncUnstarted) that uses arena-backed tracetransform via SpansWithArena and safely reuses arenas through a bounded sync.Pool. Provide Option tuning for initial batch size and max retained size to bound memory. Implement UploadTracesSync for otlptracehttp and otlptracegrpc clients by reusing existing transport logic via internal uploadTraces helper. Add internal arena implementation with chunkedStorage for KeyValue and AnyValue, plus ValueWithArena/KeyValueWithArena helpers. Arena reuse is only safe under the SyncClient contract; existing Exporter remains heap-allocated. Update doc.go and CHANGELOG.md for open-telemetry#8860. Closes open-telemetry#8860 Signed-off-by: marwan562 <mixing.gamer546@gmail.com>
marwan562
requested review from
MrAlias,
XSAM,
dashpole,
dmathieu,
flc1125 and
pellared
as code owners
August 28, 2026 11:27
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #8860
Problem
The current
Exporter/Clientcontract does not guarantee that telemetry passed toClient.UploadTracesis fully consumed when the method returns. A client may retain or process it asynchronously, which prevents safe reuse of arena-backed protobuf memory. PR #8425 demonstrated the allocation/CPU opportunity but reuse was incompatible with the existing contract.Solution
Introduce a synchronous ownership boundary:
SyncClientingo.opentelemetry.io/otel/exporters/otlp/otlptraceextendsClientwithUploadTracesSync. Contract: MUST fully consumeprotoSpansbefore returning and MUST NOT retain/reference/mutate after return.SyncExporter(NewSync/NewSyncUnstarted) usesUploadTracesSyncexclusively and reuses an arena via a boundedsync.Pool. Arena is internal; no arena-backed object escapes the synchronous operation.WithInitialBatchSize,WithMaxRetainedBatchSize) allow tuning of preallocation and bounded retention (large arenas are discarded).internal/tracetransformaddsArenawithchunkedStorageforKeyValue/AnyValueplusSpansWithArena/KeyValuesWithArenaetc. ExistingSpanspath remains heap-allocated for backward compatibility.SyncClient(both now assertvar _ SyncClient = (*client)(nil)) by factoringUploadTracesintouploadTraceshelper so both sync and async paths share the synchronous transport logic (eager marshal for HTTP, retainedpbRequestonly for the duration ofrequestFuncfor gRPC).Arena details
NewArena(size)preallocatessize*8KeyValue/AnyValue slots plus slices for string/bool/int/double wrappers.Resetclears withclear()andReset()on protobuf messages;Exceeds(maxSpans)implements bounded retention.SyncExporter.ExportSpansgets arena from pool, callsSpansWithArena, invokesUploadTracesSync, then resets and conditionally returns to pool.Testing
sync_test.goforSyncExporter: sync dispatch, error wrapping, MarshalLog filtering, arena reuse, and Option handling.arena_sync_test.goverifyingSpansWithArenamatches heapSpansfor attributes/events/links, arena reuse, andExceeds.go test ./exporters/otlp/otlptrace/...and.../otlptracehttpand.../otlptracegrpc(including race).Prior Art
Follows the proposal in #8860 and reuses logic from #8425 but confines pooling to
SyncExporterwhere the contract makes it safe. ExistingClientsemantics are unchanged.Closes #8860