Add experimental FixedSizeRoundRobinReservoir - #8305
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #8305 +/- ##
=====================================
Coverage 88.4% 88.4%
=====================================
Files 331 332 +1
Lines 21001 21064 +63
=====================================
+ Hits 18569 18632 +63
Misses 2432 2432
🚀 New features to boost your workflow:
|
|
Link failure is because the module isn't released yet. |
There was a problem hiding this comment.
Pull request overview
This PR introduces a new experimental exemplar reservoir implementation (FixedSizeRoundRobinReservoir) under go.opentelemetry.io/otel/sdk/metric/exemplar/x, intended to improve concurrent Offer performance by sharding sampling across per-bucket Algorithm‑L state and selecting buckets via round-robin.
Changes:
- Adds the new experimental reservoir implementation, plus accompanying tests and a parallel benchmark.
- Introduces a new experimental module (
sdk/metric/exemplar/x) with its owngo.mod/go.sumand documentation. - Wires the module into the repo’s release/module metadata (
versions.yaml) and documents it inCHANGELOG.md.
Reviewed changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| versions.yaml | Adds the new experimental module to the experimental-metrics module-set. |
| sdk/metric/exemplar/x/reservoir.go | Implements FixedSizeRoundRobinReservoir and supporting storage/sampling logic. |
| sdk/metric/exemplar/x/reservoir_test.go | Adds unit tests for the new reservoir. |
| sdk/metric/exemplar/x/benchmark_test.go | Adds a parallel benchmark for Offer. |
| sdk/metric/exemplar/x/README.md | Documents the experimental reservoir and intended usage tradeoffs. |
| sdk/metric/exemplar/x/go.mod | Defines the new experimental module and its dependencies. |
| sdk/metric/exemplar/x/go.sum | Dependency checksums for the new module. |
| CHANGELOG.md | Announces the new experimental reservoir. |
|
I have not found any other issues than the ones that are currently reported. |
794e941 to
e3eacf3
Compare
This moves the implementation of the round-robin reservoir from #8257 to an experimental module.
Public API Changes
This moves the
ConcurrentSafeinterface fromsdk/metric/exemplar/internal/reservoirtosdk/metric/exemplar, making it part of the public API for exemplar reservoirs. This is required to be able to use it in an experimental module.Changes
This is an alternative approach to #7447, which had to be reverted in #8249 because of #8238. That approach attempted to make a highly-performant parallel version of algorithm-L, but ultimately is is very difficult to do so correctly.
Instead, this PR tries to improve parallel performance by sharding. Instead of one algorithm-L implementation for a k-sized reservoir, just have k algorithm-L copies for 1-sized reservoirs. This solves #8236 by moving the algorithm-L logic into the storage layer, so that each bucket is individually time-weighted. It also improves the concurrent performance of the fixed-size reservoir by reducing contention on the algorithm-L implementations, since they are distributed across multiple instances.
This should have identical runtime. One k-sized reservoir that handles
nObserve calls has a runtime ofO(k(1 + log(n/k)). A 1-sized reservoirs that handlesn/kObserve calls (we have an even distribution because of our round-robin) would haveO(1(1 + log(n/k))runtime. k 1-sized reservoirs would have aO(k(1 + log(n/k))runtime.This has the following downsides:
This uses a simple round-robin strategy. This does introduce a bias. Algorithm-L by itself makes any combination of exemplars equally likely (e.g. the first k and last k are equally likely). Using round-robin makes some exemplar combinations impossible (e.g. the 1st and k+1 th). This would matter if there was periodic behavior that aligned with the reservoir size. I.e. there is something "special" about every
kthOffer call. You would be highly likely to get only one "special" exemplar, and less likely to get zero or more than one special events. This tend to makes the results more "average". If this is a concern, I think this could be addressed with some added complexity, and some performance overhead.FixedSizeRoundRobin benchmarks.
FixedSize Reservoir benchmarks
Gemini helped me write this.