Skip to content

sdk: optimize composite attribute truncation - #8912

Open
pellared wants to merge 11 commits into
open-telemetry:mainfrom
pellared:optimize-attribute-truncation-8491
Open

sdk: optimize composite attribute truncation#8912
pellared wants to merge 11 commits into
open-telemetry:mainfrom
pellared:optimize-attribute-truncation-8491

Conversation

@pellared

@pellared pellared commented Sep 1, 2026

Copy link
Copy Markdown
Member

Changes

  • Avoid rebuilding unchanged STRINGSLICE, SLICE, and MAP attribute values in the Trace and Logs SDK truncation paths.
  • Replace the preflight-plus-transform shape with a single-pass, post-order transform that lazily allocates on the first changed child.
  • Add fixed-array and reflection-backed regression cases, allocation assertions, deep nesting coverage, and matched benchmarks for both public entry shapes.

Performance analysis

The previous implementation materialized defensive copies through AsStringSlice, AsSlice, or AsMap even when a finite configured limit did not truncate anything. Nested values that did change were first searched by needsTruncation and then walked again to rebuild them, repeatedly revisiting descendants as depth increased.

This change reads the immutable array storage using the existing attrnorm raw-storage convention and performs one post-order traversal. Each collection keeps its original attribute.Value until the first changed child; only then does it allocate a result and backfill the unchanged prefix. Therefore unchanged composite values allocate nothing, while changed nested values rebuild only the affected ancestry. The traversal visits each value once and recursion uses the Go stack, providing O(depth) temporary state without introducing a heap-allocated work stack. Direct top-level scalar and composite dispatch keeps the common wrapper paths from paying an avoidable generic-dispatch call.

Highlights from the matched microbenchmark:

  • finite-limit, unchanged STRINGSLICE, SLICE, and MAP values fall to 0 B/op and 0 allocs/op for both Value (Logs) and KeyValue (Trace) entry shapes, including reflection-backed length-6 values
  • an unchanged depth-32 mixed value falls from 33 allocs/op and 3.531 KiB/op to zero
  • a changed depth-32 mixed value falls from 626 to 66 allocs/op, 65.562 KiB/op to 7.062 KiB/op, and about 80 us/op to 11.8 us/op
  • a flat STRINGSLICE that does require truncation retains 2 allocs/op and 64 B/op, but is 36-45% slower in this focused benchmark; this is the cost of the lazy per-element change detection used to eliminate the common no-op allocations
  • reflection-backed unchanged SLICE values remove their allocation and 288 B/op, with a measured 3.3-4.1% latency increase; the other composite no-op and changed cases improve or are statistically unchanged
  • scalar paths remain at 0 B/op and 0 allocs/op; STRING is 6.4% slower for the Logs-style Value case and 8.3% faster for the Trace-style KeyValue case, while BYTESLICE is statistically unchanged for Value and 2.6% slower for KeyValue

Benchmark setup

  • Base: 31d148c597cb29b2eacc1e9c74a62c98a9d140b5 plus this PR's benchmark harness only
  • Measured head: 41c3c897d7375a13731e0cd6336a559cd4801e01
  • Current PR head benchmark-relevant source and module inputs are byte-identical to the measured head
(cd sdk && GOMAXPROCS=16 go test -run '^$' -bench '^BenchmarkTruncateValue$' -benchmem -benchtime=500ms -count=10 ./internal/attrnorm)
benchstat base.txt head.txt

Raw benchstat output

goos: linux
goarch: amd64
pkg: go.opentelemetry.io/otel/sdk/internal/attrnorm
cpu: AMD EPYC 7571
                                                       │   base.txt   │               head.txt               │
                                                       │    sec/op    │    sec/op     vs base                │
TruncateValue/String/NoTruncation/Value-16                24.75n ± 0%   26.32n ±  2%   +6.36% (p=0.000 n=10)
TruncateValue/String/NoTruncation/KeyValue-16             38.60n ± 0%   35.42n ±  0%   -8.25% (p=0.000 n=10)
TruncateValue/ByteSlice/NoTruncation/Value-16             24.73n ± 0%   24.80n ±  2%        ~ (p=0.135 n=10)
TruncateValue/ByteSlice/NoTruncation/KeyValue-16          25.93n ± 0%   26.59n ±  1%   +2.55% (p=0.000 n=10)
TruncateValue/StringSlice/NoTruncation/Value-16          126.40n ± 1%   80.19n ±  0%  -36.56% (p=0.000 n=10)
TruncateValue/StringSlice/NoTruncation/KeyValue-16       139.35n ± 0%   85.08n ±  0%  -38.94% (p=0.000 n=10)
TruncateValue/StringSlice/Truncation/Value-16             142.1n ± 1%   206.4n ±  0%  +45.25% (p=0.000 n=10)
TruncateValue/StringSlice/Truncation/KeyValue-16          153.7n ± 1%   208.8n ±  0%  +35.88% (p=0.000 n=10)
TruncateValue/Slice/NoTruncation/Value-16                 131.4n ± 0%   130.4n ±  0%   -0.76% (p=0.000 n=10)
TruncateValue/Slice/NoTruncation/KeyValue-16              144.9n ± 1%   138.7n ±  0%   -4.28% (p=0.000 n=10)
TruncateValue/Slice/Truncation/Value-16                   406.5n ± 1%   340.4n ±  1%  -16.25% (p=0.000 n=10)
TruncateValue/Slice/Truncation/KeyValue-16                420.0n ± 1%   346.8n ±  1%  -17.44% (p=0.000 n=10)
TruncateValue/Map/NoTruncation/Value-16                   141.2n ± 1%   114.5n ±  0%  -18.91% (p=0.000 n=10)
TruncateValue/Map/NoTruncation/KeyValue-16                152.0n ± 1%   121.3n ±  0%  -20.20% (p=0.000 n=10)
TruncateValue/Map/Truncation/Value-16                     484.2n ± 2%   397.2n ±  1%  -17.95% (p=0.000 n=10)
TruncateValue/Map/Truncation/KeyValue-16                  499.6n ± 1%   402.0n ±  1%  -19.54% (p=0.000 n=10)
TruncateValue/StringSliceWide/NoTruncation/Value-16       604.9n ± 1%   152.0n ± 13%  -74.86% (p=0.000 n=10)
TruncateValue/StringSliceWide/NoTruncation/KeyValue-16    612.5n ± 1%   157.9n ± 17%  -74.22% (p=0.000 n=10)
TruncateValue/SliceWide/NoTruncation/Value-16             342.4n ± 1%   353.6n ±  0%   +3.26% (p=0.000 n=10)
TruncateValue/SliceWide/NoTruncation/KeyValue-16          342.8n ± 1%   356.9n ±  0%   +4.14% (p=0.000 n=10)
TruncateValue/MapWide/NoTruncation/Value-16               400.5n ± 1%   302.2n ±  1%  -24.54% (p=0.000 n=10)
TruncateValue/MapWide/NoTruncation/KeyValue-16            395.0n ± 1%   309.7n ±  0%  -21.59% (p=0.000 n=10)
TruncateValue/Nested/NoTruncation/Depth1/Value-16         194.6n ± 1%   180.9n ±  1%   -7.04% (p=0.000 n=10)
TruncateValue/Nested/NoTruncation/Depth1/KeyValue-16      197.0n ± 1%   182.7n ±  0%   -7.26% (p=0.000 n=10)
TruncateValue/Nested/Truncation/Depth1/Value-16           587.9n ± 1%   506.9n ±  1%  -13.78% (p=0.000 n=10)
TruncateValue/Nested/Truncation/Depth1/KeyValue-16        594.7n ± 1%   514.0n ±  1%  -13.56% (p=0.000 n=10)
TruncateValue/Nested/NoTruncation/Depth8/Value-16        1064.5n ± 0%   864.0n ±  1%  -18.83% (p=0.000 n=10)
TruncateValue/Nested/NoTruncation/Depth8/KeyValue-16     1077.0n ± 1%   892.5n ±  0%  -17.13% (p=0.000 n=10)
TruncateValue/Nested/Truncation/Depth8/Value-16           7.531µ ± 2%   2.999µ ±  1%  -60.18% (p=0.000 n=10)
TruncateValue/Nested/Truncation/Depth8/KeyValue-16        7.591µ ± 0%   3.026µ ±  1%  -60.14% (p=0.000 n=10)
TruncateValue/Nested/NoTruncation/Depth32/Value-16        4.158µ ± 2%   3.467µ ±  1%  -16.61% (p=0.000 n=10)
TruncateValue/Nested/NoTruncation/Depth32/KeyValue-16     4.173µ ± 1%   3.518µ ±  1%  -15.71% (p=0.000 n=10)
TruncateValue/Nested/Truncation/Depth32/Value-16          79.84µ ± 2%   11.75µ ±  0%  -85.28% (p=0.000 n=10)
TruncateValue/Nested/Truncation/Depth32/KeyValue-16       80.43µ ± 1%   11.83µ ±  1%  -85.29% (p=0.000 n=10)
geomean                                                   456.7n        324.8n        -28.88%

                                                       │    base.txt     │                 head.txt                  │
                                                       │      B/op       │     B/op      vs base                     │
TruncateValue/String/NoTruncation/Value-16                  0.000 ± 0%       0.000 ± 0%         ~ (p=1.000 n=10) ¹
TruncateValue/String/NoTruncation/KeyValue-16               0.000 ± 0%       0.000 ± 0%         ~ (p=1.000 n=10) ¹
TruncateValue/ByteSlice/NoTruncation/Value-16               0.000 ± 0%       0.000 ± 0%         ~ (p=1.000 n=10) ¹
TruncateValue/ByteSlice/NoTruncation/KeyValue-16            0.000 ± 0%       0.000 ± 0%         ~ (p=1.000 n=10) ¹
TruncateValue/StringSlice/NoTruncation/Value-16             64.00 ± 0%        0.00 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/StringSlice/NoTruncation/KeyValue-16          64.00 ± 0%        0.00 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/StringSlice/Truncation/Value-16               64.00 ± 0%       64.00 ± 0%         ~ (p=1.000 n=10) ¹
TruncateValue/StringSlice/Truncation/KeyValue-16            64.00 ± 0%       64.00 ± 0%         ~ (p=1.000 n=10) ¹
TruncateValue/Slice/NoTruncation/Value-16                   96.00 ± 0%        0.00 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/Slice/NoTruncation/KeyValue-16                96.00 ± 0%        0.00 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/Slice/Truncation/Value-16                     288.0 ± 0%       192.0 ± 0%   -33.33% (p=0.000 n=10)
TruncateValue/Slice/Truncation/KeyValue-16                  288.0 ± 0%       192.0 ± 0%   -33.33% (p=0.000 n=10)
TruncateValue/Map/NoTruncation/Value-16                     128.0 ± 0%         0.0 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/Map/NoTruncation/KeyValue-16                  128.0 ± 0%         0.0 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/Map/Truncation/Value-16                       384.0 ± 0%       256.0 ± 0%   -33.33% (p=0.000 n=10)
TruncateValue/Map/Truncation/KeyValue-16                    384.0 ± 0%       256.0 ± 0%   -33.33% (p=0.000 n=10)
TruncateValue/StringSliceWide/NoTruncation/Value-16         288.0 ± 0%         0.0 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/StringSliceWide/NoTruncation/KeyValue-16      288.0 ± 0%         0.0 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/SliceWide/NoTruncation/Value-16               288.0 ± 0%         0.0 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/SliceWide/NoTruncation/KeyValue-16            288.0 ± 0%         0.0 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/MapWide/NoTruncation/Value-16                 384.0 ± 0%         0.0 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/MapWide/NoTruncation/KeyValue-16              384.0 ± 0%         0.0 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/Nested/NoTruncation/Depth1/Value-16           128.0 ± 0%         0.0 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/Nested/NoTruncation/Depth1/KeyValue-16        128.0 ± 0%         0.0 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/Nested/Truncation/Depth1/Value-16             384.0 ± 0%       256.0 ± 0%   -33.33% (p=0.000 n=10)
TruncateValue/Nested/Truncation/Depth1/KeyValue-16          384.0 ± 0%       256.0 ± 0%   -33.33% (p=0.000 n=10)
TruncateValue/Nested/NoTruncation/Depth8/Value-16           928.0 ± 0%         0.0 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/Nested/NoTruncation/Depth8/KeyValue-16        928.0 ± 0%         0.0 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/Nested/Truncation/Depth8/Value-16           5.938Ki ± 0%     1.812Ki ± 0%   -69.47% (p=0.000 n=10)
TruncateValue/Nested/Truncation/Depth8/KeyValue-16        5.938Ki ± 0%     1.812Ki ± 0%   -69.47% (p=0.000 n=10)
TruncateValue/Nested/NoTruncation/Depth32/Value-16        3.531Ki ± 0%     0.000Ki ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/Nested/NoTruncation/Depth32/KeyValue-16     3.531Ki ± 0%     0.000Ki ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/Nested/Truncation/Depth32/Value-16         65.562Ki ± 0%     7.062Ki ± 0%   -89.23% (p=0.000 n=10)
TruncateValue/Nested/Truncation/Depth32/KeyValue-16      65.562Ki ± 0%     7.062Ki ± 0%   -89.23% (p=0.000 n=10)
geomean                                                                ²                 ?                       ² ³
¹ all samples are equal
² summaries must be >0 to compute geomean
³ ratios must be >0 to compute geomean

                                                       │   base.txt    │                head.txt                 │
                                                       │   allocs/op   │ allocs/op   vs base                     │
TruncateValue/String/NoTruncation/Value-16                0.000 ± 0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
TruncateValue/String/NoTruncation/KeyValue-16             0.000 ± 0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
TruncateValue/ByteSlice/NoTruncation/Value-16             0.000 ± 0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
TruncateValue/ByteSlice/NoTruncation/KeyValue-16          0.000 ± 0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
TruncateValue/StringSlice/NoTruncation/Value-16           2.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/StringSlice/NoTruncation/KeyValue-16        2.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/StringSlice/Truncation/Value-16             2.000 ± 0%     2.000 ± 0%         ~ (p=1.000 n=10) ¹
TruncateValue/StringSlice/Truncation/KeyValue-16          2.000 ± 0%     2.000 ± 0%         ~ (p=1.000 n=10) ¹
TruncateValue/Slice/NoTruncation/Value-16                 1.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/Slice/NoTruncation/KeyValue-16              1.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/Slice/Truncation/Value-16                   3.000 ± 0%     2.000 ± 0%   -33.33% (p=0.000 n=10)
TruncateValue/Slice/Truncation/KeyValue-16                3.000 ± 0%     2.000 ± 0%   -33.33% (p=0.000 n=10)
TruncateValue/Map/NoTruncation/Value-16                   1.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/Map/NoTruncation/KeyValue-16                1.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/Map/Truncation/Value-16                     3.000 ± 0%     2.000 ± 0%   -33.33% (p=0.000 n=10)
TruncateValue/Map/Truncation/KeyValue-16                  3.000 ± 0%     2.000 ± 0%   -33.33% (p=0.000 n=10)
TruncateValue/StringSliceWide/NoTruncation/Value-16       3.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/StringSliceWide/NoTruncation/KeyValue-16    3.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/SliceWide/NoTruncation/Value-16             1.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/SliceWide/NoTruncation/KeyValue-16          1.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/MapWide/NoTruncation/Value-16               1.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/MapWide/NoTruncation/KeyValue-16            1.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/Nested/NoTruncation/Depth1/Value-16         2.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/Nested/NoTruncation/Depth1/KeyValue-16      2.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/Nested/Truncation/Depth1/Value-16           6.000 ± 0%     4.000 ± 0%   -33.33% (p=0.000 n=10)
TruncateValue/Nested/Truncation/Depth1/KeyValue-16        6.000 ± 0%     4.000 ± 0%   -33.33% (p=0.000 n=10)
TruncateValue/Nested/NoTruncation/Depth8/Value-16         9.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/Nested/NoTruncation/Depth8/KeyValue-16      9.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/Nested/Truncation/Depth8/Value-16           62.00 ± 0%     18.00 ± 0%   -70.97% (p=0.000 n=10)
TruncateValue/Nested/Truncation/Depth8/KeyValue-16        62.00 ± 0%     18.00 ± 0%   -70.97% (p=0.000 n=10)
TruncateValue/Nested/NoTruncation/Depth32/Value-16        33.00 ± 0%      0.00 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/Nested/NoTruncation/Depth32/KeyValue-16     33.00 ± 0%      0.00 ± 0%  -100.00% (p=0.000 n=10)
TruncateValue/Nested/Truncation/Depth32/Value-16         626.00 ± 0%     66.00 ± 0%   -89.46% (p=0.000 n=10)
TruncateValue/Nested/Truncation/Depth32/KeyValue-16      626.00 ± 0%     66.00 ± 0%   -89.46% (p=0.000 n=10)
geomean                                                              ²               ?                       ² ³
¹ all samples are equal
² summaries must be >0 to compute geomean
³ ratios must be >0 to compute geomean

@codecov

codecov Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.6%. Comparing base (f081842) to head (ae268e8).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@          Coverage Diff           @@
##            main   #8912    +/-   ##
======================================
  Coverage   88.5%   88.6%            
======================================
  Files        331     333     +2     
  Lines      21090   21283   +193     
======================================
+ Hits       18681   18869   +188     
- Misses      2409    2414     +5     
Files with missing lines Coverage Δ
sdk/internal/attrnorm/truncate.go 100.0% <100.0%> (ø)
sdk/log/internal/attrnorm/truncate.go 100.0% <100.0%> (ø)

... and 36 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@pellared
pellared marked this pull request as ready for review September 1, 2026 11:22
Comment thread CHANGELOG.md Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Optimize truncation in Trace and Logs SDK

2 participants