[#33567] DocDB: Escape non-ASCII bytes in Slice::ToDebugString regardless of locale - #33568
Open
ellabaron-code wants to merge 1 commit into
Open
[#33567] DocDB: Escape non-ASCII bytes in Slice::ToDebugString regardless of locale#33568ellabaron-code wants to merge 1 commit into
ellabaron-code wants to merge 1 commit into
Conversation
On macOS in a UTF-8 locale isgraph() accepts bytes >= 0xA1, so binary fields (e.g. transaction ids) rendered via ToDebugString went into logs unescaped. Pin the C locale so non-ASCII bytes are always escaped; Linux output is unchanged. Extend SliceTest.DebugStringLength to run under a UTF-8 locale and assert the output contains no raw bytes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
isgraph()is locale-dependent: on macOS in a UTF-8 locale it accepts bytes >= 0xA1, so binary fields rendered viaSlice::ToDebugString(e.g. transaction ids in lightweight-protobuf debug output) went into logs as raw unescaped bytes, and the non-graph count often stayed under the--non_graph_characters_percentage_to_use_hexadecimal_renderingthreshold. glibc rejects these bytes, so Linux was unaffected.Evaluate
isgraph()in the classic C locale at both call sites so rendering is locale-independent; non-ASCII bytes are always escaped. Linux output is unchanged.The existing
SliceTest.DebugStringLengthis extended (and renamed toDebugStringLengthAndLocale) to run under a UTF-8 locale and assert the rendered output contains only printable ASCII — tests run in the default C locale, so without pinning the locale the regression check would pass against the unfixed code.Fixes #33567
Test plan
SliceTest.DebugStringLengthAndLocalefails against the unfixed code on macOS (raw bytes in output) and passes with the fix, on both macOS (arm64) and Linux (x86_64).RemoteBootstrapITestlogs on macOS contain no raw binary after the fix (LC_ALL=C grep -rl '[^[:print:][:space:]]'finds nothing); Linux log output unchanged.