Skip to content

Commit 6c4e07b

Browse files
authored
fix: emit one physical line per rendered structlog event (#194)
* fix: emit one physical line per rendered structlog event `MemoryLoggerFactory` attached a bare `StreamHandler`, so the default `logging.Formatter` appended `record.exc_text` after the rendered JSON. `logging.Logger.exception` hardcodes `exc_info=True`, so every `structlog.get_logger().exception(...)` shipped the traceback twice. Give that handler a formatter that emits the rendered line verbatim. Closes #193 * docs: name the rendered line and record why wrap_for_formatter is out `CONTEXT.md` had no word for the producer's JSON string, so the invariant added in the previous commit asserts a relationship between two concepts the glossary does not name. ADR-0008 gains the rejected alternative, since `wrap_for_formatter` is that ADR's constraint biting a second time rather than a new decision.
1 parent 1694963 commit 6c4e07b

4 files changed

Lines changed: 80 additions & 0 deletions

File tree

‎CONTEXT.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,3 +50,10 @@ Without a web framework — `FreeBootstrapper`, `FreeConfig`, the `free-all` ext
5050
application, so nothing to attach teardown to and no double-bootstrap guard.
5151
_Avoid_: bare "free" for PEP 703 free-threaded CPython, which this project also supports. Write
5252
"free-threaded" in full.
53+
54+
**Rendered line**:
55+
The complete JSON string the logging instrument produces for one event — message, level, metadata
56+
and traceback together. One event is one rendered line is one physical line of stdout; the three
57+
never diverge.
58+
_Avoid_: formatted message — reserve that for Sentry's own `logentry.formatted` field, which holds
59+
a rendered line only until the seam lifts the message out of it.

‎docs/adr/0008-structlog-sentry-seam-stays-a-heuristic.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,16 @@ drift impossible, and would change the emitted log shape for every user. The del
3131
top-level meta-processor whose key is not added to `STRUCTLOG_META_KEYS` still leaks, and that is a
3232
known, accepted limit.
3333

34+
**Deferring the render to `ProcessorFormatter.wrap_for_formatter`.** This is structlog's own
35+
idiomatic stdlib integration, it is already what `_configure_foreign_loggers` uses ten lines away,
36+
and it was the obvious fix for #193's double-rendered traceback. It is incompatible with this seam.
37+
`wrap_for_formatter` moves rendering to handler-flush time, so `record.msg` is the `EventDict`
38+
rather than a rendered line; `sentry_sdk` builds `logentry.formatted` from `record.getMessage()`,
39+
which for a dict `msg` is a Python repr. That repr opens with `{`, so `StructuredLogPayload.parse`
40+
accepts it, fails to decode it, and returns `None` — `skip_sentry` stops being honoured and
41+
`contexts.structlog` stops being attached, with nothing failing. The seam requires that the line
42+
be rendered before it becomes a `LogRecord`; anything downstream of the chain is transport only.
43+
3444
`IGNORED_STRUCTLOG_ATTRIBUTES` survives in `sentry_instrument.py` as a silent alias of
3545
`STRUCTLOG_META_KEYS` for external importers of the old name.
3646

‎lite_bootstrap/instruments/logging_factory.py‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,13 @@ def parse(cls, formatted: str) -> "StructuredLogPayload | None":
9696
return cls(message=message, extra=extra, skip_sentry=skip_sentry)
9797

9898

99+
class _RenderedLineFormatter(logging.Formatter):
100+
"""Emit the rendered line verbatim; stdlib appends nothing after it."""
101+
102+
def format(self, record: logging.LogRecord) -> str:
103+
return record.getMessage()
104+
105+
99106
if import_checker.is_structlog_installed:
100107
import structlog
101108

@@ -113,6 +120,7 @@ def __init__(
113120
def __call__(self, *args: typing.Any) -> logging.Logger: # noqa: ANN401
114121
logger: typing.Final = super().__call__(*args)
115122
stream_handler: typing.Final = logging.StreamHandler(stream=self.config.log_stream)
123+
stream_handler.setFormatter(_RenderedLineFormatter())
116124
handler: typing.Final = logging.handlers.MemoryHandler(
117125
capacity=self.config.logging_buffer_capacity,
118126
flushLevel=self.config.logging_flush_level,

‎tests/instruments/test_logging_instrument.py‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
import contextlib
2+
import json
23
import logging
34
from io import StringIO
45
from unittest.mock import patch
@@ -227,3 +228,57 @@ def test_logging_instrument_binds_log_stream_at_bootstrap() -> None:
227228
logging_instrument.teardown()
228229

229230
assert "bound at bootstrap" in redirected_stdout.getvalue()
231+
232+
233+
def test_one_rendered_event_is_one_physical_line(capsys: pytest.CaptureFixture[str]) -> None:
234+
"""INVARIANT: a structlog event reaches stdout as exactly one physical line.
235+
236+
The processor chain renders the whole event — traceback included, under the `exception` key —
237+
before the string ever becomes a `LogRecord`. Everything downstream of that is transport, so
238+
the handler must emit the rendered string verbatim.
239+
240+
Dropping the formatter from the stream handler `MemoryLoggerFactory` builds is what breaks it,
241+
and the break is invisible until something raises: stdlib's `Logger.exception` hardcodes
242+
`exc_info=True` on the record, so the default `logging.Formatter` appends `record.exc_text`
243+
after the JSON and the same traceback ships twice. Every line-oriented log shipper reads the
244+
suffix as four malformed events.
245+
"""
246+
logging_instrument = LoggingInstrument(bootstrap_config=LoggingConfig(logging_buffer_capacity=0))
247+
try:
248+
logging_instrument.bootstrap()
249+
logger = structlog.getLogger("one_physical_line")
250+
try:
251+
msg = "some error"
252+
raise ValueError(msg) # noqa: TRY301
253+
except ValueError:
254+
logger.exception("logging error")
255+
finally:
256+
logging_instrument.teardown()
257+
258+
lines = capsys.readouterr().out.splitlines()
259+
260+
assert len(lines) == 1, lines
261+
payload = json.loads(lines[0])
262+
assert payload["event"] == "logging error"
263+
assert "ValueError: some error" in payload["exception"]
264+
265+
266+
def test_foreign_logger_exception_is_one_physical_line(capsys: pytest.CaptureFixture[str]) -> None:
267+
"""The same one-line contract holds for a plain stdlib logger going through the root handler."""
268+
logging_instrument = LoggingInstrument(bootstrap_config=LoggingConfig(logging_buffer_capacity=0))
269+
try:
270+
logging_instrument.bootstrap()
271+
try:
272+
msg = "some error"
273+
raise ValueError(msg) # noqa: TRY301
274+
except ValueError:
275+
logging.getLogger("foreign_one_physical_line").exception("logging error")
276+
finally:
277+
logging_instrument.teardown()
278+
279+
lines = capsys.readouterr().out.splitlines()
280+
281+
assert len(lines) == 1, lines
282+
payload = json.loads(lines[0])
283+
assert payload["event"] == "logging error"
284+
assert "ValueError: some error" in payload["exception"]

0 commit comments

Comments
 (0)