Skip to content

Commit 04c278e

Browse files
authored
perf: skip SDK tracing without span processors (#402) (#430)
Preserve an external provider installed before bootstrap. Otherwise pass a local NoOpTracerProvider to framework and custom instrumentors without consuming the global registration slot. Keep SDK providers for configured span processors and leave external provider shutdown to its owner. Metrics-only and external-provider warning regressions failed first. Both OTLP exporter protocols still record and export. FastAPI/Litestar HTTP tests verify metrics collection and provider setup before/after bootstrap. Local Python 3.14.7 metrics-only ASGI benchmark: 500 warmup requests, seven rounds of 5000 requests, median elapsed request time. Metrics used ConsoleMetricExporter with an in-memory output stream to exclude network I/O. Logging, Prometheus and health routes disabled. Same harness and environment before/after: - FastAPI: 148.83 -> 108.39 us/request - Litestar: 108.17 -> 77.70 us/request Validation: 628 tests passed with 100% coverage; lint, ty and strict docs build passed. Standards and spec reviews found no remaining issues.
1 parent c3c0053 commit 04c278e

9 files changed

Lines changed: 251 additions & 25 deletions

File tree

‎docs/introduction/configuration.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -232,6 +232,28 @@ Additional parameters:
232232
- `opentelemetry_excluded_urls` - extra regexes for URLs excluded from tracing; the metrics path and (unless health-check spans are enabled) the health-check path are excluded automatically. Litestar searches them in the request path, FastAPI and FastMCP in the full URL. FastAPI and FastMCP raise `ConfigurationError` at config construction if an entry contains a comma because OpenTelemetry uses commas to separate patterns. Litestar accepts commas in config entries. The environment variable OpenTelemetry defines for the framework (`OTEL_PYTHON_LITESTAR_EXCLUDED_URLS`, `OTEL_PYTHON_FASTAPI_EXCLUDED_URLS` or `OTEL_PYTHON_STARLETTE_EXCLUDED_URLS` for FastMCP), falling back to `OTEL_PYTHON_EXCLUDED_URLS`, adds to them.
233233
- `opentelemetry_metrics_endpoint` - will be passed to `OTLPMetricExporter` as endpoint, and turns on the metrics signal. Its own field rather than a flag on `opentelemetry_endpoint`, so upgrading never starts exporting metrics you did not ask for. Under `opentelemetry_exporter_protocol="http"` this is a full URL (e.g. `http://collector:4318/v1/metrics`); `opentelemetry_exporter_protocol` and `opentelemetry_insecure` are shared with traces.
234234

235+
### Tracing without span processors
236+
237+
lite-bootstrap installs an SDK `TracerProvider` only when it adds a span processor:
238+
Pyroscope span correlation, `opentelemetry_log_traces=True`, or a trace endpoint
239+
with its exporter installed. Metrics-only configuration does not create an SDK
240+
tracer provider. A trace endpoint with a missing exporter still warns, but uses
241+
no-op tracing if no other span processor is configured.
242+
243+
Without span processors, framework and custom instrumentors receive a local
244+
`NoOpTracerProvider` unless an external tracer provider is already installed.
245+
An existing external provider is preserved and is never shut down by lite-bootstrap.
246+
Install an external provider before bootstrap if it should record framework spans.
247+
The no-op path leaves the global provider registration slot available, but installing
248+
an external provider afterward does not replace the no-op provider already passed
249+
to framework and custom instrumentors.
250+
251+
Code that previously added a span processor to `get_tracer_provider()` after a
252+
metrics-only bootstrap must instead create and install its own SDK provider before
253+
bootstrap. For metrics-only requests without external tracing, spans do not record
254+
and JSON logs omit `tracing`; there are no `span_id` or `trace_id` values to inject.
255+
Metrics collection is unchanged.
256+
235257
### Metrics
236258

237259
Left unset, no `MeterProvider` is installed and the metrics signal stays off. Set it, and the

‎docs/migration/to-2.x.md‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,3 +43,19 @@ warnings.filterwarnings("error", category=InstrumentDependencyMissingWarning)
4343

4444
Unconfigured instruments continue to be skipped silently. A configured instrument
4545
with a missing optional dependency emits `InstrumentDependencyMissingWarning`.
46+
47+
## Metrics-only OpenTelemetry tracing
48+
49+
Metrics-only configuration no longer installs an SDK `TracerProvider` or records
50+
unexported framework spans. The same applies when a trace endpoint's exporter is
51+
missing and no other span processor is configured; the missing-exporter warning remains.
52+
JSON logs from these requests omit `tracing` unless external tracing records a span.
53+
54+
If you add span processors yourself, create an SDK tracer provider, attach your
55+
processors, and install it with `opentelemetry.trace.set_tracer_provider()` before
56+
bootstrap. lite-bootstrap preserves that external provider and leaves its shutdown
57+
to your application. Installing a provider after bootstrap does not activate tracing
58+
for framework and custom instrumentors already given a no-op provider.
59+
60+
Configurations with a working trace exporter, stdout span logging, or Pyroscope
61+
span correlation continue to use an SDK tracer provider.

‎lite_bootstrap/bootstrappers/fastapi_bootstrapper.py‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,6 @@
4949

5050
if import_checker.is_opentelemetry_installed:
5151
from opentelemetry.metrics import get_meter_provider
52-
from opentelemetry.trace import get_tracer_provider
5352

5453
if import_checker.is_fastapi_opentelemetry_installed:
5554
from opentelemetry.instrumentation.asgi import OpenTelemetryMiddleware
@@ -213,7 +212,7 @@ def bootstrap(self) -> None:
213212
super().bootstrap()
214213
FastAPIInstrumentor.instrument_app(
215214
app=self.bootstrap_config.app,
216-
tracer_provider=get_tracer_provider(),
215+
tracer_provider=self.tracer_provider,
217216
meter_provider=get_meter_provider(),
218217
excluded_urls=",".join(self._build_excluded_url_patterns("FASTAPI")),
219218
exclude_spans=self.bootstrap_config.opentelemetry_exclude_spans,

‎lite_bootstrap/bootstrappers/fastmcp_bootstrapper.py‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,6 @@
3131

3232
if import_checker.is_fastmcp_opentelemetry_installed:
3333
from opentelemetry.metrics import get_meter_provider
34-
from opentelemetry.trace import get_tracer_provider
3534
from starlette.routing import BaseRoute, Match, Mount, Route
3635
from starlette.types import Scope
3736

@@ -241,7 +240,7 @@ def _instrument_http_app(self, http_application: "StarletteWithLifespan") -> "St
241240
),
242241
excluded_urls=",".join(self._build_excluded_url_patterns("STARLETTE")),
243242
exclude_spans=self.bootstrap_config.opentelemetry_exclude_spans,
244-
tracer_provider=get_tracer_provider(),
243+
tracer_provider=self.tracer_provider,
245244
meter_provider=get_meter_provider(),
246245
)
247246
setattr(http_application, _OPENTELEMETRY_INSTRUMENTED_MARKER, True)

‎lite_bootstrap/bootstrappers/faststream_bootstrapper.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ async def after_processed(
4949
if import_checker.is_opentelemetry_installed:
5050
from opentelemetry import trace
5151
from opentelemetry.metrics import Meter, MeterProvider, get_meter_provider
52-
from opentelemetry.trace import TracerProvider, get_tracer_provider
52+
from opentelemetry.trace import TracerProvider
5353

5454
tracer: typing.Final = trace.get_tracer(__name__)
5555

@@ -227,7 +227,7 @@ def bootstrap(self) -> None:
227227
config.application.brokers, config.opentelemetry_middleware_cls, "opentelemetry_middleware_cls"
228228
):
229229
broker.add_middleware(
230-
middleware_cls(tracer_provider=get_tracer_provider(), meter_provider=get_meter_provider())
230+
middleware_cls(tracer_provider=self.tracer_provider, meter_provider=get_meter_provider())
231231
)
232232

233233

‎lite_bootstrap/bootstrappers/litestar_bootstrapper.py‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,6 @@
7878

7979
if import_checker.is_opentelemetry_installed:
8080
from opentelemetry.metrics import get_meter_provider
81-
from opentelemetry.trace import get_tracer_provider
8281

8382
if import_checker.is_structlog_installed:
8483
import structlog
@@ -382,7 +381,7 @@ def bootstrap(self) -> None:
382381
application_config.plugins.append(
383382
OpenTelemetryPlugin(
384383
LitestarOpenTelemetryConfig(
385-
tracer_provider=get_tracer_provider(),
384+
tracer_provider=self.tracer_provider,
386385
meter_provider=get_meter_provider(),
387386
scope_span_details_extractor=build_litestar_route_details_from_scope,
388387
# An empty list compiles to a pattern that matches, and so skips, every path

‎lite_bootstrap/instruments/opentelemetry_instrument.py‎

Lines changed: 45 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,15 @@
2727
SimpleSpanProcessor,
2828
SpanExporter,
2929
)
30-
from opentelemetry.trace import Span, format_span_id, get_tracer_provider, set_tracer_provider
30+
from opentelemetry.trace import (
31+
NoOpTracerProvider,
32+
ProxyTracerProvider,
33+
Span,
34+
format_span_id,
35+
get_tracer_provider,
36+
set_tracer_provider,
37+
)
38+
from opentelemetry.trace import TracerProvider as APITracerProvider
3139

3240
if import_checker.is_otlp_grpc_exporter_installed:
3341
# opentelemetry-api can be present without the grpc otlp exporter package (e.g.
@@ -177,7 +185,11 @@ def on_end(self, span: "ReadableSpan") -> None:
177185
class OpenTelemetryInstrument(BaseInstrument[OpenTelemetryConfigT]):
178186
"""OpenTelemetry tracing instrument.
179187
180-
Lifecycle note: ``bootstrap()`` calls ``opentelemetry.trace.set_tracer_provider``,
188+
Without span processors, framework and custom instrumentors use an existing external
189+
provider or a local no-op provider. The global registration slot stays untouched.
190+
191+
Lifecycle note: when span processors are configured, ``bootstrap()`` calls
192+
``opentelemetry.trace.set_tracer_provider``,
181193
which the OTel SDK enforces as **set-once per process** (subsequent calls log
182194
"Overriding of current TracerProvider is not allowed" and have no effect).
183195
``teardown()`` calls ``shutdown()`` on the provider, which flushes batched
@@ -194,6 +206,7 @@ class OpenTelemetryInstrument(BaseInstrument[OpenTelemetryConfigT]):
194206
_tracer_provider: "TracerProvider | None" = dataclasses.field(
195207
default_factory=lambda: None, init=False, repr=False, compare=False
196208
)
209+
tracer_provider: "APITracerProvider" = dataclasses.field(init=False, repr=False, compare=False)
197210
_meter_provider: "MeterProvider | None" = dataclasses.field(
198211
default_factory=lambda: None, init=False, repr=False, compare=False
199212
)
@@ -337,7 +350,7 @@ def _bootstrap_metrics(self, resource: "resources.Resource") -> None:
337350
)
338351
self._meter_provider = meter_provider
339352

340-
def _apply_instrumentors(self, tracer_provider: "TracerProvider") -> None:
353+
def _apply_instrumentors(self, tracer_provider: "APITracerProvider") -> None:
341354
for one_instrumentor in self.bootstrap_config.opentelemetry_instrumentors:
342355
if isinstance(one_instrumentor, InstrumentorWithParams):
343356
one_instrumentor.instrumentor.instrument(
@@ -347,25 +360,41 @@ def _apply_instrumentors(self, tracer_provider: "TracerProvider") -> None:
347360
else:
348361
one_instrumentor.instrument(tracer_provider=tracer_provider)
349362

363+
def _build_span_processors(self) -> "list[SpanProcessor]":
364+
config = self.bootstrap_config
365+
processors: list[SpanProcessor] = []
366+
if import_checker.is_pyroscope_installed and getattr(config, "pyroscope_endpoint", None):
367+
processors.append(PyroscopeSpanProcessor())
368+
if config.opentelemetry_log_traces:
369+
processors.append(SimpleSpanProcessor(ConsoleSpanExporter(formatter=_format_span)))
370+
if config.opentelemetry_endpoint and (span_exporter := self._build_span_exporter()):
371+
processors.append(BatchSpanProcessor(span_exporter))
372+
return processors
373+
350374
def bootstrap(self) -> None:
351375
config = self.bootstrap_config
352376
self._silence_otel_loggers()
353377
resource = self._build_resource()
354-
tracer_provider = TracerProvider(resource=resource, sampler=config.opentelemetry_sampler)
355-
set_tracer_provider(tracer_provider)
356-
if get_tracer_provider() is not tracer_provider:
357-
warn_at_caller(
358-
"a TracerProvider is already installed; the configured exporter, sampler and resource will not be used"
378+
processors = self._build_span_processors()
379+
if processors:
380+
tracer_provider = TracerProvider(resource=resource, sampler=config.opentelemetry_sampler)
381+
self._tracer_provider = tracer_provider
382+
for processor in processors:
383+
tracer_provider.add_span_processor(processor)
384+
set_tracer_provider(tracer_provider)
385+
if get_tracer_provider() is not tracer_provider:
386+
warn_at_caller(
387+
"a TracerProvider is already installed; "
388+
"the configured exporter, sampler and resource will not be used"
389+
)
390+
self.tracer_provider = get_tracer_provider()
391+
else:
392+
existing_provider = get_tracer_provider()
393+
self.tracer_provider = (
394+
NoOpTracerProvider() if isinstance(existing_provider, ProxyTracerProvider) else existing_provider
359395
)
360-
self._tracer_provider = tracer_provider
361-
if import_checker.is_pyroscope_installed and getattr(config, "pyroscope_endpoint", None):
362-
tracer_provider.add_span_processor(PyroscopeSpanProcessor())
363-
if config.opentelemetry_log_traces:
364-
tracer_provider.add_span_processor(SimpleSpanProcessor(ConsoleSpanExporter(formatter=_format_span)))
365-
if config.opentelemetry_endpoint and (span_exporter := self._build_span_exporter()):
366-
tracer_provider.add_span_processor(BatchSpanProcessor(span_exporter))
367396
self._bootstrap_metrics(resource)
368-
self._apply_instrumentors(tracer_provider)
397+
self._apply_instrumentors(self.tracer_provider)
369398

370399
def teardown(self) -> None:
371400
with collect_teardown_errors() as teardown_errors:

‎tests/instruments/test_opentelemetry_instrument.py‎

Lines changed: 97 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88
from unittest.mock import patch
99

1010
import pytest
11+
from opentelemetry import trace
1112
from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter as OTLPGrpcMetricExporter
1213
from opentelemetry.exporter.otlp.proto.http.metric_exporter import OTLPMetricExporter as OTLPHttpMetricExporter
1314
from opentelemetry.instrumentation.instrumentor import BaseInstrumentor
@@ -16,6 +17,8 @@
1617
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
1718
from opentelemetry.sdk.trace import TracerProvider as SDKTracerProvider
1819
from opentelemetry.sdk.trace import sampling
20+
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
21+
from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter
1922
from opentelemetry.trace import set_tracer_provider
2023

2124
import lite_bootstrap.instruments.opentelemetry_instrument as otel_module
@@ -26,7 +29,12 @@
2629
OpenTelemetryConfig,
2730
OpenTelemetryInstrument,
2831
)
29-
from tests.conftest import CustomInstrumentor, emulate_package_missing_with_module_reload, warning_source_files
32+
from tests.conftest import (
33+
CustomInstrumentor,
34+
emulate_package_missing_with_module_reload,
35+
in_memory_opentelemetry_metric_reader,
36+
warning_source_files,
37+
)
3038

3139

3240
def test_opentelemetry_instrument() -> None:
@@ -716,3 +724,91 @@ def test_bootstrap_warns_when_a_meter_provider_is_already_installed() -> None:
716724
"a MeterProvider is already installed; the configured metrics exporter and resource will not be used"
717725
]
718726
assert warning_source_files(caught, UserWarning) == [__file__]
727+
728+
729+
class TracerCapturingInstrumentor(BaseInstrumentor):
730+
provider: trace.TracerProvider
731+
732+
def instrumentation_dependencies(self) -> typing.Collection[str]:
733+
return []
734+
735+
def _instrument(self, **kwargs: object) -> None:
736+
self.provider = typing.cast("trace.TracerProvider", kwargs["tracer_provider"])
737+
738+
def _uninstrument(self, **kwargs: object) -> None:
739+
pass
740+
741+
742+
def test_metrics_only_tracing_is_noop_without_installing_an_sdk_provider() -> None:
743+
custom = TracerCapturingInstrumentor()
744+
instrument = OpenTelemetryInstrument(
745+
bootstrap_config=OpenTelemetryConfig(
746+
opentelemetry_metrics_endpoint="localhost:4317",
747+
opentelemetry_instrumentors=[custom],
748+
)
749+
)
750+
with in_memory_opentelemetry_metric_reader():
751+
try:
752+
instrument.bootstrap()
753+
assert not isinstance(trace.get_tracer_provider(), SDKTracerProvider)
754+
with custom.provider.get_tracer(__name__).start_as_current_span("discarded") as span:
755+
assert not span.is_recording()
756+
assert isinstance(get_meter_provider(), SDKMeterProvider)
757+
finally:
758+
instrument.teardown()
759+
760+
761+
def test_metrics_only_tracing_preserves_an_external_provider_and_its_lifetime() -> None:
762+
external = SDKTracerProvider()
763+
exporter = InMemorySpanExporter()
764+
external.add_span_processor(SimpleSpanProcessor(exporter))
765+
trace.set_tracer_provider(external)
766+
custom = TracerCapturingInstrumentor()
767+
instrument = OpenTelemetryInstrument(
768+
bootstrap_config=OpenTelemetryConfig(
769+
opentelemetry_metrics_endpoint="localhost:4317",
770+
opentelemetry_instrumentors=[custom],
771+
)
772+
)
773+
try:
774+
with in_memory_opentelemetry_metric_reader(), warnings.catch_warnings(record=True) as caught:
775+
warnings.simplefilter("always")
776+
instrument.bootstrap()
777+
assert not caught
778+
assert custom.provider is external
779+
with custom.provider.get_tracer(__name__).start_as_current_span("external"):
780+
pass
781+
instrument.teardown()
782+
with external.get_tracer(__name__).start_as_current_span("after-teardown"):
783+
pass
784+
assert trace.get_tracer_provider() is external
785+
assert [span.name for span in exporter.get_finished_spans()] == ["external", "after-teardown"]
786+
finally:
787+
instrument.teardown()
788+
external.shutdown()
789+
790+
791+
@pytest.mark.parametrize("protocol", ["grpc", "http"])
792+
def test_configured_exporter_still_records_and_exports_spans(protocol: typing.Literal["grpc", "http"]) -> None:
793+
exporter = InMemorySpanExporter()
794+
custom = TracerCapturingInstrumentor()
795+
instrument = OpenTelemetryInstrument(
796+
bootstrap_config=OpenTelemetryConfig(
797+
opentelemetry_endpoint="localhost:4317",
798+
opentelemetry_exporter_protocol=protocol,
799+
opentelemetry_instrumentors=[custom],
800+
)
801+
)
802+
exporter_name = "OTLPGrpcSpanExporter" if protocol == "grpc" else "OTLPHttpSpanExporter"
803+
try:
804+
with patch.object(otel_module, exporter_name, return_value=exporter):
805+
instrument.bootstrap()
806+
provider = trace.get_tracer_provider()
807+
assert isinstance(provider, SDKTracerProvider)
808+
assert custom.provider is provider
809+
with custom.provider.get_tracer(__name__).start_as_current_span("exported") as span:
810+
assert span.is_recording()
811+
provider.force_flush()
812+
assert [span.name for span in exporter.get_finished_spans()] == ["exported"]
813+
finally:
814+
instrument.teardown()

0 commit comments

Comments
 (0)