feat: add `opentelemetry-otlp-file` crate implementing the OTLP File Exporter by garysassano · Pull Request #3763 · open-telemetry/opentelemetry-rust

garysassano · 2026-09-28T23:46:32Z

Closes #2602
Design discussion issue (if applicable) #2602

Changes

Adds a new crate, opentelemetry-otlp-file, that implements the OTLP File Exporter specification (Development status) for traces, metrics and logs.

Format: SpanExporter, MetricExporter and LogExporter write each export as one line of compact OTLP JSON: a TracesData, MetricsData or LogsData object followed by \n.
Destinations: output defaults to stdout, as the spec requires. with_file(path) appends to a file, creating it if needed, and opens it in build() so a bad path fails early. with_writer(w) accepts any std::io::Write + Send + 'static.
Writes: each line is serialized first, then written and flushed under the writer's lock (and the stdout lock on stdout), so it is complete when export returns and does not interleave with other exports or println! output. If a write fails partway through a line, the export returns an error and the rest of the line is written before the next one, so later lines stay valid. Empty batches write nothing.
Shutdown: flushes and drops the writer; later exports return AlreadyShutdown.
Configuration: MetricExporter reads OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE with the same precedence as opentelemetry-otlp (builder, then environment, then cumulative).
Encoding: reuses the opentelemetry-proto with-serde types and transforms, the same path as http-json. No new dependencies are added to the workspace.
Docs: the README covers destinations, configuration, the Collector receiver's max_log_size limit, blocking writes with the async-runtime processors, and running on AWS Lambda, including the trade-offs between flushing strategies.

Consistency with other SDKs

Python (opentelemetry-exporter-otlp-json-file): writes to a path, a stream or stdout, and flushes after every export. This crate does the same.
Java (OtlpStdout*Exporter): writes to any OutputStream and never closes stdout. with_writer and the stdout handling match this.
C++: also offers file rotation. It is not included here because the spec does not ask for it.

Why a separate crate

The exporter needs none of the transport configuration in opentelemetry-otlp (endpoint, headers, timeout, retry, TLS), and opentelemetry-otlp is being stabilized while this spec is still Development, so a separate crate keeps it out of that API. Python and Java also ship it as a separate package. In review, @lalitb agreed with keeping it as a separate crate in this repo.

Not covered

OTEL_EXPORTER_OTLP_METRICS_DEFAULT_HISTOGRAM_AGGREGATION, because PushMetricExporter cannot choose the default aggregation yet. Once feat(metrics): add OTEL_EXPORTER_OTLP_METRICS_DEFAULT_HISTOGRAM_AGGREGATION support #3433 lands, MetricExporter can implement it the same way as the OTLP exporter.
OTEL_{TRACES,METRICS,LOGS}_EXPORTER=otlp/stdout, because the Rust SDK does not select exporters from environment variables.

Testing

Unit tests per signal: encoding, a round trip through opentelemetry-proto deserialization, empty batches, shutdown, file create/append/open errors and concurrent writes.
Lints: Clippy passes with -D warnings for each feature on its own and with all features, and the workspace lib tests pass.
Collector interop: output covering every attribute type, events, links, error status, all metric kinds including exponential histograms (cumulative and delta) and structured log bodies was read by the otlp_json_file receiver in Collector contrib 0.161.0. Diffing what its file exporter wrote back against the input showed no differences for any signal.
AWS Lambda (provided.al2023, arm64) with all three exporters on stdout and force_flush at the end of each invocation: every line arrived in CloudWatch Logs as a single log event with both the Text and JSON log formats, and the extracted lines passed the same Collector check. Lines longer than the roughly 256 KiB CloudWatch event limit are split across events, which the README points out.
Collector line limit: with Collector contrib 0.162.0, a 2.3 MiB traces line was dropped without an error under the receiver's default 1 MiB max_log_size and received in full with max_log_size: 4MiB, which the README now documents.
Async runtimes: with the async-runtime batch span processor on Tokio and a writer that blocks for 3 s, a 500 ms export timeout did not fire, which the README now documents.
MSRV and public API: the crate builds with Rust 1.75, and cargo check-external-types (nightly-2025-05-04) passes with the new allowed-external-types.toml, which I added to the CI matrix.

Notes

The with-serde exemplar encoding gap I found while testing ([Bug]: opentelemetry-proto with-serde nests Exemplar values under "value" and writes asInt as a JSON number #3764) was fixed in fix(proto): encode asInt as a string and flatten exemplar values in OTLP/JSON #3765, which this branch includes. NaN and infinite doubles still become null (Follow up: Track remaining OTLP metrics JSON serde issues for NaN and integer-valued floats #3442, feat(otlp): Improved support for OTLP JSON metrics serialization (NaN,Infinity,-Infinity) #3496), which affects this crate and http-json equally.
The diff is larger than the 500-line guideline: about 1,000 lines of code, docs, examples and manifests, plus about 780 lines of tests. I can split it per signal if that is easier to review.

This was developed with AI assistance (Claude); I reviewed and tested the changes.

Merge requirement checklist

CONTRIBUTING guidelines followed
Unit tests added/updated (if applicable)
Appropriate CHANGELOG.md files updated for non-trivial, user-facing changes
Changes in public API reviewed (if applicable)

…porter SpanExporter, MetricExporter and LogExporter write each export as one line of OTLP JSON (TracesData, MetricsData or LogsData) to stdout, to a file opened for appending, or to any std::io::Write implementation, as described by the OpenTelemetry Protocol File Exporter specification.

opentelemetry-pr-dashboard · 2026-09-28T23:47:16Z

Pull request dashboard status

Waiting on the author · refreshed 2026-10-17 01:45 UTC

Respond to 4 review items (e.g. link a commit, explain why not, ask a follow-up):

Inline threads: 1, 2, 3, 4

Status above doesn't look right?

Just replied or pushed? Anything around or after the refresh time above may not be picked up yet — give it a few minutes.
Should this be with reviewers? Comment /dashboard route:reviewers to route it to them.
Anything wrong — including the routing? Report it with what you expected; it helps us improve the dashboard.

codecov · 2026-09-28T23:52:37Z

Codecov Report

❌ Patch coverage is 92.57593% with 66 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.9%. Comparing base (8d81bb7) to head (e8bf3b7).

Files with missing lines	Patch %	Lines
opentelemetry-otlp-file/src/metrics.rs	88.4%	23 Missing ⚠️
opentelemetry-otlp-file/src/output.rs	94.0%	22 Missing ⚠️
opentelemetry-otlp-file/src/logs.rs	87.4%	16 Missing ⚠️
opentelemetry-otlp-file/src/trace.rs	97.4%	5 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##            main   #3763     +/-   ##
=======================================
+ Coverage   84.1%   84.9%   +0.8%     
=======================================
  Files        123     127      +4     
  Lines      30280   31169    +889     
=======================================
+ Hits       25483   26486   +1003     
+ Misses      4797    4683    -114

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

SystemTime has 100 ns resolution on Windows, so a start time ending in ...321 ns came back as ...300 and failed the encoding assertion there.

scottgerring

Hey @garysassano thanks for the PR! -i've put some initial feedback inline but it's generally looking good! I reckon the biggest thing to agree on to move forward is where this belongs - contrib, here, or as an experimental feature on the OTLP crate. I'm kind of leaning towards the latter to avoid duplication.

Have tagged the rest of the maintainers to express opinions so we can unblock you

scottgerring · 2026-10-06T10:37:23Z

@@ -0,0 +1,53 @@
+[package]


I feel like we should host this in contrib or as an unstable feature in the OTLP crate (which will help with re-use), in particular because the spec is still in development.

Reviewing it inline anyhow as I don't think it should affect the actual contents much, but it would be good to align on this - @open-telemetry/rust-maintainers thoughts?

Happy to move it wherever you prefer. If it goes into opentelemetry-otlp, I'd suggest a separate file module behind its own feature flag (e.g. opentelemetry_otlp::file::SpanExporter) rather than a third option next to with_tonic()/with_http() on the existing builders, since it shares none of the endpoint, headers, timeout or TLS configuration. That would also let it reuse the env-var helpers you mention in the other comment. The move itself is mostly mechanical, since the encoding already goes through the same opentelemetry-proto with-serde types as http-json, so I'll wait for the decision before changing anything.

I’d keep this as a separate crate in this repo, unless other maintainers have concerns. The file exporter has a distinct configuration and dependency surface from HTTP/gRPC, and it already reuses the proto transforms. The small amount of duplicated environment-variable handling doesn’t seem sufficient reason to put it in opentelemetry-otlp or move that policy into the SDK. From my side, @garysassano can proceed with the current placement.

Thanks @lalitb, I'll keep it as a separate crate here then. The Development status is called out at the top of the README and in the crate docs. I'll leave this thread open in case @scottgerring wants to weigh in, since he leaned towards the OTLP crate.

…uildError

lalitb · 2026-10-08T21:41:01Z

@@ -0,0 +1,53 @@
+[package]


I’d keep this as a separate crate in this repo, unless other maintainers have concerns. The file exporter has a distinct configuration and dependency surface from HTTP/gRPC, and it already reuses the proto transforms. The small amount of duplicated environment-variable handling doesn’t seem sufficient reason to put it in opentelemetry-otlp or move that policy into the SDK. From my side, @garysassano can proceed with the current placement.

…t one

garysassano · 2026-10-13T16:50:56Z

/dashboard route:reviewers

opentelemetry-pr-dashboard · 2026-10-13T16:51:49Z

@garysassano, this pull request was routed to reviewers. The handoff remains active across pushes until newer actionable human feedback arrives. Top-level feedback through this request will not return; unresolved review threads remain open.

herin049 · 2026-10-14T01:03:50Z

+    /// exporter. The writer is dropped even if the flush fails.
+    pub(crate) fn shutdown(&self) -> OTelSdkResult {
+        let mut state = self.lock()?.take().ok_or(OTelSdkError::AlreadyShutdown)?;
+        state.flush().map_err(|err| self.io_error(err))


It may be worth also calling fsync when the destination is a file so that the caller can be more certain that the data is actually written durably to disk.

herin049 · 2026-10-14T01:06:43Z

+
+/// The destination configured on an exporter builder.
+#[derive(Default)]
+pub(crate) enum Output {


Can we use a more descriptive name here other than just Output, e.g. OutputStream?

herin049 · 2026-10-14T01:11:46Z

@@ -0,0 +1,667 @@
+use std::fmt;


This might just be me, but I feel like a lot of the logic in this file can be greatly simplified and that some of these helper enums and structs can be consolidated.

herin049 · 2026-10-14T01:23:39Z

+}
+
+#[cfg(test)]
+mod tests {


Can we include some true end-to-end tests that involve writing to a real file (same for the trace/log exporters)?

garysassano added 2 commits September 29, 2026 01:45

docs: link the changelog entry to its pull request

eab4861

opentelemetry-pr-dashboard Bot mentioned this pull request Sep 28, 2026

Pull Request Dashboard #3735

Open

garysassano mentioned this pull request Sep 28, 2026

[Bug]: opentelemetry-proto with-serde nests Exemplar values under "value" and writes asInt as a JSON number #3764

Closed

garysassano changed the title ~~feat: add opentelemetry-otlp-file crate implementing the OTLP File Exporter~~ feat: add opentelemetry-otlp-file crate implementing the OTLP File Exporter Sep 29, 2026

test: use span timestamps that SystemTime can represent on Windows

e430738

SystemTime has 100 ns resolution on Windows, so a start time ending in ...321 ns came back as ...300 and failed the encoding assertion there.

garysassano force-pushed the feat/otlp-file-exporter branch from 2166952 to e430738 Compare September 29, 2026 20:04

garysassano marked this pull request as ready for review September 29, 2026 20:30

garysassano requested a review from a team as a code owner September 29, 2026 20:30

garysassano added 3 commits September 30, 2026 19:25

Merge branch 'main' into feat/otlp-file-exporter

b91b3e4

Merge branch 'main' into feat/otlp-file-exporter

5c40464

Merge branch 'main' into feat/otlp-file-exporter

109c157

scottgerring reviewed Oct 6, 2026

View reviewed changes

garysassano added 2 commits October 6, 2026 19:06

chore(otlp-file): align keywords and drop non_exhaustive on ExporterB…

aec4ec7

…uildError

docs(otlp-file): compare flushing strategies on AWS Lambda

a4235c2

scottgerring mentioned this pull request Oct 7, 2026

chore: decouple opentelemetry-proto from opentelemetry-sdk #3512

Open

lalitb reviewed Oct 8, 2026

View reviewed changes

garysassano added 5 commits October 9, 2026 14:21

fix(otlp-file): complete a partly written line before writing the nex…

dbaf1b4

…t one

docs(otlp-file): document the receiver line limit and blocking writes

4b41ca8

Merge branch 'main' into feat/otlp-file-exporter

1f1c35f

chore(otlp-file): align version with the 0.33.1 workspace release

4733c95

Merge branch 'main' into feat/otlp-file-exporter

e8bf3b7

This comment has been minimized.

Sign in to view

herin049 reviewed Oct 14, 2026

View reviewed changes

Conversation

garysassano commented Sep 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Consistency with other SDKs

Why a separate crate

Not covered

Testing

Notes

Merge requirement checklist

Uh oh!

opentelemetry-pr-dashboard Bot commented Sep 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull request dashboard status

Uh oh!

codecov Bot commented Sep 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

scottgerring left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

garysassano commented Oct 13, 2026

Uh oh!

opentelemetry-pr-dashboard Bot commented Oct 13, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

herin049 Oct 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

garysassano commented Sep 28, 2026 •

edited

Loading

opentelemetry-pr-dashboard Bot commented Sep 28, 2026 •

edited

Loading

codecov Bot commented Sep 28, 2026 •

edited

Loading

herin049 Oct 14, 2026 •

edited

Loading