[aakarim]

Summary

• Add opt-in OTLP/HTTP protobuf export configured primarily through tracing: in openlore.yml, with bounded asynchronous export, sampling, and shutdown flushing.
• File settings take precedence over optional standard OTEL environment fallbacks; enabled: false prevents export. Configure the full endpoint, service name, sampling ratio, authentication headers, and resource attributes without environment variables.
• Instrument HTTP requests, MCP tool calls/listing, shell invocations, and commands without exporting arguments, paths, document contents, or output.
• Propagate current request trace context through persistent MCP and HTTP sessions; support MCP request metadata for non-HTTP transports.
• Correlate analytics events and context-aware logs with trace/span IDs while preserving existing Prometheus metrics and host-owned telemetry providers.
• Document configuration, privacy boundaries, and integration limitations.

Verification

• nix develop --command go test ./... — passed, including after rebasing onto current main.
• nix develop --command go vet ./... — passed.
• nix develop --command go test -race ./internal/config ./internal/telemetry ./cmd/openlore ./pkg/openlore ./pkg/shell ./internal/analytics ./internal/httpserver — passed.
• nix develop --command go mod verify — all modules verified.
• Built CLI exercised through actual MCP stdio with a local OTLP receiver: enabled, disabled, unconfigured, sampled-out, and collector-failure scenarios all preserved command output and exit codes. Enabled export retained the caller trace and omitted command content.
• File-config revision also exercised the built CLI against a local OTLP receiver: file-only settings, conflicting environment values, explicit disable, zero sampling, and empty headers all passed; invalid sampling is rejected. The full test suite, vet, and targeted race suite passed after this revision.
• Integration tests cover per-request parentage across persistent sessions, metadata propagation, nested execution, log/event correlation, and content-free export.

Scope and limitations

• Export remains off until an endpoint is explicitly configured; only OTLP/HTTP protobuf is supported.
• This is the tracing/correlation foundation, not new OTLP metrics/log export or deep storage tracing.
• Hosted Langfuse, Phoenix, and LangSmith ingestion/presentation have not been verified; backend-specific mapping may be needed.
• No deployment or release is included.

[Copilot]

🔵 Needs a closer look

Cross-transport propagation and exporter lifecycle changes need human validation, and nested write attribution remains unresolved.

1 open issue

• Restore invocation observer after nested eval/source execution · New

What changed in this PR

Adds opt-in OpenTelemetry tracing across OpenLore execution paths, with analytics and log correlation.

Changes:

• Adds configurable OTLP/HTTP export, sampling, and shutdown flushing.
• Propagates request context through HTTP, MCP, and shell execution.
• Adds integration tests and configuration/privacy documentation.

File	Description
pkg/​shell/​shell.go	Adds invocation and command tracing.
pkg/​shell/​cmds/​analytics_metrics.go	Passes execution context to metrics.
pkg/​shell/​analytics_test.go	Tests nested tracing and content redaction.
pkg/​openlore/​telemetry_test.go	Tests request parentage and transport propagation.
pkg/​openlore/​server.go	Adds shell context and log correlation.
pkg/​openlore/​mcp.go	Instruments MCP requests and propagates context.
pkg/​openlore/​mcp_test.go	Updates transcript tests for context.
pkg/​openlore/​mcp_http_sessions.go	Passes current request context to sessions.
openlore.yml.example	Adds tracing configuration examples.
internal/​telemetry/​telemetry.go	Implements exporter setup and correlation handlers.
internal/​telemetry/​telemetry_test.go	Tests export, configuration precedence, and sampling.
internal/​httpserver/​httpserver.go	Wraps HTTP handling with tracing.
internal/​configdocs/​main.go	Defines tracing configuration documentation.
internal/​config/​documented_keys_test.go	Tests tracing configuration validation.
internal/​config/​config.go	Loads and validates tracing settings.
internal/​analytics/​types.go	Adds trace and span identifiers.
internal/​analytics/​storage.go	Captures correlation before queuing events.
internal/​analytics/​analytics_test.go	Tests persisted trace correlation.
go.sum	Records dependency checksums.
go.mod	Adds telemetry dependencies.
docs/​openlore-yml.md	Documents tracing settings.
docs/​configure-the-server.md	Explains tracing, privacy, and limitations.
cmd/​openlore/​main.go	Initializes tracing and flushes on shutdown.

_{Review effort: Balanced}