[issue-7686] [BE][FE] feat: add OpenInference pretty formatting - #8141
[issue-7686] [BE][FE] feat: add OpenInference pretty formatting#8141d8rt8v wants to merge 1 commit into
Conversation
github-actions
Bot
added
java
df8eec5 to
82398bd
Compare
![baz-reviewer[bot]](https://avatars.githubusercontent.com/in/933528?s=60&v=4){kind=small}
| const url = content.image?.url; | ||
| if (url) { | ||
| blocks.push({ | ||
| blockType: "image", | ||
| component: PrettyLLMMessage.ImageBlock, | ||
| props: { images: [{ url, name: mediaName(url, "Image", index) }] }, |
There was a problem hiding this comment.
Untrusted media URL enables unsafe navigation
The OpenInference mapper copies unvalidated content.image.url and audio URLs into shared descriptors, so AttachmentThumbnail uses them as <a href> values and the audio component as media sources, allowing javascript:, data:text/html, or custom-scheme URLs to reach unsafe browser destinations — should we validate schemes before building the descriptors or enforce the policy at the shared destination boundary?
Want Baz to fix this for you? Activate Fixer
Other fix methods
Prompt for AI Agents
Before applying, verify this suggestion against the current code. In
apps/opik-frontend/src/shared/PrettyLLMMessage/llmMessages/providers/openinference/mapper.ts
around lines 117-134, update the `contentBlocks` image/audio handling so untrusted
`content.image.url` and `content.audio.url` values are not passed directly into
`ImageBlock` or `AudioPlayerBlock`. Validate URLs with the existing media safety policy
or a shared helper that permits only approved passive schemes (such as `http`/`https`
and explicitly supported placeholders), and skip or safely replace rejected URLs before
constructing the descriptors.
| JsonNode parsed = JsonUtils.getJsonNodeFromString(value.getStringValue()); | ||
| if (!parsed.isObject()) { | ||
| metadata.set("metadata", parsed); | ||
| return; |
There was a problem hiding this comment.
Empty metadata aborts OTLP ingestion
The OpenInference metadata branch passes empty or whitespace-only JSON to mergeMetadata, where readTree returns null and parsed.isObject() is called unconditionally, so toOpikSpan throws during OTLP normalization and aborts the batch — should we guard the no-document result before checking its shape or use the fallback parser?
Want Baz to fix this for you? Activate Fixer
Other fix methods
Prompt for AI Agents
Before applying, verify this suggestion against the current code. In
`apps/opik-backend/src/main/java/com/comet/opik/domain/mapping/otel/OpenInferenceSpanNormalizer.java`
around lines 491-510, fix the `State.mergeMetadata` method so empty or whitespace-only
metadata strings do not cause a null dereference at `parsed.isObject()`. Guard a null
parse result (or switch to the fallback parser) and preserve the original metadata
string or an explicit null value in the metadata object, ensuring marked spans continue
ingesting without aborting the OTLP batch.
| if ( | ||
| inputDetection.format === "openinference" || | ||
| outputDetection.format === "openinference" | ||
| ) { |
There was a problem hiding this comment.
Mixed-format output is silently dropped
A mixed pair with a legacy OpenInference side is routed to combineOpenInferenceMessages, whose item.text/item["completion.text"] parsing ignores OpenAI choices[].message, so the assistant output is dropped — should we restrict the combiner to compatible pairs or retain the sibling mapper result?
Want Baz to fix this for you? Activate Fixer
Other fix methods
Prompt for AI Agents
Before applying, verify this suggestion against the current code. In
`apps/opik-frontend/src/shared/PrettyLLMMessage/llmMessages/mapAndCombineMessages.ts`
around lines 31-38, update `mapAndCombineMessages` so the OpenInference combiner is not
used merely because one side detected as OpenInference. For mixed pairs such as legacy
OpenInference input and OpenAI `choices[].message` output, preserve the sibling mapper
result by restricting the combiner to compatible inputs or combining its result with the
independently mapped result.
| let processedMessage = extractOpenInferencePrettyText(message, config.type); | ||
|
|
||
| if (!isString(processedMessage)) { | ||
| processedMessage = prettifyOpenAIMessageLogic(message, config); | ||
| } |
There was a problem hiding this comment.
Unmarked messages bypass provider formatting
extractOpenInferencePrettyText runs before provider-specific formatters and accepts any role-bearing messages without an OpenInference marker, so unmarked inputs are rendered as OpenInference text instead of their existing representation — should we gate it on the marker/legacy attributes or run provider-specific formatting first at apps/opik-frontend/src/lib/traces.ts:640-644?
Want Baz to fix this for you? Activate Fixer You can also update your AI coding guidelines based on this comment by apply pr to [branch name]
Other fix methods
Prompt for AI Agents
Before applying, verify this suggestion against the current code. In
apps/opik-frontend/src/lib/traces.ts around lines 640-644, update the `prettifyMessage`
formatter chain so `extractOpenInferencePrettyText` does not process arbitrary
role-bearing `messages` arrays before provider-specific logic. Gate it on an explicit
OpenInference marker or legacy attribute, or run the existing OpenAI/provider formatters
first and use OpenInference extraction only as an appropriate fallback. Preserve the
current behavior for unmarked system, assistant, developer, and tool messages.
| const DISPLAYABLE_LEGACY_KEYS = new Set([ | ||
| OPENINFERENCE_SPAN_KIND, | ||
| "llm.finish_reason", | ||
| "llm.function_call", | ||
| ]); |
There was a problem hiding this comment.
Metadata-only records show empty tab
detectOpenInferenceFormat treats openinference.span.kind, llm.finish_reason, and malformed indexed prefixes as renderable data, so TraceDataViewer exposes an empty Messages tab when mapParsed produces no messages — should we base support on parsed renderable messages, prompts, tools, choices, or function calls while keeping metadata in Details?
Want Baz to fix this for you? Activate Fixer You can also update your AI coding guidelines based on this comment by apply pr to [branch name]
Other fix methods
Prompt for AI Agents
Before applying, verify this suggestion against the current code. In
apps/opik-frontend/src/lib/openinference.ts around lines 68-72 and 483-490, refactor the
OpenInference support predicates so `openinference.span.kind`, `llm.finish_reason`, and
malformed indexed prefixes do not by themselves mark a payload as renderable. Base
Messages-tab support on successfully parsed messages, prompts, tools, choices, or
function calls that the mapper can display, while retaining marker and finish-reason
metadata for Details. Add or update visibility tests covering marker-only, finish-only,
and malformed-index-only records so detection and rendering stay consistent.
| // OpenInference semantic attributes have already been normalized as one coherent shape. | ||
| // Skipping them here prevents generic prefix rules from processing the same key again. | ||
| if (openInference != null && openInference.consumes(key)) { | ||
| continue; |
There was a problem hiding this comment.
Breaks existing online scoring paths
Marked spans now route semantic llm.output_messages.* keys to span.output.messages instead of span.input, so OnlineScoringEngine mappings such as input.llm.output_messages... silently produce no replacement for newly ingested spans — should we retain a compatibility alias or explicitly migrate/document this released consumer contract?
Want Baz to fix this for you? Activate Fixer
Other fix methods
Prompt for AI Agents
Before applying, verify this suggestion against the current code. In
apps/opik-backend/src/main/java/com/comet/opik/domain/OpenTelemetryMapper.java around
lines 186-189, update enrichSpanWithAttributes so consumed OpenInference
llm.output_messages.* attributes remain available under the legacy span.input path used
by OnlineScoringEngine mappings, while preserving their canonical span.output.messages
representation. Add a compatibility alias during OpenInference input/output composition
(or an equivalent explicit migration path) and add regression coverage proving
input.llm.output_messages mappings still resolve for newly ingested marked spans.
|
Here are some examples 
|
1 participant
Details
OpenInference spans received through the private OTLP trace endpoint now normalize raw values, chat/completion semantics, tools, multimodal content, model/provider data, usage, cost, sessions, tags, and metadata into the canonical Opik span shape. The Messages tab and shared Pretty formatter now support both that canonical shape and historical flattened OpenInference records, including output attributes that older ingestion stored under input, without a database migration or backfill.
openinference.span.kindmarker so mixed OTLP batches and unmarked spans keep their existing behavior.openinferencefrontend format with a pure canonical/legacy parser, pair-aware message combination, multimodal/tool/reasoning rendering, role normalization, and normalized span usage.Before
Historical flattened attributes were shown as raw dotted-key data and did not open the Messages tab.
After
The same stored span opens in Messages and renders roles, tool calls/results, finish reason, and usage; newly ingested spans also preserve ordered multimodal content.
Change checklist
Issues
AI-WATERMARK
AI-WATERMARK: yes
Testing
mvn -q -Dtest=OpenTelemetryMapperTest testmvn -q -Dtest='OpenTelemetryResourceTest$ApiKey#testOpenInferenceSpanNormalization' testnpm run test -- src/lib/traces.test.ts src/shared/PrettyLLMMessage/llmMessages/providers/openai/detector.test.ts src/shared/PrettyLLMMessage/llmMessages/providers/openai/mapper.test.ts src/shared/PrettyLLMMessage/llmMessages/providers/langchain/detector.test.ts src/shared/PrettyLLMMessage/llmMessages/providers/langchain/mapper.test.ts src/shared/PrettyLLMMessage/llmMessages/providers/openinference/detector.test.ts src/shared/PrettyLLMMessage/llmMessages/providers/openinference/mapper.test.ts --run(148 tests)npm run typechecknpm run lintmvn -q spotless:checkmake precommit./opik.sh --build./opik.sh --verifytool_call_id, tool result, model/provider fields, and per-call token usageValidated JSON/text/native raw values, invalid JSON and malformed attributes, sparse and shuffled indices, input/output separation, chat/completion/tool schemas and calls, tool results, ordered text/image/audio/reasoning/tool-use content, field priority, mixed marked/unmarked batches, canonical and historical frontend shapes, false-positive detection, preview extraction, and raw-output deduplication.
Documentation
No documentation update is required. This is transparent ingestion and UI compatibility with no public API, schema, configuration, or migration change.