Skip to content

PAL-1104 document Langfuse server version detection and SDK v5 routing - #1351

Open
peter-vapi wants to merge 5 commits into
mainfrom
peter/PAL-1104-langfuse-upgrade
Open

peter-vapi wants to merge 5 commits into
mainfrom
peter/PAL-1104-langfuse-upgrade

Conversation

@peter-vapi

@peter-vapi peter-vapi commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Tracking: PAL-1288 Publish new docs gates the merge of this PR. Parent: PAL-1104 Migrate Langfuse tracing to JS SDK v5.

Documents the Langfuse integration upgrade on the existing Langfuse observability page.

What the page now covers

  • The credential walkthrough as it actually behaves: the automatic version read on Host URL blur, the Check server version trigger, save gating until a version resolves, the manual-entry fallback when /api/public/health is hidden, and the public-address requirement.
  • Export routing by resolved server version: Cloud by hostname, self-hosted 3.63+ once detected, everything else on legacy v3. Explicitly calls out that an existing self-hosted credential with no stored version stays on legacy until it is re-saved, and that nothing re-probes on its own.
  • Trace IDs and Call Correlation (new section): the trace ID is no longer the call ID verbatim, it is the first 32 hex characters of sha256(callId); metadata.callId is the recommended correlation key; a warning plus TypeScript/Python snippets for anyone whose code joins on traceId == call.id.
  • The Langfuse Cloud November 16, 2026 legacy cutoff, scoped honestly: Vapi's export is handled, legacy LLM-as-a-judge evaluators are the customer's own action, with links to Langfuse's migration and v4 upgrade guides. Self-hosted is unaffected by that date.
  • Troubleshooting for every refusal the save path can return: redirecting Host URL, non-public Host URL, undetectable server version, and mid-call tracing failure.
  • Trace metadata limits on the OTLP path (512 keys, nested values serialized, Vapi identifiers always kept).

Accompanies VapiAI/vapi#25539 (groundwork), VapiAI/vapi#25583 (version detection) and VapiAI/vapi#25581 (v5 OTLP tracing client), all merged, plus VapiAI/vapi#25708 (version-aware validation and the error taxonomy these troubleshooting entries quote). Supersedes the reference to the original VapiAI/vapi#23001, which was closed and split into those PRs. Shared with the Langfuse team in #ext-vapi-langfuse.

Before merge

  • Merge only after the backend chain lands and the tracing rollout reaches 100%, which is what PAL-1288 tracks. The page describes the version row, the error copy from VapiAI/vapi#25708, and OTLP trace IDs; merging earlier documents behavior customers do not have yet.
  • Replace the credentials screenshot (https://langfuse.com/images/docs/vapi-integration-credentials.png, hosted by Langfuse) with a fresh capture of the Langfuse card showing the Server version row, and host it under fern/static/images/providers/ so we own the asset.

After the November 16, 2026 cutoff

  • Trim the two dated Langfuse Cloud cutoff notes (the one in Server Version Detection and SDK Routing and the sentence in the Evaluate and Debug your Agent step) down to a plain pointer at Langfuse's v4 docs, so the page does not carry a stale deadline.

…oubleshooting for redirecting host urls and tracing failures

OpenCode session ID: ses_ece821fbeffeZ6yysKJ60XiP1X
@github-actions

Copy link
Copy Markdown
Contributor

The dashboard integration modal now links to this page instead of
going straight to Langfuse's docs, so readers need a path to
Langfuse's own tracing guide once they land here.

- Add a reference to Langfuse's tracing documentation right after
  Vapi's framing paragraph, before the video walkthrough

OpenCode session ID: ses_ece821fbeffeZ6yysKJ60XiP1X
@github-actions

Copy link
Copy Markdown
Contributor

@peter-vapi
peter-vapi marked this pull request as ready for review October 22, 2026 21:05
@peter-vapi
peter-vapi requested review from a team and ashwin-vapi and removed request for a team October 22, 2026 21:05

@hal-anil-vapi hal-anil-vapi left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see comments

- Note that each top-level metadata key maps to one trace
  attribute, with nested values serialized as JSON strings
- Document the 512-key cap per trace and that excess keys
  beyond the first 512 are silently dropped
- Clarify that Vapi's own identifiers (e.g. call ID) are added
  separately and always kept, even if user metadata reuses
  the same key name
- Recommend keeping metadata small and filterable, storing
  large payloads elsewhere by reference

Addresses reviewer feedback on vapi#23001

OpenCode session ID: ses_ece821fbeffeZ6yysKJ60XiP1X
@github-actions

Copy link
Copy Markdown
Contributor

Address Hal's four review comments on the Langfuse docs PR and move
the credentials screenshot to self-hosted storage.

- Describe the credential walkthrough in full: automatic version read
  on Host URL blur, the Check server version trigger, the save gate,
  the manual-entry fallback for proxied health endpoints, and the
  public-address requirement for the Host URL
- Rewrite the routing section into four explicit cases, with Langfuse
  Cloud recognized by hostname regardless of when the credential was
  saved; note that a versionless self-hosted credential stays on the
  legacy path until re-saved, and remove the false claim that a later
  probe corrects the stored version automatically
- Add a Trace IDs and Call Correlation section documenting the
  sha256-derived trace ID, metadata.callId as the recommended
  correlation key, and TypeScript/Python derivation snippets
- Expand Troubleshooting to four entries covering every refusal the
  save path can return
- Add the Langfuse Cloud legacy-API/evaluator cutoff of November 16,
  2026, with a pointer to the LLM-as-a-judge migration guide
- Host the credentials screenshot under fern/static/images/providers
  instead of hotlinking from langfuse.com, and update it to show the
  Server version row

OpenCode session ID: ses_e4d07141affeH93cc94qFsDEgV
- Pulls in main's broader alt-text pass (#1365) and other unrelated
  doc fixes (SIP dashboard nav copy, OpenAPI spec updates, traffic
  allocation GA, Vapi for CX renaming) that had landed since this
  branch was cut
- Resolves the only conflict, on the credentials Frame in the
  Langfuse integration doc: main edited caption/alt text on the same
  two lines this branch rewrote to self-host the screenshot
- Keeps both sides: this branch's local image path, plus main's
  caption and alt text, extending the alt text to describe the
  Server version row and Check server version button now visible in
  the new screenshot
- Leaves the example-trace Frame on main's version, unchanged by this
  branch
- fern check passes with no warnings on this page, clearing CI to
  schedule the PR

OpenCode session ID: ses_e4d07141affeH93cc94qFsDEgV
@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants