Skip to content

Commit f890d1f

Browse files
hassiebpcursoragentlangfuse-botlangfuse-botniklassemmler
authored
feat!: prepare v5 release (#1991)
* chore!: require Python 3.11 or newer (#1927) * chore!: require Python 3.11 or newer Python 3.10 reaches end of life in October 2026. Raise requires-python to >=3.11, target py311 in ruff, drop 3.10 from the CI unit matrix, and remove the pre-3.11 fallbacks for asyncio.create_task(context=...) and typing.NotRequired. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(observe): keep detecting legacy-marked coroutine functions inspect.iscoroutinefunction ignores asyncio's _is_coroutine marker, which asgiref's markcoroutinefunction sets on Python < 3.12. Accept both so such functions keep the async wrapper and their observation ends after the coroutine runs. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(openai)!: drop support for OpenAI SDK versions below 1.0 (#1928) * chore!: require Python 3.11 or newer Python 3.10 reaches end of life in October 2026. Raise requires-python to >=3.11, target py311 in ruff, drop 3.10 from the CI unit matrix, and remove the pre-3.11 fallbacks for asyncio.create_task(context=...) and typing.NotRequired. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(openai)!: drop support for OpenAI SDK versions below 1.0 The OpenAI integration now always uses the v1 client shapes. Remove the pre-1.0 method definitions and every _is_openai_v1() branch, and raise the dev dependency floor to openai>=1.0.0. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(observe): keep detecting legacy-marked coroutine functions inspect.iscoroutinefunction ignores asyncio's _is_coroutine marker, which asgiref's markcoroutinefunction sets on Python < 3.12. Accept both so such functions keep the async wrapper and their observation ends after the coroutine runs. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(otel)!: send x-langfuse-ingestion-version: 4 by default (#1930) * chore!: require Python 3.11 or newer Python 3.10 reaches end of life in October 2026. Raise requires-python to >=3.11, target py311 in ruff, drop 3.10 from the CI unit matrix, and remove the pre-3.11 fallbacks for asyncio.create_task(context=...) and typing.NotRequired. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(openai)!: drop support for OpenAI SDK versions below 1.0 The OpenAI integration now always uses the v1 client shapes. Remove the pre-1.0 method definitions and every _is_openai_v1() branch, and raise the dev dependency floor to openai>=1.0.0. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(otel)!: send x-langfuse-ingestion-version: 4 by default The default OTLP exporter now sends x-langfuse-ingestion-version: 4 so servers in dual-write or preview mode process exported spans directly. It is a no-op on events_only deployments, and additional_headers can still override it. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(observe): keep detecting legacy-marked coroutine functions inspect.iscoroutinefunction ignores asyncio's _is_coroutine marker, which asgiref's markcoroutinefunction sets on Python < 3.12. Accept both so such functions keep the async wrapper and their observation ends after the coroutine runs. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * test(otel): pin that a mixed-case ingestion-version override wins Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * chore: re-run CI Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(client)!: remove blocked_instrumentation_scopes (#1931) * chore!: require Python 3.11 or newer Python 3.10 reaches end of life in October 2026. Raise requires-python to >=3.11, target py311 in ruff, drop 3.10 from the CI unit matrix, and remove the pre-3.11 fallbacks for asyncio.create_task(context=...) and typing.NotRequired. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(openai)!: drop support for OpenAI SDK versions below 1.0 The OpenAI integration now always uses the v1 client shapes. Remove the pre-1.0 method definitions and every _is_openai_v1() branch, and raise the dev dependency floor to openai>=1.0.0. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(otel)!: send x-langfuse-ingestion-version: 4 by default The default OTLP exporter now sends x-langfuse-ingestion-version: 4 so servers in dual-write or preview mode process exported spans directly. It is a no-op on events_only deployments, and additional_headers can still override it. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(client)!: remove blocked_instrumentation_scopes blocked_instrumentation_scopes was deprecated in favor of should_export_span. Remove it from Langfuse, the resource manager and the span processor, and drop the leftover is_active entry from the create_prompt docstring. LANGFUSE_HOST and host keep working, but now log a deprecation warning when they decide the base URL. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(observe): keep detecting legacy-marked coroutine functions inspect.iscoroutinefunction ignores asyncio's _is_coroutine marker, which asgiref's markcoroutinefunction sets on Python < 3.12. Accept both so such functions keep the async wrapper and their observation ends after the coroutine runs. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * test(otel): pin that a mixed-case ingestion-version override wins Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(client): keep host and LANGFUSE_HOST working without a warning Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * chore: re-run CI Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(tracing)!: JSON-serialize propagated metadata values (#1932) * chore!: require Python 3.11 or newer Python 3.10 reaches end of life in October 2026. Raise requires-python to >=3.11, target py311 in ruff, drop 3.10 from the CI unit matrix, and remove the pre-3.11 fallbacks for asyncio.create_task(context=...) and typing.NotRequired. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(openai)!: drop support for OpenAI SDK versions below 1.0 The OpenAI integration now always uses the v1 client shapes. Remove the pre-1.0 method definitions and every _is_openai_v1() branch, and raise the dev dependency floor to openai>=1.0.0. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(otel)!: send x-langfuse-ingestion-version: 4 by default The default OTLP exporter now sends x-langfuse-ingestion-version: 4 so servers in dual-write or preview mode process exported spans directly. It is a no-op on events_only deployments, and additional_headers can still override it. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(client)!: remove blocked_instrumentation_scopes blocked_instrumentation_scopes was deprecated in favor of should_export_span. Remove it from Langfuse, the resource manager and the span processor, and drop the leftover is_active entry from the create_prompt docstring. LANGFUSE_HOST and host keep working, but now log a deprecation warning when they decide the base URL. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(tracing)!: JSON-serialize propagated metadata values propagate_attributes(metadata=...) coerced non-string values with str(), so a boolean became "True" and a list became its Python repr. Serialize them as JSON instead, like observation metadata, and drop None values instead of sending "None". Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(tracing): serialize propagated metadata like JSON.stringify Use compact separators, keep non-ASCII characters, and send None as "null", so the Python and JS SDKs emit byte-identical propagated metadata values. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(observe): keep detecting legacy-marked coroutine functions inspect.iscoroutinefunction ignores asyncio's _is_coroutine marker, which asgiref's markcoroutinefunction sets on Python < 3.12. Accept both so such functions keep the async wrapper and their observation ends after the coroutine runs. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * test(otel): pin that a mixed-case ingestion-version override wins Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(tracing): keep the exact digits of large integer metadata values Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(tracing): match JS for nested big ints and non-finite metadata Integers keep their exact digits as JSON numbers at any depth, and propagated metadata values containing NaN or Infinity are dropped with a warning instead of being stored, matching the JS SDK. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(client): keep host and LANGFUSE_HOST working without a warning Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(client)!: remove get_dataset_run, get_dataset_runs and delete_dataset_run (#1933) * chore!: require Python 3.11 or newer Python 3.10 reaches end of life in October 2026. Raise requires-python to >=3.11, target py311 in ruff, drop 3.10 from the CI unit matrix, and remove the pre-3.11 fallbacks for asyncio.create_task(context=...) and typing.NotRequired. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(openai)!: drop support for OpenAI SDK versions below 1.0 The OpenAI integration now always uses the v1 client shapes. Remove the pre-1.0 method definitions and every _is_openai_v1() branch, and raise the dev dependency floor to openai>=1.0.0. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(otel)!: send x-langfuse-ingestion-version: 4 by default The default OTLP exporter now sends x-langfuse-ingestion-version: 4 so servers in dual-write or preview mode process exported spans directly. It is a no-op on events_only deployments, and additional_headers can still override it. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(client)!: remove blocked_instrumentation_scopes blocked_instrumentation_scopes was deprecated in favor of should_export_span. Remove it from Langfuse, the resource manager and the span processor, and drop the leftover is_active entry from the create_prompt docstring. LANGFUSE_HOST and host keep working, but now log a deprecation warning when they decide the base URL. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(tracing)!: JSON-serialize propagated metadata values propagate_attributes(metadata=...) coerced non-string values with str(), so a boolean became "True" and a list became its Python repr. Serialize them as JSON instead, like observation metadata, and drop None values instead of sending "None". Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(client)!: remove get_dataset_run, get_dataset_runs and delete_dataset_run These helpers call the dataset-run endpoints that Langfuse v4 no longer serves, so they always fail with a 404. Read experiment runs through langfuse.api.experiments.list() and list_items() instead. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(tracing): serialize propagated metadata like JSON.stringify Use compact separators, keep non-ASCII characters, and send None as "null", so the Python and JS SDKs emit byte-identical propagated metadata values. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(observe): keep detecting legacy-marked coroutine functions inspect.iscoroutinefunction ignores asyncio's _is_coroutine marker, which asgiref's markcoroutinefunction sets on Python < 3.12. Accept both so such functions keep the async wrapper and their observation ends after the coroutine runs. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * test(otel): pin that a mixed-case ingestion-version override wins Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(tracing): keep the exact digits of large integer metadata values Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * chore: re-run CI Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(tracing): match JS for nested big ints and non-finite metadata Integers keep their exact digits as JSON numbers at any depth, and propagated metadata values containing NaN or Infinity are dropped with a warning instead of being stored, matching the JS SDK. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(client): keep host and LANGFUSE_HOST working without a warning Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(experiments)!: record experiment runs via OTEL only (#1934) * chore!: require Python 3.11 or newer Python 3.10 reaches end of life in October 2026. Raise requires-python to >=3.11, target py311 in ruff, drop 3.10 from the CI unit matrix, and remove the pre-3.11 fallbacks for asyncio.create_task(context=...) and typing.NotRequired. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(openai)!: drop support for OpenAI SDK versions below 1.0 The OpenAI integration now always uses the v1 client shapes. Remove the pre-1.0 method definitions and every _is_openai_v1() branch, and raise the dev dependency floor to openai>=1.0.0. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(otel)!: send x-langfuse-ingestion-version: 4 by default The default OTLP exporter now sends x-langfuse-ingestion-version: 4 so servers in dual-write or preview mode process exported spans directly. It is a no-op on events_only deployments, and additional_headers can still override it. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(client)!: remove blocked_instrumentation_scopes blocked_instrumentation_scopes was deprecated in favor of should_export_span. Remove it from Langfuse, the resource manager and the span processor, and drop the leftover is_active entry from the create_prompt docstring. LANGFUSE_HOST and host keep working, but now log a deprecation warning when they decide the base URL. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(tracing)!: JSON-serialize propagated metadata values propagate_attributes(metadata=...) coerced non-string values with str(), so a boolean became "True" and a list became its Python repr. Serialize them as JSON instead, like observation metadata, and drop None values instead of sending "None". Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(client)!: remove get_dataset_run, get_dataset_runs and delete_dataset_run These helpers call the dataset-run endpoints that Langfuse v4 no longer serves, so they always fail with a 404. Read experiment runs through langfuse.api.experiments.list() and list_items() instead. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(experiments)!: record experiment runs via OTEL only Generate the experiment id client-side instead of creating dataset run items via POST /dataset-run-items. Dataset-backed runs derive a stable id from project, dataset and run name (matching the server's derivation), so re-runs with the same run name keep grouping into one experiment. Local runs keep a random id. - set langfuse.experiment.item.version from the pinned dataset version - always persist run evaluations as scores on the experiment, incl. local data - return experiment_id / experiment_url (v4 results page) on results; dataset_run_id / dataset_run_url remain as deprecated aliases BREAKING CHANGE: run_experiment no longer creates Postgres dataset runs; dataset_run_id / dataset_run_url are deprecated aliases that are now also set for local-data experiments, and the URL points to the experiments page. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * test(experiments): read experiment e2e results via v4 APIs Replace dataset run, dataset run item and legacy trace reads with the experiments, observations and v3 scores APIs, and cover stable experiment ids, run-level scores for local data and the recorded item version. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * test(experiments): pin the cross-SDK experiment id vectors Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(tracing): serialize propagated metadata like JSON.stringify Use compact separators, keep non-ASCII characters, and send None as "null", so the Python and JS SDKs emit byte-identical propagated metadata values. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(observe): keep detecting legacy-marked coroutine functions inspect.iscoroutinefunction ignores asyncio's _is_coroutine marker, which asgiref's markcoroutinefunction sets on Python < 3.12. Accept both so such functions keep the async wrapper and their observation ends after the coroutine runs. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * test(otel): pin that a mixed-case ingestion-version override wins Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(tracing): keep the exact digits of large integer metadata values Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(experiments): skip orphaned run scores and offline project lookups Run-level scores are only sent when at least one item trace is sampled, because a fully sampled-out experiment never reaches the server. Clients with tracing disabled no longer resolve the project id. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * chore: re-run CI Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(tracing): match JS for nested big ints and non-finite metadata Integers keep their exact digits as JSON numbers at any depth, and propagated metadata values containing NaN or Infinity are dropped with a warning instead of being stored, matching the JS SDK. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(client): keep host and LANGFUSE_HOST working without a warning Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(langchain)!: require LangChain v1 (langchain-core >= 1) (#1940) * feat(langchain)!: drop LangChain v0 support The CallbackHandler now imports only from langchain_core and no longer requires the langchain package to be installed. Remove the langchain.__version__ branch and the langchain.callbacks / langchain.schema fallbacks for LangChain v0. The serializer fallback tests now hide langchain_core, the module the serializer actually imports, and reload the serializer module. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * build(deps)!: require langchain-core>=1 and langchain-openai>=1 Add langchain-core>=1,<2 to the dev dependencies and raise langchain-openai from >=0.0.5,<0.4 to >=1,<2. The lock moves langchain-openai 0.3.34 -> 1.1.12, which keeps langchain-core 1.2.22 and openai 2.29.0 unchanged. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(batch-evaluation)!: evaluate observations via the v2 observations API (#1936) * chore!: require Python 3.11 or newer Python 3.10 reaches end of life in October 2026. Raise requires-python to >=3.11, target py311 in ruff, drop 3.10 from the CI unit matrix, and remove the pre-3.11 fallbacks for asyncio.create_task(context=...) and typing.NotRequired. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(openai)!: drop support for OpenAI SDK versions below 1.0 The OpenAI integration now always uses the v1 client shapes. Remove the pre-1.0 method definitions and every _is_openai_v1() branch, and raise the dev dependency floor to openai>=1.0.0. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(otel)!: send x-langfuse-ingestion-version: 4 by default The default OTLP exporter now sends x-langfuse-ingestion-version: 4 so servers in dual-write or preview mode process exported spans directly. It is a no-op on events_only deployments, and additional_headers can still override it. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(client)!: remove blocked_instrumentation_scopes blocked_instrumentation_scopes was deprecated in favor of should_export_span. Remove it from Langfuse, the resource manager and the span processor, and drop the leftover is_active entry from the create_prompt docstring. LANGFUSE_HOST and host keep working, but now log a deprecation warning when they decide the base URL. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(tracing)!: JSON-serialize propagated metadata values propagate_attributes(metadata=...) coerced non-string values with str(), so a boolean became "True" and a list became its Python repr. Serialize them as JSON instead, like observation metadata, and drop None values instead of sending "None". Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(client)!: remove get_dataset_run, get_dataset_runs and delete_dataset_run These helpers call the dataset-run endpoints that Langfuse v4 no longer serves, so they always fail with a 404. Read experiment runs through langfuse.api.experiments.list() and list_items() instead. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(experiments)!: record experiment runs via OTEL only Generate the experiment id client-side instead of creating dataset run items via POST /dataset-run-items. Dataset-backed runs derive a stable id from project, dataset and run name (matching the server's derivation), so re-runs with the same run name keep grouping into one experiment. Local runs keep a random id. - set langfuse.experiment.item.version from the pinned dataset version - always persist run evaluations as scores on the experiment, incl. local data - return experiment_id / experiment_url (v4 results page) on results; dataset_run_id / dataset_run_url remain as deprecated aliases BREAKING CHANGE: run_experiment no longer creates Postgres dataset runs; dataset_run_id / dataset_run_url are deprecated aliases that are now also set for local-data experiments, and the URL points to the experiments page. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * test(experiments): read experiment e2e results via v4 APIs Replace dataset run, dataset run item and legacy trace reads with the experiments, observations and v3 scores APIs, and cover stable experiment ids, run-level scores for local data and the recorded item version. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * test(experiments): pin the cross-SDK experiment id vectors Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(batch-evaluation)!: evaluate observations via the v2 observations API Batch evaluation now reads exclusively from GET /api/public/v2/observations with cursor pagination, so it works on Langfuse platform v4. BREAKING CHANGES: - scope="traces" is replaced by scope="root_observations", which evaluates the logical root observation of each trace and attaches scores to the trace. scope="observations" is unchanged in intent. - Mappers receive an ObservationV2. input/output are raw strings and only the requested field groups are populated. - fetch_trace_fields is replaced by fields (v2 field groups), defaulting to "core,basic,io,metadata" in both the client and the runner. - filter must be a JSON array in the v2 observations filter format. - Resume tokens carry the pagination cursor and are also returned when max_items is reached while more items exist. - The private _additional_trace_tags option is removed. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * test(batch-evaluation): rewrite e2e suite for v2 observations Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(tracing): serialize propagated metadata like JSON.stringify Use compact separators, keep non-ASCII characters, and send None as "null", so the Python and JS SDKs emit byte-identical propagated metadata values. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(observe): keep detecting legacy-marked coroutine functions inspect.iscoroutinefunction ignores asyncio's _is_coroutine marker, which asgiref's markcoroutinefunction sets on Python < 3.12. Accept both so such functions keep the async wrapper and their observation ends after the coroutine runs. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * test(otel): pin that a mixed-case ingestion-version override wins Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(tracing): keep the exact digits of large integer metadata values Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(experiments): skip orphaned run scores and offline project lookups Run-level scores are only sent when at least one item trace is sampled, because a fully sampled-out experiment never reaches the server. Clients with tracing disabled no longer resolve the project id. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(batch-evaluation): one root per trace and safe resume - root_observations scope evaluates each trace once, preferring the physical root over SDK-marked app roots - resuming reuses the token's filter and rejects a different one, since the cursor only continues the query that produced it - the cursorless start-time fallback is inclusive so tied observations are not skipped Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * chore: re-run CI Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * chore: re-run CI Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(tracing): match JS for nested big ints and non-finite metadata Integers keep their exact digits as JSON numbers at any depth, and propagated metadata values containing NaN or Infinity are dropped with a warning instead of being stored, matching the JS SDK. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(client): keep host and LANGFUSE_HOST working without a warning Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(tracing)!: remove trace-level input/output setters (#1935) Remove the deprecated Langfuse.set_current_trace_io() and span.set_trace_io(), the langfuse.trace.input/output attributes, and their media handling. On Langfuse v4 a trace's input and output come from its root observation: set them with update() on the root span or update_current_span(). Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * test: run the e2e suites on Langfuse v4 events_only (#1937) * feat(tracing)!: remove trace-level input/output setters Remove the deprecated Langfuse.set_current_trace_io() and span.set_trace_io(), the langfuse.trace.input/output attributes, and their media handling. On Langfuse v4 a trace's input and output come from its root observation: set them with update() on the root span or update_current_span(). Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * test: read e2e assertions through the v4 observations API Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * ci: run e2e against an events_only server Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * test: pin that create_score logs invalid input without enqueueing Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * test: remove the v3 wait_for_trace helper Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * test: keep scalar text IO as strings and read full response_format The v2 observations API returns IO as raw strings, so only JSON objects and arrays are parsed back; text like "2" stays a string. The response_format assertion requests the untruncated metadata value. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * test: dedupe observation rows and assert streamed text as a string The events table can briefly return several rows for one span, so the observation helper keeps the latest row per id. A streamed completion of "2" is text and is now asserted as a string. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(batch-evaluation)!: evaluate observations only Remove the scope parameter and root-observation mode, so batch evaluation always runs on the observations matching the filter and scores each observation. Resume tokens no longer carry a scope. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(batch-evaluation): evaluate duplicate observation rows once The events table can briefly return several rows for one observation, within a page or across a page boundary. Keep the most recently updated row per id so each observation is evaluated and scored once. Also build exception messages before raising. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(tracing)!: require the gen_ai. prefix in the default span filter (#1929) * fix(tracing)!: require the gen_ai. prefix in the default span filter is_genai_span matched any attribute key starting with "gen_ai", so lookalikes such as gen_ai_custom or gen_airflow.model exported unrelated third-party spans. Match only the gen_ai.* namespace. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * chore: re-run CI Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * chore: re-run CI Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * chore: re-run CI Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * chore: re-run CI Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(otel)!: compress span exports with gzip by default (#1954) The default OTLP exporter now gzips span batches unless configured otherwise. Resolution order: the otel_compression argument, LANGFUSE_OTEL_COMPRESSION, OTEL_EXPORTER_OTLP_TRACES_COMPRESSION, OTEL_EXPORTER_OTLP_COMPRESSION, then gzip. Values are case-insensitive and an invalid value falls through to the next setting, matching the JS SDK. Set any of them to "none" to send uncompressed. Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * chore(api)!: drop references to API types removed from the v4 spec (#1955) * chore(api): drop references to API types removed from the v4 spec langfuse.model no longer re-exports DatasetRun, Observation and TraceWithFullDetails, which the regenerated API client removes. The api docstrings point to the v2 observations, scores_v3 and experiments reads instead of trace.get/list and the legacy v1 resources. Two unit tests no longer mock the removed dataset_run_items resource. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * chore: re-run CI Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(api): update API spec from langfuse/langfuse 21b9b33 (#1961) * feat(api): update API spec from langfuse/langfuse 21b9b33 * fix(api): adapt the legacy score-create alias to the CreateScoresRequest union scores.create now takes a request body that is one score or a list. The legacy score_v1.create alias keeps its keyword signature, builds a CreateScoreRequest from the passed arguments, and calls scores.create(request=...). The postprocessor upgrades an alias applied by an earlier template instead of leaving it in place. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> --------- Co-authored-by: langfuse-bot <langfuse-bot@langfuse.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(api): update API spec from langfuse/langfuse 5beee5d (#1962) Co-authored-by: langfuse-bot <langfuse-bot@langfuse.com> * fix(tracing): keep metadata within the span attribute limit (#1957) * fix(tracing): keep metadata within the span attribute limit Each metadata key is written as its own OTel attribute. Once a span holds more than SpanLimits.max_span_attributes (default 128) attributes, the OTel SDK evicts the oldest ones, which are the Langfuse core attributes such as input, model and version. Before writing observation attributes, count the span's existing attribute keys plus the new keys and drop the excess new metadata keys from the tail. Non-metadata attributes and overwrites of existing keys are never dropped. Log one warning per write that dropped keys instead of raising. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(tracing): reserve attribute slots for later observation updates Capping metadata at exactly the span attribute limit left no room for attributes written in later updates, so e.g. update(output=...) made the OTel SDK evict the oldest attribute (langfuse.internal.is_app_root, then input, model, ...). Metadata now also leaves one slot for each reserved observation attribute that is not on the span or in the current write yet, matching langfuse-js. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(tracing): pin span attribute limit in metadata limit tests The tests assert the OTel default of 128 attributes, so a custom OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT or OTEL_ATTRIBUTE_COUNT_LIMIT in the environment made them fail. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(tracing): drop newest attributes instead of evicting on span attribute overflow The Python OTel SDK evicts the oldest attribute once a span is full, while OTel JS drops the newest. After capping metadata, the limit helper now also drops excess new non-metadata keys from the tail when the span has no physical capacity left, so Langfuse writes never evict existing attributes. Overwrites of existing keys are always kept, and at most one warning is logged per write. Langfuse write paths on observation spans now go through the guard: the observation constructor, update(), media-processed updates, set_trace_as_public, the experiment item attribute writes, as_root marking, propagated attributes in the span processor and propagate_attributes, and the app-root marker. The warning now names SpanLimits.max_span_attributes alongside OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT, matching langfuse-js. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(experiments): keep experiment run keys when metadata hits the span limit The task span metadata put experiment_name, experiment_run_name and the dataset ids after item and experiment metadata, so the span attribute limit dropped them first. Insert the run keys first so they survive truncation, and again last so they still win over user keys of the same name. Mirrors langfuse-js. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(tracing): reserve attribute slots by observation type Span-like observations and events never write model, usage, cost, completion start time, model parameters or prompt attributes, so reserving slots for them wasted 7 metadata keys. Reserve the 7 shared keys (type, level, status message, version, environment, input, output) for span-like types and events, and all 14 for generation-like types and for writes whose observation type is unknown (as_root, propagated attributes, app-root marker). Mirrors langfuse-js. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(experiments): propagate experiment metadata as one JSON attribute Experiment metadata and experiment item metadata were propagated as one attribute per flattened key (langfuse.experiment.item.metadata.<path>), so large item metadata filled the task span and its children and left no room for later attributes such as the output. Send each as a single serialized JSON attribute (langfuse.experiment.metadata and langfuse.experiment.item.metadata), like langfuse-js. The server parses the blob at the bare prefix and flattens it to the same paths. Experiment attributes are set by the SDK and skip the 200-character validation of user-propagated values, matching langfuse-js. Remove the now unused _flatten_and_serialize_metadata_values. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(experiments): write task span metadata after the experiment attributes The experiment task span got its observation metadata at start, before the experiment attributes were copied onto it. The metadata cap could then fill the span up to the reserved slots, and the copied attributes used those slots, so the later output was dropped. Start the task span without metadata and set it inside the propagation block, before the task runs, so the cap counts the copied attributes and trims the observation metadata instead. The experiment run keys still go first. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(tracing): mask dict metadata per key when the mask function fails (#1956) * fix(tracing): mask metadata per key when the mask function fails When a custom mask function raised on dict metadata, the fallback string was written to the bare langfuse.observation.metadata attribute. The server parses that attribute as a JSON object, so the fallback was lost, and nothing showed which keys had been masked. For dict metadata, write the fallback under each key of the failed call instead, so it merges with keys from earlier updates like any other update. Non-dict metadata, input and output keep the plain-string fallback. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(tracing): apply mask to propagated trace metadata propagate_attributes now applies the client's mask to each propagated trace metadata value (keys are kept), on the raw value before serialization and the 200 character check. If the mask raises, only that value becomes the fallback string. The masked values are stored in the context and baggage, so the current span, child spans, and outgoing baggage all carry them. Matches langfuse-js #1003. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * docs(tracing): document which client's mask applies to propagated metadata The mask for propagated trace metadata comes from the client for the public key in the execution context, otherwise the only initialized client. With several clients and no public key in context, or with no client yet, propagated metadata is not masked. Document this and the workarounds, and pin the behavior with a test. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * revert(tracing): do not mask propagated trace metadata Propagated metadata is passed explicitly by the developer. Masking it adds client-resolution ambiguity and baggage/length-limit ordering complexity, and diverges from the JS SDK. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(tracing): keep string fallback when the mask fails on empty dict metadata Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * test(api): pin how POST /scores responses parse into the union (#1964) * test(api): pin how POST /scores responses parse into the union Cover the 200, 202 and 207 response shapes of scores.create, sync and async, against a mocked transport. CI also runs them on pydantic 2.7, whose union matching depends on the member order. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * test(api): assert the pydantic 2.7 overlay and describe its coverage Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * feat(tracing)!: JSON-encode observation and trace metadata values (#1958) * fix(tracing): JSON-encode metadata attribute values Per-key `langfuse.observation.metadata.<key>` attributes now carry `json.dumps(value, cls=EventSerializer)` for every value, strings included. Previously str and int were passed raw/natively, so ints beyond int64 made the OTLP encoder drop the key, ints beyond 2^53 lost precision on the server, and a server that JSON-decodes values could not tell the string "123" from the number 123. None values are still skipped so updates keep earlier keys. The bare non-dict metadata attribute is unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat(tracing)!: JSON-encode non-dict metadata too The bare `langfuse.observation.metadata` attribute (non-dict metadata) is now sent as `json.dumps(metadata, cls=EventSerializer)`, strings included, matching the per-key values and langfuse-js #1005. Previously strings were passed through raw, so `metadata="foo"` sent `foo` in Python but `"foo"` in JS. None is still not written. Input/output and other `_serialize` callers are unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(e2e): match metadata expectations to server decoding The server JSON-decodes per-key metadata values only for Python SDK major >= 5 (langfuse/langfuse#18436). On 4.x the experiment items API returns the raw JSON string, so compare against the encoded value until the version bump. test_concurrency now writes an int so it round-trips the same way with and without server-side decoding. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(tracing): write metadata JSON with compact separators and raw non-ASCII Encode observation and trace metadata values with separators=(",", ":") and ensure_ascii=False so the output is byte-identical to JS JSON.stringify, matching propagated metadata (#1932). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat(tracing): debug-log metadata keys skipped because their value is None Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat(tracing)!: JSON-encode propagated trace metadata like observation metadata propagate_attributes(metadata=...) now uses the same encoder as observation metadata, so a string such as "123" stays a string once the server decodes langfuse.trace.metadata.<key>. Ints outside the JS-safe range and NaN/Infinity become JSON strings instead of exact digits or being dropped. The 200 character limit applies to the encoded value. Plain str values skip EventSerializer and go straight to json.dumps; the output is byte-identical and pinned against EventSerializer and JSON.stringify. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(media): extract top-level data URIs from JSON-encoded metadata values Metadata values are JSON-encoded, so a top-level data URI arrives quoted and the exporter no longer recognized it, shipping the base64 inline. Decode the JSON string for Langfuse metadata attributes, process the media, and re-encode the reference string. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * style(tests): format span attribute limit test Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(tracing): drop None propagated metadata values with a warning Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * perf(media): extract JSON-encoded metadata data URIs without json.loads Strip the quotes of a JSON-encoded top-level data URI in place instead of decoding and re-encoding it. Values with backslashes, inner quotes, or no ;base64 header in the first 256 characters are left unchanged. Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * fix(tracing): do not mask attributes the caller never set (#1973) * fix(tracing): do not mask attributes the caller never set `_process_media_and_apply_mask` runs for input, output and metadata on every span creation and update, regardless of whether the caller provided them. An unset attribute is still handed to the user's mask function as `None`. A mask that assumes a dict throws, `_mask_attribute` then fails closed, and the fallback string is not `None` -- so it survives the `v is not None` filters in `create_span_attributes` / `create_generation_attributes`, and lands on the span as an attribute the caller never set. Each unset attribute also logs one ERROR per span, which makes it look like the user's mask function is broken. Repro with a mask that assumes a dict and only `output` set: mask receives [None, None, None, None, {'answer': '42'}, None] span carries langfuse.observation.input / .metadata = '<fully masked due to failed mask function>' plus 5 ERROR logs ("'NoneType' object is not a mapping") Return early when `data is None`: there is nothing to process or mask. Attributes that do have a value keep the existing fail-closed behaviour and the per-key metadata fallback. Adds two regression tests asserting that only attributes the caller set reach the mask, and that unset attributes stay unset. * push --------- Co-authored-by: Hassieb Pakzad <68423100+hassiebp@users.noreply.github.com> * test(tracing): expect JSON-encoded metadata in mask-unset test (#1985) The test asserted raw metadata values, but metadata values are JSON-encoded on prepare-v5-release, so it failed there. Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> * chore: merge main into prepare-v5-release (#1986) * docs(api): document metadata is set / is not set filter operators (#1946) Regenerated langfuse/api from the upstream Fern definition: baseline langfuse/langfuse 619dbe3 (last sync, #1866) plus langfuse/langfuse#17357 and #18404. Split out of the bundled API-spec bot PR #1943. Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(api): type Dataset JSON-schema response fields as objects (#1949) Regenerated langfuse/api from the upstream Fern definition: baseline langfuse/langfuse 619dbe3 (last sync, #1866) plus langfuse/langfuse#17892 and the datasets.yml/opentelemetry.yml part of #18406. Split out of the bundled API-spec bot PR #1943. Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * refactor(api): share cursor pagination meta via CursorMetaResponse (#1948) Regenerated langfuse/api from the upstream Fern definition: baseline langfuse/langfuse 619dbe3 (last sync, #1866) plus langfuse/langfuse#17888. Split out of the bundled API-spec bot PR #1943. Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * docs(api): document fromStartTime as required on experiments endpoints (#1947) * docs(api): document fromStartTime as required on experiments endpoints Regenerated langfuse/api from the upstream Fern definition: baseline langfuse/langfuse 619dbe3 (last sync, #1866) plus langfuse/langfuse#17603. Split out of the bundled API-spec bot PR #1943. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * docs(api): use language-neutral wording for experiments time window Regenerated from langfuse/langfuse branch docs/experiments-from-start-time-neutral-wording. The fromStartTime docstrings named `toStartTime`, but the Python parameter is `to_start_time`; they now say "an upper bound" instead. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(api): type read-side metadata as optional objects (#1950) * fix(api): type read-side metadata as optional objects Regenerated langfuse/api from the upstream Fern definition: baseline langfuse/langfuse 619dbe3 (last sync, #1866) plus langfuse/langfuse#18122 and the commons.yml part of #18406. Split out of the bundled API-spec bot PR #1943. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(api): pin read-side metadata parsing on generated models Cover omitted, null, empty and nested-JSON metadata plus rejection of non-object metadata for Trace, Observation, ObservationV2, BaseScore, BaseScoreV1 and every Score / ScoreV1 / GetScoresResponseData variant. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Niklas Semmler <1880943+niklassemmler@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> * ci: test on Python 3.15 and declare it supported (#1988) * test: run unit tests without langchain and langgraph installed Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> * build: lock dependencies with Python 3.15 wheels Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> * ci: run unit tests on Python 3.15 and declare supported versions Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> * feat(scores)!: send scores via POST /api/public/scores (#1987) * feat(scores)!: send scores via POST /api/public/scores Scores are now batched to the public scores endpoint instead of being sent as score-create events to /api/public/ingestion. The consumer keeps flush_at, flush_interval and the batch byte limit, retries transport errors, 429 and 5xx with backoff, does not retry other 4xx, and logs rejected items from a 207 partial result without retrying them. Scores for traces that the tracer sampler drops are now dropped as well; the trace id was previously never read, so sampling was skipped. The trace-create ingestion path used for trace tags is removed. BREAKING CHANGE: batch score delivery needs Langfuse platform v4.55.0 or later. Custom score timestamps need the platform release that accepts timestamp on POST /scores; older servers record the receive time. Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> * chore(api)!: drop the legacy score create compatibility layer Removes legacy.score_v1.create, the CreateScore* aliases it exposed, and the script that patched them into the generated client. Score deletion through legacy.score_v1.delete is unchanged. BREAKING CHANGE: use api.scores.create to create scores. Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> * chore(integrations)!: stop passing the deprecated usage argument BREAKING CHANGE: OpenAI and LangChain generations only report usage_details. Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> * style: format test_media Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> * test(e2e): expect custom score timestamps to fail until the platform supports them Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> * fix(scores): send naive score timestamps as UTC The scores API rejects timestamps without a UTC offset. Naive datetimes were serialized with the local offset; they are now treated as UTC. Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> * feat(scores): hint at the minimum server version when a score batch gets a 400 Langfuse servers before v4.55.0 reject list bodies on POST /scores. The first 400 in a process now logs that batched score creation requires v4.55.0 or later and that the scores were dropped. Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> * fix(scores): match the JS minimum server version hint and log repeats at debug Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> * refactor(scores): drop the minimum server version hint Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> * fix(scores): keep scores unless the trace-id sampler drops their trace Re-running a parent-based, random or attribute-based sampler with only the trace id can disagree with the decision the trace actually got, so scores are only dropped for Langfuse's trace-id ratio sampler. Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> * fix(scores): cap LANGFUSE_MAX_BATCH_SIZE_BYTES at the scores body limit Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> * feat(api): update API spec from langfuse/langfuse 99da25d (#1990) Co-authored-by: langfuse-bot <langfuse-bot@langfuse.com> * chore: merge main into prepare-v5-release (#1992) * docs(api): document metadata is set / is not set filter operators (#1946) Regenerated langfuse/api from the upstream Fern definition: baseline langfuse/langfuse 619dbe3 (last sync, #1866) plus langfuse/langfuse#17357 and #18404. Split out of the bundled API-spec bot PR #1943. Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(api): type Dataset JSON-schema response fields as objects (#1949) Regenerated langfuse/api from the upstream Fern definition: baseline langfuse/langfuse 619dbe3 (last sync, #1866) plus langfuse/langfuse#17892 and the datasets.yml/opentelemetry.yml part of #18406. Split out of the bundled API-spec bot PR #1943. Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * refactor(api): share cursor pagination meta via CursorMetaResponse (#1948) Regenerated langfuse/api from the upstream Fern definition: baseline langfuse/langfuse 619dbe3 (last sync, #1866) plus langfuse/langfuse#17888. Split out of the bundled API-spec bot PR #1943. Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * docs(api): document fromStartTime as required on experiments endpoints (#1947) * docs(api): document fromStartTime as required on experiments endpoints Regenerated langfuse/api from the upstream Fern definition: baseline langfuse/langfuse 619dbe3 (last sync, #1866) plus langfuse/langfuse#17603. Split out of the bundled API-spec bot PR #1943. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * docs(api): use language-neutral wording for experiments time window Regenerated from langfuse/langfuse branch docs/experiments-from-start-time-neutral-wording. The fromStartTime docstrings named `toStartTime`, but the Python parameter is `to_start_time`; they now say "an upper bound" instead. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(api): type read-side metadata as optional objects (#1950) * fix(api): type read-side metadata as optional objects Regenerated langfuse/api from the upstream Fern definition: baseline langfuse/langfuse 619dbe3 (last sync, #1866) plus langfuse/langfuse#18122 and the commons.yml part of #18406. Split out of the bundled API-spec bot PR #1943. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(api): pin read-side metadata parsing on generated models Cover omitted, null, empty and nested-JSON metadata plus rejection of non-object metadata for Trace, Observation, ObservationV2, BaseScore, BaseScoreV1 and every Score / ScoreV1 / GetScoresResponseData variant. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Niklas Semmler <1880943+niklassemmler@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> * fix: address v5 release review findings (#1993) * fix: budget propagated trace metadata against the span attribute limit Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> * fix: gate experiment run scores on the item span's sampling decision Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> * fix(batch-evaluation): skip boundary duplicates on resume and fetch updatedAt for dedup Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> * style: format span metadata key helper Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> * fix: count experiment items as sampled only after experiment attributes are set Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> * fix(batch-evaluation): request updatedAt when no field groups are given Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> * feat(api): update API spec from langfuse/langfuse 738b05d (#1995) Co-authored-by: langfuse-bot <langfuse-bot@langfuse.com> * refactor(experiments): drop sampling-aware gating of run-level scores (#1996) Co-authored-by: Cursor Agent <cursoragent@cursor.com> * test: read scores via api.scores.get_many in e2e helpers and docstring (#1998) * test: read scores via api.scores.get_many in e2e helpers Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> * docs: point Langfuse.api docstring at api.scores.get_many Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> * feat(api): update API spec from langfuse/langfuse f0ea0c3 (#1999) * feat(api): update API spec from langfuse/langfuse f0ea0c3 * test: drop the single-score case from the scores create response test Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> --------- Co-authored-by: langfuse-bot <langfuse-bot@langfuse.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: hassieb.pakzad <hassieb.pakzad@clickhouse.com> * feat(api): update API spec from langfuse/langfuse 7cb2eed (#2000) Co-authored-by: langfuse-bot <langfuse-bot@langfuse.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com> Co-authored-by: langfuse-bot <accounts+githubbot@langfuse.com> Co-authored-by: langfuse-bot <langfuse-bot@langfuse.com> Co-authored-by: Niklas Semmler <1880943+niklassemmler@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Neumanncheng <93020150+Neumanncheng@users.noreply.github.com>
1 parent 4961816 commit f890d1f

236 files changed

Lines changed: 11514 additions & 19105 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/langfuse-v4-dual-write.override.yml‎

Lines changed: 0 additions & 7 deletions
This file was deleted.

‎.github/workflows/ci.yml‎

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -70,17 +70,23 @@ jobs:
7070
fail-fast: false
7171
matrix:
7272
python-version:
73-
- "3.10"
7473
- "3.11"
7574
- "3.12"
7675
- "3.13"
7776
- "3.14"
77+
- "3.15"
7878

7979
name: Unit tests on Python ${{ matrix.python-version }}
8080
steps:
8181
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
8282
with:
8383
persist-credentials: false
84+
# The pinned uv only knows Python 3.15 pre-releases; install the final release for uv to pick up.
85+
- name: Set up Python 3.15
86+
if: ${{ matrix.python-version == '3.15' }}
87+
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
88+
with:
89+
python-version: "3.15"
8490
- name: Install uv and set Python version
8591
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
8692
with:
@@ -99,6 +105,14 @@ jobs:
99105
python --version
100106
uv run --frozen pytest -n auto --dist worksteal -s -v --log-cli-level=INFO tests/unit
101107
108+
# pydantic < 2.8 matches undiscriminated unions left to right, so the
109+
# CreateScoresResponse member order matters there.
110+
- name: Run score response parsing on pydantic 2.7
111+
if: ${{ matrix.python-version == '3.11' }}
112+
run: |
113+
uv run --frozen --with "pydantic==2.7.4" python -c "import pydantic; assert pydantic.VERSION == '2.7.4', pydantic.VERSION"
114+
uv run --frozen --with "pydantic==2.7.4" pytest -v tests/unit/test_scores_batch_response.py
115+
102116
e2e-tests:
103117
runs-on: ubuntu-latest
104118
timeout-minutes: 30
@@ -160,8 +174,6 @@ jobs:
160174
LANGFUSE_SERVER_SHA="$(git ls-remote https://github.com/langfuse/langfuse.git HEAD | cut -f1)"
161175
curl -fsSL "https://raw.githubusercontent.com/langfuse/langfuse/${LANGFUSE_SERVER_SHA}/docker-compose.yml" \
162176
-o ./langfuse-server/docker-compose.yml
163-
cp ./.github/langfuse-v4-dual-write.override.yml \
164-
./langfuse-server/docker-compose.override.yml
165177
echo "${LANGFUSE_SERVER_SHA}"
166178
167179
- name: Run langfuse server

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,7 @@ Minimum verification matrix:
109109
The main CI workflow currently runs:
110110
- linting on Python 3.13
111111
- mypy on Python 3.13
112-
- `tests/unit` on a Python 3.10-3.14 matrix
112+
- `tests/unit` on a Python 3.11-3.15 matrix
113113
- `tests/e2e` in 2 mechanical shards plus a serial subset inside each shard
114114
- `tests/live_provider` as one always-on suite
115115
- PR title validation for Conventional Commits

‎langfuse/_client/attributes.py‎

Lines changed: 33 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@
2020
)
2121
from langfuse._utils.serializer import EventSerializer
2222
from langfuse.api import MapValue
23+
from langfuse.logger import langfuse_logger
2324
from langfuse.model import PromptClient
2425
from langfuse.types import SpanLevel
2526

@@ -32,8 +33,6 @@ class LangfuseOtelSpanAttributes:
3233
TRACE_TAGS = "langfuse.trace.tags"
3334
TRACE_PUBLIC = "langfuse.trace.public"
3435
TRACE_METADATA = "langfuse.trace.metadata"
35-
TRACE_INPUT = "langfuse.trace.input"
36-
TRACE_OUTPUT = "langfuse.trace.output"
3736

3837
# Langfuse-observation attributes
3938
OBSERVATION_TYPE = "langfuse.observation.type"
@@ -68,20 +67,17 @@ class LangfuseOtelSpanAttributes:
6867
EXPERIMENT_METADATA = "langfuse.experiment.metadata"
6968
EXPERIMENT_DATASET_ID = "langfuse.experiment.dataset.id"
7069
EXPERIMENT_ITEM_ID = "langfuse.experiment.item.id"
70+
EXPERIMENT_ITEM_VERSION = "langfuse.experiment.item.version"
7171
EXPERIMENT_ITEM_EXPECTED_OUTPUT = "langfuse.experiment.item.expected_output"
7272
EXPERIMENT_ITEM_METADATA = "langfuse.experiment.item.metadata"
7373
EXPERIMENT_ITEM_ROOT_OBSERVATION_ID = "langfuse.experiment.item.root_observation_id"
7474

7575

7676
def create_trace_attributes(
7777
*,
78-
input: Optional[Any] = None,
79-
output: Optional[Any] = None,
8078
public: Optional[bool] = None,
8179
) -> dict:
8280
attributes = {
83-
LangfuseOtelSpanAttributes.TRACE_INPUT: _serialize(input),
84-
LangfuseOtelSpanAttributes.TRACE_OUTPUT: _serialize(output),
8581
LangfuseOtelSpanAttributes.TRACE_PUBLIC: public,
8682
}
8783

@@ -165,30 +161,24 @@ def _serialize(obj: Any) -> Optional[str]:
165161
return json.dumps(obj, cls=EventSerializer)
166162

167163

168-
def _flatten_and_serialize_metadata_values(
169-
metadata: Optional[Dict[str, Any]],
170-
) -> Optional[Dict[str, str]]:
171-
if metadata is None:
172-
return None
173-
174-
flattened_metadata: Dict[str, str] = {}
175-
176-
def flatten_value(path: str, value: Any) -> None:
177-
if isinstance(value, dict):
178-
for nested_key, nested_value in value.items():
179-
flatten_value(f"{path}.{nested_key}", nested_value)
164+
def _serialize_metadata_value(value: Any) -> str:
165+
"""JSON-encode one metadata value, strings included.
180166
181-
return
167+
The server decodes metadata values from v5 SDKs, so encoding every value
168+
keeps "123" and 123 distinct. EventSerializer turns ints outside the
169+
JS-safe range and NaN/Infinity into JSON strings and never raises, so one
170+
bad value cannot drop the other keys. Compact separators and raw non-ASCII
171+
match JS JSON.stringify.
172+
"""
173+
# Plain strings skip EventSerializer, which returns them unchanged and then
174+
# uses the same string encoder, so the output is identical but cheaper.
175+
# str subclasses (e.g. str enums) still go through EventSerializer.
176+
if type(value) is str:
177+
return json.dumps(value, ensure_ascii=False)
182178

183-
serialized_value = _serialize(value)
184-
185-
if serialized_value is not None:
186-
flattened_metadata[path] = serialized_value
187-
188-
for key, value in metadata.items():
189-
flatten_value(str(key), value)
190-
191-
return flattened_metadata
179+
return json.dumps(
180+
value, cls=EventSerializer, separators=(",", ":"), ensure_ascii=False
181+
)
192182

193183

194184
def _flatten_and_serialize_metadata(
@@ -200,16 +190,24 @@ def _flatten_and_serialize_metadata(
200190
else LangfuseOtelSpanAttributes.TRACE_METADATA
201191
)
202192

203-
metadata_attributes: Dict[str, Union[str, int, None]] = {}
193+
metadata_attributes: Dict[str, Optional[str]] = {}
204194

205195
if not isinstance(metadata, dict):
206-
metadata_attributes[prefix] = _serialize(metadata)
196+
# JSON-encode non-dict metadata too, strings included, matching the
197+
# per-key values below. None stays None so it is not written.
198+
metadata_attributes[prefix] = (
199+
None if metadata is None else _serialize_metadata_value(metadata)
200+
)
207201
else:
208202
for key, value in metadata.items():
209-
metadata_attributes[f"{prefix}.{key}"] = (
210-
value
211-
if isinstance(value, str) or isinstance(value, int)
212-
else _serialize(value)
213-
)
203+
# Skip None so an update does not overwrite an earlier value.
204+
if value is None:
205+
langfuse_logger.debug(
206+
'Observation metadata key "%s" was not written because its value is None',
207+
key,
208+
)
209+
continue
210+
211+
metadata_attributes[f"{prefix}.{key}"] = _serialize_metadata_value(value)
214212

215213
return metadata_attributes

0 commit comments

Comments
 (0)