[OPIK-8202] [BE] fix: harden ClickHouseAppender against dropped user logs - #8126
Draft
JetoPistola wants to merge 2 commits into
Draft
[OPIK-8202] [BE] fix: harden ClickHouseAppender against dropped user logs#8126JetoPistola wants to merge 2 commits into
JetoPistola wants to merge 2 commits into
GitHub Actions / Backend Tests - Integration Group 14
failed
Sep 3, 2026 in 0s
1 errors, 2 skipped, 357 pass in 4m 11s
Annotations
github-actions / Backend Tests - Integration Group 14
getLogsTraceSkipped (com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResourceTest$GetLogs) with error
apps/opik-backend/target/surefire-reports/TEST-com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResourceTest$GetLogs.xml [took 30s]
Raw output
Assertion condition defined as a Lambda expression in com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResourceTest$GetLogs
Expected size: 1 but was: 0 in:
[] within 30 seconds.
org.awaitility.core.ConditionTimeoutException:
Assertion condition defined as a Lambda expression in com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResourceTest$GetLogs
Expected size: 1 but was: 0 in:
[] within 30 seconds.
at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:119)
at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:31)
at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:1160)
at org.awaitility.core.ConditionFactory.untilAsserted(ConditionFactory.java:790)
at com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResourceTest$GetLogs.getLogsTraceSkipped(AutomationRuleEvaluatorsResourceTest.java:1650)
at java.base/java.lang.reflect.Method.invoke(Method.java:565)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1604)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1604)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1604)
Caused by: java.lang.AssertionError:
Expected size: 1 but was: 0 in:
[]
at com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResourceTest.assertLogPage(AutomationRuleEvaluatorsResourceTest.java:2079)
at com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResourceTest.assertLogResponse(AutomationRuleEvaluatorsResourceTest.java:2086)
at com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResourceTest$GetLogs.lambda$getLogsTraceSkipped$0(AutomationRuleEvaluatorsResourceTest.java:1654)
at org.awaitility.core.AssertionCondition.lambda$new$0(AssertionCondition.java:53)
at org.awaitility.core.ConditionAwaiter$ConditionPoller.call(ConditionAwaiter.java:248)
at org.awaitility.core.ConditionAwaiter$ConditionPoller.call(ConditionAwaiter.java:235)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:328)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1090)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
at java.base/java.lang.Thread.run(Thread.java:1474)
[Client action]---------------------------------------------{
1128 * Sending client request on thread main
1128 > POST http://localhost:34219/v1/private/projects
1128 > Authorization: [redacted]
1128 > Comet-Workspace: workspace-wEg1YRuVHIe6EkJVlLaL
1128 > Content-Type: application/json
{"id":"01a068bd-5720-7451-9dbc-caff9babce54","name":"project-xUoHBlStUt5EWXsiNnnelYzmmsuLjh9XzQWQ","visibility":"public","description":"eYMFvzr61N6xP6E1ngdjcsheaNp99T9OQH6xnYdi2MqvAcbQk1x3bkf6RUCfjx89YmkeoV7S3jXpLRBlGs5KxErvFEomhXQMLm03gEPV2P4hYdV8psPe2c","created_at":"2026-09-03T19:27:14.208917110Z","created_by":"XB1VDxOCPN","last_updated_at":"2026-09-03T19:27:14.209077852Z","last_updated_by":"ymGdTMr0gl","last_updated_trace_at":"2026-09-03T19:27:14.209176301Z","feedback_scores":[{"name":"2nfM0Sw3O8","value":-323746670.194318414},{"name":"gXq_IRob4r","value":607901058.745286226},{"name":"zL72U3SIUW","value":144447026.476954460},{"name":"8aZGf4AM_I","value":-302973942.194626451},{"name":"kMeZaqb8bF","value":-934136309.645113826}],"duration":{"p50":898317453.819683552,"p90":801178201.181892872,"p99":855322743.439553738},"total_estimated_cost":0.16491630577216015,"total_estimated_cost_sum":0.5940410536889386,"usage":{"wX8oIMze2O":0.43037104597014564,"jotEuAiH5E":0.6151440605198445,"NrpavuyNWS":0.5184401069525172,"gW8QYpgWzy":0.749144021219054,"kmeOZINcM5":0.39759096443345227},"trace_count":383945402003,"thread_count":383945433610,"guardrails_failed_count":383945461653,"error_count":{"count":383945495243,"deviation":383945518859,"deviation_percentage":383945541723}}
}----------------------------------------------------------
DEBUG [2026-09-03 19:27:14,212] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:14,214] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:14,214] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
DEBUG [2026-09-03 19:27:14,215] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:14,216] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:14,216] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
INFO [2026-09-03 19:27:14,217] com.comet.opik.api.resources.v1.priv.ProjectsResource: Creating project with name 'project-xUoHBlStUt5EWXsiNnnelYzmmsuLjh9XzQWQ', on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
INFO [2026-09-03 19:27:14,222] com.comet.opik.infrastructure.bi.AnalyticsServiceImpl: Analytics disabled — skipping event 'project_created'
INFO [2026-09-03 19:27:14,259] com.comet.opik.api.resources.v1.priv.ProjectsResource: Created project with name 'project-xUoHBlStUt5EWXsiNnnelYzmmsuLjh9XzQWQ', id '01a068bd-5729-72d6-8201-367327c99027', on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
127.0.0.1 - - [03/Sep/2026:19:27:14 +0000] "POST /v1/private/projects HTTP/1.1" 201 0 "-" "Jersey/3.1.11 (Async HTTP Grizzly Connector 3.1.11)" 46
[Client action]---------------------------------------------{
1128 * Client response received on thread main
1128 < 201
1128 < content-length: 0
1128 < date: Thu, 03 Sep 2026 19:27:14 GMT
1128 < location: http://localhost:34219/v1/private/projects/01a068bd-5729-72d6-8201-367327c99027
1128 < vary: Accept-Encoding
}----------------------------------------------------------
WARN [2026-09-03 19:27:14,260] uk.co.jemos.podam.api.PodamFactoryImpl: Cannot instantiate a class interface java.util.SortedSet. Resorting to uk.co.jemos.podam.api.NullExternalFactory external factory
WARN [2026-09-03 19:27:14,260] uk.co.jemos.podam.api.NullExternalFactory: Cannot instantiate interface java.util.SortedSet with arguments [class com.comet.opik.api.filter.TraceFilter]. Returning null.
[Client action]---------------------------------------------{
1129 * Sending client request on thread main
1129 > POST http://localhost:34219/v1/private/automations/evaluators/
1129 > Accept: application/json
1129 > Authorization: [redacted]
1129 > Comet-Workspace: workspace-wEg1YRuVHIe6EkJVlLaL
1129 > Content-Type: application/json
{"id":"01a068bd-5754-713e-a877-b410e4232311","project_id":"01a068bd-5754-713e-a877-b410e4232312","project_name":"EWmhetTz9O","project_ids":["01a068bd-5729-72d6-8201-367327c99027"],"name":"0_aVOeT3WL","sampling_rate":0.0,"enabled":true,"trigger_scope":"production","filters":[],"code":{"metric":"X3sAtSmWb9","arguments":{"vXsy_ZMinm":"tyjtHiYx9D","6obqau3rWD":"iBi1mmgUq8","lqnYgJEQYO":"4bOjkF4qY4","bgLK9m6EfL":"8vokqHf6Yv","f6R0OpN9Z4":"4djlFfrsdd"}},"created_at":"2026-09-03T19:27:14.263775464Z","created_by":"Wly2pitK93","last_updated_at":"2026-09-03T19:27:14.263881844Z","last_updated_by":"H1xklqU_8N","type":"user_defined_metric_python","action":"evaluator"}
}----------------------------------------------------------
DEBUG [2026-09-03 19:27:14,266] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:14,268] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:14,268] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
DEBUG [2026-09-03 19:27:14,268] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:14,271] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:14,271] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
INFO [2026-09-03 19:27:14,272] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Creating USER_DEFINED_METRIC_PYTHON evaluator for '1' projects on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
DEBUG [2026-09-03 19:27:14,280] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Creating USER_DEFINED_METRIC_PYTHON AutomationRuleEvaluator with id '01a068bd-5760-703b-a90d-7bda4a673617' in projectIds '[01a068bd-5729-72d6-8201-367327c99027]' and workspaceId 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
DEBUG [2026-09-03 19:27:14,284] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Saving 1 project associations for rule '01a068bd-5760-703b-a90d-7bda4a673617'
DEBUG [2026-09-03 19:27:14,287] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Finding AutomationRuleEvaluator with id '01a068bd-5760-703b-a90d-7bda4a673617' in projectIds '[01a068bd-5729-72d6-8201-367327c99027]' and workspaceId 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
DEBUG [2026-09-03 19:27:14,293] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Model before enrichment - id: '01a068bd-5760-703b-a90d-7bda4a673617', projectId: 'null', projectIds: '[01a068bd-5729-72d6-8201-367327c99027]'
DEBUG [2026-09-03 19:27:14,303] com.comet.opik.api.resources.v1.jobs.MetricsAlertJob: Starting metrics alert job
INFO [2026-09-03 19:27:14,303] com.comet.opik.domain.AlertServiceImpl: Fetching all enabled alerts for workspace 'null', eventTypes '[TRACE_ERRORS, TRACE_FEEDBACK_SCORE, TRACE_THREAD_FEEDBACK_SCORE, TRACE_COST, TRACE_LATENCY]'
DEBUG [2026-09-03 19:27:14,305] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Fetched '1' project names for '1' project IDs
DEBUG [2026-09-03 19:27:14,305] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Enriched rule '01a068bd-5760-703b-a90d-7bda4a673617' with 1 projects
DEBUG [2026-09-03 19:27:14,305] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Model after enrichment - id: '01a068bd-5760-703b-a90d-7bda4a673617', projectId: '01a068bd-5729-72d6-8201-367327c99027'
INFO [2026-09-03 19:27:14,306] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Created USER_DEFINED_METRIC_PYTHON evaluator '01a068bd-5760-703b-a90d-7bda4a673617' for '1' projects on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
[Client action]---------------------------------------------{
1129 * Client response received on thread main
1129 < 201
1129 < content-length: 0
1129 < date: Thu, 03 Sep 2026 19:27:14 GMT
1129 < location: http://localhost:34219/v1/private/automations/evaluators/01a068bd-5760-703b-a90d-7bda4a673617
1129 < vary: Accept-Encoding
}----------------------------------------------------------
WARN [2026-09-03 19:27:14,307] uk.co.jemos.podam.api.PodamFactoryImpl: Cannot instantiate a class interface java.util.SortedSet. Resorting to uk.co.jemos.podam.api.NullExternalFactory external factory
WARN [2026-09-03 19:27:14,307] uk.co.jemos.podam.api.NullExternalFactory: Cannot instantiate interface java.util.SortedSet with arguments [class com.comet.opik.api.filter.TraceFilter]. Returning null.
127.0.0.1 - - [03/Sep/2026:19:27:14 +0000] "POST /v1/private/automations/evaluators/ HTTP/1.1" 201 0 "-" "Jersey/3.1.11 (Async HTTP Grizzly Connector 3.1.11)" 41
DEBUG [2026-09-03 19:27:14,310] com.comet.opik.api.resources.v1.jobs.MetricsAlertJob: MetricsAlertJob finished processing all alerts
DEBUG [2026-09-03 19:27:14,315] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
[Client action]---------------------------------------------{
1130 * Sending client request on thread main
1130 > POST http://localhost:34219/v1/private/automations/evaluators/
1130 > Accept: application/json
1130 > Authorization: [redacted]
1130 > Comet-Workspace: workspace-wEg1YRuVHIe6EkJVlLaL
1130 > Content-Type: application/json
{"id":"01a068bd-5783-763a-b905-3f4110a82fc0","project_id":"01a068bd-5783-763a-b905-3f4110a82fc1","project_name":"ZOWMMgF10g","project_ids":["01a068bd-5729-72d6-8201-367327c99027"],"name":"EUyXtlM6C6","sampling_rate":0.0,"enabled":true,"trigger_scope":"experiment","filters":[],"code":{"model":{"name":"pTagGSO6PR","temperature":0.02657347858956325,"seed":1176736080,"custom_parameters":{"97TfoZWfNg":"RzihoXCniU","VhmUh0KgRn":"vUBLvnXy6l","F2wPWX4DIv":"p3TW8S4f2M","FnGadwSLFG":"G71eVJHX9Z","gIBvW1x86i":"glfksplalP"}},"messages":[{"role":"USER","content":"Dpo9CaIGSH","string_content":true,"structured_content":false},{"role":"TOOL_EXECUTION_RESULT","content":"QMrcFhNUgc","string_content":true,"structured_content":false},{"role":"USER","content":"skVMilwWN2","string_content":true,"structured_content":false},{"role":"USER","content":"g61H8Vhc4u","string_content":true,"structured_content":false},{"role":"TOOL_EXECUTION_RESULT","content":"2bBap_eX6c","string_content":true,"structured_content":false}],"variables":{"qgRKIDwA4N":"e41dV0cqYX","DxXukCbwx4":"60t6zKhpO1","ey2ihUss4A":"5CvtRgslf0","awfrFua3Bf":"cf2WpIlRjy","1y6kYFiKBt":"FRQdvT2Yqa"},"schema":[{"name":"reG8b0rVCv","type":"BOOLEAN","description":"FmmZB3B8Ry"},{"name":"XhmnEDxYwm","type":"BOOLEAN","description":"xPwa0Fjvsg"},{"name":"TVHRV0mSlS","type":"INTEGER","description":"xHAReSC7GU"},{"name":"qKhJx0vzHY","type":"BOOLEAN","description":"iGwnml_p93"},{"name":"DKWwr_Is72","type":"DOUBLE","description":"0qm1i_yB0m"}]},"created_at":"2026-09-03T19:27:14.313229539Z","created_by":"2jsb7Cx1OA","last_updated_at":"2026-09-03T19:27:14.313335118Z","last_updated_by":"gD5dPZILCt","type":"llm_as_judge","action":"evaluator"}
}----------------------------------------------------------
DEBUG [2026-09-03 19:27:14,316] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:14,316] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
DEBUG [2026-09-03 19:27:14,316] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:14,320] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:14,320] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
INFO [2026-09-03 19:27:14,321] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Creating LLM_AS_JUDGE evaluator for '1' projects on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
DEBUG [2026-09-03 19:27:14,330] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Creating LLM_AS_JUDGE AutomationRuleEvaluator with id '01a068bd-5791-75f4-824a-88dd119a9567' in projectIds '[01a068bd-5729-72d6-8201-367327c99027]' and workspaceId 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
DEBUG [2026-09-03 19:27:14,335] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Saving 1 project associations for rule '01a068bd-5791-75f4-824a-88dd119a9567'
DEBUG [2026-09-03 19:27:14,337] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Finding AutomationRuleEvaluator with id '01a068bd-5791-75f4-824a-88dd119a9567' in projectIds '[01a068bd-5729-72d6-8201-367327c99027]' and workspaceId 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
DEBUG [2026-09-03 19:27:14,344] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Model before enrichment - id: '01a068bd-5791-75f4-824a-88dd119a9567', projectId: 'null', projectIds: '[01a068bd-5729-72d6-8201-367327c99027]'
DEBUG [2026-09-03 19:27:14,356] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Fetched '1' project names for '1' project IDs
DEBUG [2026-09-03 19:27:14,356] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Enriched rule '01a068bd-5791-75f4-824a-88dd119a9567' with 1 projects
DEBUG [2026-09-03 19:27:14,356] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Model after enrichment - id: '01a068bd-5791-75f4-824a-88dd119a9567', projectId: '01a068bd-5729-72d6-8201-367327c99027'
INFO [2026-09-03 19:27:14,358] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Created LLM_AS_JUDGE evaluator '01a068bd-5791-75f4-824a-88dd119a9567' for '1' projects on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
[Client action]---------------------------------------------{
1130 * Client response received on thread main
1130 < 201
1130 < content-length: 0
1130 < date: Thu, 03 Sep 2026 19:27:14 GMT
1130 < location: http://localhost:34219/v1/private/automations/evaluators/01a068bd-5791-75f4-824a-88dd119a9567
1130 < vary: Accept-Encoding
}----------------------------------------------------------
127.0.0.1 - - [03/Sep/2026:19:27:14 +0000] "POST /v1/private/automations/evaluators/ HTTP/1.1" 201 0 "-" "Jersey/3.1.11 (Async HTTP Grizzly Connector 3.1.11)" 44
DEBUG [2026-09-03 19:27:14,369] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
[Client action]---------------------------------------------{
1131 * Sending client request on thread main
1131 > POST http://localhost:34219/v1/private/traces
1131 > Accept: application/json
1131 > Authorization: [redacted]
1131 > Comet-Workspace: workspace-wEg1YRuVHIe6EkJVlLaL
1131 > Content-Type: application/json
{"id":"01a068bd-57b7-7780-9a18-866515795241","project_name":"project-xUoHBlStUt5EWXsiNnnelYzmmsuLjh9XzQWQ","project_id":"01a068bd-5729-72d6-8201-367327c99027","name":"9yNMxXXPBF","start_time":"2026-09-03T19:27:14.359869759Z","end_time":"2026-09-03T19:27:14.359895117Z","input":{"cPHmzy5Ujz":"qYUyIHqv8C","WVDuDi6nJL":"5MPuWiJwJ6","epazQ8X9qF":"C2TuDiemXm","rPrm8k6JCo":"zKJlVdVxBZ","lOVRYAGmGv":"fV04iKs3OU"},"output":{"0BEtq9uIvd":"YJUVk1yK0B","m9ukenAEDS":"eF8arqGHAh","lvmW8uXF5Q":"RpW1ljQY4x","jCG7gvVOyK":"svPZI1Czhs","VNTBXpHqhE":"YPBWTMex00"},"metadata":{"vrywCF2Ihq":"zIZGTyl7a9","GokTS4fh6s":"bJ94DjM68v","pwtGviV651":"AX5X96lkxs","u3Ykj4qIv2":"zK6QClEpSX","LaNo0eQBL2":"A85uFWI9wW"},"tags":["WeIcFnInV0","y6bDMUtGQu","Z_Fm3BSXkT","SLI4IxZd5U","VcMnXQEYzd"],"error_info":{"exception_type":"gjCD7ZPezF","message":"QkynTixIlN","traceback":"HaXeZGwkKi"},"usage":{"9y_ql4g9Uc":384095674737,"qzmthj8JsO":384095758874,"o6j0bP2uBA":384095416678,"paR6Emi4Y9":384095591562,"19ntg_Ror8":384095505632},"created_at":"2026-09-03T19:27:14.361763523Z","last_updated_at":"2026-09-03T19:27:14.361836272Z","created_by":"5PBNw5AGRl","last_updated_by":"ssRRsVeu4Y","feedback_scores":[{"name":"SmxBaJNEv5","category_name":"H4pZaQdFwa","value":-408459251.023651361,"reason":"Ww2j8QSpuG","source":"online_scoring"},{"name":"_YOnnGinDc","category_name":"kqSkP6lojC","value":-33659034.660187840,"reason":"Cm8h5FHlVC","source":"online_scoring"},{"name":"Scx19FQZrU","category_name":"aEeowPvHQN","value":524868888.915231943,"reason":"_9kTTEYRQx","source":"ui"},{"name":"rVv2HzkI6S","category_name":"jmQWhORGt7","value":579230077.545984507,"reason":"vrgj9hbHqM","source":"online_scoring"},{"name":"RZDaLGcgsq","category_name":"Vh9zmRotzU","value":704293115.690358639,"reason":"PZ42FwfZ9S","source":"sdk"}],"span_feedback_scores":[{"name":"8jUl3mCj90","category_name":"7snbTrCqtF","value":203622854.874070168,"reason":"q11VC6zVcL","source":"online_scoring"},{"name":"wUwr7xbE2W","category_name":"sTtYnHfwwi","value":173776518.319621086,"reason":"LnBXvs6ov3","source":"online_scoring"},{"name":"Q2H6VlzALl","category_name":"UyOWaje7_c","value":-494130422.126453638,"reason":"zdfKSyA1Tc","source":"online_scoring"},{"name":"RAa74m7_m2","category_name":"_PqFkot0Y1","value":383161650.460246325,"reason":"Dt5LPO34fp","source":"ui"},{"name":"32BBD9L39A","category_name":"t6LzJJCaX4","value":-126084369.951677084,"reason":"Wu6mBMDxXW","source":"sdk"}],"comments":[{"id":"01a068bd-57ba-7294-af21-736d6ad216db","text":"zPP6lJMEns","source_queue_id":"01a068bd-57ba-7294-af21-736d6ad216dc","created_at":"2026-09-03T19:27:14.362566813Z","last_updated_at":"2026-09-03T19:27:14.362604871Z","created_by":"PjTKuApSFL","last_updated_by":"LCnO7GRuj8"},{"id":"01a068bd-57ba-7294-af21-736d6ad216dd","text":"JsrIfCkCC_","source_queue_id":"01a068bd-57ba-7294-af21-736d6ad216de","created_at":"2026-09-03T19:27:14.362981829Z","last_updated_at":"2026-09-03T19:27:14.363019495Z","created_by":"mrSBIp_kVZ","last_updated_by":"MyYs5X6Xra"},{"id":"01a068bd-57bb-72ba-b25a-081cc2814df7","text":"gOh2UFW9RQ","source_queue_id":"01a068bd-57bb-72ba-b25a-081cc2814df8","created_at":"2026-09-03T19:27:14.363430335Z","last_updated_at":"2026-09-03T19:27:14.363466409Z","created_by":"gYHS38yuFY","last_updated_by":"6s4IxmYmyn"},{"id":"01a068bd-57bb-72ba-b25a-081cc2814df9","text":"W_jIJ86U2k","source_queue_id":"01a068bd-57bb-72ba-b25a-081cc2814dfa","created_at":"2026-09-03T19:27:14.363783967Z","last_updated_at":"2026-09-03T19:27:14.363819831Z","created_by":"y71WgYESFM","last_updated_by":"CvMOq2EyAw"},{"id":"01a068bd-57bc-706c-80d3-fc0fed8a25c4","text":"wWDAoQsH0l","source_queue_id":"01a068bd-57bc-706c-80d3-fc0fed8a25c5","created_at":"2026-09-03T19:27:14.364187776Z","last_updated_at":"2026-09-03T19:27:14.364225883Z","created_by":"rwxf4WtR1d","last_updated_by":"Bzb4oLZy4A"}],"guardrails_validations":[{"span_id":"01a068bd-57bc-706c-80d3-fc0fed8a25c6","checks":[{"name":"TOPIC","result":"failed"},{"name":"LLM_JUDGE","result":"failed"},{"name":"LLM_JUDGE","result":"passed"},{"name":"PII","result":"passed"},{"name":"CUSTOM_CLASSIFIER","result":"passed"}]},{"span_id":"01a068bd-57bc-706c-80d3-fc0fed8a25c7","checks":[{"name":"CUSTOM_CLASSIFIER","result":"passed"},{"name":"PROMPT_INJECTION","result":"passed"},{"name":"PII","result":"passed"},{"name":"PROMPT_INJECTION","result":"passed"},{"name":"TOPIC","result":"passed"}]},{"span_id":"01a068bd-57bd-776f-95ce-4cca7449bc12","checks":[{"name":"TOPIC","result":"failed"},{"name":"LLM_JUDGE","result":"passed"},{"name":"TOPIC","result":"passed"},{"name":"LLM_JUDGE","result":"passed"},{"name":"TOPIC","result":"failed"}]},{"span_id":"01a068bd-57be-7463-836e-5f082d95e05e","checks":[{"name":"LLM_JUDGE","result":"passed"},{"name":"TOPIC","result":"passed"},{"name":"PII","result":"passed"},{"name":"TOPIC","result":"passed"},{"name":"TOPIC","result":"passed"}]},{"span_id":"01a068bd-57be-7463-836e-5f082d95e05f","checks":[{"name":"CUSTOM_CLASSIFIER","result":"passed"},{"name":"CUSTOM_CLASSIFIER","result":"failed"},{"name":"PROMPT_INJECTION","result":"passed"},{"name":"CUSTOM_CLASSIFIER","result":"passed"},{"name":"PII","result":"passed"}]}],"total_estimated_cost":-348399058.147834420,"span_count":1827999357,"duration":0.32019813324468166,"ttft":7.229144890197622E8,"visibility_mode":"default","llm_span_count":278142663,"has_tool_spans":true,"providers":["NSoMOxU1hL","ap_vtsnpgY","EY9nr8ryag","JF2B7xQbio","l7aSDEsLCE"],"experiment":{"id":"01a068bd-57c0-716f-a51f-ea589b771aa3","name":"dVXUItPexO","dataset_id":"01a068bd-57c0-716f-a51f-ea589b771aa4","dataset_item_id":"01a068bd-57c0-716f-a51f-ea589b771aa5"},"environment":"0SKa7OZveOmBDZkxokdWdUWVeaS9_JEZY3RWgMMtr5yj472TIdYaxZY1gIYxMUkUrHAul5SCl9BYRtUfNwkrxLBxRF4mjsNPZ7c31eQjN8wtmuku0Bx35yAmYnN3P"}
}----------------------------------------------------------
DEBUG [2026-09-03 19:27:14,373] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:14,373] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
DEBUG [2026-09-03 19:27:14,373] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:14,375] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:14,375] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
INFO [2026-09-03 19:27:14,376] com.comet.opik.api.resources.v1.priv.TracesResource: Creating trace with id '01a068bd-57b7-7780-9a18-866515795241', projectName 'project-xUoHBlStUt5EWXsiNnnelYzmmsuLjh9XzQWQ' on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
DEBUG [2026-09-03 19:27:14,387] com.comet.opik.infrastructure.redis.RedissonLockService: Trying to lock with 'Lock[key=01a068bd-57b7-7780-9a18-866515795241-Trace]'
INFO [2026-09-03 19:27:14,390] com.comet.opik.domain.TraceDAOImpl: Getting partial trace by id '01a068bd-57b7-7780-9a18-866515795241'
DEBUG [2026-09-03 19:27:14,390] com.comet.opik.infrastructure.redis.RedissonLockService: Lock acquired 'Lock[key=01a068bd-57b7-7780-9a18-866515795241-Trace]'
INFO [2026-09-03 19:27:14,428] com.comet.opik.api.resources.v1.priv.TracesResource: Created trace with id '01a068bd-57b7-7780-9a18-866515795241', projectName 'project-xUoHBlStUt5EWXsiNnnelYzmmsuLjh9XzQWQ' on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
INFO [2026-09-03 19:27:14,428] com.comet.opik.api.resources.v1.events.OnlineScoringSampler: Received TracesCreated, complete '1', total '1', workspace 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
127.0.0.1 - - [03/Sep/2026:19:27:14 +0000] "POST /v1/private/traces HTTP/1.1" 201 0 "-" "Jersey/3.1.11 (Async HTTP Grizzly Connector 3.1.11)" 59
INFO [2026-09-03 19:27:14,430] com.comet.opik.infrastructure.events.EventInterceptor: Event intercepted: TracesCreated
INFO [2026-09-03 19:27:14,430] com.comet.opik.api.resources.v1.events.TraceThreadListener: Received TracesCreated event for workspace: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', projectIds: '[[01a068bd-5729-72d6-8201-367327c99027]]'. Processing trace thread ingestion
INFO [2026-09-03 19:27:14,430] com.comet.opik.api.resources.v1.events.TraceThreadListener: Completed processing TracesCreated event for workspace: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', projectIds: '[01a068bd-5729-72d6-8201-367327c99027]'
INFO [2026-09-03 19:27:14,430] com.comet.opik.infrastructure.events.EventInterceptor: Event intercepted: TracesCreated
INFO [2026-09-03 19:27:14,428] com.comet.opik.api.resources.v1.events.OnlineScoringSampler: Scoring traces, count '1', workspace 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', projects '{projectId: 01a068bd-5729-72d6-8201-367327c99027=1}'
INFO [2026-09-03 19:27:14,430] com.comet.opik.api.resources.v1.events.OnlineScoringSampler: Fetching evaluators, traces '1', project '01a068bd-5729-72d6-8201-367327c99027', workspace 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
INFO [2026-09-03 19:27:14,430] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Finding AutomationRuleEvaluators, projectId '01a068bd-5729-72d6-8201-367327c99027', workspaceId 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', type 'null'
DEBUG [2026-09-03 19:27:14,430] com.comet.opik.infrastructure.redis.RedissonLockService: Lock released
DEBUG [2026-09-03 19:27:14,430] com.comet.opik.infrastructure.redis.RedissonLockService: Trying to lock with 'Lock[key=ce45bdc0-b2df-421e-97c2-1fac62f1bc90-environment_create]'
INFO [2026-09-03 19:27:14,431] com.comet.opik.infrastructure.events.EventInterceptor: Event intercepted: TracesCreated
INFO [2026-09-03 19:27:14,432] com.comet.opik.infrastructure.events.EventInterceptor: Event intercepted: TracesCreated
DEBUG [2026-09-03 19:27:14,432] com.comet.opik.infrastructure.redis.RedissonLockService: Lock acquired 'Lock[key=ce45bdc0-b2df-421e-97c2-1fac62f1bc90-environment_create]'
INFO [2026-09-03 19:27:14,432] com.comet.opik.infrastructure.events.EventInterceptor: Event intercepted: TracesCreated
[Client action]---------------------------------------------{
1131 * Client response received on thread main
1131 < 201
1131 < content-length: 0
1131 < date: Thu, 03 Sep 2026 19:27:14 GMT
1131 < location: http://localhost:34219/v1/private/traces/01a068bd-57b7-7780-9a18-866515795241
1131 < vary: Accept-Encoding
}----------------------------------------------------------
DEBUG [2026-09-03 19:27:14,443] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Found 2 evaluators for projectId '01a068bd-5729-72d6-8201-367327c99027', workspaceId 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', type 'null'
DEBUG [2026-09-03 19:27:14,443] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Model before enrichment - id: '01a068bd-5791-75f4-824a-88dd119a9567', projectId: 'null', projectIds: '[01a068bd-5729-72d6-8201-367327c99027]'
DEBUG [2026-09-03 19:27:14,444] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Model before enrichment - id: '01a068bd-5760-703b-a90d-7bda4a673617', projectId: 'null', projectIds: '[01a068bd-5729-72d6-8201-367327c99027]'
WARN [2026-09-03 19:27:14,447] com.comet.opik.api.resources.v1.events.ExperimentAggregateEventListener: No finished experiments to publish for workspace 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
INFO [2026-09-03 19:27:14,447] com.comet.opik.domain.EnvironmentServiceImpl: Skipping environment auto-create for workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90': cap '5' reached
WARN [2026-09-03 19:27:14,449] com.comet.opik.api.resources.v1.events.TestSuiteAssertionSampler: No LLM model resolved for test suite batch in workspace 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90' — no supported provider connected and no test_suite_model in trace metadata
INFO [2026-09-03 19:27:14,449] com.comet.opik.api.resources.v1.events.ProjectEventListener: Recorded last traces for projects '[01a068bd-5729-72d6-8201-367327c99027]'
INFO [2026-09-03 19:27:14,449] com.comet.opik.infrastructure.events.EventInterceptor: Event intercepted: TracesCreated
INFO [2026-09-03 19:27:14,449] com.comet.opik.infrastructure.events.EventInterceptor: Event intercepted: TracesCreated
DEBUG [2026-09-03 19:27:14,452] com.comet.opik.infrastructure.redis.RedissonLockService: Lock released
DEBUG [2026-09-03 19:27:14,455] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Fetched '1' project names for '1' project IDs
DEBUG [2026-09-03 19:27:14,455] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Enriched rule '01a068bd-5791-75f4-824a-88dd119a9567' with 1 projects
DEBUG [2026-09-03 19:27:14,455] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Enriched rule '01a068bd-5760-703b-a90d-7bda4a673617' with 1 projects
DEBUG [2026-09-03 19:27:14,455] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Model after enrichment - id: '01a068bd-5791-75f4-824a-88dd119a9567', projectId: '01a068bd-5729-72d6-8201-367327c99027'
DEBUG [2026-09-03 19:27:14,455] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorServiceImpl: Model after enrichment - id: '01a068bd-5760-703b-a90d-7bda4a673617', projectId: '01a068bd-5729-72d6-8201-367327c99027'
INFO [2026-09-03 19:27:14,456] com.comet.opik.api.resources.v1.events.OnlineS…673617' on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
INFO [2026-09-03 19:27:43,912] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorLogsDAOImpl: Finding logs with criteria: LogCriteria[entityId=01a068bd-5760-703b-a90d-7bda4a673617, level=null, size=1000, page=null, markers=null]
DEBUG [2026-09-03 19:27:43,912] com.comet.opik.utils.template.TemplateLogger: Template runtime error: context [anonymous] 3:4 attribute level isn't defined
DEBUG [2026-09-03 19:27:43,913] com.comet.opik.utils.template.TemplateLogger: Template runtime error: context [anonymous] 6:36 attribute offset isn't defined
INFO [2026-09-03 19:27:43,917] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Found 1 logs for automated evaluator: id '01a068bd-5760-703b-a90d-7bda4a673617' on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
127.0.0.1 - - [03/Sep/2026:19:27:43 +0000] "GET /v1/private/automations/evaluators/01a068bd-5760-703b-a90d-7bda4a673617/logs HTTP/1.1" 200 344 "-" "Jersey/3.1.11 (Async HTTP Grizzly Connector 3.1.11)" 8
[Client action]---------------------------------------------{
1625 * Client response received on thread awaitility-thread
1625 < 200
1625 < content-length: 344
1625 < content-type: application/json
1625 < date: Thu, 03 Sep 2026 19:27:43 GMT
1625 < vary: Accept-Encoding
{"content":[{"timestamp":"2026-09-03T19:27:14.456727008Z","rule_id":"01a068bd-5760-703b-a90d-7bda4a673617","level":"INFO","message":"The traceId '01a068bd-57b7-7780-9a18-866515795241' was skipped for rule: '0_aVOeT3WL' and per the sampling rate '0.0'","markers":{"trace_id":"01a068bd-57b7-7780-9a18-866515795241"}}],"page":1,"size":1,"total":1}
}----------------------------------------------------------
[Client action]---------------------------------------------{
1626 * Sending client request on thread awaitility-thread
1626 > GET http://localhost:34219/v1/private/automations/evaluators/01a068bd-5791-75f4-824a-88dd119a9567/logs
1626 > Accept: application/json
1626 > Authorization: [redacted]
1626 > Comet-Workspace: workspace-wEg1YRuVHIe6EkJVlLaL
}----------------------------------------------------------
DEBUG [2026-09-03 19:27:43,919] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:43,920] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:43,920] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
DEBUG [2026-09-03 19:27:43,920] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:43,920] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:43,920] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
INFO [2026-09-03 19:27:43,920] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Looking for logs for automated evaluator: id '01a068bd-5791-75f4-824a-88dd119a9567' on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
INFO [2026-09-03 19:27:43,921] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorLogsDAOImpl: Finding logs with criteria: LogCriteria[entityId=01a068bd-5791-75f4-824a-88dd119a9567, level=null, size=1000, page=null, markers=null]
DEBUG [2026-09-03 19:27:43,921] com.comet.opik.utils.template.TemplateLogger: Template runtime error: context [anonymous] 3:4 attribute level isn't defined
DEBUG [2026-09-03 19:27:43,921] com.comet.opik.utils.template.TemplateLogger: Template runtime error: context [anonymous] 6:36 attribute offset isn't defined
INFO [2026-09-03 19:27:43,954] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Found 0 logs for automated evaluator: id '01a068bd-5791-75f4-824a-88dd119a9567' on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
[Client action]---------------------------------------------{
1626 * Client response received on thread awaitility-thread
1626 < 200
1626 < content-length: 42
1626 < content-type: application/json
1626 < date: Thu, 03 Sep 2026 19:27:43 GMT
1626 < vary: Accept-Encoding
{"content":[],"page":1,"size":0,"total":0}
}----------------------------------------------------------
127.0.0.1 - - [03/Sep/2026:19:27:43 +0000] "GET /v1/private/automations/evaluators/01a068bd-5791-75f4-824a-88dd119a9567/logs HTTP/1.1" 200 42 "-" "Jersey/3.1.11 (Async HTTP Grizzly Connector 3.1.11)" 36
DEBUG [2026-09-03 19:27:44,046] com.comet.opik.api.resources.v1.events.OnlineScoringUserDefinedMetricPythonScorer: Backpressure drop detected: Unable to keep up with polling intervals. Polling interval tick dropped (sequence number: '221').
DEBUG [2026-09-03 19:27:44,048] com.comet.opik.api.resources.v1.events.OnlineScoringUserDefinedMetricPythonScorer: Successfully read from stream, size '0'
[Client action]---------------------------------------------{
1627 * Sending client request on thread awaitility-thread
1627 > GET http://localhost:34219/v1/private/automations/evaluators/01a068bd-5760-703b-a90d-7bda4a673617/logs
1627 > Accept: application/json
1627 > Authorization: [redacted]
1627 > Comet-Workspace: workspace-wEg1YRuVHIe6EkJVlLaL
}----------------------------------------------------------
DEBUG [2026-09-03 19:27:44,057] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:44,058] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:44,058] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
DEBUG [2026-09-03 19:27:44,058] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:44,059] com.comet.opik.api.resources.v1.events.OnlineScoringLlmAsJudgeScorer: Backpressure drop detected: Unable to keep up with polling intervals. Polling interval tick dropped (sequence number: '221').
DEBUG [2026-09-03 19:27:44,059] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:44,059] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
INFO [2026-09-03 19:27:44,059] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Looking for logs for automated evaluator: id '01a068bd-5760-703b-a90d-7bda4a673617' on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
INFO [2026-09-03 19:27:44,060] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorLogsDAOImpl: Finding logs with criteria: LogCriteria[entityId=01a068bd-5760-703b-a90d-7bda4a673617, level=null, size=1000, page=null, markers=null]
DEBUG [2026-09-03 19:27:44,060] com.comet.opik.utils.template.TemplateLogger: Template runtime error: context [anonymous] 3:4 attribute level isn't defined
DEBUG [2026-09-03 19:27:44,060] com.comet.opik.utils.template.TemplateLogger: Template runtime error: context [anonymous] 6:36 attribute offset isn't defined
INFO [2026-09-03 19:27:44,065] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Found 1 logs for automated evaluator: id '01a068bd-5760-703b-a90d-7bda4a673617' on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
127.0.0.1 - - [03/Sep/2026:19:27:44 +0000] "GET /v1/private/automations/evaluators/01a068bd-5760-703b-a90d-7bda4a673617/logs HTTP/1.1" 200 344 "-" "Jersey/3.1.11 (Async HTTP Grizzly Connector 3.1.11)" 9
[Client action]---------------------------------------------{
1627 * Client response received on thread awaitility-thread
1627 < 200
1627 < content-length: 344
1627 < content-type: application/json
1627 < date: Thu, 03 Sep 2026 19:27:44 GMT
1627 < vary: Accept-Encoding
{"content":[{"timestamp":"2026-09-03T19:27:14.456727008Z","rule_id":"01a068bd-5760-703b-a90d-7bda4a673617","level":"INFO","message":"The traceId '01a068bd-57b7-7780-9a18-866515795241' was skipped for rule: '0_aVOeT3WL' and per the sampling rate '0.0'","markers":{"trace_id":"01a068bd-57b7-7780-9a18-866515795241"}}],"page":1,"size":1,"total":1}
}----------------------------------------------------------
[Client action]---------------------------------------------{
1628 * Sending client request on thread awaitility-thread
1628 > GET http://localhost:34219/v1/private/automations/evaluators/01a068bd-5791-75f4-824a-88dd119a9567/logs
1628 > Accept: application/json
1628 > Authorization: [redacted]
1628 > Comet-Workspace: workspace-wEg1YRuVHIe6EkJVlLaL
}----------------------------------------------------------
DEBUG [2026-09-03 19:27:44,067] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:44,068] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:44,068] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
DEBUG [2026-09-03 19:27:44,068] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:44,069] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:44,069] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
INFO [2026-09-03 19:27:44,069] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Looking for logs for automated evaluator: id '01a068bd-5791-75f4-824a-88dd119a9567' on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
INFO [2026-09-03 19:27:44,069] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorLogsDAOImpl: Finding logs with criteria: LogCriteria[entityId=01a068bd-5791-75f4-824a-88dd119a9567, level=null, size=1000, page=null, markers=null]
DEBUG [2026-09-03 19:27:44,069] com.comet.opik.utils.template.TemplateLogger: Template runtime error: context [anonymous] 3:4 attribute level isn't defined
DEBUG [2026-09-03 19:27:44,069] com.comet.opik.utils.template.TemplateLogger: Template runtime error: context [anonymous] 6:36 attribute offset isn't defined
INFO [2026-09-03 19:27:44,072] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Found 0 logs for automated evaluator: id '01a068bd-5791-75f4-824a-88dd119a9567' on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
[Client action]---------------------------------------------{
1628 * Client response received on thread awaitility-thread
1628 < 200
1628 < content-length: 42
1628 < content-type: application/json
1628 < date: Thu, 03 Sep 2026 19:27:44 GMT
1628 < vary: Accept-Encoding
{"content":[],"page":1,"size":0,"total":0}
}----------------------------------------------------------
127.0.0.1 - - [03/Sep/2026:19:27:44 +0000] "GET /v1/private/automations/evaluators/01a068bd-5791-75f4-824a-88dd119a9567/logs HTTP/1.1" 200 42 "-" "Jersey/3.1.11 (Async HTTP Grizzly Connector 3.1.11)" 6
[Client action]---------------------------------------------{
1629 * Sending client request on thread awaitility-thread
1629 > GET http://localhost:34219/v1/private/automations/evaluators/01a068bd-5760-703b-a90d-7bda4a673617/logs
1629 > Accept: application/json
1629 > Authorization: [redacted]
1629 > Comet-Workspace: workspace-wEg1YRuVHIe6EkJVlLaL
}----------------------------------------------------------
DEBUG [2026-09-03 19:27:44,174] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:44,175] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:44,175] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
DEBUG [2026-09-03 19:27:44,175] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:44,176] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:44,176] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
INFO [2026-09-03 19:27:44,176] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Looking for logs for automated evaluator: id '01a068bd-5760-703b-a90d-7bda4a673617' on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
INFO [2026-09-03 19:27:44,176] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorLogsDAOImpl: Finding logs with criteria: LogCriteria[entityId=01a068bd-5760-703b-a90d-7bda4a673617, level=null, size=1000, page=null, markers=null]
DEBUG [2026-09-03 19:27:44,177] com.comet.opik.utils.template.TemplateLogger: Template runtime error: context [anonymous] 3:4 attribute level isn't defined
DEBUG [2026-09-03 19:27:44,177] com.comet.opik.utils.template.TemplateLogger: Template runtime error: context [anonymous] 6:36 attribute offset isn't defined
INFO [2026-09-03 19:27:44,182] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Found 1 logs for automated evaluator: id '01a068bd-5760-703b-a90d-7bda4a673617' on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
127.0.0.1 - - [03/Sep/2026:19:27:44 +0000] "GET /v1/private/automations/evaluators/01a068bd-5760-703b-a90d-7bda4a673617/logs HTTP/1.1" 200 344 "-" "Jersey/3.1.11 (Async HTTP Grizzly Connector 3.1.11)" 8
[Client action]---------------------------------------------{
1629 * Client response received on thread awaitility-thread
1629 < 200
1629 < content-length: 344
1629 < content-type: application/json
1629 < date: Thu, 03 Sep 2026 19:27:44 GMT
1629 < vary: Accept-Encoding
{"content":[{"timestamp":"2026-09-03T19:27:14.456727008Z","rule_id":"01a068bd-5760-703b-a90d-7bda4a673617","level":"INFO","message":"The traceId '01a068bd-57b7-7780-9a18-866515795241' was skipped for rule: '0_aVOeT3WL' and per the sampling rate '0.0'","markers":{"trace_id":"01a068bd-57b7-7780-9a18-866515795241"}}],"page":1,"size":1,"total":1}
}----------------------------------------------------------
[Client action]---------------------------------------------{
1630 * Sending client request on thread awaitility-thread
1630 > GET http://localhost:34219/v1/private/automations/evaluators/01a068bd-5791-75f4-824a-88dd119a9567/logs
1630 > Accept: application/json
1630 > Authorization: [redacted]
1630 > Comet-Workspace: workspace-wEg1YRuVHIe6EkJVlLaL
}----------------------------------------------------------
DEBUG [2026-09-03 19:27:44,183] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:44,184] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:44,184] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
DEBUG [2026-09-03 19:27:44,184] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:44,185] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:44,185] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
INFO [2026-09-03 19:27:44,185] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Looking for logs for automated evaluator: id '01a068bd-5791-75f4-824a-88dd119a9567' on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
INFO [2026-09-03 19:27:44,185] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorLogsDAOImpl: Finding logs with criteria: LogCriteria[entityId=01a068bd-5791-75f4-824a-88dd119a9567, level=null, size=1000, page=null, markers=null]
DEBUG [2026-09-03 19:27:44,186] com.comet.opik.utils.template.TemplateLogger: Template runtime error: context [anonymous] 3:4 attribute level isn't defined
DEBUG [2026-09-03 19:27:44,186] com.comet.opik.utils.template.TemplateLogger: Template runtime error: context [anonymous] 6:36 attribute offset isn't defined
INFO [2026-09-03 19:27:44,189] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Found 0 logs for automated evaluator: id '01a068bd-5791-75f4-824a-88dd119a9567' on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
[Client action]---------------------------------------------{
1630 * Client response received on thread awaitility-thread
1630 < 200
1630 < content-length: 42
1630 < content-type: application/json
1630 < date: Thu, 03 Sep 2026 19:27:44 GMT
1630 < vary: Accept-Encoding
{"content":[],"page":1,"size":0,"total":0}
}----------------------------------------------------------
127.0.0.1 - - [03/Sep/2026:19:27:44 +0000] "GET /v1/private/automations/evaluators/01a068bd-5791-75f4-824a-88dd119a9567/logs HTTP/1.1" 200 42 "-" "Jersey/3.1.11 (Async HTTP Grizzly Connector 3.1.11)" 6
[Client action]---------------------------------------------{
1631 * Sending client request on thread awaitility-thread
1631 > GET http://localhost:34219/v1/private/automations/evaluators/01a068bd-5760-703b-a90d-7bda4a673617/logs
1631 > Accept: application/json
1631 > Authorization: [redacted]
1631 > Comet-Workspace: workspace-wEg1YRuVHIe6EkJVlLaL
}----------------------------------------------------------
DEBUG [2026-09-03 19:27:44,290] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:44,291] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:44,291] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
DEBUG [2026-09-03 19:27:44,291] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:44,292] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:44,292] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
INFO [2026-09-03 19:27:44,292] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Looking for logs for automated evaluator: id '01a068bd-5760-703b-a90d-7bda4a673617' on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
INFO [2026-09-03 19:27:44,293] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorLogsDAOImpl: Finding logs with criteria: LogCriteria[entityId=01a068bd-5760-703b-a90d-7bda4a673617, level=null, size=1000, page=null, markers=null]
DEBUG [2026-09-03 19:27:44,293] com.comet.opik.utils.template.TemplateLogger: Template runtime error: context [anonymous] 3:4 attribute level isn't defined
DEBUG [2026-09-03 19:27:44,293] com.comet.opik.utils.template.TemplateLogger: Template runtime error: context [anonymous] 6:36 attribute offset isn't defined
INFO [2026-09-03 19:27:44,297] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Found 1 logs for automated evaluator: id '01a068bd-5760-703b-a90d-7bda4a673617' on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
127.0.0.1 - - [03/Sep/2026:19:27:44 +0000] "GET /v1/private/automations/evaluators/01a068bd-5760-703b-a90d-7bda4a673617/logs HTTP/1.1" 200 344 "-" "Jersey/3.1.11 (Async HTTP Grizzly Connector 3.1.11)" 7
[Client action]---------------------------------------------{
1631 * Client response received on thread awaitility-thread
1631 < 200
1631 < content-length: 344
1631 < content-type: application/json
1631 < date: Thu, 03 Sep 2026 19:27:44 GMT
1631 < vary: Accept-Encoding
{"content":[{"timestamp":"2026-09-03T19:27:14.456727008Z","rule_id":"01a068bd-5760-703b-a90d-7bda4a673617","level":"INFO","message":"The traceId '01a068bd-57b7-7780-9a18-866515795241' was skipped for rule: '0_aVOeT3WL' and per the sampling rate '0.0'","markers":{"trace_id":"01a068bd-57b7-7780-9a18-866515795241"}}],"page":1,"size":1,"total":1}
}----------------------------------------------------------
[Client action]---------------------------------------------{
1632 * Sending client request on thread awaitility-thread
1632 > GET http://localhost:34219/v1/private/automations/evaluators/01a068bd-5791-75f4-824a-88dd119a9567/logs
1632 > Accept: application/json
1632 > Authorization: [redacted]
1632 > Comet-Workspace: workspace-wEg1YRuVHIe6EkJVlLaL
}----------------------------------------------------------
DEBUG [2026-09-03 19:27:44,299] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:44,299] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:44,299] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
DEBUG [2026-09-03 19:27:44,299] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:44,300] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:44,300] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
INFO [2026-09-03 19:27:44,300] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Looking for logs for automated evaluator: id '01a068bd-5791-75f4-824a-88dd119a9567' on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
INFO [2026-09-03 19:27:44,301] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorLogsDAOImpl: Finding logs with criteria: LogCriteria[entityId=01a068bd-5791-75f4-824a-88dd119a9567, level=null, size=1000, page=null, markers=null]
DEBUG [2026-09-03 19:27:44,301] com.comet.opik.utils.template.TemplateLogger: Template runtime error: context [anonymous] 3:4 attribute level isn't defined
DEBUG [2026-09-03 19:27:44,301] com.comet.opik.utils.template.TemplateLogger: Template runtime error: context [anonymous] 6:36 attribute offset isn't defined
DEBUG [2026-09-03 19:27:44,303] com.comet.opik.api.resources.v1.jobs.MetricsAlertJob: Starting metrics alert job
INFO [2026-09-03 19:27:44,303] com.comet.opik.domain.AlertServiceImpl: Fetching all enabled alerts for workspace 'null', eventTypes '[TRACE_ERRORS, TRACE_FEEDBACK_SCORE, TRACE_THREAD_FEEDBACK_SCORE, TRACE_COST, TRACE_LATENCY]'
INFO [2026-09-03 19:27:44,304] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Found 0 logs for automated evaluator: id '01a068bd-5791-75f4-824a-88dd119a9567' on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
127.0.0.1 - - [03/Sep/2026:19:27:44 +0000] "GET /v1/private/automations/evaluators/01a068bd-5791-75f4-824a-88dd119a9567/logs HTTP/1.1" 200 42 "-" "Jersey/3.1.11 (Async HTTP Grizzly Connector 3.1.11)" 6
[Client action]---------------------------------------------{
1632 * Client response received on thread awaitility-thread
1632 < 200
1632 < content-length: 42
1632 < content-type: application/json
1632 < date: Thu, 03 Sep 2026 19:27:44 GMT
1632 < vary: Accept-Encoding
{"content":[],"page":1,"size":0,"total":0}
}----------------------------------------------------------
DEBUG [2026-09-03 19:27:44,309] com.comet.opik.api.resources.v1.jobs.MetricsAlertJob: MetricsAlertJob finished processing all alerts
[Client action]---------------------------------------------{
1633 * Sending client request on thread awaitility-thread
1633 > GET http://localhost:34219/v1/private/automations/evaluators/01a068bd-5760-703b-a90d-7bda4a673617/logs
1633 > Accept: application/json
1633 > Authorization: [redacted]
1633 > Comet-Workspace: workspace-wEg1YRuVHIe6EkJVlLaL
}----------------------------------------------------------
DEBUG [2026-09-03 19:27:44,406] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:44,407] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:44,407] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
DEBUG [2026-09-03 19:27:44,407] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:44,409] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:44,409] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
INFO [2026-09-03 19:27:44,409] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Looking for logs for automated evaluator: id '01a068bd-5760-703b-a90d-7bda4a673617' on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
INFO [2026-09-03 19:27:44,410] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorLogsDAOImpl: Finding logs with criteria: LogCriteria[entityId=01a068bd-5760-703b-a90d-7bda4a673617, level=null, size=1000, page=null, markers=null]
DEBUG [2026-09-03 19:27:44,410] com.comet.opik.utils.template.TemplateLogger: Template runtime error: context [anonymous] 3:4 attribute level isn't defined
DEBUG [2026-09-03 19:27:44,410] com.comet.opik.utils.template.TemplateLogger: Template runtime error: context [anonymous] 6:36 attribute offset isn't defined
INFO [2026-09-03 19:27:44,415] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Found 1 logs for automated evaluator: id '01a068bd-5760-703b-a90d-7bda4a673617' on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
[Client action]---------------------------------------------{
1633 * Client response received on thread awaitility-thread
1633 < 200
1633 < content-length: 344
1633 < content-type: application/json
1633 < date: Thu, 03 Sep 2026 19:27:44 GMT
1633 < vary: Accept-Encoding
{"content":[{"timestamp":"2026-09-03T19:27:14.456727008Z","rule_id":"01a068bd-5760-703b-a90d-7bda4a673617","level":"INFO","message":"The traceId '01a068bd-57b7-7780-9a18-866515795241' was skipped for rule: '0_aVOeT3WL' and per the sampling rate '0.0'","markers":{"trace_id":"01a068bd-57b7-7780-9a18-866515795241"}}],"page":1,"size":1,"total":1}
}----------------------------------------------------------
127.0.0.1 - - [03/Sep/2026:19:27:44 +0000] "GET /v1/private/automations/evaluators/01a068bd-5760-703b-a90d-7bda4a673617/logs HTTP/1.1" 200 344 "-" "Jersey/3.1.11 (Async HTTP Grizzly Connector 3.1.11)" 9
[Client action]---------------------------------------------{
1634 * Sending client request on thread awaitility-thread
1634 > GET http://localhost:34219/v1/private/automations/evaluators/01a068bd-5791-75f4-824a-88dd119a9567/logs
1634 > Accept: application/json
1634 > Authorization: [redacted]
1634 > Comet-Workspace: workspace-wEg1YRuVHIe6EkJVlLaL
}----------------------------------------------------------
DEBUG [2026-09-03 19:27:44,416] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:44,417] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:44,417] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
DEBUG [2026-09-03 19:27:44,417] com.comet.opik.infrastructure.auth.RemoteAuthService: User and workspace id not found in cache for API key
DEBUG [2026-09-03 19:27:44,418] com.comet.opik.infrastructure.auth.RemoteAuthService: Caching user and workspace id for API key
DEBUG [2026-09-03 19:27:44,418] com.comet.opik.infrastructure.auth.RemoteAuthService: setting credentials into context, userName: 'user-f00OoXJUtAykHiVpkYs1', workspaceId: 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90', workspaceName: 'workspace-wEg1YRuVHIe6EkJVlLaL', quotas: 'null'
INFO [2026-09-03 19:27:44,418] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Looking for logs for automated evaluator: id '01a068bd-5791-75f4-824a-88dd119a9567' on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
INFO [2026-09-03 19:27:44,418] com.comet.opik.domain.evaluators.AutomationRuleEvaluatorLogsDAOImpl: Finding logs with criteria: LogCriteria[entityId=01a068bd-5791-75f4-824a-88dd119a9567, level=null, size=1000, page=null, markers=null]
DEBUG [2026-09-03 19:27:44,418] com.comet.opik.utils.template.TemplateLogger: Template runtime error: context [anonymous] 3:4 attribute level isn't defined
DEBUG [2026-09-03 19:27:44,419] com.comet.opik.utils.template.TemplateLogger: Template runtime error: context [anonymous] 6:36 attribute offset isn't defined
INFO [2026-09-03 19:27:44,423] com.comet.opik.api.resources.v1.priv.AutomationRuleEvaluatorsResource: Found 0 logs for automated evaluator: id '01a068bd-5791-75f4-824a-88dd119a9567' on workspace_id 'ce45bdc0-b2df-421e-97c2-1fac62f1bc90'
127.0.0.1 - - [03/Sep/2026:19:27:44 +0000] "GET /v1/private/automations/evaluators/01a068bd-5791-75f4-824a-88dd119a9567/logs HTTP/1.1" 200 42 "-" "Jersey/3.1.11 (Async HTTP Grizzly Connector 3.1.11)" 7
[Client action]---------------------------------------------{
1634 * Client response received on thread awaitility-thread
1634 < 200
1634 < content-length: 42
1634 < content-type: application/json
1634 < date: Thu, 03 Sep 2026 19:27:44 GMT
1634 < vary: Accept-Encoding
{"content":[],"page":1,"size":0,"total":0}
}----------------------------------------------------------
Loading