Skip to content

README.md

Latest commit

 

History

History
568 lines (442 loc) · 28.1 KB

README.md

File metadata and controls

568 lines (442 loc) · 28.1 KB
Raw

oz-agent-worker

Self-hosted worker for Oz cloud agents.

📖 Documentation

Overview

oz-agent-worker is a daemon that connects to Oz via WebSocket to receive and execute cloud agent tasks on self-hosted infrastructure.

Requirements

  • Service account API key with team scope
  • Network egress to warp-server
  • One supported execution backend:
    • Docker daemon access for the Docker backend
    • Local oz CLI access plus a writable workspace root for the Direct backend
    • Kubernetes API access plus cluster credentials for the Kubernetes backend
    • An operator-provided dispatch command for the Command backend (dispatch to any runtime over any transport)

Usage

Docker (Recommended)

The worker needs access to the Docker daemon to spawn task containers. Mount the host's Docker socket into the container:

docker run -v /var/run/docker.sock:/var/run/docker.sock \
  -e WARP_API_KEY="wk-abc123" \
  warpdotdev/oz-agent-worker:<release-tag> --worker-id "my-worker"

Note: Mounting the Docker socket gives the container access to the host's Docker daemon. This is required for the worker to create and manage task containers.

Image releases and pinning

Production self-hosted workers should pin an immutable image version instead of relying on latest.

Each merge to main creates a GitHub release and publishes a multi-architecture Docker image with a UTC timestamp tag:

warpdotdev/oz-agent-worker:vYYYY-MM-DD-HH-MM-SS

The GitHub release body includes both the Docker tag and digest. Use either the timestamp tag or the digest in production deployments:

docker pull warpdotdev/oz-agent-worker:v2026-06-01-10-09-37
docker pull warpdotdev/oz-agent-worker@sha256:<digest>

latest is updated to the same image when a release is published, but it is a moving tag intended for quick testing only.

Direct

The direct backend executes tasks directly on the host instead of inside Docker or Kubernetes. It requires the oz CLI to be available on PATH (or configured explicitly with backend.direct.oz_path) and stores per-task workspaces under backend.direct.workspace_root (default: /var/lib/oz/workspaces).

Example config:

worker_id: "my-worker"
backend:
  direct:
    workspace_root: "/var/lib/oz/workspaces"
    oz_path: "/usr/local/bin/oz"

Command

The command backend hands task execution to an operator-owned runtime over any transport. Instead of running the agent itself, the worker invokes an operator-configured dispatch_command and lets that command dispatch the task however it likes (HTTP, gRPC, a cloud SDK, a message queue, SSH, etc.).

Example config:

worker_id: "my-worker"
backend:
  command:
    dispatch_command: "/opt/oz/dispatch.sh"
    cancel_command: "/opt/oz/cancel.sh"
    dispatch_timeout: "60s"
    environment:
      - name: MY_RUNTIME_TOKEN

Config keys:

  • dispatch_command (required): shell command (run via /bin/sh -c) invoked once per task to dispatch it.
  • cancel_command (optional): shell command invoked best-effort when a dispatched task is cancelled. If unset, the worker relies on agent-side cancellation.
  • dispatch_timeout (optional): how long the dispatch command may run before it is considered failed (humantime format, e.g. 60s). Defaults to 60s.
  • environment: extra environment variables exposed to the dispatch/cancel commands (same name/value semantics as the other backends; omit value to inherit from the host).

The dispatch contract:

  • The dispatch command receives the task payload as JSON on stdin. This is the only place task environment variables and secrets appear — they are deliberately kept out of the subprocess environment and argv.

  • The following variables are also set in the command's environment for convenience: OZ_RUN_ID, OZ_EXECUTION_ID, OZ_WORKER_BACKEND=command, OZ_SERVER_ROOT_URL, OZ_DOCKER_IMAGE.

  • The JSON payload looks like:

    {
  "version": 1,
  "run_id": "...",
  "execution_id": "...",
  "server_root_url": "https://app.warp.dev",
  "worker_id": "my-worker",
  "docker_image": "ubuntu:22.04",
  "base_args": ["agent", "run", "--task-id", "...", "--server-root-url", "..."],
  "env": { "GITHUB_ACCESS_TOKEN": "...", "...": "..." },
  "sidecars": [ { "image": "...", "mount_path": "/agent", "read_write": false } ],
  "task": { "id": "...", "title": "...", "task_definition": { "prompt": "..." } }
}

    base_args is the oz agent run … argument vector your runtime should launch the agent with, inside an environment built from docker_image and sidecars.

  • Exit code 0 means the task was dispatched successfully; the worker will not finalize it (the remote agent reports terminal state to Warp itself). A non-zero exit or a dispatch that exceeds dispatch_timeout marks the task failed.

  • The cancel command (when configured) receives OZ_RUN_ID, OZ_EXECUTION_ID, and OZ_WORKER_BACKEND=command in its environment.

Because dispatched tasks run independently of the worker process, the command backend does not consume a local concurrency slot for the lifetime of the remote task, and worker shutdown does not cancel already-dispatched tasks. The runtime must report completion by executing oz harness-support report-shutdown using the provided run ID.

Kubernetes

The Kubernetes backend creates one Job per task. Cluster selection is controlled by the Kubernetes client config:

  • backend.kubernetes.kubeconfig points to an explicit kubeconfig file
  • if kubeconfig is omitted, the worker uses in-cluster config when running inside Kubernetes
  • otherwise it falls back to the default kubeconfig loading rules and uses the current context

Example config:

worker_id: "my-worker"
backend:
  kubernetes:
    kubeconfig: "/path/to/kubeconfig"
    namespace: "agents"
    default_image: "my-registry.io/dev-image:latest"
    unschedulable_timeout: "2m"
    pod_template:
      nodeSelector:
        kubernetes.io/os: linux
      containers:
        - name: task
          resources:
            requests:
              cpu: "2"
              memory: 4Gi

Notes:

  • default_image sets the Docker image for task Jobs when no Warp environment is configured on the run; this lets you skip creating a Warp environment entirely if all your tasks use the same base image (precedence: Warp environment image > default_image > ubuntu:22.04)
  • namespace selects the namespace inside the chosen cluster; it does not choose the cluster itself, and defaults to default when omitted
  • unschedulable_timeout controls how long a Pod may remain unschedulable before the task is failed early; it defaults to 30s, and 0s disables that fail-fast behavior
  • image_pull_policy defaults to IfNotPresent
  • sidecar_image overrides the warp-agent sidecar image reference sent by the server (e.g. docker.io/warpdotdev/warp-agent:latest); set this when cluster nodes cannot pull directly from Docker Hub and must use an internal registry mirror or pull-through cache instead. This only affects the warp-agent sidecar (mounted at /agent), not any additional sidecars. When using this override, you are responsible for keeping your mirror in sync with docker.io/warpdotdev/warp-agent — the server normally sends the correct version-matched image per task, so a stale mirror may cause version incompatibility
  • coding_cli_sidecars maps a harness config name (e.g. claude, codex) to a custom Docker image that will be mounted as the coding CLI sidecar for runs using that harness. When set, the worker replaces the server-provided sidecar image (or injects a new entry if the server did not send one) at the standard mount path /mnt/{harness}-cli-sidecar. Use this when your cluster uses a custom or internal Claude Code binary wrapper instead of the Warp-provided image. Example:
backend:
  kubernetes:
    coding_cli_sidecars:
      claude: "registry.internal.example.com/my-claude-wrapper:v1"

The custom image must have the harness binary reachable in the path that the Warp agent entrypoint scans (typically /usr/local/bin inside the sidecar image). claude must be in PATH when the harness process is invoked

  • by default, the Kubernetes backend materializes sidecars with root init containers into emptyDir volumes, matching the existing behavior
  • set use_image_volumes: true to opt into native image volumes for sidecars; in that mode, sidecar mounts are read-only and Kubernetes/runtime support for the built-in ImageVolume Pod volume source is required
  • Kubernetes 1.35+ is the recommended and tested target for use_image_volumes: true; Kubernetes 1.33-1.34 may work if ImageVolume is enabled and the container runtime supports image volumes
  • the worker runs a short-lived startup preflight Job for the configured sidecar-loading mode and waits for either preflight success or an early controller, mount, or admission failure, so incompatible cluster/runtime policy failures surface before the worker starts accepting tasks
  • preflight_image defaults to busybox:1.36; set it if your cluster only allows pulling startup-preflight images from an internal or allowlisted registry
  • pod_template accepts standard Kubernetes PodSpec YAML and is the declarative way to configure task pod scheduling, service accounts, image pull secrets, resources, and environment
  • when using pod_template, define a container named task if you want to customize the main task container directly; otherwise the worker appends its own task container to the PodSpec
  • when a run's runner specifies an instance shape, the worker sets the task container's CPU and memory requests and limits from that shape on a per-run basis, overriding any matching resources set on the task container in pod_template (other resource entries are preserved). Runs whose runner has no instance shape keep your pod_template/cluster defaults unchanged. The Docker backend applies the same shape as container CPU/memory limits; the Direct backend runs on the host and does not enforce shapes
  • to run services (databases, brokers, caches, etc.) alongside the task container, declare them in pod_template.initContainers with restartPolicy: Always (native sidecar containers). Matching Kubernetes Job semantics, the worker ignores their exit codes when detecting task failure: the kubelet stops sidecars with SIGTERM after the task container finishes, so services that exit non-zero on SIGTERM (e.g. JVM-based services exiting 143) do not fail the task. Init containers without restartPolicy remain run-to-completion setup steps whose non-zero exit fails the task
  • use valueFrom.secretKeyRef inside pod_template to inject Kubernetes Secret values into task container environment variables:
pod_template:
  containers:
    - name: task
      env:
        - name: MY_SECRET
          valueFrom:
            secretKeyRef:
              name: my-k8s-secret
              key: secret-key

Helm Chart

This repo includes a namespace-scoped Helm chart at charts/oz-agent-worker.

The chart deploys:

  • a long-lived Deployment for oz-agent-worker
  • a namespaced ServiceAccount
  • a namespaced Role / RoleBinding
  • a ConfigMap containing the worker config
  • an optional Secret for WARP_API_KEY (or a reference to an existing Secret)

At runtime, the deployed worker connects outbound to Warp and creates one Kubernetes Job per task. The built-in Kubernetes Job controller then manages the task Pod lifecycle.

Recommended install flow:

kubectl create secret generic oz-agent-worker \
  --from-literal=WARP_API_KEY="wk-abc123" \
  --namespace agents

helm install oz-agent-worker ./charts/oz-agent-worker \
  --namespace agents \
  --create-namespace \
  --set worker.workerId=my-worker \
  --set image.tag=v2026-06-01-10-09-37

The chart assumes the worker runs inside the target cluster and uses in-cluster Kubernetes auth by default. It does not create CRDs or cluster-scoped RBAC. Set image.tag explicitly for each install so the worker image is pinned instead of defaulting to latest.

The chart always deploys a single replica for a given worker.workerId. If you want multiple workers, deploy multiple releases with distinct worker IDs rather than scaling one release horizontally.

The chart defaults the long-lived worker Deployment to a non-root security context and conservative starting resource requests of 100m CPU and 128Mi memory. Tune worker.resources for your workload and cluster policy.

The Deployment includes a default exec liveness probe that checks the worker process is still running (kill -0 1). If the worker becomes unresponsive, Kubernetes will restart the pod after three consecutive failures. Override worker.livenessProbe in your values to use a custom probe (e.g. httpGet if you add a health endpoint), or set it to null to disable.

When the long-lived worker pod is terminated by normal Kubernetes disruption (for example Karpenter node consolidation), the Kubernetes backend preserves active task Jobs instead of deleting them during worker shutdown. This protects running Oz sessions from worker pod rotation as long as the task Job and task Pod remain healthy. The default worker.terminationGracePeriodSeconds only needs to cover WebSocket close and metrics flush.

This does not make task Pods disruption-proof. If Karpenter or another cluster operation evicts the node that is actually running the task Pod, the live Oz session can still be interrupted because the process and any pod-local workspace state are on that task Pod. For stronger protection, schedule worker pods and task pods independently (for example with separate node pools, selectors, tolerations, or disruption budgets) so worker rotation does not imply task pod eviction.

When cleanup is enabled, successful task Jobs are deleted immediately by the worker when it observes completion, while failed task Jobs (and Jobs orphaned by worker disruption) are left in place for post-mortem debugging and cleaned up by the Kubernetes Job TTL (kubernetesBackend.ttlSecondsAfterFinished, default 24h). When cleanup is disabled, no TTL is set and task Jobs remain indefinitely.

Recommended namespace-scoped permissions for the worker are:

  • create, get, list, watch, delete jobs
  • get, list, watch pods
  • get pods/log
  • list events

The worker Deployment's ServiceAccount is separate from the task Job serviceAccountName you may set inside backend.kubernetes.pod_template / kubernetesBackend.podTemplate. The worker Deployment defaults to non-root. By default, task Jobs still materialize sidecars with root init containers; set kubernetesBackend.useImageVolumes=true to opt into native image volumes instead. Kubernetes 1.35+ is the recommended and tested target for that opt-in path, while Kubernetes 1.33-1.34 may work if ImageVolume is enabled and the container runtime supports image volumes. If your cluster restricts image sources for admission or policy reasons, set kubernetesBackend.preflightImage in the chart to an allowlisted image for the startup preflight Job, and configure task imagePullSecrets inside podTemplate when needed.

Go Install

go install github.com/warpdotdev/oz-agent-worker@latest
oz-agent-worker --api-key "wk-abc123" --worker-id "my-worker"

Build from Source

git clone https://github.com/warpdotdev/oz-agent-worker.git
cd oz-agent-worker
go build -o oz-agent-worker
./oz-agent-worker --api-key "wk-abc123" --worker-id "my-worker"

Environment Variables for Task Containers

Use -e / --env to pass environment variables into task containers:

# Explicit key=value
oz-agent-worker --api-key "wk-abc123" --worker-id "my-worker" -e MY_SECRET=hunter2

# Pass through from host environment
export MY_SECRET=hunter2
oz-agent-worker --api-key "wk-abc123" --worker-id "my-worker" -e MY_SECRET

# Multiple variables
oz-agent-worker --api-key "wk-abc123" --worker-id "my-worker" -e FOO=bar -e BAZ=qux

When using Docker to run the worker, note that -e flags for the worker itself (task containers) are passed as arguments, while -e flags for the worker container use Docker's syntax:

docker run -v /var/run/docker.sock:/var/run/docker.sock \
  -e WARP_API_KEY="wk-abc123" \
  warpdotdev/oz-agent-worker:<release-tag> --worker-id "my-worker" -e MY_SECRET=hunter2

When configuring the Kubernetes backend via YAML or Helm, declarative task-container env belongs in backend.kubernetes.pod_template / kubernetesBackend.podTemplate rather than a separate top-level Kubernetes env list. The -e / --env flags remain available as backend-agnostic runtime overrides.

Docker Connectivity

The worker automatically discovers the Docker daemon using standard Docker client mechanisms, in this order:

  1. DOCKER_HOST environment variable (e.g., unix:///var/run/docker.sock, tcp://localhost:2375)
  2. Default socket location (/var/run/docker.sock on Linux, ~/.docker/run/docker.sock for rootless)
  3. Docker context via DOCKER_CONTEXT environment variable
  4. Config file (~/.docker/config.json) for context settings

Additional supported environment variables:

  • DOCKER_API_VERSION - Specify Docker API version
  • DOCKER_CERT_PATH - Path to TLS certificates
  • DOCKER_TLS_VERIFY - Enable TLS verification

Example: Remote Docker Daemon

export DOCKER_HOST="tcp://remote-host:2376"
export DOCKER_TLS_VERIFY=1
export DOCKER_CERT_PATH="/path/to/certs"
oz-agent-worker --api-key "wk-abc123" --worker-id "my-worker"

Monitoring

The worker can export metrics over OpenTelemetry. Exporter selection is driven by the standard OpenTelemetry environment variables, implemented via go.opentelemetry.io/contrib/exporters/autoexport. When OTEL_METRICS_EXPORTER is unset, the worker delegates to autoexport's default, which is OTLP push to OTEL_EXPORTER_OTLP_ENDPOINT (defaulting to http://localhost:4318 for http/protobuf or http://localhost:4317 for grpc). To fully disable metrics export, set OTEL_METRICS_EXPORTER=none.

Quick start with Prometheus

export OTEL_METRICS_EXPORTER=prometheus
export OTEL_EXPORTER_PROMETHEUS_HOST=0.0.0.0
export OTEL_EXPORTER_PROMETHEUS_PORT=9464
oz-agent-worker --api-key "$WARP_API_KEY" --worker-id "my-worker"

# In another shell:
curl -s localhost:9464/metrics | grep oz_worker_

Quick start with OTLP

export OTEL_METRICS_EXPORTER=otlp
export OTEL_TRACES_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector.observability.svc:4318
oz-agent-worker --api-key "$WARP_API_KEY" --worker-id "my-worker"

Tracing is opt-in. Set OTEL_TRACES_EXPORTER to a non-none exporter such as otlp to emit per-task spans and lifecycle events; leave it unset to export metrics only.

Helm

helm install oz-agent-worker ./charts/oz-agent-worker \
  --namespace agents --create-namespace \
  --set worker.workerId=my-worker \
  --set image.tag=v2026-06-01-10-09-37 \
  --set metrics.enabled=true

With metrics.enabled=true and the default metrics.exporter=prometheus, the chart adds:

  • a containerPort: metrics (default 9464) on the worker Deployment
  • the OTEL_METRICS_EXPORTER, OTEL_EXPORTER_PROMETHEUS_HOST, and OTEL_EXPORTER_PROMETHEUS_PORT environment variables
  • a namespace-scoped Service named <release>-oz-agent-worker-metrics with prometheus.io/scrape annotations
  • optionally a PodMonitor (metrics.podMonitor.create=true) for clusters using the Prometheus Operator

For OTLP push instead, set metrics.exporter=otlp and forward the relevant endpoint variables via metrics.extraEnv:

metrics:
  enabled: true
  exporter: otlp
  extraEnv:
    - name: OTEL_TRACES_EXPORTER
      value: otlp
    - name: OTEL_EXPORTER_OTLP_ENDPOINT
      value: http://otel-collector.observability.svc:4318

Metric catalog

All metrics carry the resource attributes service.name=oz-agent-worker, service.version, worker.id, and worker.backend, so each worker process shows up as a distinct series.

  • oz_worker_connected (gauge): 1 while the worker has an active WebSocket connection to warp-server, 0 otherwise. Aggregate to count connected workers: sum(oz_worker_connected).
  • oz_worker_tasks_active (gauge / UpDownCounter): tasks currently executing on this worker. To count workers running ≥1 task: count(oz_worker_tasks_active > 0).
  • oz_worker_tasks_max_concurrent (gauge): configured concurrency limit (0 means unlimited).
  • oz_worker_tasks_rejected_total{reason} (counter): tasks the worker declined, e.g. reason="at_capacity".
  • oz_worker_tasks_completed_total{result} (counter): completed tasks labeled result="succeeded", result="failed", or result="cancelled". Success rate over 5m: sum(rate(oz_worker_tasks_completed_total{result="succeeded"}[5m])) / sum(rate(oz_worker_tasks_completed_total[5m])).
  • oz_worker_task_duration_seconds{result} (histogram): wall-clock task duration on the worker. p95: histogram_quantile(0.95, sum by (le) (rate(oz_worker_task_duration_seconds_bucket[5m]))).
  • oz_worker_task_failures_total{phase,reason} (counter): bounded failure classification for task failures, such as phase="backend" with reason="image_pull", reason="unschedulable", or reason="container_oom".
  • oz_worker_websocket_reconnects_total{reason} (counter): reconnect attempts; spikes indicate flapping workers.
  • oz_worker_info{version,backend,worker_id} (gauge, value 1): build and runtime metadata, useful for joining other series by labels.
  • oz_worker_debug_archive_requests_total{backend,ownership,outcome,reason} (counter): debug-archive log requests this worker owned. Requests for executions another instance ran are silent and are not counted here.
  • oz_worker_debug_archive_snapshot_duration_seconds{backend} and oz_worker_debug_archive_snapshot_bytes{backend} (histograms): cost and size of producing a log snapshot.
  • oz_worker_debug_archive_truncations_total{backend} (counter): snapshots that dropped bytes to stay within their bound.
  • oz_worker_debug_archive_uploads_total{result} and oz_worker_debug_archive_upload_duration_seconds{result}: snapshot upload outcomes and latency.
  • oz_worker_debug_archive_requests_in_flight (gauge): requests currently being snapshotted or uploaded, bounded by debugLogCapture.maxConcurrentUploads.
  • oz_worker_debug_archive_capture_bytes (gauge): disk currently reserved by direct-execution captures and request snapshots.
  • oz_worker_cleanup_grace_entries (gauge) and oz_worker_cleanup_grace_results_total{backend,result} (counter): executions retained past terminal state and the backend cleanups performed when their grace expired.

Sample dashboards / alerts

Direct mappings for the questions enterprise operators most commonly ask:

  • Workers available: sum(oz_worker_connected)
  • Workers active (running ≥1 task): count(oz_worker_tasks_active > 0)
  • Saturation: sum(oz_worker_tasks_active) / sum(oz_worker_tasks_max_concurrent > 0)
  • Failure rate: sum(rate(oz_worker_tasks_completed_total{result="failed"}[5m]))
  • Failure modes: sum by (phase, reason) (rate(oz_worker_task_failures_total[5m]))
  • Reconnect storms: sum(rate(oz_worker_websocket_reconnects_total[5m])) > 0.1

Debug archive log collection

Warp can assemble a debug archive for a cloud-agent run. For self-hosted executions the logs live inside your infrastructure, so Warp asks the worker that actually ran the execution for a bounded snapshot instead of reaching into your Docker daemon, Kubernetes cluster, or host.

When Warp requests logs for an execution, the worker that ran it snapshots the backed-up output, uploads it directly to a short-lived Warp-signed destination, and reports the result. Every other worker process silently ignores the request. Collection never blocks, delays, or fails a running agent: if logs cannot be captured or uploaded, the archive is simply marked partial.

Sensitive data

A snapshot contains whatever the execution wrote to stdout and stderr, which can include prompts, source code, identifiers, and secrets a process printed. Treat a debug archive as sensitive. The worker itself never logs captured bytes, upload destinations, signed headers, local capture paths, or upload response bodies.

Supported backends

Backend Source
Docker The execution container's stdout and stderr, covering the entrypoint script and the client process it starts.
Kubernetes Every init and regular container in the execution's pods, including current and best-effort previous logs after a restart.
Direct A bounded on-disk capture of setup, agent, and teardown stdout/stderr.
Command Not supported. The dispatch command hands the task to an opaque runtime with no log API, so the worker reports the source unavailable rather than passing off dispatch output as the agent's log.

Docker and Kubernetes report one merged stream per container, so those records are labeled combined rather than attributing a line to the entrypoint or the client. Only direct execution owns distinct handles, so only it labels setup, agent, and teardown phases.

Sizing the cleanup grace for failure capture

The worker retains an execution's log source for the cleanup grace it already resolves for --idle-on-complete, in this order:

  1. the run's idle_timeout_minutes
  2. the worker's idle_on_complete (worker.idleOnComplete in the chart)
  3. the Oz default of 45 minutes

There is deliberately no separate archive retention setting: one clock governs both how long the agent stays available for follow-ups and how long its logs stay retrievable.

Collection triggered by a failure has to reach the worker after the terminal event propagates to Warp, and the request itself allows up to 30 minutes. If you want reliable archives for failed runs, keep the grace comfortably longer than that; a shorter grace still works but produces a partial archive when it lapses first.

Kubernetes: kubernetesBackend.ttlSecondsAfterFinished must not be shorter than the effective grace. The Job TTL controller deletes a finished Job's pods, and once they are gone their logs are gone with them regardless of the worker's own retention. A successful Job is now deleted at the grace deadline rather than immediately at task completion.

Worker replacement: ownership is process-local. If a worker pod is replaced while a preserved Job keeps running, the replacement does not inherit the old process's ownership, and a later request for that execution yields a partial archive rather than an incorrect upload.

Capture bounds

Each execution's snapshot is capped at 64 MiB. Output above the cap keeps the first and last portions with an explicit gap marker, so both the early setup context and the terminal failure survive.

The Helm chart mounts a dedicated 1 GiB ephemeral volume for the capture root and renders the matching bounds:

debugLogCapture:
  enabled: true
  sizeLimit: 1Gi
  directory: /var/lib/oz/debug-logs
  maxTotalBytes: 1073741824
  maxExecutionBytes: 67108864
  maxConcurrentUploads: 2

Outside Kubernetes, set the same bounds under debug_log_capture in the config file; an unset value uses the default above, and an unwritable root or an invalid bound disables archive capture without affecting task execution.

The volume is scratch space, not storage: nothing in it survives worker replacement, and it does not extend the cleanup grace.

Compatibility

Self-hosted logs in debug archives require a worker built with this protocol. An older worker ignores the request and Warp records the source as unavailable, so upgrading is safe and never required for ordinary task execution. Each authenticated connection reports its build version so Warp can show exactly which worker ran an execution; a worker that reports no version still executes tasks normally and is shown as not reported.

The Kubernetes backend needs no additional permissions: the chart's existing namespace-scoped get pods/log grant is sufficient.

License

Copyright © 2026 Warp