From 73a181d32f42c1fa2782c4e5514fdbbc81ad8f38 Mon Sep 17 00:00:00 2001 From: Drew Newberry Date: Fri, 25 Sep 2026 13:53:45 -0700 Subject: [PATCH 1/6] docs: streamline README, add policy prover to architecture docs (#3718) * docs(readme): streamline README and move reference detail to docs Restructure the README as a short path from overview to quickstart to further reading. Move prerelease install steps into the installation guide and telemetry build flags into a new observability page. Fix broken docs links and outdated runtime and credential descriptions. Signed-off-by: Drew Newberry * docs(readme): describe 0.1.0 as adding new isolation primitives Signed-off-by: Drew Newberry * docs(architecture): add policy prover as a gateway component Signed-off-by: Drew Newberry * docs: describe OpenShell as a runtime for fleets of autonomous AI agents Signed-off-by: Drew Newberry * docs(architecture): describe policy prover as formal verification Signed-off-by: Drew Newberry * docs(architecture): name OpenShell Sandbox in component table Signed-off-by: Drew Newberry * docs(architecture): fold isolation backend into supervisor row Signed-off-by: Drew Newberry * docs(architecture): mention formal verification in overview Signed-off-by: Drew Newberry * docs(run-agent): use the published OpenCode image in the first-agent guide Signed-off-by: Drew Newberry * docs(inference): correct provider examples and readiness Signed-off-by: Piotr Mlocek * docs(providers): correct Google binding and provider selection Signed-off-by: Piotr Mlocek * docs(readme): sharpen value prop, how it works, and explore further Signed-off-by: Drew Newberry * docs(run-agent): use example Anthropic profile and add policy advisor step Import the example Anthropic profile, which now allows OpenCode, instead of editing it with sed. Add a step that shows how to review and approve mechanistic policy proposals as the agent needs more access. Signed-off-by: Drew Newberry * feat(providers): add OpenRouter example for OpenCode Signed-off-by: Drew Newberry * docs(run-agent): run OpenCode against OpenRouter with a free model Add an example OpenRouter provider profile scoped to OpenCode and switch the first-agent guide to it, using a free Nemotron model so readers do not need OpenRouter credits. Revert the OpenCode binary added to the example Anthropic profile. Signed-off-by: Drew Newberry * docs(readme): link first-agent guide and add agent skills section Signed-off-by: Drew Newberry --------- Signed-off-by: Drew Newberry Signed-off-by: Piotr Mlocek Co-authored-by: Piotr Mlocek --- README.md | 359 ++---------------- architecture/README.md | 5 +- crates/openshell-server/src/grpc/provider.rs | 1 + docs/about/architecture.mdx | 11 +- docs/about/installation.mdx | 37 ++ docs/about/overview.mdx | 4 +- docs/about/run-an-agent.mdx | 87 +++-- docs/how-it-works/inference.mdx | 20 +- docs/how-it-works/providers/google.mdx | 19 +- docs/how-it-works/providers/profiles.mdx | 9 +- docs/images/openshell-system-architecture.svg | 30 +- docs/index.mdx | 4 +- docs/observability/telemetry.mdx | 53 +++ providers/openrouter.yaml | 43 +++ 14 files changed, 288 insertions(+), 394 deletions(-) create mode 100644 docs/observability/telemetry.mdx create mode 100644 providers/openrouter.yaml diff --git a/README.md b/README.md index 2d9979f72d..7b12c4dc7a 100644 --- a/README.md +++ b/README.md @@ -15,358 +15,75 @@ [![Project Status](https://img.shields.io/badge/status-alpha-orange)](https://github.com/NVIDIA/OpenShell/releases) > [!IMPORTANT] -> **New in OpenShell 0.1.0:** a stable release cadence, an improved security model, an expanded extension surface, and new APIs. [Read the 0.1.0 upgrade guide](https://docs.nvidia.com/openshell/latest/upgrade/0-1-0). +> **New in OpenShell 0.1.0:** a stable release cadence, new isolation primitives, an expanded extension surface, and new APIs. [Read the 0.1.0 upgrade guide](https://docs.nvidia.com/openshell/latest/upgrade/0-1-0). -OpenShell is the safe, private runtime for autonomous AI agents. It provides sandboxed execution environments that protect your data, credentials, and infrastructure — governed by declarative YAML policies that prevent unauthorized file access, data exfiltration, and uncontrolled network activity. +OpenShell is the safe, private runtime for fleets of autonomous AI agents. Agents are most useful when they can read files, install packages, call APIs, and use credentials. OpenShell gives them that capability without giving them unrestricted access to your data, secrets, or network. You declare what each agent can touch in a policy, and OpenShell enforces it. -OpenShell is built agent-first. It ships public agent skills for using and operating OpenShell, plus separate repository-aware workflows for contributors and maintainers. +## How It Works -## Install OpenShell +OpenShell governs what agents can do in two ways: it instruments the kernel to enforce policy on every file access, system call, and network connection at runtime, and it uses formal verification to check what a policy change would allow before it is applied. -### Prerequisites +- **Kernel-level enforcement.** Each agent runs in an isolated sandbox. Kernel controls confine which files it can access and which system calls it can make, and every network connection passes through a policy check before it leaves the sandbox. Agents never see real credentials; OpenShell adds them only to requests bound for approved endpoints. +- **Formally verified policy changes.** Before a policy change is approved, OpenShell uses formal verification to flag risky new access it would grant, such as reaching a new host with credentials or calling a new API method, so those changes wait for human review. -- **A supported host** — Linux, macOS (Apple Silicon), or Windows with WSL 2 (experimental). -- **A local runtime** — Docker, Podman, or host virtualization enabled for MicroVM-backed sandboxes. +See [Architecture](https://docs.nvidia.com/openshell/latest/about/architecture) for how the gateway, supervisor, and sandbox fit together. -### Install +## Quickstart -**Local installation:** +You need Linux, macOS on Apple Silicon, or Windows with WSL 2 (experimental), plus Docker, Podman, or host virtualization. See the [Support Matrix](https://docs.nvidia.com/openshell/latest/about/support-matrix) for details. -```bash +```shell curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | sh -``` - -The installer installs the latest stable release by default. See [Prerelease and development builds](#prerelease-and-development-builds) to install an upcoming release or the latest commit on `main`. - -**Kubernetes installation:** - -> **Experimental** — the Kubernetes deployment path is under active development. Expect rough edges and breaking changes. -> **Required:** Your cluster CNI MUST enforce Kubernetes `NetworkPolicy` for -> ingress and egress in every sandbox namespace. OpenShell creates the policies, -> but Kubernetes does not verify that the CNI applies them. - -Deploy the OpenShell gateway into a Kubernetes cluster from the OCI chart published to GHCR: - -```bash -helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart -``` - -See [`deploy/helm/openshell/README.md`](deploy/helm/openshell/README.md) for available versions, dev tag conventions, and configuration. - -For deploying OpenShell on OpenShift, see [`deploy/helm/openshell/README.md#install-on-openshift`](deploy/helm/openshell/README.md#install-on-openshift). - -### Create a sandbox - -```bash -openshell sandbox create --name demo -``` - -The gateway defaults to `nvcr.io/nvidia/base/ubuntu:24.04`, a minimal Ubuntu -Noble workload. To run an agent, build or select an OCI image that contains the -agent and pass its explicit reference: - -```bash -openshell sandbox create --from registry.example.com/agents/my-agent:1.0 -- my-agent -``` - -Attach the providers and policy required by that workload. - -### See network policy in action - -Every sandbox starts with **minimal outbound access**. You open additional access with a network rule that the proxy enforces at the HTTP method and path level, without restarting anything. - -```bash -# 1. Create a sandbox (starts with minimal outbound access) openshell sandbox create --name demo - -# 2. Inside the sandbox — blocked -sandbox$ curl -sS https://api.github.com/zen -curl: (56) Received HTTP code 403 from proxy after CONNECT - -# 3. Back on the host — add a read-only GitHub API rule -sandbox$ exit -openshell policy update demo --rule-name github_api --binary /usr/bin/curl \ - --add-endpoint api.github.com:443:read-only:rest:enforce --wait - -# 4. Reconnect — GET allowed, POST blocked by L7 -openshell sandbox connect demo -sandbox$ curl -sS https://api.github.com/zen -Anything added dilutes everything else. - -sandbox$ curl -sS -X POST https://api.github.com/repos/octocat/hello-world/issues -d '{"title":"oops"}' -{"error":"policy_denied","detail":"POST /repos/octocat/hello-world/issues not permitted by policy"} ``` -See the [full walkthrough](examples/sandbox-policy-quickstart/) or run the automated demo: - -```bash -bash examples/sandbox-policy-quickstart/demo.sh -``` +The installer sets up the CLI and a local gateway. The default sandbox image is minimal Ubuntu with no agent installed. To run a real agent, follow [Run Your First Agent](https://docs.nvidia.com/openshell/latest/about/run-your-first-agent): it runs OpenCode against a free OpenRouter model and shows how to approve new access as the agent needs it. -## SDKs +## Explore Further -OpenShell provides client SDKs for Python, TypeScript, Go, and Rust. SDK packages connect applications to an OpenShell gateway; they do not install the `openshell` CLI. Use the SDK and gateway from the same OpenShell release when possible. +- [Sandboxes](https://docs.nvidia.com/openshell/latest/how-it-works/sandboxes/overview): images, runtimes, GPUs, and lifecycle. +- [Policies](https://docs.nvidia.com/openshell/latest/how-it-works/policies/overview): filesystem, network, and process rules, with the [advisor](https://docs.nvidia.com/openshell/latest/how-it-works/policies/advisor) and [prover](https://docs.nvidia.com/openshell/latest/how-it-works/policies/prover) for reviewing changes. +- [Providers](https://docs.nvidia.com/openshell/latest/how-it-works/providers/overview): credentials that work only at approved endpoints, including [inference](https://docs.nvidia.com/openshell/latest/how-it-works/inference). +- [Gateways](https://docs.nvidia.com/openshell/latest/how-it-works/gateways/overview): the control plane for sandboxes, policy, and access. +- [Kubernetes](https://docs.nvidia.com/openshell/latest/kubernetes/setup): deploy the gateway with Helm. Your CNI must enforce `NetworkPolicy`. +- [Extensibility](https://docs.nvidia.com/openshell/latest/extensibility/overview): middleware, interceptors, and compute drivers. +- [Tutorials](https://docs.nvidia.com/openshell/latest/tutorials/first-network-policy): step-by-step policy and agent walkthroughs. +- [Prerelease and development builds](https://docs.nvidia.com/openshell/latest/about/installation#prerelease-and-development-builds): try an upcoming release or the latest commit on `main`. -### Python +## Agent Skills -The [Python SDK](python/openshell/) is published to [PyPI](https://pypi.org/project/openshell/): +Install the public OpenShell skills for your coding agent: ```shell -uv add openshell -``` - -### TypeScript - -The [TypeScript SDK](sdk/typescript/README.md) is published to GitHub Packages as `@nvidia/openshell-sdk`. Configure the `@nvidia` npm scope for `https://npm.pkg.github.com`, authenticate with a token that has `read:packages`, and install it: - -```shell -npm install @nvidia/openshell-sdk -``` - -### Go - -Add the [Go SDK](sdk/go/README.md) to a Go module: - -```shell -go get github.com/NVIDIA/OpenShell/sdk/go@latest -``` - -### Rust - -The [Rust SDK](crates/openshell-sdk/README.md) is currently consumed from source. Pin the Git dependency to the same OpenShell release as the gateway: - -```shell -cargo add openshell-sdk \ - --git https://github.com/NVIDIA/OpenShell \ - --tag -``` - -## How It Works - -OpenShell isolates each sandbox in its own container with policy-enforced egress routing. A lightweight gateway coordinates sandbox lifecycle, and every outbound connection is intercepted by the policy engine, which does one of three things: - -- **Allows** — the destination and binary match a policy block. -- **Binds credentials to endpoints** — injects provider credentials only after policy admits a request to a profile-authorized endpoint. -- **Denies** — blocks the request and logs it. - -| Component | Role | -| ------------------ | -------------------------------------------------------------------------------------------- | -| **Gateway** | Control-plane API that coordinates sandbox lifecycle and acts as the auth boundary. | -| **Sandbox** | Isolated runtime with container supervision and policy-enforced egress routing. | -| **Policy Engine** | Enforces filesystem, network, and process constraints from application layer down to kernel. | -| **Provider Access** | Profile-defined endpoints, binary policy, and endpoint-bound credential injection for model APIs and other services. | - -OpenShell runs a gateway control plane that manages sandbox lifecycle through a configured compute driver. Supported compute platforms include Docker, Podman, MicroVM, and Kubernetes. - -## Protection Layers - -OpenShell applies defense in depth across four policy domains: - -| Layer | What it protects | When it applies | -| ---------- | --------------------------------------------------- | --------------------------- | -| Filesystem | Prevents reads/writes outside allowed paths. | Locked at sandbox creation. | -| Network | Blocks unauthorized outbound connections. | Hot-reloadable at runtime. | -| Process | Blocks privilege escalation and dangerous syscalls. | Locked at sandbox creation. | -| Providers | Grants endpoint-bound credentials and network access. | Hot-reloadable at runtime. | - -Policies are declarative YAML files. Static sections (filesystem, process) are locked at creation; network policy and provider attachments can be updated on a running sandbox. - -## Providers - -Agents need credentials — API keys, tokens, service accounts. OpenShell manages these as **providers**: named credential bundles that are injected into sandboxes at creation. Credentials never leak into the sandbox filesystem; they are injected as environment variables at runtime. - -A provider is created from a **provider profile**, which declares the credentials, endpoints, and client binaries the provider needs. Profiles are import-only: a gateway serves exactly the profiles you imported with `openshell provider profile import`, and ships none of its own. The [`providers/`](providers/) directory holds reviewable examples to copy and adapt. Once a profile is imported, the CLI can auto-discover credentials for its provider from your shell environment, or you can create providers explicitly with `openshell provider create`. - -Inference access uses the same provider workflow. Attach an inference-capable provider to a sandbox, call the provider's native endpoint, and select the model in the client. Provider profiles contribute the endpoint policy and bind credential placeholders to the authorized destination. - -## GPU Support (Experimental) - -> **Experimental** — GPU passthrough works on supported hosts but is under active development. Expect rough edges and breaking changes. - -OpenShell can pass host GPUs into sandboxes for local inference, fine-tuning, or any GPU workload. Add `--gpu` when creating a sandbox: - -```bash -openshell sandbox create --gpu --from registry.example.com/your-org/gpu-agent:latest -- claude -``` - -Docker-backed GPU sandboxes auto-select CDI when available and otherwise fall back to Docker's NVIDIA GPU request path (`--gpus all`). - -**Requirements:** NVIDIA drivers and the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) must be installed on the host. The sandbox image itself must include the appropriate GPU drivers and libraries for your workload — the default Ubuntu image does not. See the [BYOC example](https://github.com/NVIDIA/OpenShell/tree/main/examples/bring-your-own-container) for building a custom sandbox image with GPU support. - -## Running Agents - -OpenShell can run Linux agents packaged in OCI images. The default Ubuntu -workload does not bundle agent CLIs. Build or select an image containing your -agent, then authorize its binary paths, service endpoints, and credentials. See -[Run Your First Agent](https://docs.nvidia.com/openshell/latest/about/run-an-agent) -for the image, provider, and policy workflow. - -## Key Commands - -| Command | Description | -| ---------------------------------------------------------- | ----------------------------------------------- | -| `openshell sandbox create -- ` | Create a sandbox and launch an agent. | -| `openshell sandbox connect [name]` | SSH into a running sandbox. | -| `openshell sandbox list` | List all sandboxes. | -| `openshell provider create --type [type] --from-existing` | Create a credential provider from env vars. | -| `openshell sandbox provider attach ` | Attach a provider to a running sandbox. | -| `openshell policy set --policy file.yaml` | Apply or update a policy on a running sandbox. | -| `openshell policy get ` | Show the active policy. | -| `openshell logs [name] --tail` | Stream sandbox logs. | -| `openshell term` | Launch the real-time terminal UI for debugging. | - -See the [full documentation](https://docs.nvidia.com/openshell/latest) for command guides, tutorials, and reference material. - -## Terminal UI - -OpenShell includes a real-time terminal dashboard for monitoring gateways, sandboxes, and providers — inspired by [k9s](https://k9scli.io/). - -```bash -openshell term -``` - -

- OpenShell Terminal UI -

- -The TUI gives you a live, keyboard-driven view of your gateway and sandboxes. Navigate with `Tab` to switch panels, `j`/`k` to move through lists, `Enter` to select, and `:` for command mode. Gateway health and sandbox status auto-refresh every two seconds. - -## Workload Images and BYOC - -Use `--from` with an explicit OCI image reference: - -```bash -docker build -t my-sandbox:latest ./my-sandbox-dir -openshell sandbox create --from my-sandbox:latest - -podman build -t localhost/my-sandbox:latest ./my-sandbox-dir -openshell sandbox create --from localhost/my-sandbox:latest - -openshell sandbox create --from registry.example.com/agents/my-agent:1.0 -``` - -Build with the container engine used by your local gateway. For a remote -gateway, push the image to a registry that the gateway can pull from. See the -[BYOC example](https://github.com/NVIDIA/OpenShell/tree/main/examples/bring-your-own-container). - -## Use OpenShell with Your Agent - -OpenShell provides four portable skills for users and operators: CLI workflows (`openshell-cli`), gateway troubleshooting (`debug-openshell-cluster`), inference troubleshooting (`debug-inference`), and policy generation (`generate-sandbox-policy`). Install them with the Agent Skills CLI: - -```bash npx skills add NVIDIA/OpenShell ``` -These public, installable skills live in [`skills/`](skills/) and use the installed CLI help and [published documentation](https://docs.nvidia.com/openshell/latest/index.html) as their sources of truth. They do not require an OpenShell source checkout. +The skills teach your agent to drive the OpenShell CLI, write sandbox policies, and debug gateways and inference routing. They live in [`skills/`](skills/) and work without an OpenShell source checkout. -## Built With Agents - -OpenShell is developed using the same agent-driven workflows it enables. Contributor and maintainer skills live separately in [`.agents/skills/`](.agents/skills/); they automate work on the OpenShell repository and are not included when users install the public skills: +## SDKs -- **Spike and build:** Investigate a problem with `create-spike`; a human accepts it with `state:accepted` or [roadmap](https://github.com/orgs/NVIDIA/projects/233) placement, or declines it. Accepted work can remain human-owned or enter the optional, human-gated `agent:*` planning and implementation workflow. -- **Triage and route:** Community issues are assessed with `triage-issue`. Agents establish technical validity and impact; humans decide whether the project should act and where the work sits on the roadmap. -- **Security review:** `review-security-issue` produces a severity assessment and remediation plan. `fix-security-issue` implements it. -- **Repository maintenance:** `sync-agent-infra`, `update-docs-from-commits`, and other internal workflows keep code, documentation, and agent infrastructure consistent. +SDKs connect applications to an OpenShell gateway. They do not install the CLI. Use the same OpenShell release for the SDK and the gateway when possible. -Agent implementation is human-directed: a user may request a phase directly, or maintainers may use the optional `agent:*` workflow to queue and approve planning and implementation. See [AGENTS.md](AGENTS.md) for the full workflow chain documentation. +| Language | Install | Docs | +|---|---|---| +| Python | `uv add openshell` | [README](python/openshell/) | +| TypeScript | `npm install @nvidia/openshell-sdk` (GitHub Packages) | [README](sdk/typescript/README.md) | +| Go | `go get github.com/NVIDIA/OpenShell/sdk/go@latest` | [README](sdk/go/README.md) | +| Rust | Git dependency pinned to a release tag | [README](crates/openshell-sdk/README.md) | -## Getting Help +## Community - **Questions and discussion:** [GitHub Discussions](https://github.com/NVIDIA/OpenShell/discussions) -- **Bug reports:** [GitHub Issues](https://github.com/NVIDIA/OpenShell/issues) — use the bug report template -- **Security vulnerabilities:** See [SECURITY.md](SECURITY.md) — do not use GitHub Issues -- **Agent-assisted help:** Install the public OpenShell skills with `npx skills add NVIDIA/OpenShell` - -## Learn More - -- [Full Documentation](https://docs.nvidia.com/openshell/latest/index.html) — overview, architecture, tutorials, and reference -- [Run Your First Agent](https://docs.nvidia.com/openshell/latest/about/run-an-agent) — prepare an image, attach providers, and launch an agent -- [GitHub Sandbox Tutorial](https://docs.nvidia.com/openshell/latest/tutorials/github-sandbox) — end-to-end scoped GitHub repo access -- [Architecture](https://github.com/NVIDIA/OpenShell/tree/main/architecture) — detailed architecture docs and design decisions -- [Roadmap](https://github.com/orgs/NVIDIA/projects/233) — planned work and project priorities -- [RFC Board](https://github.com/orgs/NVIDIA/projects/233/views/6) — RFC proposals tracked on the OpenShell Roadmap with the `rfc` label -- [Support Matrix](https://docs.nvidia.com/openshell/latest/about/support-matrix) — platforms, versions, and kernel requirements -- [Brev Launchable](https://brev.nvidia.com/launchable/deploy/now?launchableID=env-3Ap3tL55zq4a8kew1AuW0FpSLsg) — try OpenShell on cloud compute without local setup -- [Agent Instructions](AGENTS.md) — system prompt and workflow documentation for agent contributors - -## Prerelease and development builds - -Use a prerelease candidate to evaluate an upcoming release, or use the rolling development build to test the latest commit on `main`. These builds may change before the next stable release. The matching documentation is published in the [development channel](https://docs.nvidia.com/openshell/dev/index.html). - -Prerelease packages are retained as GitHub Actions artifacts for 90 days and require an authenticated [GitHub CLI](https://cli.github.com/) session. The `pre` alias installs the latest prerelease: +- **Bug reports and feature requests:** [GitHub Issues](https://github.com/NVIDIA/OpenShell/issues), using the issue templates +- **Security vulnerabilities:** follow [SECURITY.md](SECURITY.md). Do not open a GitHub issue. +- **Roadmap:** [OpenShell Roadmap](https://github.com/orgs/NVIDIA/projects/233) and the [RFC board](https://github.com/orgs/NVIDIA/projects/233/views/6) +- **Try it in the cloud:** [Brev Launchable](https://brev.nvidia.com/launchable/deploy/now?launchableID=env-3Ap3tL55zq4a8kew1AuW0FpSLsg) -```shell -gh auth login -curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | \ - OPENSHELL_VERSION=pre sh -``` - -The installer checks prerelease tags from newest to oldest, selects an unexpired artifact from a successful release run for the current platform, and downloads only that artifact. Installed packages retain the candidate's exact version, such as `0.1.0-pre.3`. Prerelease tags do not create entries on the GitHub Releases page. -On Linux, prereleases and explicit release tags use Debian or RPM packages even if `snap` is installed. - -The rolling [`dev` release](https://github.com/NVIDIA/OpenShell/releases/tag/dev) does not require GitHub authentication: - -```shell -curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | \ - OPENSHELL_VERSION=dev sh -``` - -For Kubernetes, select the corresponding Helm chart version. Helm chart versions omit the leading `v` from release tags: - -```shell -# Pin an exact candidate -helm upgrade --install openshell \ - oci://ghcr.io/nvidia/openshell/helm-chart \ - --version 0.1.0-pre.3 - -# Rolling development build -helm upgrade --install openshell \ - oci://ghcr.io/nvidia/openshell/helm-chart \ - --version 0.0.0-dev -``` - -Prerelease charts use exact `-pre.N` versions. Development charts are also published as immutable `0.0.0-dev.` versions when you need to pin a specific commit. See the [Helm chart documentation](deploy/helm/openshell/README.md#available-versions) for version and configuration details. - -## Contributing - -OpenShell is built agent-first. Issues should include a user story, problem statement, impact, and acceptance criteria. The impact should explain the consequences of the current behavior and why existing workarounds are insufficient. Feature requests also require a workflow-level proposed design and alternatives; bug reports add reproduction steps, environment details, and relevant logs. Once work is authorized through the project workflow or a direct request, contributors should use the skills in `.agents/skills/` to investigate the current code and behavior, implement the change, and verify it. If an issue contains earlier diagnostics, verify them rather than relying on them. See [CONTRIBUTING.md](CONTRIBUTING.md) for the full agent skills table, contribution workflow, and development setup. +OpenShell is built agent-first: it is developed with the same agent-driven workflows it enables. See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and the contribution workflow, and [AGENTS.md](AGENTS.md) for the contributor agent skills and workflow chains. ## Telemetry -OpenShell collects anonymous telemetry to help improve the project for developers. This data is not used to track individual user behavior. It helps us understand aggregate usage of sandbox, provider, and policy workflows so we can prioritize product improvements and share usage trends with the community. - -Disable telemetry at runtime by setting `OPENSHELL_TELEMETRY_ENABLED=false` on the gateway deployment. For Helm installs, set `server.telemetryEnabled=false`. OpenShell propagates this deployment setting into sandbox supervisor environments so sandbox-side telemetry collection is disabled as well. - -You can also compile telemetry out entirely. Telemetry support is a default-on `telemetry` Cargo feature, and each crate that carries it also defines a `defaults-without-telemetry` alias covering every other default feature. Build telemetry-free artifacts with `--no-default-features --features defaults-without-telemetry`: - -```shell -cargo build --release -p openshell-gateway --no-default-features --features defaults-without-telemetry -cargo build --release -p openshell-sandbox --no-default-features --features defaults-without-telemetry -cargo build --release -p openshell-driver-vm --no-default-features --features defaults-without-telemetry -``` - -The resulting binaries contain no telemetry endpoint, no telemetry HTTP client, and no emission code. With telemetry compiled out, the gateway emits nothing and reports telemetry disabled to the sandboxes it launches. Cargo has no way to subtract a single default feature, so `defaults-without-telemetry` must be paired with `--no-default-features`; passing it on its own leaves the defaults in place and fails the build rather than producing a binary that still emits. - -The gateway also exposes separate Cargo features for its built-in compute drivers: `compute-driver-kubernetes`, `compute-driver-docker`, `compute-driver-podman`, `compute-driver-vm`, and `compute-driver-mxc`. Disable the default feature set, then enable only the drivers and telemetry mode required by the target binary. For example: - -```shell -# Docker only, with telemetry support. -cargo build --release -p openshell-gateway --no-default-features --features telemetry,compute-driver-docker - -# Docker and VM only, with telemetry compiled out. -cargo build --release -p openshell-gateway --no-default-features --features compute-driver-docker,compute-driver-vm - -# Windows MXC only, with telemetry support and bundled Z3. -cargo build --release -p openshell-gateway --no-default-features --features telemetry,compute-driver-mxc,bundled-z3 -``` - -Regular builds retain their platform driver set through the default `in-tree-compute-drivers` compatibility feature. On Windows, `compute-driver-mxc` selects MXC; the other four features install unsupported-driver stubs. On other platforms, MXC is excluded. - -Telemetry events are limited to anonymous operational categories and counts, such as sandbox lifecycle outcomes, provider profile buckets, policy decision counts, and aggregate network activity denial categories. OpenShell telemetry does not collect sandbox names or IDs, hostnames, file paths, binary paths, prompts, credentials, provider names, model names, or user content. - -Opting out applies only to telemetry emitted by OpenShell. Third-party services, model providers, inference endpoints, agents, or tools that you configure and use with OpenShell may have their own terms and privacy practices. - -We publish aggregate usage trends from this telemetry every two weeks. See the [community telemetry reports](telemetry/README.md) for the latest summary. +OpenShell collects anonymous telemetry, limited to operational categories and counts, to help improve the project. It does not collect sandbox names, hostnames, file paths, prompts, credentials, provider or model names, or user content. To disable it, set `OPENSHELL_TELEMETRY_ENABLED=false` on the gateway, or `server.telemetryEnabled=false` for Helm installs. You can also compile telemetry out entirely. See [Telemetry](https://docs.nvidia.com/openshell/latest/observability/telemetry) for details and the [community telemetry reports](telemetry/README.md) for published usage trends. ## Notice and Disclaimer diff --git a/architecture/README.md b/architecture/README.md index 706db67b1f..f8c6b24048 100644 --- a/architecture/README.md +++ b/architecture/README.md @@ -1,6 +1,6 @@ # OpenShell Architecture -OpenShell runs autonomous AI agents in sandboxed environments with explicit +OpenShell runs fleets of autonomous AI agents in sandboxed environments with explicit policy, credential, identity, and network boundaries. The target architecture is built around three stable runtime components: the **CLI**, the **Gateway**, and the **Supervisor**. @@ -29,6 +29,7 @@ flowchart TB subgraph CP["Control Plane"] GW["Gateway core"] + PROVER["Policy prover"] DB[("Shared persistence")] COMPUTE["Compute"] CREDS["Credentials"] @@ -59,6 +60,7 @@ flowchart TB TUI -->|"gRPC / HTTP"| GW GW --> DB + GW -->|"proposed policy changes"| PROVER GW --> COMPUTE GW --> CREDS GW --> CPIDENT @@ -89,6 +91,7 @@ flowchart TB |---|---| | CLI, SDK, TUI | User-facing management surfaces. They talk to the gateway and do not need to know which infrastructure drivers are active. | | Gateway | Authenticated control plane, API server, durable state, policy and settings delivery, provider config, supervisor session ownership, and relay coordination. | +| Policy prover | Gateway-linked formal verification of proposed policy changes. It reports categorical findings (new credentialed reach, new HTTP methods, L7 bypass, link-local reach); any finding blocks auto-approval. The same crate backs the standalone `openshell-prover check` boundary command, which runs on local files without a gateway. See [Security Policy](security-policy.md). | | Compute subsystem | Sandbox lifecycle semantics: creation, deletion, watching, reconciliation, and state transitions. Platform provisioning details belong to the compute driver. | | Credentials subsystem | Logical provider and credential resolution. Secret storage and platform-native credential access belong to credentials drivers. | | Control-plane identity | Authentication and authorization for users, operators, and API clients. External identity verification belongs to identity drivers. | diff --git a/crates/openshell-server/src/grpc/provider.rs b/crates/openshell-server/src/grpc/provider.rs index 7cbc2749fa..2141ecb4c2 100644 --- a/crates/openshell-server/src/grpc/provider.rs +++ b/crates/openshell-server/src/grpc/provider.rs @@ -6302,6 +6302,7 @@ mod tests { "google-vertex-ai", "nvidia", "openai", + "openrouter", "pypi" ] ); diff --git a/docs/about/architecture.mdx b/docs/about/architecture.mdx index 8c8ab55679..468fc922c7 100644 --- a/docs/about/architecture.mdx +++ b/docs/about/architecture.mdx @@ -11,8 +11,9 @@ position: 2 ![OpenShell system architecture showing a trusted supervisor separated from a network-isolated sandbox workload. The workload can connect only to the supervisor.](../images/openshell-system-architecture.svg) OpenShell separates control-plane state from sandbox enforcement. The gateway -owns sandbox state, policy, providers, and access. A compute driver provisions -the workload and its isolation boundary. +owns sandbox state, policy, providers, and access, and uses formal verification +to evaluate proposed policy changes before they are approved. A compute driver +provisions the workload and its isolation boundary. The trusted supervisor runs outside the workload. `openshell-sandbox` runs inside it, owns the agent process, and mediates its network requests. All other @@ -25,10 +26,10 @@ gateway for configuration, credentials, logs, and interactive sessions. |---|---| | [Gateway](/how-it-works/gateways/overview) | Checks who you are and remembers everything about your sandboxes. It delivers policy and settings, attaches providers, decides who can do what, and coordinates connections into sandboxes. | | [Compute runtime](/how-it-works/sandboxes/runtimes) | Creates the sandbox, starts the supervisor and the workload, sets up the private channel between them, and builds the network fence. It reports status back and cleans up when the sandbox goes away. | -| [Supervisor](#inside-the-sandbox-boundary) | Lives on the trusted side of the boundary. It checks requests against policy, supplies credentials, resolves DNS, opens approved connections, and keeps the link to the gateway alive. | -| [Isolation backend](/extensibility/isolation-backends) | Gives the supervisor one consistent way to work with any runtime: confirm the boundary, start the agent, run commands, forward connections, and see network requests. | -| [`openshell-sandbox`](#inside-the-sandbox-boundary) | Lives inside the boundary with the agent. It owns the agent's processes, knows which program made each request, applies process controls, and forwards TCP and DNS traffic to the supervisor. | +| [Supervisor](#inside-the-sandbox-boundary) | Lives on the trusted side of the boundary. It checks requests against policy, supplies credentials, resolves DNS, opens approved connections, and keeps the link to the gateway alive. It works with every runtime through one [isolation backend](/extensibility/isolation-backends) interface to confirm the boundary, start the agent, run commands, forward connections, and see network requests. | +| [OpenShell Sandbox](#inside-the-sandbox-boundary) | Lives inside the boundary with the agent. It owns the agent's processes, knows which program made each request, applies process controls, and forwards TCP and DNS traffic to the supervisor. | | [Outer network fence](/security/best-practices#deny-by-default-egress) | Denies all network egress from the workload except its protected connection to the supervisor. Each runtime builds this with its own native tools. | +| [Policy prover](/how-it-works/policies/prover) | Runs in the gateway and uses formal verification to check each proposed policy change before approval. It flags changes such as new credentialed reach, new HTTP methods, or access to cloud metadata endpoints, and any finding blocks auto-approval. It also ships as the standalone `openshell-prover` command for checking a policy against a boundary in CI. | | [Policies](/how-it-works/policies/overview) | Describe what the agent can touch: files, processes, network destinations, API calls, and where provider credentials can go. | | [Providers](/how-it-works/providers/overview) | Connect a service name to a stored credential. The supervisor hands that credential out only where policy allows it. | diff --git a/docs/about/installation.mdx b/docs/about/installation.mdx index f6912e79e0..350164c2a7 100644 --- a/docs/about/installation.mdx +++ b/docs/about/installation.mdx @@ -26,6 +26,43 @@ openshell status To install a specific release, set `OPENSHELL_VERSION` to a release tag. Release artifacts are also on the [GitHub Releases](https://github.com/NVIDIA/OpenShell/releases) page. +## Prerelease and Development Builds + +Use a prerelease candidate to evaluate an upcoming release, or the rolling development build to test the latest commit on `main`. These builds may change before the next stable release. The matching documentation is published in the [development channel](https://docs.nvidia.com/openshell/dev/index.html). + +Prerelease packages are retained as GitHub Actions artifacts for 90 days and require an authenticated [GitHub CLI](https://cli.github.com/) session. The `pre` alias installs the latest prerelease: + +```shell +gh auth login +curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | \ + OPENSHELL_VERSION=pre sh +``` + +The installer checks prerelease tags from newest to oldest, selects an unexpired artifact from a successful release run for the current platform, and downloads only that artifact. Installed packages keep the candidate's exact version, such as `0.1.0-pre.3`. Prerelease tags do not create entries on the GitHub Releases page. On Linux, prereleases and explicit release tags use Debian or RPM packages even if `snap` is installed. + +The rolling [`dev` release](https://github.com/NVIDIA/OpenShell/releases/tag/dev) does not require GitHub authentication: + +```shell +curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | \ + OPENSHELL_VERSION=dev sh +``` + +For Kubernetes, select the corresponding Helm chart version. Helm chart versions omit the leading `v` from release tags: + +```shell +# Pin an exact candidate +helm upgrade --install openshell \ + oci://ghcr.io/nvidia/openshell/helm-chart \ + --version 0.1.0-pre.3 + +# Rolling development build +helm upgrade --install openshell \ + oci://ghcr.io/nvidia/openshell/helm-chart \ + --version 0.0.0-dev +``` + +Prerelease charts use exact `-pre.N` versions. Development charts are also published as immutable `0.0.0-dev.` versions when you need to pin a specific commit. + ## Supported Runtimes The local gateway auto-detects an available runtime. To pin one, set `compute_driver` in the gateway TOML file. See [Sandbox Runtimes](/how-it-works/sandboxes/runtimes). diff --git a/docs/about/overview.mdx b/docs/about/overview.mdx index 63ada5a910..29ca8931e1 100644 --- a/docs/about/overview.mdx +++ b/docs/about/overview.mdx @@ -3,12 +3,12 @@ # SPDX-License-Identifier: Apache-2.0 title: "Overview of NVIDIA OpenShell" sidebar-title: "Overview" -description: "OpenShell is the safe, private runtime for autonomous AI agents. Run agents in sandboxed environments that protect your data, credentials, and infrastructure." +description: "OpenShell is the safe, private runtime for fleets of autonomous AI agents. Run agents in sandboxed environments that protect your data, credentials, and infrastructure." keywords: "Generative AI, Cybersecurity, AI Agents, Sandboxing, Security, Privacy, Provider Credentials" position: 1 --- -NVIDIA OpenShell is an open-source runtime for executing autonomous AI agents in sandboxed environments with kernel-level isolation. It combines sandbox runtime controls and a declarative YAML policy so teams can run agents without giving them unrestricted access to local files, credentials, and external networks. +NVIDIA OpenShell is an open-source runtime for executing fleets of autonomous AI agents in sandboxed environments with kernel-level isolation. It combines sandbox runtime controls and a declarative YAML policy so teams can run agents without giving them unrestricted access to local files, credentials, and external networks. New in OpenShell 0.1.0: a [stable release cadence](/about/support-matrix#releases), new [isolation primitives](/about/architecture), and an expanded [extension surface](/extensibility/overview), along with much more. diff --git a/docs/about/run-an-agent.mdx b/docs/about/run-an-agent.mdx index fb3169a6bd..96ab04abe7 100644 --- a/docs/about/run-an-agent.mdx +++ b/docs/about/run-an-agent.mdx @@ -15,41 +15,36 @@ with the agent as its main process. **1. Configure what the agent has access to.** A [provider profile](/how-it-works/providers/profiles) defines the credentials, service -endpoints, and executable paths an agent may use. For Claude Code, review the -[example profile](https://github.com/NVIDIA/OpenShell/blob/main/providers/claude-code.yaml), -including its executable paths for your image, then import it and -[create a provider](/how-it-works/providers/overview#create-a-provider) that -stores its credentials. +endpoints, and executable paths an agent may use. + +This example runs OpenCode +against OpenRouter. Review the +[example profile](https://github.com/NVIDIA/OpenShell/blob/main/providers/openrouter.yaml), +which allows `/usr/local/bin/opencode` to reach `openrouter.ai`, then import +it and [create a provider](/how-it-works/providers/overview#create-a-provider) +that stores its credentials. ```shell openshell profile import \ - --url https://raw.githubusercontent.com/NVIDIA/OpenShell/main/providers/claude-code.yaml + --url https://raw.githubusercontent.com/NVIDIA/OpenShell/main/providers/openrouter.yaml openshell provider create \ - --name claude-code \ - --type claude-code \ + --name openrouter \ + --type openrouter \ --from-existing ``` -Set `ANTHROPIC_API_KEY` before creating the provider. OpenShell stores its -value in the credential store. +Set `OPENROUTER_API_KEY` before creating the provider. OpenShell stores its +value in the credential store and sends it only to `openrouter.ai`. -**2. Configure the sandbox image.** +**2. Choose the sandbox image.** -Build or select a [sandbox image](/how-it-works/sandboxes/overview#sandbox-images) -containing Claude Code and every tool it needs: +Use any [sandbox image](/how-it-works/sandboxes/overview#sandbox-images) that +contains the agent and every tool it needs. -```shell -docker build -t agent:local . -``` +In this example, the OpenCode image installs `opencode` at `/usr/local/bin/opencode`. -For a remote gateway, push the image to a registry that the gateway can pull -from. Use an explicit version instead of a mutable tag for reproducible -deployments. - -The built-in fallback policy grants access to the working directory and common -runtime paths, and denies network egress unless an attached provider contributes -an endpoint. Create a custom [sandbox policy](/how-it-works/policies/overview) when the +Create a custom [sandbox policy](/how-it-works/policies/overview) when the agent needs more filesystem access, package registries, source hosts, tool servers, or other destinations. @@ -61,14 +56,48 @@ provider, and pass the agent command after `--`: ```shell openshell sandbox create \ --name my-agent \ - --from agent:local \ - --provider claude-code \ - -- claude + --from ghcr.io/anomalyco/opencode:latest \ + --provider openrouter \ + -- opencode -m openrouter/nvidia/nemotron-3.5-lightning:free +``` + +OpenCode detects the OpenRouter credential and starts with the model passed to +`-m`. This example uses a free model, so it works without OpenRouter credits. +Choose any model from `opencode models openrouter`. + +For more control over what the agent can access, refer to +[Policies](/how-it-works/policies/overview). + +**4. Grant more access as the agent needs it.** + +When the agent reaches a destination that policy does not allow, OpenShell +denies the request and drafts a rule for it. The +[policy advisor](/how-it-works/policies/advisor)'s mechanistic mapper groups +recent denials by host, port, and calling program, then submits a narrow +proposal to the gateway. The gateway checks each proposal with the +[policy prover](/how-it-works/policies/prover) before you review it. + +List pending proposals from the host: + +```shell +openshell rule get my-agent --status pending +``` + +Approve a proposal when its rule matches the access you intend to grant, or +reject it with a reason: + +```shell +openshell rule approve my-agent --chunk-id + +openshell rule reject my-agent \ + --chunk-id \ + --reason "Not needed for this task." ``` -Add `--policy sandbox-policy.yaml` when the workload needs a custom policy. The -command after `--` becomes the sandbox's main process, and every child process -it starts remains inside the sandbox boundary. +Approved rules hot-reload into the running sandbox without a restart, so the +agent can retry the request. To let the agent propose its own narrower rules, +or to approve proposals automatically when the prover finds no new risk, refer +to [Policy Advisor](/how-it-works/policies/advisor). ## Examples diff --git a/docs/how-it-works/inference.mdx b/docs/how-it-works/inference.mdx index 72c361e654..db2335320a 100644 --- a/docs/how-it-works/inference.mdx +++ b/docs/how-it-works/inference.mdx @@ -83,7 +83,7 @@ client = OpenAI( ) response = client.responses.create( - model="nvidia/nemotron-3-nano-30b-a3b", + model="nvidia/nemotron-3-super-120b-a12b", input="Hello!", ) print(response.output_text) @@ -103,24 +103,26 @@ starting templates you copy and adapt, not as defaults. ## Attach a Provider to a Running Sandbox ```shell -openshell sandbox provider attach inference-demo nvidia-prod +openshell sandbox provider attach inference-demo nvidia-prod --wait openshell sandbox provider list inference-demo ``` -Running sandboxes poll for provider and effective-policy changes. Launch a new -process after attachment so it receives the new credential placeholder: +Running sandboxes poll for provider and effective-policy changes. `--wait` +confirms the sandbox has applied the attachment. Launch a new process so it +receives the new credential placeholder: ```shell -openshell sandbox exec inference-demo -- python app.py +openshell sandbox exec --name inference-demo -- python app.py ``` An already-running process does not gain new environment variables. Credential -rotation and detach still take effect immediately at placeholder resolution. +rotation and detach take effect once the sandbox applies the provider change. +Use `--wait` to confirm that change before relying on it. Detach the provider to revoke its policy and credential access: ```shell -openshell sandbox provider detach inference-demo nvidia-prod +openshell sandbox provider detach inference-demo nvidia-prod --wait ``` ## Use a Custom or Self-hosted Endpoint @@ -238,7 +240,7 @@ Earlier releases configured one shared route: openshell provider create --name nvidia-prod --type nvidia --from-existing openshell inference set \ --provider nvidia-prod \ - --model nvidia/nemotron-3-nano-30b-a3b \ + --model nvidia/nemotron-3-super-120b-a12b \ --timeout 300 ``` @@ -259,7 +261,7 @@ openshell provider create \ --name nvidia-native-prod \ --type nvidia-native \ --from-existing -openshell sandbox provider attach inference-demo nvidia-native-prod +openshell sandbox provider attach inference-demo nvidia-native-prod --wait ``` OpenShell does not export stored credential values or change a provider's diff --git a/docs/how-it-works/providers/google.mdx b/docs/how-it-works/providers/google.mdx index 1df8cead1a..2c7091476c 100644 --- a/docs/how-it-works/providers/google.mdx +++ b/docs/how-it-works/providers/google.mdx @@ -164,6 +164,10 @@ The metadata emulator serves tokens with the `cloud-platform` OAuth2 scope, which grants access to any GCP API the underlying service account has IAM permissions for. Add the target API hosts to your sandbox network policy: +Attach `my-gcp` to the sandbox first. Because the `google-cloud` profile has no +endpoints, each API endpoint must bind to that provider instance. Without the +binding, OpenShell withholds its access token from the sandbox. + ```yaml network_policies: gcp_apis: @@ -174,6 +178,8 @@ network_policies: protocol: rest access: read-write enforcement: enforce + credential_binding: + provider: my-gcp binaries: - { path: /usr/bin/curl } - { path: /usr/bin/node } @@ -181,16 +187,13 @@ network_policies: - { path: "/sandbox/.venv/**" } ``` -Or update a live sandbox directly: +For a running sandbox, export its base policy. Remove the metadata header above +the `---` line, add the endpoint and binaries shown above, then apply the file: ```shell -openshell policy update my-sandbox \ - --add-endpoint "*.googleapis.com:443:read-write:rest:enforce" \ - --binary "/usr/bin/curl" \ - --binary "/usr/bin/node" \ - --binary "/sandbox/.uv/python/**" \ - --binary "/sandbox/.venv/**" \ - --wait +openshell policy get my-sandbox --base > sandbox-policy.yaml +# Edit sandbox-policy.yaml to add the gcp_apis rule shown above. +openshell policy set my-sandbox --policy sandbox-policy.yaml --wait ``` ## Network Policy diff --git a/docs/how-it-works/providers/profiles.mdx b/docs/how-it-works/providers/profiles.mdx index 3cb9a6bfba..f28124c82c 100644 --- a/docs/how-it-works/providers/profiles.mdx +++ b/docs/how-it-works/providers/profiles.mdx @@ -891,10 +891,11 @@ Each attached provider with a matching profile contributes a provider policy lay Updating a custom provider profile affects every provider instance whose `type` matches that profile ID. Provider instances are not rewritten, and sandbox-authored policies are not modified. Running sandboxes observe the updated provider-derived policy on their next config sync. If a gateway-global policy is active, provider-derived policy layers remain suppressed. -The CLI can infer a provider profile from a recognized sandbox command and -auto-create the provider from profile discovery. Attach a different existing -provider explicitly with `--provider`, or use `openshell sandbox provider -attach` after creation. +Name a provider with `--provider` when creating a sandbox. If no provider has +that name and it matches an imported profile ID, the CLI can create the +provider from local profile discovery. The trailing sandbox command does not +select a provider. Use `openshell sandbox provider attach` to add an existing +provider after creation. List providers attached to a sandbox: diff --git a/docs/images/openshell-system-architecture.svg b/docs/images/openshell-system-architecture.svg index 7a6d4b59fa..c910ff8c5a 100644 --- a/docs/images/openshell-system-architecture.svg +++ b/docs/images/openshell-system-architecture.svg @@ -3,7 +3,7 @@ OpenShell system architecture - User interfaces connect to the gateway. The OpenShell Runtime places a trusted supervisor separately from a network-isolated sandbox workload. The workload's only allowed network path is its mediated connection to the supervisor, which connects to approved external services. + User interfaces connect to the gateway, which uses the policy prover to formally verify proposed policy changes before approval. The OpenShell Runtime places a trusted supervisor separately from a network-isolated sandbox workload. The workload's only allowed network path is its mediated connection to the supervisor, which connects to approved external services. @@ -51,17 +51,21 @@ GATEWAY (CONTROL PLANE) - - API Server - API, authentication, lifecycle, relays - - Durable state - sandboxes · policies · providers · settings - - Compute driver - places and fences each sandbox - - + + API Server + API, authentication, lifecycle, relays + + Policy prover + formal verification of policy changes + + Durable state + sandboxes · policies · providers · settings + + Compute driver + places and fences each sandbox + + + OPENSHELL RUNTIME (DATA PLANE) @@ -98,7 +102,7 @@ Deny all egress except to the supervisor - + gRPC / HTTP OUTBOUND diff --git a/docs/index.mdx b/docs/index.mdx index c05fe445cf..a5e2e310d7 100644 --- a/docs/index.mdx +++ b/docs/index.mdx @@ -2,7 +2,7 @@ # SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 title: "NVIDIA OpenShell Developer Guide" -description: "OpenShell is the safe, private runtime for autonomous AI agents. Run agents in sandboxed environments that protect your data, credentials, and infrastructure." +description: "OpenShell is the safe, private runtime for fleets of autonomous AI agents. Run agents in sandboxed environments that protect your data, credentials, and infrastructure." keywords: "Generative AI, Cybersecurity, AI Agents, Sandboxing, Security, Privacy, Provider Credentials" position: 1 --- @@ -30,7 +30,7 @@ import { CommandTerminal } from "./_components/CommandTerminal"; ]} /> -NVIDIA OpenShell is the safe, private runtime for autonomous AI agents. It provides sandboxed execution environments +NVIDIA OpenShell is the safe, private runtime for fleets of autonomous AI agents. It provides sandboxed execution environments that protect your data, credentials, and infrastructure. Agents run with exactly the permissions they need and nothing more, governed by declarative policies that prevent unauthorized file access, data exfiltration, and uncontrolled network activity. diff --git a/docs/observability/telemetry.mdx b/docs/observability/telemetry.mdx new file mode 100644 index 0000000000..e78e0f7d00 --- /dev/null +++ b/docs/observability/telemetry.mdx @@ -0,0 +1,53 @@ +--- +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +title: "Telemetry" +sidebar-title: "Telemetry" +description: "What anonymous telemetry OpenShell collects and how to disable it at runtime or compile it out." +keywords: "Generative AI, Cybersecurity, Telemetry, Privacy, Opt Out, Cargo Features, Observability" +--- + +OpenShell collects anonymous telemetry to help improve the project. The data shows aggregate usage of sandbox, provider, and policy workflows. It is not used to track individual users. Aggregate usage trends are published every two weeks in the [community telemetry reports](https://github.com/NVIDIA/OpenShell/tree/main/telemetry). + +## What Is Collected + +Telemetry events are limited to anonymous operational categories and counts, such as sandbox lifecycle outcomes, provider profile buckets, policy decision counts, and aggregate network activity denial categories. + +OpenShell telemetry does not collect sandbox names or IDs, hostnames, file paths, binary paths, prompts, credentials, provider names, model names, or user content. + +Opting out applies only to telemetry emitted by OpenShell. Third-party services, model providers, inference endpoints, agents, or tools that you configure and use with OpenShell may have their own terms and privacy practices. + +## Disable Telemetry at Runtime + +Set `OPENSHELL_TELEMETRY_ENABLED=false` on the gateway deployment. For Helm installs, set `server.telemetryEnabled=false`. OpenShell propagates this setting into sandbox supervisor environments, so sandbox-side telemetry collection is disabled as well. + +## Compile Telemetry Out + +Telemetry support is a default-on `telemetry` Cargo feature. Each crate that carries it also defines a `defaults-without-telemetry` alias covering every other default feature. Build telemetry-free artifacts with `--no-default-features --features defaults-without-telemetry`: + +```shell +cargo build --release -p openshell-gateway --no-default-features --features defaults-without-telemetry +cargo build --release -p openshell-sandbox --no-default-features --features defaults-without-telemetry +cargo build --release -p openshell-driver-vm --no-default-features --features defaults-without-telemetry +``` + +The resulting binaries contain no telemetry endpoint, no telemetry HTTP client, and no emission code. With telemetry compiled out, the gateway emits nothing and reports telemetry disabled to the sandboxes it launches. + +Cargo cannot subtract a single default feature, so `defaults-without-telemetry` must be paired with `--no-default-features`. Passing it on its own leaves the defaults in place and fails the build rather than producing a binary that still emits. + +### Select Compute Drivers + +The gateway exposes separate Cargo features for its built-in compute drivers: `compute-driver-kubernetes`, `compute-driver-docker`, `compute-driver-podman`, `compute-driver-vm`, and `compute-driver-mxc`. Disable the default feature set, then enable only the drivers and telemetry mode the target binary needs: + +```shell +# Docker only, with telemetry support. +cargo build --release -p openshell-gateway --no-default-features --features telemetry,compute-driver-docker + +# Docker and VM only, with telemetry compiled out. +cargo build --release -p openshell-gateway --no-default-features --features compute-driver-docker,compute-driver-vm + +# Windows MXC only, with telemetry support and bundled Z3. +cargo build --release -p openshell-gateway --no-default-features --features telemetry,compute-driver-mxc,bundled-z3 +``` + +Regular builds keep their platform driver set through the default `in-tree-compute-drivers` compatibility feature. On Windows, `compute-driver-mxc` selects MXC and the other four features install unsupported-driver stubs. On other platforms, MXC is excluded. diff --git a/providers/openrouter.yaml b/providers/openrouter.yaml new file mode 100644 index 0000000000..9e357b82c5 --- /dev/null +++ b/providers/openrouter.yaml @@ -0,0 +1,43 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# Example provider profile. OpenShell does not load it; import it explicitly: +# openshell provider profile lint -f providers/openrouter.yaml +# openshell provider profile import -f providers/openrouter.yaml --global +# +# Copy and edit this file rather than importing it unchanged. `binaries` is the +# least-privilege control that decides which processes may reach the endpoints +# below, so it has to name the paths in *your* image. +# +# Client binaries: OpenCode. +# Reference layout: OpenCode at /usr/local/bin/opencode, as in +# ghcr.io/anomalyco/opencode. Add curl or an SDK interpreter +# if the workload calls the API another way. +# Credential scope: OPENROUTER_API_KEY, sent as a bearer token to +# openrouter.ai and nowhere else. +# Endpoint access: openrouter.ai, read-write, L7 enforced. +# Smoke test: openshell sandbox create --provider \ +# --from ghcr.io/anomalyco/opencode:latest -- \ +# opencode run -m openrouter/nvidia/nemotron-3.5-lightning:free 'reply with OK' + +id: openrouter +display_name: OpenRouter +description: OpenRouter inference API +category: inference +inference_capable: true +credentials: + - name: api_key + description: OpenRouter API key + env_vars: [OPENROUTER_API_KEY] + required: true + auth_style: bearer + header_name: authorization +discovery: + credentials: [api_key] +endpoints: + - host: openrouter.ai + port: 443 + protocol: rest + access: read-write + enforcement: enforce +binaries: [/usr/local/bin/opencode] From 496ebba293f5cc2bb2753444dddd534f0b4aeb6a Mon Sep 17 00:00:00 2001 From: Drew Newberry Date: Fri, 25 Sep 2026 21:04:39 +0000 Subject: [PATCH 2/6] docs(readme): remove alpha status badge (#3723) Signed-off-by: Drew Newberry --- README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/README.md b/README.md index 7b12c4dc7a..ae90b3d2ae 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,6 @@ [![PyPI](https://img.shields.io/badge/PyPI-openshell-orange?logo=pypi)](https://pypi.org/project/openshell/) [![Security Policy](https://img.shields.io/badge/Security-Report%20a%20Vulnerability-red)](SECURITY.md) [![Documentation](https://img.shields.io/badge/docs-latest-brightgreen)](https://docs.nvidia.com/openshell/latest/index.html) -[![Project Status](https://img.shields.io/badge/status-alpha-orange)](https://github.com/NVIDIA/OpenShell/releases) > [!IMPORTANT] > **New in OpenShell 0.1.0:** a stable release cadence, new isolation primitives, an expanded extension surface, and new APIs. [Read the 0.1.0 upgrade guide](https://docs.nvidia.com/openshell/latest/upgrade/0-1-0). From d4f5034d7fb46883930f405202a65fd23563dfef Mon Sep 17 00:00:00 2001 From: Drew Newberry Date: Fri, 25 Sep 2026 23:43:18 +0000 Subject: [PATCH 3/6] fix(gateway)!: make the WebSocket tunnel opt-in (#3727) The /_ws_tunnel endpoint pipes a WebSocket into the full gRPC service. On a plaintext loopback gateway any web page could open it, since browsers do not apply CORS to WebSocket upgrades. Mount the tunnel only when enable_websocket_tunnel is set (config file, --enable-websocket-tunnel, or OPENSHELL_ENABLE_WEBSOCKET_TUNNEL; server.enableWebsocketTunnel in Helm). BREAKING CHANGE: gateways behind an authenticating edge proxy must set enable_websocket_tunnel = true for CLI edge-tunnel connections. Signed-off-by: Drew Newberry --- architecture/gateway.md | 3 +- crates/openshell-core/src/config.rs | 12 ++++++ crates/openshell-server/src/cli.rs | 41 ++++++++++++++++++- crates/openshell-server/src/config_file.rs | 3 ++ crates/openshell-server/src/http.rs | 14 ++++--- crates/openshell-server/src/lib.rs | 24 +++++++++++ deploy/helm/openshell/README.md | 1 + .../openshell/templates/gateway-config.yaml | 1 + .../openshell/tests/gateway_config_test.yaml | 16 ++++++++ deploy/helm/openshell/values.yaml | 3 ++ docs/how-it-works/gateways/authentication.mdx | 2 +- docs/how-it-works/gateways/configuration.mdx | 2 + e2e/with-docker-gateway.sh | 2 + e2e/with-podman-gateway.sh | 2 + skills/debug-openshell-cluster/SKILL.md | 4 +- 15 files changed, 120 insertions(+), 10 deletions(-) diff --git a/architecture/gateway.md b/architecture/gateway.md index 45cb5efd70..a301fae123 100644 --- a/architecture/gateway.md +++ b/architecture/gateway.md @@ -9,7 +9,8 @@ attachments; and asks compute runtimes to create or delete sandbox workloads. - Authenticate clients and sandbox supervisor sessions. - Serve gRPC APIs for sandbox lifecycle, provider management, policy updates, settings, logs, watch streams, and relay forwarding. -- Serve HTTP endpoints for health, WebSocket tunnels, and edge-auth flows. +- Serve HTTP endpoints for health and edge-auth flows, plus the opt-in + WebSocket tunnel for edge-proxy deployments. - Persist domain objects in SQLite or Postgres. - Resolve endpoint-bound provider environments for sandbox supervisors. - Coordinate supervisor relay sessions for connect, exec, file sync, and diff --git a/crates/openshell-core/src/config.rs b/crates/openshell-core/src/config.rs index 0d4a9f4325..c85fc4308a 100644 --- a/crates/openshell-core/src/config.rs +++ b/crates/openshell-core/src/config.rs @@ -194,6 +194,10 @@ pub struct Config { /// Gateway user authentication behavior. pub auth: GatewayAuthConfig, + /// Allow the WebSocket tunnel used by authenticated edge proxies. + /// Disabled for local gateways by default. + pub enable_websocket_tunnel: bool, + /// Disabled-by-default gateway interceptor service configs. pub gateway_interceptors: Vec, @@ -850,6 +854,7 @@ impl Config { tls, oidc: None, auth: GatewayAuthConfig::default(), + enable_websocket_tunnel: false, gateway_interceptors: Vec::new(), provider_profile_sources: vec![GatewayProviderProfileSourceConfig::User], mtls_auth: MtlsAuthConfig::default(), @@ -1022,6 +1027,13 @@ impl Config { self.service_routing.enable_loopback_service_http = enabled; self } + + /// Enable the WebSocket tunnel for an authenticated edge proxy. + #[must_use] + pub const fn with_websocket_tunnel(mut self, enabled: bool) -> Self { + self.enable_websocket_tunnel = enabled; + self + } } impl Default for ServiceRoutingConfig { diff --git a/crates/openshell-server/src/cli.rs b/crates/openshell-server/src/cli.rs index ca7396f2bc..f531d88c5d 100644 --- a/crates/openshell-server/src/cli.rs +++ b/crates/openshell-server/src/cli.rs @@ -257,6 +257,15 @@ struct RunArgs { action = ArgAction::Set )] enable_loopback_service_http: bool, + + /// Enable the WebSocket tunnel for an authenticated edge proxy. + #[arg( + long, + env = "OPENSHELL_ENABLE_WEBSOCKET_TUNNEL", + default_value_t = false, + action = ArgAction::Set + )] + enable_websocket_tunnel: bool, } pub fn command() -> Command { @@ -508,7 +517,8 @@ fn prepare_server_config_with_drivers( .unwrap_or_default(), ) .with_server_sans(args.server_sans.clone()) - .with_loopback_service_http(args.enable_loopback_service_http); + .with_loopback_service_http(args.enable_loopback_service_http) + .with_websocket_tunnel(args.enable_websocket_tunnel); if let Some(sources) = file .as_ref() .and_then(|file| file.openshell.gateway.provider_profile_sources.clone()) @@ -1089,6 +1099,11 @@ fn merge_file_into_args(args: &mut RunArgs, file: &GatewayFileSection, matches: { args.enable_loopback_service_http = enabled; } + if let Some(enabled) = file.enable_websocket_tunnel + && arg_defaulted(matches, "enable_websocket_tunnel") + { + args.enable_websocket_tunnel = enabled; + } if let Some(mtls_auth) = &file.mtls_auth && arg_defaulted(matches, "enable_mtls_auth") { @@ -1440,6 +1455,30 @@ mod tests { assert!(cli.run.enable_loopback_service_http); } + #[test] + fn websocket_tunnel_is_disabled_by_default_and_can_be_enabled_from_file() { + let _lock = ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + let _guard = EnvVarGuard::remove("OPENSHELL_ENABLE_WEBSOCKET_TUNNEL"); + let (mut args, matches) = + parse_with_args(&["openshell-gateway", "--db-url", "sqlite::memory:"]); + assert!(!args.enable_websocket_tunnel); + + let file = config_file_from_toml("[openshell.gateway]\nenable_websocket_tunnel = true\n"); + merge_file_into_args(&mut args, &file.openshell.gateway, &matches); + assert!(args.enable_websocket_tunnel); + + let (mut args, matches) = parse_with_args(&[ + "openshell-gateway", + "--db-url", + "sqlite::memory:", + "--enable-websocket-tunnel=false", + ]); + merge_file_into_args(&mut args, &file.openshell.gateway, &matches); + assert!(!args.enable_websocket_tunnel, "CLI flag must override file"); + } + #[test] fn command_disables_loopback_service_http_with_false_value() { let _lock = ENV_LOCK diff --git a/crates/openshell-server/src/config_file.rs b/crates/openshell-server/src/config_file.rs index 6cde6aa461..87180766ab 100644 --- a/crates/openshell-server/src/config_file.rs +++ b/crates/openshell-server/src/config_file.rs @@ -132,6 +132,9 @@ pub struct GatewayFileSection { /// Enable plaintext HTTP routing for loopback sandbox service URLs. #[serde(default)] pub enable_loopback_service_http: Option, + /// Enable the WebSocket tunnel for an authenticated edge proxy. + #[serde(default)] + pub enable_websocket_tunnel: Option, // ── Sandbox client TLS ─────────────────────────────────────────────── #[serde(default)] diff --git a/crates/openshell-server/src/http.rs b/crates/openshell-server/src/http.rs index 63b59edf04..cf37aa18b9 100644 --- a/crates/openshell-server/src/http.rs +++ b/crates/openshell-server/src/http.rs @@ -179,12 +179,14 @@ async fn render_metrics(State(handle): State) -> impl IntoResp /// Create the HTTP router served on the multiplexed gateway port. pub fn http_router(state: Arc) -> Router { - crate::ws_tunnel::router(state.clone()) - .merge(crate::auth::router(state.clone())) - .layer(middleware::from_fn_with_state( - state, - sandbox_service_routing_first, - )) + let mut router = crate::auth::router(state.clone()); + if state.config.enable_websocket_tunnel { + router = router.merge(crate::ws_tunnel::router(state.clone())); + } + router.layer(middleware::from_fn_with_state( + state, + sandbox_service_routing_first, + )) } /// Create the plaintext loopback-only router for browser service endpoints. diff --git a/crates/openshell-server/src/lib.rs b/crates/openshell-server/src/lib.rs index a51e17a9ad..ea1f74c0f4 100644 --- a/crates/openshell-server/src/lib.rs +++ b/crates/openshell-server/src/lib.rs @@ -1913,6 +1913,9 @@ mod tests { use tokio::sync::watch; use crate::tls_test_utils::generate_test_certs_with_ca; + use axum::body::Body; + use http::{Request, StatusCode}; + use tower::ServiceExt; fn tls_enabled_config() -> Config { Config::new(Some(openshell_core::TlsConfig { @@ -2174,6 +2177,27 @@ mod tests { )) } + #[tokio::test] + async fn websocket_tunnel_is_mounted_only_when_enabled() { + let state = test_state("127.0.0.1:17670".parse().unwrap(), true).await; + let response = super::http_router(state.clone()) + .oneshot(Request::get("/_ws_tunnel").body(Body::empty()).unwrap()) + .await + .unwrap(); + assert_eq!(response.status(), StatusCode::NOT_FOUND); + + let mut enabled = state; + Arc::get_mut(&mut enabled) + .unwrap() + .config + .enable_websocket_tunnel = true; + let response = super::http_router(enabled) + .oneshot(Request::get("/_ws_tunnel").body(Body::empty()).unwrap()) + .await + .unwrap(); + assert_ne!(response.status(), StatusCode::NOT_FOUND); + } + async fn start_tls_gateway_listener( bind_addr: &str, enable_loopback_service_http: bool, diff --git a/deploy/helm/openshell/README.md b/deploy/helm/openshell/README.md index 4ab3517b27..4d09c32de8 100644 --- a/deploy/helm/openshell/README.md +++ b/deploy/helm/openshell/README.md @@ -306,6 +306,7 @@ discovery endpoint or its TLS CA. | server.drivers.kubernetes.workspaceMode | string | `"shared"` | How workspaces map to Kubernetes namespaces. "shared" (default): all sandboxes in a single namespace. "managed": auto-creates per-workspace namespaces. "operator": uses pre-provisioned namespaces. | | server.enableLoopbackServiceHttp | bool | `true` | Enable plaintext HTTP routing for loopback sandbox service URLs on TLS-enabled gateways. | | server.enableUserNamespaces | bool | `false` | Enable Kubernetes user namespace isolation (hostUsers: false) for sandbox pods. Requires Kubernetes 1.33+ with user namespace support available (beta through 1.35, GA in 1.36+), plus a supporting container runtime and Linux 5.12+. When enabled, container UID 0 maps to an unprivileged host UID and capabilities become namespaced. | +| server.enableWebsocketTunnel | bool | `false` | Enable the WebSocket tunnel used by CLI/SDK clients behind an authenticated edge proxy. Leave disabled for direct gateway installs. | | server.externalDbSecret | string | `""` | Name of a pre-existing Opaque Secret containing a PostgreSQL connection URI (key: uri). When set, the gateway reads OPENSHELL_DB_URL from this Secret instead of using dbUrl. The Secret must contain a `uri` key, e.g. postgresql://user:pass@host:5432/dbname. | | server.grpcEndpoint | string | `""` | gRPC endpoint sandboxes call back into the gateway. Leave empty to derive it from the chart fullname, release namespace, service port, and disableTls flag, for example https://openshell.openshell.svc.cluster.local:8080. Override only when sandboxes must reach the gateway via a different hostname (e.g. an external ingress or a host alias). | | server.grpcRateLimit.requests | int | `0` | Maximum gRPC requests allowed per window. Must be positive (alongside windowSeconds) to enable rate limiting; 0 (default) disables it. | diff --git a/deploy/helm/openshell/templates/gateway-config.yaml b/deploy/helm/openshell/templates/gateway-config.yaml index b1ec38bd0a..0449871f7d 100644 --- a/deploy/helm/openshell/templates/gateway-config.yaml +++ b/deploy/helm/openshell/templates/gateway-config.yaml @@ -57,6 +57,7 @@ data: disable_tls = true {{- end }} enable_loopback_service_http = {{ .Values.server.enableLoopbackServiceHttp }} + enable_websocket_tunnel = {{ .Values.server.enableWebsocketTunnel }} {{- $sans := list -}} {{- if and .Values.certManager.enabled .Values.certManager.serverDnsNames }} {{- $sans = .Values.certManager.serverDnsNames }} diff --git a/deploy/helm/openshell/tests/gateway_config_test.yaml b/deploy/helm/openshell/tests/gateway_config_test.yaml index 83fbb7aa08..eacfdfa2c4 100644 --- a/deploy/helm/openshell/tests/gateway_config_test.yaml +++ b/deploy/helm/openshell/tests/gateway_config_test.yaml @@ -13,6 +13,22 @@ release: namespace: my-namespace tests: + - it: disables the WebSocket tunnel unless explicitly enabled for an edge proxy + template: templates/gateway-config.yaml + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?m)^enable_websocket_tunnel\s*=\s*false$' + + - it: enables the WebSocket tunnel when requested + template: templates/gateway-config.yaml + set: + server.enableWebsocketTunnel: true + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?m)^enable_websocket_tunnel\s*=\s*true$' + - it: defaults to label admission with caller driver config disabled template: templates/gateway-config.yaml asserts: diff --git a/deploy/helm/openshell/values.yaml b/deploy/helm/openshell/values.yaml index b48a7c269f..426f3ae424 100644 --- a/deploy/helm/openshell/values.yaml +++ b/deploy/helm/openshell/values.yaml @@ -328,6 +328,9 @@ server: # -- Enable plaintext HTTP routing for loopback sandbox service URLs on # TLS-enabled gateways. enableLoopbackServiceHttp: true + # -- Enable the WebSocket tunnel used by CLI/SDK clients behind an + # authenticated edge proxy. Leave disabled for direct gateway installs. + enableWebsocketTunnel: false # -- Posture when a candidate sandbox policy fails validation. `fail_closed` # deactivates the previous policy; `retain_last_valid` keeps it active. policyValidationFailureMode: fail_closed diff --git a/docs/how-it-works/gateways/authentication.mdx b/docs/how-it-works/gateways/authentication.mdx index a05cf7f678..7abb989276 100644 --- a/docs/how-it-works/gateways/authentication.mdx +++ b/docs/how-it-works/gateways/authentication.mdx @@ -246,7 +246,7 @@ a workspace. For membership commands, refer to ### Edge JWT (cloud gateways) -For gateways behind a reverse proxy that handles authentication (e.g. Cloudflare Access), the CLI uses a browser-based login flow and routes traffic through a WebSocket tunnel. +For gateways behind a reverse proxy that handles authentication (e.g. Cloudflare Access), the CLI uses a browser-based login flow and routes traffic through a WebSocket tunnel. The tunnel is disabled by default; enable it with `enable_websocket_tunnel = true` under `[openshell.gateway]` (Helm: `server.enableWebsocketTunnel=true`). **Registration flow** (`openshell gateway add https://gateway.example.com`): diff --git a/docs/how-it-works/gateways/configuration.mdx b/docs/how-it-works/gateways/configuration.mdx index c1963c3253..fc9d5aa8c2 100644 --- a/docs/how-it-works/gateways/configuration.mdx +++ b/docs/how-it-works/gateways/configuration.mdx @@ -149,6 +149,8 @@ policy_validation_failure_mode = "fail_closed" server_sans = ["openshell", "*.dev.openshell.localhost"] # Allow plaintext HTTP routing for loopback sandbox service URLs. enable_loopback_service_http = true +# Serve the WebSocket tunnel for CLI clients behind an authenticating edge proxy. +enable_websocket_tunnel = false # Set true only for local plaintext gateways or trusted TLS termination. disable_tls = false diff --git a/e2e/with-docker-gateway.sh b/e2e/with-docker-gateway.sh index 9d1197a21d..4b96693ab1 100755 --- a/e2e/with-docker-gateway.sh +++ b/e2e/with-docker-gateway.sh @@ -662,6 +662,8 @@ GATEWAY_ARGS=( --compute-driver docker --tls-cert "${PKI_DIR}/server/tls.crt" --tls-key "${PKI_DIR}/server/tls.key" + # edge_tunnel_e2e exercises the opt-in WebSocket tunnel. + --enable-websocket-tunnel true --db-url "sqlite:${STATE_DIR}/gateway.db?mode=rwc" ) diff --git a/e2e/with-podman-gateway.sh b/e2e/with-podman-gateway.sh index 3b3c5cf46c..7350dd07e5 100755 --- a/e2e/with-podman-gateway.sh +++ b/e2e/with-podman-gateway.sh @@ -856,6 +856,8 @@ GATEWAY_ARGS=( --health-port "${HEALTH_PORT}" --tls-cert "${PKI_DIR}/server/tls.crt" --tls-key "${PKI_DIR}/server/tls.key" + # edge_tunnel_e2e exercises the opt-in WebSocket tunnel. + --enable-websocket-tunnel true --db-url "sqlite:${STATE_DIR}/gateway.db?mode=rwc" --log-level info ) diff --git a/skills/debug-openshell-cluster/SKILL.md b/skills/debug-openshell-cluster/SKILL.md index 8e865f13ec..d654d38094 100644 --- a/skills/debug-openshell-cluster/SKILL.md +++ b/skills/debug-openshell-cluster/SKILL.md @@ -123,7 +123,9 @@ Gateway configuration requires `[openshell] version = 2`, a singular `--drivers` selectors rather than silently migrating them. One valid, nonempty `OPENSHELL_DRIVERS` value remains a deprecated environment-only alias when the canonical selector is absent; the gateway selects that driver with a warning. -Empty, invalid, comma-delimited, or conflicting values fail startup. Homebrew +Empty, invalid, comma-delimited, or conflicting values fail startup. The +WebSocket tunnel for edge-proxy CLI access is off unless +`enable_websocket_tunnel = true` (`server.enableWebsocketTunnel` in Helm). Homebrew and RPM package startup migrates only exact package-generated v1 defaults. If an upgraded package still reports an unsupported version, inspect the active prefix or `~/.config/openshell/gateway.toml`; an edited v1 file must follow the published schema-v2 migration steps and must not be overwritten. From b52eed72e30d26b73272d92858472fa321f8cad1 Mon Sep 17 00:00:00 2001 From: grs Date: Sat, 26 Sep 2026 00:01:04 +0000 Subject: [PATCH 4/6] test(binary-identity): stabilize procfs identity fixtures (#3708) * test(binary-identity): stabilize procfs identity fixtures Signed-off-by: Gordon Sim * test(binary-identity): wait for child fixture readiness Signed-off-by: Gordon Sim --------- Signed-off-by: Gordon Sim --- crates/openshell-binary-identity/src/lib.rs | 39 +++++++++++++++++---- 1 file changed, 32 insertions(+), 7 deletions(-) diff --git a/crates/openshell-binary-identity/src/lib.rs b/crates/openshell-binary-identity/src/lib.rs index ed4dde362a..5f9b72de6c 100644 --- a/crates/openshell-binary-identity/src/lib.rs +++ b/crates/openshell-binary-identity/src/lib.rs @@ -462,6 +462,8 @@ fn cmdline_absolute_paths(cmdline: &[u8]) -> Vec { mod tests { use super::*; + const IDENTITY_HELPER_READY: &str = "identity helper ready\n"; + struct ChildGuard(std::process::Child); impl Drop for ChildGuard { @@ -474,20 +476,22 @@ mod tests { #[cfg(target_os = "linux")] #[test] fn resolver_cache_is_owned_and_only_explicit_clones_share_it() { - let first = ProcfsIdentityResolver::for_pid_namespace(); + let pid = std::process::id(); + let first = ProcfsIdentityResolver::for_process_tree(pid); let shared = first.clone(); - let separate = ProcfsIdentityResolver::for_pid_namespace(); + let separate = ProcfsIdentityResolver::for_process_tree(pid); assert!(Arc::ptr_eq(&first.cache, &shared.cache)); assert!(!Arc::ptr_eq(&first.cache, &separate.cache)); - first.resolve(std::process::id()).unwrap(); + first.resolve(pid).unwrap(); assert!(!shared.cache.lock().unwrap().is_empty()); assert!(separate.cache.lock().unwrap().is_empty()); } #[test] fn resolves_current_process_from_live_executable() { - let identity = ProcfsIdentityResolver::for_pid_namespace() - .resolve(std::process::id()) + let pid = std::process::id(); + let identity = ProcfsIdentityResolver::for_process_tree(pid) + .resolve(pid) .expect("resolve current process"); assert!(identity.executable.path.is_absolute()); @@ -497,16 +501,37 @@ mod tests { #[test] #[ignore = "subprocess fixture for executable identity tests"] fn identity_helper_process() { - std::thread::sleep(std::time::Duration::from_secs(30)); + let mut stdout = std::io::stdout(); + std::io::Write::write_all(&mut stdout, IDENTITY_HELPER_READY.as_bytes()) + .expect("signal helper readiness"); + std::io::Write::flush(&mut stdout).expect("flush helper readiness"); + + // Stay alive until the parent closes stdin or ChildGuard terminates us. + let _ = std::io::Read::read(&mut std::io::stdin(), &mut [0_u8]); } #[test] fn resolves_child_and_hashes_ancestor_chain() { let parent_pid = std::process::id(); - let child = std::process::Command::new(std::env::current_exe().unwrap()) + let mut child = std::process::Command::new(std::env::current_exe().unwrap()) .args(["--ignored", "--exact", "tests::identity_helper_process"]) + .stdin(std::process::Stdio::piped()) + .stdout(std::process::Stdio::piped()) .spawn() .expect("spawn child"); + let mut child_stdout = std::io::BufReader::new(child.stdout.take().expect("child stdout")); + let mut line = String::new(); + // Wait until the helper has entered the test before taking the + // fail-closed procfs snapshot of the newly spawned process. + loop { + line.clear(); + let bytes_read = std::io::BufRead::read_line(&mut child_stdout, &mut line) + .expect("read child readiness"); + assert_ne!(bytes_read, 0, "child exited before signaling readiness"); + if line == IDENTITY_HELPER_READY { + break; + } + } let child = ChildGuard(child); let identity = ProcfsIdentityResolver::for_process_tree(parent_pid) From f1558995279f63b3d1d0b1ed4f164551d6281f02 Mon Sep 17 00:00:00 2001 From: Johnny Greco Date: Sat, 26 Sep 2026 01:38:59 +0000 Subject: [PATCH 5/6] docs(tutorials): run Pi with OpenRouter (#3722) * docs(tutorials): run Pi with OpenRouter Switch the Pi tutorial from Anthropic to OpenRouter, add fd to the Pi image so Pi does not try to download it from GitHub inside the sandbox, and rename the page to Run Pi with OpenRouter with a dev redirect from the old URL. Signed-off-by: Johnny Greco * docs(tutorials): drop redirect for dev-only Pi page Signed-off-by: Johnny Greco * docs(tutorials): make the Pi tutorial easier to follow Add prerequisites and steps, explain providers and the profile fields in plain terms, inspect the sandbox while Pi is still running, and add clean-up and troubleshooting sections. Signed-off-by: Johnny Greco * docs(tutorials): clarify Pi's providers and sandbox additions Signed-off-by: Johnny Greco * docs(tutorials): make the Pi tutorial more conversational Signed-off-by: Johnny Greco --------- Signed-off-by: Johnny Greco --- docs/about/run-an-agent.mdx | 4 +- docs/index.yml | 5 +- docs/tutorials/index.mdx | 4 +- docs/tutorials/run-pi-with-openrouter.mdx | 233 ++++++++++++++++++++++ docs/tutorials/run-pi.mdx | 168 ---------------- 5 files changed, 240 insertions(+), 174 deletions(-) create mode 100644 docs/tutorials/run-pi-with-openrouter.mdx delete mode 100644 docs/tutorials/run-pi.mdx diff --git a/docs/about/run-an-agent.mdx b/docs/about/run-an-agent.mdx index 96ab04abe7..b5f7af0aae 100644 --- a/docs/about/run-an-agent.mdx +++ b/docs/about/run-an-agent.mdx @@ -101,8 +101,8 @@ to [Policy Advisor](/how-it-works/policies/advisor). ## Examples -For a complete worked example, follow [Run Pi in OpenShell](/tutorials/run-pi). -The tutorial builds a Pi image, configures narrowly scoped Anthropic access, +For a complete worked example, follow [Run Pi with OpenRouter](/tutorials/run-pi-with-openrouter). +The tutorial builds a Pi image, configures narrowly scoped OpenRouter access, uploads a project, and starts the agent inside the sandbox boundary. ## Next Steps diff --git a/docs/index.yml b/docs/index.yml index e4b3ff9245..defa2106e6 100644 --- a/docs/index.yml +++ b/docs/index.yml @@ -103,8 +103,9 @@ navigation: - section: "Tutorials" slug: tutorials contents: - - page: "Run Pi" - path: tutorials/run-pi.mdx + - page: "Run Pi with OpenRouter" + path: tutorials/run-pi-with-openrouter.mdx + slug: run-pi-with-openrouter - page: "First Network Policy" path: tutorials/first-network-policy.mdx - page: "GitHub Push Access" diff --git a/docs/tutorials/index.mdx b/docs/tutorials/index.mdx index 1beee6d434..22e9647f69 100644 --- a/docs/tutorials/index.mdx +++ b/docs/tutorials/index.mdx @@ -11,9 +11,9 @@ Hands-on walkthroughs that teach OpenShell concepts by building real configurati - + -Build a Pi coding-agent image, configure Anthropic access, and run it inside an OpenShell sandbox. +Build a Pi coding-agent image, configure OpenRouter access, and run it inside an OpenShell sandbox. diff --git a/docs/tutorials/run-pi-with-openrouter.mdx b/docs/tutorials/run-pi-with-openrouter.mdx new file mode 100644 index 0000000000..4e57903f6c --- /dev/null +++ b/docs/tutorials/run-pi-with-openrouter.mdx @@ -0,0 +1,233 @@ +--- +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +title: "Run Pi with OpenRouter" +description: "Build a Pi coding-agent image, configure OpenRouter access, and run Pi inside an OpenShell sandbox." +keywords: "Generative AI, Cybersecurity, Tutorial, Pi Coding Agent, OpenRouter, Sandbox, Provider Profile" +--- + +In this tutorial, we'll run [Pi](https://pi.dev), a terminal coding agent, +inside an OpenShell sandbox. Pi works with many model providers out of the box, +including [OpenRouter](https://openrouter.ai), which we use here. + +We'll build a Pi image, give the sandbox access to OpenRouter, and start Pi. +The sandbox lets Pi reach OpenRouter but nothing else on the network, and Pi +never sees your real API key. + +## Prerequisites + +You'll need: + +- A working OpenShell installation with an active gateway. Refer to + [Installation](/about/installation). +- Docker or Podman to build the image. +- An OpenRouter API key. + + + +## Build the Pi Image + +Pi doesn't publish a container image, so we'll build one from its npm package, +based on Pi's [container guide](https://pi.dev/docs/latest/containerization). +Save this Dockerfile as `Dockerfile.pi`: + +```dockerfile +FROM node:24-bookworm-slim + +ARG PI_VERSION=latest + +RUN apt-get update \ + && apt-get install -y --no-install-recommends bash ca-certificates fd-find git ripgrep \ + && ln -s /usr/bin/fdfind /usr/local/bin/fd \ + && rm -rf /var/lib/apt/lists/* + +RUN npm install -g --ignore-scripts "@earendil-works/pi-coding-agent@${PI_VERSION}" + +RUN mkdir -p /workspace && chown node:node /workspace +USER node +WORKDIR /workspace + +ENV PI_CODING_AGENT_DIR=/tmp/pi-agent +``` + +We add two things to Pi's guide because the sandbox limits what Pi can reach: + +- We install `fd`. Pi uses `fd` and `ripgrep` to search files, and if they're + missing, it downloads them from GitHub on first use. The sandbox blocks that + download, so the image includes both. +- We set `PI_CODING_AGENT_DIR` so Pi keeps its settings and sessions in `/tmp`, + one of the few places the sandbox lets Pi write. + +Now build the image: + +```shell +docker build -t pi-agent:local -f Dockerfile.pi . +``` + + +If your gateway uses Podman, build with +`podman build -t localhost/pi-agent:local -f Dockerfile.pi .` and use +`localhost/pi-agent:local` in the commands below. If your gateway runs on +another machine, push the image to a registry the gateway can pull from, and +use that image reference with `--from`. + + +## Give Pi Access to OpenRouter + +A sandbox has no network access until you grant it. To let Pi reach +OpenRouter, we'll create a provider, which stores your API key and opens access +to the service. Pi only ever sees a placeholder for the key. OpenShell swaps in +the real key only when Pi calls OpenRouter. + +A provider is built from a provider profile, which describes the service. Save +this one as `provider-openrouter.yaml`: + +```yaml +id: openrouter +display_name: OpenRouter +description: Direct OpenRouter inference access +category: inference +inference_capable: true + +credentials: + - name: api_key + description: OpenRouter API key + env_vars: + - OPENROUTER_API_KEY + required: true + auth_style: bearer + header_name: authorization + +discovery: + credentials: + - api_key + +endpoints: + - host: openrouter.ai + port: 443 + protocol: rest + access: read-write + enforcement: enforce + +binaries: + - /usr/local/bin/node +``` + +Here's what the main sections do: + +- `credentials` declares the API key. Pi finds the placeholder in + `OPENROUTER_API_KEY`. +- `endpoints` allows only `openrouter.ai`, and OpenShell sends the key nowhere + else. +- `binaries` allows only `node` to use this access. Pi runs on Node.js, so + that includes Pi. + +Check the profile, import it, and create a provider named `openrouter`: + +```shell +openshell profile lint -f provider-openrouter.yaml +openshell profile import -f provider-openrouter.yaml + +OPENROUTER_API_KEY=your-key \ + openshell provider create \ + --name openrouter \ + --type openrouter \ + --from-existing +``` + +Replace `your-key` with your OpenRouter API key, or drop that prefix if +`OPENROUTER_API_KEY` is already set in your shell. `--from-existing` tells +OpenShell to read the key from that variable. + +## Start Pi in a Sandbox + +Next, create a sandbox from the image, attach the provider, and start Pi: + +```shell +openshell sandbox create \ + --name pi \ + --from pi-agent:local \ + --provider openrouter \ + -- pi +``` + +Everything after `--` runs inside the sandbox, so Pi and any commands it runs +stay contained. + +Pi starts with its default OpenRouter model. To pick a different one, type +`/model` and choose any OpenRouter model from the list. OpenRouter usually has +some free models, marked with a `:free` suffix, which are handy for trying this +out without spending credits. + +Then give Pi a try: + +```text +Explain what files are available in this workspace. +``` + +The workspace starts empty, but we'll give Pi your own code in a later step. + +## Inspect the Sandbox + +While Pi is running, open another terminal and take a look at the sandbox's +status, access, providers, and logs: + +```shell +openshell sandbox list +openshell policy get pi --full +openshell sandbox provider list pi +openshell logs pi --tail +``` + +`openshell logs` streams the sandbox's activity, including any connections it +blocks. When you're done, type `/quit` in Pi to exit. + +## Work on a Local Project + +To point Pi at your own code, run this from your project directory. `--upload` +copies the directory into the sandbox before Pi starts, skipping anything +`.gitignore` excludes. + +```shell +openshell sandbox create \ + --name pi-project \ + --from pi-agent:local \ + --provider openrouter \ + --upload .:/workspace \ + -- pi +``` + +Pi's changes stay inside the sandbox. To copy them back, run +`openshell sandbox download` from another terminal while Pi is still running. +Refer to [Transfer Files](/how-it-works/sandboxes/overview#transfer-files). + +## Clean Up + +When you're finished, delete both sandboxes: + +```shell +openshell sandbox delete pi pi-project +``` + + + +## Troubleshooting + +- If the gateway can't find `pi-agent:local`, make sure you built the image + with the same container engine your gateway uses, or pushed it to a registry + the gateway can reach. +- If Pi's requests to OpenRouter are denied, make sure the `binaries` path in + the profile matches where `node` is installed in your image. In the image + above, that's `/usr/local/bin/node`. + +## Next Steps + +- To give Pi more tools, add them to the image. To let Pi reach package + registries, source hosts, or other services, grant access with a + [sandbox policy](/how-it-works/policies/overview). +- To use a different model service, write a provider profile for it. The + [`providers/`](https://github.com/NVIDIA/OpenShell/tree/main/providers) + directory has examples, and [Profiles](/how-it-works/providers/profiles) + describes each field. +- To learn about images, resources, file transfer, and the sandbox lifecycle, + refer to [Sandboxes](/how-it-works/sandboxes/overview). diff --git a/docs/tutorials/run-pi.mdx b/docs/tutorials/run-pi.mdx deleted file mode 100644 index 141db5571f..0000000000 --- a/docs/tutorials/run-pi.mdx +++ /dev/null @@ -1,168 +0,0 @@ ---- -# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -# SPDX-License-Identifier: Apache-2.0 -title: "Run Pi in OpenShell" -sidebar-title: "Run Pi" -description: "Build a Pi coding-agent image, configure Anthropic access, and run Pi inside an OpenShell sandbox." -keywords: "Generative AI, Cybersecurity, Tutorial, Pi Coding Agent, Anthropic, Sandbox, Provider Profile" ---- - - -This tutorial runs [Pi](https://pi.dev), a terminal coding agent, with the -Anthropic API. You will build Pi into an OCI image, configure narrowly scoped -model access, and run it inside OpenShell. - -## Build the Pi Image - -Pi does not publish an official OCI image. Its -[container guide](https://pi.dev/docs/latest/containerization) installs the -maintained `@earendil-works/pi-coding-agent` npm package into a Node.js image. -Create `Dockerfile.pi` with the same approach: - -```dockerfile -FROM node:24-bookworm-slim - -ARG PI_VERSION=latest - -RUN apt-get update \ - && apt-get install -y --no-install-recommends bash ca-certificates git ripgrep \ - && rm -rf /var/lib/apt/lists/* - -RUN npm install -g --ignore-scripts "@earendil-works/pi-coding-agent@${PI_VERSION}" - -RUN mkdir -p /workspace && chown node:node /workspace -USER node -WORKDIR /workspace - -ENV PI_CODING_AGENT_DIR=/tmp/pi-agent -``` - -Build the image with the container engine used by your local gateway: - -```shell -docker build -t pi-agent:local -f Dockerfile.pi . -``` - -For Podman, use `podman build -t localhost/pi-agent:local -f Dockerfile.pi .` -and substitute `localhost/pi-agent:local` in the commands below. For a remote -gateway, push the image to a registry the gateway can pull from. Pin -`PI_VERSION` to a reviewed release when you publish a reproducible image. - -## Configure Anthropic Access - -Create `pi-anthropic.yaml`. The profile permits only the Pi executable from -this image to reach the Anthropic API. OpenShell supplies a credential -placeholder to Pi and replaces it only on requests to the declared endpoint. - -```yaml -id: pi-anthropic -display_name: Pi with Anthropic -description: Anthropic model access for the Pi coding agent -category: agent -inference_capable: true -credentials: - - name: api_key - description: Anthropic API key used by Pi - env_vars: [ANTHROPIC_API_KEY] - required: true - auth_style: header - header_name: x-api-key -discovery: - credentials: [api_key] -endpoints: - - host: api.anthropic.com - port: 443 - protocol: rest - access: read-write - enforcement: enforce -binaries: - - /usr/local/bin/pi - - /usr/local/lib/node_modules/@earendil-works/pi-coding-agent/** -``` - -Lint and import the profile into your current workspace, then store an -Anthropic API key in a provider: - -```shell -openshell provider profile lint -f pi-anthropic.yaml -openshell provider profile import -f pi-anthropic.yaml - -ANTHROPIC_API_KEY=your-key \ - openshell provider create \ - --name pi-anthropic \ - --type pi-anthropic \ - --from-existing -``` - -## Create the Pi Sandbox - -Create the sandbox, attach the provider, and run Pi interactively: - -```shell -openshell sandbox create \ - --name pi \ - --from pi-agent:local \ - --provider pi-anthropic \ - -- pi -``` - -The command after `--` is the sandbox's main process. Pi and every command it -starts run inside the sandbox boundary. Try asking Pi: - -```text -Explain what files are available in this workspace. -``` - -To give Pi a local project, upload it before the main process starts: - -```shell -openshell sandbox create \ - --name pi-project \ - --from pi-agent:local \ - --provider pi-anthropic \ - --upload .:/workspace \ - -- pi -``` - -The provider grants Pi access to `api.anthropic.com`. The built-in fallback -policy grants access to the working directory and standard runtime paths while -denying all other network egress. Add a reviewed custom policy when Pi needs -package registries, source hosts, tool servers, or other destinations. - -## Adapt the Pi Workflow - -Pi supports multiple model providers. Create or adapt a provider profile for -the service you use, and make its `binaries` paths match the Pi installation in -your image. The OpenShell repository contains reviewable profile examples in -[`providers/`](https://github.com/NVIDIA/OpenShell/tree/main/providers). - -Install any additional compilers, package managers, and agent tools in the -image. Grant their filesystem and network requirements through -[sandbox policy](/how-it-works/policies/overview) instead of giving the workload broad -access. Refer to [Profiles](/how-it-works/providers/profiles) for credential and endpoint -configuration. - -## Verify and Troubleshoot - -Inspect the sandbox, effective policy, provider attachments, and logs: - -```shell -openshell sandbox list -openshell policy get pi --full -openshell sandbox provider list pi -openshell logs pi --tail -``` - -If `pi` is missing, confirm that the image was built with the selected local -container engine or pushed to a registry visible to the gateway. If a model -request is denied, confirm that the profile's `binaries` paths match -`command -v pi` and `npm root -g` inside the image. Review any other denied -network or filesystem operation before updating the sandbox policy. - -## Next Steps - -- To give Pi access to source control, package registries, or tool servers, add - the corresponding provider profiles or a reviewed [sandbox policy](/how-it-works/policies/overview). -- To understand image, resource, upload, and lifecycle options, refer to - [Sandboxes](/how-it-works/sandboxes/overview). -- To configure another model service, refer to [Profiles](/how-it-works/providers/profiles). From a67567e583454021d6665c056d092769136a3edd Mon Sep 17 00:00:00 2001 From: Drew Newberry Date: Sat, 26 Sep 2026 01:50:02 +0000 Subject: [PATCH 6/6] fix(snap): require mTLS for the snap gateway (#3726) * fix(snap): require mTLS for the snap gateway Replace the installer opt-in with an authenticated snap gateway. The wrapper no longer forces plaintext, so the gateway serves TLS from the bundle it already generates in $SNAP_COMMON/tls. The install hook writes a config that enables mTLS user auth instead of unauthenticated access, and a new post-refresh hook migrates the exact legacy default on existing installs. install.sh waits for the gateway, detects whether it serves TLS, copies the client bundle into the target user's snap state directory, and registers the gateway over HTTPS. Older plaintext snap revisions still register over HTTP with a warning. The release canary asserts mTLS auth and HTTPS registration. Signed-off-by: Drew Newberry * fix(snap): pass config preflight and detect the mTLS gateway reliably An explicit [openshell.gateway.mtls_auth] table fails config preflight, which validates mTLS auth before the local TLS bundle supplies the client CA. Write a default that pins the Docker driver instead; with the wrapper's TLS bundle the gateway requires client certificates and enables mTLS user auth automatically, as the native packages do. The mTLS gateway rejects TLS handshakes without a client certificate, and it still answers plaintext loopback HTTP for sandbox service routing, so the installer could misdetect it as a legacy plaintext gateway. Probe HTTPS with the root-owned client bundle, and treat a gateway as legacy only when a plaintext gRPC Health call succeeds. Signed-off-by: Drew Newberry * chore(snap): simplify install hook comment Signed-off-by: Drew Newberry * fix(snap): migrate insecure gateway configs on refresh Signed-off-by: Drew Newberry * refactor(snap): simplify mTLS detection and config migration Detect the mTLS snap from the installed revision's post-refresh hook instead of probing plaintext gRPC, and drop the scheme global. Remove the installer's pre-hook config fallback, which is dead now that every channel ships the install hook and which wrote the insecure default. Give the install hook a single write path with a simple backup name, and shorten the manual client certificate steps. Signed-off-by: Drew Newberry * fix(snap): stop keeping a copy of replaced insecure configs Signed-off-by: Drew Newberry * feat(install): make the snap an opt-in install method Stop selecting the OpenShell snap just because the snap command exists. Linux installs default to the Debian or RPM package; OPENSHELL_INSTALL_METHOD=snap (or deb, rpm) selects the package explicitly. Hosts that already have the OpenShell snap keep refreshing it rather than gaining a second gateway on the same port. The release canary and snap repro script opt in explicitly. Signed-off-by: Drew Newberry * fix(snap): let the gateway auto-detect its compute driver Signed-off-by: Drew Newberry * fix(snap): restart the gateway after refresh Published revisions use refresh-mode: endure, and snapd honors the old revision's setting during a refresh, so the plaintext gateway kept running with the migrated config unused until a manual restart. Restart the gateway from the post-refresh hook so the mTLS config takes effect immediately. Signed-off-by: Drew Newberry * docs(snap): drop refresh notes from the snap description Signed-off-by: Drew Newberry * docs(snap): trim snap refresh notes from installation docs Signed-off-by: Drew Newberry --------- Signed-off-by: Drew Newberry --- .github/workflows/release-canary.yml | 7 + architecture/build.md | 32 ++-- docs/about/installation.mdx | 24 +-- install.sh | 121 +++++++------ nix/test-guest/scripts/snap-gateway-repro.sh | 4 +- python/openshell/release_formula_test.py | 6 +- skills/debug-openshell-cluster/SKILL.md | 1 + snap/hooks/install | 33 ++-- snap/hooks/post-refresh | 12 ++ snapcraft.yaml | 39 ++--- tasks/scripts/snap-gateway-wrapper.sh | 16 +- tasks/scripts/test-install-sh.sh | 173 +++++++++++++------ tasks/scripts/test-packaging-assets.sh | 11 +- tasks/scripts/test-snap-gateway-wrapper.sh | 26 +-- tasks/scripts/test-snap-install-hook.sh | 88 ++++++++++ 15 files changed, 404 insertions(+), 189 deletions(-) create mode 100755 snap/hooks/post-refresh diff --git a/.github/workflows/release-canary.yml b/.github/workflows/release-canary.yml index 239f49eced..f6e729fb40 100644 --- a/.github/workflows/release-canary.yml +++ b/.github/workflows/release-canary.yml @@ -195,6 +195,8 @@ jobs: name: Ubuntu Snap with system Docker if: ${{ github.event_name == 'workflow_dispatch' || github.event.workflow_run.conclusion == 'success' }} runs-on: ubuntu-latest + env: + OPENSHELL_INSTALL_METHOD: snap timeout-minutes: 20 steps: - name: Install snapd @@ -232,6 +234,9 @@ jobs: sudo snap connections openshell | grep -E '^docker +openshell:docker +:docker +' openshell --version sudo snap services openshell + sudo journalctl -b -u snap.openshell.gateway.service --no-pager | + grep -F "mTLS user authentication enabled" + openshell gateway list | grep -F "https://127.0.0.1:17670" openshell status - name: Create and exercise a sandbox @@ -262,6 +267,8 @@ jobs: name: Ubuntu Snap Docker preflight if: ${{ github.event_name == 'workflow_dispatch' || github.event.workflow_run.conclusion == 'success' }} runs-on: ubuntu-latest + env: + OPENSHELL_INSTALL_METHOD: snap timeout-minutes: 20 steps: - name: Install snapd diff --git a/architecture/build.md b/architecture/build.md index 43161b8c0c..639dde8ba8 100644 --- a/architecture/build.md +++ b/architecture/build.md @@ -324,24 +324,30 @@ for direct executable installation on every environment. Release Dev and Release Tag run Ubuntu conformance through the Debian package, while Fedora continues using direct executable installation until RPM coverage is available. The release canary separately exercises the public installer on Ubuntu. The -OpenShell Snap requires a compatible, preinstalled non-Snap Docker daemon. Its -positive canary uses system Docker; negative preflight coverage verifies that -the installer rejects both missing Docker and the Docker Snap before installing -OpenShell. Its Debian lane removes snapd before running the installer so Snap -precedence cannot change the package under test. -Explicit release tags and the `pre` alias bypass Snap selection and use the -native Debian or RPM package path even when `snap` is available. The `pre` alias +installer selects the OpenShell Snap only with `OPENSHELL_INSTALL_METHOD=snap` +or when the Snap is already installed; otherwise it uses the native Debian or +RPM package. The Snap requires a compatible, preinstalled non-Snap Docker +daemon. Its positive canary uses system Docker; negative preflight coverage +verifies that the installer rejects both missing Docker and the Docker Snap +before installing OpenShell. +Explicit release tags and the `pre` alias always use the native Debian or RPM +package path. The `pre` alias checks matching Git tags in version order, then looks up the exact platform artifact and verifies the release run instead of listing every repository artifact. Snapd runs the gateway as a root-owned system service. Its generated client -certificates reside in root-owned snap state and are unavailable to ordinary CLI -users, so the Snap uses plaintext loopback transport and enables unauthenticated -local users by default. Debian and RPM packages instead run systemd user services -and use user-owned mTLS material. Bootstrap creates the default configuration -only when it is missing. Sandbox-to-gateway sessions remain authenticated with -gateway-minted JWTs. +certificates reside in root-owned snap state. The installer copies the client +bundle into the target user's private Snap state and registers the TLS endpoint; +direct Snap installs require the same enrollment. The install and post-refresh +hooks replace configs that explicitly enable plaintext or unauthenticated access +with the secure default. Snap refreshes +restart the gateway so the migrated config takes effect immediately. + +Debian and RPM packages instead run systemd user services with user-owned mTLS +material. Sandbox-to-gateway sessions remain authenticated with gateway-minted +JWTs. + The Debian qualification profile keeps candidate-image overrides outside the operator-owned gateway configuration: it writes a harness-owned file under `/var/lib/openshell-qualification` and selects it through the packaged systemd diff --git a/docs/about/installation.mdx b/docs/about/installation.mdx index 350164c2a7..d41639cddc 100644 --- a/docs/about/installation.mdx +++ b/docs/about/installation.mdx @@ -38,7 +38,7 @@ curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | OPENSHELL_VERSION=pre sh ``` -The installer checks prerelease tags from newest to oldest, selects an unexpired artifact from a successful release run for the current platform, and downloads only that artifact. Installed packages keep the candidate's exact version, such as `0.1.0-pre.3`. Prerelease tags do not create entries on the GitHub Releases page. On Linux, prereleases and explicit release tags use Debian or RPM packages even if `snap` is installed. +The installer checks prerelease tags from newest to oldest, selects an unexpired artifact from a successful release run for the current platform, and downloads only that artifact. Installed packages keep the candidate's exact version, such as `0.1.0-pre.3`. Prerelease tags do not create entries on the GitHub Releases page. On Linux, prereleases and explicit release tags use Debian or RPM packages. The rolling [`dev` release](https://github.com/NVIDIA/OpenShell/releases/tag/dev) does not require GitHub authentication: @@ -86,7 +86,7 @@ The gateway reads `~/.config/openshell/gateway.toml` if it exists, otherwise the ## Linux -The script uses the [Snap](#snap) package when `snap` is available. Otherwise, or when you set `OPENSHELL_VERSION` to a release tag, it installs a Debian package on Debian and Ubuntu or an RPM package on Fedora and RHEL. Linux packages require glibc 2.28 or newer. +The script installs a Debian package on Debian and Ubuntu or an RPM package on Fedora and RHEL. Set `OPENSHELL_INSTALL_METHOD=snap` to install the [Snap](#snap) package instead; hosts that already have the OpenShell snap keep refreshing it. Linux packages require glibc 2.28 or newer. The gateway runs as a systemd user service at `https://127.0.0.1:17670` and reads `~/.config/openshell/gateway.toml`. @@ -110,20 +110,22 @@ The snap requires Docker Engine installed from your distribution or Docker's pac sudo snap install openshell ``` -The snap does not migrate existing Debian, RPM, or Homebrew installs. Remove any existing installation first, then rerun the script with `OPENSHELL_ACK_BREAKING_UPGRADE=1`. +The snap does not migrate existing Debian, RPM, or Homebrew installs. Remove any existing installation first, then rerun the script with `OPENSHELL_INSTALL_METHOD=snap OPENSHELL_ACK_BREAKING_UPGRADE=1`. -The gateway runs as a system service at `http://127.0.0.1:17670` and reads `/var/snap/openshell/common/gateway.toml`. - - -The snap gateway allows unauthenticated access from the local host. Any local user or process can operate it. Do not expose it beyond the local host. - - -Snap refreshes do not restart the gateway, so active sandboxes keep running. Restart it to pick up a new version: +The gateway runs as a system service at `https://127.0.0.1:17670` and reads `/var/snap/openshell/common/gateway.toml`. It requires a client certificate. The install script copies that certificate to the installing user's Snap state and registers the gateway automatically. If you installed with `sudo snap install openshell`, give each trusted user the certificate and register the gateway from that user's account: ```shell -sudo systemctl restart snap.openshell.gateway +d=~/snap/openshell/common/.local/state/openshell/tls +mkdir -p -m 700 "$d" "$d/client" +sudo install -o "$USER" -m 600 /var/snap/openshell/common/tls/ca.crt "$d/" +sudo install -o "$USER" -m 600 -t "$d/client" \ + /var/snap/openshell/common/tls/client/tls.crt /var/snap/openshell/common/tls/client/tls.key +openshell gateway add https://127.0.0.1:17670 --local --name openshell +openshell status ``` +Keep the client key private. + To install a locally built snap, connect its interfaces manually: ```shell diff --git a/install.sh b/install.sh index 60bfd6f64c..83b7c479be 100755 --- a/install.sh +++ b/install.sh @@ -61,21 +61,22 @@ ENVIRONMENT VARIABLES: OPENSHELL_ACK_BREAKING_UPGRADE Set to 1 only after backing up and cleaning up a pre-v0.0.37 or non-snap installation. + OPENSHELL_INSTALL_METHOD + Linux package to install: snap, deb, or rpm. Unset + selects deb or rpm from the host package manager. NOTES: When OPENSHELL_VERSION is unset, this resolves the latest tagged release from ${GITHUB_URL}/releases/latest. - On Linux, the installer uses the OpenShell snap when the snap command is - available and OPENSHELL_VERSION is unset or dev. Snap installs use - latest/stable by default and latest/edge for dev. Explicit release tags - and prereleases use Debian or RPM packages. The OpenShell snap requires a - running Docker Engine installed from a system package or Docker's package + Linux installs the Debian package on amd64/arm64 or the RPM packages on + x86_64/aarch64, depending on the host package manager. Set + OPENSHELL_INSTALL_METHOD=snap to install the OpenShell snap instead; hosts + that already have the OpenShell snap keep refreshing it. Snap installs use + latest/stable by default and latest/edge for dev, and do not support + explicit release tags or prereleases. The OpenShell snap requires a running + Docker Engine installed from a system package or Docker's package repository. The Docker snap is not currently compatible with OpenShell. - - For explicit versions or without snap, Linux installs the Debian package - on amd64/arm64 or the RPM packages on x86_64/aarch64, depending on the - host package manager. macOS installs the release Homebrew formula on Apple Silicon and starts a brew services-backed local gateway. EOF @@ -658,9 +659,20 @@ local_gateway_endpoint() { } linux_package_method() { + case "${OPENSHELL_INSTALL_METHOD:-}" in + snap | deb | rpm) + echo "$OPENSHELL_INSTALL_METHOD" + return 0 + ;; + '') ;; + *) error "unsupported OPENSHELL_INSTALL_METHOD=${OPENSHELL_INSTALL_METHOD}; use snap, deb, or rpm" ;; + esac + + # Keep refreshing an existing snap install instead of adding a second + # gateway on the same port. case "${OPENSHELL_VERSION:-}" in '' | dev) - if has_cmd snap; then + if has_cmd snap && snap list openshell >/dev/null 2>&1; then echo "snap" return 0 fi @@ -1220,43 +1232,6 @@ openshell_snap_channel() { esac } -ensure_snap_gateway_config() { - _config_file="${1:-/var/snap/openshell/common/gateway.toml}" - - as_root sh -c ' - set -eu - config_file=$1 - if [ -e "$config_file" ] || [ -L "$config_file" ]; then - exit 0 - fi - - config_dir=${config_file%/*} - mkdir -p "$config_dir" - umask 077 - temporary_file=$(mktemp "${config_file}.tmp.XXXXXX") - trap '\''rm -f "$temporary_file"'\'' 0 HUP INT TERM - - cat >"$temporary_file" <<'\''EOF'\'' -[openshell] -version = 2 - -[openshell.gateway] - -[openshell.gateway.auth] -allow_unauthenticated_users = true -EOF - - if ! ln "$temporary_file" "$config_file"; then - if [ -e "$config_file" ] || [ -L "$config_file" ]; then - exit 0 - fi - exit 1 - fi - rm -f "$temporary_file" - trap - 0 HUP INT TERM - ' sh "$_config_file" -} - wait_for_docker_daemon() { _timeout="${OPENSHELL_INSTALL_DOCKER_TIMEOUT:-30}" _elapsed=0 @@ -1280,9 +1255,39 @@ wait_for_docker_daemon() { error "Docker daemon did not become reachable within ${_timeout}s" } +# Copy the snap gateway's client bundle into the target user's snap state +# directory, where `openshell gateway add --local` imports it. Root only reads +# the source files; the target user writes the copies into their own home. +copy_snap_client_bundle() { + _src="${OPENSHELL_SNAP_TLS_DIR:-/var/snap/openshell/common/tls}" + _dst="${TARGET_HOME}/snap/openshell/common/.local/state/openshell/tls" + + as_target_user mkdir -p "${_dst}/client" + as_target_user chmod 700 "$_dst" "${_dst}/client" + for _file in ca.crt client/tls.crt client/tls.key; do + as_root cat "${_src}/${_file}" | + as_target_user sh -c 'umask 077; cat >"$1"' sh "${_dst}/${_file}" + as_target_user chmod 600 "${_dst}/${_file}" + done +} + +# Snap revisions that require mTLS ship the post-refresh hook that migrates +# older plaintext configs. +snap_gateway_uses_mtls() { + [ -e "${OPENSHELL_SNAP_DIR:-/snap/openshell/current}/meta/hooks/post-refresh" ] +} + register_snap_gateway() { _register_bin="${OPENSHELL_REGISTER_BIN:-/snap/bin/openshell}" - _endpoint="http://127.0.0.1:${LOCAL_GATEWAY_PORT}" + + if snap_gateway_uses_mtls; then + _endpoint="https://127.0.0.1:${LOCAL_GATEWAY_PORT}" + info "copying the gateway client certificate for ${TARGET_USER}..." + copy_snap_client_bundle + else + _endpoint="http://127.0.0.1:${LOCAL_GATEWAY_PORT}" + warn "this OpenShell snap revision serves plaintext HTTP without client authentication; any local user can operate the gateway" + fi if _add_output="$(as_target_user "$_register_bin" gateway add "$_endpoint" --local --name openshell 2>&1)"; then [ -z "$_add_output" ] || print_gateway_add_output "$_add_output" @@ -1304,15 +1309,28 @@ register_snap_gateway() { esac } +# The mTLS gateway rejects TLS handshakes without a client certificate, so +# probe it with the root-owned client bundle. wait_for_snap_gateway_listener() { _timeout="${OPENSHELL_INSTALL_GATEWAY_TIMEOUT:-30}" _elapsed=0 _last_output="" - _probe_url="http://127.0.0.1:${LOCAL_GATEWAY_PORT}/" + _tls_dir="${OPENSHELL_SNAP_TLS_DIR:-/var/snap/openshell/common/tls}" + + if snap_gateway_uses_mtls; then + _probe_url="https://127.0.0.1:${LOCAL_GATEWAY_PORT}/" + _probe_as=as_root + set -- --cacert "${_tls_dir}/ca.crt" \ + --cert "${_tls_dir}/client/tls.crt" --key "${_tls_dir}/client/tls.key" + else + _probe_url="http://127.0.0.1:${LOCAL_GATEWAY_PORT}/" + _probe_as="" + set -- + fi info "waiting for local gateway listener to become reachable..." while [ "$_elapsed" -lt "$_timeout" ]; do - if _last_output="$(curl -sS --max-time 2 -o /dev/null "$_probe_url" 2>&1)"; then + if _last_output="$($_probe_as curl -sS --max-time 2 "$@" -o /dev/null "$_probe_url" 2>&1)"; then info "local gateway listener is reachable" return 0 fi @@ -1350,13 +1368,12 @@ Install Docker Engine from a system package or Docker's package repository, then as_root snap install openshell --channel="$_channel" fi - ensure_snap_gateway_config as_root snap restart openshell.gateway info "installed OpenShell snap from ${_channel}" + wait_for_snap_gateway_listener info "registering local gateway as ${TARGET_USER}..." register_snap_gateway - wait_for_snap_gateway_listener OPENSHELL_REGISTER_BIN="/snap/bin/openshell" wait_for_local_gateway_status } diff --git a/nix/test-guest/scripts/snap-gateway-repro.sh b/nix/test-guest/scripts/snap-gateway-repro.sh index d92800b523..43b7f3949c 100755 --- a/nix/test-guest/scripts/snap-gateway-repro.sh +++ b/nix/test-guest/scripts/snap-gateway-repro.sh @@ -99,7 +99,7 @@ for attempt in $(seq 1 "${attempts}"); do echo "==> install.sh Snap ${mode} reproduction attempt ${attempt}/${attempts}" if [ "${mode}" != system-docker ]; then output=$(mktemp) - if OPENSHELL_VERSION=dev sh "${install_script}" >"${output}" 2>&1; then + if OPENSHELL_INSTALL_METHOD=snap OPENSHELL_VERSION=dev sh "${install_script}" >"${output}" 2>&1; then echo "install.sh unexpectedly succeeded in ${mode} mode" >&2 cat "${output}" >&2 rm -f "${output}" @@ -127,7 +127,7 @@ for attempt in $(seq 1 "${attempts}"); do fi sandbox="snap-${attempt}-$$" - if ! OPENSHELL_VERSION=dev sh "${install_script}" || + if ! OPENSHELL_INSTALL_METHOD=snap OPENSHELL_VERSION=dev sh "${install_script}" || ! sudo snap list openshell >/dev/null || ! snap info openshell | grep -Eq '^tracking: +latest/edge$' || ! docker_is_ready || diff --git a/python/openshell/release_formula_test.py b/python/openshell/release_formula_test.py index 61fff150ac..4db21b0951 100644 --- a/python/openshell/release_formula_test.py +++ b/python/openshell/release_formula_test.py @@ -155,7 +155,11 @@ def test_snap_wrapper_uses_optional_gateway_config_without_generating_toml() -> 'export OPENSHELL_DB_URL="${OPENSHELL_DB_URL:-sqlite:${SNAP_COMMON}/gateway.db?mode=rwc}"' in wrapper ) - assert 'export OPENSHELL_DISABLE_TLS="${OPENSHELL_DISABLE_TLS:-true}"' in wrapper + assert "OPENSHELL_DISABLE_TLS" not in wrapper + assert ( + 'export OPENSHELL_LOCAL_TLS_DIR="${OPENSHELL_LOCAL_TLS_DIR:-${SNAP_COMMON}/tls}"' + in wrapper + ) assert ( 'exec "${SNAP}/bin/openshell-gateway" --config "$CANONICAL_CONFIG_FILE" "$@"' in wrapper diff --git a/skills/debug-openshell-cluster/SKILL.md b/skills/debug-openshell-cluster/SKILL.md index d654d38094..85360a77ad 100644 --- a/skills/debug-openshell-cluster/SKILL.md +++ b/skills/debug-openshell-cluster/SKILL.md @@ -72,6 +72,7 @@ Common findings: - `No active gateway`: register one with `openshell gateway add `. - Connection refused: gateway process is not running, service exposure is wrong, or a port-forward/proxy is not active. - TLS/certificate errors: the endpoint scheme or trust chain is wrong, a local mTLS bundle does not match the gateway CA, or TLS termination does not match the gateway listener. +- A Snap refresh restarts the gateway with its migrated mTLS config. The secure Snap gateway uses `https://127.0.0.1:17670` and requires a client bundle in the user's Snap state. Refresh replaces insecure configs without keeping a copy; follow the published Snap installation steps to re-register an old HTTP client. - `Unauthenticated` from an edge or OIDC gateway: refresh stored credentials with `openshell gateway login [name]`, then retry. Use `gateway logout` only when intentionally clearing local credentials. - A direct development endpoint with a private or self-signed certificate can be isolated with `--gateway-endpoint --gateway-insecure`; do not persist or recommend insecure verification for shared gateways. diff --git a/snap/hooks/install b/snap/hooks/install index c5fb3baf87..78efeefdf8 100755 --- a/snap/hooks/install +++ b/snap/hooks/install @@ -2,10 +2,22 @@ # SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 +# Create the mTLS default and replace insecure configs on refresh. + set -eu config_file="${SNAP_COMMON}/gateway.toml" -if [ -e "$config_file" ] || [ -L "$config_file" ]; then +insecure='^[[:space:]]*(allow_unauthenticated_users|disable_tls)[[:space:]]*=[[:space:]]*true([[:space:]#]|$)' + +# Keep secure operator configs, symlinks, and directories. Replace a config +# that explicitly allows plaintext or anonymous access, even if it has other +# edits. +if [ -L "$config_file" ]; then + exit 0 +elif [ -f "$config_file" ]; then + grep -Eq "$insecure" "$config_file" || exit 0 + echo "openshell: replacing insecure gateway config with the mTLS default" >&2 +elif [ -e "$config_file" ]; then exit 0 fi @@ -13,24 +25,11 @@ mkdir -p "$SNAP_COMMON" umask 077 temporary_file=$(mktemp "${config_file}.tmp.XXXXXX") trap 'rm -f "$temporary_file"' 0 HUP INT TERM - -cat >"$temporary_file" <<'EOF' +cat >"$temporary_file" <<'CONFIG' [openshell] version = 2 [openshell.gateway] - -[openshell.gateway.auth] -allow_unauthenticated_users = true -EOF - -# A hard link publishes the config atomically without replacing a path created -# concurrently. SNAP_COMMON and the temporary file are on the same filesystem. -if ! ln "$temporary_file" "$config_file"; then - if [ -e "$config_file" ] || [ -L "$config_file" ]; then - exit 0 - fi - exit 1 -fi -rm -f "$temporary_file" +CONFIG +mv -f "$temporary_file" "$config_file" trap - 0 HUP INT TERM diff --git a/snap/hooks/post-refresh b/snap/hooks/post-refresh new file mode 100755 index 0000000000..28eb20373d --- /dev/null +++ b/snap/hooks/post-refresh @@ -0,0 +1,12 @@ +#!/bin/sh +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# Replace insecure gateway configs on refresh, then restart the gateway so the +# new config takes effect even when the previous revision used +# refresh-mode: endure and kept its plaintext gateway running. + +set -eu + +"${SNAP}/meta/hooks/install" +snapctl restart "${SNAP_INSTANCE_NAME}.gateway" diff --git a/snapcraft.yaml b/snapcraft.yaml index 4c602620cf..6c943c1387 100644 --- a/snapcraft.yaml +++ b/snapcraft.yaml @@ -29,18 +29,18 @@ description: | sudo systemctl reset-failed snap.openshell.gateway.service sudo snap restart openshell.gateway - 2. Verify the gateway and register it locally: + 2. Give your user the gateway client certificate and register the gateway. + The gateway requires mTLS; only users holding this certificate can use + it, so copy it only for trusted users: snap services openshell.gateway + d=~/snap/openshell/common/.local/state/openshell/tls + mkdir -p -m 700 "$d" "$d/client" + sudo install -o "$USER" -m 600 /var/snap/openshell/common/tls/ca.crt "$d/" + sudo install -o "$USER" -m 600 -t "$d/client" \ + /var/snap/openshell/common/tls/client/tls.crt /var/snap/openshell/common/tls/client/tls.key + openshell gateway add https://127.0.0.1:17670 --local --name openshell openshell status - openshell gateway add http://127.0.0.1:17670 --local --name openshell-gateway - - After a snap refresh, restart the gateway to pick up the new revision: - - sudo snap restart openshell.gateway - - Restarting the gateway will interrupt active sandbox sessions. The gateway - is not restarted automatically to avoid disconnecting running sandboxes. base: core24 grade: stable @@ -88,17 +88,16 @@ apps: gateway: command: bin/openshell-gateway-wrapper daemon: simple - # refresh-mode: endure prevents snapd from restarting the gateway daemon - # during snap refreshes, which would kill active sandbox sessions. - # Operators must manually restart the service after a refresh if needed. - refresh-mode: endure - # Snapd runs this daemon as root, so ordinary CLI users cannot read the - # client certificate generated in root-owned snap state. The wrapper sets - # OPENSHELL_DISABLE_TLS=true for the loopback-only listener and uses - # $SNAP_COMMON/gateway.db. Before startup it bootstraps package-managed - # credentials and validates the selected operator-provided config without - # creating or rewriting it. A nonempty OPENSHELL_GATEWAY_CONFIG takes - # precedence over gateway.toml. + # Refresh must activate the migrated mTLS config immediately. This + # interrupts active sandbox sessions. + refresh-mode: restart + # Snapd runs this daemon as root. The wrapper serves TLS from the bundle + # generated in $SNAP_COMMON/tls, and the default config requires client + # certificates; the installer copies the client bundle to the target + # user. The wrapper uses $SNAP_COMMON/gateway.db. Before startup it + # bootstraps package-managed credentials and validates the selected + # operator-provided config without creating or rewriting it. A nonempty + # OPENSHELL_GATEWAY_CONFIG takes precedence over gateway.toml. environment: XDG_DATA_HOME: "$SNAP_COMMON" XDG_RUNTIME_DIR: "$SNAP_COMMON" diff --git a/tasks/scripts/snap-gateway-wrapper.sh b/tasks/scripts/snap-gateway-wrapper.sh index 4a98962559..a047cf72e3 100755 --- a/tasks/scripts/snap-gateway-wrapper.sh +++ b/tasks/scripts/snap-gateway-wrapper.sh @@ -3,16 +3,16 @@ # SPDX-License-Identifier: Apache-2.0 # Snap wrapper for openshell-gateway. Sets snap-specific defaults: -# - OPENSHELL_DB_URL -> sqlite:$SNAP_COMMON/gateway.db (overridable) -# - OPENSHELL_DISABLE_TLS -> true -# It bootstraps package-managed credentials and validates, but never creates or -# rewrites, an operator-provided config before starting the gateway. +# - OPENSHELL_DB_URL -> sqlite:$SNAP_COMMON/gateway.db (overridable) +# - OPENSHELL_LOCAL_TLS_DIR -> $SNAP_COMMON/tls (overridable) +# The gateway serves TLS from the generated bundle and requires client +# certificates. It bootstraps package-managed credentials and validates, but +# never creates or rewrites, an operator-provided config before starting. set -eu CANONICAL_CONFIG_FILE="${SNAP_COMMON}/gateway.toml" export OPENSHELL_DB_URL="${OPENSHELL_DB_URL:-sqlite:${SNAP_COMMON}/gateway.db?mode=rwc}" -export OPENSHELL_DISABLE_TLS="${OPENSHELL_DISABLE_TLS:-true}" export OPENSHELL_LOCAL_TLS_DIR="${OPENSHELL_LOCAL_TLS_DIR:-${SNAP_COMMON}/tls}" # Mirror clap's CLI-over-environment precedence so preflight always inspects @@ -64,9 +64,9 @@ if [ "$expect_config_path" = true ] || { [ "$config_seen" = true ] && [ -z "$cli exit 2 fi -# Docker sandboxes require gateway-minted, launch-scoped credentials for the -# supervisor. Generate the local JWT bundle alongside the otherwise-unused TLS -# material; generate-certs is idempotent and preserves an existing bundle. +# Generate the local TLS bundle and the JWT bundle used for launch-scoped +# supervisor credentials; generate-certs is idempotent and preserves an +# existing bundle. "${SNAP}/bin/openshell-gateway" generate-certs \ --output-dir "$OPENSHELL_LOCAL_TLS_DIR" \ --server-san host.openshell.internal diff --git a/tasks/scripts/test-install-sh.sh b/tasks/scripts/test-install-sh.sh index 8cc6102148..286fa3f1ff 100755 --- a/tasks/scripts/test-install-sh.sh +++ b/tasks/scripts/test-install-sh.sh @@ -100,25 +100,30 @@ assert_glibc_preflight_fails \ "OpenShell Linux packages require glibc >= 2.28; detected musl or unsupported libc." \ setup_ldd_musl +# snap_state: 0 = no snap command, 1 = snap command only, +# installed = the OpenShell snap is already installed. assert_linux_package_method() { local name=$1 - local requested_version=$2 - local snap_present=$3 - local dpkg_present=$4 - local rpm_present=$5 - local expected=$6 + local install_method=$2 + local requested_version=$3 + local snap_state=$4 + local dpkg_present=$5 + local rpm_present=$6 + local expected=$7 local actual actual="$( + export OPENSHELL_INSTALL_METHOD="$install_method" export OPENSHELL_VERSION="$requested_version" has_cmd() { case "$1" in - snap) [ "$snap_present" = "1" ] ;; + snap) [ "$snap_state" != "0" ] ;; dpkg) [ "$dpkg_present" = "1" ] ;; rpm) [ "$rpm_present" = "1" ] ;; *) return 1 ;; esac } + snap() { [ "$*" = "list openshell" ] && [ "$snap_state" = "installed" ]; } linux_package_method )" if [ "$actual" != "$expected" ]; then @@ -127,16 +132,30 @@ assert_linux_package_method() { fi } -assert_linux_package_method "snap takes precedence over deb and rpm" "" 1 1 1 snap -assert_linux_package_method "dev uses snap" dev 1 1 1 snap -assert_linux_package_method "pre uses deb despite snap" pre 1 1 1 deb -assert_linux_package_method "numbered prerelease uses deb despite snap" v0.1.0-pre.3 1 1 1 deb -assert_linux_package_method "pre uses rpm despite snap" pre 1 0 1 rpm -assert_linux_package_method "pinned stable uses deb despite snap" v1.2.3 1 1 1 deb -assert_linux_package_method "pinned stable uses rpm despite snap" v1.2.3 1 0 1 rpm -assert_linux_package_method "deb is selected without snap" "" 0 1 1 deb -assert_linux_package_method "dev uses deb without snap" dev 0 1 1 deb -assert_linux_package_method "rpm is selected without snap or deb" "" 0 0 1 rpm +assert_linux_package_method "deb is the default despite snap" "" "" 1 1 1 deb +assert_linux_package_method "rpm is the default despite snap" "" "" 1 0 1 rpm +assert_linux_package_method "dev uses deb despite snap" "" dev 1 1 1 deb +assert_linux_package_method "snap is opt-in" snap "" 1 1 1 snap +assert_linux_package_method "snap opt-in with dev" snap dev 1 1 1 snap +assert_linux_package_method "explicit deb" deb "" installed 0 1 deb +assert_linux_package_method "explicit rpm" rpm "" 1 1 1 rpm +assert_linux_package_method "existing snap install keeps refreshing" "" "" installed 1 1 snap +assert_linux_package_method "existing snap install keeps refreshing dev" "" dev installed 1 1 snap +assert_linux_package_method "pre uses deb despite existing snap" "" pre installed 1 1 deb +assert_linux_package_method "numbered prerelease uses deb despite existing snap" "" v0.1.0-pre.3 installed 1 1 deb +assert_linux_package_method "pinned stable uses rpm despite existing snap" "" v1.2.3 installed 0 1 rpm +assert_linux_package_method "deb is selected without snap" "" "" 0 1 1 deb +assert_linux_package_method "rpm is selected without snap or deb" "" "" 0 0 1 rpm + +if (OPENSHELL_INSTALL_METHOD=flatpak linux_package_method) >"$out" 2>"$err"; then + echo "FAIL: unsupported OPENSHELL_INSTALL_METHOD should be rejected" >&2 + exit 1 +fi +if ! grep -Fq "unsupported OPENSHELL_INSTALL_METHOD=flatpak" "$err"; then + echo "FAIL: unsupported OPENSHELL_INSTALL_METHOD was not explained" >&2 + cat "$err" >&2 + exit 1 +fi if ! ( find_existing_native_openshell_bin() { return 1; } @@ -269,7 +288,6 @@ assert_snap_install_flow() { as_root() { printf 'root:%s\n' "$*"; } set_linux_target_runtime_dir() { :; } wait_for_docker_daemon() { printf '%s\n' "wait:docker"; } - ensure_snap_gateway_config() { printf '%s\n' "ensure:gateway-config"; } register_snap_gateway() { printf '%s\n' "register:gateway"; } wait_for_snap_gateway_listener() { printf '%s\n' "wait:gateway-listener"; } wait_for_local_gateway_status() { printf '%s\n' "wait:gateway-status"; } @@ -293,10 +311,9 @@ assert_snap_install_flow \ 1 0 "" \ "wait:docker root:snap install openshell --channel=latest/stable -ensure:gateway-config root:snap restart openshell.gateway -register:gateway wait:gateway-listener +register:gateway wait:gateway-status" assert_snap_install_flow \ @@ -304,10 +321,9 @@ assert_snap_install_flow \ 1 1 "" \ "wait:docker root:snap refresh openshell --channel=latest/stable -ensure:gateway-config root:snap restart openshell.gateway -register:gateway wait:gateway-listener +register:gateway wait:gateway-status" assert_snap_install_rejected() { @@ -363,34 +379,6 @@ assert_snap_install_rejected \ 1 1 \ "the Docker snap is not currently compatible with OpenShell" -snap_config_dir="${tmpdir}/snap-config" -snap_config="${snap_config_dir}/gateway.toml" -if ! (as_root() { "$@"; }; ensure_snap_gateway_config "$snap_config"); then - echo "FAIL: Snap gateway config bootstrap should create a missing config" >&2 - exit 1 -fi -if ! grep -Fq 'allow_unauthenticated_users = true' "$snap_config"; then - echo "FAIL: Snap gateway config must permit the plaintext local CLI" >&2 - exit 1 -fi -if [[ -z $(find "$snap_config" -perm 600) ]]; then - echo "FAIL: Snap gateway config must be mode 0600" >&2 - exit 1 -fi - -printf '\noperator setting = true\n' >>"$snap_config" -cp "$snap_config" "${tmpdir}/snap-config-before" -(as_root() { "$@"; }; ensure_snap_gateway_config "$snap_config") -cmp -s "${tmpdir}/snap-config-before" "$snap_config" - -broken_config="${tmpdir}/broken-gateway.toml" -ln -s "${tmpdir}/missing-gateway.toml" "$broken_config" -(as_root() { "$@"; }; ensure_snap_gateway_config "$broken_config") -if [[ $(readlink "$broken_config") != "${tmpdir}/missing-gateway.toml" ]]; then - echo "FAIL: Snap gateway config bootstrap replaced a broken operator symlink" >&2 - exit 1 -fi - attempts_file="${tmpdir}/docker-attempts" root_probes_file="${tmpdir}/docker-root-probes" printf '0\n' >"$attempts_file" @@ -450,7 +438,11 @@ registration_calls_file="${tmpdir}/registration-calls" : >"$registration_calls_file" if ! ( as_target_user() { printf 'target:%s\n' "$*" >>"$registration_calls_file"; } + copy_snap_client_bundle() { printf 'copy:client-bundle\n' >>"$registration_calls_file"; } print_gateway_add_output() { :; } + info() { :; } + TARGET_USER=test-user + snap_gateway_uses_mtls() { return 0; } register_snap_gateway ) >"$out" 2>"$err"; then echo "FAIL: Snap gateway registration should succeed" >&2 @@ -458,11 +450,92 @@ if ! ( exit 1 fi registration_calls="$(cat "$registration_calls_file")" +if [ "$registration_calls" != "copy:client-bundle +target:/snap/bin/openshell gateway add https://127.0.0.1:17670 --local --name openshell" ]; then + echo "FAIL: mTLS Snap gateway registration must copy the client bundle and use HTTPS" >&2 + printf '%s\n' "$registration_calls" >&2 + exit 1 +fi + +: >"$registration_calls_file" +if ! ( + as_target_user() { printf 'target:%s\n' "$*" >>"$registration_calls_file"; } + copy_snap_client_bundle() { printf 'copy:client-bundle\n' >>"$registration_calls_file"; } + print_gateway_add_output() { :; } + snap_gateway_uses_mtls() { return 1; } + register_snap_gateway +) >"$out" 2>"$err"; then + echo "FAIL: legacy plaintext Snap gateway registration should succeed" >&2 + cat "$err" >&2 || true + exit 1 +fi +registration_calls="$(cat "$registration_calls_file")" if [ "$registration_calls" != "target:/snap/bin/openshell gateway add http://127.0.0.1:17670 --local --name openshell" ]; then - echo "FAIL: Snap gateway registration must use the Snap CLI as the target user" >&2 + echo "FAIL: legacy Snap gateway registration must use HTTP without copying certificates" >&2 printf '%s\n' "$registration_calls" >&2 exit 1 fi +if ! grep -Fq "without client authentication" "$err"; then + echo "FAIL: legacy Snap gateway registration must warn about unauthenticated access" >&2 + exit 1 +fi + +assert_snap_listener_probe() { + local name=$1 + local uses_mtls=$2 + local expected=$3 + local actual + + actual="$( + as_root() { printf 'root:'; "$@"; } + curl() { printf '%s\n' "$*"; } + snap_gateway_uses_mtls() { [ "$uses_mtls" = "1" ]; } + info() { :; } + OPENSHELL_SNAP_TLS_DIR=/tls + wait_for_snap_gateway_listener >/dev/null + printf '%s\n' "$_last_output" + )" + if [ "$actual" != "$expected" ]; then + echo "FAIL: ${name}: expected ${expected}, got ${actual}" >&2 + exit 1 + fi +} + +assert_snap_listener_probe "mTLS snap probes HTTPS with the client bundle as root" 1 \ + "root:-sS --max-time 2 --cacert /tls/ca.crt --cert /tls/client/tls.crt --key /tls/client/tls.key -o /dev/null https://127.0.0.1:17670/" +assert_snap_listener_probe "legacy snap probes plaintext HTTP" 0 \ + "-sS --max-time 2 -o /dev/null http://127.0.0.1:17670/" + +snap_tls_src="${tmpdir}/snap-tls" +mkdir -p "${snap_tls_src}/client" +printf 'ca\n' >"${snap_tls_src}/ca.crt" +printf 'cert\n' >"${snap_tls_src}/client/tls.crt" +printf 'key\n' >"${snap_tls_src}/client/tls.key" +snap_user_home="${tmpdir}/snap-user-home" +mkdir -p "${snap_user_home}/snap/openshell/common/.local/state/openshell/tls/client" +printf 'old key\n' >"${snap_user_home}/snap/openshell/common/.local/state/openshell/tls/client/tls.key" +chmod 644 "${snap_user_home}/snap/openshell/common/.local/state/openshell/tls/client/tls.key" +( + as_root() { "$@"; } + as_target_user() { "$@"; } + TARGET_HOME="$snap_user_home" + OPENSHELL_SNAP_TLS_DIR="$snap_tls_src" copy_snap_client_bundle +) +snap_user_tls="${snap_user_home}/snap/openshell/common/.local/state/openshell/tls" +for file in ca.crt client/tls.crt client/tls.key; do + if ! cmp -s "${snap_tls_src}/${file}" "${snap_user_tls}/${file}"; then + echo "FAIL: Snap client bundle copy missing ${file}" >&2 + exit 1 + fi + if [[ -z $(find "${snap_user_tls}/${file}" -perm 600) ]]; then + echo "FAIL: Snap client bundle ${file} must be mode 0600" >&2 + exit 1 + fi +done +if [[ -z $(find "$snap_user_tls" -maxdepth 0 -perm 700) ]]; then + echo "FAIL: Snap client bundle directory must be mode 0700" >&2 + exit 1 +fi if [ "$(PLATFORM=darwin local_gateway_endpoint)" != "https://localhost:17670" ]; then echo "FAIL: macOS local gateway endpoint must use a TLS-compatible loopback hostname" >&2 diff --git a/tasks/scripts/test-packaging-assets.sh b/tasks/scripts/test-packaging-assets.sh index 345a0e3964..04f69b08be 100755 --- a/tasks/scripts/test-packaging-assets.sh +++ b/tasks/scripts/test-packaging-assets.sh @@ -123,7 +123,14 @@ if [[ ! -x "$snap_install_hook" ]]; then echo "FAIL: Snap install hook must be executable" >&2 exit 1 fi -assert_contains "$snap_install_hook" 'allow_unauthenticated_users = true' +assert_not_contains "$snap_install_hook" 'compute_driver' +assert_not_contains "$snap_install_hook" 'allow_unauthenticated_users = true' +assert_contains "$snapcraft" 'refresh-mode: restart' +if [[ ! -x "$(dirname "$snap_install_hook")/post-refresh" ]]; then + echo "FAIL: Snap post-refresh hook must be executable" >&2 + exit 1 +fi +assert_not_contains "$ROOT/tasks/scripts/snap-gateway-wrapper.sh" 'OPENSHELL_DISABLE_TLS' bash "$ROOT/tasks/scripts/test-snap-install-hook.sh" "$snap_install_hook" assert_not_contains "$snap_install_docs" "snap connect openshell:home" assert_not_contains "$snap_install_docs" "snap connect openshell:network" @@ -132,7 +139,7 @@ assert_contains "$snap_install_docs" "snap connect openshell:docker :docker" assert_contains "$snap_canary" "install.sh | sh" assert_contains "$snap_canary" "ubuntu-snap-system-docker:" assert_contains "$snap_canary" "ubuntu-snap-docker-preflight:" -assert_contains "$snap_repro" 'OPENSHELL_VERSION=dev sh "${install_script}"' +assert_contains "$snap_repro" 'OPENSHELL_INSTALL_METHOD=snap OPENSHELL_VERSION=dev sh "${install_script}"' assert_contains "$snap_repro" "system-docker" assert_contains "$snap_repro" "missing-docker" assert_contains "$snap_repro" "docker-snap" diff --git a/tasks/scripts/test-snap-gateway-wrapper.sh b/tasks/scripts/test-snap-gateway-wrapper.sh index a14076b121..691a1e7628 100755 --- a/tasks/scripts/test-snap-gateway-wrapper.sh +++ b/tasks/scripts/test-snap-gateway-wrapper.sh @@ -82,9 +82,9 @@ cp "$override" "$work/override-before" : >"$log" run_wrapper "$override" assert_log "config preflight -- --trace -env:$override|sqlite:$common/gateway.db?mode=rwc|true +env:$override|sqlite:$common/gateway.db?mode=rwc| --trace -env:$override|sqlite:$common/gateway.db?mode=rwc|true" +env:$override|sqlite:$common/gateway.db?mode=rwc|" cmp -s "$work/override-before" "$override" cli_config="$work/cli.toml" @@ -98,9 +98,9 @@ env \ FAKE_GATEWAY_LOG="$log" \ "$wrapper" --trace --config "$cli_config" assert_log "config preflight -- --trace --config $cli_config -env:$override|sqlite:$common/gateway.db?mode=rwc|true +env:$override|sqlite:$common/gateway.db?mode=rwc| --trace --config $cli_config -env:$override|sqlite:$common/gateway.db?mode=rwc|true" +env:$override|sqlite:$common/gateway.db?mode=rwc|" cmp -s "$work/cli-before" "$cli_config" : >"$log" @@ -115,7 +115,7 @@ if env \ exit 1 fi assert_log "config preflight -- --config=$cli_config -env:$override|sqlite:$common/gateway.db?mode=rwc|true" +env:$override|sqlite:$common/gateway.db?mode=rwc|" cmp -s "$work/cli-before" "$cli_config" : >"$log" @@ -130,7 +130,7 @@ if env \ exit 1 fi assert_log "config preflight -- --grpc-rate-limit-requests 10 -env:$override|sqlite:$common/gateway.db?mode=rwc|true" +env:$override|sqlite:$common/gateway.db?mode=rwc|" for invalid_selector in terminator nested-config; do : >"$log" @@ -162,9 +162,9 @@ env \ FAKE_GATEWAY_LOG="$log" \ "$wrapper" --config=--dash-leading assert_log "config preflight -- --config=--dash-leading -env:$override|sqlite:$common/gateway.db?mode=rwc|true +env:$override|sqlite:$common/gateway.db?mode=rwc| --config=--dash-leading -env:$override|sqlite:$common/gateway.db?mode=rwc|true" +env:$override|sqlite:$common/gateway.db?mode=rwc|" canonical="$common/gateway.toml" printf 'valid schema-v2\n' >"$canonical" @@ -172,18 +172,18 @@ cp "$canonical" "$work/canonical-before" : >"$log" run_wrapper unset assert_log "config preflight -- --config $canonical --trace -env:|sqlite:$common/gateway.db?mode=rwc|true +env:|sqlite:$common/gateway.db?mode=rwc| --config $canonical --trace -env:|sqlite:$common/gateway.db?mode=rwc|true" +env:|sqlite:$common/gateway.db?mode=rwc|" cmp -s "$work/canonical-before" "$canonical" rm "$canonical" : >"$log" run_wrapper unset assert_log "config preflight -- --trace -env:|sqlite:$common/gateway.db?mode=rwc|true +env:|sqlite:$common/gateway.db?mode=rwc| --trace -env:|sqlite:$common/gateway.db?mode=rwc|true" +env:|sqlite:$common/gateway.db?mode=rwc|" assert_preflight_failure() { local name=$1 @@ -193,7 +193,7 @@ assert_preflight_failure() { exit 1 fi assert_log "config preflight -- --config $canonical --trace -env:|sqlite:$common/gateway.db?mode=rwc|true" +env:|sqlite:$common/gateway.db?mode=rwc|" } printf 'legacy version = 1\n' >"$canonical" diff --git a/tasks/scripts/test-snap-install-hook.sh b/tasks/scripts/test-snap-install-hook.sh index aacea97bbb..e82cb64b55 100755 --- a/tasks/scripts/test-snap-install-hook.sh +++ b/tasks/scripts/test-snap-install-hook.sh @@ -15,6 +15,14 @@ cat >"$expected" <<'EOF' [openshell] version = 2 +[openshell.gateway] +EOF + +legacy="${work}/legacy.toml" +cat >"$legacy" <<'EOF' +[openshell] +version = 2 + [openshell.gateway] [openshell.gateway.auth] @@ -34,6 +42,81 @@ cp "$common/gateway.toml" "${work}/operator-before" SNAP_COMMON="$common" "$hook" cmp -s "${work}/operator-before" "$common/gateway.toml" +common="${work}/legacy" +mkdir -p "$common" +cp "$legacy" "$common/gateway.toml" +chmod 644 "$common/gateway.toml" +SNAP_COMMON="$common" "$hook" +if ! cmp -s "$expected" "$common/gateway.toml"; then + echo "FAIL: install hook must migrate the legacy unauthenticated config" >&2 + exit 1 +fi +if [[ -z $(find "$common/gateway.toml" -perm 600) ]]; then + echo "FAIL: migrated config must be mode 0600" >&2 + exit 1 +fi + +common="${work}/legacy-edited" +mkdir -p "$common" +cp "$legacy" "$common/gateway.toml" +printf '\n# operator note\n' >>"$common/gateway.toml" +cp "$common/gateway.toml" "${work}/legacy-edited-before" +SNAP_COMMON="$common" "$hook" +cmp -s "$expected" "$common/gateway.toml" + +common="${work}/custom-insecure" +mkdir -p "$common" +cat >"$common/gateway.toml" <<'EOF' +[openshell] +version = 2 + +[openshell.gateway] +compute_driver = "docker" +disable_tls = true # old local override + +[openshell.gateway.auth] +allow_unauthenticated_users = true # old local override +EOF +cp "$common/gateway.toml" "${work}/custom-insecure-before" +SNAP_COMMON="$common" "$hook" +cmp -s "$expected" "$common/gateway.toml" + +common="${work}/custom-secure" +mkdir -p "$common" +cat >"$common/gateway.toml" <<'EOF' +[openshell] +version = 2 + +[openshell.gateway] +compute_driver = "docker" +# allow_unauthenticated_users = true +EOF +cp "$common/gateway.toml" "${work}/custom-secure-before" +SNAP_COMMON="$common" "$hook" +cmp -s "${work}/custom-secure-before" "$common/gateway.toml" + +common="${work}/post-refresh" +mkdir -p "$common" "${work}/snap/meta/hooks" +cp "$hook" "${work}/snap/meta/hooks/install" +cp "${work}/legacy-edited-before" "$common/gateway.toml" +mkdir -p "${work}/bin" +cat >"${work}/bin/snapctl" <>"${work}/snapctl.log" +EOF +chmod 755 "${work}/bin/snapctl" +PATH="${work}/bin:$PATH" SNAP="${work}/snap" SNAP_COMMON="$common" \ + SNAP_INSTANCE_NAME=openshell "${hook_dir}/post-refresh" +if ! cmp -s "$expected" "$common/gateway.toml"; then + echo "FAIL: post-refresh hook must migrate an edited insecure config" >&2 + exit 1 +fi +if [[ $(cat "${work}/snapctl.log") != "restart openshell.gateway" ]]; then + echo "FAIL: post-refresh hook must restart the gateway" >&2 + cat "${work}/snapctl.log" >&2 + exit 1 +fi + common="${work}/broken-link" mkdir -p "$common" ln -s "${work}/missing-target" "$common/gateway.toml" @@ -51,4 +134,9 @@ if [[ ! -d "$common/gateway.toml" ]]; then exit 1 fi +if [[ -n $(find "$work" -name 'gateway.toml.pre-mtls*') ]]; then + echo "FAIL: install hook must not keep copies of replaced configs" >&2 + exit 1 +fi + echo "Snap install hook tests passed"