refactor(compute): decouple gateway driver composition

[drew]

Summary

Move first-party compute-driver composition and VM process ownership into openshell-gateway, leaving openshell-server independent of compute backend implementations. The standard gateway installs drivers through a generic registry, can build without linked compute backends, and preserves the existing public compute-driver RPC path for external drivers.

Related Issue

Part of #2417.

Changes

• add openshell-gateway as the binary and first-party compute-driver composition crate
• gate linked first-party registrations behind the in-tree-compute-drivers feature
• replace backend-specific server construction with generic driver registrations, factories, metadata, and build context
• move VM subprocess ownership and first-party Docker, Podman, Kubernetes, VM, and Windows MXC composition to the gateway boundary
• preserve driver-free gateway builds and existing external-driver operation for Docker, Podman, Kubernetes, and VM
• move Docker and Podman socket discovery into their driver crates, including native and machine-backed Podman discovery
• keep compute-driver telemetry categories bounded and move driver-specific tracing setup to registrations
• carry user_namespaces as a typed compute-driver field, translate it to Kubernetes hostUsers, and accept the legacy platform_config.host_users encoding during rolling upgrades
• update architecture, public compute-driver documentation, packaging, deployment scripts, E2E wiring, Windows build guidance, and related agent skills
• remove stale backend-specific comments and obsolete openshell-server binary build references

Testing

• mise run pre-commit
• OPENSHELL_SYSTEM_GATEWAY_DIR=/tmp/openshell-empty-system-config-codex-pr2823 mise run test
• mise run helm:test (106 tests)
• cargo check -p openshell-gateway --no-default-features
• cargo check -p openshell-gateway --features in-tree-compute-drivers
• focused server, gateway registration, and Kubernetes user-namespace tests
• mise run e2e:gateway:no-compute-drivers
• external-driver E2E matrix (requested through test:e2e)
• Windows MSVC build-only lane (not run locally)

Checklist

• Changes are scoped to accepted issue refactor(compute): define driver shutdown and restart lifecycle contract #2417
• Follows Conventional Commits
• Commits are signed off (DCO)
• Architecture documentation is updated
• Public compute-driver documentation is updated
• Public compute-driver changes are additive and retain legacy Kubernetes input compatibility

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-2823.docs.buildwithfern.com/openshell

[drew]

/ok to test

[drew]

/ok to test

[drew]

/ok to test

[drew]

/ok to test

[elezar]

@drew This branch update is rebase-only: PR #2823 was rebased onto the restored PR #2822 head. No intentional feature or scope changes were added to this PR; the conflict resolution preserves the parent stack behavior while retaining the existing gateway-composition and telemetry commits.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[drew]

/ok to test 8d0eed7

[drew]

/ok to test 904c94b0d9b4d4c87330ecbb384ce401025115e3

[copy-pr-bot]

/ok to test 904c94b0d9b4d4c87330ecbb384ce401025115e3

@drew, there was an error processing your request: E2

See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/2/

[drew]

/ok to test 904c94b

[github-actions]

Label test:e2e applied for 1cf8a8b. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.