Repository navigation
Configure native SSH and SCP with sandbox ssh-setup - #7590
Conversation
dc841c7 to
b9424a8
Compare
5cd926a to
417c077
Compare
|
Commit: Regression test reportSelected 1 changed tests (limit=20, 0 not selected) 1 tests: regression=1 unreleased=0 coverage=0 failing=0
TestAccept/cmd/sandbox/ssh-setup: branch ✅ | main (7f25e1e) ❌ | latest (v1.27.0) ❌main (7f25e1e): latest (v1.27.0): same output as main (7f25e1e) |
✅ Integration test reportCommit: d8d0d72
Top 3 slowest tests (at least 2 minutes):
|
b9424a8 to
c26a360
Compare
417c077 to
5441e69
Compare
c26a360 to
a725f0c
Compare
9434fda to
637d42e
Compare
a725f0c to
1a2cf4d
Compare
637d42e to
c4637bc
Compare
b69ba16 to
acd94d2
Compare
c4637bc to
aab1071
Compare
987588f to
4d9dfba
Compare
69946b9 to
12e4dd1
Compare
anton-107
left a comment
There was a problem hiding this comment.
Looks good. go test -race passes for experimental/ssh/sandbox, internal/sshconfig and cmd/sandbox, and TestAccept/cmd/sandbox/ssh-setup passes locally. I also had real OpenSSH (ssh -G) parse a generated config with a HOME containing a space and %d, a CLI path with a space, a profile named p 1, and a user ControlPath ~/.ssh/cm-%r@%h:%p. The %% escaping is correct, and the short alias keeps ControlPath within the length limit.
Non-blocking: cmd/sandbox/README.md still says a broken forwarding connection ends the SSH session. Since #7589 negotiates resume v2, that paragraph should describe resume instead.
LGTM.
There was a problem hiding this comment.
Automated review at 4d9dfba. Each finding was verified against the code.
No blocking issues found. 2 non-blocking comment(s) below.
This PR adds databricks sandbox ssh-setup to write a persistent SSH Host entry for native ssh/scp/sftp through the existing sandbox proxy. The implementation is clean: alias validation is thorough, idempotency and overwrite protection work correctly, and the acceptance and unit tests cover the main paths well. Two minor items below.
88a9af8 to
e7e1fc7
Compare
6ab5086 to
c2d75cd
Compare
|
@anton-107 Updated the README in e7e1fc7 to describe 25-minute stream renewal, resuming the same session for up to 60 seconds after an unexpected disconnect, and termination if recovery fails or the remote session is gone. This PR's own binary passed CLI SSH and ssh-setup/native SSH/SCP/SFTP against separate freshly created Gateway and public API sandboxes. Setup while stopped, idempotency, configuration parsing, overwrite protection, interactive sessions, PTY, exit status, and exact 1 MiB transfers passed. The test plan records the commands and results; local checks and Isaac review also pass. |
c2d75cd to
49d6175
Compare
747b807 to
7af91ee
Compare
7af91ee to
d8d0d72
Compare
❌ Integration test reportCommit: 1ede87e
27 interesting tests: 23 RECOVERED, 4 KNOWN
Top 50 slowest tests (at least 2 minutes):
|
🥞 Stacked PR
Use this link to review incremental changes.
Changes
Add
databricks sandbox ssh-setup SANDBOX_ID --name ALIAS --profile PROFILEto write a local SSH Host configuration using the existing sandbox proxy mode. Nativessh,scp, andsftpcan then connect using that alias. Setup requires an explicit Databricks profile, reuses local keys, retains a selected custom Databricks configuration file, and preserves unrelated SSH configuration. It does not start the sandbox.Sandbox SSH configuration helpers live alongside the other new sandbox implementation under
experimental/ssh/sandbox.Why
Support native SSH tools through the same authenticated forwarding path as
databricks sandbox ssh. Interactive CLI SSH continues to work without a persistent Host entry or an explicit setup step.Tests
Unit tests cover profile requirements, alias validation, idempotency, overwrite protection, and preservation of unrelated configuration. Installed OpenSSH parses the generated configuration. Acceptance tests exercise setup, repeated setup, saved connection settings, and invalid input.
./task test-sandbox test-exp-ssh, affected race tests, and./task fmt checks lintpass. The package-layout change was reviewed locally with Isaac, with no blocking findings.Real-sandbox validation (2026-11-19)
Passed against newly created staging sandboxes from both APIs:
POST /api/2.0/lakebox/sandboxesandPOST /api/2.0/sandboxes. Tested clean builds of this PR at88a9af89871bdc738512594c2c95cfb16df6297cand PR2 at6ab5086fb2f078c21c056ee47d14b69f0abc0be2. PR2's binary was tested directly for CLI SSH; this PR's binary generated the Host configuration used by native tools.Each sandbox was stopped before setup. Setup succeeded without starting it; the subsequent native SSH connection started it automatically. Tests used a separate SSH configuration file, preserving the user's normal configuration. Native commands used the generated alias, identity, host-key settings, and ProxyCommand without supplying transport arguments manually.
Payload:
bytes(range(256)) * 4096; SHA-256fbbab289f7f94b25736c58be46a994c441fd02552cc6022352e3d86d2fab7c83.The earlier live failures exposed recovery defects in PR2's transport, now fixed there. A local test proxy forwarded authenticated requests to a real sandbox while injecting startup failures. The previous build reproduced both visible failures; the tested revision recovered automatically, with no caller retries:
A separate 120-second real session survived a forced network interruption with the same remote process and all 2,481,600 stream bytes verified. The original upstream interruption could not be attributed from the available logs; the client failure modes were reproduced and fixed.
Local sandbox and SSH suites, race checks, formatting, checks, and linters pass. Isaac's final local review has no remaining findings. The installed OpenSSH client was 8.2, so native SCP tested legacy SCP; native SFTP was tested separately. OpenSSH 9's default-SFTP SCP and desktop IDEs were not exercised.
Validation after nonblocking review fixes (2026-11-19)
Tested a clean build of this PR at
e7e1fc7696145f9a5b87f6b071bf2cd231e092bbagainst two newly created real staging sandboxes, one created throughPOST /api/2.0/lakebox/sandboxesand one through the public Sandboxes API,POST /api/2.0/sandboxes. Each PR/API combination used its own sandbox; binaries were checked against their commit versions and SHA-256 before and after testing.PR3 was tested using its own binary for CLI SSH,
ssh-setup, and every native-tool ProxyCommand. Setup ran while the sandbox was stopped and did not start it. Repeated setup left both configuration files unchanged;ssh -Gparsed the saved profile, identity, host-key checks, and ProxyCommand. Tests also checked alias overwrite protection and explicit overwrite while preserving unrelated configuration.The PTY test launched each interactive command in a local pseudo-terminal, waited for the remote shell prompt, typed a unique marker,
id -un,pwd, andexit 29, then checked the output and exit status. Binary tests usedbytes(range(256)) * 4096and compared stdout and downloaded files byte for byte. Payload SHA-256:fbbab289f7f94b25736c58be46a994c441fd02552cc6022352e3d86d2fab7c83.The full sandbox and compute SSH suites, affected race tests, formatting, checks, lint, and local Isaac review passed.
Commands above show the tested operations with workspace-specific values and unique remote file names replaced by variables. Installed OpenSSH 8.2 uses legacy SCP; SFTP was exercised separately. The earlier long-session handover results remain historical; this rerun covers the nonblocking fixes and both creation APIs.
This PR was written with Codex.