Skip to content

Use bespokelabs-sandbox to run code in sandboxes - #716

Merged
madiator merged 10 commits into
mainfrom
sandbox
Mar 28, 2026
Merged

madiator merged 10 commits into
mainfrom
sandbox

Conversation

@madiator

@madiator madiator commented Mar 28, 2026 •

Copy link
Copy Markdown
Contributor

Note

Medium Risk
Replaces multiple execution backends with a new bespokelabs-sandbox wrapper and changes the default backend to local, which may affect runtime behavior, file output paths, and timeout/exit-code handling across environments.

Overview
Code execution backends are consolidated onto bespokelabs-sandbox. The PR removes the bespoke multiprocessing, ray, docker, and e2b backends and introduces SandboxCodeExecutionBackend, created via a simplified factory with aliasing (multiprocessing -> local).

API/config and examples are updated for the new sandbox model. CodeExecutor now defaults to backend="local", backend config adds image (with legacy docker_image/base_url supported), and examples/tests are adjusted for new output file paths and params (e.g., docker_image -> image) plus new unit coverage for sandbox execution/error cases; CI no longer installs the removed code_execution extra and instead depends on bespokelabs-sandbox.

Written by Cursor Bugbot for commit d7dd3f6. This will update automatically on new commits. Configure here.

madiator and others added 4 commits March 27, 2026 19:49
Remove the four built-in execution backends (multiprocessing, docker, ray, e2b)
and replace them with a single unified backend powered by bespokelabs-sandbox.
This delegates sandbox lifecycle, isolation, and multi-provider support to the
sandbox package while keeping curator's orchestration layer (caching, retries,
request/response files) intact.

- Add SandboxCodeExecutionBackend using bespokelabs.sandbox.Sandbox
- bespokelabs-sandbox is a required dep; "local" backend works out of the box
- code_execution extra installs docker/ray/e2b SDKs for non-local backends
- Preserve legacy docker_image and base_url config params for backwards compat
- Tar archives retain directory structure (os.path.relpath, not basename)
- Regenerate poetry.lock for the new dependency graph
- Update tests and examples for new backend names

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Backend SDKs are managed by bespokelabs-sandbox's own extras
(e.g. pip install bespokelabs-sandbox[docker]).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The code_execution extra was removed from pyproject.toml since backend
SDKs are now managed by bespokelabs-sandbox's own extras.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
madiator and others added 2 commits March 27, 2026 20:22
Non-zero exit codes are valid outcomes (e.g. code-verification workloads).
Setting error on non-zero exits caused validate_existing_response_file to
discard and re-run those responses on every resume.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comment thread src/bespokelabs/curator/code_executor/code_execution_backend/sandbox_backend.py Outdated
madiator and others added 2 commits March 27, 2026 20:53
- Test all dataclass add/append methods on TokenUsage, CostInfo,
  RequestStats, PerformanceStats
- Test CuratorResponse save/load, to_dict, get_failed_requests,
  __getattr__ deprecation, update_tracker_stats offline path
- Test safe_model_dump normal and fallback paths

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The sandbox lifecycle timeout (timeout_secs) is a safety net, not the
per-execution limit. Use `timeout N` in the shell command to kill the
python process after exactly N seconds, matching the old subprocess.run
timeout behavior.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

madiator and others added 2 commits March 27, 2026 21:16
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Distinguish exit code 124 (timeout) with message="timeout"
- Format non-zero exit code errors with stderr details
- Preserve stdout/stderr/files in exception handler
- Add unit tests with fake sandbox for all error paths

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@madiator
madiator merged commit 8f26600 into main Mar 28, 2026
1 of 2 checks passed
madiator added a commit that referenced this pull request Jun 11, 2026
Since #716, bespokelabs-sandbox (a required dependency) installs into the
same bespokelabs.* namespace package that CI measures with
--cov=bespokelabs, so ~1,100 statements of third-party sandbox code
(docker/e2b/modal/ray/... backends, mostly unexercised here) entered the
coverage denominator and dropped the total from 79.5% to ~72%, tripping
the 80% gate. Measure only curator's own package instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
madiator pushed a commit that referenced this pull request Jun 11, 2026
* Fix code execution on the local sandbox backend

Since #716, CodeExecutor with the default "local" backend always failed:
the command `cd /workspace && ...` escapes the sandbox path rebasing.
Host-style sandbox backends (local, ray) map absolute paths under a temp
$SANDBOX_ROOT via a shell prelude that wraps common file utilities, but
`cd` is not among them, so the shell tried to enter the host's
nonexistent /workspace and every execution returned an error output.
This is also why tests/code_executor/test_code_execution.py::
test_simple_code_execution_local has been failing CI on main.

Rebase the workspace path explicitly with "${SANDBOX_ROOT:-}/workspace":
host-style backends expand their sandbox root, while container backends
(docker, e2b, ...) leave SANDBOX_ROOT unset and keep using the real
/workspace.

Also invoke `python3` instead of `python`: stock macOS hosts have no
bare `python` binary, so the local backend failed on Macs even with the
cd fix; python3 is present on CI runners and in python docker images.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Scope CI coverage to bespokelabs.curator

Since #716, bespokelabs-sandbox (a required dependency) installs into the
same bespokelabs.* namespace package that CI measures with
--cov=bespokelabs, so ~1,100 statements of third-party sandbox code
(docker/e2b/modal/ray/... backends, mostly unexercised here) entered the
coverage denominator and dropped the total from 79.5% to ~72%, tripping
the 80% gate. Measure only curator's own package instead.
@madiator madiator mentioned this pull request Jun 14, 2026
3 of 5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant