Replace the Python openjd-sessions-for-python implementation with bindings
to the Rust openjd-sessions crate, exposed through the existing
openjd._openjd_rs native extension in openjd-model-for-python.
openjd-model-for-python (Rust extension)
└── openjd._openjd_rs ← single .so, already ships expr + model
├── expr bindings ← done
├── model bindings ← done
└── sessions bindings ← new: wraps openjd-sessions crate
openjd-sessions-for-python (Python)
└── openjd.sessions
└── imports from openjd._openjd_rs (via openjd.model / openjd.expr)
Sessions is a dependency of model, so it can import from openjd._openjd_rs.
The Rust sessions crate depends on openjd-expr and openjd-model, which
are already linked into the extension.
The openjd-sessions crate provides:
| Rust type | Description |
|---|---|
Session |
Core state machine: enter/exit environments, run tasks, cancel, cleanup |
SessionConfig |
Configuration for creating a session |
SessionState |
Enum: Ready, Running, Canceling, ReadyEnding, Ended |
ActionState |
Enum: Running, Success, Failed, Canceled, Timeout |
ActionStatus |
Progress/status/fail message/exit code for current action |
ActionResult |
Final result of an action (state, exit code, stdout, stderr) |
ActionMessage |
Parsed openjd stdout messages (progress, env, status, fail) |
SessionError |
Error enum for all session operations |
ScriptRunnerState |
Enum: Ready, Running, Canceling, Canceled, Timeout, Failed, Success |
CancelMethod |
Enum: Terminate, NotifyThenTerminate |
EmbeddedFiles |
File materialization with scope (Step/Env) |
SessionUser trait |
Cross-user execution (PosixSessionUser, WindowsSessionUser) |
SubprocessResult |
Exit state + code + stdout from a subprocess |
Key methods on Session:
with_config(SessionConfig)— constructorenter_environment(env, resolved_symtab, identifier, os_env_vars)— asyncexit_environment(identifier, resolved_symtab, keep_running, os_env_vars)— asyncrun_task(script, task_params, resolved_symtab, os_env_vars)— asyncrun_subprocess(command, args, timeout, os_env_vars, ...)— asynccancel_action(time_limit, mark_failed)cleanup()build_symbol_table(task_params, base)— symbol table constructionevaluate_env_vars(extra)— cumulative env var evaluation- Properties:
session_id,state,working_directory,files_directory,action_status
class SessionState:
READY = ...
RUNNING = ...
CANCELING = ...
READY_ENDING = ...
ENDED = ...
class ActionState:
RUNNING = ...
SUCCESS = ...
FAILED = ...
CANCELED = ...
TIMEOUT = ...class ActionStatus:
state: ActionState
progress: Optional[float]
status_message: Optional[str]
fail_message: Optional[str]
exit_code: Optional[int]
class ActionResult:
state: ActionState
exit_code: Optional[int]
stdout: strclass Session:
def __init__(
self,
*,
session_id: str,
job_parameter_values: dict[str, ParameterValue],
path_mapping_rules: Optional[list[PathMappingRule]] = None,
callback: Optional[Callable] = None,
os_env_vars: Optional[dict[str, str]] = None,
retain_working_dir: bool = False,
session_root_directory: Optional[Path] = None,
user: Optional[SessionUser] = None,
job_template: Optional[JobTemplate] = None,
environment_templates: Optional[list[EnvironmentTemplate]] = None,
caller_limits: Optional[CallerLimits] = None,
): ...
# Properties
session_id: str
state: SessionState
working_directory: Path
files_directory: Path
action_status: Optional[ActionStatus]
# Environment lifecycle (non-blocking — starts action, returns immediately)
def enter_environment(
self,
*,
environment: Environment,
identifier: Optional[str] = None,
os_env_vars: Optional[dict[str, str]] = None,
) -> str: ...
def exit_environment(
self,
*,
identifier: str,
os_env_vars: Optional[dict[str, str]] = None,
) -> None: ...
# Task execution (non-blocking — starts action, returns immediately)
def run_task(
self,
*,
step_script: StepScript,
task_parameter_values: Optional[dict] = None,
os_env_vars: Optional[dict[str, str]] = None,
) -> None: ...
# Subprocess execution (non-blocking)
def run_subprocess(
self,
*,
command: str,
args: Optional[list[str]] = None,
timeout: Optional[float] = None,
os_env_vars: Optional[dict[str, str]] = None,
use_session_env_vars: bool = True,
log_banner_message: Optional[str] = None,
) -> None: ...
# Control
def cancel_action(
self,
*,
time_limit: Optional[float] = None,
mark_action_failed: bool = False,
) -> None: ...
def cleanup(self) -> None: ...caller_limits takes the same
CallerLimits
a submitting service passes to decode_job_template and create_job.
The session applies its run-time half — the resolved-value caps and the
per-expression evaluation budgets — to the values it resolves, and
ignores the fields with no run-time meaning. This is the enforcement
boundary for those caps: a worker can run a job that never passed
through the validating process, so a session that is given no limits
enforces nothing beyond the spec.
The SessionUser types involve OS-specific credential validation (sudo
checks, Windows logon tokens) that are better kept in Python for now.
The Rust SessionUser trait accepts user/group strings — the Python side
validates credentials and passes the identity through.
class PosixSessionUser:
user: str
group: str
class WindowsSessionUser:
user: str
password: Optional[str]The Python callback Callable[[str, ActionStatus], None] is invoked from
Rust via PyO3 when action status changes. The Rust drive_action loop
sends ActionMessage values through a channel; the binding converts them
to ActionStatus and calls the Python callback on the main thread.
The Rust sessions crate uses tokio async for subprocess I/O. The Python
sessions API is non-blocking: run_task(), enter_environment(), and
exit_environment() start a subprocess in a background thread and return
immediately. The caller polls session.action_status or receives a callback
when the action completes.
Options:
-
Spawn tokio task, return immediately — Match the Python API. Each
run_task/enter_environmentcall spawns a tokio task on a background runtime. The Python method returns immediately. The callback fires from the tokio runtime when the action completes. The caller pollsaction_statusor waits for the callback. -
Block until complete — Change the Python API to block. Would break existing callers (worker agent, CLI) that rely on non-blocking behavior.
Decision: Option 1 (non-blocking, matching current Python API). We
maintain a single tokio::Runtime on the PySession object. Each action
method spawns a task on that runtime and returns immediately. The runtime
thread drives subprocess I/O and calls the Python callback (acquiring the
GIL) when the action finishes or status changes.
#[pymethods]
impl PySession {
fn run_task(&mut self, ...) -> PyResult<()> {
// Spawn on the session's tokio runtime — returns immediately
let handle = self.runtime.spawn(async move {
session.run_task(script, task_params, ...).await
});
self.action_handle = Some(handle);
Ok(())
}
#[getter]
fn action_status(&self) -> Option<PyActionStatus> {
// Read current status from shared state
}
}The GIL is released during subprocess execution since the tokio runtime runs on its own thread.
Add to rust-bindings/src/sessions/ in the model-for-python extension:
SessionStateenumActionStateenum (replaces PythonActionState(str, Enum))ActionStatusclassActionResultclassSessionError→ Python exceptionScriptRunnerStateenum
Wire into lib.rs module registration. Update openjd.sessions._types
to import from openjd._openjd_rs instead of defining locally.
Test: Import enums, check values match Python originals.
Session.__init__wrappingSession::with_config- Properties:
session_id,state,working_directory,files_directory cleanup()methodaction_statuspropertycancel_action()method
Test: Create session, check properties, cleanup.
enter_environment()— wraps asyncSession::enter_environmentexit_environment()— wraps asyncSession::exit_environment- Callback plumbing (ActionMessage → ActionStatus → Python callback)
- Env var tracking (openjd_env, openjd_unset_env)
Test: Enter/exit environment with onEnter/onExit scripts, verify env vars are set/unset, callback fires.
run_task()— wraps asyncSession::run_task- Symbol table construction with path mapping
- Embedded file materialization
- Let binding evaluation
- Action filter (openjd_progress, openjd_status, openjd_fail)
Test: Run tasks with parameters, embedded files, let bindings. Verify action status, progress, stdout capture.
run_subprocess()— for worker agent install/sync operations- Cancelation with time limits
- Cross-user execution (PosixSessionUser passthrough)
- Redacted environment variables
evaluate_env_vars()for external consumers
Test: Run subprocess, cancel mid-execution, cross-user scenarios.
- Remove Python runner classes (
_runner_base.py,_runner_step_script.py,_runner_env_script.py) - Remove Python embedded files (
_embedded_files.py) - Remove Python action filter (
_action_filter.py) - Simplify
_session.pyto thin wrapper over RustSession - Remove subprocess management (
_subprocess.py) - Keep
_session_user.py(credential validation stays in Python) - Keep
_logging.py(Python logging integration)
# rust-bindings/Cargo.toml additions
openjd-sessions = { git = "...", branch = "prototype" }
tokio = { version = "1", features = ["rt", "process", "io-util", "sync", "time"] }
uuid = { version = "1", features = ["v4"] }The Rust sessions crate uses the log crate. We bridge to Python logging:
- Set up a
log→ Pythonloggingbridge in the PyO3 module init - The Rust
session_log!macro tags messages with three extra attributes:session_id: the session correlation identifier, represented as a Pythonstropenjd_log_content: theLogContentbitmask, represented as a decimal Pythonstropenjd_timestamp_usec: microseconds since the Unix epoch, represented as a decimal Pythonstr
pyo3-logstringifies Rust key-values while constructing the PythonLogRecord. Theopenjd.sessions._v1wrapper normalizes the content bitmask to its publicLogContenttype and the timestamp to anintfor consumers.- Rust records not emitted through
session_log!may omit some or all of these extra attributes.
This preserves the existing log output that the worker agent and CLI parse.
The Python Session class signature stays the same. Callers (worker agent,
CLI openjd run) don't need changes. Internal implementation moves from
Python subprocess management to Rust async subprocess management.
Breaking changes:
ActionStatebecomes a Rust enum (notstr, Enum) — string comparison likestate == "running"breaks, must usestate == ActionState.RUNNING- Internal types (
ScriptRunnerState,ActionMonitoringFilter) are no longer importable from sessions submodules
Pickle is used by the Deadline Cloud worker-agent IPC layer, so all
value types that flow across that boundary are pickleable. The
Session class itself is not — it owns live OS resources (subprocess,
working directory, file handles) that are not meaningful to serialize.
| Type | Reduces through |
|---|---|
SessionState |
variant name (READY, RUNNING, ENDED, …) |
ActionState |
variant name (RUNNING, SUCCESS, FAILED, …) |
ScriptRunnerState |
variant name |
ActionStatus |
private _from_state classmethod that round-trips state, progress, status_message, fail_message, exit_code, started_at, ended_at |
ActionResult |
constructor arguments (state, exit_code, stdout) |
PosixSessionUser |
constructor arguments (user, group) |
WindowsSessionUser |
constructor arguments (user, password, logon_token) — see caveats below |
SessionError, BadCredentialsException |
standard exception pickle, under their canonical openjd.sessions._v1 module path |
WindowsSessionUser pickle caveats:
- The password is stored plaintext in the pickle output. Avoid pickling
WindowsSessionUserto disk or over an untrusted channel. This matches the legacy Pythondataclasswhich also storedpasswordas a plain field. logon_tokenis a process-local Win32HANDLE. The integer value pickles correctly but does not refer to a valid handle in another process; unpickling on a different process will fail atLogonUservalidation.- Unpickling on a non-Windows host raises
RuntimeError, matching the construction-time behavior.
Session is intentionally not pickleable — pickling a live session
object would silently produce a stub that cannot run anything. Take a
checkpoint of the inputs (Session.__init__ arguments and current
ActionStatus) instead.