Skip to content

Commit a3d4f4e

Browse files
authored
fix(sessions): Escape strings in the sessions __repr__ output (#361)
* fix(sessions): Escape strings in the sessions __repr__ output The sessions reprs were built with `format!("{:?}")`, which is not a Python literal writer. Rust's `Debug` for `str` agrees with Python on the quote, the backslash and the C0 controls, but renders anything else non-printable as `\u{a0}`, and CPython wants exactly four hex digits after `\u`. The literal then does not parse at all: >>> repr(ActionResult(state=ActionState.SUCCESS, exit_code=0, ... stdout="a\xa0b")) 'ActionResult(state=SUCCESS, exit_code=Some(0), stdout="a\\u{a0}b")' SyntaxError: truncated \uXXXX escape That made a repr corruptible by its own data. `ActionResult.stdout` is captured process output, so a non-ASCII byte in a job's stdout is ordinary rather than adversarial, and `PosixSessionUser` carries a user name that arrives from outside. Both reach log lines and exception messages. Every string field now goes through CPython's own `repr()` via a new `py_repr` module. Delegating retires the bug class instead of reimplementing Python's escaping table: the output is by construction whatever the running interpreter produces, including its per-string choice of quote character. openjd-rs#374 added an escaping writer upstream for `ExprValue`, but it is `pub(crate)` in openjd-expr and so unreachable from this crate; PyO3 exposes the interpreter's `repr()` directly, which needs no upstream change and is the same oracle the tests assert against. Two neighbouring defects in the same expressions went with it, because leaving them would have kept the repr un-evaluable and so left the fix unverifiable by round-trip: * `Debug` for `Option` emitted `exit_code=Some(0)`, a `NameError`. * The state field rendered as a bare `Success` / `SUCCESS`, also a `NameError`. It now uses the spelling the enum's own repr already produces, `ActionState.SUCCESS`. `eval(repr(x)) == x` now holds for `ActionResult` across all 16 test inputs, which is what pins the escaping rather than a spelling assertion alone. Four sites changed: `ActionResult`, `ActionStatus`, `PosixSessionUser`, `WindowsSessionUser` and `Session`. `SessionState.__repr__` and `ScriptRunnerState.__repr__` were already correct. Adds test/openjd/sessions/, which did not exist -- the sessions surface had no Python tests beyond a module-name check. 113 cases over 16 inputs: the characters Rust renders as `\u{...}`, the ones it escapes correctly, printable non-ASCII that must survive verbatim, and negative controls for text needing no escaping. `group` is asserted separately from `user` because a fix applied to only the first argument would pass every `user` case. Mutation-checked, each behaviour reverted independently against a green 113-case baseline: escaping 67 failed, Option 44 failed, enum spelling 43 failed. No stub regeneration: a `Python<'_>` token is invisible to Python and `PyResult<String>` still maps to `str`, so the emitted signature is unchanged. `SymbolTable.__repr__` already uses this form and its stub entry is `def __repr__(self) -> builtins.str`. WindowsSessionUser is unverified -- it cannot be constructed off Windows (`RuntimeError: Only available on Windows systems`), so its change is by inspection and shares the helper the other three sites test. Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com> * test(sessions): Gate the PosixSessionUser repr tests on posix The binding gates construction on `#[cfg(unix)]` and raises `RuntimeError: Only available on posix systems.` elsewhere, so the four tests that build one failed on the Windows matrix: 49 failures, exactly the 16+16+16+1 cases those tests parametrize. Mirrors the Rust guard with `os.name`. WindowsSessionUser gets no counterpart: off the process user it demands a password or a logon token, so it cannot be built with an arbitrary name just to read its repr. Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com> * fix(sessions): Address review findings on the repr escaping Five defects from review, each measured before fixing. **py_repr::py_str could panic instead of raising.** `.to_string()` on a `Bound<PyString>` resolves to PyO3's `Display`, which calls `str()` on the object and has nowhere to put a `PyErr` -- `ToString` treats the formatter error as unreachable and panics. That put an unwind across the FFI boundary in the one function four reprs route through, and reprs are evaluated from logging and exception formatting where an unwind loses the diagnostic that prompted it. Now uses `to_cow()?`, which reads the UTF-8 directly, propagates failure, and skips a redundant interpreter round-trip. **Session's repr held the snapshot mutex across a CPython call.** `py_str` allocates, an allocation can trigger a GC pass, and a finalizer run by that pass may re-enter this Session and re-lock a non-reentrant `Mutex`. `lock_recover` maps a poisoned lock to `into_inner()`, so a panic there would not even surface. Reads `session_id` and `state` out from under the guard and drops it before formatting. Recorded in the module docs as a rule for the helper rather than a one-off. **Session's repr used `id=`, not the constructor's `session_id=`.** It parsed and then raised `TypeError: got an unexpected keyword argument 'id'` -- arguably worse than the old form, which failed at `compile()`. **PathMappingRule corrupted Windows paths silently.** Not a `{:?}` at all, so it was invisible to the search that found the others: it hand-rolled `'{}'` with no escaping. `C:\temp` rendered as `'C:\temp'`, which Python reads as `C:` + TAB + `emp` -- it parses and gives back a different string. An apostrophe in a path closed the literal early. This is the only repr found that corrupts on its most typical input, a Windows destination path, which is what path mapping exists to produce. Now routes through `py_str`. **ESC was missing from the test inputs.** Rust's `Debug` special-cases only the quote, the backslash, and NUL/tab/CR/LF; every other control falls through to the unparseable brace form. So ANSI colour sequences in captured stdout were almost certainly the highest-frequency trigger of this bug in the field, and no case exercised them. Adds ESC, U+0001 and U+001F, and narrows the module docs, which previously implied only non-ASCII was affected. `"a\x00b"` was also mislabelled as representative of C0 when it passes only because NUL has its own arm. Also corrects two claims of mine that were wrong: * A test comment said `ActionStatus` has no `__eq__`. It does (types.rs:337), comparing seven fields. The real reason its repr cannot round-trip is that the repr emits two of them, and it cannot be fixed by adding fields because `started_at`/`ended_at` are not constructor arguments. The misleading `eval()` assertion is replaced by one that pins the lossiness explicitly, so a later change to the field list is deliberate. * `py_repr`'s docs said delegating "retires the bug class". It retires it for `sessions/`; `model/` and the rest of `expr/` still carry it. Scoped the wording and pointed at the tracking note. Adds coverage for `Session` and `PathMappingRule`, which the first revision omitted -- both are constructible in a unit test, so the omissions were oversights rather than constraints. `Session` skips the NUL case: the id becomes a working-directory path component and the filesystem rejects it before any repr is taken, which is a constructor constraint, not a repr gap. 176 tests in the sessions module, up from 113. Two new mutants against a green baseline: reverting PathMappingRule to hand-rolled quoting fails 21, reverting Session's keyword fails 19. The pre-existing `TestPathMappingRuleRepr` in test/openjd/expr/test_path_mapping.py passes unmodified, which is the evidence that the change is a no-op for well-behaved paths. Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com> * test(sessions): Bound the Session repr cases by filename portability Windows CI failed 8 of the new TestSessionRepr cases: Session names its working directory after the session_id, so the inputs are bounded by what a filename may contain, not by what the repr can render. ESC, U+0001, U+001F, tab, newline, CRLF, the double quote and the backslash are all legal on APFS and ext4 and rejected by Windows -- 'The filename, directory name, or volume label syntax is incorrect. (os error 123)'. The previous revision excluded only NUL, which is the one case macOS caught. That fixed the instance rather than the class. Now filtered by predicate -- the C0 controls plus the punctuation Windows reserves -- so a new HOSTILE_STRINGS entry is classified automatically instead of silently breaking one platform. 10 of 19 cases survive, including U+00A0 and U+3000, the characters this change exists for. The 9 excluded are not left unverified: ActionResult and PosixSessionUser parametrize the full list through the same py_repr::py_str helper and touch no disk. What Session verifies is that it routes through that helper at all, and that its repr keyword matches its constructor. Adds test_the_case_filter_keeps_the_canonical_trigger so the filter cannot quietly empty out and leave the sweep asserting nothing. Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com> --------- Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com>
1 parent efba36c commit a3d4f4e

8 files changed

Lines changed: 402 additions & 25 deletions

File tree

‎rust-bindings/src/expr/path_mapping.rs‎

Lines changed: 11 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -122,16 +122,22 @@ impl PyPathMappingRule {
122122
&self.inner.destination_path
123123
}
124124

125-
fn __repr__(&self) -> String {
125+
fn __repr__(&self, py: Python<'_>) -> PyResult<String> {
126126
// Render `source_path_format` using its Python name
127127
// (`PathFormat.POSIX`) rather than the underlying Rust
128128
// enum's `Debug` name (`Posix`). Matches the Python
129129
// convention for enum repr.
130130
let fmt: PyPathFormat = self.inner.source_path_format.into();
131-
format!(
132-
"PathMappingRule(source_path_format=PathFormat.{}, source_path='{}', destination_path='{}')",
133-
fmt.variant_name(), self.inner.source_path, self.inner.destination_path
134-
)
131+
// The paths go through CPython's repr rather than `'{}'`. Hand-rolled
132+
// quoting corrupted a Windows destination silently: `C:\temp` emitted
133+
// `'C:\temp'`, which Python reads as `C:` + TAB + `emp`, and an
134+
// apostrophe in a path closed the literal early.
135+
Ok(format!(
136+
"PathMappingRule(source_path_format=PathFormat.{}, source_path={}, destination_path={})",
137+
fmt.variant_name(),
138+
crate::py_repr::py_str(py, &self.inner.source_path)?,
139+
crate::py_repr::py_str(py, &self.inner.destination_path)?,
140+
))
135141
}
136142

137143
/// Two `PathMappingRule`s compare equal when they have the

‎rust-bindings/src/lib.rs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
mod expr;
55
mod model;
66
mod pickle_helpers;
7+
mod py_repr;
78
mod sessions;
89

910
use pyo3::prelude::*;

‎rust-bindings/src/py_repr.rs‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
2+
// SPDX-License-Identifier: Apache-2.0
3+
4+
//! Rendering values into `__repr__` output that Python can parse.
5+
//!
6+
//! `format!("{:?}", s)` is not a Python literal writer. Rust's `Debug` for
7+
//! `str` special-cases only the quote, the backslash, and NUL, tab, CR and
8+
//! LF; every other control character and every non-printable falls through
9+
//! to Rust's brace form, `\u{1b}` or `\u{a0}`. Python wants exactly four
10+
//! hex digits after `\u`, so those do not parse. ESC is the one to keep in
11+
//! mind: ANSI colour sequences in captured process output hit this far more
12+
//! often than any exotic codepoint does.
13+
//!
14+
//! Delegating to CPython's own `repr()` removes the guesswork rather than
15+
//! reimplementing its escaping table: the output is by construction
16+
//! whatever the running interpreter produces, including its per-string
17+
//! choice of quote character.
18+
//!
19+
//! Scope: the `sessions` reprs route through here. Reprs under `model/`
20+
//! and the rest of `expr/` still use `{:?}` or hand-rolled quoting and
21+
//! carry the same defect — see the tracking note in the pull request that
22+
//! introduced this module. A new repr should use these helpers.
23+
//!
24+
//! Callers must not hold a lock across `py_str`: it re-enters the
25+
//! interpreter, which can run arbitrary Python (allocation may trigger a
26+
//! GC pass and with it `__del__` and weakref callbacks). Read what you
27+
//! need out from under the guard, drop it, then format.
28+
29+
use pyo3::prelude::*;
30+
use pyo3::types::{PyString, PyStringMethods};
31+
32+
/// CPython's `repr()` of `value`, ready to embed in a `__repr__`.
33+
pub(crate) fn py_str(py: Python<'_>, value: &str) -> PyResult<String> {
34+
// `to_cow` reads the UTF-8 directly and propagates failure. Going via
35+
// `to_string()` would resolve to PyO3's `Display`, which calls `str()`
36+
// on the object -- a second interpreter round-trip whose error has
37+
// nowhere to go but a panic out of `__repr__`.
38+
Ok(PyString::new(py, value).repr()?.to_cow()?.into_owned())
39+
}
40+
41+
/// An optional `int` as Python spells it. `Debug` would emit `Some(0)`,
42+
/// which evaluates to a `NameError`.
43+
pub(crate) fn py_opt_int(value: Option<i32>) -> String {
44+
match value {
45+
Some(v) => v.to_string(),
46+
None => "None".to_string(),
47+
}
48+
}

‎rust-bindings/src/sessions/session.rs‎

Lines changed: 14 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -691,8 +691,19 @@ impl PySession {
691691
}
692692
}
693693

694-
fn __repr__(&self) -> String {
695-
let snap = lock_recover(&self.snapshot);
696-
format!("Session(id={:?}, state={:?})", snap.session_id, snap.state)
694+
fn __repr__(&self, py: Python<'_>) -> PyResult<String> {
695+
// Read out from under the guard and drop it before calling into
696+
// CPython: `py_str` allocates, an allocation can trigger a GC pass,
697+
// and a finalizer run by that pass may re-enter this Session and
698+
// re-lock `snapshot`, which is not reentrant.
699+
let (session_id, state) = {
700+
let snap = lock_recover(&self.snapshot);
701+
(snap.session_id.clone(), snap.state)
702+
};
703+
Ok(format!(
704+
"Session(session_id={}, state=SessionState.{})",
705+
crate::py_repr::py_str(py, &session_id)?,
706+
crate::sessions::types::PySessionState::from(state).name()
707+
))
697708
}
698709
}

‎rust-bindings/src/sessions/session_user.rs‎

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -63,12 +63,12 @@ impl PyPosixSessionUser {
6363
self.inner.is_process_user()
6464
}
6565

66-
fn __repr__(&self) -> String {
67-
format!(
68-
"PosixSessionUser(user={:?}, group={:?})",
69-
self.inner.user(),
70-
self.inner.group()
71-
)
66+
fn __repr__(&self, py: Python<'_>) -> PyResult<String> {
67+
Ok(format!(
68+
"PosixSessionUser(user={}, group={})",
69+
crate::py_repr::py_str(py, self.inner.user())?,
70+
crate::py_repr::py_str(py, self.inner.group())?
71+
))
7272
}
7373

7474
/// Pickle support — round-trips through `__init__(user, *, group=...)`.
@@ -307,8 +307,11 @@ impl PyWindowsSessionUser {
307307
self.inner.is_process_user()
308308
}
309309

310-
fn __repr__(&self) -> String {
311-
format!("WindowsSessionUser(user={:?})", self.inner.user())
310+
fn __repr__(&self, py: Python<'_>) -> PyResult<String> {
311+
Ok(format!(
312+
"WindowsSessionUser(user={})",
313+
crate::py_repr::py_str(py, self.inner.user())?
314+
))
312315
}
313316

314317
/// Pickle support — round-trips through `__init__(user, *,

‎rust-bindings/src/sessions/types.rs‎

Lines changed: 10 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ impl From<SessionState> for PySessionState {
5353
impl PySessionState {
5454
/// Variant name as a string (e.g. `"READY"`).
5555
#[getter]
56-
fn name(&self) -> &'static str {
56+
pub(crate) fn name(&self) -> &'static str {
5757
match self {
5858
Self::READY => "READY",
5959
Self::RUNNING => "RUNNING",
@@ -323,8 +323,9 @@ impl PyActionStatus {
323323

324324
fn __repr__(&self) -> String {
325325
format!(
326-
"ActionStatus(state={:?}, exit_code={:?})",
327-
self.inner.state, self.inner.exit_code
326+
"ActionStatus(state=ActionState.{}, exit_code={})",
327+
self.state().name(),
328+
crate::py_repr::py_opt_int(self.inner.exit_code)
328329
)
329330
}
330331

@@ -525,13 +526,13 @@ impl PyActionResult {
525526
}
526527
}
527528

528-
fn __repr__(&self) -> String {
529-
format!(
530-
"ActionResult(state={}, exit_code={:?}, stdout={:?})",
529+
fn __repr__(&self, py: Python<'_>) -> PyResult<String> {
530+
Ok(format!(
531+
"ActionResult(state=ActionState.{}, exit_code={}, stdout={})",
531532
self.state.name(),
532-
self.exit_code,
533-
self.stdout,
534-
)
533+
crate::py_repr::py_opt_int(self.exit_code),
534+
crate::py_repr::py_str(py, &self.stdout)?,
535+
))
535536
}
536537

537538
fn __eq__(&self, other: &Self) -> bool {

‎test/openjd/sessions/__init__.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
2+
# SPDX-License-Identifier: Apache-2.0

0 commit comments

Comments
 (0)