Skip to content

Commit e27a4fb

Browse files
committed
fix: Enforce the adaptive indexing refusal, and three review follow-ups
Four findings from review on #349, all against bfa28bc. `__getitem__` now rejects an adaptive space explicitly. Review pointed out that it resolved a negative index against `self.len` — the count `__len__` refuses to report for an adaptive space, on the grounds that it is not knowable — and that `it[-1]` therefore declined only because `get` happened to decline downstream. Confirmed by reading it: `self.len` is captured unconditionally at construction and `__getitem__` had no adaptive check. Since 0.6.0 just extended `get` to contiguous chunked spaces, a later extension to adaptive would have made `it[-1]` answer against a count `len()` still withholds. The guard makes the documented behaviour the enforced one and lets the message name the reason. The message is now the pure-Python reference's, verbatim. The type stays `IndexError` rather than moving to v0's bare `LookupError`: `IndexError` is a `LookupError` subclass, so a caller catching either type is served by both implementations, while changing the type would break anyone catching `IndexError`. Full type parity is a breaking change and is not smuggled into a dependency bump. The spec now records the divergence, which review noted the rewritten text had made read as though the area were at parity. `test_indexing_observes_the_override` had a docstring the previous commit falsified: it said random access needs a non-sequential space and that a contiguous chunked space is always sequential, then pointed at "the test below" — which is now the test proving both clauses wrong. Replaced with the reason the NONCONTIGUOUS choice actually still holds, which review identified correctly: it renders a single-task chunk as a bare `1` where CONTIGUOUS renders `1-1`, and that is what the assertions depend on. `scripts/check_third_party_licenses.sh` no longer creates a second temp file. The `$generated.eol` sibling the previous commit introduced was not registered with the `trap`, so it leaked if the script died between the redirect and the `mv`. Folding `sed` into the redirect that already builds `$generated` drops the file, the `mv` and the portability caveat at once. Review suggested `tr -d`; kept `sed 's/\r//'` because `tr -d` also deletes a CR that is not a line ending, and the point here is EOL normalization. The script still reports the file up to date, so the output is byte-identical. Verified: 5567 passed, 24 skipped, 3 xfailed with the 94% coverage gate; cargo clippy --all-targets -D warnings, cargo fmt, ruff, black, mypy clean; scripts/check_third_party_licenses.sh reports up to date. Removing the new guard fails `test_an_adaptive_space_still_refuses_indexing` and nothing else, from a green baseline of 18 in that class. Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com>
1 parent 65da216 commit e27a4fb

4 files changed

Lines changed: 62 additions & 19 deletions

File tree

‎rust-bindings/src/model/step_param_space.rs‎

Lines changed: 20 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -214,6 +214,26 @@ impl PyStepParameterSpaceIterator {
214214
}
215215

216216
fn __getitem__(&self, py: Python<'_>, index: isize) -> PyResult<Py<PyDict>> {
217+
// Random access uses a fresh iterator — don't disturb the
218+
// persistent iter's cursor or its adaptive Arc. It must carry the
219+
// same chunk override, or indexing would report chunks that
220+
// iteration never yields.
221+
let iter =
222+
StepParameterSpaceIterator::new_with_chunk_override(&self.space, self.chunk_override)
223+
.map_err(model_err_to_py)?;
224+
// Rejected here rather than left to `get`, because the negative-index
225+
// arithmetic below resolves against `self.len` — the count `__len__`
226+
// refuses to report for an adaptive space. Without this, `it[-1]` declines
227+
// only incidentally, and would start answering against that hidden count
228+
// if `get` were ever extended to adaptive spaces the way 0.6.0 extended it
229+
// to contiguous ones. The message is the pure-Python reference's; the type
230+
// stays `IndexError`, which is a `LookupError` as v0 raises, so `except`
231+
// clauses for either keep working.
232+
if iter.chunks_adaptive() {
233+
return Err(pyo3::exceptions::PyIndexError::new_err(
234+
"Items cannot be retrieved by index because the parameter space uses adaptive chunking.",
235+
));
236+
}
217237
let idx = if index < 0 {
218238
let adjusted = self.len as isize + index;
219239
if adjusted < 0 {
@@ -225,13 +245,6 @@ impl PyStepParameterSpaceIterator {
225245
} else {
226246
index as usize
227247
};
228-
// Random access uses a fresh iterator — don't disturb the
229-
// persistent iter's cursor or its adaptive Arc. It must carry the
230-
// same chunk override, or indexing would report chunks that
231-
// iteration never yields.
232-
let iter =
233-
StepParameterSpaceIterator::new_with_chunk_override(&self.space, self.chunk_override)
234-
.map_err(model_err_to_py)?;
235248
match iter.get(idx) {
236249
Some(params) => task_param_set_to_py(py, &params),
237250
None => Err(pyo3::exceptions::PyIndexError::new_err(

‎scripts/check_third_party_licenses.sh‎

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -150,17 +150,15 @@ awk -v re="^[*][*] ($workspace_pattern); version " '
150150

151151
# ── Combine ───────────────────────────────────────────────────────────
152152

153+
# Piped through `sed` on the way in, so the EOL strip needs neither a second temp
154+
# file nor `sed -i`, which wants a backup suffix on BSD sed and refuses one on GNU.
153155
{
154156
echo ""
155157
echo ""
156158
cat "$python_section"
157159
echo ""
158160
cat "$rust_section"
159-
} > "$generated"
160-
161-
# Ensure consistent EOL. Rewritten through a temp file rather than `sed -i`,
162-
# which needs a backup suffix on BSD sed and rejects one on GNU sed.
163-
sed 's/\r//' "$generated" > "$generated.eol" && mv "$generated.eol" "$generated"
161+
} | sed 's/\r//' > "$generated"
164162

165163
if [[ "$mode" == "update" ]]; then
166164
cp "$generated" "$OUTPUT_FILE"

‎specs/python-model-interface.md‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1201,10 +1201,24 @@ the template's.
12011201

12021202
One current-implementation limitation remains. An *adaptive* space has no
12031203
knowable count until it is walked, so `len(it)` raises `ValueError` and
1204-
every index — including `-1` — raises `IndexError`. Iteration still
1204+
`it[i]` is refused for every index, negative included. Iteration still
12051205
yields, which is what distinguishes an unknown count from an empty space.
12061206
Supplying the override makes the space static and lifts both.
12071207

1208+
That refusal is enforced rather than incidental: `__getitem__` rejects an
1209+
adaptive space before it resolves a negative index, because the
1210+
arithmetic for a negative index would otherwise run against the length
1211+
`__len__` declines to report.
1212+
1213+
One divergence from the v0 reference here, in the exception *type*. v0
1214+
raises a bare `LookupError` for an adaptive `it[i]`; v1 raises
1215+
`IndexError`, which is a `LookupError` subclass, so a caller catching
1216+
either type is served by both implementations. The messages agree.
1217+
Because `IndexError` also means "past the end" in v1, the two conditions
1218+
are told apart by the message rather than the type — the adaptive refusal
1219+
says `Items cannot be retrieved by index because the parameter space uses
1220+
adaptive chunking.` where a real overrun says `index out of range`.
1221+
12081222
### `StepDependencyGraph`
12091223

12101224
Step dependency graph for topological ordering.

‎test/openjd/model_v1/test_step_param_space_iter.py‎

Lines changed: 24 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -817,8 +817,10 @@ def test_indexing_observes_the_override(self) -> None:
817817
"""``__getitem__`` builds a fresh iterator, so it has to carry the override too,
818818
or indexing reports chunks iteration never yields.
819819
820-
Uses NONCONTIGUOUS because random access needs a non-sequential space, and a
821-
CONTIGUOUS chunked space is always sequential (see the test below).
820+
NONCONTIGUOUS because that is what the assertions below depend on: it renders a
821+
single-task chunk as a bare ``"1"``, where CONTIGUOUS renders ``"1-1"``. Both
822+
support random access — see
823+
``test_a_contiguous_space_supports_indexing_with_or_without_the_override``.
822824
"""
823825
it = StepParameterSpaceIterator(
824826
step=self._step({"defaultTaskCount": 5, "rangeConstraint": "NONCONTIGUOUS"}),
@@ -863,14 +865,30 @@ def test_a_contiguous_space_supports_indexing_with_or_without_the_override(self)
863865
def test_an_adaptive_space_still_refuses_indexing(self) -> None:
864866
"""The negative control for the test above, and the remaining limitation
865867
recorded in ``specs/python-model-interface.md``: an adaptive space has no
866-
knowable count, so ``len()`` raises and every index is out of range. Iteration
867-
still yields, which is what distinguishes "unknown count" from "empty"."""
868+
knowable count, so ``len()`` raises and every index is refused. Iteration
869+
still yields, which is what distinguishes "unknown count" from "empty".
870+
871+
The refusal is enforced rather than incidental. ``__getitem__`` resolves a
872+
negative index against the length ``__len__`` declines to report, so leaving
873+
the rejection to ``get`` would let ``it[-1]`` answer against that hidden count
874+
if ``get`` were extended to adaptive spaces, as 0.6.0 extended it to
875+
contiguous ones. The message therefore names the reason instead of saying
876+
"index out of range", and it is the pure-Python reference's wording.
877+
"""
868878
it = StepParameterSpaceIterator(step=self._step(self._ADAPTIVE))
869879
assert it.chunks_adaptive is True
870-
for index in (0, -1):
880+
for index in (0, 1, -1, -100):
871881
with pytest.raises(IndexError) as excinfo:
872882
_ = it[index]
873-
assert str(excinfo.value) == "index out of range"
883+
assert str(excinfo.value) == (
884+
"Items cannot be retrieved by index because the parameter space "
885+
"uses adaptive chunking."
886+
)
887+
# v0 raises a bare LookupError here, not an IndexError. IndexError is a
888+
# LookupError subclass, so a caller catching either is served by both
889+
# implementations; the type itself still differs, which the spec records.
890+
with pytest.raises(LookupError):
891+
_ = it[0]
874892
assert self._frames(StepParameterSpaceIterator(step=self._step(self._ADAPTIVE))) == [
875893
"1-5",
876894
"6-10",

0 commit comments

Comments
 (0)