Skip to content

Commit 546fa3e

Browse files
authored
fix: Make task parameter type names case-insensitive under EXPR (#350)
Template Schemas §2 says "job parameter and task parameter type names become case-insensitive" when the EXPR extension is enabled, and so are case-sensitive without it. `_normalize_parameter_type_case` already implemented that, gated on the extension, but it was registered only on `parameterDefinitions` in `JobTemplate` and `EnvironmentTemplate`. `StepParameterSpaceDefinition` had no such registration, so a task parameter spelled `type: int` was rejected with `Input tag 'int' found using 'type' does not match any of the expected tags` even with EXPR declared. The conformance fixture pinning this is `EXPR/job_templates/proposed/3.4.1--task-param-type-case-insensitive.yaml`, added by openjd-specifications#166, which failed here and in openjd-rs alike. The fix is the same three-line registration the other two models carry, on `taskParameterDefinitions`, before discriminated-union resolution. Nothing else was needed: `TaskParameterList` is a list of dicts, the shape the shared function was already written for, and `context.extensions` holds the effective set by then because `JobTemplate` declares `extensions` before `steps`. That covers all five task parameter types, `CHUNK[INT]` included -- §3.4.1.5 makes it a task parameter type name, so EXPR makes `chunk[int]` a spelling of it. The fold now uses an ASCII-only translation table instead of `str.upper()`. `str.upper()` is Unicode-aware and folds ten codepoints wholly into the type-name alphabet, five of which spell a real type: `ıNT` (U+0131), `ſTRING` (U+017F), `flOAT` (U+FB02) and `ſtRING`/`stRING` (U+FB05/U+FB06) were all accepted under EXPR. Spec type names are ASCII, so none of those is a spelling variant. This is a second defect rather than part of the fixture's, and no fixture pins it, but it shares the line being edited -- and registering the fold on task parameters would otherwise have extended it to a surface that was accidentally safe by not folding at all. 81 tests across three files, one class per registration site, each covering the four cases of EXPR on/off by canonical/mis-cased spelling. Nine mutations, all caught: un-registering each of the three sites, dropping `mode="before"`, dropping the EXPR gate, reverting to `str.upper()`, neutering the translation table, removing the two isinstance guards, and folding only the first list element. Three findings came from auditing the tests rather than the code, and each added tests that were missing. The `EnvironmentTemplate` registration was covered by nothing: neutering it left the entire model_v0 suite green at 2743 passed, and the ASCII change had silently altered that shipped path in both directions with no test either way. The two isinstance guards in the shared function became load-bearing on a new surface, since registering the fold routed `taskParameterDefinitions` through them for the first time; without them `taskParameterDefinitions:` left empty raises `TypeError` and `type: 3` raises `AttributeError` out of `decode_job_template` instead of `DecodeValidationError`. And because a mis-cased name now resolves to a variant, it reaches validators it never used to: two mis-cased `chunk[int]` parameters now report the one-CHUNK[INT] rule, and `int`/`INT` under one name now reports the duplicate-name rule. One test was renamed after mutation testing showed it did not pin what its name claimed. A template declaring EXPR to a caller that does not allowlist it is rejected on the extension name, before the type name is reached, so it survives removal of the EXPR gate; it now says so and asserts the extension message. The matching openjd-rs change is not yet raised. Both fixtures stay in `proposed/` until both implementations release, since a fixture is promotable only when it passes on both. Design note: SuperDaveDocs docs/conformance-0901/4.3-parameter-insensitive/fix.md Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com>
1 parent 2726bbc commit 546fa3e

4 files changed

Lines changed: 462 additions & 1 deletion

File tree

‎src/openjd/model/v2023_09/_model.py‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1829,6 +1829,13 @@ class StepParameterSpaceDefinition(OpenJDModel_v2023_09):
18291829
reshape_field_to_dict={"taskParameterDefinitions": "name"},
18301830
)
18311831

1832+
# §2 makes task parameter type names case-insensitive under EXPR, the same as
1833+
# job parameter type names. Must run before discriminated-union resolution.
1834+
@field_validator("taskParameterDefinitions", mode="before")
1835+
@classmethod
1836+
def _normalize_parameter_type_case(cls, v: Any, info: ValidationInfo) -> Any:
1837+
return _normalize_parameter_type_case(v, info)
1838+
18321839
@field_validator("taskParameterDefinitions")
18331840
@classmethod
18341841
def _validate_parameters(cls, v: TaskParameterList) -> TaskParameterList:
@@ -3948,11 +3955,18 @@ def _expr_param_gate(value: JobParameterType, info: ValidationInfo) -> JobParame
39483955
return value
39493956

39503957

3958+
# Parameter type names are ASCII (§2). str.upper() is Unicode-aware and folds
3959+
# U+0131 to 'I', which would make 'ıNT' a spelling of 'INT'.
3960+
_ASCII_UPPERCASE = str.maketrans("abcdefghijklmnopqrstuvwxyz", "ABCDEFGHIJKLMNOPQRSTUVWXYZ")
3961+
3962+
39513963
def _normalize_parameter_type_case(value: Any, info: ValidationInfo) -> Any:
39523964
"""Uppercase the ``type`` discriminator of each parameter definition when
39533965
the EXPR extension is enabled (RFC 0007 makes parameter type names
39543966
case-insensitive, e.g. ``int`` == ``INT``, ``list[int]`` == ``LIST[INT]``).
39553967
Runs before discriminated-union resolution.
3968+
3969+
Applies to job parameter and task parameter type names alike, per §2.
39563970
"""
39573971
context = cast(Optional[ModelParsingContext], info.context)
39583972
if not (context and "EXPR" in context.extensions):
@@ -3962,7 +3976,7 @@ def _normalize_parameter_type_case(value: Any, info: ValidationInfo) -> Any:
39623976
normalized: list[Any] = []
39633977
for item in value:
39643978
if isinstance(item, dict) and isinstance(item.get("type"), str):
3965-
item = {**item, "type": item["type"].upper()}
3979+
item = {**item, "type": item["type"].translate(_ASCII_UPPERCASE)}
39663980
normalized.append(item)
39673981
return normalized
39683982

‎test/openjd/model_v0/v2023_09/test_environment_template.py‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
import pytest
66
from pydantic import ValidationError
77

8+
from openjd.model import DecodeValidationError, decode_environment_template
89
from openjd.model._parse import _parse_model
910
from openjd.model.v2023_09 import EnvironmentTemplate
1011

@@ -256,3 +257,72 @@ def test_parse_fails(self, data: dict[str, Any], expected_num_errors: int) -> No
256257

257258
# THEN
258259
assert len(excinfo.value.errors()) == expected_num_errors
260+
261+
262+
class TestEnvironmentTemplateParameterTypeNameCase:
263+
"""Template Schemas §2: job parameter type names are case-sensitive in base
264+
2023-09 and case-insensitive when the EXPR extension is enabled. An
265+
environment template's ``parameterDefinitions`` carries the same
266+
``JobParameterDefinition`` union as a job template's, so the same rule applies.
267+
268+
``EnvironmentTemplate`` is the third registration site of the shared fold, and
269+
it is the one an audit found unpinned: neutering it left the whole model_v0
270+
suite green. The job template counterpart is
271+
``test_list_parameters.py::TestJobParameterTypeNameCase`` and the task
272+
parameter one is ``test_parameter_space.py::TestTaskParameterTypeNameCase``.
273+
"""
274+
275+
@staticmethod
276+
def _tmpl(type_name: str, *, extensions: tuple[str, ...] = ("EXPR",)) -> dict[str, Any]:
277+
template: dict[str, Any] = {
278+
"specificationVersion": "environment-2023-09",
279+
"parameterDefinitions": [{"name": "P", "type": type_name}],
280+
"environment": ENVIRONMENT,
281+
}
282+
if extensions:
283+
template["extensions"] = list(extensions)
284+
return template
285+
286+
@staticmethod
287+
def _decode(template: dict[str, Any]) -> None:
288+
decode_environment_template(template=template, supported_extensions=["EXPR"])
289+
290+
# (canonical spelling, a mis-cased spelling)
291+
TYPES: tuple = (
292+
pytest.param("STRING", "string", id="string"),
293+
pytest.param("INT", "iNt", id="int"),
294+
pytest.param("PATH", "pAtH", id="path"),
295+
)
296+
297+
@pytest.mark.parametrize("canonical, miscased", TYPES)
298+
def test_no_expr_canonical_case_accepted(self, canonical: str, miscased: str) -> None:
299+
# Case 1 of 4.
300+
self._decode(self._tmpl(canonical, extensions=()))
301+
302+
@pytest.mark.parametrize("canonical, miscased", TYPES)
303+
def test_no_expr_miscased_rejected(self, canonical: str, miscased: str) -> None:
304+
# Case 2 of 4. Fails if the fold is registered without its EXPR gate.
305+
with pytest.raises(DecodeValidationError) as excinfo:
306+
self._decode(self._tmpl(miscased, extensions=()))
307+
message = str(excinfo.value)
308+
assert "parameterDefinitions[0]" in message, message
309+
assert f"'{miscased}'" in message, message
310+
311+
@pytest.mark.parametrize("canonical, miscased", TYPES)
312+
def test_with_expr_canonical_case_accepted(self, canonical: str, miscased: str) -> None:
313+
# Case 3 of 4.
314+
self._decode(self._tmpl(canonical))
315+
316+
@pytest.mark.parametrize("canonical, miscased", TYPES)
317+
def test_with_expr_miscased_accepted(self, canonical: str, miscased: str) -> None:
318+
# Case 4 of 4. Fails if the fold is not registered on this model at all,
319+
# which is the state an audit found untested.
320+
self._decode(self._tmpl(miscased))
321+
322+
@pytest.mark.parametrize("type_name", ("\u0131NT", "\u017fTRING", "\ufb02OAT"))
323+
def test_non_ascii_lookalike_rejected_with_expr(self, type_name: str) -> None:
324+
# str.upper() folds U+0131 to 'I', U+017F to 'S' and U+FB02 (fl) to 'FL',
325+
# so a Unicode-aware fold would read these as INT, STRING and FLOAT.
326+
with pytest.raises(DecodeValidationError) as excinfo:
327+
self._decode(self._tmpl(type_name))
328+
assert f"'{type_name}'" in str(excinfo.value), str(excinfo.value)

‎test/openjd/model_v0/v2023_09/test_list_parameters.py‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -109,3 +109,82 @@ def test_requires_expr(self):
109109
def test_rejects(self, param):
110110
with pytest.raises(DecodeValidationError):
111111
_decode(_tmpl(param))
112+
113+
114+
class TestJobParameterTypeNameCase:
115+
"""Template Schemas §2: job parameter type names are case-sensitive in base
116+
2023-09 and case-insensitive when the EXPR extension is enabled.
117+
118+
``TestListValid.test_case_insensitive_type`` covers one of the four
119+
extension-by-spelling combinations. This covers all four, on both an
120+
EXPR-only type and a base type, and pins that the shared fold is ASCII.
121+
122+
The task parameter counterpart is
123+
``test_parameter_space.py::TestTaskParameterTypeNameCase``.
124+
"""
125+
126+
# (canonical spelling, a mis-cased spelling, a valid default)
127+
TYPES: tuple = (
128+
("STRING", "string", "a"), # base type, available without EXPR
129+
("INT", "iNt", 1), # base type
130+
("LIST[INT]", "list[int]", [1, 2]), # EXPR-only type
131+
("LIST[LIST[INT]]", "List[List[Int]]", [[1], [2]]), # EXPR-only, nested brackets
132+
)
133+
134+
@staticmethod
135+
def _param(type_name, default):
136+
return {"name": "P", "type": type_name, "default": default}
137+
138+
@pytest.mark.parametrize("canonical, miscased, default", TYPES)
139+
def test_no_expr_canonical_case(self, canonical, miscased, default):
140+
# Case 1 of 4. Without EXPR the spec spelling is the only one accepted.
141+
# A base type is accepted; an EXPR-only type is rejected for needing EXPR,
142+
# which is a different rejection from the casing one in case 2.
143+
template = _tmpl(self._param(canonical, default), extensions=())
144+
if canonical.startswith("LIST["):
145+
with pytest.raises(DecodeValidationError, match="requires the EXPR extension"):
146+
_decode(template)
147+
else:
148+
_decode(template)
149+
150+
@pytest.mark.parametrize("canonical, miscased, default", TYPES)
151+
def test_no_expr_miscased_rejected(self, canonical, miscased, default):
152+
# Case 2 of 4. Without EXPR, type names are case-sensitive.
153+
with pytest.raises(DecodeValidationError) as excinfo:
154+
_decode(_tmpl(self._param(miscased, default), extensions=()))
155+
message = str(excinfo.value)
156+
assert "parameterDefinitions[0]" in message, message
157+
assert f"'{miscased}'" in message, message
158+
# Rejected for the casing, not for the extension. An EXPR-only type spelled
159+
# correctly would say "requires the EXPR extension" instead.
160+
assert "requires the EXPR extension" not in message, message
161+
162+
@pytest.mark.parametrize("canonical, miscased, default", TYPES)
163+
def test_with_expr_canonical_case_accepted(self, canonical, miscased, default):
164+
# Case 3 of 4. Enabling EXPR must not break the spec spelling.
165+
_decode(_tmpl(self._param(canonical, default)))
166+
167+
@pytest.mark.parametrize("canonical, miscased, default", TYPES)
168+
def test_with_expr_miscased_accepted(self, canonical, miscased, default):
169+
# Case 4 of 4.
170+
_decode(_tmpl(self._param(miscased, default)))
171+
172+
# ── The fold is ASCII ──
173+
174+
# str.upper() folds each of these wholly into the type-name alphabet: U+0131
175+
# dotless i to 'I', U+017F long s to 'S', U+FB02 ligature fl to 'FL', U+FB06
176+
# ligature st to 'ST'. A Unicode-aware fold reads them as real type names.
177+
LOOKALIKES = ("\u0131NT", "\u017fTRING", "\ufb02OAT", "\ufb06RING")
178+
179+
@pytest.mark.parametrize("type_name", LOOKALIKES)
180+
def test_non_ascii_lookalike_rejected_with_expr(self, type_name):
181+
with pytest.raises(DecodeValidationError) as excinfo:
182+
_decode(_tmpl(self._param(type_name, None)))
183+
assert f"'{type_name}'" in str(excinfo.value), str(excinfo.value)
184+
185+
@pytest.mark.parametrize("type_name", LOOKALIKES)
186+
def test_non_ascii_lookalike_rejected_without_expr(self, type_name):
187+
# Inert against this change by design: without EXPR no fold runs. Present
188+
# so the pair covers both extension states.
189+
with pytest.raises(DecodeValidationError):
190+
_decode(_tmpl(self._param(type_name, None), extensions=()))

0 commit comments

Comments
 (0)