Skip to content

Commit e7a17b3

Browse files
committed
fix: Bound floatstring range element expansion by the field's char cap
Rendering a <floatstring> range element with format(value, 'f') is unbounded in the element's exponent, which review found has two reachable faces. Unbounded expansion. Decimal construction from a string is not bounded by the context -- Emax/Emin constrain arithmetic results, not construction -- so Decimal('1e999999999') is finite and the is_finite() guard did not stop it. The plain-notation length is exponent + 1, measured: 10**7 gives 10,000,001 characters. Parsing a FLOAT range of ['1e100000000'] on the instantiation target took 0.37s and 266 MiB peak RSS from 11 characters of template text; the 1e999999999 case is 10x that and killed the probe process. It is reachable from an ordinary template, because the template layer accepts the element and keeps it as a TaskParameterStringValue. Silently exceeding the field cap, far lower down. A TaskRangeList element's str member is TaskParameterStringValueAsJob, capped at 1024 characters. Measured: '1E+1022' -- 7 characters, accepted before this branch -- expands to 1025 and so fails that member, and pydantic then falls through to the numeric members, rendering the element as a 1023-digit int. In the other direction '1E-1023' expands to 1025, fails the same way, and renders 0.0, silently losing the value. Both were valid text-rendering elements on mainline, which rendered '1E+1030' as 1E+1030. Bound the length from value.adjusted() and the exponent before materializing anything. The limit is the field's own 1024-character cap rather than a new number, because an expansion past it cannot be carried here as text at all, so there is nothing to gain by producing one. An element over the bound keeps its source text -- what this function already does for anything it cannot normalize, and how such an element rendered before normalization was introduced -- so there is no new rejection path and no new error type. The bound is exact at the boundary: '1E+1021' and '1E-1022' expand to exactly 1024 and are still normalized. After the fix '1e999999999' parses in 0.01ms at 42 MiB peak RSS. Model suite 5521 -> 5527 passed (6 new tests, 24 skipped, 3 xfailed). The two boundary tests fail against 2109e2f, rendering 1000...000 and 0.0. Conformance 3.4.1.1, 3.4.1.2, 3.4--float-parameter and expr1.3.4--float-passthrough all pass. Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com>
1 parent d6d5540 commit e7a17b3

2 files changed

Lines changed: 79 additions & 2 deletions

File tree

‎src/openjd/model/v2023_09/_model.py‎

Lines changed: 32 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1208,7 +1208,10 @@ def __new__(cls, value: str, *, context: ModelParsingContextInterface = ModelPar
12081208
list[Union[Decimal, TaskParameterStringValue]], Field(min_length=1, max_length=1024)
12091209
]
12101210
StringRangeList = Annotated[list[TaskParameterStringValue], Field(min_length=1, max_length=1024)]
1211-
TaskParameterStringValueAsJob = Annotated[str, StringConstraints(min_length=0, max_length=1024)]
1211+
_MAX_TASK_PARAM_VALUE_LEN = 1024
1212+
TaskParameterStringValueAsJob = Annotated[
1213+
str, StringConstraints(min_length=0, max_length=_MAX_TASK_PARAM_VALUE_LEN)
1214+
]
12121215

12131216
TaskRangeList = list[Union[TaskParameterStringValueAsJob, int, float, Decimal]]
12141217

@@ -1246,7 +1249,8 @@ def _normalized_range_element(elem: str, to_int: bool) -> Any:
12461249
"""The number an ``<intstring>``/``<floatstring>`` range element denotes -- an
12471250
``int`` for ``<intstring>``, and for ``<floatstring>`` the plain-notation text
12481251
of the value, since ``str(Decimal)`` cannot render every magnitude without an
1249-
exponent. Returns the element unchanged when it does not denote a number."""
1252+
exponent. Returns the element unchanged when it does not denote a number, or
1253+
when its plain-notation form would not fit this field's character cap."""
12501254
try:
12511255
if to_int:
12521256
return int(elem)
@@ -1255,6 +1259,32 @@ def _normalized_range_element(elem: str, to_int: bool) -> Any:
12551259
# Decimal accepts 'nan'/'inf', which are not <floatstring>s; leave
12561260
# them as text rather than changing how they render.
12571261
return elem
1262+
# Bound the expansion from the exponent before performing it. Decimal
1263+
# construction from a string is unbounded -- the context's Emax/Emin
1264+
# constrain arithmetic results, not construction, so Decimal('1e999999999')
1265+
# is finite and the guard above does not stop it -- and the plain-notation
1266+
# length is linear in the exponent, so 11 characters of template text
1267+
# would otherwise materialize ~10**9 characters.
1268+
#
1269+
# The bound is the character cap this field already enforces, because an
1270+
# expansion past it cannot be carried here as text at all: the value is
1271+
# stored in a TaskRangeList element, whose str member is
1272+
# TaskParameterStringValueAsJob, and pydantic's union falls through to
1273+
# the numeric members instead, rendering the element as an integer, or as
1274+
# inf, or as 0.0. Such an element keeps its source text -- what this
1275+
# function already does for anything it cannot normalize, and how it
1276+
# rendered before normalization was introduced.
1277+
exponent = cast(int, value.as_tuple().exponent) # finite: never 'n'/'N'/'F'
1278+
# Length of the '[-]<integer>.<fraction>' text returned below. Stripping
1279+
# redundant zeros can only shorten it, so this is an upper bound.
1280+
expansion_len = (
1281+
(1 if value.is_signed() else 0)
1282+
+ max(value.adjusted() + 1, 1) # integer digits, at least one
1283+
+ 1 # the decimal point
1284+
+ max(-exponent, 1) # fraction digits, at least one
1285+
)
1286+
if expansion_len > _MAX_TASK_PARAM_VALUE_LEN:
1287+
return elem
12581288
# format(value, 'f') is exact and plain at every magnitude: with no
12591289
# precision in the format spec it neither rounds to getcontext().prec
12601290
# nor switches to exponent notation, so an embedding app that sets a

‎test/openjd/model_v0/v2023_09/test_parameter_space.py‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -513,6 +513,53 @@ def test_floatstring_elements_carry_their_value(self, element: str, expected: st
513513
# THEN the element renders as the number it denotes
514514
assert [str(v) for v in model.range] == [expected]
515515

516+
@pytest.mark.parametrize(
517+
"element",
518+
(
519+
# Decimal's context bounds arithmetic results, not construction from
520+
# a string, so these are finite: 11 characters of template text
521+
# whose plain-notation expansion is ~10**9 characters.
522+
pytest.param("1e999999999", id="huge positive exponent"),
523+
pytest.param("1e-999999999", id="huge negative exponent"),
524+
# One character past the cap in each direction: both expand to 1025.
525+
pytest.param("1E+1022", id="one past the cap, positive exponent"),
526+
pytest.param("1E-1023", id="one past the cap, negative exponent"),
527+
),
528+
)
529+
def test_element_too_long_to_expand_keeps_its_source_text(self, element: str) -> None:
530+
# WHEN the instantiation target parses a <floatstring> whose plain-notation
531+
# form would not fit TaskParameterStringValueAsJob's 1024-character cap
532+
model = _parse_model(
533+
model=RangeListTaskParameterDefinition, obj={"type": "FLOAT", "range": [element]}
534+
)
535+
536+
# THEN it keeps its source text rather than being expanded. Expanding it
537+
# would allocate the whole expansion, and the result could not be carried
538+
# as text here anyway: it fails the str member of TaskRangeList's union,
539+
# and pydantic then falls through to the numeric members, rendering the
540+
# element as an integer, or as inf, or as 0.0.
541+
assert [str(v) for v in model.range] == [element]
542+
543+
@pytest.mark.parametrize(
544+
"element,expected",
545+
(
546+
pytest.param("1E+1021", "1" + "0" * 1021 + ".0", id="positive exponent"),
547+
pytest.param("1E-1022", "0." + "0" * 1021 + "1", id="negative exponent"),
548+
),
549+
)
550+
def test_element_at_the_cap_is_still_normalized(self, element: str, expected: str) -> None:
551+
# GIVEN a <floatstring> whose plain-notation form is exactly 1024
552+
# characters — at the cap, not past it
553+
assert len(expected) == 1024
554+
555+
# WHEN the instantiation target parses it
556+
model = _parse_model(
557+
model=RangeListTaskParameterDefinition, obj={"type": "FLOAT", "range": [element]}
558+
)
559+
560+
# THEN the length bound has not cost it its normalization
561+
assert [str(v) for v in model.range] == [expected]
562+
516563
def test_floatstring_normalization_ignores_the_decimal_context(self) -> None:
517564
# GIVEN an embedding application that has narrowed the process-wide
518565
# decimal context, and a <floatstring> with more significant digits

0 commit comments

Comments
 (0)