Skip to content

Commit b0d20ac

Browse files
committed
fix: enforce deterministic JSON depth limits across runtimes
1 parent 7a7031e commit b0d20ac

7 files changed

Lines changed: 70 additions & 3 deletions

File tree

Makefile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
.SHELLFLAGS := -eu -c
12
PYTHON ?= python3
23
VENV ?= .venv
34
BIN := $(VENV)/bin

README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# MemoryFlow Lab
22

3+
[Engineering notes](docs/engineering-notes.md) · [Core model](src/memoryflow/simulator.py) · [Reproduction protocol](#reproduce)
4+
35
[![CI](https://github.com/KIM3310/memoryflow-lab/actions/workflows/ci.yml/badge.svg)](https://github.com/KIM3310/memoryflow-lab/actions/workflows/ci.yml)
46
[![Live results](https://img.shields.io/badge/live-results-0d6447)](https://kim3310.github.io/memoryflow-lab/)
57
[![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-3776ab)](https://www.python.org/)

docs/engineering-notes.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# A stable JSON input boundary across Python versions
2+
3+
The input loader relied on the JSON decoder raising `RecursionError` for excessively nested documents. On this machine's Python 3.12 runtime, the 2,000-level regression input was accepted by the decoder, making both CLI and API behavior depend on the interpreter.
4+
5+
The loader now performs a bounded, non-recursive nesting scan before decoding, with an explicit maximum of 64 object/array levels. Quoted braces and escaped quotes do not count as containers. Existing byte-size, duplicate-key, finite-number, Unicode, and scenario-schema checks remain in place.
6+
7+
`tests/test_json_depth_contract.py` checks the exact boundary, quoted syntax, and rejection before invoking the decoder. `make verify` also regenerates and compares simulation evidence, checks measurement artifacts, and builds and validates the source distribution and wheel. The Makefile now fails immediately when either deterministic comparison fails.
8+
9+
Committed simulation evidence was regenerated from the changed source. The pre-existing MPS measurement files were not remeasured or relabeled; they remain aggregate summaries of the original limited experiments.

evidence/benchmark-summary.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# Reproducible Benchmark Summary
22

33
Scenario-set SHA-256: `6c494ca01417e6b51ee6aec7359b084d57e17c1cca8e8b9b48347cded529e302`
4-
Model-source SHA-256: `b2ff2bfb644c3aca0a1307731b046add648dd46a6b8c5f3b1c44c06bae429a7c`
4+
Model-source SHA-256: `eff89ab1a969bb96a3739ed950b36d55213884d224344b75d46da29b845ff6a4`
55
Generator SHA-256: `dd9d905137eba4496e3865caa8dbc167703c286f165fd95f2a4a38aafb6cbfc8`
66

77
All policy results below use bundled synthetic hardware knobs. They are deterministic first-order estimates, not measurements or product claims.

site/results.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1394,15 +1394,15 @@
13941394
"command": "python -m scripts.build_evidence",
13951395
"generator_path": "scripts/build_evidence.py",
13961396
"generator_sha256": "dd9d905137eba4496e3865caa8dbc167703c286f165fd95f2a4a38aafb6cbfc8",
1397-
"model_source_sha256": "b2ff2bfb644c3aca0a1307731b046add648dd46a6b8c5f3b1c44c06bae429a7c",
1397+
"model_source_sha256": "eff89ab1a969bb96a3739ed950b36d55213884d224344b75d46da29b845ff6a4",
13981398
"model_sources": [
13991399
{
14001400
"path": "src/memoryflow/domain.py",
14011401
"sha256": "2eaecabd38fe7a7c16c9d4eb97eecfd7f9d2930418ed4ebb4594a6bbc7e7b72f"
14021402
},
14031403
{
14041404
"path": "src/memoryflow/io.py",
1405-
"sha256": "5ea2f52a00f46bdbb2a20c09ff1001d8a7f53ad597743ef66450d8d755ab0f55"
1405+
"sha256": "5bb80100a2daa56ea17bab78300829e5ef2825324203d21bd42e1e6aace36571"
14061406
},
14071407
{
14081408
"path": "src/memoryflow/simulator.py",

src/memoryflow/io.py

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616

1717
SCENARIO_SCHEMA_VERSION = "2.0"
1818
MAX_SCENARIO_BYTES = 1_048_576
19+
MAX_JSON_NESTING = 64
1920
T = TypeVar("T", Workload, MemorySystem, PlacementPolicy, ScenarioProvenance)
2021

2122

@@ -101,6 +102,29 @@ def _validate_parsed_scalars(payload: object) -> None:
101102
pending.extend(value)
102103

103104

105+
def _validate_json_nesting(text: str) -> None:
106+
"""Bound container nesting before the interpreter-dependent JSON decoder."""
107+
depth = 0
108+
in_string = False
109+
escaped = False
110+
for char in text:
111+
if in_string:
112+
if escaped:
113+
escaped = False
114+
elif char == "\\":
115+
escaped = True
116+
elif char == '"':
117+
in_string = False
118+
elif char == '"':
119+
in_string = True
120+
elif char in "[{":
121+
depth += 1
122+
if depth > MAX_JSON_NESTING:
123+
raise ValueError("JSON nesting exceeds the parser limit")
124+
elif char in "]}":
125+
depth -= 1
126+
127+
104128
def json_object_from_bytes(
105129
content: bytes, *, maximum_bytes: int = MAX_SCENARIO_BYTES
106130
) -> dict[str, Any]:
@@ -110,6 +134,7 @@ def json_object_from_bytes(
110134
text = content.decode("utf-8")
111135
except UnicodeDecodeError as exc:
112136
raise ValueError("JSON input must use UTF-8 encoding") from exc
137+
_validate_json_nesting(text)
113138
try:
114139
payload = json.loads(
115140
text,

tests/test_json_depth_contract.py

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
import json
2+
from unittest.mock import patch
3+
4+
import pytest
5+
6+
from memoryflow.io import MAX_JSON_NESTING, json_object_from_bytes
7+
8+
9+
def nested_arrays(depth: int) -> bytes:
10+
return b'{"x":' + b"[" * (depth - 1) + b"0" + b"]" * (depth - 1) + b"}"
11+
12+
13+
def test_exact_container_depth_boundary() -> None:
14+
assert "x" in json_object_from_bytes(nested_arrays(MAX_JSON_NESTING))
15+
with pytest.raises(ValueError, match="nesting exceeds"):
16+
json_object_from_bytes(nested_arrays(MAX_JSON_NESTING + 1))
17+
18+
19+
def test_quoted_braces_and_escaped_quotes_do_not_count_as_containers() -> None:
20+
payload = {"text": '[{\\"' * 1000 + "}]" * 1000}
21+
assert json_object_from_bytes(json.dumps(payload).encode()) == payload
22+
23+
24+
def test_excessive_depth_is_rejected_before_json_decode() -> None:
25+
with (
26+
patch("memoryflow.io.json.loads") as decoder,
27+
pytest.raises(ValueError, match="nesting exceeds"),
28+
):
29+
json_object_from_bytes(nested_arrays(2000))
30+
decoder.assert_not_called()

0 commit comments

Comments
 (0)