Skip to content

Commit 22cba0a

Browse files
committed
fix(text-json): JsonArray/JsonObject enumeration is fail-fast (#1889)
Closes two measured defects. An iterator held across a reallocating Add was an ASan-confirmed heap-use-after-free -- a SIGSEGV in a build without a sanitizer (J11). One held across Clear() silently returned a value from DESTROYED STORAGE with no diagnostic in any build (J12), which is the worse of the two precisely because nothing traps. The repository's standard idiom, the one List<T> and BitArray already use, and CLAUDE.md's counter invariant followed exactly: detail::MutationCounter, never a bare integer, because ++ on a signed counter is undefined at INTCS_MAX and an implicit assignment would transplant the source's counter into the destination. NarrowMutationCounter gains no user. JsonArray and JsonObject go 48 -> 56. JsonNode (24) and JsonValue (40) are unchanged, which is what shows the counter went where the enumerators are and nowhere else. Zero consumer sites. THE MODULE EDGE WAS MADE EXPLICIT BY THE VALIDATOR, NOT BY ME. MutationCounter lives in modules/collections and the two headers are PUBLIC, so Text.Json needed Collections.Core as a PUBLIC dependency where it had been private -- and the boundary validator rejected the private declaration outright. A local copy was never an option; CLAUDE.md forbids it in terms. The graph is UNCHANGED at 41/93: the edge existed and only its kind moved, a smaller change than #1814's. Catalogue regenerated; --check passes. Two of my own measurements were wrong and the compiler corrected both. "Zero first-party begin()/end() sites" missed JsonNodeParseDepthTests iterating with `it->second`, so the enumerator needs operator-> (guarded there too). And a shipped #1886 layout static_assert fired at compile time with a runtime one failing beside it; both updated, the two unmoved figures left as #1886 wrote them, and the growth additionally asserted as a relationship. Six mutations, all caught -- but M6 only after the test was strengthened, and the first result is recorded rather than quietly fixed: a begin() that bumps the counter leaves a single range-for working perfectly, because the one enumerator snapshots the version begin() just produced. It is observable only with TWO enumerators over the same unmutated container. SetItem earns its own row for the mirror reason -- it changes no element count, so only the counter can notice it. Gate: 17,589 run, 17,589 passed, 0 failed, 0 skipped across 38 executables (+4). Build directory: build/ only, --parallel 2 throughout.
1 parent 65b8d24 commit 22cba0a

8 files changed

Lines changed: 356 additions & 15 deletions

File tree

CLAUDE.md

Lines changed: 1 addition & 1 deletion
Large diffs are not rendered by default.

docs/ComponentCatalog.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ for maintainers and are not part of the consumer include surface.
4343
| `Numerics` | `modules/numerics` | static | `Buffers`, `Collections.Core`, `Core.Base` |||| `System/Numerics/BigInteger.hpp` |
4444
| `Runtime` | `modules/runtime` | static | `Collections.Core`, `Core.Base` |||| `System/Runtime/AmbiguousImplementationException.hpp` |
4545
| `Text` | `modules/text` | static | `Buffers`, `Core.Base` |||| `System/Text/ASCIIEncoding.hpp` |
46-
| `Text.Json` | `modules/text-json` | static | `Core.Base`, `Text` | `Collections.Core` | `Collections.Core` || `System/Text/Json/JsonCommentHandling.hpp` |
46+
| `Text.Json` | `modules/text-json` | static | `Collections.Core`, `Core.Base`, `Text` | | `Collections.Core` || `System/Text/Json/JsonCommentHandling.hpp` |
4747
| `Text.RegularExpressions` | `modules/text-regular-expressions` | interface | `Core.Base` |||| `System/Text/RegularExpressions/Capture.hpp` |
4848
| `Threading` | `modules/threading` | static | `Core.Base`, `TimeZone` |||| `System/AsyncCallback.hpp` |
4949
| `Threading.Tasks` | `modules/threading-tasks` | static | `Core.Base`, `Threading` |||| `System/IAsyncDisposable.hpp` |
Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
<!-- SPDX-License-Identifier: MIT -->
2+
# Migration — `JsonArray`/`JsonObject` enumerators are fail-fast (#1889)
3+
4+
Ticket **#1889** (SR-AUD-327, CCF-019), landed 2026-08-19 on an explicit approval after being
5+
declined since July.
6+
7+
## The two measured defects
8+
9+
`begin()`/`end()` handed out **raw `std::vector` iterators with no version guard**:
10+
11+
| probe | what it did |
12+
|---|---|
13+
| **J11** | an iterator held across a **reallocating `Add`** was an **ASan-confirmed heap-use-after-free** — a SIGSEGV in a build without a sanitizer |
14+
| **J12** | an iterator held across **`Clear()`** silently returned a value from **destroyed storage, with no diagnostic in any build** |
15+
16+
J12 is the worse of the two: `Clear()` does not reallocate, so nothing traps — the caller simply
17+
gets a plausible wrong answer.
18+
19+
## What changed
20+
21+
The repository's standard fail-fast idiom, the same one `List<T>` and `BitArray` use: each
22+
container holds a `System::Collections::detail::MutationCounter`, the enumerator snapshots a
23+
`detail::MutationVersion`, and a stale **dereference or advance** raises
24+
`InvalidOperationException` through `detail::requireUnmodified`.
25+
26+
**CLAUDE.md's collection invariant is binding and was followed exactly**: the counter must be
27+
`detail::MutationCounter` and never a bare integer — `++` on a signed counter is undefined at
28+
`INTCS_MAX`, and an implicitly declared assignment operator would transplant the *source's* counter
29+
into the destination, leaving an enumerator apparently valid over storage the assignment destroyed.
30+
`detail::NarrowMutationCounter` gains no user.
31+
32+
| type | before | after |
33+
|---|---|---|
34+
| `JsonArray` | 48 | **56** |
35+
| `JsonObject` | 48 | **56** |
36+
| `JsonNode`, `JsonValue` | 24, 40 | **unchanged** |
37+
38+
Exactly one counter per container, and the two non-containers are untouched — which is what shows
39+
it went where the enumerators are and nowhere else. Silent binary break; consumers rebuild.
40+
Measured: **zero** `JsonArray`/`JsonObject` sites in `cna` and `mobile-eggbert`.
41+
42+
## The module edge, made explicit by the validator rather than by me
43+
44+
`MutationCounter` lives in `modules/collections`, and `JsonArray.hpp`/`JsonObject.hpp` are **public**
45+
headers — so `Text.Json` needed `Collections.Core` as a **public** dependency, where it had been
46+
**private**. The module-boundary validator rejected the private declaration outright, which is how
47+
the edge became explicit instead of accidental. A local copy of the counter was not an option:
48+
CLAUDE.md forbids it in terms.
49+
50+
**The graph is unchanged at 41 modules / 93 edges** — the edge already existed; only its *kind*
51+
moved from private to public. That is a smaller change than #1814's, which added one. The generated
52+
catalogue was regenerated and `--check` passes.
53+
54+
## Two corrections to my own measurements, both found by the compiler
55+
56+
1. **"Zero first-party `begin()`/`end()` sites" was wrong.** `JsonNodeParseDepthTests.cpp` iterates
57+
with `it->second`, which the first grep pattern missed. The enumerator therefore needs
58+
`operator->`, which it now has — with the guard running there too, not only on `operator*`.
59+
2. **A shipped `#1886` layout pin fired at compile time**, exactly as it was written to, and a
60+
second runtime one failed. Both are updated; the two figures that did **not** move are left as
61+
#1886 wrote them, and the growth is additionally asserted as a **relationship**
62+
(`48 + sizeof(MutationVersion)`) so a later member cannot hide behind a hand-updated literal.
63+
64+
## Mutation testing
65+
66+
Six mutations, all caught — **but M6 only after the test was strengthened, and the first result is
67+
recorded rather than quietly fixed**:
68+
69+
| # | Mutation | Caught by |
70+
|---|---|---|
71+
| M1 | `SetItem` does not bump | the every-door case |
72+
| M2 | `Clear` does not bump | the J12 case |
73+
| M3 | dereference unguarded | the J11 case |
74+
| M4 | advance unguarded | the J11 case |
75+
| M5 | `JsonObject::Remove` does not bump | the every-door case |
76+
| M6 | **`begin()` itself bumps** | the every-door case, **after repair** |
77+
78+
M6 was **NOT CAUGHT** at first. A `begin()` that bumps the counter leaves a single range-for
79+
working perfectly — the one enumerator snapshots the version `begin()` just produced — so nothing
80+
noticed. It is observable only with **two enumerators over the same unmutated container**, which is
81+
a legitimate thing to hold, and that is what the case now asserts.
82+
83+
`SetItem` is worth its own row for the mirror-image reason: it changes no element **count**, so only
84+
the counter can notice it at all.

modules/text-json/CMakeLists.txt

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,11 @@ sharp_runtime_register_module(
55
NAME Text.Json
66
TARGET sharp_runtime_text_json
77
TYPE STATIC
8-
PUBLIC_DEPENDENCIES Core.Base Text
9-
PRIVATE_DEPENDENCIES Collections.Core
8+
# Collections.Core became a PUBLIC edge with ticket #1889: JsonArray and JsonObject hold a
9+
# System::Collections::detail::MutationCounter in their PUBLIC headers, because CLAUDE.md's
10+
# collection invariant requires exactly that type for a fail-fast enumerator and forbids a
11+
# local copy. The module-boundary validator rejected the private declaration, which is what
12+
# made the edge explicit rather than accidental. Same shape as #1814's Net.Http.Json edge.
13+
PUBLIC_DEPENDENCIES Core.Base Text Collections.Core
1014
TEST_DEPENDENCIES Collections.Core
1115
)

modules/text-json/include/System/Text/Json/Nodes/JsonArray.hpp

Lines changed: 63 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@
33
// Portions based on .NET runtime API (MIT License, Copyright .NET Foundation and Contributors)
44
#pragma once
55
#include <memory>
6+
#include "System/Collections/detail/MutationCounter.hpp"
7+
#include "System/Collections/IEnumerator.hpp"
68
#include <vector>
79
#include "SharpRuntime/SharpRuntimeHelper.hpp"
810
#include "System/ArgumentOutOfRangeException.hpp"
@@ -20,6 +22,13 @@ namespace System::Text::Json::Nodes {
2022
class JsonArray : public JsonNode {
2123
std::vector<std::shared_ptr<JsonNode>> items_;
2224

25+
/// Fail-fast enumeration counter (#1889). CLAUDE.md's collection invariant is binding here:
26+
/// it must be `detail::MutationCounter` and never a bare integer -- `++` on a signed counter
27+
/// is undefined at INTCS_MAX, and the implicitly declared assignment operator would
28+
/// transplant the SOURCE's counter into the destination, leaving an enumerator apparently
29+
/// valid over storage the assignment destroyed. `MutationCounter` solves both.
30+
System::Collections::detail::MutationCounter version_;
31+
2332
public:
2433
explicit JsonArray(JsonNodeOptions options = {}) : JsonNode(options) {}
2534

@@ -64,12 +73,14 @@ namespace System::Text::Json::Nodes {
6473
if (value) value->AssignParent(this);
6574
if (auto& old = items_[static_cast<size_t>(index)]) old->DetachParent();
6675
items_[static_cast<size_t>(index)] = std::move(value);
76+
++version_; // #1889
6777
}
6878

6979
/** @brief Appends @p item to the end of the array. */
7080
void Add(std::shared_ptr<JsonNode> item) {
7181
if (item) item->AssignParent(this);
7282
items_.push_back(std::move(item));
83+
++version_; // #1889
7384
}
7485

7586
/** @brief Inserts @p item at @p index. @throws System::ArgumentOutOfRangeException if out of range. */
@@ -78,6 +89,7 @@ namespace System::Text::Json::Nodes {
7889
throw System::ArgumentOutOfRangeException("index");
7990
if (item) item->AssignParent(this);
8091
items_.insert(items_.begin() + index, std::move(item));
92+
++version_; // #1889
8193
}
8294

8395
/** @brief Removes the item at @p index. @throws System::ArgumentOutOfRangeException if out of range. */
@@ -86,6 +98,7 @@ namespace System::Text::Json::Nodes {
8698
throw System::ArgumentOutOfRangeException("index");
8799
if (auto& item = items_[static_cast<size_t>(index)]) item->DetachParent();
88100
items_.erase(items_.begin() + index);
101+
++version_; // #1889
89102
}
90103

91104
/**
@@ -104,6 +117,7 @@ namespace System::Text::Json::Nodes {
104117
void Clear() {
105118
for (auto& item : items_) if (item) item->DetachParent();
106119
items_.clear();
120+
++version_; // #1889
107121
}
108122

109123
/** @return The index of @p item within the array, or -1 if not found (pointer identity). */
@@ -113,8 +127,55 @@ namespace System::Text::Json::Nodes {
113127
return -1;
114128
}
115129

116-
[[nodiscard]] auto begin() const { return items_.begin(); }
117-
[[nodiscard]] auto end() const { return items_.end(); }
130+
/**
131+
* @brief Fail-fast enumerator over the array's elements.
132+
*
133+
* Ticket **#1889**. `begin()`/`end()` used to hand out **raw `std::vector` iterators with
134+
* no version guard**, and two measured defects followed:
135+
* * an iterator held across a reallocating `Add` was an **ASan-confirmed
136+
* heap-use-after-free** -- a SIGSEGV in a build without a sanitizer (probe case J11);
137+
* * an iterator held across `Clear()` **silently returned a value from destroyed
138+
* storage, with no diagnostic in any build** (J12).
139+
*
140+
* This is the repository's standard fail-fast idiom, the same one `List<T>` and `BitArray`
141+
* use: the container holds a `detail::MutationCounter`, the enumerator snapshots a
142+
* `detail::MutationVersion`, and a stale dereference or advance raises
143+
* `InvalidOperationException` rather than reading freed memory.
144+
*/
145+
class Enumerator {
146+
const JsonArray* owner_;
147+
std::size_t index_;
148+
System::Collections::detail::MutationVersion version_;
149+
150+
void requireCurrent() const {
151+
System::Collections::detail::requireUnmodified(version_ == owner_->version_);
152+
}
153+
154+
public:
155+
Enumerator(const JsonArray* owner, std::size_t index)
156+
: owner_(owner), index_(index), version_(owner->version_) {}
157+
158+
[[nodiscard]] const std::shared_ptr<JsonNode>& operator*() const {
159+
requireCurrent();
160+
return owner_->items_[index_];
161+
}
162+
/// Provided because callers iterate with `it->`; a forward iterator must offer it,
163+
/// and the guard runs here too rather than only on `operator*`.
164+
[[nodiscard]] auto operator->() const { return &**this; }
165+
166+
Enumerator& operator++() {
167+
requireCurrent();
168+
++index_;
169+
return *this;
170+
}
171+
[[nodiscard]] bool operator==(const Enumerator& other) const {
172+
return owner_ == other.owner_ && index_ == other.index_;
173+
}
174+
[[nodiscard]] bool operator!=(const Enumerator& other) const { return !(*this == other); }
175+
};
176+
177+
[[nodiscard]] Enumerator begin() const { return Enumerator(this, 0); }
178+
[[nodiscard]] Enumerator end() const { return Enumerator(this, items_.size()); }
118179

119180
[[nodiscard]] JsonValueKind GetValueKind() const override { return JsonValueKind::Array; }
120181

modules/text-json/include/System/Text/Json/Nodes/JsonObject.hpp

Lines changed: 52 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@
33
// Portions based on .NET runtime API (MIT License, Copyright .NET Foundation and Contributors)
44
#pragma once
55
#include <memory>
6+
#include "System/Collections/detail/MutationCounter.hpp"
7+
#include "System/Collections/IEnumerator.hpp"
68
#include <string>
79
#include <utility>
810
#include <vector>
@@ -29,6 +31,10 @@ namespace System::Text::Json::Nodes {
2931
class JsonObject : public JsonNode {
3032
std::vector<std::pair<std::string, std::shared_ptr<JsonNode>>> properties_;
3133

34+
/// Fail-fast enumeration counter (#1889); see JsonArray for why it must be
35+
/// `detail::MutationCounter` rather than a bare integer.
36+
System::Collections::detail::MutationCounter version_;
37+
3238
// Verified against JsonObject.IDictionary.cs's CreateDictionary(): real .NET's backing
3339
// dictionary uses StringComparer.OrdinalIgnoreCase (when PropertyNameCaseInsensitive is
3440
// set) as the comparer for every operation -- lookup, ContainsKey, the duplicate-key
@@ -94,6 +100,7 @@ namespace System::Text::Json::Nodes {
94100
throw System::ArgumentException("An item with the same key has already been added.", "propertyName");
95101
if (value) value->AssignParent(this);
96102
properties_.emplace_back(propertyName, std::move(value));
103+
++version_; // #1889
97104
}
98105

99106
/** @brief Removes the property named @p propertyName. @return true if it was present. */
@@ -102,6 +109,7 @@ namespace System::Text::Json::Nodes {
102109
if (idx < 0) return false;
103110
if (auto& value = properties_[static_cast<size_t>(idx)].second) value->DetachParent();
104111
properties_.erase(properties_.begin() + idx);
112+
++version_; // #1889
105113
return true;
106114
}
107115

@@ -139,17 +147,59 @@ namespace System::Text::Json::Nodes {
139147
} else {
140148
if (value) value->AssignParent(this);
141149
properties_.emplace_back(propertyName, std::move(value));
150+
++version_; // #1889
142151
}
143152
}
144153

145154
/** @brief Removes all properties from the object. */
146155
void Clear() {
147156
for (auto& [name, value] : properties_) if (value) value->DetachParent();
148157
properties_.clear();
158+
++version_; // #1889
149159
}
150160

151-
[[nodiscard]] auto begin() const { return properties_.begin(); }
152-
[[nodiscard]] auto end() const { return properties_.end(); }
161+
/**
162+
* @brief Fail-fast enumerator over the object's properties (#1889).
163+
*
164+
* The same repository idiom as `JsonArray::Enumerator`, for the same two measured
165+
* defects (J11, J12): raw `std::vector` iterators had no version guard, so one held
166+
* across a reallocating `Add` was an ASan-confirmed heap-use-after-free and one held
167+
* across `Clear()` silently read destroyed storage with no diagnostic in any build.
168+
*/
169+
class Enumerator {
170+
const JsonObject* owner_;
171+
std::size_t index_;
172+
System::Collections::detail::MutationVersion version_;
173+
174+
void requireCurrent() const {
175+
System::Collections::detail::requireUnmodified(version_ == owner_->version_);
176+
}
177+
178+
public:
179+
Enumerator(const JsonObject* owner, std::size_t index)
180+
: owner_(owner), index_(index), version_(owner->version_) {}
181+
182+
[[nodiscard]] const std::pair<std::string, std::shared_ptr<JsonNode>>& operator*() const {
183+
requireCurrent();
184+
return owner_->properties_[index_];
185+
}
186+
/// Provided because callers iterate with `it->`; a forward iterator must offer it,
187+
/// and the guard runs here too rather than only on `operator*`.
188+
[[nodiscard]] auto operator->() const { return &**this; }
189+
190+
Enumerator& operator++() {
191+
requireCurrent();
192+
++index_;
193+
return *this;
194+
}
195+
[[nodiscard]] bool operator==(const Enumerator& other) const {
196+
return owner_ == other.owner_ && index_ == other.index_;
197+
}
198+
[[nodiscard]] bool operator!=(const Enumerator& other) const { return !(*this == other); }
199+
};
200+
201+
[[nodiscard]] Enumerator begin() const { return Enumerator(this, 0); }
202+
[[nodiscard]] Enumerator end() const { return Enumerator(this, properties_.size()); }
153203

154204
[[nodiscard]] JsonValueKind GetValueKind() const override { return JsonValueKind::Object; }
155205

0 commit comments

Comments
 (0)