fix(xml,xml-linq): reject code points outside XML's Char production (#2349)

Split from #2085, which repaired the NUL truncation and left this policy open.
Measured: 28 of the 29 non-Char bytes in 0x00-0x1F were EMITTED RAW at every writer
content door and every Xml.Linq direct door, so the emitted document was not
well-formed XML.

THE TICKET RECORDED FIVE PRICED OPTIONS AND CALLED IT A USER DECISION. The blocker
was that /rv was absent. The reference collapses the table to one:

    if (_checkCharacters) throw XmlConvert.CreateInvalidCharException((char)ch, '\0');
    else { ... entitize or write raw ... }   -- XmlEncodedRawTextWriter.cs:1635-1653

with CheckCharacters defaulting to true (XmlWriterSettings.cs:513). .NET rejects by
default and the flag is what turns that off. That is option B, DERIVED rather than
chosen, and SA-5 covers it.

BOTH OF THE TICKET'S PRICING COMPLICATIONS ARE DISSOLVED, NOT ACCEPTED.

  1. It priced enforcement on XmlConvert::VerifyXmlChars, which iterates `char` and
     so checks BYTES -- it accepts U+FFFE, U+FFFF and a lone surrogate encoded in
     UTF-8, all genuinely outside Char, so "enforce with the validator we already
     ship" would have bought C0 controls only. Ticket #2354, EARLIER THE SAME DAY,
     moved a code-point decoder into Core.Base, and both modules/xml and
     modules/xml-linq already depend on it. The correct check now costs one call and
     no new component edge; the graph stays at 41 modules / 93 edges.

  2. It priced option B as "not a same-shaped change on both sides", because
     XNode::SerializeTo takes no settings and SaveOptions has no such value. .NET'S
     DO NOT EITHER: XNode.GetXmlWriterSettings constructs a DEFAULT
     XmlWriterSettings and touches only Indent and NamespaceHandling
     (XNode.cs:681-687), inheriting CheckCharacters = true. So the Linq side needs no
     settings channel, no new SaveOptions value and no ambient default -- it checks
     unconditionally, and the two door families agree BY CONSTRUCTION rather than by
     coordination. That was the whole reason this looked like two changes.

TWO EXCEPTION TYPES, DELIBERATELY. NUL keeps XmlException, because that is this
port's own truncation guard (#2085) -- a length boundary at the tinyxml2 const char*
API, not a transcription of a .NET check. Every other non-Char code point raises
ArgumentException, because that is what XmlConvert.CreateInvalidCharException
produces (XmlConvert.cs:1614-1622), carrying .NET's text.

WHAT IS STILL OPEN IS STATED RATHER THAN LEFT IMPLICIT. XmlReaderSettings
::CheckCharacters also defaults true and is also unenforced, and this port's reader
ACCEPTS these characters -- so writing and reading now disagree. Options A, B and D
all leave that asymmetry and only C and E avoid it. .NET has none, because its
reader enforces too; the reader here is tinyxml2, which this port does not drive
character by character, so closing it is not a matter of adding a call.

Both scope pins are UPDATED, not deleted, exactly as the ticket anticipated:
NonNulControlCharacters_StillEmitted_PinnedScopeBoundary and
NonNulControlCharacters_StillEmittedByTheDirectDoor become Fix2349_* on the same
inputs. Two cases replacing two, so the count does not move.

Four mutations, all caught, each at BOTH door families: the check goes byte-wise
again; U+FFFE and U+FFFF become Char; tab/LF/CR stop being Char; the C0 controls
become Char.

Downstream: neither cna nor mobile-eggbert references XmlWriter, XDocument or
XElement -- zero sites in both.

Gate: 17,333 run, 17,333 passed, 0 failed, 0 skipped across 38 executables, GREEN.
Module boundaries valid (41 modules, 93 edges).

docs/Migration-XmlCheckCharacters.md

Author: robertvokac

CLAUDE.md

@@ -0,0 +1,99 @@
+<!-- SPDX-License-Identifier: MIT -->
+<!-- Copyright (c) Robert Vokac and contributors -->
+
+# Migration — the writer doors reject characters outside XML's `Char` production (ticket #2349)
+
+*2026-08-18.* `XmlWriter::WriteString("a\x01b")` and `XText("a\x01b").ToString()` now raise
+`ArgumentException`. They used to emit the byte raw, producing a document that is not well-formed
+XML.
+
+Landed under `docs/StandingApprovals.md` SA-5. No signature, layout or `noexcept` change.
+
+---
+
+## 1. What changed
+
+| Value | Was | Is |
+|---|---|---|
+| `0x01`, `0x08`, `0x0B`, `0x0C`, `0x0E`–`0x1F` in content | emitted raw | `ArgumentException` |
+| `U+FFFE`, `U+FFFF` | emitted raw | `ArgumentException` |
+| a lone surrogate in UTF-8 (`ED A0 80`) | emitted raw | `ArgumentException` |
+| `\t`, `\n`, `\r` | emitted | **unchanged** — they *are* `Char` |
+| any other text, including multi-byte | — | **unchanged, byte for byte** |
+| `NUL` | `XmlException` (#2085) | **unchanged** — §4 |
+
+Measured before: **28 of the 29 non-`Char` bytes in `0x00`–`0x1F`** went through both door
+families.
+
+## 2. The ticket recorded five priced options and called it a user decision
+
+`docs/SystemXmlNamespaceReviewPlan.md` SS22 carries the table: *A reject unconditionally, B honour
+the flag, C keep today's permissiveness, D writer only, E widen the reader too.* The blocker was
+that `/rv` was absent.
+
+**The reference collapses the table to one option.**
+`XmlWriterSettings.CheckCharacters` defaults to `true` (`XmlWriterSettings.cs:513`) and is
+enforced:
+
+```csharp
+if (_checkCharacters) throw XmlConvert.CreateInvalidCharException((char)ch, '\0');
+else { … entitize or write raw … }               // XmlEncodedRawTextWriter.cs:1635-1653
+```
+
+.NET rejects by default; the flag is what turns that off. That is option B, derived rather than
+chosen.
+
+## 3. Both of the ticket's pricing complications are dissolved, not accepted
+
+**(1) "Enforcing with the validator we already ship buys C0 controls only."** True of
+`XmlConvert::VerifyXmlChars`, which iterates `char` and so checks **bytes** — it accepts `U+FFFE`,
+`U+FFFF` and a lone surrogate. Ticket **#2354**, earlier the same day, moved a **code-point**
+decoder into `Core.Base`, and both `modules/xml` and `modules/xml-linq` already depend on it. A
+code-point-correct check now costs one call and **no new component edge**; the graph stays at
+41/93. The ticket priced this when that decoder was not there.
+
+**(2) "Option B is not a same-shaped change on both sides, because the Xml.Linq direct doors
+cannot see the flag."** **.NET's cannot either.** `XNode.GetXmlWriterSettings` constructs a
+*default* `XmlWriterSettings` and touches only `Indent` and `NamespaceHandling`
+(`XNode.cs:681-687`), so it inherits `CheckCharacters = true` and checks exactly as any
+default-settings writer does. The Linq side needs **no** new settings channel, no new
+`SaveOptions` value and no ambient default: it checks unconditionally, and the two door families
+agree **by construction** rather than by coordination.
+
+## 4. Two exception types, deliberately
+
+`NUL` keeps `XmlException`, because that is this port's own truncation guard (#2085) — a length
+boundary at the tinyxml2 `const char*` API, not a transcription of a .NET check. Every other
+non-`Char` code point raises `ArgumentException`, because that is what
+`XmlConvert.CreateInvalidCharException` produces (`XmlConvert.cs:1614-1622`), with .NET's text:
+*"'…', hexadecimal value 0x01, is an invalid character."*
+
+## 5. What is still open, and it is stated rather than left implicit
+
+`XmlReaderSettings::CheckCharacters` also defaults to `true` and is also unenforced. This port's
+reader **accepts** these characters, so writing and reading now disagree — options A, B and D all
+leave that asymmetry and only C and E avoid it. .NET has no such asymmetry, because its reader
+enforces too.
+
+The reader here is tinyxml2, which this port does not drive character by character, so closing it
+is not a matter of adding a call. It is recorded here rather than silently tolerated.
+
+## 6. To migrate
+
+A document containing these characters was never valid XML — the `Char` production excludes them
+and no character reference can carry them either, since a reference must itself match `Char`.
+Strip or replace them before writing.
+
+## 7. Evidence
+
+| Mutation | Caught |
+|---|---|
+| The check is byte-wise again (the shipped `VerifyXmlChars` shape) | ✅ (both door families) |
+| `U+FFFE` and `U+FFFF` become `Char` | ✅ (both) |
+| Tab, LF and CR stop being `Char` | ✅ (4 tests, including pre-existing ones) |
+| The C0 controls become `Char` (the pre-#2349 behaviour) | ✅ (both) |
+
+## 8. Downstream
+
+Neither `cna` nor `mobile-eggbert` references `XmlWriter`, `XDocument` or `XElement` — zero sites
+in both.

docs/Migration-XmlCheckCharacters.md

@@ -6,6 +6,7 @@
 
 #include "System/Xml/XmlException.hpp"
 #include "System/Xml/detail/XmlLexicalSanitizer.hpp"
+#include "System/ArgumentException.hpp"
 
 namespace System::Xml::Linq::detail {
 
@@ -32,12 +33,12 @@ namespace System::Xml::Linq::detail {
      * why `System::Xml::detail::ContainsNul` (the single detector, shared with the writer door)
      * says so in its own doc-comment.
      *
-     * **What this deliberately does NOT decide.** Characters outside `Char` other than NUL
-     * (0x01, 0x0C, 0x0E–0x1F) are a different question: both doors emit them faithfully rather
-     * than losing them, this runtime's own reader accepts them, and both "reject" and "emit" are
-     * implementable — so whether they are rejected is a `XmlWriterSettings::CheckCharacters`
-     * decision tracked separately. A flag can only govern a choice whose branches both exist.
-     * This guard tests for a NUL and nothing else.
+     * **The other non-`Char` characters, settled by #2349.** They were a different question when
+     * this comment was written; the reference settles it. .NET's Xml.Linq save path builds a
+     * DEFAULT `XmlWriterSettings` and touches only `Indent` and `NamespaceHandling`
+     * (`XNode.cs:681-687`), so it inherits `CheckCharacters = true` and checks exactly as any
+     * default-settings writer does. That is why this door needs no settings channel of its own,
+     * and why the two doors agree by construction rather than by coordination.
      *
      * The diagnostic deliberately mirrors `XmlWriter.cpp`'s file-local `ThrowIfContainsNul`, so
      * the two doors of one node kind report the same cause under different member names.
@@ -55,6 +56,19 @@ namespace System::Xml::Linq::detail {
             throw System::Xml::XmlException(std::string(member) + ": the " + what +
                                             " contains a NUL character, which cannot be "
                                             "represented in XML.");
+
+        // #2349, the same check the writer door runs and for the same reason. The exception TYPE
+        // is .NET's ArgumentException, not this door's XmlException: the NUL case above is this
+        // port's own truncation guard (#2085) while this one transcribes
+        // XmlConvert.CreateInvalidCharException (XmlConvert.cs:1614-1622).
+        const std::size_t bad = System::Xml::detail::FindNonCharCodePoint(text);
+        if (bad != std::string::npos) {
+            std::uint32_t cp = 0;
+            std::size_t   len = 0;
+            if (!System::detail::TryDecodeUtf8Scalar(text, bad, cp, len))
+                cp = static_cast<unsigned char>(text[bad]);
+            throw System::ArgumentException(System::Xml::detail::InvalidCharacterMessage(cp));
+        }
     }
 
 } // namespace System::Xml::Linq::detail

modules/xml-linq/include/System/Xml/Linq/detail/XLinqSerializationGuards.hpp

@@ -52,6 +52,7 @@
 #include <string>
 
 #include "System/Xml/Linq/SaveOptions.hpp"
+#include "System/ArgumentException.hpp"
 #include "System/Xml/Linq/XAttribute.hpp"
 #include "System/Xml/Linq/XCData.hpp"
 #include "System/Xml/Linq/XComment.hpp"
@@ -334,20 +335,32 @@ TEST(XLinqNulRejectionTests, EmptyValuesAreStillAccepted) {
     EXPECT_EQ(direct(XComment("")), "<!---->");
 }
 
-TEST(XLinqNulRejectionTests, NonNulControlCharacters_StillEmittedByTheDirectDoor) {
-    // SCOPE BOUNDARY, and the mirror of the pin #2085 left at the writer door. 0x01, 0x0C and
-    // 0x1F are outside the XML `Char` production but are EMITTED faithfully rather than lost,
-    // and this runtime's own reader accepts them -- so both "reject" and "emit" are implementable
-    // and choosing between them is the XmlWriterSettings::CheckCharacters decision, tracked
-    // separately. #2201 deliberately does NOT make it. If a later ticket decides to reject them,
-    // it updates this pin rather than deleting it.
-    for (const char* v : {"a\x01" "b", "a\x0c" "b", "a\x1f" "b"}) {
-        std::string value(v);
-        EXPECT_NO_THROW((void)direct(XText(value))) << "0x" << static_cast<int>(value[1]);
-        EXPECT_EQ(direct(XText(value)), value);
-        EXPECT_NO_THROW((void)direct(XComment(value)));
-        EXPECT_NO_THROW((void)XAttribute(XName("a"), value).ToString());
+// FLIPPED by #2349 (2026-08-18), the mirror of the writer door's pin flipping in the same change.
+// #2201 deliberately did not make the CheckCharacters decision; the reference makes it.
+//
+// AND IT DISSOLVES THE REASON THIS DOOR WAS THOUGHT TO BE DIFFERENT. #2349 priced option B as
+// "not a same-shaped change on both sides", because XNode::SerializeTo takes no settings.
+// .NET's does not either: XNode.GetXmlWriterSettings builds a DEFAULT XmlWriterSettings and
+// touches only Indent and NamespaceHandling (XNode.cs:681-687), inheriting CheckCharacters = true.
+// So this door needs no settings channel of its own, and the two agree by construction.
+TEST(XLinqNulRejectionTests, Fix2349_NonCharCodePointsAreRejectedAtTheDirectDoorToo) {
+    for (const char* v : {"a\x01" "b", "a\x0c" "b", "a\x1f" "b", "a\x0b" "b"}) {
+        const std::string value(v);
+        EXPECT_THROW((void)direct(XText(value)), System::ArgumentException)
+            << "0x" << static_cast<int>(value[1]);
+        EXPECT_THROW((void)direct(XComment(value)), System::ArgumentException);
+        EXPECT_THROW((void)XAttribute(XName("a"), value).ToString(), System::ArgumentException);
+    }
+
+    // Code points, not bytes: U+FFFE, U+FFFF and a lone surrogate are outside Char too, and a
+    // byte-wise check accepts all three.
+    for (const char* utf8 : {"\xEF\xBF\xBE", "\xEF\xBF\xBF", "\xED\xA0\x80"}) {
+        EXPECT_THROW((void)direct(XText(std::string(utf8))), System::ArgumentException) << utf8;
     }
+
+    // Tab, LF and CR are Char and still pass, and so does ordinary multi-byte text.
+    EXPECT_NO_THROW((void)direct(XText(std::string("a\tb\nc\rd"))));
+    EXPECT_EQ(direct(XText(std::string("h\xC3\xA9llo"))), "h\xC3\xA9llo");
 }
 
 TEST(XLinqNulRejectionTests, ConstructionAndMutationStillAcceptANul) {

modules/xml-linq/tests/System/Xml/Linq/XLinqNulRejectionTests.cpp

@@ -3,6 +3,9 @@
 // Portions based on .NET runtime API (MIT License, Copyright .NET Foundation and Contributors)
 #pragma once
 #include <string>
+#include "System/detail/Utf8Scalar.hpp"
+#include <cstdint>
+#include <cstdio>
 
 namespace System::Xml::detail {
 
@@ -166,14 +169,75 @@ namespace System::Xml::detail {
      * document; "write it in full" is not an implementable branch. This runtime's own parser
      * agrees, rejecting an embedded NUL in text with `XML_ERROR_PARSING_TEXT`.
      *
-     * Other characters outside `Char` (0x01, 0x0C, ...) are a **different** question and are
-     * deliberately not covered here: they are emitted faithfully rather than lost, this
-     * runtime's own reader accepts them, and both "reject" and "emit" are implementable -- so
-     * whether they are rejected is a `XmlWriterSettings::CheckCharacters` decision, tracked
-     * separately. A flag can only govern a choice whose branches both exist.
+     * Other characters outside `Char` (0x01, 0x0C, ...) were a **different** question, settled by
+     * ticket #2349 -- see FindNonCharCodePoint below.
      */
     [[nodiscard]] inline bool ContainsNul(const std::string& text) {
         return text.find('\0') != std::string::npos;
     }
 
+    /**
+     * @brief Finds the first code point in @p text outside the XML 1.0 `Char` production.
+     *
+     * Ticket #2349 (2026-08-18), split from #2085. Characters outside `Char` other than NUL --
+     * `0x01`, `0x0C`, `0x0E`-`0x1F`, `U+FFFE`, `U+FFFF` -- were **emitted raw** at every writer
+     * content door and at every Xml.Linq direct door, so the emitted document was not
+     * well-formed XML. Measured: 28 of the 29 non-`Char` bytes in `0x00`-`0x1F` went through.
+     *
+     * **THE TICKET RECORDED THIS AS A USER DECISION WITH FIVE PRICED OPTIONS, AND THE REFERENCE
+     * COLLAPSES THEM TO ONE.** `XmlWriterSettings.CheckCharacters` defaults to `true`
+     * (`XmlWriterSettings.cs:513`) and is enforced: `XmlEncodedRawTextWriter.InvalidXmlChar`
+     * throws `XmlConvert.CreateInvalidCharException` when it is set and entitizes when it is not
+     * (`:1630-1654`). So .NET rejects by default, and the flag is what turns that off.
+     *
+     * **Both of the ticket's pricing complications are dissolved rather than accepted.**
+     *
+     *   1. It priced enforcement on `XmlConvert::VerifyXmlChars`, which iterates `char` and so
+     *      checks **bytes**: it accepts `U+FFFE`, `U+FFFF` and a lone surrogate encoded in UTF-8.
+     *      Ticket **#2354** moved a code-point decoder into `Core.Base` earlier the same day, and
+     *      both `modules/xml` and `modules/xml-linq` already depend on it -- so a code-point
+     *      correct check costs one call and no new component edge. The ticket priced this when
+     *      that decoder was not there.
+     *   2. It priced option B as "not a same-shaped change on both sides", because the Xml.Linq
+     *      direct doors take no settings. **.NET's do not either**: `XNode.GetXmlWriterSettings`
+     *      constructs a default `XmlWriterSettings` and touches only `Indent` and
+     *      `NamespaceHandling` (`XNode.cs:681-687`), inheriting `CheckCharacters = true`. So the
+     *      Linq side needs no new settings channel; it checks, exactly as a default-settings
+     *      writer does, and the two doors agree by construction.
+     *
+     * An **ill-formed UTF-8 sequence** is rejected too, and that is not an extension: .NET works
+     * in UTF-16 and a lone surrogate is outside `Char` there as surely as it is here.
+     *
+     * @return The offending code point's byte offset, or `std::string::npos` when every code
+     *         point is a `Char`.
+     */
+    [[nodiscard]] inline std::size_t FindNonCharCodePoint(const std::string& text) {
+        std::size_t i = 0;
+        while (i < text.size()) {
+            std::uint32_t cp  = 0;
+            std::size_t   len = 0;
+            if (!System::detail::TryDecodeUtf8Scalar(text, i, cp, len)) return i;
+            // Char ::= #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF].
+            // The decoder has already refused a surrogate and anything above U+10FFFF, so what is
+            // left to exclude is the C0 controls other than tab/LF/CR, and the two non-characters
+            // at the end of the BMP.
+            const bool isChar = cp == 0x09 || cp == 0x0A || cp == 0x0D ||
+                                (cp >= 0x20 && cp <= 0xD7FF) ||
+                                (cp >= 0xE000 && cp <= 0xFFFD) ||
+                                (cp >= 0x10000 && cp <= 0x10FFFF);
+            if (!isChar) return i;
+            i += len;
+        }
+        return std::string::npos;
+    }
+
+    /** @brief `.NET`'s `SR.Xml_InvalidCharacter`, formatted for @p codePoint. */
+    [[nodiscard]] inline std::string InvalidCharacterMessage(std::uint32_t codePoint) {
+        char hex[16] = {};
+        std::snprintf(hex, sizeof(hex), "0x%02X", codePoint);
+        std::string rendered;
+        if (codePoint >= 0x20 && codePoint < 0x7F) rendered.push_back(static_cast<char>(codePoint));
+        return "'" + rendered + "', hexadecimal value " + hex + ", is an invalid character.";
+    }
+
 } // namespace System::Xml::detail

modules/xml/include/System/Xml/detail/XmlLexicalSanitizer.hpp

@@ -53,6 +53,22 @@ static void ThrowIfContainsNul(const std::string& text, const char* member, cons
     if (detail::ContainsNul(text))
         throw XmlException(std::string("XmlWriter::") + member + ": the " + what +
                            " contains a NUL character, which cannot be represented in XML.");
+
+    // Ticket #2349. The other characters outside the XML 1.0 Char production were emitted RAW,
+    // so the emitted document was not well-formed XML. .NET rejects them when
+    // XmlWriterSettings.CheckCharacters is set, which it is by default
+    // (XmlWriterSettings.cs:513; XmlEncodedRawTextWriter.cs:1630-1654). The exception TYPE is
+    // .NET's ArgumentException rather than this door's XmlException, because that is what
+    // XmlConvert.CreateInvalidCharException produces (XmlConvert.cs:1614-1622) -- the NUL case
+    // above keeps XmlException because it is this port's own truncation guard (#2085), not a
+    // transcription of a .NET check.
+    const std::size_t bad = detail::FindNonCharCodePoint(text);
+    if (bad != std::string::npos) {
+        std::uint32_t cp = 0;
+        std::size_t   len = 0;
+        if (!System::detail::TryDecodeUtf8Scalar(text, bad, cp, len)) cp = static_cast<unsigned char>(text[bad]);
+        throw System::ArgumentException(detail::InvalidCharacterMessage(cp));
+    }
 }
 
 static void ThrowIfClosed(const XmlWriterState* state, const char* member) {