Skip to content

Add QUIC_PARAM_CONN_PATH_STATISTICS for per-path network statistics - #82

Merged
masa-koz merged 2 commits into
seera-mainfrom
masa-koz/path-statistics
Aug 22, 2026
Merged

Add QUIC_PARAM_CONN_PATH_STATISTICS for per-path network statistics#82
masa-koz merged 2 commits into
seera-mainfrom
masa-koz/path-statistics

Conversation

@masa-koz

@masa-koz masa-koz commented Aug 22, 2026

Copy link
Copy Markdown

Description

QUIC_PARAM_CONN_NETWORK_STATISTICS only ever reports the first path:

Connection->Paths[0].PathID->CongestionControl.QuicCongestionControlGetNetworkStatistics(
    Connection, &Connection->Paths[0].PathID->CongestionControl, Stats);

On a multipath connection that leaves the other paths unobservable. QUIC_PARAM_CONN_PATH_STATISTICS returns one entry per path instead.

typedef struct QUIC_PATH_STATISTICS {
    uint32_t PathId;
    uint64_t Rtt;                        // Smoothed RTT, microseconds
    uint64_t MinRtt;                     // Zero until the path has an RTT sample
    uint64_t MaxRtt;                     // Zero until the path has an RTT sample
    uint16_t Mtu;
    QUIC_NETWORK_STATISTICS NetworkStatistics;
} QUIC_PATH_STATISTICS;

#define QUIC_PARAM_CONN_PATH_STATISTICS  0x05000026  // QUIC_PATH_STATISTICS[]

The path count is not known ahead of time and changes over the connection's life, so it is retrieved the usual two-step way: a BufferLength of 0 reports the size needed, and on success BufferLength is the number of bytes written, giving the entry count. Paths with no path ID assigned yet — added before the handshake is confirmed — are not reported, having nothing to identify them by and no congestion control to read. Works with or without multipath negotiated; a single-path connection returns one entry.

Two things beyond the field list

PathId is included. Array position is not stable — removing a path moves the ones behind it up — so without an identifier the caller cannot tell which entry belongs to which path across calls. It matches the PathId used by QUIC_PARAM_CONN_PATH_STATUS.

It is not unique, however. QuicConnGetPathForPacket assigns a path ID to a rebound path without clearing it from the path being replaced, so during a rebind two entries carry the same PathId. Both are real paths, and sharing a path ID means sharing its congestion control, so their NetworkStatistics agreeing is correct rather than a defect; the per-path Rtt, MinRtt, MaxRtt and Mtu are what tell them apart. Documented rather than filtered — suppressing one would hide a path that exists from an API whose purpose is to report all of them.

MinRtt / MaxRtt are normalised. QuicPathInitialize sets MinRtt = UINT32_MAX as its "no sample yet" sentinel and leaves MaxRtt at zero. Passing that through would report a minimum RTT of roughly 4295 seconds, so a path with nothing measured reports zero for both. Rtt needs no such treatment: it starts from the configured InitialRttMs.

This differs from QUIC_STATISTICS_V2, which passes the sentinel through as-is.

The network statistics hook now takes a path

Both implementations read the RTT off Connection->Paths[0]:

const QUIC_PATH* Path = &Connection->Paths[0];

Congestion control is per path ID, so every path would have reported path 0's RTT — and, in cubic, path 0's bandwidth, which is derived from it.

Deriving the path from the congestion control's owning path ID does not work, because that back reference is not always current. QuicPathSetActive's non-multipath branch swaps the contents of the two slots:

QUIC_PATH PrevActivePath = Connection->Paths[0];
Connection->Paths[0] = *Path;
*Path = PrevActivePath;

The PathID pointer travels with the contents while the path ID's back reference stays behind, and QuicPathRemove only repairs back references under if (MultipathNegotiated). After a migration without multipath the back reference points at the slot the old path was moved into — and, once that path is removed, at a slot past PathsCount holding stale data.

So the path is a parameter of the hook instead, and no caller depends on the back reference:

  • QuicConnGetNetworkStatistics and the BBR connection event pass &Connection->Paths[0], the expression the hook used internally before, so both keep their behaviour exactly.
  • QuicConnGetPathStatistics passes the path it is reporting on, which also rules out an entry's Rtt disagreeing with its own NetworkStatistics.SmoothedRTT.

BbrTest.cpp and CubicTest.cpp are updated for the signature.

Testing

New Basic/WithFamilyArgs.PathStatistics, registered in MsQuicTests.h, quic_gtest.cpp and winkernel/control.cpp. It queries a one-path connection, brings up a second path, and checks the array grows by exactly one entry; that both entries carry distinct path IDs, a non-zero MTU and RTT, and a MinRtt no greater than MaxRtt; that a buffer one entry short is refused with the required size; and that each entry's Rtt equals its own NetworkStatistics.SmoothedRTT.

That last assertion is what pins the per-path plumbing. Instrumented, the two paths report genuinely different figures:

path[0] id=0 rtt=2168   min=2168 max=2168 mtu=1280 cwnd=12000 nsRtt=2168   bw=5 inflight=0
path[1] id=1 rtt=333000 min=0    max=0    mtu=1248 cwnd=12200 nsRtt=333000 bw=0 inflight=2440

Pointing QuicConnGetPathStatistics back at Paths[0] turns path[1]'s nsRtt into path 0's and fails the test.

Sweep Result
PathStatistics × 3 repeats 6/6 pass
*Multipath*:*Path*:*Migration*:*UnconnectedSocket*:*KeepAlive*:*Statistics* 112 tests pass
*Basic*:*Datagram*:*Receive*:*Recv* 626 tests pass
msquiccoretest *Bbr*:*Cubic* 128 tests pass
cargo test --features preview-api 13 pass

Build is clean. connection.c, cubic.c and bbr.c were run through the CI's clang-tidy 21 in ghcr.io/microsoft/msquic/linux-build-xcomp:ubuntu-26.04-cross, since the local one is too old to reproduce what -CodeCheck sees.

Not covered by a test: the non-multipath migration case above. It is the reason the hook signature changed, but reaching it needs a connection to migrate and then have the old path removed, which the existing path tests do not set up. The two connection-level callers pass the same expression the hook used before, so the change is behaviour-preserving there by construction rather than by test.

Documentation

docs/Settings.md gains the table row and a section covering the two-step sizing, what PathId is for and why it is not unique, the zero-until-sampled RTTs, and the fact that paths without a path ID yet are not reported.

The parameter table had a blank line in the middle, which terminated it early and left the last rows rendering as literal | ... | text. Removed, which also repairs the QUIC_PARAM_CONN_UNCONNECTED_UDP_SOCKET row that had been broken since it was added.

Rust bindings are regenerated; win_bindings.rs cannot be regenerated on Linux, so the identical hunk was applied by hand and diffed against the Linux one. The C# bindings are unchanged, as with previous fork settings.

QUIC_PARAM_CONN_NETWORK_STATISTICS only ever reports Paths[0], which on a
multipath connection leaves every other path unobservable. The new parameter
returns one QUIC_PATH_STATISTICS per path: the path's Rtt, MinRtt, MaxRtt and
Mtu, plus the QUIC_NETWORK_STATISTICS read from that path's own congestion
control.

The path count is not known ahead of time and changes over the connection's
life, so it is retrieved the usual two-step way: a BufferLength of 0 reports
the size needed, and on success BufferLength is the number of bytes written.
Paths with no path ID assigned yet -- added before the handshake is confirmed
-- are not reported, having nothing to identify them by and no congestion
control to read. Works with or without multipath negotiated.

PathId is carried in each entry because array position is not stable: removing
a path moves the ones behind it up, so a caller cannot otherwise tell which
entry belongs to which path across calls. It matches the PathId used by
QUIC_PARAM_CONN_PATH_STATUS.

MinRtt and MaxRtt are reported as zero until the path has an RTT sample.
QuicPathInitialize uses MinRtt = UINT32_MAX as its "no sample yet" sentinel and
leaves MaxRtt at zero; passing that through would read as a minimum RTT of
roughly 4295 seconds. This differs from QUIC_STATISTICS_V2, which passes the
sentinel through as-is.

Both congestion control implementations of the network statistics hook read the
RTT off Connection->Paths[0] rather than the path the instance belongs to, so
every path would have reported path 0's RTT -- and, in cubic, path 0's
bandwidth, which is derived from it. Both now take the path from the path ID
that owns the instance, the way the rest of both files already reach it. Single
path connections are unaffected.

Basic/WithFamilyArgs.PathStatistics brings up a second path and checks the array
grows by one, that the entries carry distinct path IDs and their own figures,
and that a short buffer is refused with the required size. Asserting each
entry's Rtt against its own NetworkStatistics.SmoothedRTT is what pins the
congestion control correction: reverting the cubic change alone fails it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Comment thread src/core/cubic.c Outdated
// Congestion control is per path ID, so the RTT to report is the one of the
// path this instance belongs to, not whichever path happens to be first.
//
const QUIC_PATH* Path = QuicCongestionControlGetPathID(Cc)->Path;

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QuicCongestionControlGetPathID(Cc)->Path is not a reliable pointer to the path this congestion control is currently driving when multipath is not negotiated, so this swaps a always-valid pointer for one that can be stale.

QuicPathSetActive() (src/core/path.c, non-multipath branch) swaps the contents of Connection->Paths[0] and *Path and never fixes up PathID->Path. QuicPathRemove() only re-points PathID->Path inside if (Connection->State.MultipathNegotiated).

Concrete scenario (server, multipath off, client migrates / NAT rebind):

  1. QuicConnGetPathForPacket() creates the new path at Paths[1] for the existing path ID and sets PathID->Path = &Paths[1].
  2. The migration is accepted and QuicPathSetActive(Connection, &Paths[1]) swaps the two slots — the active path is now Paths[0], but PathID->Path still points at Paths[1], which now holds the old path.
  3. QUIC_PARAM_CONN_NETWORK_STATISTICS and the QUIC_CONNECTION_EVENT_NETWORK_STATISTICS event now report the old path's SmoothedRtt. Before this change they read Connection->Paths[0], which is always the active path.
  4. Once the old path is removed, PathID->Path refers to a slot past PathsCount holding stale data, so the reported RTT is whatever was left there.

Multipath connections are fine (the back-references are maintained there), which is why the new test passes. Consider having the caller supply the path (e.g. pass the QUIC_PATH* into the hook, or fill SmoothedRTT in QuicConnGetPathStatistics/QuicConnGetNetworkStatistics from the path being iterated) rather than depending on the PathID->Path back-pointer.

Comment thread src/core/bbr.c Outdated
// Congestion control is per path ID, so the RTT to report is the one of the
// path this instance belongs to, not whichever path happens to be first.
//
const QUIC_PATH* Path = QuicCongestionControlGetPathID(Cc)->Path;

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue as in cubic.c: QuicCongestionControlGetPathID(Cc)->Path is not guaranteed to be the path this CC is driving when multipath is not negotiated.

QuicPathSetActive() (src/core/path.c) swaps Connection->Paths[0] with *Path in the non-multipath branch without updating PathID->Path, and QuicPathRemove() only repairs the back-references under if (Connection->State.MultipathNegotiated).

Scenario: non-multipath server connection, client migrates. QuicConnGetPathForPacket() sets PathID->Path = &Paths[1]; QuicPathSetActive() then swaps Paths[0]/Paths[1], leaving PathID->Path on the now-inactive old path. BbrCongestionControlGetNetworkStatistics (used by both QUIC_PARAM_CONN_NETWORK_STATISTICS and the QUIC_CONNECTION_EVENT_NETWORK_STATISTICS indication) then reports that stale path's SmoothedRtt; the previous &Connection->Paths[0] was always the active path. After the old path is removed the pointer refers to a slot beyond PathsCount and the RTT reported is leftover data.

Comment thread src/core/connection.c
// the handshake is confirmed has neither yet.
//
uint8_t PathCount = 0;
for (uint8_t i = 0; i < Connection->PathsCount; ++i) {

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This counts/emits one entry per QUIC_PATH, but a QUIC_PATHID is not one-to-one with a QUIC_PATH: two paths can hold the same PathID pointer at the same time.

QuicConnGetPathForPacket() (src/core/path.c) handles a rebind/migration by creating a new QUIC_PATH at Paths[1] for an existing path ID (QuicPathIDAddRef(PathID, QUIC_PATHID_REF_PATH); Path->PathID = PathID; PathID->Path = Path;) while the old path, still carrying the same PathID pointer, remains in the array until it is validated away.

In that window QUIC_PARAM_CONN_PATH_STATISTICS returns two entries with the same PathId (0 on a non-multipath connection, where every path shares path ID 0) and byte-identical NetworkStatistics, since both read the one PathID->CongestionControl. That directly contradicts the documented contract added in docs/Settings.md ("PathId identifies which path an entry describes") and leaves the caller unable to tell the two apart.

Related: PathStats->Rtt is read from Paths[i].SmoothedRtt while PathStats->NetworkStatistics.SmoothedRTT is read (via the CC hook) from PathID->Path->SmoothedRtt. Whenever those are not the same object — the duplicate case above, or the non-multipath QuicPathSetActive() swap — the two fields of a single entry disagree, breaking the TEST_EQUAL(PathStats[i].Rtt, PathStats[i].NetworkStatistics.SmoothedRTT) invariant the new test relies on.

Comment thread docs/Settings.md
| `QUIC_PARAM_CONN_REMOVE_CANDIDATE_ADDRESS` <br> 34| QUIC_CANDIDATE_ADDRESS | Set-only | Remove a candidate address. Client only. |

| `QUIC_PARAM_CONN_UNCONNECTED_UDP_SOCKET` <br> 37 | uint8_t (BOOLEAN) | Both | Set on client only. Must be set before start, and requires `QUIC_PARAM_CONN_SHARE_UDP_BINDING`. See [QUIC_PARAM_CONN_UNCONNECTED_UDP_SOCKET](#quic_param_conn_unconnected_udp_socket). |
| `QUIC_PARAM_CONN_PATH_STATISTICS` <br> 38 | QUIC_PATH_STATISTICS[] | Get-only | Network statistics for every path at once, one array entry per path. See [QUIC_PARAM_CONN_PATH_STATISTICS](#quic_param_conn_path_statistics). |

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rendering nit, but it makes the new row invisible in the docs: line 225 is blank, which terminates the parameter table above. A block of |-delimited lines with no header/delimiter row is not a GFM table, so this row (and the QUIC_PARAM_CONN_UNCONNECTED_UDP_SOCKET row above it, which has the same pre-existing problem) renders as literal | ... | text rather than as table rows.

Moving both rows up so they are contiguous with the table that ends at QUIC_PARAM_CONN_REMOVE_CANDIDATE_ADDRESS (deleting the blank line at 225) fixes it.

Review of #82 found that deriving the path from the congestion control's
owning path ID regresses QUIC_PARAM_CONN_NETWORK_STATISTICS on connections
without multipath.

QuicPathSetActive swaps the *contents* of Paths[0] and the promoted path:

    QUIC_PATH PrevActivePath = Connection->Paths[0];
    Connection->Paths[0] = *Path;
    *Path = PrevActivePath;

The PathID pointer travels with the contents, but the path ID's own back
reference is left behind, and QuicPathRemove only repairs back references
under if (MultipathNegotiated). So after a migration on a non-multipath
connection, PathID->Path points at the slot the old path was moved into, and
the statistics would have come from the inactive path -- and, once that path
was removed, from a slot past PathsCount holding stale data.

The path is now a parameter of the hook, so no caller depends on that back
reference. QuicConnGetNetworkStatistics and the BBR connection event pass
&Connection->Paths[0], which is the expression the hook used internally
before, so both keep their existing behaviour exactly.
QuicConnGetPathStatistics passes the path it is reporting on, which also
removes the possibility of an entry's Rtt disagreeing with its own
NetworkStatistics.SmoothedRTT.

Review also found that a rebind leaves two paths sharing one path ID for a
while -- QuicConnGetPathForPacket assigns the path ID to the new path without
clearing it from the old one -- so PathId is not unique in the array while
that lasts. Both entries are real paths and the shared congestion control
makes their NetworkStatistics agree; documented rather than filtered, since
dropping one would hide a path that exists.

The parameter table in docs/Settings.md had a blank line in the middle, which
terminated it early and rendered the last rows as literal text. Removed; this
also fixes the QUIC_PARAM_CONN_UNCONNECTED_UDP_SOCKET row added earlier.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@masa-koz

Copy link
Copy Markdown
Author

All four findings were real. Verified each against the source before acting; fixed in ac05aa7.

The cubic/bbr regression (findings 1 and 2). Confirmed. QuicPathSetActive's non-multipath branch swaps the contents of the two slots:

QUIC_PATH PrevActivePath = Connection->Paths[0];
Connection->Paths[0] = *Path;
*Path = PrevActivePath;

The PathID pointer travels with the contents while the path ID's back reference stays behind, and QuicPathRemove only repairs back references under if (MultipathNegotiated). So deriving the path from QuicCongestionControlGetPathID(Cc)->Path was wrong exactly where the old &Connection->Paths[0] was right.

Rather than reinstate the hardcoded index, the path is now a parameter of the hook, so no caller depends on that back reference at all. QuicConnGetNetworkStatistics and the BBR connection event pass &Connection->Paths[0] — the same expression the hook used internally before, so their behaviour is unchanged — and QuicConnGetPathStatistics passes the path it is reporting on. That also closes the Rtt vs NetworkStatistics.SmoothedRTT disagreement you noted at the end of finding 3, since both now come from the same QUIC_PATH.

The unit tests in BbrTest.cpp and CubicTest.cpp were updated for the new signature.

Duplicate PathId during a rebind (finding 3). Confirmed. QuicConnGetPathForPacket does Path->PathID = PathID; PathID->Path = Path; for the new path without clearing the pointer from the old one, so both carry it until the old path goes away.

Documented rather than filtered. Both entries are real paths, and since they share one path ID they share its congestion control, so their NetworkStatistics agreeing is correct rather than a defect — what distinguishes them is the per-path Rtt, MinRtt, MaxRtt and Mtu. Suppressing one would hide a path that exists from an API whose job is to report all of them. docs/Settings.md now says PathId is not guaranteed unique and why.

The docs table (finding 4). Confirmed — a blank line after the REMOVE_CANDIDATE_ADDRESS row terminated the table, so the last two rows rendered as literal text. Removed, which also repairs the QUIC_PARAM_CONN_UNCONNECTED_UDP_SOCKET row that had been broken since it was added.

Verification after the fix. PathStatistics ×3 repeats 6/6, *Multipath*:*Path*:*Migration*:*UnconnectedSocket*:*KeepAlive*:*Statistics* 112 pass, *Basic*:*Datagram*:*Receive*:*Recv* 626 pass, msquiccoretest *Bbr*:*Cubic* 128 pass, and clang-tidy 21 clean on the three core files via the CI container. Pointing QuicConnGetPathStatistics back at Paths[0] still fails the test, so the per-path assertion is still doing its job.

@codecov

codecov Bot commented Aug 22, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.93939% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/core/connection.c 93.75% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

@masa-koz
masa-koz merged commit 880a765 into seera-main Aug 22, 2026
583 of 598 checks passed
masa-koz added a commit that referenced this pull request Aug 22, 2026
Brings in #82 (QUIC_PARAM_CONN_PATH_STATISTICS). One conflict:

  src/test/bin/quic_gtest.cpp
    Both sides added a TEST_P after PathKeepAlive -- qmux-01 the three QMux
    tests, seera-main the PathStatistics one. Kept both, PathStatistics
    first so it stays next to the other path tests.

QMux connections are unaffected by the new parameter. QuicConnQMuxAlloc
never initializes Paths[], so PathsCount is zero and the new
QuicConnGetPathStatistics iterates nothing: it reports zero entries rather
than reaching for a path ID or its congestion control.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant