Skip to content

Commit 633ab11

Browse files
committed
fix(const)!: give tcp.flags.Flags their wire masks (#2310)
Problem: the crawler generated each member as 1 << offset in IANA's most-significant-first numbering, so FIN was 0x8000 and int() of a parsed SYN/ACK's connection was 0x4800 where the header carries 0x012. Fix: generate 1 << (15 - offset), each flag's mask in the 12-bit flags field, and narrow the range guard to 0 <= value <= 0xFFF. The const module is regenerated from the crawler. Tests and the dumpkit docstring that pinned the old integers, or used Flags for undeclared bits it no longer has, move to the new values or to BindingUpdateFlag. Test: test_const_tcp_flags_wire_masks_2310_unit checks every member's mask, composites, the guard, a parsed SYN/ACK, every flags octet reading back as itself with a byte-exact rebuild, and the crawler's output. Changelog entry under pcapkit.const, CHANGELOG.md regenerated.
1 parent bfe184b commit 633ab11

11 files changed

Lines changed: 255 additions & 94 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and
1919
- **a breaking change to** `ExtensionHeader`: it is generated from IANA's IPv6 Extension Header Types registry rather than the Protocol Numbers registry's flag, and the two disagree on 147 (`BIT-EMU`) ([#925](https://github.com/JarryShaw/PyPCAPKit/issues/925), [#926](https://github.com/JarryShaw/PyPCAPKit/pull/926)).
2020
- the TCP and UDP `AppType` registries load on first use rather than at import ([#1538](https://github.com/JarryShaw/PyPCAPKit/issues/1538), [#1560](https://github.com/JarryShaw/PyPCAPKit/pull/1560)).
2121
- **a breaking change to** `LLACode`: `Wilcard` is spelled `Wildcard` and `MH` is `MN`, the names [RFC 5568](https://datatracker.ietf.org/doc/html/rfc5568) gives codes 0 and 2, with no alias for either old name ([#1848](https://github.com/JarryShaw/PyPCAPKit/issues/1848), [#1872](https://github.com/JarryShaw/PyPCAPKit/pull/1872)).
22+
- **a breaking change to** `tcp.flags.Flags`: each member is its wire mask in the flags field, `FIN` `0x001` up to `AE` `0x100`, rather than `1 << offset` in IANA's most-significant-first bit numbering, which put `FIN` at `0x8000`. `int(tcp.connection)` of a SYN/ACK is `0x012`, as the header carries it, not `0x4800`, and `Flags(0x12)` is `SYN | ACK`. The range guard narrows to the 12-bit flags field, `0 <= value <= 0xFFF`. Code that stored or compared the old integers has to change; membership tests and names are unaffected ([#2310](https://github.com/JarryShaw/PyPCAPKit/issues/2310)).
2223

2324
#### Fixed
2425

‎docs/source/changelog/1.5.0.rst‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,14 @@ Changed
9090
* **a breaking change to** :class:`~pcapkit.const.mh.lla_code.LLACode`: ``Wilcard`` is
9191
spelled ``Wildcard`` and ``MH`` is ``MN``, the names :rfc:`5568` gives codes 0 and 2,
9292
with no alias for either old name (:issue:`1848`, :pr:`1872`).
93+
* **a breaking change to** :class:`tcp.flags.Flags <pcapkit.const.tcp.flags.Flags>`:
94+
each member is its wire mask in the flags field, ``FIN`` ``0x001`` up to ``AE``
95+
``0x100``, rather than ``1 << offset`` in IANA's most-significant-first bit numbering,
96+
which put ``FIN`` at ``0x8000``. ``int(tcp.connection)`` of a SYN/ACK is ``0x012``,
97+
as the header carries it, not ``0x4800``, and ``Flags(0x12)`` is ``SYN | ACK``. The
98+
range guard narrows to the 12-bit flags field, ``0 <= value <= 0xFFF``. Code that
99+
stored or compared the old integers has to change; membership tests and names are
100+
unaffected (:issue:`2310`).
93101

94102
Fixed
95103
~~~~~

‎pcapkit/const/tcp/flags.py‎

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -26,40 +26,40 @@ class Flags(EnumRegistry, IntFlag):
2626
"""[Flags] TCP Header Flags"""
2727

2828
#: Reserved for future use [:rfc:`9293`]
29-
Reserved_4 = 1 << 4
29+
Reserved_4 = 1 << 11
3030

3131
#: Reserved for future use [:rfc:`9293`]
32-
Reserved_5 = 1 << 5
32+
Reserved_5 = 1 << 10
3333

3434
#: Reserved for future use [:rfc:`9293`]
35-
Reserved_6 = 1 << 6
35+
Reserved_6 = 1 << 9
3636

3737
#: AE (Accurate ECN) [:rfc:`9768`]
38-
AE = 1 << 7
38+
AE = 1 << 8
3939

4040
#: CWR (Congestion Window Reduced) [:rfc:`3168`]
41-
CWR = 1 << 8
41+
CWR = 1 << 7
4242

4343
#: ECE (ECN-Echo) [:rfc:`3168`]
44-
ECE = 1 << 9
44+
ECE = 1 << 6
4545

4646
#: Urgent Pointer field is significant (URG) [:rfc:`9293`]
47-
URG = 1 << 10
47+
URG = 1 << 5
4848

4949
#: Acknowledgment field is significant (ACK) [:rfc:`9293`]
50-
ACK = 1 << 11
50+
ACK = 1 << 4
5151

5252
#: Push Function (PSH) [:rfc:`9293`]
53-
PSH = 1 << 12
53+
PSH = 1 << 3
5454

5555
#: Reset the connection (RST) [:rfc:`9293`]
56-
RST = 1 << 13
56+
RST = 1 << 2
5757

5858
#: Synchronize sequence numbers (SYN) [:rfc:`9293`]
59-
SYN = 1 << 14
59+
SYN = 1 << 1
6060

6161
#: No more data from sender (FIN) [:rfc:`9293`]
62-
FIN = 1 << 15
62+
FIN = 1 << 0
6363

6464
@classmethod
6565
def _validate_value(cls, value: 'Any') -> 'None':
@@ -80,7 +80,7 @@ def _validate_value(cls, value: 'Any') -> 'None':
8080
unaffected.
8181
8282
"""
83-
if not (isinstance(value, int) and 0 <= value <= 0xFFFF):
83+
if not (isinstance(value, int) and 0 <= value <= 0xFFF):
8484
raise EnumValueError(f'{value!r} is not a valid {cls.__name__}')
8585

8686
@classmethod
@@ -91,6 +91,6 @@ def _missing_(cls, value: 'int') -> 'Flags':
9191
value: Value to get enum item.
9292
9393
"""
94-
if not (isinstance(value, int) and 0 <= value <= 0xFFFF):
94+
if not (isinstance(value, int) and 0 <= value <= 0xFFF):
9595
raise ValueError(f'{value!r} is not a valid {cls.__name__}')
9696
return super()._missing_(value)

‎pcapkit/dumpkit/common.py‎

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -220,16 +220,18 @@ def render_enum(o: 'enum.Enum | aenum.Enum') -> 'str':
220220
plausible member name, so a consumer splitting the rendering on ``::``
221221
cannot tell it from a member genuinely so named -- and ``NONE`` *is* a
222222
declared name elsewhere in the library. And the defect is not confined to
223-
zero: ``Flags(1)``, ``Flags(8)`` and ``Flags(15)`` are every bit as
224-
nameless, so a guard written against ``value == 0`` would fix one case
225-
and leave the rest.
223+
zero: a registry whose field has undeclared bits, such as
224+
:class:`~pcapkit.const.mh.binding_update_flag.BindingUpdateFlag`, is
225+
every bit as nameless at ``1`` or ``8``, so a guard written against
226+
``value == 0`` would fix one case and leave the rest.
226227
227228
The fallback is the value's own decimal spelling, which is what the
228229
enumeration libraries themselves already use for an undeclared residue:
229-
``Flags(2057).name`` is ``'ACK|9'``, naming the declared bit and giving
230-
the leftovers as one number. A wholly-undeclared value is that same
231-
rendering with no declared bit to precede it, so ``Flags(9)`` becomes
232-
``'Flags::9 [9]'`` and ``Flags(0)`` becomes ``'Flags::0 [0]'``. It also
230+
``BindingUpdateFlag(32777).name`` is ``'A|9'``, naming the declared bit
231+
and giving the leftovers as one number. A wholly-undeclared value is that
232+
same rendering with no declared bit to precede it, so
233+
``BindingUpdateFlag(9)`` becomes ``'BindingUpdateFlag::9 [9]'`` and
234+
``Flags(0)`` becomes ``'Flags::0 [0]'``. It also
233235
cannot be mistaken for a member name, since a Python identifier may not
234236
begin with a digit.
235237

‎pcapkit/protocols/transport/tcp.py‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -528,9 +528,9 @@ def read(self, length: 'Optional[int]' = None, **kwargs: 'Any') -> 'Data_TCP':
528528
# never runs, and ``Enum_Flags.SYN in self._flags`` would raise ``TypeError:
529529
# argument of type 'int' is not a container or iterable``. A segment with any flag
530530
# set would mask that. :class:`Enum_Flags` is an ``aenum.IntFlag`` whose
531-
# ``_missing_`` accepts any value from ``0`` to ``0xFFFF``, so ``Enum_Flags(0)`` is
532-
# a valid flagless member that compares equal to ``0`` and ORs as usual. This is the type :attr:`connection`
533-
# advertises.
531+
# ``_missing_`` accepts any value in the 12-bit flags field, ``0`` to ``0xFFF``,
532+
# so ``Enum_Flags(0)`` is a valid flagless member that compares equal to ``0`` and
533+
# ORs as usual. This is the type :attr:`connection` advertises.
534534
_flag = Enum_Flags(0)
535535
for key, val in schema.flags.items():
536536
if val == 1:

‎pcapkit/vendor/tcp/flags.py‎

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -104,11 +104,13 @@ class Flags(Vendor):
104104
"""TCP Header Flags"""
105105

106106
#: Value limit checker. The registry indexes *bit offsets* 4 through 15,
107-
#: but the members it generates are ``1 << offset``, so the value domain a
108-
#: lookup has to accept is the 16-bit field those bits live in -- a
107+
#: counted from the most significant end of the 16-bit Data Offset/flags
108+
#: word, and the members it generates are their wire masks,
109+
#: ``1 << (15 - offset)`` (:issue:`2310`). So the value domain a lookup has
110+
#: to accept is the low 12 bits of that word, the flags field -- a
109111
#: composite such as ``SYN | ACK`` is a legitimate value and only the
110112
#: composite path reaches :meth:`~pcapkit.const.tcp.flags.Flags._missing_`.
111-
FLAG = 'isinstance(value, int) and 0 <= value <= 0xFFFF'
113+
FLAG = 'isinstance(value, int) and 0 <= value <= 0xFFF'
112114
#: Link to registry.
113115
LINK = 'https://www.iana.org/assignments/tcp-parameters/tcp-header-flags.csv'
114116

@@ -146,7 +148,10 @@ def process(self, data: 'list[str]') -> 'list[str]': # type: ignore[override]
146148
name = DATA.get(int(code), dscp.split(' (')[0]).replace('Reserved for future use', 'Reserved')
147149
renm = self.rename(name or 'Unassigned', code, original=dscp)
148150

149-
pres = f"{renm} = 1 << {code}"
151+
# NOTE: IANA numbers the bits from the most significant end of the
152+
# 16-bit Data Offset/flags word, so the wire mask of offset ``n`` is
153+
# ``1 << (15 - n)`` -- FIN, at offset 15, is the low bit (#2310).
154+
pres = f"{renm} = 1 << {15 - int(code)}"
150155
sufs = f'#: {desc}'
151156

152157
# if len(pres) > 74:

‎tests/const/test_const_enum_builtin_parity.py‎

Lines changed: 22 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -543,7 +543,7 @@ class ConstFlagCompositeTests(unittest.TestCase):
543543
``FLAG = 'isinstance(value, int) and 4 <= value <= 15'`` checker in
544544
:mod:`pcapkit.vendor.tcp.flags` would have done had the template ever
545545
interpolated it: those are *bit offsets*, and the values they generate are
546-
``1 << offset``.
546+
the wire masks ``1 << (15 - offset)`` (#2310).
547547
"""
548548

549549
def setUp(self) -> None:
@@ -565,19 +565,16 @@ def test_tcp_flag_composites_resolve(self) -> None:
565565
every = 0
566566
for member in Flags:
567567
every |= int(member)
568-
self.assertEqual(every, 0xFFF0)
568+
self.assertEqual(every, 0xFFF)
569569
self.assertEqual(int(Flags(every)), every)
570570

571-
# One past the 16-bit field the registry indexes bits of.
572-
for beyond in (0x10000, -1, -0x10000, UNRESOLVABLE):
571+
# One past the 12-bit flags field, where the data offset begins, and
572+
# the values outside any width.
573+
for beyond in (0x1000, 0x10000, -1, -0x1000, UNRESOLVABLE):
573574
with self.subTest(value=beyond):
574575
with self.assertRaises(ValueError):
575576
Flags(beyond)
576577

577-
# An unassigned low bit is in range and still composes, rather than
578-
# being rejected: bits 0-3 of that field are the data offset.
579-
self.assertEqual(int(Flags(1)), 1)
580-
581578
def test_the_other_flag_registry_composes(self) -> None:
582579
"""``CommandType`` composes; ``TransportProtocol`` is not in this group.
583580
@@ -717,11 +714,11 @@ def test_the_guard_leaves_the_other_lookup_paths_alone(self) -> None:
717714
broken either would fail rather than merely go unmeasured.
718715
719716
The fallback is probed with a two-part check: a *registered* default
720-
(``1 << 14``, ``SYN``) is genuinely consulted and resolves, while
717+
(``1 << 1``, ``SYN``) is genuinely consulted and resolves, while
721718
``Flags.get(UNRESOLVABLE, 0)`` raises the same original-key error as
722719
no default at all. ``0`` is in-bounds for ``Flags``'s own
723-
``_missing_`` (``0 <= value <= 0xFFFF``), but it is not a *registered*
724-
``Flags`` member (``Flags``'s smallest declared value is ``1 << 4``),
720+
``_missing_`` (``0 <= value <= 0xFFF``), but it is not a *registered*
721+
``Flags`` member (``Flags``'s smallest declared value is ``1 << 0``),
725722
and GitHub issue #864 restricts ``default`` to a plain
726723
``_value2member_map_`` lookup -- the same accepted cost as an
727724
unassigned integer range, here on an :class:`~aenum.IntFlag` registry
@@ -748,12 +745,12 @@ def test_the_guard_leaves_the_other_lookup_paths_alone(self) -> None:
748745

749746
# Integer paths, which do.
750747
self.assertIs(TransportProtocol.get(1), TransportProtocol.tcp)
751-
self.assertIs(Flags.get(1 << 14), Flags.SYN)
748+
self.assertIs(Flags.get(1 << 1), Flags.SYN)
752749

753750
# And the fallback the guard's ValueError is what triggers: GitHub issue
754751
# #584's ``get(key, default)``, which an EnumError would have walked past.
755752
# A registered default is genuinely consulted and resolves.
756-
self.assertIs(Flags.get(UNRESOLVABLE, 1 << 14), Flags.SYN)
753+
self.assertIs(Flags.get(UNRESOLVABLE, 1 << 1), Flags.SYN)
757754
# ``0`` is not a registered ``Flags`` member (GitHub issue #864), so
758755
# it does not resolve either -- the original key's error propagates,
759756
# the same as omitting the default outright.
@@ -1227,26 +1224,30 @@ def test_the_tcp_flags_checker_covers_the_value_domain_not_the_bit_offsets(self)
12271224
"""The template's checker, and why a bit-offset one could not be kept.
12281225
12291226
A ``Flags.FLAG`` of ``4 <= value <= 15`` covers the *offsets* the IANA
1230-
registry indexes, while the members it generates are ``1 << offset``.
1231-
The template never interpolated it, so it was never wrong in practice;
1232-
interpolating it unchanged would have rejected every composite, every
1233-
single member above bit 3, and ``Flags(0)``.
1227+
registry indexes, while the members it generates are their wire masks,
1228+
``1 << (15 - offset)`` (#2310). The template never interpolated it, so
1229+
it was never wrong in practice; interpolating it unchanged would have
1230+
rejected ``FIN``, ``SYN``, every member from ``ACK`` up, most
1231+
composites, and ``Flags(0)``.
12341232
"""
12351233
from pcapkit.vendor.tcp.flags import Flags as VendorFlags
12361234

12371235
checker = VendorFlags.FLAG
1238-
self.assertIn('0xFFFF', checker)
1236+
self.assertRegex(checker, r'<= 0xFFF$')
12391237

12401238
from pcapkit.const.tcp.flags import Flags
12411239

1240+
old = 'isinstance(value, int) and 4 <= value <= 15'
12421241
for member in Flags:
12431242
value = int(member)
12441243
with self.subTest(member=member.name):
12451244
self.assertTrue(eval(checker, {'value': value}), # pylint: disable=eval-used
12461245
f'the checker rejects its own member {member.name}={value}')
1247-
self.assertFalse(eval('isinstance(value, int) and 4 <= value <= 15',
1248-
{'value': value}), # pylint: disable=eval-used
1249-
f'the old checker accepted {member.name}={value}')
1246+
for value in (0, int(Flags.FIN), int(Flags.SYN | Flags.ACK), int(Flags.CWR)):
1247+
with self.subTest(value=hex(value)):
1248+
self.assertTrue(eval(checker, {'value': value})) # pylint: disable=eval-used
1249+
self.assertFalse(eval(old, {'value': value}), # pylint: disable=eval-used
1250+
f'the old checker accepted {value:#x}')
12501251

12511252
def test_self_check_of_the_repr_percent_format_walk(self) -> None:
12521253
""":func:`percent_format_lines_in_repr` against its own fixtures.

‎tests/const/test_const_registry_protocol.py‎

Lines changed: 18 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -201,25 +201,27 @@ class _PreConversionFlags(IntFlag):
201201
hand-copied ``get`` (omitted here -- irrelevant to ``_missing_``) and this
202202
``_missing_``, copied verbatim. Kept inline rather than fetched from git
203203
history so :class:`TCPFlagsConversionTests` runs offline and the
204-
comparison is exact rather than approximate.
204+
comparison is exact rather than approximate. The member values and the
205+
range check track the generated module's, wire masks in the 12-bit flags
206+
field since #2310.
205207
"""
206208

207-
Reserved_4 = 1 << 4
208-
Reserved_5 = 1 << 5
209-
Reserved_6 = 1 << 6
210-
AE = 1 << 7
211-
CWR = 1 << 8
212-
ECE = 1 << 9
213-
URG = 1 << 10
214-
ACK = 1 << 11
215-
PSH = 1 << 12
216-
RST = 1 << 13
217-
SYN = 1 << 14
218-
FIN = 1 << 15
209+
Reserved_4 = 1 << 11
210+
Reserved_5 = 1 << 10
211+
Reserved_6 = 1 << 9
212+
AE = 1 << 8
213+
CWR = 1 << 7
214+
ECE = 1 << 6
215+
URG = 1 << 5
216+
ACK = 1 << 4
217+
PSH = 1 << 3
218+
RST = 1 << 2
219+
SYN = 1 << 1
220+
FIN = 1 << 0
219221

220222
@classmethod
221223
def _missing_(cls, value: 'Any') -> 'Any':
222-
if not (isinstance(value, int) and 0 <= value <= 0xFFFF):
224+
if not (isinstance(value, int) and 0 <= value <= 0xFFF):
223225
raise ValueError(f'{value!r} is not a valid {cls.__name__}')
224226
return super()._missing_(value)
225227

@@ -276,14 +278,14 @@ def test_missing_resolves_identically_to_the_pre_conversion_shape(self) -> None:
276278
:class:`~aenum.Flag` machinery exactly as it did when ``Flags``
277279
inherited from ``IntFlag`` directly. Swept rather than sampled at a
278280
few boundary values, across every declared member, every boundary of
279-
the 16-bit field the range check bounds, and several composites,
281+
the 12-bit field the range check bounds, and several composites,
280282
so a widened or narrowed range check would be caught here rather
281283
than only in the composite-specific tests in
282284
``tests/const/test_const_enum_builtin_parity.py``.
283285
"""
284286
from pcapkit.const.tcp.flags import Flags
285287

286-
probes = [0, 1, 0xF, 0x10, 0xFFFF, 0x10000, -1, -0x10000,
288+
probes = [0, 1, 0xF, 0x10, 0xFFF, 0x1000, 0xFFFF, 0x10000, -1, -0x1000,
287289
1 << 70, 0xFFF0]
288290
for member in _PreConversionFlags:
289291
probes.append(int(member))

0 commit comments

Comments
 (0)