Skip to content

Commit 14be18a

Browse files
committed
feat(vendor): add the TLS content-type/handshake-type crawlers, close pairing gap (#2495)
- pcapkit/vendor/tls/content_type.py, handshake_type.py: new LINK-and-scrape crawlers against IANA's tls-parameters-5.csv and -7.csv. Both override process()/count() for the registry's 5-column CSV, lower-case the wire name into the member identifier, and for HandshakeType strip IANA's _RESERVED suffix from identifiers while keeping it out of the comment. - Regenerated pcapkit/const/tls/content_type.py, handshake_type.py from the crawlers. Fixes a real behaviour bug: ContentType resolved 32-63 to Unassigned via a flat _missing_ fallback where IANA (and the crawler) call that band Reserved (RFC 9147) -- the committed docstring already said "Reserved", the code did not. Drops ContentType.invalid=0 (no IANA counterpart, unused anywhere); value 0 now resolves into the 0-19 Unassigned band like every other unassigned value there, which SSL's wire-type allowlist already excluded by construction, so this is a behavioural no-op in the protocol layer. HandshakeType gains explicit unassigned_7/unassigned_255 members (IANA's two single-row Unassigned entries) instead of leaving them to _missing_, and several members' RFC citations now match IANA's current RFC 9846 instead of the superseded 5246/6066/4680. - Moved the hand-written protocol-specific prose (record-layer header citations, Unassigned/Reserved band rationale, _RESERVED-suffix handling) from the const modules' docstrings into the crawlers' own docstrings, matching pcapkit.vendor.linux.arphrd's pattern; the generated const modules now carry only the plain templated docstring. - pcapkit/protocols/application/ssl.py: updated two comments that named the now-removed invalid(0) member. - docs/.../const-vendor-pairing.rst: rewrote the TLS section -- running either crawler is now a true no-op against the committed const files, closing the gap the previous revision left open as follow-up work. - tests/const/test_const_tls_vendor_pairing_2495_unit.py: new, pins the Reserved-band fix, the dropped invalid member and the two new unassigned members; all five assertions fail against the pre-fix files. Verified: tests/vendor, tests/const, test_ssl_tls_dtls_unit.py and test_tls_dtls_roundtrip_unit.py all green (651 passed, 26949 subtests). Regenerating both const modules twice in a row produces byte-identical output.
1 parent 57ff1c8 commit 14be18a

10 files changed

Lines changed: 544 additions & 92 deletions

File tree

‎docs/source/contributing/conventions/const-vendor-pairing.rst‎

Lines changed: 42 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -40,15 +40,47 @@ module docstring and comments, alongside the ``DATA`` literal it documents; the
4040
generated ``const`` module carries only the plain, templated docstring every
4141
other generated module does.
4242

43-
Known Violation
44-
~~~~~~~~~~~~~~~
43+
``LINK``-and-Scrape Pairing: TLS Content Types and HandshakeType
44+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
4545

4646
:class:`~pcapkit.const.tls.content_type.ContentType` and
47-
:class:`~pcapkit.const.tls.handshake_type.HandshakeType` (:pr:`2441`) both cite a live
48-
IANA registry and claim to name every value IANA assigns, with no
49-
:mod:`pcapkit.vendor` crawler behind either one -- no ``pcapkit/vendor/tls/``
50-
directory exists, and neither class appears in :data:`pcapkit.vendor.__all__`. This
51-
was flagged in that PR's own review and merged anyway on the stated reasoning that the
52-
crawler would follow later. Tracked as :issue:`2495`. Unlike ARPHRD, TLS's source
53-
*is* a live, web-published IANA table, so its eventual crawler takes the normal
54-
``LINK``-and-scrape shape, not the ``DATA`` attribute.
47+
:class:`~pcapkit.const.tls.handshake_type.HandshakeType` (:pr:`2441`) now have
48+
matching crawlers -- :mod:`pcapkit.vendor.tls.content_type` and
49+
:mod:`pcapkit.vendor.tls.handshake_type`, exported as ``TLS_ContentType`` and
50+
``TLS_HandshakeType`` -- closing the gap :issue:`2495` tracked. Both take the
51+
normal ``LINK``-and-scrape shape against IANA's ``tls-parameters-5.csv`` and
52+
``tls-parameters-7.csv``, as this doc said the eventual crawler would; unlike
53+
ARPHRD, TLS's source is a live, web-published table, so neither reads a
54+
``DATA`` literal.
55+
56+
Running either crawler is a true no-op against the committed ``const``
57+
modules: regenerating both reproduces the tree byte-for-byte, the same
58+
guarantee ARPHRD and the ``sslv2`` restack carry. Getting there meant
59+
regenerating the ``const`` modules committed alongside :pr:`2441`, which
60+
predated the crawler and had drifted from it in three ways, caught in
61+
cross-review of :pr:`2503`:
62+
63+
* the committed ``ContentType`` resolved values 32-63 to ``Unassigned`` via a
64+
flat ``_missing_`` fallback, while IANA's registry -- and the crawler --
65+
call that band ``Reserved`` (:rfc:`9147`); the committed docstring already
66+
said "Reserved", the code did not. Regenerating fixes the resolved name for
67+
that band.
68+
* the committed ``ContentType`` carried an ``invalid = 0`` sentinel with no
69+
IANA counterpart and no reference anywhere in the codebase (IANA leaves 0
70+
inside its 0-19 ``Unassigned`` band, unnamed); it is dropped, and value 0
71+
now resolves to ``Unassigned`` like the rest of that band.
72+
:class:`~pcapkit.protocols.application.ssl.SSL` never checked this member
73+
by name -- its wire-type allowlist already excluded value 0 by construction
74+
-- so the sentinel's removal is behaviourally a no-op there too.
75+
* the committed ``HandshakeType`` cited RFC 5246, 6066 and 4680 for several
76+
members where IANA's registry now cites :rfc:`9846`, and left IANA's two
77+
single-row ``Unassigned`` entries (values 7 and 255) to ``_missing_``
78+
rather than naming them; the crawler renders both as explicit members
79+
(``unassigned_7``, ``unassigned_255``), matching the registry.
80+
81+
The protocol-specific prose the hand-curated modules carried -- the
82+
record-layer header citations, the Unassigned/Reserved band rationale, the
83+
``_RESERVED``-suffix handling -- moved into the crawlers' own module
84+
docstrings and comments, the way :mod:`pcapkit.vendor.linux.arphrd` already
85+
does for ARPHRD; the generated ``const`` modules carry only the plain,
86+
templated docstring every other generated module does.

‎pcapkit/const/tls/content_type.py‎

Lines changed: 33 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -6,17 +6,7 @@
66
.. module:: pcapkit.const.tls.content_type
77
88
This module contains the constant enumeration for **TLS/DTLS Content Types**,
9-
as assigned by IANA in the ``TLS ContentType`` registry [*]_. The record-layer
10-
header -- :rfc:`9846#section-5.1` for TLS, :rfc:`9147#section-4`'s
11-
``DTLSPlaintext`` for DTLS -- opens with this type in both.
12-
13-
Every value IANA assigns is named below. Everything else in the 0-255 range it
14-
lists as Unassigned (0-19 and 28-31, plus 64-255, which :rfc:`9443` reserves
15-
for multiplexing with other protocols on the same port) or as Reserved (32-63,
16-
:rfc:`9147`), and :meth:`ContentType._missing_` resolves those to an
17-
``Unassigned`` member without growing the registry.
18-
19-
.. [*] https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-5
9+
which is automatically generated from :class:`pcapkit.vendor.tls.content_type.ContentType`.
2010
2111
"""
2212

@@ -36,45 +26,47 @@
3626
class ContentType(EnumRegistry, IntEnum):
3727
"""[ContentType] TLS/DTLS Content Types"""
3828

39-
#: Invalid -- a sentinel :rfc:`9846#section-5.1` names so that a record
40-
#: carrying it can be rejected, never a value sent on the wire; IANA keeps
41-
#: 0 inside its Unassigned 0-19 band rather than assigning it [:rfc:`9846`]
42-
invalid = 0
43-
44-
#: ChangeCipherSpec [:rfc:`9846`]
29+
#: change_cipher_spec [:rfc:`9846`]
4530
change_cipher_spec = 20
4631

47-
#: Alert [:rfc:`9846`]
32+
#: alert [:rfc:`9846`]
4833
alert = 21
4934

50-
#: Handshake [:rfc:`9846`]
35+
#: handshake [:rfc:`9846`]
5136
handshake = 22
5237

53-
#: Application Data [:rfc:`9846`]
38+
#: application_data [:rfc:`9846`]
5439
application_data = 23
5540

56-
#: Heartbeat [:rfc:`6520`]
41+
#: heartbeat [:rfc:`6520`]
5742
heartbeat = 24
5843

59-
#: ``tls12_cid`` [:rfc:`9146`]
44+
#: tls12_cid [:rfc:`9146`]
6045
tls12_cid = 25
6146

62-
#: ACK -- DTLS 1.3 only [:rfc:`9147`]
47+
#: ACK [:rfc:`9147`]
6348
ack = 26
6449

65-
#: ``return_routability_check`` -- DTLS connection-ID path validation
66-
#: [:rfc:`9853`]
50+
#: return_routability_check [:rfc:`9853`]
6751
return_routability_check = 27
6852

6953
@classmethod
7054
def _validate_value(cls, value: 'Any') -> 'None':
7155
"""Reject ``value`` unless it passes the range check :meth:`_missing_` makes.
7256
57+
Overrides :meth:`~pcapkit.corekit.enum.EnumLookup._validate_value`
58+
with that check and the message it raises, so
59+
:meth:`~pcapkit.corekit.enum.EnumRegistry.register` refuses exactly the
60+
values :meth:`~pcapkit.corekit.enum.EnumLookup.get` refuses, in the
61+
same words (:issue:`1816`).
62+
7363
Args:
7464
value: Value about to be looked up or minted.
7565
7666
Raises:
77-
EnumValueError: If ``value`` fails that check.
67+
EnumValueError: If ``value`` fails that check. A
68+
:exc:`ValueError`, so an ``except ValueError`` caller is
69+
unaffected.
7870
7971
"""
8072
if not (isinstance(value, int) and 0 <= value <= 255):
@@ -90,5 +82,18 @@ def _missing_(cls, value: 'int') -> 'ContentType':
9082
"""
9183
if not (isinstance(value, int) and 0 <= value <= 255):
9284
raise ValueError('%r is not a valid %s' % (value, cls.__name__))
93-
#: Unassigned
94-
return cls._unregistered_member(value, 'Unassigned')
85+
if 0 <= value <= 19:
86+
#: Unassigned (Requires coordination; see [RFC 9443])
87+
#: [:rfc:`5764`][:rfc:`9443`]
88+
return cls._unregistered_member(value, 'Unassigned')
89+
if 28 <= value <= 31:
90+
#: Unassigned
91+
return cls._unregistered_member(value, 'Unassigned')
92+
if 32 <= value <= 63:
93+
#: Reserved [:rfc:`9147`]
94+
return cls._unregistered_member(value, 'Reserved')
95+
if 64 <= value <= 255:
96+
#: Unassigned (Requires coordination; see [RFC 9443])
97+
#: [:rfc:`5764`][:rfc:`9443`]
98+
return cls._unregistered_member(value, 'Unassigned')
99+
return super()._missing_(value)

‎pcapkit/const/tls/handshake_type.py‎

Lines changed: 54 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -6,19 +6,7 @@
66
.. module:: pcapkit.const.tls.handshake_type
77
88
This module contains the constant enumeration for **TLS/DTLS HandshakeType**,
9-
as assigned by IANA in the ``TLS HandshakeType`` registry [*]_, including the
10-
DTLS-only connection-ID messages :rfc:`9147#section-5.2` adds to that same
11-
registry. Every value IANA assigns is named below; the ranges it lists as
12-
Unassigned fall through to :meth:`_missing_` like any other unassigned value.
13-
14-
Each member's name is the message name the protocol uses on the wire. IANA
15-
suffixes ``_RESERVED`` onto the ones TLS 1.3 retired -- it keeps the code point
16-
reserved rather than reassigning it -- which is a statement about the registry
17-
rather than a different message name, so the suffix is not carried into the
18-
member name and each such member instead records IANA's own comment on why the
19-
assignment is retired.
20-
21-
.. [*] https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-7
9+
which is automatically generated from :class:`pcapkit.vendor.tls.handshake_type.HandshakeType`.
2210
2311
"""
2412

@@ -38,95 +26,110 @@
3826
class HandshakeType(EnumRegistry, IntEnum):
3927
"""[HandshakeType] TLS/DTLS HandshakeType"""
4028

41-
#: ``hello_request`` -- TLS 1.2 and earlier only [:rfc:`5246`]
29+
#: hello_request [:rfc:`9846`] -- Used in TLS versions prior to 1.3.
4230
hello_request = 0
4331

44-
#: ``client_hello`` [:rfc:`9846`]
32+
#: client_hello [:rfc:`9846`]
4533
client_hello = 1
4634

47-
#: ``server_hello`` [:rfc:`9846`]
35+
#: server_hello [:rfc:`9846`]
4836
server_hello = 2
4937

50-
#: ``hello_verify_request`` -- DTLS 1.2 and earlier only; DTLS 1.3 moved
51-
#: its return-routability role to the ``cookie`` extension
52-
#: (:rfc:`9147#section-5.1`) [:rfc:`6347`]
38+
#: hello_verify_request [:rfc:`6347`][:rfc:`9846`] -- Assigned for interim
39+
#: draft, but the functionality was moved to a different message.
5340
hello_verify_request = 3
5441

55-
#: ``new_session_ticket`` [:rfc:`9846`]
42+
#: new_session_ticket [:rfc:`4507`][:rfc:`9846`][:rfc:`8447`][RFC Errata 7250]
5643
new_session_ticket = 4
5744

58-
#: ``end_of_early_data`` [:rfc:`9846`]
45+
#: end_of_early_data [:rfc:`9846`]
5946
end_of_early_data = 5
6047

61-
#: ``hello_retry_request`` -- historical; the functionality moved to an
62-
#: extension, and TLS 1.3 carries it on a ``server_hello`` instead
63-
#: (:rfc:`9846#section-4.2.4`) [:rfc:`9846`]
48+
#: hello_retry_request [:rfc:`9846`] -- Assigned for interim draft, but the
49+
#: functionality was moved to an extension.
6450
hello_retry_request = 6
6551

66-
#: ``encrypted_extensions`` [:rfc:`9846`]
52+
#: Unassigned
53+
unassigned_7 = 7
54+
55+
#: encrypted_extensions [:rfc:`9846`]
6756
encrypted_extensions = 8
6857

69-
#: ``request_connection_id`` -- DTLS 1.3 only [:rfc:`9147`]
58+
#: request_connection_id [:rfc:`9147`]
7059
request_connection_id = 9
7160

72-
#: ``new_connection_id`` -- DTLS 1.3 only [:rfc:`9147`]
61+
#: new_connection_id [:rfc:`9147`]
7362
new_connection_id = 10
7463

75-
#: ``certificate`` [:rfc:`9846`]
64+
#: certificate [:rfc:`9846`]
7665
certificate = 11
7766

78-
#: ``server_key_exchange`` -- TLS 1.2 and earlier only [:rfc:`5246`]
67+
#: server_key_exchange [:rfc:`9846`] -- Used in TLS versions prior to 1.3.
7968
server_key_exchange = 12
8069

81-
#: ``certificate_request`` [:rfc:`9846`]
70+
#: certificate_request [:rfc:`9846`]
8271
certificate_request = 13
8372

84-
#: ``server_hello_done`` -- TLS 1.2 and earlier only [:rfc:`5246`]
73+
#: server_hello_done [:rfc:`9846`] -- Used in TLS versions prior to 1.3.
8574
server_hello_done = 14
8675

87-
#: ``certificate_verify`` [:rfc:`9846`]
76+
#: certificate_verify [:rfc:`9846`]
8877
certificate_verify = 15
8978

90-
#: ``client_key_exchange`` -- TLS 1.2 and earlier only [:rfc:`5246`]
79+
#: client_key_exchange [:rfc:`9846`] -- Used in TLS versions prior to 1.3.
9180
client_key_exchange = 16
9281

93-
#: ``client_certificate_request`` -- TLS 1.2 and earlier only, per IANA's
94-
#: own comment on this assignment [:rfc:`9261`]
82+
#: client_certificate_request [:rfc:`9261`] -- Used in TLS versions prior to
83+
#: 1.3.
9584
client_certificate_request = 17
9685

97-
#: ``finished`` [:rfc:`9846`]
86+
#: finished [:rfc:`9846`]
9887
finished = 20
9988

100-
#: ``certificate_url`` -- TLS 1.2 and earlier only [:rfc:`6066`]
89+
#: certificate_url [:rfc:`6066`][:rfc:`9846`] -- Used in TLS versions prior to
90+
#: 1.3.
10191
certificate_url = 21
10292

103-
#: ``certificate_status`` -- TLS 1.2 and earlier only [:rfc:`6066`]
93+
#: certificate_status [:rfc:`6066`][:rfc:`9846`] -- Used in TLS versions prior
94+
#: to 1.3.
10495
certificate_status = 22
10596

106-
#: ``supplemental_data`` -- TLS 1.2 and earlier only [:rfc:`4680`]
97+
#: supplemental_data [:rfc:`4680`][:rfc:`9846`] -- Used in TLS versions prior
98+
#: to 1.3.
10799
supplemental_data = 23
108100

109-
#: ``key_update`` [:rfc:`9846`]
101+
#: key_update [:rfc:`9846`]
110102
key_update = 24
111103

112-
#: ``compressed_certificate`` [:rfc:`8879`]
104+
#: compressed_certificate [:rfc:`8879`]
113105
compressed_certificate = 25
114106

115-
#: ``ekt_key`` [:rfc:`8870`]
107+
#: ekt_key [:rfc:`8870`]
116108
ekt_key = 26
117109

118-
#: ``message_hash`` [:rfc:`9846`]
110+
#: message_hash [:rfc:`9846`]
119111
message_hash = 254
120112

113+
#: Unassigned
114+
unassigned_255 = 255
115+
121116
@classmethod
122117
def _validate_value(cls, value: 'Any') -> 'None':
123118
"""Reject ``value`` unless it passes the range check :meth:`_missing_` makes.
124119
120+
Overrides :meth:`~pcapkit.corekit.enum.EnumLookup._validate_value`
121+
with that check and the message it raises, so
122+
:meth:`~pcapkit.corekit.enum.EnumRegistry.register` refuses exactly the
123+
values :meth:`~pcapkit.corekit.enum.EnumLookup.get` refuses, in the
124+
same words (:issue:`1816`).
125+
125126
Args:
126127
value: Value about to be looked up or minted.
127128
128129
Raises:
129-
EnumValueError: If ``value`` fails that check.
130+
EnumValueError: If ``value`` fails that check. A
131+
:exc:`ValueError`, so an ``except ValueError`` caller is
132+
unaffected.
130133
131134
"""
132135
if not (isinstance(value, int) and 0 <= value <= 255):
@@ -142,5 +145,10 @@ def _missing_(cls, value: 'int') -> 'HandshakeType':
142145
"""
143146
if not (isinstance(value, int) and 0 <= value <= 255):
144147
raise ValueError('%r is not a valid %s' % (value, cls.__name__))
145-
#: Unassigned
146-
return cls._unregistered_member(value, 'Unassigned')
148+
if 18 <= value <= 19:
149+
#: Unassigned
150+
return cls._unregistered_member(value, 'Unassigned')
151+
if 27 <= value <= 253:
152+
#: Unassigned
153+
return cls._unregistered_member(value, 'Unassigned')
154+
return super()._missing_(value)

‎pcapkit/protocols/application/ssl.py‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -60,9 +60,10 @@
6060
_DTLS_VERSION_MAJOR = 0xFE
6161

6262
#: The content types a genuine record may carry. An allowlist rather than
63-
#: every :class:`~pcapkit.const.tls.content_type.ContentType` member: the
64-
#: registry names ``invalid(0)``, a sentinel :rfc:`9846#section-5.1` rejects,
65-
#: and mints a member for any unassigned value it is asked for.
63+
#: every :class:`~pcapkit.const.tls.content_type.ContentType` member: 0 is
64+
#: never assigned, a value :rfc:`9846#section-5.1` rejects outright, and the
65+
#: enum mints an ``Unassigned``/``Reserved`` member for any value IANA has
66+
#: not named.
6667
#:
6768
#: ``tls12_cid`` (25, :rfc:`9146#section-4`) is left out: its connection ID
6869
#: precedes ``length`` and has a negotiated length the record does not carry,
@@ -235,8 +236,9 @@ def _validate_tls_record(self, schema: 'Any', *, available: 'int', dtls: 'bool')
235236
"""
236237
# NOTE: ``make()`` leaves ``content_type`` a bare :obj:`int`, not the enum member a
237238
# parse yields, so resolve it through the registry to cover both.
238-
# NOTE: checked against ``_WIRE_CONTENT_TYPES``, not by name: ``invalid(0)`` is
239-
# named yet never a record's content type.
239+
# NOTE: checked against ``_WIRE_CONTENT_TYPES``, not by name: an
240+
# unassigned or reserved value still resolves to a named member via
241+
# ``_missing_``, yet is never a record's genuine content type.
240242
content_type = Enum_ContentType.get(int(schema.content_type))
241243
if content_type not in _WIRE_CONTENT_TYPES:
242244
raise ProtocolError(

‎pcapkit/vendor/__init__.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,7 @@ def __getattr__(name: 'str') -> 'typing.NoReturn':
120120
from pcapkit.vendor.ospf import *
121121
from pcapkit.vendor.sctp import *
122122
from pcapkit.vendor.tcp import *
123+
from pcapkit.vendor.tls import *
123124
from pcapkit.vendor.vlan import *
124125

125126
# Crawlers only. This list is also the default target list of
@@ -180,6 +181,8 @@ def __getattr__(name: 'str') -> 'typing.NoReturn':
180181
'SCTP_PayloadProtocolIdentifier',
181182
# TCP
182183
'TCP_Checksum', 'TCP_Option', 'TCP_MPTCPOption', 'TCP_Flags',
184+
# TLS
185+
'TLS_ContentType', 'TLS_HandshakeType',
183186
# VLAN
184187
'VLAN_PriorityLevel',
185188
# PCAPNG

0 commit comments

Comments
 (0)