Skip to content

Commit 11cf6ba

Browse files
committed
feat(application): parse TLS handshake extensions (#2514)
- Add the IANA TLS ExtensionType registry: pcapkit.vendor.tls.extension_type and the pcapkit.const.tls.extension_type module it generates. - Add SSL.__extension__ with register_extension and the register_tls_extension wrapper; decode server_name, supported_groups, signature_algorithms, ALPN, supported_versions and key_share in ClientHello, ServerHello, HelloRetryRequest and EncryptedExtensions. - Keep unknown extensions, and bodies their decoder rejects, as raw octets, the rejection quiet under CaughtByCaller; decode EncryptedExtensions. - Build extensions from data models, raw octets or (type, kwargs) pairs, byte-exact both ways; add the tls-extension options-generator family. - Document the models, registry and wrapper, and the decoded bodies in the DTLS models, the roadmap and the TLS/DTLS pages; bump the const pins.
1 parent f4aa577 commit 11cf6ba

34 files changed

Lines changed: 2421 additions & 88 deletions

‎docs/source/contributing/pep.rst‎

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -238,8 +238,12 @@ DTLS's own divergence (the epoch/sequence-number pair in its record header,
238238
and the fragment header each handshake message carries) staying in
239239
:class:`~pcapkit.protocols.application.dtls.DTLS` alone. ``ClientHello`` and
240240
``ServerHello`` decode down to their cipher suite(s), compression method(s)
241-
and extension list (TYPE+LENGTH framed, extension bodies undecoded); every
242-
other handshake message is message-TYPE-level framing only. Decryption is a
241+
and extension list, and ``EncryptedExtensions`` down to its extension list. An
242+
extension body decodes where
243+
:attr:`SSL.__extension__ <pcapkit.protocols.application.ssl.SSL.__extension__>`
244+
registers a decoder for its type, and stays raw octets where none is
245+
registered or the decoder rejects it (:issue:`2514`). Every other handshake
246+
message is message-TYPE-level framing only. Decryption is a
243247
separate question and is not needed to make the record structure legible, in
244248
the same way :class:`ESP <pcapkit.protocols.internet.esp.ESP>` parses
245249
without keys.
@@ -1146,8 +1150,9 @@ group-record list, each a shape of its own.
11461150

11471151
**Done within wave 2 — TLS/SSL with DTLS.** The `DTLS`_ section above lists
11481152
what is decoded: the shared :class:`~pcapkit.protocols.application.ssl.SSL`
1149-
base, TLS, DTLS, handshake message framing, and handshake-fragment
1150-
reassembly (:issue:`2440`).
1153+
base, TLS, DTLS, handshake message framing, the hello and
1154+
``EncryptedExtensions`` extension lists with their registered extension bodies
1155+
(:issue:`2514`), and handshake-fragment reassembly (:issue:`2440`).
11511156

11521157
**Done ahead of wave 2 — handlers for the** ``DLT_NULL``, ``DLT_RAW``,
11531158
``DLT_LINUX_SLL`` **and** ``DLT_LINUX_SLL2`` **link types.** Each is a

‎docs/source/ext.rst‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -406,7 +406,9 @@ writes them:
406406
+ + +-----------------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
407407
| | | :attr:`SCTP.__cause__ <pcapkit.protocols.transport.sctp.SCTP.__cause__>` | :func:`~pcapkit.foundation.registry.protocols.register_sctp_cause` |
408408
+-------------------+------------------------------------------------------------+-----------------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
409-
| Application Layer | :class:`~pcapkit.protocols.application.httpv2.HTTPv2` | :attr:`HTTPv2.__frame__ <pcapkit.protocols.application.httpv2.HTTPv2.__frame__>` | :func:`~pcapkit.foundation.registry.protocols.register_http_frame` |
409+
| | :class:`~pcapkit.protocols.application.httpv2.HTTPv2` | :attr:`HTTPv2.__frame__ <pcapkit.protocols.application.httpv2.HTTPv2.__frame__>` | :func:`~pcapkit.foundation.registry.protocols.register_http_frame` |
410+
+ Application Layer +------------------------------------------------------------+-----------------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
411+
| | :class:`~pcapkit.protocols.application.ssl.SSL` | :attr:`SSL.__extension__ <pcapkit.protocols.application.ssl.SSL.__extension__>` | :func:`~pcapkit.foundation.registry.protocols.register_tls_extension` |
410412
+-------------------+------------------------------------------------------------+-----------------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
411413
| | | :attr:`PCAPNG.__block__ <pcapkit.protocols.misc.pcapng.PCAPNG.__block__>` | :func:`~pcapkit.foundation.registry.protocols.register_pcapng_block` |
412414
+ + +-----------------------------------------------------------------------------------------------+----------------------------------------------------------------------------+

‎docs/source/pcapkit/const/tls.rst‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@ enumerations include:
1616
- TLS/DTLS Content Types [*]_
1717
* - :class:`TLS_HandshakeType <pcapkit.const.tls.handshake_type.HandshakeType>`
1818
- TLS/DTLS HandshakeType [*]_
19+
* - :class:`TLS_ExtensionType <pcapkit.const.tls.extension_type.ExtensionType>`
20+
- TLS/DTLS ExtensionType Values [*]_
1921

2022
.. versionadded:: 1.6.0
2123

@@ -47,7 +49,21 @@ DTLS-only messages :rfc:`9147#section-5.2` adds to the registry.
4749
:undoc-members:
4850
:show-inheritance:
4951

52+
TLS/DTLS ExtensionType Values
53+
=============================
54+
55+
.. module:: pcapkit.const.tls.extension_type
56+
57+
This module contains the constant enumeration for **TLS/DTLS ExtensionType Values**,
58+
which is automatically generated from :class:`pcapkit.vendor.tls.extension_type.ExtensionType`.
59+
60+
.. autoclass:: pcapkit.const.tls.extension_type.ExtensionType
61+
:members:
62+
:undoc-members:
63+
:show-inheritance:
64+
5065
.. rubric:: Footnotes
5166

5267
.. [*] https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-5
5368
.. [*] https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-7
69+
.. [*] https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#tls-extensiontype-values-1

‎docs/source/pcapkit/foundation/registry.rst‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,8 @@ Application Layer Registries
142142

143143
.. autofunction:: pcapkit.foundation.registry.protocols.register_http_frame
144144

145+
.. autofunction:: pcapkit.foundation.registry.protocols.register_tls_extension
146+
145147
Miscellaneous Protocol Registries
146148
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
147149

‎docs/source/pcapkit/protocols/application/dtls.rst‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -121,9 +121,9 @@ Data Models
121121
:members:
122122
:show-inheritance:
123123

124-
See :doc:`ssl` for ``ClientHello``, ``ServerHello``, ``Extension``, ``Alert``
125-
and ``ChangeCipherSpec``, shared verbatim with
126-
:class:`~pcapkit.protocols.application.tls.TLS`.
124+
See :doc:`ssl` for ``ClientHello``, ``ServerHello``, ``EncryptedExtensions``,
125+
``Extension`` and its decoded bodies, ``Alert`` and ``ChangeCipherSpec``,
126+
shared verbatim with :class:`~pcapkit.protocols.application.tls.TLS`.
127127

128128
.. rubric:: Footnotes
129129

‎docs/source/pcapkit/protocols/application/ssl.rst‎

Lines changed: 78 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -26,9 +26,9 @@ version and the length, and DTLS adds a fragment header to each handshake
2626
message. Each record header is declared in its subclass's own schema, and
2727
:class:`~pcapkit.protocols.application.dtls.DTLS` reads the epoch and sequence
2828
number. This base class carries the logic the two share -- content-type
29-
dispatch, handshake-message decoding (including the DTLS fragment header), and
30-
the record-level validation that keeps dispatch from accepting bytes that
31-
cannot be a genuine record.
29+
dispatch, handshake-message decoding (including the DTLS fragment header), the
30+
extension registry, and the record-level validation that keeps dispatch from
31+
accepting bytes that cannot be a genuine record.
3232

3333
.. versionadded:: 1.6.0
3434

@@ -41,6 +41,23 @@ cannot be a genuine record.
4141
:show-inheritance:
4242

4343
.. automethod:: id
44+
.. automethod:: register_extension
45+
46+
.. automethod:: _read_ext_unknown
47+
.. automethod:: _read_ext_server_name
48+
.. automethod:: _read_ext_supported_groups
49+
.. automethod:: _read_ext_signature_algorithms
50+
.. automethod:: _read_ext_alpn
51+
.. automethod:: _read_ext_supported_versions
52+
.. automethod:: _read_ext_key_share
53+
54+
.. automethod:: _make_ext_unknown
55+
.. automethod:: _make_ext_server_name
56+
.. automethod:: _make_ext_supported_groups
57+
.. automethod:: _make_ext_signature_algorithms
58+
.. automethod:: _make_ext_alpn
59+
.. automethod:: _make_ext_supported_versions
60+
.. automethod:: _make_ext_key_share
4461

4562
.. automethod:: _validate_tls_record
4663

@@ -58,9 +75,15 @@ cannot be a genuine record.
5875
.. automethod:: _read_tls_server_hello
5976
.. automethod:: _make_tls_server_hello
6077

78+
.. automethod:: _read_tls_encrypted_extensions
79+
.. automethod:: _make_tls_encrypted_extensions
80+
6181
.. automethod:: _read_tls_extensions
6282
.. automethod:: _make_tls_extensions
6383

84+
.. autoattribute:: __extension__
85+
:no-value:
86+
6487
Data Models
6588
-----------
6689

@@ -78,10 +101,62 @@ Data Models
78101
:members:
79102
:show-inheritance:
80103

104+
.. autoclass:: pcapkit.protocols.data.application.tls.EncryptedExtensions
105+
:members:
106+
:show-inheritance:
107+
81108
.. autoclass:: pcapkit.protocols.data.application.tls.Extension
82109
:members:
83110
:show-inheritance:
84111

112+
.. autoclass:: pcapkit.protocols.data.application.tls.ExtensionData
113+
:members:
114+
:show-inheritance:
115+
116+
.. autoclass:: pcapkit.protocols.data.application.tls.ServerNameList
117+
:members:
118+
:show-inheritance:
119+
120+
.. autoclass:: pcapkit.protocols.data.application.tls.ServerName
121+
:members:
122+
:show-inheritance:
123+
124+
.. autoclass:: pcapkit.protocols.data.application.tls.NamedGroupList
125+
:members:
126+
:show-inheritance:
127+
128+
.. autoclass:: pcapkit.protocols.data.application.tls.SignatureSchemeList
129+
:members:
130+
:show-inheritance:
131+
132+
.. autoclass:: pcapkit.protocols.data.application.tls.ProtocolNameList
133+
:members:
134+
:show-inheritance:
135+
136+
.. autoclass:: pcapkit.protocols.data.application.tls.SupportedVersionsClientHello
137+
:members:
138+
:show-inheritance:
139+
140+
.. autoclass:: pcapkit.protocols.data.application.tls.SupportedVersionsServerHello
141+
:members:
142+
:show-inheritance:
143+
144+
.. autoclass:: pcapkit.protocols.data.application.tls.KeyShareClientHello
145+
:members:
146+
:show-inheritance:
147+
148+
.. autoclass:: pcapkit.protocols.data.application.tls.KeyShareHelloRetryRequest
149+
:members:
150+
:show-inheritance:
151+
152+
.. autoclass:: pcapkit.protocols.data.application.tls.KeyShareServerHello
153+
:members:
154+
:show-inheritance:
155+
156+
.. autoclass:: pcapkit.protocols.data.application.tls.KeyShareEntry
157+
:members:
158+
:show-inheritance:
159+
85160
.. autoclass:: pcapkit.protocols.data.application.tls.Alert
86161
:members:
87162
:show-inheritance:

‎docs/source/pcapkit/protocols/application/tls.rst‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ extractor for the TLS/SSLv3+ record-layer family [*]_ -- SSLv3 (:rfc:`6101`),
1010
TLS 1.0 (:rfc:`2246`), TLS 1.1 (:rfc:`4346`), TLS 1.2 (:rfc:`5246`) and TLS
1111
1.3 (:rfc:`9846`, which obsoletes :rfc:`5077`, :rfc:`5246`, :rfc:`6961`,
1212
:rfc:`7627`, :rfc:`8422` and :rfc:`8446`) -- at the record layer, plus
13-
handshake message TYPE-level framing.
13+
handshake message framing and the bodies `Decoded Fields`_ lists.
1414

1515
Version Identification
1616
----------------------
@@ -76,7 +76,10 @@ Decoded Fields
7676
decides which are):
7777

7878
* ``ClientHello``/``ServerHello`` down to their cipher suites, compression
79-
methods and extensions (type and length only).
79+
methods and extensions, and ``EncryptedExtensions`` down to its
80+
extensions; an extension's body decodes where
81+
:attr:`SSL.__extension__ <pcapkit.protocols.application.ssl.SSL.__extension__>`
82+
registers a decoder for its type.
8083
* Every other handshake message at type and length only.
8184
* ``Alert`` and ``ChangeCipherSpec`` in full.
8285

@@ -157,8 +160,9 @@ Data Model
157160
:members:
158161
:show-inheritance:
159162

160-
See :doc:`ssl` for ``ClientHello``, ``ServerHello``, ``Handshake``, ``Extension``,
161-
``Alert`` and ``ChangeCipherSpec``, shared verbatim with
163+
See :doc:`ssl` for ``ClientHello``, ``ServerHello``, ``EncryptedExtensions``,
164+
``Handshake``, ``Extension`` and its decoded bodies, ``Alert`` and
165+
``ChangeCipherSpec``, shared verbatim with
162166
:class:`~pcapkit.protocols.application.dtls.DTLS`.
163167

164168
.. rubric:: Footnotes

‎docs/source/pcapkit/vendor/tls.rst‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,8 @@ include:
1717
- TLS/DTLS Content Types [*]_
1818
* - :class:`TLS_HandshakeType <pcapkit.vendor.tls.handshake_type.HandshakeType>`
1919
- TLS/DTLS HandshakeType [*]_
20+
* - :class:`TLS_ExtensionType <pcapkit.vendor.tls.extension_type.ExtensionType>`
21+
- TLS/DTLS ExtensionType Values [*]_
2022

2123
.. versionadded:: 1.6.0
2224

@@ -44,7 +46,20 @@ which is automatically generating :class:`pcapkit.const.tls.handshake_type.Hands
4446
:members: FLAG, LINK
4547
:show-inheritance:
4648

49+
TLS/DTLS ExtensionType Values
50+
=============================
51+
52+
.. module:: pcapkit.vendor.tls.extension_type
53+
54+
This module contains the vendor crawler for **TLS/DTLS ExtensionType Values**,
55+
which is automatically generating :class:`pcapkit.const.tls.extension_type.ExtensionType`.
56+
57+
.. autoclass:: pcapkit.vendor.tls.extension_type.ExtensionType
58+
:members: FLAG, LINK
59+
:show-inheritance:
60+
4761
.. rubric:: Footnotes
4862

4963
.. [*] https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-5
5064
.. [*] https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-7
65+
.. [*] https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#tls-extensiontype-values-1

‎examples/generators/options.py‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1166,6 +1166,63 @@ def _httpv2_rebuild(frame: 'Any') -> 'Any':
11661166
frame=frame)
11671167

11681168

1169+
###############################################################################
1170+
# TLS -- handshake extensions
1171+
###############################################################################
1172+
#
1173+
# Each case is one extension in a ``ClientHello``, built from keyword arguments
1174+
# by the extension's ``_make_ext_*``. No capture is written: the ``tcp``
1175+
# envelope is addressed to port 80, which dispatches to HTTP rather than TLS.
1176+
1177+
1178+
def _tls_extension_registry() -> 'Any':
1179+
from pcapkit.protocols.application.tls import TLS
1180+
return TLS.__extension__
1181+
1182+
1183+
def _tls_extension_overrides() -> 'dict[Any, dict[str, Any]]':
1184+
from pcapkit.const.tls.extension_type import ExtensionType as Enum_ExtensionType
1185+
from pcapkit.protocols.data.application.tls import KeyShareEntry, ServerName
1186+
return {
1187+
# Each of these defaults to an empty list.
1188+
Enum_ExtensionType.server_name: {
1189+
'server_name_list': (ServerName(name_type=0, name=b'example.com'),)},
1190+
Enum_ExtensionType.supported_groups: {'named_group_list': (0x001D, 0x0017)},
1191+
Enum_ExtensionType.signature_algorithms: {'supported_signature_algorithms': (0x0403, 0x0804)},
1192+
Enum_ExtensionType.application_layer_protocol_negotiation: {
1193+
'protocol_name_list': (b'h2', b'http/1.1')},
1194+
Enum_ExtensionType.supported_versions: {'versions': (0x0304, 0x0303)},
1195+
Enum_ExtensionType.key_share: {
1196+
'client_shares': (KeyShareEntry(group=0x001D, key_exchange=bytes(range(32))),)},
1197+
}
1198+
1199+
1200+
def _tls_client_hello(extensions: 'Any') -> 'Any':
1201+
"""A TLS record carrying one ``ClientHello`` with ``extensions``."""
1202+
from pcapkit.const.tls.content_type import ContentType as Enum_ContentType
1203+
from pcapkit.const.tls.handshake_type import HandshakeType as Enum_HandshakeType
1204+
from pcapkit.protocols.application.tls import TLS
1205+
from pcapkit.protocols.data.application.tls import ClientHello, Handshake
1206+
1207+
hello = ClientHello(version=0x0303, random=bytes(32), session_id=b'', cookie=None,
1208+
cipher_suites=(0x1301,), compression_methods=(0,), extensions=extensions)
1209+
return TLS(content_type=Enum_ContentType.handshake, version=0x0303,
1210+
content=(Handshake(msg_type=Enum_HandshakeType.client_hello, length=0, body=hello),))
1211+
1212+
1213+
def _tls_extension_build(code: 'Any', kwargs: 'dict[str, Any]') -> 'Any':
1214+
return _tls_client_hello(((code, dict(kwargs)),))
1215+
1216+
1217+
def _tls_parse(octets: 'bytes') -> 'Any':
1218+
from pcapkit.protocols.application.tls import TLS
1219+
return TLS(octets, len(octets))
1220+
1221+
1222+
def _tls_extension_extract(parsed: 'Any') -> 'Any':
1223+
return parsed.info.content[0].body.extensions
1224+
1225+
11691226
###############################################################################
11701227
# PCAP-NG -- blocks, block options, name records and decryption secrets
11711228
###############################################################################
@@ -1595,6 +1652,10 @@ def _pcapng_secrets_build(code: 'Any', kwargs: 'dict[str, Any]') -> 'Any':
15951652
Family('hip-parameter', _hip_registry, _hip_overrides,
15961653
_hip_build, _hip_parse, _hip_extract, _hip_rebuild,
15971654
capture='options-internet.pcap', envelope='ipv6', proto=139),
1655+
# No capture -- see the section comment above ``_tls_extension_registry``.
1656+
Family('tls-extension', _tls_extension_registry, _tls_extension_overrides,
1657+
_tls_extension_build, _tls_parse, _tls_extension_extract,
1658+
_tls_client_hello, capture=None, envelope=None),
15981659
# The PCAP-NG families write no capture -- see the section comment
15991660
# above ``PCAPNG_TIME`` for why the construction API's defaults do not
16001661
# make a reproducible one, and what already covers that space instead.

‎pcapkit/all.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,7 @@
106106
'register_tcp_option', 'register_tcp_mp_option',
107107
'register_sctp_chunk', 'register_sctp_parameter', 'register_sctp_cause',
108108
'register_http_frame',
109+
'register_tls_extension',
109110
'register_pcapng_block', 'register_pcapng_option', 'register_pcapng_secrets',
110111
'register_pcapng_record',
111112
'register_extractor_engine',

0 commit comments

Comments
 (0)