Skip to content

Commit e78fb2a

Browse files
authored
feat(application): parse TLS handshake extensions (#2595)
- Add the IANA TLS ExtensionType registry: pcapkit.vendor.tls.extension_type and the pcapkit.const.tls.extension_type module it generates. - Add SSL.__extension__ with register_extension and the register_tls_extension wrapper; decode server_name, supported_groups, signature_algorithms, ALPN, supported_versions and key_share in ClientHello, ServerHello, HelloRetryRequest and EncryptedExtensions. - Keep unknown extensions, and bodies their decoder rejects, as raw octets, the rejection quiet under CaughtByCaller; decode EncryptedExtensions. - Build extensions from data models, raw octets or (type, kwargs) pairs, byte-exact both ways; add the tls-extension options-generator family. - Add the shared value helpers to Protocol: _read_vector, _read_whole_vector, _read_uints, _make_uint, _make_vector and _make_uints, which the SSL extension readers and makers call. - Document the models, registry and wrapper, and the decoded bodies in the DTLS models, the roadmap and the TLS/DTLS pages; bump the const pins.
1 parent e789677 commit e78fb2a

36 files changed

Lines changed: 2744 additions & 89 deletions

‎docs/source/contributing/pep.rst‎

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -238,8 +238,12 @@ DTLS's own divergence (the epoch/sequence-number pair in its record header,
238238
and the fragment header each handshake message carries) staying in
239239
:class:`~pcapkit.protocols.application.dtls.DTLS` alone. ``ClientHello`` and
240240
``ServerHello`` decode down to their cipher suite(s), compression method(s)
241-
and extension list (TYPE+LENGTH framed, extension bodies undecoded); every
242-
other handshake message is message-TYPE-level framing only. Decryption is a
241+
and extension list, and ``EncryptedExtensions`` down to its extension list. An
242+
extension body decodes where
243+
:attr:`SSL.__extension__ <pcapkit.protocols.application.ssl.SSL.__extension__>`
244+
registers a decoder for its type, and stays raw octets where none is
245+
registered or the decoder rejects it (:issue:`2514`). Every other handshake
246+
message is message-TYPE-level framing only. Decryption is a
243247
separate question and is not needed to make the record structure legible, in
244248
the same way :class:`ESP <pcapkit.protocols.internet.esp.ESP>` parses
245249
without keys.
@@ -1167,8 +1171,9 @@ group-record list, each a shape of its own.
11671171

11681172
**Done within wave 2 — TLS/SSL with DTLS.** The `DTLS`_ section above lists
11691173
what is decoded: the shared :class:`~pcapkit.protocols.application.ssl.SSL`
1170-
base, TLS, DTLS, handshake message framing, and handshake-fragment
1171-
reassembly (:issue:`2440`).
1174+
base, TLS, DTLS, handshake message framing, the hello and
1175+
``EncryptedExtensions`` extension lists with their registered extension bodies
1176+
(:issue:`2514`), and handshake-fragment reassembly (:issue:`2440`).
11721177

11731178
**Done ahead of wave 2 — handlers for the** ``DLT_NULL``, ``DLT_RAW``,
11741179
``DLT_LINUX_SLL`` **and** ``DLT_LINUX_SLL2`` **link types.** Each is a

‎docs/source/ext.rst‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -412,7 +412,9 @@ writes them:
412412
+ +------------------------------------------------------------+-----------------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
413413
| | :class:`~pcapkit.protocols.transport.dccp.DCCP` | :attr:`DCCP.__option__ <pcapkit.protocols.transport.dccp.DCCP.__option__>` | :func:`~pcapkit.foundation.registry.protocols.register_dccp_option` |
414414
+-------------------+------------------------------------------------------------+-----------------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
415-
| Application Layer | :class:`~pcapkit.protocols.application.httpv2.HTTPv2` | :attr:`HTTPv2.__frame__ <pcapkit.protocols.application.httpv2.HTTPv2.__frame__>` | :func:`~pcapkit.foundation.registry.protocols.register_http_frame` |
415+
| | :class:`~pcapkit.protocols.application.httpv2.HTTPv2` | :attr:`HTTPv2.__frame__ <pcapkit.protocols.application.httpv2.HTTPv2.__frame__>` | :func:`~pcapkit.foundation.registry.protocols.register_http_frame` |
416+
+ Application Layer +------------------------------------------------------------+-----------------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
417+
| | :class:`~pcapkit.protocols.application.ssl.SSL` | :attr:`SSL.__extension__ <pcapkit.protocols.application.ssl.SSL.__extension__>` | :func:`~pcapkit.foundation.registry.protocols.register_tls_extension` |
416418
+-------------------+------------------------------------------------------------+-----------------------------------------------------------------------------------------------+----------------------------------------------------------------------------+
417419
| | | :attr:`PCAPNG.__block__ <pcapkit.protocols.misc.pcapng.PCAPNG.__block__>` | :func:`~pcapkit.foundation.registry.protocols.register_pcapng_block` |
418420
+ + +-----------------------------------------------------------------------------------------------+----------------------------------------------------------------------------+

‎docs/source/pcapkit/const/tls.rst‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@ enumerations include:
1616
- TLS/DTLS Content Types [*]_
1717
* - :class:`TLS_HandshakeType <pcapkit.const.tls.handshake_type.HandshakeType>`
1818
- TLS/DTLS HandshakeType [*]_
19+
* - :class:`TLS_ExtensionType <pcapkit.const.tls.extension_type.ExtensionType>`
20+
- TLS/DTLS ExtensionType Values [*]_
1921

2022
.. versionadded:: 1.6.0
2123

@@ -47,7 +49,21 @@ DTLS-only messages :rfc:`9147#section-5.2` adds to the registry.
4749
:undoc-members:
4850
:show-inheritance:
4951

52+
TLS/DTLS ExtensionType Values
53+
=============================
54+
55+
.. module:: pcapkit.const.tls.extension_type
56+
57+
This module contains the constant enumeration for **TLS/DTLS ExtensionType Values**,
58+
which is automatically generated from :class:`pcapkit.vendor.tls.extension_type.ExtensionType`.
59+
60+
.. autoclass:: pcapkit.const.tls.extension_type.ExtensionType
61+
:members:
62+
:undoc-members:
63+
:show-inheritance:
64+
5065
.. rubric:: Footnotes
5166

5267
.. [*] https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-5
5368
.. [*] https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-7
69+
.. [*] https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#tls-extensiontype-values-1

‎docs/source/pcapkit/foundation/registry.rst‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -146,6 +146,8 @@ Application Layer Registries
146146

147147
.. autofunction:: pcapkit.foundation.registry.protocols.register_http_frame
148148

149+
.. autofunction:: pcapkit.foundation.registry.protocols.register_tls_extension
150+
149151
Miscellaneous Protocol Registries
150152
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
151153

‎docs/source/pcapkit/protocols/application/dtls.rst‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -121,9 +121,9 @@ Data Models
121121
:members:
122122
:show-inheritance:
123123

124-
See :doc:`ssl` for ``ClientHello``, ``ServerHello``, ``Extension``, ``Alert``
125-
and ``ChangeCipherSpec``, shared verbatim with
126-
:class:`~pcapkit.protocols.application.tls.TLS`.
124+
See :doc:`ssl` for ``ClientHello``, ``ServerHello``, ``EncryptedExtensions``,
125+
``Extension`` and its decoded bodies, ``Alert`` and ``ChangeCipherSpec``,
126+
shared verbatim with :class:`~pcapkit.protocols.application.tls.TLS`.
127127

128128
.. rubric:: Footnotes
129129

‎docs/source/pcapkit/protocols/application/ssl.rst‎

Lines changed: 78 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -26,9 +26,9 @@ version and the length, and DTLS adds a fragment header to each handshake
2626
message. Each record header is declared in its subclass's own schema, and
2727
:class:`~pcapkit.protocols.application.dtls.DTLS` reads the epoch and sequence
2828
number. This base class carries the logic the two share -- content-type
29-
dispatch, handshake-message decoding (including the DTLS fragment header), and
30-
the record-level validation that keeps dispatch from accepting bytes that
31-
cannot be a genuine record.
29+
dispatch, handshake-message decoding (including the DTLS fragment header), the
30+
extension registry, and the record-level validation that keeps dispatch from
31+
accepting bytes that cannot be a genuine record.
3232

3333
.. versionadded:: 1.6.0
3434

@@ -41,6 +41,23 @@ cannot be a genuine record.
4141
:show-inheritance:
4242

4343
.. automethod:: id
44+
.. automethod:: register_extension
45+
46+
.. automethod:: _read_ext_unknown
47+
.. automethod:: _read_ext_server_name
48+
.. automethod:: _read_ext_supported_groups
49+
.. automethod:: _read_ext_signature_algorithms
50+
.. automethod:: _read_ext_alpn
51+
.. automethod:: _read_ext_supported_versions
52+
.. automethod:: _read_ext_key_share
53+
54+
.. automethod:: _make_ext_unknown
55+
.. automethod:: _make_ext_server_name
56+
.. automethod:: _make_ext_supported_groups
57+
.. automethod:: _make_ext_signature_algorithms
58+
.. automethod:: _make_ext_alpn
59+
.. automethod:: _make_ext_supported_versions
60+
.. automethod:: _make_ext_key_share
4461

4562
.. automethod:: _validate_tls_record
4663

@@ -58,9 +75,15 @@ cannot be a genuine record.
5875
.. automethod:: _read_tls_server_hello
5976
.. automethod:: _make_tls_server_hello
6077

78+
.. automethod:: _read_tls_encrypted_extensions
79+
.. automethod:: _make_tls_encrypted_extensions
80+
6181
.. automethod:: _read_tls_extensions
6282
.. automethod:: _make_tls_extensions
6383

84+
.. autoattribute:: __extension__
85+
:no-value:
86+
6487
Data Models
6588
-----------
6689

@@ -78,10 +101,62 @@ Data Models
78101
:members:
79102
:show-inheritance:
80103

104+
.. autoclass:: pcapkit.protocols.data.application.tls.EncryptedExtensions
105+
:members:
106+
:show-inheritance:
107+
81108
.. autoclass:: pcapkit.protocols.data.application.tls.Extension
82109
:members:
83110
:show-inheritance:
84111

112+
.. autoclass:: pcapkit.protocols.data.application.tls.ExtensionData
113+
:members:
114+
:show-inheritance:
115+
116+
.. autoclass:: pcapkit.protocols.data.application.tls.ServerNameList
117+
:members:
118+
:show-inheritance:
119+
120+
.. autoclass:: pcapkit.protocols.data.application.tls.ServerName
121+
:members:
122+
:show-inheritance:
123+
124+
.. autoclass:: pcapkit.protocols.data.application.tls.NamedGroupList
125+
:members:
126+
:show-inheritance:
127+
128+
.. autoclass:: pcapkit.protocols.data.application.tls.SignatureSchemeList
129+
:members:
130+
:show-inheritance:
131+
132+
.. autoclass:: pcapkit.protocols.data.application.tls.ProtocolNameList
133+
:members:
134+
:show-inheritance:
135+
136+
.. autoclass:: pcapkit.protocols.data.application.tls.SupportedVersionsClientHello
137+
:members:
138+
:show-inheritance:
139+
140+
.. autoclass:: pcapkit.protocols.data.application.tls.SupportedVersionsServerHello
141+
:members:
142+
:show-inheritance:
143+
144+
.. autoclass:: pcapkit.protocols.data.application.tls.KeyShareClientHello
145+
:members:
146+
:show-inheritance:
147+
148+
.. autoclass:: pcapkit.protocols.data.application.tls.KeyShareHelloRetryRequest
149+
:members:
150+
:show-inheritance:
151+
152+
.. autoclass:: pcapkit.protocols.data.application.tls.KeyShareServerHello
153+
:members:
154+
:show-inheritance:
155+
156+
.. autoclass:: pcapkit.protocols.data.application.tls.KeyShareEntry
157+
:members:
158+
:show-inheritance:
159+
85160
.. autoclass:: pcapkit.protocols.data.application.tls.Alert
86161
:members:
87162
:show-inheritance:

‎docs/source/pcapkit/protocols/application/tls.rst‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ extractor for the TLS/SSLv3+ record-layer family [*]_ -- SSLv3 (:rfc:`6101`),
99
TLS 1.0 (:rfc:`2246`), TLS 1.1 (:rfc:`4346`), TLS 1.2 (:rfc:`5246`) and TLS
1010
1.3 (:rfc:`9846`, which obsoletes :rfc:`5077`, :rfc:`5246`, :rfc:`6961`,
1111
:rfc:`7627`, :rfc:`8422` and :rfc:`8446`) -- at the record layer, plus
12-
handshake message TYPE-level framing.
12+
handshake message framing and the bodies `Decoded Fields`_ lists.
1313

1414
Version Identification
1515
----------------------
@@ -75,7 +75,10 @@ Decoded Fields
7575
decides which are):
7676

7777
* ``ClientHello``/``ServerHello`` down to their cipher suites, compression
78-
methods and extensions (type and length only).
78+
methods and extensions, and ``EncryptedExtensions`` down to its
79+
extensions; an extension's body decodes where
80+
:attr:`SSL.__extension__ <pcapkit.protocols.application.ssl.SSL.__extension__>`
81+
registers a decoder for its type.
7982
* Every other handshake message at type and length only.
8083
* ``Alert`` and ``ChangeCipherSpec`` in full.
8184

@@ -150,8 +153,9 @@ Data Model
150153
:members:
151154
:show-inheritance:
152155

153-
See :doc:`ssl` for ``ClientHello``, ``ServerHello``, ``Handshake``, ``Extension``,
154-
``Alert`` and ``ChangeCipherSpec``, shared verbatim with
156+
See :doc:`ssl` for ``ClientHello``, ``ServerHello``, ``EncryptedExtensions``,
157+
``Handshake``, ``Extension`` and its decoded bodies, ``Alert`` and
158+
``ChangeCipherSpec``, shared verbatim with
155159
:class:`~pcapkit.protocols.application.dtls.DTLS`.
156160

157161
.. rubric:: Footnotes

‎docs/source/pcapkit/vendor/tls.rst‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,8 @@ include:
1717
- TLS/DTLS Content Types [*]_
1818
* - :class:`TLS_HandshakeType <pcapkit.vendor.tls.handshake_type.HandshakeType>`
1919
- TLS/DTLS HandshakeType [*]_
20+
* - :class:`TLS_ExtensionType <pcapkit.vendor.tls.extension_type.ExtensionType>`
21+
- TLS/DTLS ExtensionType Values [*]_
2022

2123
.. versionadded:: 1.6.0
2224

@@ -44,7 +46,20 @@ which is automatically generating :class:`pcapkit.const.tls.handshake_type.Hands
4446
:members: FLAG, LINK
4547
:show-inheritance:
4648

49+
TLS/DTLS ExtensionType Values
50+
=============================
51+
52+
.. module:: pcapkit.vendor.tls.extension_type
53+
54+
This module contains the vendor crawler for **TLS/DTLS ExtensionType Values**,
55+
which is automatically generating :class:`pcapkit.const.tls.extension_type.ExtensionType`.
56+
57+
.. autoclass:: pcapkit.vendor.tls.extension_type.ExtensionType
58+
:members: FLAG, LINK
59+
:show-inheritance:
60+
4761
.. rubric:: Footnotes
4862

4963
.. [*] https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-5
5064
.. [*] https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-7
65+
.. [*] https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#tls-extensiontype-values-1

‎examples/generators/options.py‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1325,6 +1325,63 @@ def _httpv2_rebuild(frame: 'Any') -> 'Any':
13251325
frame=frame)
13261326

13271327

1328+
###############################################################################
1329+
# TLS -- handshake extensions
1330+
###############################################################################
1331+
#
1332+
# Each case is one extension in a ``ClientHello``, built from keyword arguments
1333+
# by the extension's ``_make_ext_*``. No capture is written: the ``tcp``
1334+
# envelope is addressed to port 80, which dispatches to HTTP rather than TLS.
1335+
1336+
1337+
def _tls_extension_registry() -> 'Any':
1338+
from pcapkit.protocols.application.tls import TLS
1339+
return TLS.__extension__
1340+
1341+
1342+
def _tls_extension_overrides() -> 'dict[Any, dict[str, Any]]':
1343+
from pcapkit.const.tls.extension_type import ExtensionType as Enum_ExtensionType
1344+
from pcapkit.protocols.data.application.tls import KeyShareEntry, ServerName
1345+
return {
1346+
# Each of these defaults to an empty list.
1347+
Enum_ExtensionType.server_name: {
1348+
'server_name_list': (ServerName(name_type=0, name=b'example.com'),)},
1349+
Enum_ExtensionType.supported_groups: {'named_group_list': (0x001D, 0x0017)},
1350+
Enum_ExtensionType.signature_algorithms: {'supported_signature_algorithms': (0x0403, 0x0804)},
1351+
Enum_ExtensionType.application_layer_protocol_negotiation: {
1352+
'protocol_name_list': (b'h2', b'http/1.1')},
1353+
Enum_ExtensionType.supported_versions: {'versions': (0x0304, 0x0303)},
1354+
Enum_ExtensionType.key_share: {
1355+
'client_shares': (KeyShareEntry(group=0x001D, key_exchange=bytes(range(32))),)},
1356+
}
1357+
1358+
1359+
def _tls_client_hello(extensions: 'Any') -> 'Any':
1360+
"""A TLS record carrying one ``ClientHello`` with ``extensions``."""
1361+
from pcapkit.const.tls.content_type import ContentType as Enum_ContentType
1362+
from pcapkit.const.tls.handshake_type import HandshakeType as Enum_HandshakeType
1363+
from pcapkit.protocols.application.tls import TLS
1364+
from pcapkit.protocols.data.application.tls import ClientHello, Handshake
1365+
1366+
hello = ClientHello(version=0x0303, random=bytes(32), session_id=b'', cookie=None,
1367+
cipher_suites=(0x1301,), compression_methods=(0,), extensions=extensions)
1368+
return TLS(content_type=Enum_ContentType.handshake, version=0x0303,
1369+
content=(Handshake(msg_type=Enum_HandshakeType.client_hello, length=0, body=hello),))
1370+
1371+
1372+
def _tls_extension_build(code: 'Any', kwargs: 'dict[str, Any]') -> 'Any':
1373+
return _tls_client_hello(((code, dict(kwargs)),))
1374+
1375+
1376+
def _tls_parse(octets: 'bytes') -> 'Any':
1377+
from pcapkit.protocols.application.tls import TLS
1378+
return TLS(octets, len(octets))
1379+
1380+
1381+
def _tls_extension_extract(parsed: 'Any') -> 'Any':
1382+
return parsed.info.content[0].body.extensions
1383+
1384+
13281385
###############################################################################
13291386
# PCAP-NG -- blocks, block options, name records and decryption secrets
13301387
###############################################################################
@@ -1765,6 +1822,10 @@ def _pcapng_secrets_build(code: 'Any', kwargs: 'dict[str, Any]') -> 'Any':
17651822
Family('icmpv6-type', _icmpv6_registry, _icmpv6_overrides,
17661823
_icmpv6_build, _icmpv6_parse, _icmp_extract, _icmpv6_rebuild,
17671824
capture=None, envelope=None),
1825+
# No capture -- see the section comment above ``_tls_extension_registry``.
1826+
Family('tls-extension', _tls_extension_registry, _tls_extension_overrides,
1827+
_tls_extension_build, _tls_parse, _tls_extension_extract,
1828+
_tls_client_hello, capture=None, envelope=None),
17681829
# The PCAP-NG families write no capture -- see the section comment
17691830
# above ``PCAPNG_TIME`` for why the construction API's defaults do not
17701831
# make a reproducible one, and what already covers that space instead.

‎pcapkit/all.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,7 @@
107107
'register_sctp_chunk', 'register_sctp_parameter', 'register_sctp_cause',
108108
'register_dccp_option',
109109
'register_http_frame',
110+
'register_tls_extension',
110111
'register_pcapng_block', 'register_pcapng_option', 'register_pcapng_secrets',
111112
'register_pcapng_record',
112113
'register_extractor_engine',

0 commit comments

Comments
 (0)