Skip to content

Commit f5b9c25

Browse files
JarryShawJarryShaw
committed
feat(application): implement TLS/SSL and DTLS record layer + handshake framing
pcapkit.protocols.application.ssl.SSL is the shared record-layer base over the part TLS and DTLS genuinely share (RFC 8446 §5.1 / RFC 9147 §4.1): ContentType/legacy-version/length and content-type dispatch. tls.py and dtls.py are the two concrete subclasses; DTLS alone carries the epoch + sequence number in its record header and the message_seq/fragment_offset/ fragment_length header each of its handshake messages adds (RFC 6347 §4.2.2), plus a cookie field in its ClientHello that TLS's does not carry (RFC 6347 §4.2.1 / RFC 9147 §5.3). - ClientHello/ServerHello decode to their version, random, session ID, cipher suite(s), compression method(s) and extension list (TYPE+LENGTH framed, extension bodies undecoded); every other handshake type is message-TYPE-level framing only. Alert and ChangeCipherSpec decode in full; application_data and unrecognised content types stay opaque bytes. - SSL._validate_tls_record rejects a content type the registry has no member for, a legacy version outside TLS's/DTLS's range, or a declared length that runs past the buffer, so dispatch falls back to Raw instead of fabricating a record from unrelated bytes (e.g. a mid-stream TLS ciphertext continuation on port 443). - New const registries pcapkit.const.tls.{content_type,handshake_type}, following the generated-shape convention (EnumRegistry + IntEnum); HandshakeType now also names 17/21/22/23, previously unnamed gaps. - TCP port 443 (https) and UDP port 5684 (coaps) are bound to TLS/DTLS. - DTLS handshake-fragment reassembly is deliberately out of scope, tracked as #2440. Every message shape implemented has a hand-crafted byte fixture proven byte-exact both ways (parse-then-rebuild and make-then-parse), including a genuine DTLS ClientHello with a cookie. Build/test: mypy and pylint clean on every changed file; isort clean; the full tests/protocols, tests/project and tests/const suites pass. Refs #2437
1 parent 83c7b22 commit f5b9c25

38 files changed

Lines changed: 2599 additions & 79 deletions

‎CHANGELOG.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,12 @@
22

33
Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had no dissector at all.
44

5+
### pcapkit.protocols
6+
7+
#### Added
8+
9+
- TLS and DTLS record-layer and handshake-message framing [[RFC 5246](https://datatracker.ietf.org/doc/html/rfc5246), [RFC 8446](https://datatracker.ietf.org/doc/html/rfc8446), [RFC 6347](https://datatracker.ietf.org/doc/html/rfc6347), [RFC 9147](https://datatracker.ietf.org/doc/html/rfc9147)] ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). `SSL` is a shared base over the record header and content-type dispatch TLS and DTLS hold in common; `TLS` and `DTLS` are the two concrete subclasses, DTLS alone carrying the epoch/sequence-number pair and the fragment header each handshake message adds. `ClientHello`/`ServerHello` decode to their version, random, session ID, DTLS's `cookie`, cipher suite(s), compression method(s) and extension list (TYPE+LENGTH framed, extension bodies undecoded); every other handshake message is message-TYPE-level framing only. TCP port 443 (`https`) and UDP port 5684 (`coaps`) are bound to TLS and DTLS respectively. Reassembling a handshake message DTLS has fragmented across several records is left to its own follow-up ([#2440](https://github.com/JarryShaw/PyPCAPKit/issues/2440)).
10+
511
---
612

713
Full changelog: <https://jarryshaw.github.io/PyPCAPKit/changelog.html>

‎docs/source/changelog/1.6.0.rst‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,3 +4,25 @@
44
Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the
55
protocol pairs that wanted a shared abstract base, and the two that simply had
66
no dissector at all.
7+
8+
pcapkit.protocols
9+
-----------------
10+
11+
Added
12+
~~~~~
13+
14+
* TLS and DTLS record-layer and handshake-message framing [:rfc:`5246`,
15+
:rfc:`8446`, :rfc:`6347`, :rfc:`9147`] (:issue:`2437`, :pr:`2441`).
16+
:class:`~pcapkit.protocols.application.ssl.SSL` is a shared base over the
17+
record header and content-type dispatch TLS and DTLS hold in common;
18+
:class:`~pcapkit.protocols.application.tls.TLS` and
19+
:class:`~pcapkit.protocols.application.dtls.DTLS` are the two concrete
20+
subclasses, DTLS alone carrying the epoch/sequence-number pair and the
21+
fragment header each handshake message adds. ``ClientHello``/``ServerHello``
22+
decode to their version, random, session ID, DTLS's ``cookie``, cipher
23+
suite(s), compression method(s) and extension list (TYPE+LENGTH framed,
24+
extension bodies undecoded); every other handshake message is
25+
message-TYPE-level framing only. TCP port 443 (``https``) and UDP port 5684
26+
(``coaps``) are bound to TLS and DTLS respectively. Reassembling a
27+
handshake message DTLS has fragmented across several records is left to its
28+
own follow-up (:issue:`2440`).

‎docs/source/contributing/pep.rst‎

Lines changed: 59 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -43,14 +43,19 @@ the ones outstanding are,
4343
* Link Layer: DSL, EAPOL, FDDI, ISDN, LINUX_SLL, LINUX_SLL2, NDP, PPP
4444
* Internet Layer: ECN, ICMP, ICMPv6, IGMP, Shim6
4545
* Transport Layer: DCCP, RSVP
46-
* Application Layer: BGP, DHCP, DHCPv6, DNS, DTLS, IMAP, LDAP, MQTT, NNTP,
47-
NTP, ONC/RPC, POP, QUIC, RIP, RTP, SIP, SMTP, SNMP, SSH, Telnet, TLS/SSL,
46+
* Application Layer: BGP, DHCP, DHCPv6, DNS, IMAP, LDAP, MQTT, NNTP,
47+
NTP, ONC/RPC, POP, QUIC, RIP, RTP, SIP, SMTP, SNMP, SSH, Telnet,
4848
XMPP
4949

5050
Several of these come in pairs that want a shared abstract base rather than two
5151
independent implementations, in the way
5252
:class:`~pcapkit.protocols.internet.ip.IP` already covers its family: ICMP with
53-
ICMPv6, TLS/SSL with DTLS, and ``LINUX_SLL`` with ``LINUX_SLL2``.
53+
ICMPv6, and ``LINUX_SLL`` with ``LINUX_SLL2``. **Done** for the third such
54+
pair, TLS/SSL with DTLS: :class:`~pcapkit.protocols.application.ssl.SSL` is
55+
the shared record-layer base, with
56+
:class:`~pcapkit.protocols.application.tls.TLS` and
57+
:class:`~pcapkit.protocols.application.dtls.DTLS` as its two concrete
58+
implementations -- see the `DTLS`_ section below.
5459
:class:`~pcapkit.protocols.link.vlan.VLAN` is the closer precedent for a pair
5560
whose *layout* is identical -- it holds the whole of the tag, and
5661
:class:`~pcapkit.protocols.link.c_tag.C_Tag` and
@@ -62,7 +67,7 @@ whose *layout* is identical -- it holds the whole of the tag, and
6267
``NotImplemented`` folder. **Those files are not a roadmap.** They are
6368
scratch reminders, so a stub's presence does not mean a protocol is planned or
6469
claimed, and its absence does not mean the protocol is unwanted:
65-
``LINUX_SLL``, ``LINUX_SLL2``, ``QUIC`` and **DTLS** have no stub and are on the
70+
``LINUX_SLL``, ``LINUX_SLL2`` and ``QUIC`` have no stub and are on the
6671
list above,
6772
while **NGAP** has none either and is implemented. Take this page as the
6873
record and ignore the folder.
@@ -222,35 +227,43 @@ looks like an ordinary field and is not:
222227
DTLS
223228
~~~~
224229

225-
**Not started**, and it has no stub in the tree -- of the TLS/SSL and DTLS pair,
226-
only TLS/SSL does. It earns its own entry because the registries
227-
have moved ahead of it, and a growing part of SCTP's surface names a protocol
228-
that does not exist:
229-
230-
* the DTLS chunk, type 65 of :class:`~pcapkit.const.sctp.chunk.Chunk`, and the
231-
DTLS Key Management chunk parameter, ``0x8006`` of
232-
:class:`~pcapkit.const.sctp.parameter.Parameter` -- both fall through to the
233-
generic handlers, so they parse as opaque;
234-
* its error causes, 100 to 103 of
235-
:class:`~pcapkit.const.sctp.cause_code.CauseCode`;
236-
* payload protocol identifiers -- 47 (Diameter over DTLS/SCTP), 66 to 69
237-
(NGAP, XnAP, F1AP and E1AP, each over DTLS over SCTP) and 4242 (DTLS chunk
238-
key management).
239-
240-
That last group is the one that bites. NGAP over DTLS over SCTP is PPID 66, and
241-
even with an NGAP dissector registered against it the bytes arriving are a DTLS
242-
record rather than an NGAP PDU, so they can only degrade to
243-
:class:`~pcapkit.protocols.misc.raw.Raw`. DTLS therefore blocks NGAP over DTLS,
230+
**Done** for the record layer and handshake message framing:
231+
:class:`~pcapkit.protocols.application.tls.TLS` (TLS 1.2, :rfc:`5246`; TLS
232+
1.3, :rfc:`8446`) and :class:`~pcapkit.protocols.application.dtls.DTLS`
233+
(DTLS 1.2, :rfc:`6347`; DTLS 1.3, :rfc:`9147`) share
234+
:class:`~pcapkit.protocols.application.ssl.SSL` as their record-layer base --
235+
content type, legacy version, length, and content-type dispatch -- with
236+
DTLS's own divergence (the epoch/sequence-number pair in its record header,
237+
and the fragment header each handshake message carries) staying in
238+
:class:`~pcapkit.protocols.application.dtls.DTLS` alone. ``ClientHello`` and
239+
``ServerHello`` decode down to their cipher suite(s), compression method(s)
240+
and extension list (TYPE+LENGTH framed, extension bodies undecoded); every
241+
other handshake message is message-TYPE-level framing only. Decryption is a
242+
separate question and was never needed to make the record structure legible,
243+
in the same way :class:`ESP <pcapkit.protocols.internet.esp.ESP>` parses
244+
without keys.
245+
246+
**Not yet done: reassembling a handshake message DTLS has fragmented across
247+
several records** (:rfc:`6347#section-4.2.3`). That needs a
248+
:class:`~pcapkit.foundation.reassembly.reassembly.Reassembly` subclass keyed
249+
on the UDP 5-tuple plus epoch, which is substantial enough that it was left
250+
for a follow-up rather than landing alongside the dissector
251+
(:issue:`2440`). Until then, a fragmented message's body is kept as that one
252+
fragment's raw octets -- see
253+
:attr:`DTLSHandshake.is_fragmented <pcapkit.protocols.data.application.dtls.DTLSHandshake.is_fragmented>`
254+
-- rather than attempted as a decode only a slice of the message could ever
255+
back correctly. An unfragmented message, the common case, is unaffected.
256+
257+
NGAP over DTLS over SCTP (PPID 66,
258+
:class:`~pcapkit.const.sctp.payload_protocol_identifier.PayloadProtocolIdentifier`),
244259
asked for in `discussion #251
245-
<https://github.com/JarryShaw/PyPCAPKit/discussions/251>`__, independently of
246-
whether NGAP itself is implemented.
247-
248-
A record-layer dissector is enough to unblock that -- content type, version,
249-
epoch, sequence number, length, and the fragment offset and length DTLS adds to
250-
handshake messages. Decryption is a separate question and is not needed to make
251-
the record structure legible, in the same way :class:`ESP
252-
<pcapkit.protocols.internet.esp.ESP>` parses without keys. DTLS over UDP is the
253-
other half of the same work and wants the same dissector.
260+
<https://github.com/JarryShaw/PyPCAPKit/discussions/251>`__, is **not**
261+
unblocked by this: :attr:`SCTP.__proto__ <pcapkit.protocols.transport.sctp.SCTP.__proto__>`
262+
still maps PPID 66 straight to :class:`~pcapkit.protocols.application.ngap.NGAP`
263+
rather than through :class:`~pcapkit.protocols.application.dtls.DTLS`, so those
264+
bytes still degrade straight to :class:`~pcapkit.protocols.misc.raw.Raw`. Rewiring
265+
that dispatch, and deciding what an undecrypted DTLS-wrapped NGAP PDU should
266+
report, is left to its own follow-up.
254267

255268
Registered, but Not Dissected
256269
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -307,10 +320,15 @@ Some wanted only a table entry, because the dissector was already there:
307320
*name* rather than a parse -- which is the right answer for a data channel
308321
carrying an arbitrary file.
309322
* **Done.** HTTP is additionally bound on 8080 (IANA ``http-alt``, "HTTP
310-
Alternate (see port 80)") on both TCP and UDP. **8443 is deliberately left
311-
unbound**: IANA registers it as ``pcsync-https``, not as an HTTP alternate,
312-
and de-facto 8443 traffic is TLS-wrapped, which pcapkit cannot parse -- see
313-
the ``tls`` stub above. Binding it would feed a TLS record to an HTTP parser.
323+
Alternate (see port 80)") on both TCP and UDP, and TCP 443 (IANA ``https``)
324+
is bound to :class:`~pcapkit.protocols.application.tls.TLS`, now that it
325+
parses the record layer de-facto 443 traffic carries; UDP 5684 (IANA
326+
``coaps``) is bound to :class:`~pcapkit.protocols.application.dtls.DTLS`
327+
likewise. **8443 is deliberately left unbound**: IANA registers it as
328+
``pcsync-https``, not as an HTTP alternate, and binding it to HTTP would
329+
feed a TLS record to an HTTP parser; binding it to TLS instead would be a
330+
guess this registry does not make for a service IANA does not actually
331+
name there.
314332
* **Done.** The service VLAN tag identifier (S-Tag), ``0x88A8``, is bound to
315333
:class:`~pcapkit.protocols.link.s_tag.S_Tag`, and the customer tag ``0x8100``
316334
to :class:`~pcapkit.protocols.link.c_tag.C_Tag`. Both subclass the abstract
@@ -1078,6 +1096,11 @@ pair alone would answer it twice. ``LINUX_SLL`` matters out of proportion to its
10781096
size: every ``tcpdump -i any`` capture uses it, and neither variant has even a
10791097
stub.
10801098

1099+
**Done within wave 2 — TLS/SSL with DTLS.** See the `DTLS`_ section above for
1100+
what landed (the shared :class:`~pcapkit.protocols.application.ssl.SSL` base,
1101+
both concrete protocols, and handshake message framing) and what is left
1102+
(DTLS handshake-fragment reassembly, tracked as its own follow-up).
1103+
10811104
**Done ahead of wave 2 — handlers for the** ``DLT_NULL`` **and** ``DLT_RAW``
10821105
**link types.** Both are genuine DLTs that a capture can legitimately be rooted
10831106
in — ``DLT_NULL`` (0) is BSD's loopback encapsulation, ``DLT_RAW`` (101) an IP

‎docs/source/pcapkit/protocols/application/dtls.rst‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
DTLS - Datagram Transport Layer Security
2+
==============================================
3+
4+
.. module:: pcapkit.protocols.application.dtls
5+
6+
:mod:`pcapkit.protocols.application.dtls` contains
7+
:class:`~pcapkit.protocols.application.dtls.DTLS` only, which implements an
8+
extractor for Datagram Transport Layer Security (DTLS) [*]_ -- DTLS 1.2
9+
(:rfc:`6347`) and DTLS 1.3 (:rfc:`9147`) -- at the record layer, plus the
10+
epoch/sequence-number fields and the handshake-message fragment header that
11+
set DTLS apart from :class:`~pcapkit.protocols.application.tls.TLS`, whose
12+
reliable, ordered transport needs neither.
13+
14+
Known Limitation
15+
-----------------
16+
17+
**A handshake message split across several DTLS records is not reassembled.**
18+
DTLS fragments a handshake message when it does not fit in one datagram
19+
(:rfc:`6347#section-4.2.3`); reassembling that would need
20+
:class:`~pcapkit.foundation.reassembly.reassembly.Reassembly` keyed on the
21+
UDP 5-tuple plus epoch, which is substantial enough that it did not fit
22+
cleanly alongside this dissector in the same PR (:issue:`2440`). A fragment's
23+
:attr:`~pcapkit.protocols.data.application.dtls.DTLSHandshake.is_fragmented`
24+
says whether this is the case; such a message's
25+
:attr:`~pcapkit.protocols.data.application.dtls.DTLSHandshake.body` is kept
26+
as that one fragment's raw octets rather than attempted as a (necessarily
27+
wrong, since only a slice of it is present) decode. An *unfragmented* message
28+
-- ``fragment_offset == 0`` and ``fragment_length == length`` -- decodes
29+
exactly as :class:`~pcapkit.protocols.application.tls.TLS` would, which is
30+
the common case: a ``ClientHello`` or ``ServerHello`` that fits one datagram
31+
never fragments.
32+
33+
.. versionadded:: 1.6.0
34+
35+
:issue:`2437`.
36+
37+
.. raw:: html
38+
39+
<br />
40+
41+
.. autoclass:: pcapkit.protocols.application.dtls.DTLS
42+
:no-members:
43+
:show-inheritance:
44+
45+
.. autoproperty:: name
46+
.. autoproperty:: length
47+
48+
.. automethod:: read
49+
.. automethod:: make
50+
51+
.. automethod:: __length_hint__
52+
53+
.. automethod:: _make_data
54+
55+
Header Schema
56+
-------------
57+
58+
.. module:: pcapkit.protocols.schema.application.dtls
59+
60+
.. autoclass:: pcapkit.protocols.schema.application.dtls.DTLS
61+
:members:
62+
:show-inheritance:
63+
64+
Data Models
65+
-----------
66+
67+
.. module:: pcapkit.protocols.data.application.dtls
68+
69+
.. autoclass:: pcapkit.protocols.data.application.dtls.DTLS
70+
:members:
71+
:show-inheritance:
72+
73+
.. autoclass:: pcapkit.protocols.data.application.dtls.DTLSHandshake
74+
:members:
75+
:show-inheritance:
76+
77+
See :doc:`ssl` for ``ClientHello``, ``ServerHello``, ``Extension``, ``Alert``
78+
and ``ChangeCipherSpec``, shared verbatim with
79+
:class:`~pcapkit.protocols.application.tls.TLS`.
80+
81+
.. rubric:: Footnotes
82+
83+
.. [*] https://en.wikipedia.org/wiki/Datagram_Transport_Layer_Security

‎docs/source/pcapkit/protocols/application/index.rst‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,11 +22,14 @@ function rather than encapsulation, see
2222
ngap
2323
ospf
2424
rarp
25+
ssl
26+
tls
27+
dtls
2528

2629
.. todo::
2730

2831
Implement BGP, DHCP, DHCPv6, DNS, IMAP, LDAP, MQTT, NNTP, NTP, ONC:RPC,
29-
POP, RIP, RTP, SIP, SMTP, SNMP, SSH, TELNET, TLS/SSL, XMPP.
32+
POP, RIP, RTP, SIP, SMTP, SNMP, SSH, TELNET, XMPP.
3033

3134
Protocol Registry
3235
-----------------

‎docs/source/pcapkit/protocols/application/ngap.rst‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -91,8 +91,11 @@ Known Limitations
9191
-----------------
9292

9393
* **PPID 66 payloads are not decoded.** NGAP over DTLS over SCTP wraps the
94-
``NGAP-PDU`` in a DTLS record, and :mod:`pcapkit` implements no DTLS, so the
95-
bytes reaching :meth:`NGAP.read` are not an APER encoding. The PPID is
94+
``NGAP-PDU`` in a DTLS record, and :attr:`SCTP.__proto__
95+
<pcapkit.protocols.transport.sctp.SCTP.__proto__>` still maps PPID 66
96+
straight to :class:`~pcapkit.protocols.application.ngap.NGAP` rather than
97+
through :class:`~pcapkit.protocols.application.dtls.DTLS`, so the bytes
98+
reaching :meth:`NGAP.read` are not an APER encoding. The PPID is
9699
registered so that it is *named* rather than anonymous; the payload itself
97100
degrades to :class:`~pcapkit.protocols.misc.raw.Raw`.
98101
* **The specification version is that of the installed decoder, not this

‎docs/source/pcapkit/protocols/application/ssl.rst‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
SSL - Secure Sockets Layer
2+
===============================
3+
4+
.. module:: pcapkit.protocols.application.ssl
5+
6+
:mod:`pcapkit.protocols.application.ssl` contains
7+
:class:`~pcapkit.protocols.application.ssl.SSL` only,
8+
which is a base class for Transport Layer Security (TLS) protocol
9+
family [*]_, e.g.
10+
:class:`~pcapkit.protocols.application.tls.TLS` and
11+
:class:`~pcapkit.protocols.application.dtls.DTLS`.
12+
13+
The record-layer header -- a :class:`~pcapkit.const.tls.content_type.ContentType`,
14+
a legacy record version, and a length -- is byte-identical in shape between
15+
TLS (:rfc:`8446#section-5.1`) and DTLS (:rfc:`9147#section-4.1`); DTLS's own
16+
divergence (an explicit epoch and sequence number in the record header, and
17+
handshake-message fragmentation that a reliable TLS stream never needs) stays
18+
entirely in :class:`~pcapkit.protocols.application.dtls.DTLS`. This base class
19+
carries the logic the two genuinely share -- content-type dispatch, handshake-
20+
message decoding, and the record-level validation that keeps dispatch from
21+
accepting bytes that cannot be a genuine record.
22+
23+
.. versionadded:: 1.6.0
24+
25+
:issue:`2437`.
26+
27+
.. raw:: html
28+
29+
<br />
30+
31+
.. autoclass:: pcapkit.protocols.application.ssl.SSL
32+
:no-members:
33+
:show-inheritance:
34+
35+
.. automethod:: id
36+
37+
.. automethod:: _validate_tls_record
38+
39+
.. automethod:: _read_tls_content
40+
.. automethod:: _make_tls_content
41+
42+
.. automethod:: _read_tls_handshake_messages
43+
.. automethod:: _make_tls_handshake_messages
44+
45+
.. automethod:: _read_tls_client_hello
46+
.. automethod:: _make_tls_client_hello
47+
48+
.. automethod:: _read_tls_server_hello
49+
.. automethod:: _make_tls_server_hello
50+
51+
.. automethod:: _read_tls_extensions
52+
.. automethod:: _make_tls_extensions
53+
54+
Data Models
55+
-----------
56+
57+
.. module:: pcapkit.protocols.data.application.tls
58+
59+
.. autoclass:: pcapkit.protocols.data.application.tls.Handshake
60+
:members:
61+
:show-inheritance:
62+
63+
.. autoclass:: pcapkit.protocols.data.application.tls.ClientHello
64+
:members:
65+
:show-inheritance:
66+
67+
.. autoclass:: pcapkit.protocols.data.application.tls.ServerHello
68+
:members:
69+
:show-inheritance:
70+
71+
.. autoclass:: pcapkit.protocols.data.application.tls.Extension
72+
:members:
73+
:show-inheritance:
74+
75+
.. autoclass:: pcapkit.protocols.data.application.tls.Alert
76+
:members:
77+
:show-inheritance:
78+
79+
.. autoclass:: pcapkit.protocols.data.application.tls.ChangeCipherSpec
80+
:members:
81+
:show-inheritance:
82+
83+
.. rubric:: Footnotes
84+
85+
.. [*] https://en.wikipedia.org/wiki/Transport_Layer_Security

0 commit comments

Comments
 (0)